Just because it's 4am doesn't mean the warrant isn't valid or they should come back when it's convenient for you. If it's 4am and they are serving a warrant, it's because the warrant is for something important, and they want to get you at a time when you are less likely to dodge it.