Unless the gate agent is a real jerk, or you are, the worst that can happen is that you'll be told to wait for your own group. Sometimes you may slil through and be allowed to board while on other occasions the gate agent may ask you to wait until your boarding group is called.