Accusations of adultery and even of plotting against the King's life were levelled against the Queen, her brother and a small group of courtiers. Anne was arrested on 2 May 1536 and taken by barge to the Tower of London, arriving at the private postern gate (now the Byward Tower).