The only way to visit the Colosseum in the evening is via a guided tour, so you will have a group and a guide with you during your experience. That being said, there are simply so few people around compared to during the day that you can't help but revel in the lack of crowds.