What is the closest beach to Charleston? Sullivan's Island. To be fair, though, the difference is statistically insignificant: the drive from downtown Charleston to Sullivan's Island is about 20 minutes, or 9.5 miles. The drive from downtown to Folly Beach, the next closest, is about 21 minutes, or 11.5 miles.