Charleston, South Carolina's Magnolia Plantation & Gardens claims to be the nation's oldest public garden, although it opened to visitors in 1870, two decades after the national botanic garden's debut. It's also allegedly home to oldest collection of Indica Azaleas in the country.