Such trains had limited stops (only went to a certain station). Fewer stops on a long journey make a faster trip, so one would want to take an express train (which avoided local stops). This is how it became synonymous with fast. Trains used to be referred to as the express versus the local in this sense.