There's plenty to do in a day. That sounds like a fine amount of time to see the island for a first time. The ferry will take an hour. You can bike the loop from Old Harbor to the Southeast Light/Mohegan Bluffs and around to New Harbor (about 6 1/2 miles), Have lunch, do some shopping, walk on the beach, etc.