Southport Harbor has had a long and varied history. Known earlier as Mill River, Southport was active during the Revolution as a haven for privateers and as a launching point for whale boat raids against the British in Long Island Sound.

Southport was one of the leading harbors on the Sound, shipping most of its produce to New York City. The fall and winter months were busy times for the harbor. Onions, the main crop of the area, would be harvested and warehoused along with other produce in the buildings lining the harbor side. Sailing day would see the village streets lined with carts of local farmers with goods for New York City.

In 1849, when the railroad went through, commercial interests in the harbor declined until by the turn of the century, the market boats had virtually disappeared. They were replaced by a rise in private yachts. Housed in the remodeled stores from the 1800s, Pequot Yacht Club now dominates the waterfront.