WHILE VACATIONING ONE summer in the late 1930s, Maude Terry decided to go fishing. On her way to Gardiners Bay in eastern Long Island, she came across a secluded, underdeveloped, marshy, wooded area that faced a beach. Immediately, she felt a sense of tranquillity in the sylvan space, surrounded by tall old oak and walnut trees. Green shrubbery and weeds grew amid the sand at her feet, and her skin turned sticky in the salt air. It was heaven.

At the time, Terry was a Brooklyn schoolteacher who spent most summers with her husband, Frederick Richards, and her daughter, Iris, who were both doctors at Harlem Hospital; her sister Amaza Lee Meredith, the chair of the art department of Virginia State University in Ettrick, Va. (who was also one of the first Black female architects in the United States), would occasionally join them. The sisters had grown up in Lynchburg, Va., and lived most of their lives up and down the East Coast: Come summer, Terry would usually rent a cottage in Eastville, an area on the outskirts of Sag Harbor, the beachfront village that — although it straddles the rich, mostly white enclaves of Southampton and East Hampton — has always remained a bit more subdued, at least compared to Long Island’s other storied warm-weather escapes, which begin at the eastern edge of Queens and stretch more than 100 miles out into the Atlantic Ocean.

Just over two square miles large, Sag Harbor had grown into a bustling port town by the late 1700s, after an influx of whalers, ship captains and their crews had settled in the area. But until the mid-1900s, Eastville remained an outlier, several blocks that were singular in the region for their embrace of diversity, welcoming Native Americans, manumitted Black people and European immigrants from France, Portugal’s Azores and Cape Verde, Africa. The neighborhood was one of few places where Black and Native Americans could coexist without experiencing daily, virulent oppression. Eastville, in fact, had been a forerunner in welcoming Black men who were formerly enslaved, many of whom found work in oceanside towns as whalers, fishermen or shipbuilders. The women, meanwhile, worked as seamstresses, launderers or bakers to earn money while their husbands were at sea for years at a time. Often, the wives were the property owners so that they could keep their home and family together in the event that a ship didn’t make it back, which offered these women unprecedented agency.

Even long after the abolition of slavery, most Black people in the United States had difficulty becoming homeowners, primarily because they were discriminated against when they tried to get bank loans. Their mere presence in a neighborhood was also thought to devalue property, so many white residents wouldn’t live near them. In Eastville, things were different: In 1840, in response to segregated conditions at Sag Harbor’s churches, Black people were even able to build their own church, St. David A.M.E Zion, which is believed to have been a stop on the Underground Railroad.