When the Supreme Court decided earlier this summer that federal judges cannot interfere if states draw election district boundaries that favor the party in power, Chief Justice John Roberts invoked the Founding Fathers. Even James Madison had been the target of gerrymandering, Roberts wrote in the 5-4 Rucho v. Common Cause decision, using that as evidence to show the founders tolerated this practice and viewed it as an unavoidable part of our political system.

But before you conclude gerrymandering had the blessing of the founders and we are forever stuck with it, it helps to have the full story. What the Supreme Court did not mention was that if Madison’s political opponents had kept him out of the first Congress, the results would have been catastrophic: There likely would have been no Bill of Rights, then or perhaps ever, and Anti-Federalists might have succeeded in completely rewriting the Constitution. In other words, a gerrymander—before it was officially called by that name, of course—might have irreparably damaged the very document Chief Justice Roberts has spent his career defending.

The story begins in 1788, after Virginia ratified the U.S. Constitution in June. The state’s General Assembly had to create 10 congressional districts, deciding which counties to assign to each one. At the time, Patrick Henry, a member of the state House, was Virginia’s most powerful legislator and had almost complete control over the House and Senate. His influence over the Legislature was so obvious that George Washington wrote to Madison, “[Henry] has only to say let this be Law—and it is Law.”

Henry was also a leading Anti-Federalist who opposed the new plan and feared the Constitution would create a “consolidated” government that would be too powerful. Henry had tried to stop Virginia from approving the Constitution at the state’s ratifying convention. For three weeks in a sweltering converted theater in downtown Richmond, he had vigorously debated Madison, another delegate at the convention and the principal defender of the Constitution. Madison prevailed—the state voted to ratify the document.