For 10 months, the world’s most valuable coin sat wrapped in plastic on a folding chair in a little cagelike compartment behind a bright blue door at the Federal Reserve Bank of New York. It was only a step or two away from billions of dollars’ worth of neatly stacked bars of gold bullion.

On Monday, a man in a dark suit stashed the coin in his briefcase and coolly walked out of the Fed’s heavily guarded limestone-and-sandstone building, a couple of blocks from the New York Stock Exchange in Lower Manhattan. He nodded politely to the guards on the front steps of the Fed. They did not stop him.

The man with the briefcase, David N. Redden, an auction-house executive, was not pulling off a heist. He was taking the coin on a 6.7-mile ride to the New-York Historical Society on Central Park West....