From the 1830s to 1965 Castle Williams on Governors Island served as a military prison. Over the years prisoners have attempted to escape by swimming across the Buttermilk Channel to Red Hook.

The New York Times Tuesday, AUGUST 29, 1901

ESCAPED BY SWIMMING Prisoner on Governors Island Makes a Dive for Freedom. Supposed to Have Been Picked Up by Passing Tug—Third Escape of Notorious Deserter.

Ex-Private George Harvey of the Second Cavalry. United States Army, who, up to 3 o'clock yesterday afternoon, was a military prisoner on Governors Island, and who is notorious throughout the service as a deserter, escaped yesterday by swimming half way across Buttermilk Channel, where he Is supposed to have been picked up by a passing tug. It was the third time that Harvey has eluded the vigilance of the authorities. All the afternoon and last night a squad of men, under Sergt. Way, the Supervisor of Castle Williams, were searching for the escaped prisoner in Brooklyn, where he is supposed to be in hiding. That he was picked up by a tug was vouched for by several civilians, who averred that they had seen the man when he jumped into the water, and had watched his subsequent movements up to the time he was fished out and carried away. They did not get the name of the tug.