The H. Lee White Marine Museum is housed in the former administration building for a huge, then state-of the-art 1920's grain elevator complex on the Oswego Harbor's west side pier. The grain elevator itself was demolished in 1998, but the two-story steel frame, smooth stucco-finished administration building was retained by the Port of Oswego Authority and generously provided to the Museum.

During its decades as an administration building for the grain elevator, the structure housed a machine shop, stock room, heating plant, main switchboard and superintendent's office on the first floor. The second floor comprised "welfare rooms" for the workmen and other offices.

The administration building today is a fully-functioning museum of maritime history, with exhibit spaces, storage rooms, offices and online connections to the outside world. Traces of its original uses can still be seen by visitors on the main floor, where a few of the ceiling-mounted, belt-driven pulleys of the old machine shop have been left in place.