Port Elizabeth's Main Public Library is situated in the north-western corner of Market Square and was constructed in 1835. It was used as a courthouse from 1854 until the present library building was officially opened in 1902. The building is regarded as an excellent example of Victorian Gothic architecture. Its striking facade was manufactured in England and rebuilt piece by piece on the present site.

The interior of the building complements the facade perfectly. It features a stained-glass dome and windows as well as a myriad of little balconies. The Sicilian marble statue of Queen Victoria, at the library entrance, was erected and unveiled in 1903.