New London Literary and Scientific Institution - In 1854, the Ladies Boarding House (later called Heidelberg) was built on what is now the New London green to accommodate up to 40 female students and the female faculty. Anthony Colby also purchased the original New London town meeting house and moved it to campus, where it was renovated to provide 20 double rooms for the male students. The building is called Colby Hall. In 1870, a brick Academy building was dedicated, located on the present site of Colgate Hall. The building provided dormitory space for 100 female students as well as classrooms, laboratories, library, gymnastic facilities, chapel, dining room, kitchen and laundry facilities. It burned in 1892.
The New London Literary and Scientific Institution was in 1878 renamed Colby Academy in tribute to the ongoing support of the Colby family of New London.