On May 4, 1814, after months of preparations by the Building Committee of St. Paul’s Church and builder Robert Carey Long, Rev. Dr. James Kemp gathered his congregation at the northern end of Charles Street to lay their cornerstone for their new church. In a sermon delivered at St. Paul’s Church in 1878, Revered John Sebastian Bach Hodges commented on the pressing need for a new building in 1814:

The ground upon which the Church stood, we have already remarked, was high, but that occupied by St. Paul Street and by the first five houses which front on Lexington Street counting from the corner of St. Paul had, as early as the beginning of the century, been reduced to the present level. The consequence was caving in of the hill upon which the Church stood, which, about 1811 or ’12, was obliged to be protected by a strong wall the defence proved inadequate, and the consequence was that it was indispensable to build a new Church, hence the erection of the present S. Paul’s, which was consecrated in the year 1817.