In 1278 a leprosy hospital with a chapel and a cemetery was founded outside the city walls. In the 16th century the cemetery became the only burial place of the city and it became necessary to enlarge it. During the 16th and 17th century the cemetery was expanded several times and at the same time was rearranged in the style of a Campo Santo. The vaults there were either inherited or bought, but they were very expensive and even famous people like Bach or Gellert were unable to buy one.

During the Thirty-Years-War it was used as a camp by swedish troops that ravaged it. In September 1813, one month before the Battle of Leipzig it became hospital and prison because the facilities in the city were already overcrowded. The soldiers lived in the vaults and used the wood of the caskets as firewood, while the undertaker, Johann Daniel Ahlemann, continued his work. His notes from these days were later published.

On October 18th 1813, the last day of the Battle of Leipzig, the cemetery suffered further destruction when French troops entrenched inside.

In the fourth Section, that was opened in 1805, exist unmarked mass graves with victims from the Seven-Years War and the Battle of Leipzig.

The last person was buried here on December 24th 1883. The first and second section (the area around the church to the back of todays Grassi Museum) were secularized in the same year.

Between 1894 and 1897 the St. John's church got a new nave. During the preparation for the construction the grave of J. S. Bach was found.

Between 1925 and 1929 the old St. John's hospital was pulled down and the New Grassi Museum was build. In the night of December 4th, 1943 most of the church of St. John was destroyed only the tower could be saved. The crypt with the graves of Gellert and Bach was opened and the bones of the two were moved to the University Church (Paulinerkirche) and St. Thomas Church. In 1963 the tower of the church was blown up. Between 1980 and 1995 the cemetery was closed and old gravestones were reconstructed and repaired.

On June 21th, 1995 the cemetery was reopened as a park and museum cemetery. Some stones from New St. Johns Cemetery were placed in the south part, from where they had been saved in the seventies when that cemetery was closed to the public and secularized.

During the changes in the last 120 years (the secularization, the extension of the streets left and right of the cemetery, the building of the Grassi Museum and the Gutenberg school in the back of the cemetery) all the old family vaults were destroyed and many gravestones were either lost or were moved from their original place.