Northcote Cemetery is quite a small cemetery set on about 1 acre of land tucked in between houses just off Separation Street on its north side, a little east of High Street. There are entrances at the north and south ends of the cemetery, however the northern entrance is permanently locked. The pedestrian gate at the south end is open 24 hours a day.

The land on which the cemetery sits was purchased by four German settlers, August Schwaebsch, Albert Fritsche, Moritz Heiner, and Edward Hellwig, from Peter McArthur in 1861 to be held in trust as a cemetery for the residents of Northcote. However the cemetery was already in use prior to 1861, with the mother of Moritz Heiner, Dorothea Heiner, being the earliest known burial at the cemetery in January 1855.

The northern part of the area now called Northcote was briefly named "Newmarket" in the mid 1850's, when the area was just being settled, and thus the cemetery was briefly known as "Newmarket Cemetery".

The western end of Separation Street, where it meets High Street, was mostly settled by German immigrants, in the second half of the 19th century, and they referred to that area also as "Cawdortown". Separation Street, which was named in celebration of the separation of the State of Victoria from the State of New South Wales, was also for a time called "German Lane". Many of the local place names and street names in the area were German but were changed during Australia's participation in World War 1, when there was a lot of anti-German sentiment. During the second half of the 19th century the cemetery was also sometimes referred to as "The German Cemetery", although it became the final resting place for many people not of German descent as well.

The cemetery fell into disrepair for a number of decades in the second half of the 20th century, and many headstones were destroyed or lost. There are more than 1500 people buried in the tiny cemetery, but only headstones with engravings for about 200 of them.