Albion was the son of Anson Barton of Sidney, Kennebec County, Maine and Rhoda Sisson of Douglas Parish, York County, New Brunswick, Canada. Rhoda and Anson were married on 05 January 1826 in Douglas Parish, York County, New Brunswick, Canada. Rhoda and Anson had twelve children. They were: Albion Barton was born on 03 October 1826 in Douglas Parish, York County, New Brunswick, Canada; George Sisson Barton was born in August 1828 in Douglas Parish, York County, New Brunswick, Canada; Greenleaf Barton was born on 02 May 1831 in Sidney, Kennebec County, Maine; Herrick S. Barton was born on 14 March 1833 in Hallowell, Kennebec County, Maine; Jonathan Quincy Barton was born on 20 March 1835 in Sidney, Kennebec County, Maine; Anson Bliss Barton was born on 27 November 1836 in Sidney, Kennebec County, Maine; Mary Hellen Barton was born on 26 October 1838 in Sidney, Kennebec County, Maine; William Henry Harrison Barton was born on 13 March 1840 in Sidney, Kennebec County, Maine; Charles Carroll Barton was born on 21 March 1842 in Sidney, Kennebec County, Maine; Ellen Victoria Barton was born on 08 May 1844 in Sidney, Kennebec County, Maine; Mary Ellen Barton was born on 07 October 1845 in Sidney, Kennebec County, Maine; and Caroline Augusta Barton was born on 01 March 1848 in Sidney, Kennebec County, Maine.

Albion married Elizabeth Arnold Carr in November 1850 in Maine. Elizabeth and Albion had five children. They were: Charles Albion Barton was born on 17 June 1852; Frederick Eugene Barton was born on 20 August 1856; Frederick E. Barton was born on 20 December 1857; Willard Anson Barton was born on 24 November 1865; and Arthur Flint Barton was born on 13 February 1872. All the children were born in Sidney, Kennebec County, Maine.