Madison Ownbey grew up on his father's Broad River farm. He apparently grew up fast because, according to the 1820 Rutherford County census, Madison was married and had one child by the time he was 18. He married Nancy Morgan around 1818, and the family moved to Swannanoa by 1822. In that year, Madison was baptized by Elder Stephen Morgan. He became a member of the Baptist Church at the North Fork of the Swannanoa (probably what is now Mountain View Baptist Church).

Madison Ownbey and his father, James, both moved to Fairview by 1830. In 1827 James Ownbey received 150 acres on Gap Creek from the State of North Carolina. Madison Ownbey is listed on the 1830 Buncombe County census with 8 children, living next to his brother, William, and his father. They appear to have been living on Gap Creek, judging by who their neighbors were.

Madison was a very religious man and was very active in the Baptist church from the time he joined in 1822 until his death. He joined Cane Creek (now Fairview) Baptist Church when he moved to Fairview in the 1820s. In 1831 he was the messenger from Cane Creek Baptist Church to the French Broad Association (the local association that united the Baptist churches in the Buncombe-Henderson area) and attended a meeting held at the Flat Creek Meeting House in August. He was chosen to write the Nolachucky Tennessee Association in 1833.

In 1838 Cane Creek Baptist Church sent a group of messengers to the formation meeting of the Salem Baptist Association at Fletcher. The messengers were the Pastor, Rev. Bailey Bruce, D. Garrison, Sr., J. W. Freeman, Joshua Whitaker, and Madison Ownbey. The meeting was held September 14–15 at Blake's Meeting House. The nine churches attending, Beulah, Cane Creek, Crab Creek, Ebenezer, French Broad, Mills River, Mt. Pleasant (now First Baptist of Asheville), Mud Creek, and New Bethany (the forerunner of Broad River Baptist Church), broke away from the French Broad Association and formed the Salem Association. Madison Ownbey was chosen as the clerk of the new association. He was instructed to have 400 copies of the minutes printed and distributed.

Madison continued to be active over the next several years. In 1838 he was the messenger from Cane Creek Baptist Church to the French Broad Association. The following year he was chosen to write the 12 Mile Association (Baptist) correspondence letter. In 1840 he was the messenger from Cane Creek to the Salem Association, and in 1841 he was again the messenger to the Salem Association and was chosen to write the French Broad Association correspondence letter.

In 1842 Madison Ownbey decided to follow the lead of many Fairview families and move to Gilmer County in Georgia. Gilmer County was part of the lands opened up to settlement after the Cherokee Removal. Many families moved there to take advantage of the cheap land, but Madison apparently was not satisfied with the new territory. He moved back to the Fairview area before 1844, and settled in the Flat Creek area near Nesbitt's Chapel.

Madison's first wife, Nancy Morgan Ownbey, died in childbirth, either on the way to Georgia or on the way back. She was said to have died in the Nantahala Mountains. Madison remarried shortly after his return to Fairview. His new wife was Elizabeth Clements (Clemmons), the daughter of Cornelius Clements (Clemmons) and his first wife, Isabella Porter. The Clementses lived near Nesbitt's Chapel on Flat Creek, and this may have been the reason Madison moved to that area.

Madison joined New Bethany Baptist Church, located in the Broad River area, and was ordained a deacon there in 1844. (New Bethany Church ceased to exist by the time of the Civil War.) By 1847, however, Madison was again a member of Cane Creek Baptist Church, and that year he was again chosen to be a messenger from Cane Creek Baptist to the Salem Association. Ownbey was named administrator of the estate of his father-in-law Cornelius Clements when he died in 1850.

The Ownbey family moved to Mills River in Henderson County some time in the early 1850s. Two of Madison Ownbey's sons-in-law, Soloman Whitaker and Riley Whitaker, moved to Mills River about the same time. Madison and his wife, Elizabeth, joined Mills River Baptist Church in 1859, and on the day they joined, Madison was appointed the church's clerk pro tem. He served in Mills River Church as deacon, clerk, messenger, and any other position where he was needed, almost to the day he died.

Madison died November 27, 1868, at his Mills River farm. He was 67 years old. Elizabeth Clements Ownbey went to live with some of her children, but we do not know when she died.

Madison Ownbey was buried on a hill in back of his house. The following poem was written on his tombstone:

"His trials and conflicts all are past, His body's mouldering in the dust, His soul has reached sweet heaven at last, To dwell with Jesus Christ we trust. No more his kind voice is heard, To cheer the dear ones he has left, The sorrows he here with them shared, Are only felt by us bereft. Weep not for me this loved one said, I hope in heaven to find a rest, That when this earthly body's dead, My soul shall be forever blest."

Madison Ownbey was the father of 25 children by his 2 wives. His first child was born around 1819, and his last child was born 47 years later in 1866. They used to say in Mills River: "Here comes Uncle Madison Ownbey hobbling down the road on his walking stick with a baby under his arm."