John Connally was the oldest of two sons born to the marriage of George Connally and Margaret Mathis. His younger brother was Charles. John's mother died circa 1774 (based on the birthdate of his oldest half sibling), when he was seventeen. His father remarried to Frances Moore (daughter of Robert Moore) at Brunswick County, Virginia circa 1774. John's younger half siblings were Thomas Oldham, Angelico, Rebecca, Sarah, Ann, Kitturah "Kitty," George Oldham, William Isaac and Frances "Fanny" Connally.

John was born in Virginia. It has not been determined if he was born at Brunswick County or Richmond County. His father is reported to have been born at North Farnham Parish in Richmond County, but he may have been born at Essex County. His father was residing at Brunswick County in 1781 and Amelia County in 1782. John was a witness to a land deed in 1782 at Halifax County, VA which borders Caswell County, NC. The land deed was a purchase made by William Moore for 100 acres in Halifax County. His father's land in Halifax County, VA crossed over into Caswell County, NC.

John married Sicily Moore on May 23, 1788 at Caswell County, NC. She was the daughter of William Moore, who had been residing at Caswell County, NC since at least 1777. It is unknown if/how William Moore is related to John's step grandfather, Robert Moore.

John and Sicily were the parents of John Oldham, Drury Moore, Mary "Polly", Charles, Archibald, George W., Mildred "Milly" and Angelica Connally.

John and Sicily remained in Caswell County until 1800 when they moved to Franklin County, Georgia along with John's brother, Charles. Both John and Charles died in 1807. Sicily died in 1809. John's estate was finally settled in 1820. Neither John nor Sicily left a will. Their minor children were placed in the care of John's half brother, Thomas Connally, and returned to Caswell County, NC with him.