Adam is the son of Adam Oliver and Mary Marshall. Born January 31, 1825, he was baptized at the Presbyterian Church in Killinchy, Down, Ireland on February 4, 1825.
~ Ireland, Births and Baptisms, 1620-1881

Adam and his brothers of working age are found working in various households after arriving in New York City.

1850 United States Census
Gates, Monroe, New York
Dwelling 154 Stephen C Sprout Household
Adam Oliver 23 Ireland
Mathew Marshal 24 Ireland
John Stewart 27 Ireland
James Marshal 26 Ireland
Robert Marshal 19 Ireland
Dwelling 129 Henry L Monroe Household
Samuel Oliver 17 Ireland
Dwelling 367 John D Esky Household
James Oliver 16 Ireland
Dwelling 128 Joseph Scott Household
James Oliver 15 Ireland,
Living in New York City, New York
Dwelling 338 Household of Elenor Bucklin
John Oliver 23 Ireland Painter

Per family records, Adam married Nancy Marshall about 1852 in Monroe County, New York, possibly a sister of the Marshall boys on the above census.

"Adam Oliver, Farmer; Sec. 27; P.O. Harrison; born in Ireland, Feb. 2, 1825; married Mary Marshall, in 1852; went to California, where she died, in 1857; had one child, Mary Jane; returned to New York, and married Agnes Marshall, June 17, 1858; she was born in Ireland, July 17, 1836; have seven children living: Libbie, born May 10, 1859; Annie L., Sept. 3, 1860; Ellen, Sept. 18, 1863; Martha, March 8, 1865; Hattie, April 16, 1866; Ida F., Feb. 5, 1868; and James R., Jan. 17, 1870; lost three: ___, May 8, 1863; Robert, Sept. 17, 1862; and Nettie, Feb. 27, 1869; they all attend the Presbyterian Church; the mother of Mrs. Oliver (Mary Marshall), aged 93, resides with her; is very smart and active; has the use of all her faculties; can use her needle as deftly as any girl; is supposed to be the oldest person now living in Harrison.
~The History of Winnebago County, Ill., Its Past and Present. Chicago : H. F. Kett & Company 1877 pp 599-600

1860 United States Census
The Town Of Harrison, Winnebago, Illinois
Dwelling 4425
Adam Oliver 35 Ireland Farmer
Nancy Oliver 24 Ireland
Mary J Oliver 7 New York
Elizabeth Oliver 1 New York
Dwelling 4426
James Oliver 25 Ireland Farmer
Robert Oliver 22 Ireland
Martha Oliver 16 Ireland
Dwelling 4428 possibly Nancy's family
James Marshall 29 Ireland Farmer
Mary Marshall 70 Ireland
Alexander Marshall 12 Ireland
Living in Troy, Rensselaer, New York
Dwelling 858 Martin J Mosher Household
Mary Oliver 20 Ireland Servant
Living in The Town Of Caledonia, New York
Sarah Edna Oliver Boyd, Samuel & children

1870 United States Census
The Town Of Harrison, Winnebago, Illinois
Dwelling 4
Adam Oliver 42 Ireland
Nancy Oliver 32 Ireland
Elizabeth Oliver 11 Illinois
Anna Oliver 10 Illinois
Ellen Oliver 8 Illinois
Martha Oliver 5 Illinois
Hattie Oliver 4 Illinois
Ida Oliver 2 Illinois
Robert Gilmore 12 New York
Dwelling 5
James Oliver, wife Jane Champion & family
Dwelling 6
Sarah Oliver Boyd, Samuel & family
Dwelling 123
Martha Oliver Miller, William & family
Dwelling 127
Mary Oliver 67 Ireland [Mother]
Living in Burritt, Winnebago, Illinois
Mary Jane Oliver Shields, Nelson, children
John Oliver, wife Sarah Miller & family
Living in Owen, Winnebago, Illinois
Robert Oliver, wife Jane Atkinson, children
Living in Jenkins Twp, Mitchell, Iowa
Eliza Oliver Allen, T W & family

1880 United States Census
Harrison, Winnebago, Illinois
Dwelling 76
Adam Oliver Self 55 Ireland
Agnes Oliver Wife 44 Ireland
Elizabeth Oliver Daughter 21 IL
Ellen Oliver Daughter 16 IL
Martha Oliver Daughter 15 IL
Hattie Oliver Daughter 14 IL
Ida Oliver Daughter 12 IL
James Oliver Son 9 IL
Dwelling 75
James Oliver, wife Jane Champion, children
Dwelling 41
Robert Oliver, wife Jane Atkinson, children
Dwelling 23
Chester Branning & wife Sarah Edna Boyd, granddaughter of Adam Oliver & Mary Marshall
Dwelling 124
Sarah Oliver Boyd, Samuel & family
Living in Burritt, Winnebago, Illinois
John Oliver & wife Sarah Miller & children
Living in Brookings, Dakota Territory
Mary Jane Oliver Shields, Nelson, children
Martha Oliver Miller, William, children
Living in Jenkins Twp, Mitchell, Iowa
Eliza Oliver Allen, T W & family

Adam Oliver was married 1st to Nancy Marshall in 1852. They went to California. Mary Jane Oliver was born c1853. Nancy Marshall Oliver died there in 1857, perhaps in childbirth. Adam married 2nd Agnes Marshall, in New York, 17 June 1858.

Irish girls named Aignéis were often called Nainsí, until the 18th century, when parents started naming their daughters Nancy. This would explain why Adam's wife Agnes is called Nancy on the census records in 1860 and 1870.

Nancy Agnes Marshall Oliver died 23 August 1893 in Harrison, Winnebago County, Illinois.

Using photo editing, I tried to enhance and clarify Nancy's tombstone. It appears to say "Nancy A - Wife of Adam Oliver - Died - ??? 23, 1893 - Aged 57 yrs 10 mos 4 dys"

New York Passenger Lists
Immigration 2 Jan 1839-29 Jun 1839
Ship Westchester
George Marshall 30 Ireland Laborer
Jane Marshall 30 Ireland
Nancy Marshall 3 Ireland