In 1841 whilst at Cambridge, he met Henrietta Phillipps and they became engaged. Her father, Sir Thomas Phillipps, was a renowned book collector who by the age of only 6 owned over 100 books and went on to amass the greatest collection of books and manuscripts in this country. His obsessional collecting caused him to work through a large fortune eventually driving his family into debt.

Halliwell was accused of stealing manuscripts from the Trinity College library which so enraged Sir Thomas Phillips that he refused them permission to marry causing them to elope and eventually marry in 1842. Sir Thomas refused to see either of them again. In 1848 Halliwell published his Life of Shakespeare illustrated by John Thomas Blight, a Cornishman. Blight had a mental breakdown partly caused by Halliwell never paying him for his work.