The 79-year-old married Haines when he was 22-year-old and accepts that he could not cope up with things after he became a father.
"Pat became pregnant and we had our wonderful daughter Dominique, who is now 56. But it was the wonderful straw that broke an already very weak camel's back," thesun.co.uk quoted Caine as saying.
"I walked out of the marriage. The marriage breakdown was entirely my fault. I was too young and immature to take on the poverty and personal and professional failure," he added.
Caine is currently married to actress Shakira Baksh. He was recently seen in, 'The Dark Knight Rises'.
Photos: Katrina Kaif, Shamita Shetty at Mumbai airport
Photos: Varun Dhawan, Alia Bhatt promote 'Badri...' on 'Dil Hai Hindustani'
In pictures: 15 facts about kissing that will surprise you
Photos: Shahid Kapoor and Mira Rajput at cafe in Bandra
Photos: Deepika Padukone, Neha Dhupia and Soha Ali Khan at event