Michael Caine is recalling an awkward encounter he had with a costar — and her ex.

The Alfie actor details his time working on Woody Allen‘s Hannah and Her Sisters in his new memoir Don’t Look Back, You’ll Trip Over — including the awkward incident where his on-screen wife, Mia Farrow, had a surprise visitor on set. “I remember one embarrassing moment when I was doing a love scene in bed with Mia – and I look up and there’s her ex-husband, André Previn!” Caine writes.

The Dark Knight star says that he was immediately apprehensive about the presence of Previn, who was married to Farrow from 1970 to 1979 and shared three children with her. ” I thought, bloody hell this is going to be tricky,” Caine recalls. “But it was fine.”

Perhaps more awkward was the fact that Allen and Farrow had actually been romantically entangled since 1980 — which Caine takes some credit for in his book. “In a funny way, it was down to me that she and Woody got together at all — they had that long relationship and made lots of movies,” he writes. “[My wife] Shakira and I used to go to Elaine’s, which was a great New York restaurant, you always saw interesting and famous people there. We loved going there. And so did Woody Allen, among many others.”

Caine says that he knew Farrow “from way back,” and that she used to visit him in London when she was beginning her career. “And then in New York one evening, she said to me, ‘Do you know Woody Allen?'” he recalls. “I said, ‘Yes, as a matter of fact, we do know him. Not very well, but he’s often in Elaine’s. He always seems to come in around the same time in the evening as we’re there.’ So she said, ‘Well, if you’re ever there and you see Woody Allen at another table, invite me along to dinner.’ And so I did. Which is how they really met.”

While filming Hannah and Her Sisters, the Italian Job star says that he witnessed some strain on the relationship between Allen and Farrow, who ultimately separated in 1992 after the filmmaker began a sexual relationship with the actress’ daughter, Soon-Yi Previn. 

“Mind you, somehow or other, Woody and Mia didn’t get on so well,” Caine writes. “You could see the tensions, even then. But I was obviously focused on the part. I was never close enough to them to know that it was all going to end so horribly. Very sad.”

Caine remembers Hannah and Her Sisters, which ultimately earned him his first Oscar for Best Supporting Actor, with nothing but fondness. “I loved the script and New York as a setting, and I had a great time trying to figure out playing this guy who has everything but is so restless and unhappy,” he writes. “He’s running around the city like a fool in this big coat trying to invent reasons to bump into the Barbara Hershey character.”

Want more movie news? Sign up for Entertainment Weekly’s free newsletter to get the latest trailers, celebrity interviews, film reviews, and more.

He continues, “He’s romantic but also pathetic. It was a fantastic role, I loved it — a complicated, middle-aged guy who lives in his head and isn’t even slightly heroic.”

Don’t Look Back, You’ll Trip Over is on shelves now.