The "Iron Man" star suffered with substance abuse for years before getting clean after a spell behind bars and now admits he was not always sober on film sets.
In an appearance on the new season of David Letterman's Netflix talk show, "My Next Guest Needs No Introduction". Downey opened up about his drug abuse, telling the veteran TV host cocaine and heroin were among his drugs of choice.
Recalling his work on "Home for the Holidays", which Foster directed, the actor said, "I call it the most relaxed performance in the history of cinema. She was really critical of just being like, 'Well, looks like you're getting away with it on this one (scene). I wouldn't try this again because we're kind of a forgiving group.' "
"I was like, 'Wasn't that last take great?' She's like, 'Yeah, you're great. It's going great!' "
"And then when I was locked up in a penitentiary, she sent me a letter saying, 'Let me tell you about what I meant by it's going great.' "
The actor then credited his producer wife, Susan, for getting him straight and helping him stay sober, revealing that she had been secretly assigned to chaperone him for one film project during his battle with addiction.
But Robert insisted he was never "blotto" on set, but once had to be revived from a "near-coma" to shoot scenes.
"Back then, when I was a kid, doing that, you could say that it was fun, but it was pretty stressful too," he said.
As part of the chat with Letterman, the two men toured the actor's Malibu, California 'farm,' where he keeps rescued alpacas, pigs, and goats, among other pets.