The 65-year-old actor presented the Video of the Year prize to the star for her tune "You Need to Calm Down", when he confused the drag queen - who appears in the video as Taylor, 29 - for the singer herself.
But speaking to Dallas-Fort Worth radio station Hot 93.3 on Thursday, he insisted he "was looking for" the "Shake It Off" star amid the chaos, as there were "so many people that bombarded the stage".
"The video has me trying to find her, and you know, I thought it was so funny the way it was interpreted," the "Grease" actor said. "And it's cool, I didn't care."
The star also referenced the viral moment from the Academy Awards five years ago, when he called Idina Menzel by the name Adele Dazeem, quipping: "If I shave my head, it's headlines. If I mispronounce something, that's headlines, and I know that about me."
He also joked "it would have been awesome" had he handed the award to Jolie, and that he "should have just gone all the way with it".
The drag star has performed as Taylor for several years and often takes the stage at Play Louisville in Kentucky.