The 29-year-old was run out on 99 on the final day of the drawn match and landed in a controversy after a window in the dressing room was shattered, leaving a spectator in the members' area with minor cuts from the broken glass.
Prior was merely reprimanded by the ICC after the incident and the wicketkeeper also played it down.
"Every time I looked up at the pavilion during Sri Lanka's second innings, I just saw this big broken window and I felt stupid more than anything else," Prior told 'London Evening Standard.'
"When you make a scene like that, you're thinking, 'Oh no, what are people going to say?' It didn't look good, obviously, and I'd just been run out, but people were putting two and two together and coming up with five, seven, nine and 10," he added.
Prior asserted that he was not angry and did not throw his gear around.
Deccan Herald is on WhatsApp Channels | Join now for Breaking News & Editor's Picks