Business card faux pas?

One of my English students is an anesthesiologist, and we’ve developed a strong rapport. We really enjoy talking, and he recently gave me his business card. Last night during class, I told him that I had hung his business card on my refrigerator with a magnet, and he paused and said “Refrigerator? So, you’re saying you used a magnet to hang my business card on your refrigerator?” I said “.. Yes 😬” His demeanor seemed to change and we continued the lesson. After I noticed his reaction, I explained that Westerners hang things they consider important on their refrigerators, and he he just said “Ok”.

I’m guessing that was probably not cool? I’m bummed, and I hope I did not offend him. His demeanor after I told him was that of someone trying hard to be nice about something that’s upsetting to them.