First off, if I happen to spot a loofa and some sort of body wash in my man's bathroom, odds are, I'd never give it a second thought. If for some reason my mind decided to wonder in that direction, there would be no negative thoughts unless he was stashing something like a Sponge Bob loofa and Bubble Gum scented body wash. Sensitive men are a turn on. Children are not.
Personally, I find it a real turn off when men are passively homophobic enough to get hung up on stupid stuff like thinking that it's gay to wear pink, use loofas or scent their bodies. It's much more important to care for the body which a loofa helps to do. Those things are great for helping to control things like callouses and general body itching due to dry, flakey skin.
Regarding body wash scents, it's important to find something that mingles well in the room and works with your personal body chemistry. What smells great on one person may reak on another. Until you find what seems to work best, have fun experimenting.
And for God's sake, have the balls to call a loofa a loofa! Men havinig sex with other men is gay. Calling a loofa something other than a loofa and using loofas along with body washes is not gay. In fact, if anything, I would tend to think that sort of man would be more self assured and much more sensual in bed.