Part myth, part truth. Once you have aggro'd a critter, they will stick to you unless they find a bigger threat, kill you, die, or start running away in fear of your awesomeness. You can aggro critters by attacking, taunting, debuffing, OR being seen. This is why you can stand in a huge crowd if you have complete invisibility (whatever combo you use to get there), until you take some aggressive action, including taunt and debuff auras. But if you have no stealth, critters will attack as soon as they see you.
Personally, I don't turn off my stealth auras because most of them have additional benefits that outweigh not being seen. If I or anyone around me wants to be seen, they just need to click some buttons.