Rubbermaid Growbox in closet... Will my clothes smell?

I recently built a rubbermaid stealth grow box. I'm wondering if I should put it in my closet so the light won't bother me when i'm asleep, or bother the plant during flowering. So I want to put it in the closet in my room. I was wondering if my clothes will end up smelling? i've got a small computer fan for exhaust and will be adding a carbon filter soon. Is itsafe to have in my closet?