Is it really that big of a deal?

No.They're adults they can do what they want.If they choose to smoke a little and let loose at the day's end then there's nothing anyone can do,they're not doing anything wrong.