Am I being too demanding here about my ex's treatment of our son?

I am very picky about my son. He isn't in daycare, he socializes with other people who have similar ideas about children as me. He eats mostly organic, depending on the season. Isn't allowed sugar or extra salt and never asks for it since he is only 2 and doesn't know about it(he gets cake for bday okay?) I believe in dressing him in nice clothes, not wife-beaters or anything a bit ghetto because children need to learn to be individual but also decent for jobs later on. Etc etc. My ex smokes, my son comes home reeking of it, though he claims he isn't smoking in front of him and I worry about my son inheriting my father's asthma and allergies that were caused by second-hand smoke and are severe. He sprays him in fabric softner to cover the smell and I think it's what has caused the rash on my son's face. My son comes home constipated from a poor diet or from drinking too much milk (doctor verified). He comes home saying words incorrectly or pointing out people farting...which is embaring and his ability to speak in complete sentences at such a young age is due to me teaching how to talk properly, not baby talk. His father has him for just 2 days a week and he watches nothing but tv or goes to play at McDonalds or he tries to take him to things like the demolition derby even though my son is scared to death of loud noises or places like the Keenland race track with lots of stairs, smoke, and drunk idiots. SO my question is, should I limit daddy time? My husband is a great step-dad and we regularly take him to fun kid things and I think he would be better for it. Especially since he comes home with this screaming attitude that I have to spend 2 days fixing. I don't like spending 2 days a week punishing and upsetting my son due to a father's inability to teach him how to behave (and I don't , I hurt his feelings by making him sit down). I have argued and talk and tried everything, nothing is getting better.