a noob question about white flies (after reading many posts)

OK, I have read thru many many posts about white flies. My plants are infested with them. I have only 3 wks to go intil harvest so I am worried about spraying plants with neem oil or similar because I dont want to get bud rot. From what I understand the white fly larva does damage to the leaves only. I have found several people asking the same question I am about to ask. This leads me to my question:

If I just leave the flies alone and shake plants well before harvest will that be OK? It doesnt bother me if I smoke a few strangling white flies. I dont care if they munch a few leaves just want my bud safe. Thanx

i found out what i needed about white flies at another site.