No. They are not hurting the plant. The plant will use the energy from them before it discards them and they fall off. If you slightly tug on them and they stay put, leave them alone. When they are ready to come off, they will almost fall off without even pulling.
And yes, that would be the...