Spray with bt. Bacillus Thuringiensis, I think that's how it's spelled. It's a bacteria, specific to caterpillars iirc. After they eat it it destroys their digestive system and they die. It's suitable for organic growing and a lot of guys use it as a preventive.
If the caterpillar you found wasn't the cause of the damage, it would have made it worse, and if there's one there are likely to be more, or at least more eggs.