If there weren't limits, a legit third party candidate might be able to raise more funds than a mainstream candidate just through grassroots campaigning and reaching more people willing to contribute.
How could they possibly raise more funds with rich people able to write million and billion dollar checks to mainstream candidates?