It's all personal preference, for the most part. I'd gauge it based on how bushy the plant is and how you want it to grow out. Right now, you're basically laying the foundation for your house. Whether you want an open floor plan, or a 3 story townhouse is up to you.