Ok, you can pick them any time, they don't really get "ripe". If you pick them small they will be more potent. I suggest you pick them just before or after the veil breaks but that is just to avoid the purple mess.
If you enjoy the way they look and many of us do, wait till the caps are...