like how in humans the baby in the womb is becoming either male or female.
Same thing there as well, sex is determined in humans the moment the sperm penetrates the egg. The sperm carries the missing chromosome, the egg already carries the X chromosome. From there the sperm delivers a X or Y chromosome to complete the chain. XX, XY.
Stress can cause more hermaphrodites. But a better way to say it would be, trying to grow a plant in a environment it is not used to being grown in will cause more hermaphrodites.
Most cases of hermaphrodites are caused by environmental stress, those that are not are caused by genetics and will happen even given the best growing conditions.
I have also observed what some people refer to as " hermaphrodite " is really just a case of over ripening or a genetic asexual trigger that seems to happen in strains.
To me, a hermaphrodite will clearly express both male and female organs at the nodes. In other words, it will grow both pistils and axial limbs with pollen sacks on them.
Over ripening and the genetic asexual feature that I refer to show up as banana's growing straight out of the bud during late development. I do not consider this to be a " true " hermaphrodite.