Depends on the scene. Just keeping shooting and stitching until you can't shoot or stitch no more.
Most important thing I reckon is to make sure the frames overlap by enough (I try to get 1/3 to 1/2 of each shot overlapping) for the stitch to work. And make sure the end frames extend beyond where you actually want the shot to end.