You're partly correct in that the streaks are caused by star drift and correction in the stacking process, but the prime cause is noise. If alignment was perfect and there was no flex etc there wouldn't be streaking, but the stack would still be noisier than if it was dithered.
Give it a try and see what happens.
|