Let's see...
Clouds, wind, fog, rain, work (getting in the way of my astronomy of course

), work (the fact that I need to), light pollution, social / family events on clear nights, gear that doesn't work as it should or isn't reliable, the jet stream, that creaking noise that says my mount is garotting itself with a cable, kicking a tripod leg (after polar alignment or during an image), Packign the car and driving to a dark sky site with the scope only to find some critical part of the rig has been left at home

(I left the tripod at home once

no idea how...), getting set up, polar aligned and then having the sky cloud over...
But hey, they are all character building facets of our hobby...

Al.