An interesting project for sure. You may need to put in a raised floor as the walls will be quite high. You will also need a long sturdy pier down to a solid base to reduce any vibration and keep the scope and mount steady.
The advantage of an Observatory of course is that it is permanently setup and aligned so minimal time lost when you want to use it.
And being a 'Horse Float' it may fool the cloud gods and give you some clear skies ...