There's no way I'd go into serious debt over a telescope or observatory - unless it could make me money somehow. That's the general rule of borrowing money - it must, in turn, make money. For example, buying a house: properties generally increase in value at a higher rate than the mortgage. Therefore, a property is making money. Another example is a vehicle. If you use a vehicle to get you to work, which allows you to earn money, then that is a positive investment.
If you use your telescope gear to make money, whether it be through astrophotography which you sell to a magazine, or through paid scientific research, then yes, it is a worthwhile loan.
Just my ten cents