Hi Bart
I've been fiddling with my OAG config recently and feel your pain
Suggest you first of all pull out the QHY5, point the scope at something bright (e.g. a white wall) and look through the OAG port - hopefully you see a reasonably unobstructed FOV? If not, get the OAG adjusted so you're seeing out the front of the main scope ok.
Then, with the QHY5 back in, focus your main camera on a bright star, or a large bright cluster. Use a ruler to measure distance from OAG prism to face of your QHY9 sensor - that's the distance the chip in your 5 has to be at.
Adjust your QHY5, set guide program to 1sec images, move QHY5 in and out slowly and smoothly until you see stars.
Good luck (what could possibly go wrong!!)
If you're convinced your spacings are wrong, you should be able to figure things out during the day once you know the required focus distance for your QHY9 on a star at night.
P.S. If your 'puter isn't close to the rig while doing this, consider installing something like Teamviewer so you have a screen on your phone or tablet with you showing your guide software right next to you while adjusting the guide cam.