You're not doing anything wrong, they are both tricky objects.
With the Rosette, the size is HUGE - it takes up a large area of the sky. In my 8" f/6 with a 25mm ep I can only see a quarter of it at a time, and then it is very faint and hard to detect. So low power for this one, or as wide a field as you can go and dont expect to see a circle, more an arc of nebulosity.
I can't help you with the Cone, I've never gone for it from memory (or maybe I have, and was unsuccessful and just don't remember...) but it might be smaller than you think...? Nope, don't know, we need someone else to anser there.
