I think that the "buy binoculars first" line really comes from the "don't buy a cheap telescope" line.
Someone that is already interest in astronomy and stargazing might be able to make acceptable use out of a 60mm ebay refractor (hi ving

) despite its shortcomings (wobbly mount, terrible optics, terrible eyepieces). But it's not true to say that an inexperienced newbie will find the same challenge enjoyable, and they may give up the hobby in frustration because of those problems and what they were led to believe by the pictures on the box (525x magnification!!).
So that's why we say to people "don't buy a cheap telescope", and instead recommend something like a 6" or 8" dob for newbies. But obviously that costs more and might be out of the budget to start with. Plus, the average newbie might not even know if astronomy is a hobby they want to invest that sort of money in. $500 can be a large investment for someone who it turns out hates the cold, the mozzies, the challenge of finding things, all there is to learn. So the scope sits in the garage/cupboard, never to come out again.
That's why it's common to recommend a pair of binoculars first. Obviously the larger aperture the better, but even a pair of $30 10x25's can see Jupiter's moons, star clusters and bright nebulas. A pair of 7x50's or 10x50's will be even better, just scanning through the milky way is enjoyable seeing faint fuzzies and star clusters and bright star fields popping out at you.
And combined with a $20 planisphere, it's the best way to learn the night sky and figure out if astronomy is a hobby you want to pursue further before outlaying the money needed for a decent telescope.
I was given the same advice, because I was looking for a cheap telescope too. But I'm glad I took that advice.. I got a pair of 11x70 binoculars and used them for 6 months before saving up enough for a (and deciding on which) telescope. I saw comets, clusters, nebula, the moon, Jupiters moons, Saturns shape, etc, and it was a great introduction into the hobby.
Even in quite light polluted areas i think binoculars will still be better than naked eye. Obviously they're great in dark skies, but i'm still amazed by what you can see in binoculars compared to naked eye.
While we might be able to show someone Saturn in a good 8 or 10" well collimated scope with a reasonably good eyepiece and blow them away, the same can't be said for a cheap refractor and one of those 4mm eyepieces they supply. Saturn will be a fuzzy white blob with handles, I don't think that's going to blow anyone away, especially when they try to align on something with those shaky alt/az tripods while it wobbles in the wind.
With the price of good quality telescopes (like the 8" GSO dob) coming down lately, and the gap between decent binoculars and a decent scope closing, the advice may not be as relevant as it was a year or 2 ago, but I still think it's relevant for someone who isn't yet sure if they have an interest in astronomy.
Anyway just my 2c.. Good discussion!