Stars are essentially mathematical points. It is diffraction caused by the opening of the tube that introduces an apparent disc size (known as the spurious disc) to the image. The fainter the star, the smaller is the spurious disc. In the case of the 8" scope you mention, the magnification enhancement is 11.4X, and the apparent per-square-arc-second brightness of the background sky in the telescope falls by a factor of that magnification increase squared, or to 1/130.6 That's a difference of over 6 magnitudes apparent brightness in the background sky.
A star that has no contrast with the night sky at low power will suddenly have significant contrast with the night sky at the higher power and be quite visible. The star image is still a point at 320X, so it is not dimmed.
Magnification beyond that level will impart both aberrations due to seeing conditions and optical quality and expand the size of the spurious disc, so at some point, any calculator that suggests that higher magnification yields the visibility of fainter stars will fall down due to the conditions of use.
In practice, in the field, higher magnifications generally yield the visibility of fainter stars.
Under ideal conditions, I was able to reach magnitude 15.6 with an 8", though this was not the theoretical limit per the calculator. The reason had to do with the telescope used. A brand new version of the same scope would have gone deeper. At that point, the coatings (standard Al) were 11 years old.
Now it has generally been said that once the spurious disc of the star becomes visible with an apparent size (often quoted as being around 1X/mm of aperture), further increases of magnification subjects the spurious disc to the same dimming due to the square of the area rule. This would account for the reason small scopes could see a dimming of the star image with higher magnifications--the spurious disc is larger in them and lower magnifications make it visible.
In practice, though, the faintest stars visible have exceedingly small spurious discs and higher magnifications than 1X/mm can be used to see fainter stars. This can be due to both the small sizes of the discs and the increased darkening of the background sky with magnification.
One last note: good seeing (steady air) is essential to see the faintest stars. In my 12.5", for instance, I have seen the central star in M57 constantly with direct vision on a couple nights of superb seeing, but on a typical night with mediocre seeing, I am lucky to catch a glimpse 10% of the time with averted vision in skies of near-identical darkness).
Here is a calculator that takes other factors into account when predicting a magnitude limit for a scope:
http://scopecity.com/limiting-magnitude-calculator.cfm
in case you're curious, here are some other calculators you might find useful:
http://scopecity.com/astronomy-calcu...my+calculators
Don