Volans, mate... no, it's not the atmospheric presure that does the work, "it gets in like liquid into chalk"!
What I believe happens (anyone with a better understanding of chemistry and physics please feel free to correct me) is that air (i.e. the gases in the air) are soluble in water. The solubility increases with decreasing temperature (at least I'm sure this is the case with oxygen - that's how fish "breathe"!).
I would expect the water in the rain gauge to freeze from the outside in. In fact, you can see in the photos the white bulge where the last bit to freeze is. Once the surface is frozen, the gas can't escape so as the ice layer around the outside thickens the gas concentration in the remaining liquid water increases and it becomes supersaturated. When a jagged little bit of ice forms, a gas bubble forms on that, and then the water freezes around it and so on.
The last bit where the ice is white, is that colour for two reasons I think. Lots of tiny micro-bubbles of gas in the ice and fractures in the older ice where the freshly freezing ice is creating force to bend it... just so it can bust the side out of my rain gauge!
Al.