Unfortunately, there's no single infinity point on a camera lens. It tends to vary with temperature, and most lenses will in fact focus "past" infinity.
Modern SLR lenses aren't designed for very precise manual focusing, so it can be tricky. On wide aperture telephoto lenses, you can often just use autofocus on a bright star like the Pointers to the Southern Cross. Wide angle is tricky - my Canon 20D can't even autofocus on Alpha Centauri with a 24 mm f/1.4 lens.
If you have a laptop handy, you can use software such as
Backyard EOS (only for Canon EOS cameras) to help with focusing - even without LiveView.