To isolate single star, you need a mask in focal plane, with appropriate aperture.
Then, you need a lens, to form a more-or-less parallel beam (not wider than your grating).
See here:
http://www.astrosurf.com/buil/us/sta.../design_us.htm
http://www.astrosurf.com/buil/us/spe2/hresol.htm
http://www.astrosurf.com/buil/us/spectro1.htm
Your resolution (not scale!) will be limited with number of grating lines engaged with the beam... apart from other variables (the quality of optics, the atmospheric visibility, etc.)