About 940 NGC images

[rogerg (Roger)]

G'day all,

Some may be aware that for the last year or so my "project" has been to photograph all NGC objects visible to me from my observatory. I have 940+ now but have long struggled with making them available on the internet. The solution I have come up with for now is a PDF document containing the results.

I think the PDF might be very useful to people especially as it grows and the image quality improves (as I replace bad images). It's sequenced on RA then Magnitude so you can look up NGC objects by what's currently visible by RA and then what's brightest and see what the object looks like and how big it is as compared to other objects. I would sure find that useful when doing observing! In fact I looked for a book like that years ago and never found one.

Producing results like this wasn't the aim of the project but is a side effect which has come about. Info on the project here.

The PDF is compressed with low image quality so that it is small enough to place on the web. This significantly degrades the image quality and in some rare cases almost obliterates the NGC object but overall I don't think detracts from the likely uses. I can provide a 50mb higher quality PDF on CD or something. The full PDF on the website now is a relatively large 9mb.

The PDF will be updated as I take more photo's. With the weather and moon the way they've been behaving lately, that'll be a little while (hopefully an update each month).

http://www.rogergroom.com/page/project_ngc_download

If you do look at it, let me know what you think, comments on improvements are appreciated also.

I might do a new PDF in the next few days to get rid of some shots which clearly should have been culled before the PDF was created.

I still hope to finish the project in the next 18 months, that's another 3800 objects. (I can't see all 7900 objects/)

Finally, sorry to those people who have PM'd me in the past asking for images to use for SN searching and the like. It's just been too hard to get them out there. While I am still happy to give the RAW images, it's still a bit hard, other than DVD.

Roger.

[multiweb (Marc)]

I was reading your website about the automation and your work flow on this project. Very interesting read. Kudos to you. I'm impressed.

[rmcpb (Rob)]

Roger,

Just had a quick look at the document. What an amazing amount of work!

Hats off to you.

Cheers

[Phil]

Great stuff well done downloaded it.
Thanks
Phil

[Dennis]

Hi Roger

That sounds terrific - a true labour of love! I just tried both the http links above and got the ubiquitous “The webpage cannot be found” HTTP 404 error.

Cheers

Dennis

[rogerg (Roger)]

Originally Posted by Dennis

“The webpage cannot be found” HTTP 404 error.

Thanks Dennis.... impeccable timing, it was down for maintenance for a couple of minutes Back now

[rogerg (Roger)]

Thanks to Phil, Rob, Marc also for your replies. If anyone ends up using it for real, let me know how it can be improved

[leinad (Dan)]

Hi Roger,

Fantastic work! I will definitely be interested in a printed copy when available.

[Garyh]

I agree, mammoth effort there Roger! The compression has reeked havoc with some of the images in the 9 meg version.
I will be very interested in the big version on cd when you are finished all the NGC entries!

[jase (Jason)]

Roger,
A miraculous effort indeed. I find it inspiring when one embarks on such challenges like this. Really is a lot of work and takes considerable dedication. There are a couple of things I'm not certain of. Within the PDF documents, many images are repeated NGC1934 through to NCG2117. While I don't have any problem with the repeated images as I gathered that there are multiple targets in the single frame, but I really don't know what target its referring to. Perhaps you need to place cross-hairs on those individual targets in each frame to give the viewer and indication of what NGC object is being referenced. I'm also not certain if your choice of format being PDF is ideal. Clearly, as you progress with acquiring more and more images, the ugliness of data management will emerge. You may wish to consider piping this into a mySQL database with a simple web front end. The SQL database will certainly give you the structure and flexibility you desire. You could then give the viewer the opportunity to search for different criteria, i.e. show me all NGC objects in the constellation Hydra or show me objects that are larger the 1.2 x 1.0 arcsecs. It is possible to embed the binary images into the SQL database itself to make the solution clean. It could get rather funky depending how far you want to take it. I look forward to seeing more of your own digital sky survey...wow...the possibilities...you could also start on the IC objects in parallel. This is where a SQL database could become handy i.e. when a single target has multiple designations. Food for thought anyway. Well done and keep it going.

[rogerg (Roger)]

Originally Posted by jase

Roger,
A miraculous effort indeed. I find it inspiring when one embarks on such challenges like this. Really is a lot of work and takes considerable dedication.

That's good to know, I hope it is inspring to some people at least.

Originally Posted by jase

There are a couple of things I'm not certain of. Within the PDF documents, many images are repeated NGC1934 through to NCG2117.

Thank you! Well spotted. It seems conversion from FIT to JPG of some images has failed or not been done, and those are not being included in the PDF, and instead of showing nothing, it is showing the most recently available JPG (in sequence) until it finds another JPG!

I'm in the process now of re-generating all the JPG's and hopefully ironing out that problem.

There should be 'near repeats' where images look similr but not exactly the same, where multiple objects are in the one FOV, but what you found is plain wrong.

Originally Posted by jase

Clearly, as you progress with acquiring more and more images, the ugliness of data management will emerge. You may wish to consider piping this into a mySQL database with a simple web front end.

Yes, ugliness is the right word

My website is actually data driven already, it uses DB2 & Java. The stuff I work with in my day job. There's two reasons I haven't got these images up there individually like you propose: 1) time to code the pages 2) the amount of data traffic required. To address them in more detail:

1) The current PDF is actually a MS Access report which is then saved as PDF. It took me all of about a few (2-4) hour to create the query, word document for the front page, and then report structure, and get it all working (or so I thought!)

To build the web interface is more work than that so I just haven't got around to it. need more time in life I hope to, I agree it would be much better. I suppose one reason I haven't given it priority is reason 2 below

2) My website is kindly hosted free by the company I work for, and there's only so much traffic I can take before I start to pay AU prices for it (as opposed to heap US priced hosting). Just having this 9mb PDf on there I estimate will use about 6GB traffic for the month, based on my average webstats. I think to put all the images I have already on the web would be about 300mb. So the search engine's alone will chew up a few GB of traffic. So I need to deal with that somehow. I will probably buy a cheap US hosted domain, put the pictures there, and just link to them from my website's database. I've almost done that a few times, but haven't got around to it. I tell ya, the PDF is a lot easier to "just get done" ... even if the coding and domain setup would only take "some hours".

It would be neat to have a searchable index up there like you say, find all of a constellation etc, that would be cool. I really look forward to having that up and going

Anyhow, thank you very much for your great input Jase

[Dennis]

Hi Roger

Phew! I’m exhausted just flipping the pages and looking at the glittering array of jewels you have presented in this mammoth project – you must be made of strong stuff to have pulled this much off so far!

I’m not sure if it is my monitor, but some of the images look heavily posterised, as if the tones have been stretched too severely?

Fantastic work indeed!

Cheers

Dennis

[rogerg (Roger)]

OK, PDF has been updated with ones that hopefully don't contain repeat images

Originally Posted by Dennis

I’m not sure if it is my monitor, but some of the images look heavily posterised, as if the tones have been stretched too severely?

Yeap, there's a few reasons there's such variations in image quality. Most of those are because some of the exposures were obscured by cloud to some degree. It goes a bit like this:
- 100% cloud: processing fails and I remove the images all together
- Partial cloud: if stacking succeds then I get what I can out of the images by stretching the levels, resulting in images like you have noticed.

It's those partial ones which I will gradually replace but for now I've left them in because there's so many others to get initially, let alone re-do

Another reason the images sometimes end up like that, or "wacky" in some way with respect to brightness is if due to stacking there is some black left around the border, the levels & curves actions i run don't get the black point right. Some I notice when running the actions and tweak or crop manually, others I just let slip through for now hoping to get back to them on a rainy day.

Thanks,
Roger.

[Dennis]

Hi Roger

I think your monumental work will become a classic in the annals of amateur astronomy – certainly a prodigious undertaking requiring bucket loads of stamina, and happily, it looks like that you are well on your way to completion!

Cheers

Dennis

[strongmanmike (Michael)]

Excellent stuff Roger!

A higher res verison of this set when finished would be a great publication on DVD or even in print, although I'd imagine a book would have to be pretty big? A collection of all NGC objects visible from mid southern latitudes, including a reasonable quality picture of each plus basic object information would be very useful.

You could group it into constellations even?

Nice work so far

Mike

[rogerg (Roger)]

Originally Posted by Dennis

...I think your monumental work will become a classic in the annals of amateur astronomy...

Well, I don't expect that, but I can see it might be useful and hope it will be

Originally Posted by Dennis

... – certainly a prodigious undertaking requiring bucket loads of stamina, and happily, it looks like that you are well on your way to completion!

Yes, quite a task! but I've always thought I'd get through them without a problem, so will continue with the positive attitude and hopefully get there soon Thanks for your reply Dennis

Originally Posted by strongmanmike

A higher res verison of this set when finished would be a great publication on DVD or even in print, although I'd imagine a book would have to be pretty big?

Yes, a bit of a problem for publishing in a book, not just web. I have thought of sticking to the idea of volumes, split by RA, Constellation or whatever ends up useful. At the moment the full PDF is 244 pages, double sided would be 122 pages. That's OK. But 5 times that isn't so it would have to be split or re-arrnged some how. I look forward to the challenge though, I suspect it would be most useful in a printed format.

Originally Posted by strongmanmike

You could group it into constellations even?

I have been wondering what grouping/sequencing would be best. So any suggestions on how people would find it useful are appreciated. By constellation is a good one, probably on of the most suited to visual observers?

Thanks,
Roger.

[Ric]

A fantastic piece of work Roger, I am in awe of what you have achieved with more to come.

Top stuff indeed.

[rogerg (Roger)]

G'day all,

One final note on this...

I won't be posting a message here whenever I update the NGC document, I'm sure everyone would get sick of me going on about it. Updating the document is something ongoing which will happen now and then. For example within the next week I will do a new copy which has more information about the objects, more images, some images re-processed, and some bad images removed.

So, if you are interested in keeping up to date with the new documents as I publish them, please sign up to my list of people which I will notify. I have put a subscribe feature at the bottom of the page: http://www.rogergroom.com/page/project_ngc_download

If you want to be updated, just put in your email & name, subscribe, confirm your subscription (you will receive a confirmation email you need to action) and whenever I put a new significant upgrade of the document up on my site I'll send out a message to subscribed people. You can 'unsubscribe' at any time (and I won't be offended when you do!).

Thanks,
Roger.

[Lester]

Hi Roger, there are hours/months and years of dedication evident there, downloaded it for later perusal.

Thanks.

[Jen]

wow now that is a very dedicated astronomer there
nice work Roger. i have always tried to find a web site to show me what i can actually see and what i cant see with my 6inch scope (every time i think i get an answer its quite vague)
keep up the great work cheers
you got some nice pics there by the way