Desktop computer advice needed

[gregbradley]

I am wanting to upgrade my computer for image processing.

I am currently using a HP Pavilion Laptop i7 with 8gm RAM and 1 TB HD.

Its way too slow and too little RAM.

My cameras both do around 32mb for a single image. So if I take multiple hours of images I may have as many as 30 x 32mb image files to process at one time.

I use CCDstack to do the initial processing and its arduous and slow.
It usually will limit me to about 9 x 32mb images before it starts to reject them. Doing some CCDstack operations can take quite a while.

This makes image processing a bit of a drain.

I am thinking of:

1. A desktop not a laptop.
2. Not a HP. This laptop is starting to fail and its less than 2 years old.
3. Latest 6th gen i7 4ghz processor. I have read some refer to Skylake and it seems to be the latest.
4. 32gb DDR4 RAW minimum perhaps 64GB. Here is where I am not sure. Some computers mention quad channel for DDR4 memory allowing 8 slots and others mention dual channel allowing 4 DDR4 slots. I assume as RAM is critical that quad channel is the way to go
5. 3 TB hard disk.
6. 500gb SSD.
7. Upgraded cooling and power supplies.
8. I am not sure how important the graphics card is. Geforce seem to be dominant. Should I go for an upgrade here or is it not important? I won't be using the computer for gaming or if I do its very minor usage. However doing image processing does require accurate colour so perhaps a good card makes sense?
9. I also read one supplier talking about a hyper speed tech where programs open and close instantly. That sounds good. Is this generally available?

Anything else I should consider?

Greg.

[alocky (Andrew lockwood)]

Hi Greg - it's not always hardware related, but I'd definitely go for a desktop, two monitors, and go for the latest multicore server setup (these become obsolete overnight so I won't suggest whatever weird name and number they've got today), make sure you've got as much ram as you can shove in it, at least two ssd drives and get a decent graphics card, because a lot of software like photoshop can actually use the gpu instead.
Then use pixinsight for your calibration and stacking, and use the ssd drives for throughput.
I've been processing 100s of raw files at a time from my full frame Nikons for 5 years on my current setup and it's still quick.
You might want to consider an external NAS device for long term archiving.
Anyway - not a laptop. And not really a gaming machine either.
I got mine built to the specs we use in our industry for managing and visualising truly enormous geophysical datasets, but went through a local small company who were so excited they wanted an extra few days to 'benchmark' with their favourite games...
Cheers
Andrew

[RickS (Rick)]

Greg,

1. A desktop system is a good choice for performance
2. Don't have a view on particular brands. Have been burnt by most of them at some time
3. Skylake is the current latest. Early next year Kaby Lake desktop CPUs will be available but will only be an incremental improvement
4. Lots of memory is good. I have 24GB in my 6 year old desktop system and it is perfectly adequate for very big integrations in PI. Sounds like CCDStack isn't as efficient. Quad channel DDR4 will give max performance with 4 slots filled (one channel each.) 8 slots is good if you decide you need more RAM later.
5. Lots of HDD makes sense
6. SSD for your boot/Windows disk will certainly improve boot time and application start times.
7. Some power & cooling headroom is useful
8. You don't need a fancy gaming card. Have a look at the Nvidia Quadro range of desktop workstation cards (you don't need a high end one.)
9. No idea what this is. SSD makes a big difference to program start times. For processing you won't care that much since you'll be running the same program for hours, or days

Cheers,
Rick.

[RickS (Rick)]

Originally Posted by alocky

get a decent graphics card, because a lot of software like photoshop can actually use the gpu instead.

That's certainly true if you're doing a lot of PS. I live in PI the whole time and it doesn't use the GPU. Neither does CCDStack.

As Andrew says, PI is very efficient at calibration and stacking and uses multiple cores/threads to great advantage.

[pluto (Hugh)]

Even if you aren't gaming I would still go for a decent entry level gaming card, like a GTX970 or GTX1060. The reason is that most Adobe software now relies on GPU for many functions and this will continue to expand with more and more functions being shifted to the GPU.

Definitely get an SSD, and if your case will fit it you can get some extra cheap fast storage by buying 3 or 4 1-2tb drives and raiding them. I do this at work as a kind of scratch drive and it's great. My OS and applications still run from an SSD though.

[pluto (Hugh)]

Originally Posted by RickS

8. You don't need a fancy gaming card. Have a look at the Nvidia Quadro range of desktop workstation cards (you don't need a high end one.)

Not sure what you mean by this but a Quadro is way more overkill than a gaming card, and much less bang-for-buck for the kind of stuff Greg will be doing.
I've used many Quadros in the past and have never seen the value of them over a good gaming card - and I work in CG using the kind of tools those cards are designed for.

[RickS (Rick)]

Originally Posted by pluto

Not sure what you mean by this but a Quadro is way more overkill than a gaming card, and much less bang-for-buck for the kind of stuff Greg will be doing.
I've used many Quadros in the past and have never seen the value of them over a good gaming card - and I work in CG using the kind of tools those cards are designed for.

If his bottleneck is doing large stacking runs in CCDStack then a gaming card won't help at all. It will only make sense if he's doing hefty processing in Photoshop. An entry level, but good quality, workstation graphics card is a reasonable alternative to consider. Astro image processing is not very graphics intensive.

[pluto (Hugh)]

Originally Posted by RickS

If his bottleneck is doing large stacking runs in CCDStack then a gaming card won't help at all. It will only make sense if he's doing hefty processing in Photoshop. An entry level, but good quality, workstation graphics card is a reasonable alternative to consider. Astro image processing is not very graphics intensive.

I agree that a gaming card won't help for stacking, neither will a Quadro. However, I think Greg does do all his other processing in PS...?
It's by no means the most important component in his build but I wouldn't build a box, for processing in PS, without at least an entry level video card.

I guess you're right that a workstation card could be considered, but personally I wouldn't recommend a Quadro over a GTX for use with Adobe software as, for the money, you're not going to get any performance benefit. Plus, in my experience, because you're using different drivers you can have problems running certain software which would run fine on a GTX. That's just my experience though, and it's why I don't buy Quadros anymore.

[gregbradley]

Thanks for the advice guys. Photoshop is not really slow even on my laptop. It takes a little while to boot up but I couldn't say its slow.
Its CCDstack that is slow. Callibration and forming master LRGB files takes the time.

To do a hot pixel data reject on 10 32mb files in CCDstack takes a few minutes.

With 8gb RAM I can only open 10 files.

I'll have to apply what I learnt at your PI workshop and get the calibration workflow going in PI. It sounded like once its setup its quite fast and you can reuse all the setup.

Greg.

[pluto (Hugh)]

I recently stacked about 30 raws from a 5DmkIII in PI on my little MBA11 with only 4gb of RAM (running Win10) and it worked fine. Took a couple of hours though...

[ChrisV (Chris)]

How do you feel about DIY ? I got sick of prebuilt nonsense and have done a few home/work desktop builds getting components from the likes of MSY. Can do it way cheaper and get exactly what you want. Whack the operating system etc on a smallish 250-500GB SSD and get a few mega-hard drives. And don't go crazy on a graphics card.

The most expensive part is usually WINDOWS ! And the most complicated = installing it.

[gregbradley]

Originally Posted by pluto

I recently stacked about 30 raws from a 5DmkIII in PI on my little MBA11 with only 4gb of RAM (running Win10) and it worked fine. Took a couple of hours though...

CCDStack has never really handled memory very well and has been a very memory intensive program. Hopefully 64gb ddr4 RAM will suffice.

I would rather someone with more current knowledge of compatible bits and pieces to put something together rather than DIY. But its a good idea.

Greg.

[Somnium (Aidan)]

Originally Posted by gregbradley

CCDStack has never really handled memory very well and has been a very memory intensive program. Hopefully 64gb ddr4 RAM will suffice.

I would rather someone with more current knowledge of compatible bits and pieces to put something together rather than DIY. But its a good idea.

Greg.

i run ccd stack with 16 gig of ddr4 ram and it seems to run fine. granted my image size is probably not nearly as bit as yours. i am running an 8300 chip.

[Windston (Dan)]

Originally Posted by ChrisV

How do you feel about DIY ? I got sick of prebuilt nonsense and have done a few home/work desktop builds getting components from the likes of MSY. Can do it way cheaper and get exactly what you want. Whack the operating system etc on a smallish 250-500GB SSD and get a few mega-hard drives. And don't go crazy on a graphics card.

The most expensive part is usually WINDOWS ! And the most complicated = installing it.

I also really, REALLY recommended ditching prebuilt pc's. I am 16 and have build over 15 pc's in my years and haven't had a problem with one. As long as you dont throw the parts, you will be fine. It is a fool proof system, if the plug doesnt fit, it isnt meant to go there! My first one took me a good 4 hours, now I can do it in under 45.

You can buy new or used (Overclockers Australia is good but you have to be a member for 3 months, but they do have really really cheap used pc parts classifieds, never had a problem there either), or new, from Umart, MSY, Mwave etc.

You could do a NAS if you wanted and feel that you would benefit from it, I just installed one at my house, moving the file server from my pc to a dedicated server pc in the cupboard with all of the file storage for the family on there, it is really great and quick especially if you have internet. I can come in from shooting and transfer all files within one minute, or even over wifi when I am near enough to the house. Otherwise just make a shared folder over the network for you desktop so that you can transfer, otherwise just do it oldschool with USB hard drives. From there I use my custom built desktop to process.

I have an i7 4770k which is a great chip, really quick, but only have 8gb of ram at the moment which is a killer. Star Tools uses it all and often crashes my pc! Over 16gb is a must, over 32gb is a tad overkill.

Anyway it is much much cheaper, customization and easily up-gradable compared to prebuilt if you are willing to build it. Having a friend on you first build will often make it easier, but I did it at 12 after a few youtube videos without and problems so I am sure that you will be fine!

Cheers
Dan

[gary]

Hi Greg,

I concur with those who suggested you should build the machine yourself.

It is not difficult and can be done in an afternoon.

Suggest you start by looking for the best quality motherboard with the
specifications you require. For example, most at the moment are built
around the Intel Z170 Chipset.

The Gigabyte Ultra Durable range might be a good starting point to look -
http://www.gigabyte.com/mb/100-ud/Model
http://www.gigabyte.com/mb/100-ud/Durable-Safe

Then pick a CPU with the right price/performance point to match the motherboard socket.
At any one point in time, there will be a sweet spot which you can
ascertain by looking at the PC suppliers price lists.

32GB of DRAM.
SSD, hard drives, etc.
Consider two identical hard drives mirrored in a RAID 1 array for redundant
reliability.

Then pick a good case. High quality with slide-out disk storage bays and
plenty of quiet fans and removable filters.

Add a power supply.

You can always add a graphics card with GPUs later if need be.

The motherboards all come with manuals that have diagrams of the various connectors.
Once the motherboard is screwed into the case using the standoffs, the cpu goes
in easy enough (it comes with its own install instructions) and the DRAM just snaps
into the sockets. Which cable plugs into where is the only task that requires any
concentration and most of the time the plug types and sockets are unique enough that
it is difficult to get any swapped. The octopus of cables from the power supply has a multitude
plugs but it will become self evident which plug into the motherboard and which power the SATA drives.

In the end you will end up with a system with the best components (you chose them) and the best bang
for your buck and one that you know was assembled with care.

[lazjen (Chris)]

And once you start building your own machine, ditch Windows and build the OS using Linux.

Building your own machine is the way to go. It's not that difficult and you get far better value for your dollar - and exactly what you want. I've been building my desktops for years now and each system has been more powerful and quieter than the previous one.

[sil (Steve)]

I don't think the OP will be able to build a PC, if they could the original questions should be part of their knowledge.

I can highly recommend getting a machine built from Scorptek, specced to what you need: fast cpu, more ram, storage, gfx card etc. the build damn good machines and have a variety of specced machines on their website for all budget needs.

I used to build custom machines to compete in the CG industry when SGI started to get non cost effective.
I recommend Intel CPU over AMD as software is typically more stable and optimised to make full use of cpu features than amd ones.
Also I prefer Radeon gfx cards, over Nvidia for driver stability issues. However while its true some software can use GPUs for processing, they only use either nvidia OR radeon as each has its own methods of GPU processing which are incompatible with each other. So check the software you use to see if it uses nvidia or radeon gpus and decide based on what your actual software uses. a good monitor and colour calibration is important too.

Just speak to Scorptek about a machine for your needs, theres some real inappropriate advice in this thread.

[rally]

Greg,

Is a 27" iMac out of the question ?
Or a Mac server with multiple 4k screen support !

The iMac has a colour calibrated screen to die for, multiple big screen support
OSX is Unix/Linix based and so handles multicores and multithreading very well
PI is supported natively as is Photoshop etc

Anything that is Win specific you choose to keep can be run in a virtual machine and you can mix and match Win and Mac programs using Parallels.
In fact you can have multiple Win OS's running different programs simultaneously and creating a new virtual machine for say testing new software that you may or may not want to keep can be as simple as a single file copy.

Best of all worlds really

[pluto (Hugh)]

The iMacs do have nice screens, and they are very shiny, but only i5 processor? and the price!!!

I didn't think Apple made servers anymore...? We have an old Mac Pro shoved next to the rack as a file server because they stopped making them...

[rally]

Not sure where you are looking Hugh?

My first iMac 6 years ago was an i7 and they are still available in various i5 and i7 configurations today.

You build it online and get it how you want from Apple - you can get an i7 in a number of processor options.

I just fiddled online and configured a 4.0GHz quad-core Intel Core i7, Turbo Boost up to 4.2GHz with 32gb RAM, AMD Radeon R9 M395X with 4GB, 1TB flash drive
The big flash drive made it a bit exy, but a standard 3TB hard drive was OK.

The "servers" (traditional box without screen) are called Mac Pro's
The standard processor in them is a 3.5GHz 6-Core Intel Xeon E5 processor but if you have plenty of $$ you can get a 12 core Xeon
They come standard with Dual AMD FirePro D500 with 3GB VRAM graphic cards.
Expensive as a personal PC but powerful and definitely still available !

You dont get the variety of options if you buy the bits and build a WinPC yourself, but you also dont get the potential problems either.

Originally Posted by pluto

The iMacs do have nice screens, and they are very shiny, but only i5 processor? and the price!!!

I didn't think Apple made servers anymore...? We have an old Mac Pro shoved next to the rack as a file server because they stopped making them...