Go Back   IceInSpace > Equipment > Software and Computers

Reply
 
Thread Tools Rate Thread
  #1  
Old 30-01-2017, 11:27 AM
g__day's Avatar
g__day (Matthew)
Tech Guru

g__day is offline
 
Join Date: Dec 2005
Location: Sydney
Posts: 2,807
Pondering a Workstation build for astro imaging processing - peoples experiences!

This year I would like to build a rig to handle astro-photography image processing workloads (which means it could likely also handle games).!

Astro image processing can require stacking of 100s of shots per night - each shot could be 10 - 40MB - and there is a lot of mathematical processing (that no software that I know of off loads to the GPU). So I was pondering getting a 10 Core Intel beast and doing a rig built for Astro first and games second.... when a thought hit me that I first entertained a decade or two ago but never acted on... what about a dual CPU server board? Then a simple google and youtube search showed me several folk have gone down this path and got great results - at a fraction of the price I was expecting - simply because old Xeon chips sell for about $200 for a 8 core 2.4 GHz chip.

Now I only remember high end Tyan motherboards (like the S7070) being dual cpu - but a simple search shows many people are using the Asus Z10PE-d16 https://www.asus.com...ds/Z10PED16_WS/ - which surprisingly supports DDR4 RAM (16 slots - up to a Terabyte of RAM) and has 6 PCI-E Gen3 x16 slots and can support NVidia in 3 way SLI or AMD in 4 way cross fire.

These boards sell in Australia for about $850 - or you might find dated rigs on Gumtree build on these boards (variant unknown) between $300 to $2000.

So a check on youtube and gumtree shows folks selling old Xeon CPUs that are 8 core, 16 thread, meaning this sort of build will give you a 2.4 - 2.6 GHz CPus with 16 cores and 32 threads to handle workloads that like many core solutions. I am yet to see if anyone astronomer has used the programs I like Deep Sky Stacker (freeware) or CCDStack on a multi core monster - but DSS does seem to keep all 4 core on my (or my kids latest i7 quad core machines) maxed out.

So my long winded question. What advice would folk have. I can guess that this rig could play games - but it would act much like a 4 core machine throttled at 2.4 - 2.8 Ghz even if it was running sat a pair of 1080s in SLI. I would have to check everything - OS (win10 x64 is supported) - memory types, Powersupplies, drivers - absolutely everything would work. Again there are folks building these rigs and they scream on 3d rendering that can use all the cores and they are decent at games (paired with a NVidia 980 or Titan card/s normally).

Have folk found - is your image processing CPU, memory or I/O bound?

What set up would be optimal for Image processing?
Reply With Quote
  #2  
Old 30-01-2017, 12:05 PM
pluto's Avatar
pluto (Hugh)
Astro Noob

pluto is offline
 
Join Date: Dec 2011
Location: Sydney
Posts: 1,982
I do a bit of processing on my workstation at work (for 3D VFX/animation/comp) which is a dual ten-core Xeon (E5-2650v3) - so with HT that shows as 40 cores, 64GB ram, win 10.
I use PixInsight*1, DSS, RegiStax, Autostakkert!, Startools, Photoshop, etc.
Obviously it's very fast for software that uses multithreading efficiently

Personally if I was building an astro processing box, with my own money , I would just go with a current I7 over an older dual Xeon box. I think you'll probably get similar performance with a current I7 and a dual 8 core Xeon (just a guess - depends on which Xeon and age of course) for a similar price and in that case I'd always go with the more recent technology*2.

I should also note that I do the majority of my processing on my lappy (2013 XPS15, I7, 16GB) and I've never had anything that I could do on my workstation that I couldn't do on my lappy.


*1 when I first got this workstation I was still running Linux (CentOS) and PixInsight would only use all the cores from one Physical processor - so it would report 20 cores whereas on the same box on Windows it reports 40 cores... weird...

*2 we had an older Xeon that we had to retire recently because it didn't have SSE2 and the renderer we use had an update that required it - took me ages to work out why it wasn't working on just that box!

Last edited by pluto; 30-01-2017 at 12:15 PM.
Reply With Quote
  #3  
Old 30-01-2017, 12:25 PM
sil's Avatar
sil (Steve)
Not even a speck of dust

sil is offline
 
Join Date: Jun 2012
Location: Canberra
Posts: 1,474
I don't think ideal exists and the throttle is the person using the machine.
Putting any software on a Windows machine immediately slows its performance and uninstalling that software won't give you back performance.

Pick a purpose AND STICK TO IT. In your case you want a machine for gaming and a machine for astroimage processing. Therefore you have TWO purposes. How I've done this in the past for the CG industry is you have two OS hard drives, put your games and office crap, one a third drive and your CG apps on a fourth, a fifth drive is ultra high speed type with two partitions (understand the different between a drive and a partition then reread my post) which you dedicate to putting page files and temp/working space for each OS drive. So these days you use SSD drives for each of these and the point isto reduce I/O queues due to the software wanting to read part of a hard drive and the OS needing to read from a different part of the same drive plus the pagefile wanting to update another all at the same just because you pressed "Open file" in a program. So you gain performance plus stability by separating them. The page file drive you should fully format periodically to ensure it performs well and hasn't developed errors from stress. You don't need TB for an OS drive or a Programs drive, 240GB is plenty and you want performance. Add TB drives for temporary storage, but a RAIDED pair Could be used for where you put your source frames and generated registered etc frames too as you can eat up drive space fast, but again reformat periodically.

Anyway I yet to use my dual gfx cards for anything in my atrophotography so use whatever cards you want but not all software will use multiple cores/cpus so I would say go with fast cpu (higher GHz). Age of CPUs will matter, we've had 3.0GHz (not overclocked) Intel CPUs for 15yrs so a current generation 2.4GHZ i7 I bet will give you better performance than a 5yr old 2.4GHz Xeon plus will.

Money no object, I would get dual cpu of whatevers todays best intel cpu is, not necessarily the fastest but close with most cores. Simple way to calculate would be Speed * Cores = processing power and rank them that way choosing the best ranking power.

RAM and GFx cards are easier to upgrade reliably down the track, so for initial outlay you can save there.
Reply With Quote
  #4  
Old 30-01-2017, 02:13 PM
g__day's Avatar
g__day (Matthew)
Tech Guru

g__day is offline
 
Join Date: Dec 2005
Location: Sydney
Posts: 2,807
The cores most people going the dual processor path use tend to be cores happy to run at 2.8GHz - so that is 16 cores and 32 threads (virtualised). Why this build is so attractive is these CPUs can often be sourced for $200 - 8 core Kaby Lake CPUs are faster clock speed but a lot more expensive! The specified dual CPU motherboards are expensive (brand new around $900) but they have 8 PCIE 16 slots and DDR4 support, plus excellent independent and fast memory and I/O lanes and M2 support. You could have up to a terabyte of RAM if that were warranted.

My current rig is an old Conroe Core2 Quad chip at 2.4 Ghz - on an old Gigabyte P35 board with 8GB RAM, 4 SSDs and 8 SATA-2 drives and a NVidia GTX 1070 graphics card. It runs a 30" UHD screen plus two 24" screens (in Portrait) configuration quite well. I don't play games often but it handles them fine.

What is slow is Deep Sky Stacker. Even running all files on SSDs the processing load max's out the CPU. Trying the same work load on my sons latest i7 with 16GB of RAM and a 512GB SSD isn't that much faster - say 30% - so I intuit the software is CPU bound.

A 16 core solution that is economically priced is worth investigating I imagine!
Reply With Quote
  #5  
Old 30-01-2017, 04:20 PM
rmuhlack's Avatar
rmuhlack (Richard)
Professional Nerd

rmuhlack is offline
 
Join Date: Nov 2011
Location: Strathalbyn, SA
Posts: 916
have a look at these Pixinsight benchmarks. Will give you an idea of the performance of different hardware configurations for astro processing

http://pixinsight.com/benchmark/
Reply With Quote
  #6  
Old 31-01-2017, 08:17 AM
g__day's Avatar
g__day (Matthew)
Tech Guru

g__day is offline
 
Join Date: Dec 2005
Location: Sydney
Posts: 2,807
Interesting to see a Xeon E5-2695 at 2.4 GHz leads the field by a long way, followed by another Xeon, followed by a I7 6700K at half the benchmark of the first placed Xeon!
Reply With Quote
  #7  
Old 31-01-2017, 09:02 AM
pluto's Avatar
pluto (Hugh)
Astro Noob

pluto is offline
 
Join Date: Dec 2011
Location: Sydney
Posts: 1,982
It's worth noting with those PI benchmarks that a very significant portion of the final result seems due to the swap speed.
This seems a bit strange to me as I don't think I've ever used anywhere near my full 64GB of ram in PI and so it's never had to use swap - which is slow on my box and pulls my final score down.

Here are the numbers for my box (this was an old one in Win8.1, can't find my CentOS score and I've never bothered doing it again in Win10):
Total performance ...... 5157
CPU performance ........ 8683
Swap performance ....... 1929

So going purely by the total score you'd be better off using an i7 with seriously fast swap when in reality, as my box never seems to run out of ram in PI, my box will be faster.


I do think it's time for another SSD upgrade though!
Reply With Quote
  #8  
Old 31-01-2017, 09:58 AM
g__day's Avatar
g__day (Matthew)
Tech Guru

g__day is offline
 
Join Date: Dec 2005
Location: Sydney
Posts: 2,807
So Hugh - if you created a 32GB RAMdrive and ran your test again from there - what performance numbers would you expect it could deliver? Not even raid 0 960 Pro would be as fast as a large RAM Drive to hold all the I/O.

I would be very interested and thankful if you can try this at some point.

Cheers, Matthew
Reply With Quote
  #9  
Old 31-01-2017, 11:28 AM
pluto's Avatar
pluto (Hugh)
Astro Noob

pluto is offline
 
Join Date: Dec 2011
Location: Sydney
Posts: 1,982
I'll give it a go later, got too much on at the moment.

I did just discover this though which instantly gave about a 40% increase in PI swap speed!
Reply With Quote
  #10  
Old 31-01-2017, 02:39 PM
g__day's Avatar
g__day (Matthew)
Tech Guru

g__day is offline
 
Join Date: Dec 2005
Location: Sydney
Posts: 2,807
Many thanks - and that is a great find!
Reply With Quote
  #11  
Old 01-02-2017, 07:36 AM
sil's Avatar
sil (Steve)
Not even a speck of dust

sil is offline
 
Join Date: Jun 2012
Location: Canberra
Posts: 1,474
keep in mind if you're using the software and hardware properly no machine is ever fast enough. speaking from experience in 3D modelling/rendering since the late '80s.

long tasks in PixInsight, like registration and integration I just set up and let it go while I head to bed or work. Once I have an integrated or drizzle integrated file to process its where my time and attention is needed so registration and integration I will do on sets of images I've taken and leave further processing to later instead of devoting all my time and all my computers time to just one image from capture to final image before moving on, I usually have a bunch of images "on the go" in various states of completeness in my workflow. Eventually final images result, sometimes I rush through a simple processing workflow just to see the potential in my capture. Its all a constant state of experimenting and optimizing each step in the workflow...and wishing my machine was faster (so I usually watch movies on another machine next to me just to keep me occupied).
Reply With Quote
  #12  
Old 04-02-2017, 12:56 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
Interesting thread, in particular that I just run the script and my computer got a total of 172 points in PI benchmark score. It looks like I really do need to buy a newer machine...
Reply With Quote
  #13  
Old 04-02-2017, 04:47 PM
RickS's Avatar
RickS (Rick)
PI cult recruiter

RickS is offline
 
Join Date: Apr 2010
Location: Brisbane
Posts: 10,584
Quote:
Originally Posted by pluto View Post
It's worth noting with those PI benchmarks that a very significant portion of the final result seems due to the swap speed.
This seems a bit strange to me as I don't think I've ever used anywhere near my full 64GB of ram in PI and so it's never had to use swap - which is slow on my box and pulls my final score down.
PI "swap" space is not used for VM paging. It's where PI stores image data, including the backup copies of images used to implement undo/redo and image history.

Cheers,
Rick.
Reply With Quote
  #14  
Old 08-02-2017, 06:51 PM
Slawomir's Avatar
Slawomir (Suavi)
Registered User

Slawomir is offline
 
Join Date: Sep 2014
Location: North Queensland
Posts: 3,240
A quick update.

I decided to get a modest PC (i5 6600, 32GB RAM) to replace my archaic one that was giving PixInsight benchmark scores around 200...


For the new machine...

Performance Indices
Total performance ...... 3224
CPU performance ........ 5568
Swap performance ....... 1178

Not revolutionary results, but these indicate that the new PC should be about 15x faster than my previous, now retired PC. Happy with that.

EDIT: After creating 4 swap storage directories in PI the performance has improved significantly:

Performance Indices
Total performance ...... 4422
CPU performance ........ 5616
Swap performance ....... 2356

Last edited by Slawomir; 09-02-2017 at 06:28 PM.
Reply With Quote
  #15  
Old 09-02-2017, 09:09 AM
rustigsmed's Avatar
rustigsmed (Russell)
Registered User

rustigsmed is offline
 
Join Date: Mar 2012
Location: Mornington Peninsula, Australia
Posts: 3,950
maybe wait for the new amd ryzen 8 core 16 thread which is out literally in a matter of weeks.
Reply With Quote
  #16  
Old 09-02-2017, 12:51 PM
Camelopardalis's Avatar
Camelopardalis (Dunk)
Drifting from the pole

Camelopardalis is offline
 
Join Date: Feb 2013
Location: Brisbane
Posts: 5,425
Quote:
Originally Posted by rustigsmed View Post
maybe wait for the new amd ryzen 8 core 16 thread which is out literally in a matter of weeks.
Yeah, building a PC might just get interesting again with a bit more competition in the market. Intel has had the game to itself for far too long, so kudos to AMD for pulling their socks up and (hopefully) getting something out the door that is competitive.
Reply With Quote
  #17  
Old 17-02-2017, 04:33 PM
lazjen's Avatar
lazjen (Chris)
PI cult member

lazjen is offline
 
Join Date: Dec 2012
Location: Flaxton, Qld
Posts: 2,064
This is what you need: http://www.anandtech.com/show/11121/...c-48t-9000-usd
Reply With Quote
  #18  
Old 18-10-2017, 01:18 AM
g__day's Avatar
g__day (Matthew)
Tech Guru

g__day is offline
 
Join Date: Dec 2005
Location: Sydney
Posts: 2,807
An update to this old thread as I am planning on going down this path shortly. I found two decent stores Zcomputers and TheServerStore in the USA that focus on Workstations - the HP Z820 in particular which seems a very decent beast. I am tossing up two configurations:

HP Z820 Workstation USD $1,824
Processors: 2x Xeon® E5-2695 v2 2.4GHz Twelve Core Processors, (24C/48T) - $760.00
Memory: 64GB (8x 8GB) DDR3 Memory - $160.00
Primary Hard Drive: 400GB SSD - $180.00
Second Hard Drive: No Hard Drive
Third Hard Drive: No Hard Drive
Fourth Hard Drive: No Hard Drive
Video Card: NVIDIA GeForce GTX 680 XLR8 - $160.00
Optical Drive: DVD-RW
Workstation PCIe Expansion: None Installed
Operating System: Windows 10 Professional 64-bit Installed - $45.00

or

HP Z820 Workstation USD $1,604
Processors: 2x Xeon® E5-2690 v2 3.0GHz Ten Core Processors, (20C/40T) - $700.00
Memory: 64GB (8x 8GB) DDR3 Memory - $160.00
Primary Hard Drive: 400GB SSD - $180.00
Second Hard Drive: No Hard Drive
Third Hard Drive: No Hard Drive
Fourth Hard Drive: No Hard Drive
Video Card: AMD Radeon HD 8350
Optical Drive: DVD-RW
Workstation PCIe Expansion: None Installed
Operating System: Windows 10 Professional 64-bit Installed - $45.00


The dual 10 core 3.0 GHZ Xeons would probably run poorly mutli threaded applications (like games) better because of its 25% higher clock speed across all threads. So its a trade off between 20 vs 24 physical cores.


The supplier also confirmed it can run a single NVidia GTX 1070 or 1080 (not SLI because only Workstation cards - think NVidia Quadro cards have that enabled).


One thing I wasn't aware of is dual cpu machines take about two minutes from power on to the normal BIOS boot sequence appearing on the screen. Apparently they go thru a prorietary hardware okay check and nothing much appears on the screen until that is finished!


So from the lasy post in February - those 12 core CPUs at 2.4GHz are available for about USD $760 the pair!
Reply With Quote
  #19  
Old 18-10-2017, 01:29 AM
g__day's Avatar
g__day (Matthew)
Tech Guru

g__day is offline
 
Join Date: Dec 2005
Location: Sydney
Posts: 2,807
Just a PS - I see folk in Pixel Insight benchmarks with insanely high scores on these rigs - some are achieving half their 15K plus scores from their I/O throughput - up their with Intel PCIE 750 SSD throughput - sure enough they are using multiple RAMDRIVES to get 2.4GB/sec throughput. But hey RAM is cheap so a 64GB RAM disk of larger is I guess a totally valid way to speed things along!

http://pixinsight.com/benchmark/benc...16T3UWPO9548R0
Reply With Quote
  #20  
Old 18-10-2017, 09:01 AM
ChrisV's Avatar
ChrisV (Chris)
Registered User

ChrisV is offline
 
Join Date: Aug 2015
Location: Sydney
Posts: 1,737
Do you need the video card?

I wasn't aware that GPU acceleration was implemented in pixinsight yet - and it seemed that they have been talking about it for ages ? If I'm wrong someone let me know.

And might be worth getting an M2 gen 3 SSD. Uber fast and coming down in price. But need appropriate motherboard
Reply With Quote
Reply

Bookmarks

Thread Tools
Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +10. The time is now 08:41 AM.

Powered by vBulletin Version 3.8.7 | Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Advertisement
Testar
Advertisement
Bintel
Advertisement