If you look into that benchmark - the dual Xeons are faster than the M4 - its the NVME drives that owner has paired them with that is retarding his score.
But regardless it is rather impressive that a M4 can keep up with a ten year old dual Xeon - likely because PI is not compiled to leverage the Xeon CPUs specific matrix manipulation capabilities
A further update from STAstro - he has found two interesting insights:
1. WBPP is faster under Windows then Ubuntu - total surprise there
2. Under Windows v1.8.9-1 is materially slower in WBPP than v1.8.9-3 - most of this accrues to Local Normailsation being about 40% faster in the latest version of PI running WBPP v2.7.8
That linux benchmark is my PC, a used Dell Precision T7910 workstation that I bought off eBay for $1200 in Jan 2024. What does a Macbook Pro M4 Max (the machine used in the other comparison test liked above) set you back...?
If you look into that benchmark - the dual Xeons are faster than the M4 - its the NVME drives that owner has paired them with that is retarding his score.
curious about your comment re NVME swap drives compromising performance. The swap storage directories (16 in all) are on a fast NVME SSD that is separate to both the OS drive and the PI working directory.
Other than using a ramdisk for PI swap, how else would you suggest setting up the swap directories to improve performance?
That linux benchmark is my PC, a used Dell Precision T7910 workstation that I bought off eBay for $1200 in Jan 2024. What does a Macbook Pro M4 Max (the machine used in the other comparison test liked above) set you back...?
nice job with the setup / benchmark.
i think you're looking at about $5k for that particular model.
I am not sure how he has set up his NVME drives - but if it used an add on Gen 4 PCIE card and set it up in RAID 0 that score would double and - if he is on Gen 5 then its performance would be double again! RAMDisks sometimes don't score as high as the very best NVME drives in RAID 0 - not sure why that is the case though...
BTW PixInsight v1.9 dropped yesterday - for a brief moment in time - I had the two fastest Windows based systems scores in the world that's right - my setup were the only two Windows enteries. V1.9 for Windows scores about 2,000 points lower than V1.8.9-3 - I haven't tried WBPP on it yet though!
Well a very interesting latest release by PixInsight v1.9.3 that delivers 20% - 30% performance improvements (real world and benchmarks) by the thoughtful practice of once-off performance monitoring of worker thread numbers and sub threads spawned versus image sizes. This is basically individual machine processor optimisation of thread counts for the most frequent and heavily used tasks in PI.
So once this update is installed and all patches download - users are encouraged to run processor optimisation to create their own optimisation profile. On 8 core machines this takes about 10 - 12 minutes. On my 36 core Xeons it took about 35 minutes. One user with a 48 core / 96 thread platinum Xeon took 70 minutes running this diagnostic.
So the end results - my best run with PI version 1.9.2 scored 17,523 - my best run with version 1.9.3 is 20,207 - a very nice improvement for free!
Interesting to see one the third fast rig on the table - a Platnium Xeon 8558P - its scores top out as 21,473 for Windows and 55,091 on Linux - with exactly the same gear.
Makes me ponder would my scores would also increase by 2.5x (giving me the 4th fast rig benchmarked - vs my current 28th ranking!
Lastly a user with Xeon Gold 6536N processor has not only mapped Windows vs Linux scores - he has done this with 8, 16, 24, 32, 48, 56 and 64 threads - and the surprising conclusion is this (and most recent versions of) PI scale very poorly with increased processor threads in Windows - but on the exact same gear performance scales linearly in Linux - pointing to a poor Windows coding issue!
Wow, I want your computer, I must check Grey's auctions.
I tried a couple of old servers but they weren't that good. I may have been running Windows server edition on them, I don't remember, one was a 4 processor unit but old processors and they sucked energy like sand sucks water.
So the end results - my best run with PI version 1.9.2 scored 17,523 - my best run with version 1.9.3 is 20,207 - a very nice improvement for free!
Interesting to see one the third fast rig on the table - a Platnium Xeon 8558P - its scores top out as 21,473 for Windows and 55,091 on Linux - with exactly the same gear.
Makes me ponder would my scores would also increase by 2.5x (giving me the 4th fast rig benchmarked - vs my current 28th ranking!
Lastly a user with Xeon Gold 6536N processor has not only mapped Windows vs Linux scores - he has done this with 8, 16, 24, 32, 48, 56 and 64 threads - and the surprising conclusion is this (and most recent versions of) PI scale very poorly with increased processor threads in Windows - but on the exact same gear performance scales linearly in Linux - pointing to a poor Windows coding issue!
so plenty more free performance to be had with an OS upgrade! would be interesting to see your results if you decide to add a linux partition.
I may try the Linux path in the future - if the PI team can't figure out what in the thread management is bottlenecking PI. The very fast Linux running under Windows runs far faster then running directly under Windows should give them some hint of where to look for what is going askew!
Back from a long trip in Europe and now attempting my latest software aided journey - exploring different O/S for Pix Insight image processing.
So an interesting challenge I have comming up. I want to try dual booting my system across WIndows 10 Pro and Windows 11 Pro operatings systems - and may eventually add a third O/S on a thrid SSD - likely Ubuntu (Linux).
I intend to set each O/S up on its own specific SATA SSD - so imagine 3 SSDs each with its own O/S.
I need to share one or in future two large HDDs across each O/S (both formated in NTFS) - this should be simple.
The challenge is I have a scratch file drive that will be heavily used - it is currently a Windows 10 Pro created dynamic disk on a PCIE x16 card comprising of four seperate 2TB NVMe M2 Crucial drives - created by the Windows 1-0 Pro O/S as a dynamic RAID 0 array.
Now my challenge is this RAID 0 array was created by the Windows 10 Pro O/S instance - not in the BIOS by the HP Z640 workstations motherboard - this maybe something it can do - but I haven't gone down this path yet - seeing if the HP Z640 can create a RAID array on a PCIE x16 card - thus possibly making it visibile to any O/S I choose to launch.
When I try and dual boot Windows 11 Pro next - I am wondering 1) where I should put the boot loader - on the intial C: drive that holds Windows 10 Pro - letting it see the first original SSD as the Windows 10 Pro drive and making the new second drive a Windows 11 Pro drive (that when it launches it will likely assign letter c: to that second drive and 2) will the RAID 0 array (my R: drive in WIndows 10 Pro) be usable to Windows 11 Pro - or will it just see 4 un-initialised disks - becuase it didn't create the dynamic drive?
Then later I will create a Ubuntu drive - so I face the problems - where do I store and what boot loader do I choose to launch 3 O/S and will the Ubuntu O/S be able to see a WIndows created RAID 0 array.
My fear is if one O/S creates a RAID array - the other O/S may not be able to understand it - forcing hem to nuke it and recreate it from scratch every time they are launched and I read somewhere when a HP Z640 BIOS tries to create a RAID array it sometimes forces all drives attached to try and become RAID arrays - so I have to look into that more.
The last (as if they aren't enough) challenge I will face is how a dual Xeon workstation launches an O/S - you don't generally see anything for 60 seconds until the Windows boot screen appears - so how or even will I see a multi boot loader to choose which O/S I desire? Unlike a normal PC - where the BIOS power on is quick and rather visible - a dual CPU workstation boot seems to be hidden until the OS splash screen appears.
I am off on a learning curve - any help that can be given will be greatly appreciated!
Many thanks,
Matthew
PS
If it all becomes too hard - I realise I can simply delete my 4 x 2TB RAID 0 array (R and create 4 x 2TB simple drives (R: S: T: U and tell my software (PixInsight) that requires fast I/0 to use 4 specific drives (with multiple shares for i/o speed) for holding astro images and the scratch files required to process them in large batches (stacking of 1,000s of images).
I'm running a six year old AMD Threadripper 3970X under Windoze10 which cost me some
coin in the day but still performs quite well with a PI benchmark in the 28k region.
That said , the latest Threadrippers are quite remarkable. I'd likely avoid
Intel and go there again if I were to update my current machine.
So an interesting challenge I have comming up. I want to try dual booting my system across WIndows 10 Pro and Windows 11 Pro operatings systems
My son does this between Windows 8.x, Windows 10 and Windows 11 (all pro) and the easiest way is he boots from a flash drive with the particular OS on it set up as bootable (ISO from memory, I forget more than I remember now). Multi boot hard drives can be problematic at times.
Quote:
Originally Posted by Peter Ward
I'm running a six year old AMD Threadripper 3970X under Windoze10 which cost me some
coin in the day but still performs quite well with a PI benchmark in the 28k region.
That said , the latest Threadrippers are quite remarkable. I'd likely avoid
Intel and go there again if I were to update my current machine.
AMD Ryzen Threadripper Pro 7995WX Desktop Processor $17299
That's just the processor.
96 core 192 threads but very little software to take full advantage in astronomy circles (possibly in paid software I can't afford to play with, I haven't researched that).
I must admit I like AMD processors, they've been better bang for the buck for a long time and were the first to 500Mhz, then 1Ghz.
I spent years studying IT and building and repairing PCs at a component level on boards back in the XT and 286/386/486 days when everything wasn't plug and play and I hear so many people say "don't even get me started on AMD builds", hinting there's always a problem. I've never come across any problems with AMD builds. The bigger issue is people not doing the research, RAM for Intel is different from RAM for AMD (didn't used to be but is now) and if matching of hardware is researched and done correctly they are an amazing machine.
...........
I must admit I like AMD processors, they've been better bang for the buck for a long time .............. they are an amazing machine.
Indeed I can echo those sentiments.
I ran Intel for decades and while I did have a failure with an early AMD CPU the current crop has proved to be bulletproof.
BTW just tweaked the BIOS and am now scoring a 31k Pixinsight benchmark
Not bad for a 6 year old machine.
I'm not in a position to buy new equipment, pensions don't make good for people with too many hobbies (not complaining but I went from a 6 figure salary in 91 down the $12K annually, slight change in life style was necessary) but I mainly run used stuff bought at auction, cheap. I don't need the latest and greatest for what I play with but I keep my son in reasonably current gear but never the latest and greatest, he has a nice 2 in 1 Lenovo for most of his programming needs and a games suitable desktop, bought used (MB and graphics, new RAM) near new goods at a bargain price from a local notice board.
Were I building a new system it would be only AMD and when I still occasionally throw systems together for friends I always insist AMD is the way to go.
I am just in the process of making a current recovery USB for my Win 10 Pro machine - before I try installing Windows 11 Pro on new drive.
Hey Leo - I saw an amazing Youtube video of a guy taking a Gigagbyte MS73-HB1 motherboard (which is dual Xeon platinium board with multiple Gen 5 PCIE lanes (Gen 5 is 4x the speed of Gen 3) then bought two 56 core Xeon Platinium ES chips (engineering samples) to create a 112 core / 224 thread CPU for under $1,600. I would be impressed to see what it can with PI.
I am wondering if sharing a software RAID array on PCIE x16 is difficult across Operating Systems - should I lean towards try creating a hardware RAID array - using the PCIE ASUS Hyperdrive card and my PCs BIOS - or just revert the 4 NVMe drives on the Hyperdrive card to individual disks and just give PI 4 x 2TB NVMe drives to play with.
Sorry Matthew, I replied last night then bumped the back button on my mouse (I hate this mouse) and lost it all and I was tired.
Hardware RAID is not dependant on the software as is software RAID and would be your best bet if running multiple OS's
I should imagine sharing it across operating systems could present some problems but I haven't played with it. I long got out of any networking, that's my sons forte now. I will see what I can find either in my books or online however.
I have a Lenovo dual CPU mother board my son was rewriting the BIOS code for because of a corrupt code in the chip but he's sort of stopped most things with his depression, still a work in progress. Twin CPU's however far from cheap for anything decent (and restricted to MB) but I doubt in Aus we'd be getting twin 56 core processors for that price, maybe US?
Well my dual boot is working fine, all device drivers for Windows 11 Pro installed and according to device manager - no issues. All Windows 11 Pro updates applied fine.
A pleasant surprise - the RAID 0 NVMe PCIE based drive created as a dynamic disk under WIndows 10 Pro appears and works fine under Windows 11 Pro - I wasn't suspecting that - total win! CrystalDiskMark shows about 8,000MB/sec read write speeds - so all good.
But under my Astrophotography suite (PixInsight) - set up the same way under Win 10 and 11 - Win 10 is almost twice as fast (a lot higher CPU utilisation under the benchmarks) and the swap file - my RAID arrays scratch directory shows 3 times the throughput under Windows 10 and Windows 11 - which is frankly wierd - off to research this!
I checked each set up has the same swap file point (the RAID array) the same number of shares and the same number of read and write threads. I also ensured under Windows 11 Pro that xisf file types and PixInsight.exe and the QT web engine exe were all added to Win 11 Virus and Security exclusions.
So I am trying to see why Win 11 is processing CPU at 50% of the rate of Win 10 (all 36 cores are utilised - just to a lower extent) and why I/O throughput is a third that of Win 10.
in terms of multi booting, perhaps it is worth looking into "rEFInd"boot manager to see if there is an easy solution there..
i can't really provide much guidance on your raid situation other than i recall hearing tech people say hardware raid is pretty much considered dead (since ~2022).
since my earlier posts i've ended up getting a 9950x3d, repurposed the old machine for a homelab/personal ai. the new cpu has pretty much doubled my benchmark to 60693 - 3rd fastest machine on the current benchmark. although realistically i don't intend on running the PBO function 24/7 which impacts the score somewhat.
what was the purposeo fo w10 and w11? just to check any performance differences?
*note i just saw you have made a post since i started on this
