Dual Xeons - a processing platform I am playing with

[Camelopardalis (Dunk)]

Maybe Win11 forces all the speculative execution mitigations that the Intel chips have, and Win10 doesn’t? Just guessing, but it’s the kind of scenario where you would feel the impact.

Dual booting Linux is easy, it’ll setup the bootloader when it’s done.

[Camelopardalis (Dunk)]

Originally Posted by rustigsmed

… 3rd fastest machine on the current benchmark. although realistically i don't intend on running the PBO function 24/7 which impacts the score somewhat.

Why would you disable PBO?

[g__day (Matthew)]

I wanted to see if Windows 11 Pro offers a discernible upgrade to Windows 10 Pro performance (but it seems a severe downgrade so far).

I thought I would try Win 11 first and then tri-boot to Ubuntu next.

For me it was 80% about seeking more performance and 20% having a fallback if PI on Windows 10 Pro stops doing what it is supposed to!

Nice performance by the way

[g__day (Matthew)]

Well this is totally unexpected by me - I ran WBPP in High Quality mode against 437 subs I captured of NGC 6537 last year that I had yet to process.

I ran it twice on Windows 10 Pro and twice on Windows 11 Pro.

PixInsight's WBPP stacking at high quality - Windows 11 Pro is 3.5 times faster than Windows 10 Pro!

Windows 11 Pro took 46 minutes vs WIndows 10 Pro 2 hours and 36 mins!

LN reference generation took around 60 secs in Win 11 and 17 minutes in Win 10

Local Normalisation was similarily much, much faster in Win11.

So for WBPP I think I will be booting into Win11 henceforth!

Note that for general post stacking processing - Windows 10 Pro seems to be about 30% faster under Windows 10 Pro!

PS

Just re-processed NGC 6744 - WBPP on High Quality settings took 3 hrs 6 mins originally on Windows 10 Pro; on Windows 11 Pro - 56 mins - very, very nice performance boost!

[rustigsmed (Russell)]

Originally Posted by g__day

Well this is totally unexpected by me - I ran WBPP in High Quality mode against 437 subs I captured of NGC 6537 last year that I had yet to process.

I ran it twice on Windows 10 Pro and twice on Windows 11 Pro.

PixInsight's WBPP stacking at high quality - Windows 11 Pro is 3.5 times faster than Windows 10 Pro!

Windows 11 Pro took 46 minutes vs WIndows 10 Pro 2 hours and 36 mins!

LN reference generation took around 60 secs in Win 11 and 17 minutes in Win 10

Local Normalisation was similarily much, much faster in Win11.

So for WBPP I think I will be booting into Win11 henceforth!

Note that for general post stacking processing - Windows 10 Pro seems to be about 30% faster under Windows 10 Pro!

PS

Just re-processed NGC 6744 - WBPP on High Quality settings took 3 hrs 6 mins originally on Windows 10 Pro; on Windows 11 Pro - 56 mins - very, very nice performance boost!

nice improvement!! (still aren't you curious on what it might be on linux??

[g__day (Matthew)]

I certainly do - whilst I expect Ubuntu to be discernibly faster overall - noting the PI benchmarks often show a 60% - 100% improvement under Linux - WBPP is a bit of a surprise beast - I remember a user with a very high core count system (close to 100 cores) found Windows was the fastest for WBPP - which seems totally unexpected and wrong to me.

I seriously think PixInsight team should pay more care with benchmarks - and organise them by operating system variant and have a benchmark for WBPP too. The Benchmarks as I have said over and over on their forums can tell you if something fundamental is going wrong - but generally Jaun and team tend not to pick this priority - as users are always clamouring for more and improved functionality. My view is you have to keep quality of product high too - else tech debt looms and sooner or later everyone will have to pay the price...

Eventually I do plan to benchmark it - the only real way to know performance is to throw a real stacking workload and trying it across all operating systems!

I would love to be able to review the Thread Optimsiation data file - to see if Windows 10 Pro has different settings under Windows 10 Pro.

If it were easy I could take the Win 11 settings and try it in the Win 10 system - and see if that is better or worse - and the same for Win 10.

I still ponder the Thread Optimsation logic may get the logical core count wrong under Windows 10 in some use cases!

[g__day (Matthew)]

My latest tests results - the speed continues

In case anyone is wondering what a lot of CPUs can do with WBPP in high quality mode under Windows 11 Pro - well this test is rather definitive - 717 subs encompassing 55 hours and 34 minutes of data fully processed in 54 minutes - so way faster than my original 8 physical core CPU - which took 4.5 hours.

The performance walk went:

Intel 7820X - 4.5 hours on Windows 11 Pro - on my 7 year old big rig workstation!
Dual Xeons - 36 cores 72 logical processors - 3 hours under Windows 10 Pro
Dual Xeons - 36 cores and 36 logical processors - 2 hours under Windows 10 Pro
Dual Xeons - 36 cores - hyper threading off - 54 minutes under Windows 11 Pro!

[g__day (Matthew)]

A key update in all this analysis - establishing a single setting that hugely affect WBPP performance (confirmed with Adam Block today).

In PixInsight's WBPP - under the light frames tab, when Local Normalisation is used - you can set the selection method to one of two choices - PSF Flux evaluation or Multiscale analysis. Given you are on Windows 11 - if your image has sufficient stars - one should always set the Local Normalisation selection method to PSF Flux evaluation method.

Whilst this setting makes no major difference to run times under Windows 10 - under Windows 11 it makes a major difference.

I tried Win 11 with this option set to multiscale analysis and WBPP times went back up to 2.5 hours - with no improvement in the final master quality. So a lot more calculations - and many of them single threaded - meaning slow - for no gain.

Adam confirmed this to me today - stating the PSF flux evaluation method was the newer algorithim - and should be relied on as the default approach. It is only when there are insufficient stars that one should consider falling back to multiscale analysis! Adam stated when I queired which method I should use that "PSF flux evaluation is correct. All this means is that is it calculating the scaling factor (for normalization purposes) by doing the photometry (of sorts) on the stars. The Multiscale method is the old method... the PSF Flux is the default nowadays. I hope to goodness I never recommended using Multiscale (with exceptions in special cases where there are no stars). PSF flus measurements are the default in WBPP as well."

So the data again with the Scale evaluation method specified:

Intel 7820X - 4.5 hours on Windows 11 Pro - PSF Flux evaluation
Dual Xeons - 36 cores 72 logical processors - 3 hours under Windows 10 Pro - PSF Flux evaluation
Dual Xeons - 36 cores and 36 logical processors - 2 hours under Windows 10 Pro - PSF flux
Dual Xeons - 36 cores - hyper threading off - 2 hours 24 minutes under Windows 11 Pro - Multiscale analysis!
Dual Xeons - 36 cores - hyper threading off - 54 minutes under Windows 11 Pro - PSF flux evaluation!

So bottom line - for WBPP on a Microsoft platform - use Windows 11 Pro and with Local Normalisation ensure you set the selection method to PSF Flux evaluation!