How to download JWST raw data?

[Stephane]

Hi all,

Antoine Grelin (Galactic Hunter) just released a video on Youtube (https://www.youtube.com/watch?v=PTrUwfSrOak&t=390s) explaining how to download raw calibrated exposures from the JWST in .fits format.

The procedure only works for Macs, not Windows. Does anyone know how to obtain the data on Windows? It would be fun to process these images.

Many thanks,
Stéphane

[AstroViking]

Hey Stephane,

Too easy!

Download the ".sh" file from the 'JWST AWS Bulk Download Scripts' page for the data set you are interested in. It's a Unix script file (plain text, nothing fancy), so you can open it with NotePad - maybe save the file with a '.txt' extension to make it easier.

There are a bunch of lines that look like this:

cat <<EOT
<<< Downloading File: https://stpubdata-jwst.stsci.edu/ero/jw02731/jw02731001003/jw02731001003_02105_00001_nrca1_i2d .fits
To: ./jw02731/jw02731001003/jw02731001003_02105_00001_nrca1_i2d .fits
EOT

curl --globoff --progress-bar --create-dirs -C - --output ./jw02731/jw02731001003/jw02731001003_02105_00001_nrca1_i2d .fits 'https://stpubdata-jwst.stsci.edu/ero/jw02731/jw02731001003/jw02731001003_02105_00001_nrca1_i2d .fits'


Look at the end of the line starting with 'curl --globoff...' and then simply cut-n-paste the URL (without the single-quote marks) into your browser or download manager and away you go.

Just make sure you have LOTS of free disk space.

Cheers,
V

[OICURMT]

I assume people are aware that curl is part of Win10/11 now.



Microsoft Windows [Version 10.0.19044.1766]
(c) Microsoft Corporation. All rights reserved.






> curl --help
Usage: curl [options...] <url>
-d, --data <data> HTTP POST data
-f, --fail Fail silently (no output at all) on HTTP errors
-h, --help <category> Get help for commands
-i, --include Include protocol response headers in the output
-o, --output <file> Write to file instead of stdout
-O, --remote-name Write output to a file named as the remote file
-s, --silent Silent mode
-T, --upload-file <file> Transfer local FILE to destination
-u, --user <user:password> Server user and password
-A, --user-agent <name> Send User-Agent <name> to server
-v, --verbose Make the operation more talkative
-V, --version Show version number and quit

This is not the full help, this menu is stripped into categories.
Use "--help category" to get an overview of all categories.
For all options use the manual or "--help all".

[mura_gadi]

Command line transfers, that's a full circle for me, heck of a flash back...

[AstroViking]

Some of us never really left the CLI world. :-)

[Stephane]

Thanks Steve! I’ll try your method as it looks easy.

In the mean time, DSO Imager released a video for Windows (https://m.youtube.com/watch?v=ejVB7p8pl_c).

Thanks again,
Stéphane

[DarwinAstro]

Hi All, Here's another vid for JWST data and processing by Nico Carver from Nebula Photos.
https://www.youtube.com/watch?v=DVuonz26P0w&ab_channel=NebulaPhotos