Direkt zum Inhalt | Direkt zur Navigation

Personal tools

Sektionen

Script Based Download

last modified Aug 25, 2016 10:32 AM
Access to data in the ENES data federation from the Linux or MacOSX command line of your computer

To automatically access collections of files you have the option to download a Wget script from one of the ESGF portals.

!! This option is only valid for Linux or Mac operating systems !!

If you work behind a firewall, port 7512 (TCP) must be open.

Please perform the following steps:

  • Login on one of the portals and add data to your DataCart
  • Unless you are already registered, become member in the right access control group(s) for the data you need
  • Download is limited to 1000 files per script. Optional but helpful: Reduce the number of files with the text filter function. Without the following sub-steps you will create a download script for whole datasets with all variables and that maybe many!
    • In "My Data Cart" fill in the names of the variables you need into the text field (step 1)
    • Click on "Apply" (step 2)

Text field for file selection in ESGF DataCart

  • Select "WGET Script"
  • You will get a browser notification to download a file called similar to "wget-some_long_number.sh". Press OK to download.
  • Optional but helpful: You may edit your script with a text editor. Delete lines with download files you don't need
  • Execute the script:
    • If you have Oracle Java 1.7+, use the default script call
bash wget-some_long_number.sh
    • Else try -H option
bash wget-some_long_number.sh -H
  • You will be asked for your OpenID and password. The script now contacts the portal where you are registered to retrieve security information
  • Your download now starts - take a coffee

More detailed information can be found in the CoG User Tutorial Downloading data using ESGF Wget scripts.