cso_dataspace
module¶
The cso_dataspace
module provides classes for accessing data from the
Copernicus DataSpace.
To browse through the data, use the Browser.
Account setup¶
To be able to download data from the DataSpace, first Register and create an account.
On a Linux system, login/passwords for websites can be stored in the users .netrc
file
in the home directory. Create this file if it does not exist yet, and add the following
line with the login name of the account (your email) and the chosen password:
machine zipper.dataspace.copernicus.eu login Your.Name@institute.org password ***********
The file should be readible and writable for you only:
chmod 400 ~/.netrc
DataSpace API’s¶
The DataSpace could be access with a number of different APIs.
Currently the OpenSearch API is used as that was the first that worked as needed.
In future the STAC API might be used, as this is becoming more and more the standard in the Earth Observation community. Within CSO it was already used by for example PAL API, but could not get working yet for the DataSpace. See the STAC product catalog for more information.
Class hierchy¶
The classes and are defined according to the following hierchy:
Classes¶
- class cso_dataspace.CSO_DataSpace_Inquire(rcfile, rcbase='', env={}, indent='')¶
Bases:
UtopyaRc
Inquire available Sentinel data from the Copernicus DataSpace.
Before data could be downloaded from the DataSpace, setup your Account setup.
Currently the OpenSearch API is used as that was the first that worked as needed; in future, the STAC product catalog might be used. See also the OpenSearch Description for the various parameters that the API accepts.
A query is sent to search for products that are available for a certain time and overlap with a specified region. The result is a list with orbit files and instructions on how to download them.
In the settings, specify the time range over which files should be searched:
<rcbase>.timerange.start : 2018-07-01 00:00 <rcbase>.timerange.end : 2018-07-01 23:59
Specify the base url of the API:
<rcbase>.url : https://catalogue.dataspace.copernicus.eu/resto/api
Define the collection name with:
<rcbase>.collection : Sentinel5P
Provide a product type:
! product type (always 10 characters!): <rcbase>.producttype : L2__NO2___
Eventually specify a target area, only orbits with some pixels within the defined box will be downloaded:
! target area, leave empty for globe; format: west,south,east,north <rcbase>.area : !<rcbase>.area : -30,30,35,76
The table will also create the url’s to download a file; specifity the template that should be used:
! template for download url given “{product_id}”: <rcbase>.download_url : https://zipper.dataspace.copernicus.eu/odata/v1/Products({product_id})/$value
Name of output csv file:
! output table, here including date of today: <rcbase>.output.file : ${my.work}/PAL_S5P_NO2_%Y-%m-%d.csv
Example records (with extra whitespace to show the columns):
orbit;start_time ;end_time ;processing;collection;processor_version;filename ;href 11488;2020-01-01 02:34:16;2020-01-01 04:15:46;RPRO ;03 ;020400 ;S5P_RPRO_L2__CH4____20200101T023416_20200101T041546_11488_03_020400_20221120T003820.nc;https://zipper.dataspace.copernicus.eu/odata/v1/Products(b3f240e6-505d-4cae-97ea-43a8778a318d)/$value 11487;2020-01-01 00:52:46;2020-01-01 02:34:16;RPRO ;03 ;020400 ;S5P_RPRO_L2__CH4____20200101T005246_20200101T023416_11487_03_020400_20221120T003818.nc;https://zipper.dataspace.copernicus.eu/odata/v1/Products(a3d40f81-6c86-44bc-bc4b-457ff069b121)/$value :
- class cso_dataspace.NullAuth¶
Bases:
AuthBase
Force requests to ignore the
~/.netrc
file.Some sites do not support regular authentication, but we still want to store credentials in the
~/.netrc
file and submit them as form elements. Without this, requests would otherwise use the~/.netrc
which leads, on some sites, to a 401 error.Use with:
requests.get( url, auth=NullAuth() )
Source:
- class cso_dataspace.CSO_DataSpace_Downloader¶
Bases:
object
Class to download single file from Copernicus DataSpace. The object defined by the class is used to store an access token that is re-used until it is expired.
The
DownloadFile
method should be used to actually download a file.Usage:
# initialize downloader: downloader = CSO_DataSpace_Downloader() # download file: downloader.DownloadFile( "https://zipper.dataspace.copernicus.eu/odata/v1/Products('d483baa0-3a61-4985-aa0c-5642a83c9214')/$value", "orbit.nc" )
- CreateToken(href, indent='')¶
Create access token.
- DownloadFile(href, output_file, maxtry=10, nsec_wait=5, nsec_wait_max=600, dmode=None, indent='')¶
Download file from DataSpace.
If a request fails it is tried again up to a maximum of
maxtry
times, with a delay of initiallynsec_wait
between requsts.Arguments:
href
: download url, for example:https://zipper.dataspace.copernicus.eu/odata/v1/Products('d483baa0-3a61-4985-aa0c-5642a83c9214')/$value
output_file
: target file
Optional arguments:
maxtry
: number of times to try again if download failsnsec_wait
: initial delay in seconds between requests; with every new attempt, this will be increased up to a maximum ofnsec_wait_max
dmode
: directory creation mode, e.g.777