How to query the web archive
The Sentinel-1 MPC website provides a queryable API at https://sar-mpc.eu/api/v1/
to find IPF ADF files.
This web api was introduced to allow users to quickly and efficiently find ADF files instead of having to scrape the html listing pages from the MPC website.
The web api covers the following types of products:
product_type | description |
---|---|
AUX_PP1 |
L1 Processor Parameters |
AUX_CAL |
Calibration Auxiliary Data |
AUX_INS |
Instrument Auxiliary Data |
AUX_PP2 |
L2 Processor Parameters |
AUX_SCS |
Simulated Cross Spectra |
AUX_ML2 |
Machine Learning Models for L2 |
AUX_SCF |
SETAP Configuration File |
AUX_ITC |
Instrument Timing Calibration |
The metadata that is returned for each product uses the data model as used by the Muninn software. This consists of a generic set of core metadata fields (that are Sentinel-1 independent) and optional sets of additional metadata fields grouped into namespaces. There is a sentinel1
namespace that provides additional metadata that is common for the Sentinel-1 mission and an adf
namespace with ADF specific version information.
Examples
Note that the examples below will show an HTML page with results when executed from within a browser. When using a command line tool or programmatic interface the result will be a JSON structure.
List all products of a specific product_type
To list all Calibration Auxiliary files use:
https://sar-mpc.eu/api/v1/?product_type=AUX_CAL
Find the right AUX file for a product
Based on the validity start and instrument configuration id of a product (e.g. validity start = 2018-01-02T12:34:56 and ICID = 6) the applicable auxiliary file (e.g. AUX_PP1) can be found as follows:
https://sar-mpc.eu/api/v1/?product_type=AUX_PP1&sentinel1__instr_conf_id=6&adf__active=1&validity_start__lte=2018-01-02T12:34:56&adf__stop_time__gt=2018-01-02T12:34:56
Metadata sync
A clone of the archive can be kept by requesting the changes made to the archive after a certain date (the last time the clone was updated). This will return the metadata entries for all new and updated products.
Caveat: since the archive does not keep track of removed entries, the API does not provide information for these cases.
Example:
https://sar-mpc.eu/api/v1/?metadata_date__gt=2018-10-13&ordering=metadata_date&mode=extended
Query parameters
The core query parameters are:
- uuid (UUID)
- hash (Text)
- size (Long)
- metadata_date (Timestamp)
- product_type (Text)
- product_name (Text)
- physical_name (Basename)
- validity_start (Timestamp)
- validity_stop (Timestamp)
- creation_date (Timestamp)
- remote_url (Remote)
Sentinel1 specific query parameters. These are prefixed with sentinel1__
:
- mission (Text)
- S1A, S1B, S1C
- instr_conf_id (Integer)
- processing_facility (Text)
- processor_name (Text)
- processor_version (Text)
ADF specific query parameters (applicable to AUX_CAL
, AUX_INS
, AUX_PP1
, AUX_PP2
, AUX_SCS
, AUX_ML2
product types). These are prefixed with adf__
:
- active (Boolean)
- stop_time (Timestamp)
- change_description (Text)
Query parameter types
Fields might be set to null, this can be checked with the isnull
modifier.
Providing a value ousite the scope of the parameter type leads to undefined results.
Text
Free text field
Number
Number fields (Integer, Long) can contain only digits.
Timestamp
Dates are UTC. They are represented in ISO 8601 format without time zone information, i.e.:
2017-10-01T00:39:50
20171001T003950
The time part can be omitted, i.e.
2017-10-01
20171001
Boolean
A boolean field is either:
true
or1
false
or0
The string values are case-insensite.
Query parameter lookups
Other criteria are available besides providing an exact value.
Which criteria is available depends on the pareameter type.
These criteria are used by appending __<lookup_name>
to the field name:
_lookup_ | Boolean | Number | Timestamp | Text | description |
---|---|---|---|---|---|
exact | x | x | x | x | == (default lookup) |
ne | x | x | x | x | != different |
in | x | x | x | x | comma-separated list of exact values |
isnull | x | x | x | x | Takes either true or false |
lt | x | x | < less than |
||
gt | x | x | > greater than |
||
lte | x | x | <= less than or equals |
||
gte | x | x | >= greater than or equals |
||
range | x | x | range test (inclusive) | ||
date | x | ignores the time part | |||
year | x | year | |||
month | x | month, 1 (January) to 12 (December) |
|||
day | x | day of month | |||
week | x | week number, 1 to 52 or 53 |
|||
week_day | x | day of week, 1 (Sunday) to 7 (Saturday) |
|||
time | x | ignores the date part | |||
hour | x | hour, 0 to 23 |
|||
minute | x | minute, 0 to 59 |
|||
second | x | second, 0 to 59 |
|||
startswith | x | ||||
endswith | x | ||||
contains | x | ||||
iexact | x | case-insensitive exact |
|||
istartswith | x | case-insensitive startswith |
|||
iendswith | x | case-insensitive endswith |
|||
icontains | x | case-insensitive contains |
Pagination
If there are more than 50 products matching a query, the results will be paginated. The following properties will be available along the results:
count
- total number of resultsnext
- URI to get the next page of resultsprevious
- URI to get the previous page of results
If needed, the number of results per page can be adjusted by adding a page_size
parameter (to a maximum of 1000):
https://sar-mpc.eu/api/v1/?page_size=10
Ordering
The default ordering of results is by ascending validity_start.
The sort order can be customized, using the ordering
query parameter.
Custom ordering needs to be enabled, see optional configuration section below.
sort by descending metadata_date
https://sar-mpc.eu/api/v1/?ordering=-metadata_date
multiple fields can be specified
https://sar-mpc.eu/api/v1/?ordering=product_type,-adf__stop_time
Output control
By default, the output will contain only the core metadata. To get the full metadata record, specify mode=extended
in the query string. This will include all the namespaces (sentinel1
and adf
), links and tags for each product.