.. _data-access: *************************************************** Accessing Data through the Graphical User Interface *************************************************** The graphical user interface (Dashboard) for the TOAR phase 2 database is currently under development and will be described here as soon as it is available. For the time being, data from the TOAR database version 2 can only be accessed via the REST API (see :numref:`chapter-rest-api`). Access to version 1 of the database (from TOAR-I) is available through the GUI at https://toar-data.fz-juelich.de/gui/v1/ which redirects to https://join.fz-juelich.de. This web interface requires registration and is described at https://join.fz-juelich.de/static/documentation/JOIN_FAQ.pdf. .. _chapter-rest-api: ***************************************************************** Accessing Data through the REST Application Programming Interface ***************************************************************** A Representational State Transfer (REST) service allows querying all metadata and data products from the TOAR database of surface ozone observations. This API can be used in a web browser or from within a program, from a Unix shell, or in a graphical web application. This section describes the URL structure and sample queries of the TOAR V2 REST interface. For general information on REST, please consult other resources [#f6]_ . ------------------- General Information ------------------- ~~~~~~~~ Base URL ~~~~~~~~ https://toar-data.fz-juelich.de/api/v2/ Response: Description and documentation of the available REST services. ~~~~~~~~ Services ~~~~~~~~ The following information services are available and described individually below. Each service is invoked by appending its name and possible query arguments to the base URL. * stationmeta: query station ids, station names, and station location from the database * timeseries: query the data series id and specific metadata of a series from the database * data: get timeseries data from the database * variables: query information on variables * contacts: query information on contacts * controlled_vocabulary: query the controlled vocabulary and their description from the database * ontology: query the used ontology of the database * database_statistics: only provides number of users, stations, time series and data records, there is nothing from any kind of statistical product * analysis: get bulk time series data and aggregated time series data * geolocation_urls: query information about geolocation urls * stationmeta_changelog: query information on stationmeta * timeseries_changelog: query information on timeseries * search: query for stations / time series with certain metadata * statistics: a separate package providing various statistics on TOAR data ~~~~~~~~~~~~~~~ Query Arguments ~~~~~~~~~~~~~~~ In order to control the database queries and hence the response of the TOAR REST service, you can add arguments to the service URL. These arguments must adhere to the format ``argumentname=value``. The first argument is prepended by a **?** character, all other arguments are separated by **&** characters. ~~~~~~~~~~~~~~~ Response Format ~~~~~~~~~~~~~~~ The default response format is json. You can control the format with the ``format=`` option in the data and ontology queries. Currently, `json `_ and `csv `_ are supported. --------------------------- Description of the Services --------------------------- For all services the default for the number of returned entries is 10, in case you want to see more entries use the query option ``?limit=`` .. _subsection-stationmeta: ~~~~~~~~~~~ Stationmeta ~~~~~~~~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/stationmeta/[id/][?QUERY-OPTIONS] where QUERY-OPTIONS are: limit= (examples: 10) bounding_box=,,, country=,, ... (country code defined in ISO-3166 ALPHA-2) htap_region_tier1_year2010= ... Response: Each query result consists of all fields of station metadata. If no QUERY-OPTIONS are given, the complete set of stations will be returned. Example: https://toar-data.fz-juelich.de/api/v2/stationmeta/CPT134S00/ Further query items are: * /stationmeta/{station_code} * /stationmeta/id/{station_id} * /stationmata/? * * /stationmeta_changelog/{station_id} ~~~~~~~~~~~ Time Series ~~~~~~~~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/timeseries/[?QUERY-OPTIONS] where QUERY-OPTIONS are: limit= station_code=,.... variable_id= format= (json|csv) Response: Each query result consists of all fields of time series metadata. If no QUERY-OPTIONS are given, the complete set of time series will be returned. Example (1), query the first time series: https://toar-data.fz-juelich.de/api/v2/timeseries/?limit=1 Example (2), query the time series with id 25: https://toar-data.fz-juelich.de/api/v2/timeseries/25 Example (3), query the timeseries of ozone measurements at the three listed stations (variable_id 5 = ozone) https://toar-data.fz-juelich.de/api/v2/timeseries/?station_code=MX_PED,CPT134S00,CH0001G&variable_id=5 Further query items are: * /timeseries/{timeseries_id} * /timeseries/id/{timeseries_id} * /timeseries/unique/ * /timeseries/citation/{timeseries_id} * /timeseries/? * * /timeseries_changelog/{timeseries_id} ~~~~ Data ~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/data/timeseries/[?QUERY-OPTIONS] where QUERY-OPTIONS are: format = (json|csv) flags = (see controlled vocabulary for data-flags: https://esde.pages.jsc.fz-juelich.de/toar-data/toardb_fastapi/docs/toardb_fastapi.html#data-flag) Response: Each query result consists of the fields that are specified in the columns argument. If columns are not specified, the output of each record will consist of the fieldsseries_id, network_name, station_id, parameter_label as the series query. If no QUERY-OPTIONS are given, the complete set of stations will be returned. Example (1), query data of time series with id “52”: https://toar-data.fz-juelich.de/api/v2/data/timeseries/52 Example (2), query data of time series with id “52” and return the result as comma separated list: https://toar-data.fz-juelich.de/api/v2/data/timeseries/52/?format=csv Further query items are: * /data/{timeseries_id} * /data/id/{timeseries_id} * /data/timeseries/{timeseries_id}?flags={flag_name} * ... .. _subsection-variables: ~~~~~~~~~ Variables ~~~~~~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/variables/[id/][?QUERY-OPTIONS] where QUERY-OPTIONS are: limit= (default: 10) Response: Each query result consists of a list of variables with name, longname, dispalyname, cf_standardname, units, chemical-formular, and its internal id, which can be used to directly query that specific variable. Further query items are: * /variables/{name} or /variable/?name={name} * /variables/id/{variable_id} or /variables/?id={variable_id} ~~~~~~~~ Contacts ~~~~~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/contacts/[persons/|organisations/|id/][?QUERY-OPTIONS] where QUERY-OPTIONS are: limit= (default: 10) Response: Each query result consists of a list of contacts, either all kinds, persons, organisations, or the information for a specific id. Further query items are: * /contacts/persons/id/{person_id} * /contacts/persons/{name} * /contacts/organisations/id/{organisation_id} * /contacts/organisations/{name} * /contacts/id/{contact_id} ~~~~~~~~~~~~~~~~~~~~~ Controlled Vocabulary ~~~~~~~~~~~~~~~~~~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/controlled_vocabulary/ Response: List of the complete vocabulary in json (raw) format. Further query items are: * /controlled_vocabulary/{name} ~~~~~~~~~~~~~~~~~~~ Database Statistics ~~~~~~~~~~~~~~~~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/database_statistics/ Response: The database statistics is given: number of users, number of stations, number of time series, and the number of data records. You can also query for only one of these numbers by using its name. Further query items are: * /database_statistics/{name} ~~~~~~~~ Ontology ~~~~~~~~ :: Query: https://toar-data.fz-juelich.de/api/v2/ontology/[?QUERY-OPTIONS] where QUERY-OPTIONS are: format = (xml|owl|doc) Response: By default, the query will return the ontology in xml format. Example: https://toar-data.fz-juelich.de/api/v2/ontology/?format=xml ~~~~~~ Search ~~~~~~ As basis for formulating searches use https://toar-data.fz-juelich.de/api/v2/#stationmeta, https://toar-data.fz-juelich.de/api/v2/#timeseries to list all metadata fields and their definitions as well as the controlled vocabulary defined for a subset of the metadata fields. All metadata fields can be used in searches and combined in one query with **&**: :: Query: https://toar-data.fz-juelich.de/api/v2/search/[?QUERY-Options] where QUERY-OPTIONS are any metadata field = value (or comma separated list of values) Response: all metadata of all stations and timeseries fullfilling the criteria. Example: https://toar-data.fz-juelich.de/api/v2/search/?bounding_box=49,7,50,8&variable_id=5,4 will get you all the time series of stations within an area between 49°N 7°E and 50°N 8°E that record ozone or pm1 https://toar-data.fz-juelich.de/api/v2/search/?name=Aachen will provide all stations located in the town of Aachen (done via similarity search) ~~~~~~~~ Analysis ~~~~~~~~ The base URL for the analysis package, a web-service for the calculation of various anaylsis on TOAR data, is https://toar-data.fz-juelich.de/api/v2/analysis . There the API is documented, especially all available analysis methods are listed with their definitions. :: Query: https://toar-data.fz-juelich.de/api/v2/analysis/[ENDPOINT]/[?QUERY-OPTIONS] where ENDPOINT is: data/timeseries: bulk time series download statistics: aggregated time series download where QUERY-Options are any metadata field = value (or comma separated list of values) flags = (see controlled vocabulary for data-flags: https://esde.pages.jsc.fz-juelich.de/toar-data/toardb_fastapi/docs/toardb_fastapi.html#data-flag) sampling = temporal aggregation (only for endpoint=statistics) metrics = statistical aggregation (only for endpoint=statistics) Response: A zip file with the requested data in csv format. Example: Example (1), query data of all German time series: https://toar-data.fz-juelich.de/api/v2/analysis/data/timeseries/?country=DE&limit=None Example (2), query annual median values of all German time series: https://toar-data.fz-juelich.de/api/v2/analysis/statistics/?sampling=annual&metrics=median&country=DE&limit=None .. rubric:: Footnotes .. [#f6] e.g. https://restfulapi.net/ or https://mlsdev.com/blog/81-a-beginner-s-tutorial-for-understanding-restful-api