# Analysis Service (Get statistics)

## Access for anonymous users

Anonymous user can only download an example dataset.

## Using the Analysis Service

Clicking on 'Get statistics' shows the following window for a logged in user (role: power user):

![Analysis Service interface](images/analysis_service.jpg)

Here, you need to select the required parameters. Starting with the **Daterange** by providing start and end day, a **Statistics** for the analysis and a **Sampling** strategy (e. g. daily, monthly, yearly).
For more details on the possible statistical analysis click the little 'i' to be redirected to the documentation.  
The **Crops** field will only be active, if the **Sampling** 'vegseason' has been chosen.

There are also optional parameters.  
The **Flags** field refers to the data quality flags. We use 'all OK' as a default value, which request our internal quality checks to be passed for each value. For more details you can simply click the little 'i' to open the documentation in a separate tab of your browser.
Different output formats are available for the users' convenience. You can choose between 'raw' and 'by_statistic' by selecting this from the dropdown of **Output format**. The default is 'raw'.
For more details on the possible output formats click the little 'i' to be redirected to the documentation.  

The ***Merge Time Series*** option: The timeseries at a single station are merged into a single timeseries before the statistical analysis is done.  
As described in the technical guide on data processing [Chapter Three: The TOAR data processing workflow](https://toar-data.fz-juelich.de/sphinx/TOAR_TG_Vol02_Data_Processing/build/latex/toardataprocessing--technicalguide.pdf), a station can have several timeseries, including two or more timeseries for the same variable. This can be the case for different instruments measuring the same variables or through a timeseries being distributed over different networks, like national and European institutions.

To filter the data used in the calculation, click on the 'Filter (0)' button.
This opens the same 'Filter' modal as if it had been invoked from the centre of the top of the dashboard (see [Filtering](filtering.md)).
The number of currently active filters is indicated in brackets after the word 'Filters'. Filtering is useful for limiting the amount of data used in the analysis, since different user roles (anonymous user, registered user, TOAR user and power user) are associated with different restrictions and permissions (see [Registration](registration.md)), and some calculations should be divided into smaller subtasks for performance reasons.

Filling out the fields will lead to the calculation of the required TOAR Billing Units (TBU):
![Example Request](images/analysis_service_example.jpg)

The bottom row gives an information on the size of this request, the monthly limits and the ***Confirm*** button to submit the request. The TBU fields have mouse over tool tips providing more details on the billing.

After submitting the request, the query for the request will be displayed in an extension of the previous window.
![Submitted request](images/analysis_service_request.jpg)
The request can be used by developers who want to programmatically access the data rather than just using the visual interface.
The results of the request can be accessed through the clickable status link. This link is also stored in the user profile of the dashboard.

The status is the result of the underling service and served in JSON format. Opening the link will display the content and give information on the status of the job. As soon as the processing is finished, it will redirect to the download of the results.
![Status from getting statistics](images/analysis_service_status.jpg)

## Filtering for the Analysis Service

The details on the filtering have been described in a [previous section](filtering.md). We pass all filters as additional metadata.

## Resulting datasets

The resulting datasets are provided as an zip-archive containing csv-files. The metadata of the dataset include a large number of information, including the used requests and filtered metadata.

## Additional Resource:

We offer toarstats as a project on our [gitlab](https://gitlab.jsc.fz-juelich.de/esde/toar-public/toarstats). It can be used as an offline package.