---
title: "Quick start guide to TheiaR package"
author: "Xavier Laviron"
date: "`r Sys.Date()`"
output: rmarkdown::html_vignette
  
vignette: >
  %\VignetteIndexEntry{Quick start guide to TheiaR package}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

# TheiaR: search, download and manage data from Theia

The TheiaR package provides an efficient and clean interface to search, download
and manage products from [Theia website](https://theia.cnes.fr/atdistrib/rocket/#/home).


## Overview

The basic functionalities are:

- Search available tiles through Theia API
- Download tiles resulting from a search
- Download tiles contained in a cart (`.meta4` file) obtained from Theia
  website.
- Read band into `RasterStack` objects (with the `raster` library)
- Read band into `gdalcubes` objects (with the `gdalcubes` library)
- Extract archives (not recommended)

__NOTE:__ To search and download data from Theia, you will need to [register to
their website](https://sso.theia-land.fr/theia/register/register.xhtml).

__NOTE:__ In order to use _Landsat_ or _SpotWorldHeritage_ products, you'll need
to make a first manual download to agree to the license and validate your
account.


## Installation

You can install the latest stable version from Github with:

```{r , eval=FALSE}
devtools::install_github('norival/theiaR')

# or, to install the development version
devtools::install_github('norival/theiaR', 'devel')
```

Or, you can install it from CRAN:

```{r , eval=FALSE}
isntall.packages('theiaR')
```


## Complete example

A workflow to search and download tiles would be something like:

```{r , eval=FALSE}
library(theiaR)

# create a list containing the query
myquery <- list(collection = "SENTINEL2",
                town       = "Grenoble",
                start.date = "2018-07-01",
                end.date   = "2018-07-06")

# create a collection from the query
mycollection <- TheiaCollection$new(query = myquery, dir.path = ".", check = TRUE)

# check available tiles fro the query
mycollection$status

# download the tiles into 'dir.path'
mycollection$download(auth = "path/to/auth/file.txt")
```


## Step-by-step guide

First, load the package.

```{r , eval=TRUE}
library(theiaR)
```

To search and download data from Theia, you will need to [register to their
website](https://sso.theia-land.fr/theia/register/register.xhtml).

__NOTE:__ In order to use _Landsat_ or _SpotWorldHeritage_ products, you'll need
to make a first manual download to agree to the license and validate your
account.


### Create a collection of tiles

The first step is to create a collection of tile(s). This can be done either
from a query or from a cart file.


#### Create a collection from a query

A query is simply a named `list` of search terms. For example:

```{r , eval=T}
myquery <- list(collection = "SENTINEL2",
                town       = "Grenoble",
                start.date = "2018-07-01",
                end.date   = "2018-07-06")
```

will create a query to Theia database, looking for tiles from Sentinel2
satellite around Grenoble, between 2018-07-01 and 2018-07-06.

It accepts the following terms.

* __collection__: The collection to look for. Accepted values are: `SENTINEL2`,
  `LANDSAT`, `Landsat57`, `SpotWorldHeritage`, `Snow`. Defaults to `SENTINEL2`.

* __platform__: The platform to look for. Accepted values are: `LANDSAT5`,
  `LANDSAT7`, `LANDSAT8`, `SPOT1`, `SPOT2`, `SPOT3`, `SPOT4`, `SPOT5`,
  `SENTINEL2A`, `SENTINEL2B`.

* __level__: Processing level of products. Accepted values are: `LEVEL1C`,
  `LEVEL2A` and `LEVEL3A`, `N2A`. Defaults to `LEVEL2A` (or `N2A` if querying
  Landsat57 collection).


To specify the location of the tiles, several alternatives are available.
You can specify the town around which you want your data with:

* __town__: The location to look for. Give a not too frequent town name.


You can specify directly the tile ID if you know it:

* __tile__: The tile identifier to retrieve (_e.g._ T31TGK)

You can specify a point by giving its x/y coordinates:

* __latitude__: The x coordinate of a point.

* __longitude__: The y coordinate of a point.


Or you can specify a rectangle by giving its min/max coordinates:

* __latmin__: The minimum latitude to search.

* __latmax__: The maximum latitude to search.

* __lonmin__: The minimum longitude to search.

* __lonmax__: The maximum longitude to search.


You can also look for a specific orbit number or relative orbit number:

* __orbit.number__: The orbit number

* __rel.orbit.number__: The relative orbit number


Finally, you can filter results by giving the date range and the maximum cloud
cover:

* __max.clouds__: The maximum of cloud cover wanted (0-100).

* __start.date__: The first date to look for (format: `YYYY-MM-DD`).

* __end.date__: The last date to look for (format: `YYYY-MM-DD`).


You can then create your collection with:

```{r , eval=FALSE}
mycollection <- TheiaCollection$new(query = myquery, dir.path = ".", check = TRUE)
```

where `dir.path` is the path you want your tiles to be further downloaded (This
only queries Theia's catalog for available tiles, nothing is downloaded). If
tiles are already present in `dir.path`, they will be checked by computing a
checksum and comparing it to the hash provided by Theia (only available for
Sentinel2 data, no hash is provided for other collections, and files are then
assumed to be correct). This ensures that the files have been correctly
downloaded. Set `check = FALSE` to skip file's check.

```{r , eval=FALSE}
print(mycollection)
```


#### Create a collection from a cart file

Alternatively, you can download a cart from Theia. To create a cart, login to
Theia website, make a [search](https://theia.cnes.fr/atdistrib/rocket/#/home)
for tiles, and add wanted tiles to your cart. Then, download your cart and save
the resulting `.meta4` file to your disk.

You can then create your collection using this file:

```{r , eval=T}
cart.path <- system.file("extdata", "cart.meta4", package = "theiaR")

mycollection <- TheiaCollection$new(cart.path = cart.path,
                                    dir.path  = ".",
                                    check     = TRUE)

print(mycollection)
```

As above, it will check the hash of files if they are already present in
`dir.path`.


### Download your tiles

The next step is to download your collection. You can get the status of your
collection by running:

```{r , eval=TRUE}
mycollection$status
```

To download all tiles in a collection, simply run:

```{r , eval=FALSE}
mycollection$download(auth = myauth)
```

where myauth is the path to file storing your Theia credentials. If it does not
exist yet, you will be securely prompted for your login and password, and the
file will be created.

This will check if files are present, check their hashes, and download them if
needed (if files do not exist or checksums are wrong). To overwrite existing
files, run:

```{r , eval=FALSE}
mycollection$download(auth = myauth, overwrite = TRUE)
```

### Read bands from zip files

Alternatively, you can read bands directly from the zip archives (by using the
`vsizip` interface provided by GDAL). Use:

```{r , eval=FALSE}
mytile$bands
```

to get a list of available bands. Then:

```{r , eval=FALSE}
mybands <- mytile$read(bands = c("B5", "B6"))
```

to load the bands into memory (returns a `RasterStack` object). It performs the
necessary corrections on the values.

You can also read bands from a collection by running:

```{r , eval=FALSE}
mybands <- mycollection$read(bands = c("B5", "B6"))
```

which returns a `list` of `RasterStack` objects.

_NOTE: loading several tiles needs a lot of memory (~900MB/tile)_


### Create a `gdalcubes` collection

Alternatively, you can use the great [gdalcubes](https://github.com/appelmar/gdalcubes_R)
package to create a three dimensional representation of the tiles. Simply run:

```{r , eval=FALSE}
library(gdalcubes)

gdalcubes <- mycollection$as_gdalcube("path/to/gdalcubes.sqlite")
```

where `path/to/gdalcubes.sqlite` is the path to store the gdalcubes object data.


### Extract tiles

If you want to extract the archives, you can run:

```{r , eval=FALSE}
file.path <- mycollection$extract()
```

which will extract tiles into the same directory as the archives.

**This is not recommended, as this will take a large amount of disk space**


## Acknowledgment

Thanks to Olivier Hagolle for his work on `theia_download.py`
([github](https://theia.cnes.fr/atdistrib/rocket/#/home)), which has inspired
this package.