iota2 feature maps ################## Assuming iota2 is :doc:`fully operational `, this chapter presents an usage of iota2: the production of features maps using satelitte images time series. Introduction to data ******************** iota2 handles several sensors images : * Landsat 5 and 8 (old and new THEIA format) * Sentinel 1, Sentinel 2 L2A(THEIA and Sen2cor), Sentinel L3A (Theia Format) * Various other images already processed, with the ``userFeat`` sensor In this chapter, only the use of Sentinel 2 L2A will be illustrated. To use other sensor, it is necessary to adapt the inputs parameters. To produce spectral indices, an user can use all spectral indices already computed by iota "link to spectral features in iota2 (NDVI...)" ad provide python code to produce custom features. Get the data-set ================ .. include:: i2_tutorial_dataset_description.rst Understand the configuration file ================================= iota2 exploits hundreds of parameters, native or coming from other libraries such as scikit-learn or OTB. These combinations of parameters will make it possible to choose the treatments to be carried out, the choice of the algorithms used or which output to write. A documentation of all these parameters is available :doc:`parameters descriptions `. To simplify the use, iota2 read a configuration file which is a simple text file containing sections and fields. The minimal configuration file contains all required fields to produce a land cover map. .. include:: examples/config_tutorial_features_maps.cfg :literal: For an end user, launching iota2 requires to fill correctly the configuration file. In the above example, replace the ``XXXX`` by the path where the archive has been extracted. Running the chain ================= iota2 launch ************ To launch the chain, simply use the command line .. code-block:: console Iota2.py -config /XXXX/IOTA2_TESTS_DATA/i2_tutorial_features_map.cfg First, the chain displays the list of all steps activated by the configuration file .. include:: examples/steps_features_maps.txt :literal: Once the processing start, a large amount of information will be printed, most of them concerning the dask-scheduler. Did it all go well? =================== iota2 displays a lot of information during execution. It becomes difficult to identify if everything went well or if an error occurred. To this end, in the ``output_path`` a log directory is available : ``output_path/logs``. The logs directory is ordered as: .. code-block:: console ├── CommonMasks │   └── common_mask_T31TCJ.out ├── html │   └── * ├── merge_feat_maps_by_tiles.out ├── merge_feat_maps.out ├── run_informations.txt ├── sensorsPreprocess │   └── preprocessing_T31TCJ.out ├── tasks_status_i2_features_map_1.svg └── write_features_map ├── write_features_T31TCJ_chunk_0.out ├── write_features_T31TCJ_chunk_10.out ├── write_features_T31TCJ_chunk_11.out ├── write_features_T31TCJ_chunk_12.out ├── write_features_T31TCJ_chunk_13.out ├── write_features_T31TCJ_chunk_14.out ├── write_features_T31TCJ_chunk_15.out ├── write_features_T31TCJ_chunk_16.out ├── write_features_T31TCJ_chunk_17.out ├── write_features_T31TCJ_chunk_18.out ├── write_features_T31TCJ_chunk_19.out ├── write_features_T31TCJ_chunk_1.out ├── write_features_T31TCJ_chunk_2.out ├── write_features_T31TCJ_chunk_3.out ├── write_features_T31TCJ_chunk_4.out ├── write_features_T31TCJ_chunk_5.out ├── write_features_T31TCJ_chunk_6.out ├── write_features_T31TCJ_chunk_7.out ├── write_features_T31TCJ_chunk_8.out └── write_features_T31TCJ_chunk_9.out Each step has its own log directory. In these directories two kind of log can be found ``*_out.log`` and ``*_err.log``. The error are compiled in "err" file and the standard output in "out" file. With the scheduler dask, iota2 go as far as possible while the data required for the next steps is available. To simplify the error identification, an interactive graph is produced in a html page. Nodes in graph can have three colors (red: error, blue: done, white: not yielded). By clicking on graph node, the corresponding log file is openned. If despite all this information, the errors can not be identifiyed or solved, the iota2 can help all users. The simplest way to ask help is to create an issue on framagit by adding the archive available in log directory. Output tree structure ===================== In this section, all output folders are described. .. code-block:: console ├── by_tiles │   └── T31TCJ.tif ├── customF │   ├── T31TCJ_chunk_0.tif │   ├── T31TCJ_chunk_10.tif │   ├── T31TCJ_chunk_11.tif │   ├── T31TCJ_chunk_12.tif │   ├── T31TCJ_chunk_13.tif │   ├── T31TCJ_chunk_14.tif │   ├── T31TCJ_chunk_15.tif │   ├── T31TCJ_chunk_16.tif │   ├── T31TCJ_chunk_17.tif │   ├── T31TCJ_chunk_18.tif │   ├── T31TCJ_chunk_19.tif │   ├── T31TCJ_chunk_1.tif │   ├── T31TCJ_chunk_2.tif │   ├── T31TCJ_chunk_3.tif │   ├── T31TCJ_chunk_4.tif │   ├── T31TCJ_chunk_5.tif │   ├── T31TCJ_chunk_6.tif │   ├── T31TCJ_chunk_7.tif │   ├── T31TCJ_chunk_8.tif │   └── T31TCJ_chunk_9.tif ├── features │   └── T31TCJ │   └── tmp │   ├── MaskCommunSL.dbf │   ├── MaskCommunSL.prj │   ├── MaskCommunSL.shp │   ├── MaskCommunSL.shx │   ├── MaskCommunSL.tif │   ├── Sentinel2_T31TCJ_input_dates.txt │   ├── Sentinel2_T31TCJ_interpolation_dates.txt │   └── Sentinel2_T31TCJ_reference.tif ├── final │   └── features_map.tif ├── IOTA2_tasks_status.txt ├── logs │   ├── * ├── logs.zip └── T31TCJ customF ------- Temporary data used for reassemble maps for each tiles. Can be removed by hand by user at the end of the processing. by_tiles -------- The tile chunks are merged to produce a map for each tile final ----- This folder contains the final products of features map builder. features -------- For each tiles, contains useful information: - ``tmp/MaskCommunSL`` : the mask of all sensors for this tile - ``tmp/Sentinel2_T31TCJ_reference.tif`` : the image used for reprojecting data - ``tmp/Sentinel2_T31TCJ_input_dates.txt`` : the list of date detected in ``s2_path`` - ``tmp/Sentinel2_T31TCJ_interpolation_dates.txt`` : the list of output interpolation dates Final products ============== Features map ------------ .. figure:: ./Images/Features_map.png :scale: 50 % :align: center :alt: features map Features_map.tif Example To go further ============= .. toctree:: :maxdepth: 1 External features .. raw:: html :file: interactive-tree.html