The WGCV undertakes a variety of activities related to the calibration and validation (cal/val) of Earth Observation data. Our current activities focus mainly on the requirements identified by the intergovernmental Group on Earth Observations (GEO) and their goal to achieve a Global Earth Observation System of Systems (GEOSS). To this end, GEO has identified a series of tasks related to calibration and validation in which we play a lead role. Some of our current activities include:
SRIX4Veg: Surface Reflectance Intercomparison Exercise for Vegetation
SRIX4Veg represents a joint effort between ESA, CSIRO and Geoscience Australia to test user-based differences in UAV-based surface reflectance fiducial reference measurements used for validation of surface reflectance satellite products, and to ensure consensus on UAV-based surface reflectance validation good practice protocols, which is conducted in the framework of the ESA FRM4Veg initiative.
The objectives of SRIX4Veg are:
- Testing user-based differences in UAV-based surface reflectance measurements, including instrument and operator biases as well as measurement collection procedures.
- Helping design field measurement protocols and validation methodologies that are clear and can be easily applied by all users.
- Ensuring international buy-in and consensus on the UAV-based surface reflectance validation protocols.
- Publish UAV-based surface reflectance inter-comparison results.
These objectives shall be achieved inviting the scientific community to a round-robin inter-comparison exercise and two international workshops to discuss the design of the experiment and the main outcomes.
The main requirement for participation in the inter-comparison is a UAV-mounted hyperspectral imager capable of measuring 400 nm – 1000 nm contiguously: <= 10 nm spectral resolution
The first SRIX4Veg workshop was held online on 28th March 2022.
The second SRIX4Veg workshop will take place at ESA ESRIN on Thursday 23rd and Friday 24th November 2023 (with a hybrid option available for those that cannot travel). For more information visit the workshop website: https://frm4veg.org/srix4veg/workshop-2/. Registration is available here.
Please visit the SRIX4Veg website for more details: https://frm4veg.org/srix4veg/
DEMIX: Digital Elevation Model Intercomparison eXercise (DEMIX) Working Group
Established in 2020, the DEMIX working group is developing qualitative and quantitative methods to provide DEM users with open tools and data to compare and select the most appropriate digital elevation model for their application. The working group has been divided into three subgroups to deal with the variety of issues stemming from implementing such a feat for global products.
For a brief overview of the project please refer to the ISPRS2021 conference paper or watch this 10 minute introductory video. Further detail about the work in the subgroups can be found below.
DEMIX Subgroup 1 – Terminology and Analytical Basis
Chairs: Peter Guth (pguth@usna.edu), Igor Florinsky (iflor@mail.ru)
Subgroup 1 is in charge of a common terminology and analytical basis to facilitate cooperation of experts from different domains. It is doing so by reviewing existing terminology and its use in literature in order to propose a consistent set of terms which also matches with the analytical basis of geomorphometry. Its task is furthermore to review the methodological developments in SG2 in order to ensure their coherence and consistency with the agreed set of definitions.
The subgroup has published a peer-reviewed paper in support of its mandate, available at https://doi.org/10.3390/rs13183581.
DEMIX Subgroup 2 – Algorithms and Software
Chairs: Conrad Bielski (conrad.bielski@eoxplore.com), Carlos Grohmann (guano@usp.br)
The DEMIX subgroup 2 is concentrating their efforts on identifying methods and implementations for the quantitative and/or qualitative intercomparison of DEM products. While there is a significant body of scientific literature covering a dizzying number of quality characteristics, the majority have focused on very local products and/or special cases. The goal of DEMIX is to provide options to end-users that can help guide them in deciding the most appropriate DEM product for their application and support geospatial practitioners defend their choices.
With the support and feedback from subgroup 1, the methods proposed will be implemented by subgroup 3 in a manner to promote open science, communication and discovery of DEM product quality fit for purpose.
DEMIX Subgroup 3 – Platforms and Processing
Chair: Serge Riazanoff (serge.riazanoff@visioterra.fr)
Subgroup 3 concentrates on applying the methodological framework developed within the other subgroups. Specifically, it aims at making available to a wider audience tools for the visualisation of the results of the inter-comparison of DEMs. This will be achieved by providing preferably open-source tools and performance tests in a variety of geographical regions around the world in order to demonstrate and verify the applicability of the DEM inter-comparison approach.
Furthermore, subgroup 3 will help to define the minimum standards and requirements regarding user access, data availability, programming interfaces and processing power, to which open candidate platforms should comply in order to host DEM benchmarking activities. Finally, this subgroup will be in charge of implementing and monitoring DEM benchmarking and comparison at global scale.
BRIX-2: Second Biomass Retrieval Intercomparison eXercise
BRIX-2 represents a joint effort between ESA and NASA to intercompare algorithms specifically for biomass mapping using current and future spaceborne missions.
The objectives of BRIX-2 are:
- Provide an objective, standardized comparison and assessment of biomass retrieval algorithms developed for the Biomass, NISAR and GEDI missions, and fusion of these mission datasets.
- Establish a forum to involve scientists in the development of retrievals that have so far not been part of the biomass community.
- The adoption of vetted validation standards and methods to compare biomass estimates to reference datasets (e.g., field plots or airborne lidar biomass maps).
- Collect inputs from the biomass user and scientific community on data formats and characteristics towards the generation of Analysis Ready Data.
These objectives will be achieved by making available standardized test cases (based on airborne campaign and spaceborne simulated data), inviting the scientific community to develop and apply retrieval algorithms based on these test cases, and finally comparing and evaluating the performance of submitted results.
The first BRIX-2 workshop was held online, from 29-30 April 2021.
Please visit the BRIX-2 website for more details: http://polinsar-biomass2021.esa.int/brix-2/
ACIX/CMIX – Atmospheric Correction and Cloud Mask Intercomparison Exercises
NASA and ESA have collaborated within the framework of CEOS to conduct three workshops thus far on the intercomparison of Atmospheric Correction and Cloud Mask algorithms. The impetus for such an effort has been the evolution of multiple high-resolution sensors that have facilitated the analysis of long time-series. The essential step of creating consistent surface reflectance (SR) products from these sensors was begun by mulitple entities, for SR products from sensors such as Landsat-8 and Sentinel-2. This intercomparison of atmospheric correction processors began in early 2016, and thier aim to point out strengths and weaknesses as well as commonalities and discrepancies of the various processors being implemented.
Many teams with atmospheric correction algorithms were invited to pariticipate, and 12 teams partook in the first exercise, ACIX I. Currently, a total of 35 teams are participating from US, Europe, Australia and China. In the second exercise, ACIX II, the atmospheric correction is divided into two parts, one over land, and the other over water. And, in parallel, a Cloud Mask intercomparison exercise is also implemented.
The current status of these activities, following the first ACIX II/CMIX workshops is:
- Agreement to include surface reflectance measurements from RadCalNet (LaCrau and Gobabeb), campaign data from DLR and surface reflectance measurements from GeoSciences Australia.
- Consensus by all the teams on the need for automated surface reflectances over vegetated/low-reflectance targets.
- The test sites of ACIX II-Land will be all the AERONET sites with currently available measurements (≈150 sites identified).
- For ACIX II-Water, the sites will be confirmed depending on the availability of in situ data, which are currently being collected by the coordinators.
- For CMIX, the sites/scenes will be selected based on the various available validation datasets.
- There are anticipated issues with the download and processing of large volumes of data involved in the comparison. Multiple alternatives are being considered.
- The follow-on workshop will be take place at ESRIN in mid to late 2019.
A summary report of the first ACIX experiment, [Doxani et al. (2018)], is available here.
For additional details, visit the workshop website, or contact Georgia Doxani
Radiometric Calibration Network (RadCalNet) Web Site Operational
Following years of planning, development, and beta testing, we are pleased to announce the release of the RadCalNet web portal to the public!
RadCalNet provides satellite operators with SI-traceable Top-of-Atmosphere (TOA) spectrally-resolved reflectances to aid in the post-launch radiometric calibration and validation of optical sensors onboard Earth Observation satellites.
The initial set of instrumented reference test sites are:
- Railroad Valley Playa, USA, operated by the University of Arizona
- La Crau, France, operated by CNES
- Baotou, China, operated by the Chinese Academy of Opto-electronics (AoE)
- Gobabeb, Namibia, operated by ESA/CNES
Each member site takes responsibility for the quality assurance of the surface/atmosphere measurements provided and is subject to peer review and rigorous comparison to ensure site-to-site consistency and SI traceability. The archive is continuously updated, and new sites that meet RadCalNet requirements may be added.
Contact: Kurtis Thome or RadCalNet admin.
Land Product Validation Subgroup Protocols
One of the primary objectives of WGCV’s Land Product Validation (LPV) subgroup is the development of best practice protocols for Essential Climate and Biodiversity Variables (ECVs and EBVs). The LPV subgroup has published five protocols thus far. The LPV focus areas on LAI, Land Surface Temperature, Albedo, Soil Moisture, and Aboveground Biomass have all published protocols that can be found on the CEOS Best Practices and Guidelines page (https://ceos.org/ourwork/lessons-learned-best-practices/) as well as the LPV Subgroup website.
The LPV Biomass focus area just had their protocol on Aboveground Woody Biomass products endorsed by CEOS in the Spring of 2021. The focus area enlisted a large, multi-institutional group of contributors for this effort. In addition to the protocol effort, the Biomass focus area leads are engaging with the ESA BIOMASS and NASA GEDI missions, CCI Biomass, GFOI, and the CEOS Working Group on Climate, to be inclusive of all stakeholders.
The LPV subgroup is also conducting an activity to identify land product validation supersites that could fulfill the needs of multiple products within their focus areas (https://lpvs.gsfc.nasa.gov/LPV_Supersites/LPVsites.html).
For more information contact the LPV Subgroup Chair, Michael Cosh.