WGCV undertakes a variety of activities related to the calibration and validation (Cal/Val) of Earth Observation data. Some past and current exercises include:
CEOS-FRM: Pilot exercises for the CEOS Fiducial Reference Measurement (FRM) Assessment Framework
CEOS WGCV is establishing a formal framework to assess and endorse a distinct class of observations, labelled ‘CEOS Fiducial Reference Measurements (CEOS-FRM),’ specifically designed to meet the needs of satellite Cal/Val. CEOS-FRM compliant data enable satellite operators to clearly understand the quality, traceability, and fitness for purpose of Cal/Val data for specific satellite products. A number of in-situ reference networks and sensors have undergone preliminary evaluations against the CEOS-FRM Assessment Framework. These include:
- PGN (Pandonia Global Network)
- RadCalNet (Radiometric Calibration Network)
- ICOS (Integrated Carbon Observation System – Ecosystem Component)
- FRM4DOAS (Ground-Based DOAS Air-Quality Observations)
- BAQUNIN (Boundary-layer Air Quality-analysis Using Network of Instruments Super Site)
- DION (Darkwater Inland Observatory Network)
- Brewer Spectrometer – NO2 Vertical Column Densities (VCDs) Measurements
- FRM4GHG 2.0 project – COCCON
- CREGARS (Centre for Reactive Trace Gases Remote Sensing)
SRIX4Veg: Surface Reflectance Intercomparison Exercise for Vegetation
SRIX4Veg represents a joint effort between ESA, CSIRO and Geoscience Australia to test user-based differences in UAV-based surface reflectance fiducial reference measurements used for validation of surface reflectance satellite products, and to ensure consensus on UAV-based surface reflectance validation good practice protocols, which is conducted in the framework of the ESA FRM4Veg initiative.
The objectives of SRIX4Veg are:
- Testing user-based differences in UAV-based surface reflectance measurements, including instrument and operator biases as well as measurement collection procedures.
- Helping design field measurement protocols and validation methodologies that are clear and can be easily applied by all users.
- Ensuring international buy-in and consensus on the UAV-based surface reflectance validation protocols.
- Publish UAV-based surface reflectance inter-comparison results.
One of the key outputs of the SRIX4VEG campaigns and workshops was the development of a community-agreed best practice protocol for UAV-based surface reflectance validation. Version 1 of the protocol has now been endorsed by WGCV and can be accessed here: https://ceos.org/document_management/Working_Groups/WGCV/GPG_UAV_SR_CEOS_version1.pdf.
Emphasis was placed on adopting meteorological practices and aligning with the CEOS Fiducial Reference Measurements (CEOS-FRM) concept. Particular attention was devoted to assessing the uncertainty budget associated with these measurements, ensuring their suitability for validating satellite-based surface reflectance products. The protocol is aimed at the users of UAV-mounted instruments capable of validating surface reflectance products and those wishing to enter this field.
The FRM4Veg team would like to take the opportunity to express their gratitude to the community for their contribution towards the document process and their enthusiastic participation in the SRIX4Veg campaigns in Barrax (Spain) in 2022 and Calperum (Australia) in 2024.
Please visit the SRIX4Veg website for more details: https://frm4veg.org/srix4veg/
DEMIX: Digital Elevation Model Intercomparison eXercise (DEMIX) Working Group
Established in 2020, the DEMIX working group is developing qualitative and quantitative methods to provide DEM users with open tools and data to compare and select the most appropriate digital elevation model for their application. The working group has been divided into three subgroups to deal with the variety of issues stemming from implementing such a feat for global products.
For a brief overview of the project please refer to the ISPRS2021 conference paper or watch this 10 minute introductory video. Further detail about the work in the subgroups can be found below.
A final report is currently being prepared, alongside follow-on activities to continue criteria development, refine benchmarking methods, and provide access to data, tools, and results.
DEMIX Subgroup 1 – Terminology and Analytical Basis
Chairs: Peter Guth (pguth@usna.edu), Igor Florinsky (iflor@mail.ru)
Subgroup 1 is in charge of a common terminology and analytical basis to facilitate cooperation of experts from different domains. It is doing so by reviewing existing terminology and its use in literature in order to propose a consistent set of terms which also matches with the analytical basis of geomorphometry. Its task is furthermore to review the methodological developments in SG2 in order to ensure their coherence and consistency with the agreed set of definitions.
The subgroup has published a peer-reviewed paper in support of its mandate, available at https://doi.org/10.3390/rs13183581.
DEMIX Subgroup 2 – Algorithms and Software
Chairs: Conrad Bielski (conrad.bielski@eoxplore.com), Carlos Grohmann (guano@usp.br)
The DEMIX subgroup 2 is concentrating their efforts on identifying methods and implementations for the quantitative and/or qualitative intercomparison of DEM products. While there is a significant body of scientific literature covering a dizzying number of quality characteristics, the majority have focused on very local products and/or special cases. The goal of DEMIX is to provide options to end-users that can help guide them in deciding the most appropriate DEM product for their application and support geospatial practitioners defend their choices.
With the support and feedback from subgroup 1, the methods proposed will be implemented by subgroup 3 in a manner to promote open science, communication and discovery of DEM product quality fit for purpose.
DEMIX Subgroup 3 – Platforms and Processing
Chair: Serge Riazanoff (serge.riazanoff@visioterra.fr)
Subgroup 3 concentrates on applying the methodological framework developed within the other subgroups. Specifically, it aims at making available to a wider audience tools for the visualisation of the results of the inter-comparison of DEMs. This will be achieved by providing preferably open-source tools and performance tests in a variety of geographical regions around the world in order to demonstrate and verify the applicability of the DEM inter-comparison approach.
Furthermore, subgroup 3 will help to define the minimum standards and requirements regarding user access, data availability, programming interfaces and processing power, to which open candidate platforms should comply in order to host DEM benchmarking activities. Finally, this subgroup will be in charge of implementing and monitoring DEM benchmarking and comparison at global scale.
BRIX-2: Second Biomass Retrieval Intercomparison eXercise
BRIX-2 represents a joint effort between ESA and NASA to intercompare algorithms specifically for biomass mapping using current and future spaceborne missions.
The objectives of BRIX-2 are:
- Provide an objective, standardized comparison and assessment of biomass retrieval algorithms developed for the Biomass, NISAR and GEDI missions, and fusion of these mission datasets.
- Establish a forum to involve scientists in the development of retrievals that have so far not been part of the biomass community.
- The adoption of vetted validation standards and methods to compare biomass estimates to reference datasets (e.g., field plots or airborne lidar biomass maps).
- Collect inputs from the biomass user and scientific community on data formats and characteristics towards the generation of Analysis Ready Data.
These objectives will be achieved by making available standardized test cases (based on airborne campaign and spaceborne simulated data), inviting the scientific community to develop and apply retrieval algorithms based on these test cases, and finally comparing and evaluating the performance of submitted results.
The first BRIX-2 workshop was held online, from 29-30 April 2021.
Please visit the BRIX-2 website for more details: http://polinsar-biomass2021.esa.int/brix-2/
ACIX: Atmospheric Correction Intercomparison eXercise
Conducted within WGCV, the Atmospheric Correction Inter-Comparison eXercise (ACIX) aims to address challenges of atmospheric correction with a comprehensive assessment of algorithms for multispectral and hyperspectral sensors. The activity began in 2016 and concluded its third exercise in 2025, ACIX-III, which assessed the performance of algorithms for the EnMAP and PRISMA hyperspectral missions over both land and water surfaces.
ACIX-III provides a rigorous and comprehensive benchmark for validating and improving AC algorithms and informs users about their strengths and weaknesses. The two separate studies, ACIX-III Land and ACIX-III Aqua, intercompared in-situ and spaceborne hyperspectral data over land, inland and coastal waters validation test sites around the globe.
Read more: ACIX-III Land paper | ACIX-III Aqua Paper
Cloud Masking Inter-comparison eXercise (CMIX)
CMIX is an international collaborative effort aimed at inter-comparing cloud detection algorithms for medium-spatial resolution (10-30 m) spaceborne optical sensors. The focus of CMIX is on open and free imagery acquired by the Landsat 8 (NASA/USGS), Sentinel-2 (ESA), and PRIMSA (ASI) missions.
View the findings of the first CMIX (2022) here.
Ground Control Point Intercomparison eXercise (GCPIX)
Established in 2025 under the WGCV Terrain Mapping Subgroup (TMSG), GCPIX aims to:
- Clarify pertinent terminology
- Define assessment criteria of individual GCPs and ensembles
- Establish threshold requirements to set standards or good practices for selection of GCPs in reference databases for specific categories
- Establish protocols and formats for documenting and sharing GCPs and respective libraries
- Harmonization of existing sources from the different CEOS agencies towards a unified database
Global Reference Grid Intercomparison eXercise (GRGIX)
GRGIX is an exercise established in 2025 by WGCV TMSG to:
- Clarify various ‘grid’ concepts and terminology
- Develop a comprehensive taxonomy of global grids
- Develop specifications and metrics for grid characterisation
- Establish a database of major used or conceptualised global grid (systems) and apply the proposed taxonomy
- Identify pertinent use cases requiring global grid solutions
- Select representative grid and grid system candidates for testing
- Perform a systematic grid system intercomparison and benchmarking
- Document, report, and publish findings
Read more on about WGCV activities and exercises on the Cal/Val Portal