CEOS Development Environment

Architecture for Interoperability: Building the Foundations


Interoperability in Earth observation (EO) must be addressed in a structured and comprehensive manner, bringing together multiple factors, including Vocabulary, Architecture, Interface, Quality and Policy. The CEOS Interoperability Framework provides a common approach for ensuring that EO data and services can be discovered, accessed, combined, and used effectively across different systems and organisations. 

Architecture is one factor of the CEOS Interoperability Framework

Within the CEOS Interoperability Framework, the Architecture factor describes the organisational structure of concepts, processes, and assets, including data. It defines how these elements are arranged and governed across the full data lifecycle, from collection, storage, and archiving to documentation and publication. It provides a common blueprint for designing interoperable EO systems and enables organisations to align how data is managed, shared, and accessed across the ecosystem. Without a coherent architectural approach, interoperability becomes fragmented, with systems requiring custom integrations that limit scalability, increase complexity, and reduce the ability to efficiently combine EO data and services across different platforms.

The CEOS Interoperability Handbook identifies three main architectural components that support interoperability:

  1. Preservation Architecture
  2. Data and Metadata Architecture
  3. Publishing Architecture.

Preservation Architecture focuses on the long-term archiving and stewardship of EO data. Its primary objective is to ensure that data remains accessible and usable both now and in the future. This requires coordination between multiple stakeholders, including data providers, archive managers, and users.

Data and Metadata Architecture defines how EO data is structured, described, and organised. A consistent approach ensures that datasets can be understood, interpreted, and combined across systems without ambiguity, which is essential for interoperability. This component also covers key recommendations for collection management functions, including data production, management, packaging and documentation.

Publishing Architecture governs how data and services are exposed to users and applications. It represents the final stage in making data accessible to external users and organisations. These recommendations support both data access and the replication of collections, including in cloud-based environments, to enable interoperability.

The 25 recommendations across the Preservation, Data and Metadata, and Publishing sub-sections can be found below. CEOS welcomes review and feedback from the community via GitHub.

IDPreservation Recommendation

DPRES#1 
Archival systems should comply with the Reference Model for an Open Archival Information System (OAIS) and with the “OAIS-Interoperability Framework” to facilitate interoperability between archives.
DPRES#2Data should be appraised and properly documented before ingestion in the archives following the CEOS Data Appraisal Procedure.
DPRES#3Data and associated information should be ingested, archived and preserved following internationally recognised standards and best practices (e.g., those produced by WGISS and Producer-Archive Interface Methodology AbstractStandard) with any tailoring documented.
DPRES#4Periodically perform archival system/media upgrade to the most adequate proven technology to ensure data and information long term preservation. Perform migration, with an integrity check, of archived data from old to new systems.
DPRES#5Archive and preserve the information, code and software needed to handle the archived data, following the CEOS guidelines.

DPRES#6
When performing archived data and information repackaging and/or reformatting, for example to comply with new standard formats and/or exchange formats, properly document changes made to the archived data and ensure data integrity.

DPRES#7
Periodically verify the integrity of the archive collection/content through integrity check on a representative set of the archived data.
DPRES#8Manage evolution of archived data collections according to the Shared Collection Lifecycle Management Principles for EO Data best practice.
DPRES#9Keep archives equipment (hardware and software) up-to-date and in conformance with vendor recommendations to preserve data and associated information integrity and facilitate interoperability between archives.
IDData and Metadata Recommendations
DATA#1
CEOS-ARD Framework should be used as a starting point for development of Analysis Ready Data.

DATA#2
CEOS-ARD Product Family Specifications (PFS) should be used for development and assessment of ARD products, including both self-assessments and peer review.

DATA#3
Compliance with CEOS-ARD Product specifications should be performed using the Self assessment Guide and should be followed by peer review process.

DATA#4
The ISO 19115 series of standards (or similar) should be used to produce geospatial metadata.

DATA#5
A Collection of data should have all granules packaged consistently and produced with consistent quality.

DATA#6
Collection-specific metadata formats may be used, but packaging should also include STAC documents at the Collection and Granule/Item level. Refer to the CEOS EO collection and granule discovery best practices with STAC.

DATA#7
Checksums for all files in a packaged granule should be available, to ensure integrity.

DATA#8
Where pixel-level metadata is available, such as scene quality masks, these should be clearly documented with a reference to lookup tables.

DATA#9
File names and folder or path structures should be consistent and include appropriate information to distinguish the specific granule. This could include the platform, time and date of acquisition, band(s), and product version.

DATA#10
Assign a Persistent Identifier to data archived and published to users and ensure the availability of all associated information in the relevant Landing Page following the CEOS Persistent Identifiers Best Practice.

DATA#11
The CEOS supported Open Data Cube family of software can be taken as a reference Datacube implementation.
IDPublishing Recommendations
PUBLISH#1Each data collection that is published as a publicly-accessible product should include a public granule-level notification including for when a granule is added, updated and deleted/archived. This supports management and maintenance of replicas collections.

PUBLISH#2
A collection should have a full listing of all available granules in a standard format, preferably cloud optimized. For example, STAC-geoparquet is used by some providers.

PUBLISH#3
Granule data stored in the cloud should be accessible in cloud-optimized formats, e.g., Zarr or Cloud-Optimized GeoTIFF (COG).

PUBLISH#4
Granules should not be zipped when stored in the cloud, so that cloud optimized data formats can be leveraged.

PUBLISH#5
Where possible, cloud providers’ standard interfaces should be used in preference to self-developed solutions, enabling interoperability of tools that work with that cloud provider.