Public comment requested: Zarr Community Standard Work Item Justification
The Open Geospatial Consortium (OGC) is considering the Zarr v2 Storage Specification for adoption as an official OGC Community Standard. A new Work Item justification to begin the Community Standard endorsement process is available for public comment.
Zarr is an open-source specification for the storage of multi-dimensional arrays of data (also known as N-dimensional arrays, ND-arrays, or tensors). Such arrays are ubiquitous in scientific research and engineering.
Zarr stores metadata using .json text files and array data as (optionally) compressed binary chunks. Zarr can store data into most storage systems, including databases, standard ‘directory based’ file systems, and cloud object stores, such as Amazon S3. This flexibility allows implementations to experiment with novel storage technologies while maintaining a uniform API for downstream libraries and users.
Zarr arose in genomics research in 2016. It was created by Alistair Miles of Oxford as a library optimized for massively parallel array analytics. It has since grown into a community project with a range of developers and users from fields such as genomics, bioimaging, astronomy, physics, quantitative finance, oceanography, atmospheric science, climate science, and geospatial imaging.
Because it can represent very large array datasets in a simple, scalable way, and is compatible with cloud object storage, Zarr is an ideal format for analysis-ready geospatial data in the cloud. Indeed, Zarr has already been adopted by several OGC communities as a format for cloud-optimized, analysis-ready geospatial data. Examples include:
- Climate Science: The CMIP6 Google Cloud Public Dataset
- Oceanography: The ECCOv4r3 Ocean State Estimate
- Atmospheric Science: Global cloud-resolving aquaplanet simulations with the System for Atmospheric Modeling
While Zarr is not inherently a geospatial-specific format, it has been put forward by the Zarr Steering Council for adoption as an OGC community standard because of its rapid growth and adoption in geospatial and related fields.
An approved OGC Community Standard is an official standard of OGC that is considered to be a widely used, mature specification, but was developed outside of OGC’s standards development and approval process. The originator of the standard brings to OGC a “snapshot” of their work that is then endorsed by OGC membership so that it can become part of the OGC Standards Baseline.
Comments can be submitted to a dedicated email reflector for a thirty day period ending on the "Close request date" listed above, Comments received will be consolidated and reviewed by OGC members for incorporation into the document. Please submit your comments using the following link: requests [at] lists.opengeospatial.org (Click here to submit comments) The link provided above should include a standard template in the message body. If the preloaded message body does not work properly using your mail client, please refer to the following template for the message body: Comments Template