Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

The following is a list of prototype converters that translate metadata between CDIF and other widely used metadata formats, created with the assistance of Anthropic’s Claude Code. These are intended to demonstrate how common standard metadata formats can be used to generate CDIF-compliant metadata. CDIF is intended to function as an integration layer - a “lingua franca” - between domains and infrastructures, so being able to go to and from common standard formats which may be used in any particular domain is a fundamental approach. These tools show how that can be achieved.

The converters here are not necessarily sufficient for use in a production setting, and should be tested carefully before being deployed. They are all accompanied by sample metadata instances to illustrate what kinds of information are handled. Further, the mappings which are supported can be found in the Simple Standard for Sharing Ontological Mappings (SSSOM) format. They are tested against the example corpora in the repository, but the mappings are still being reviewed and output should be validated before it is published. Feedback and contributions are welcome through the repository’s issue tracker. These converters are intended primarily as a demonstration of what is possible, and as an initial step in building a production solution.

The conversion tools live in the CDIF converters repository on GitHub. They are Python command-line tools; most also expose their conversion as a function that can be imported into other code. All of the tools are licensed under the Apache License Version 2.0. The SSSOM mappings are licensed under Creative Commons Attribution 4.0 International (CC BY 4.0).

Converters at a glance

Science-on-Schema.org (SOSO)

SOSO and CDIF are both schema.org profiles, so this conversion is structural alignment rather than vocabulary translation. The converters are plain Python with no third-party dependencies, and the correspondences are coded directly. The SSSOM tables cdif-to-soso and soso-to-cdif document the mapping rather than drive it; a drift-checker (soso/check_soso_mappings.py) runs in CI to confirm that the tables and the code still agree on the example records. Properties with no counterpart are passed through unchanged, since both sides are open-world.

MLCommons Croissant

Croissant describes machine-learning datasets with nested RecordSet and Field objects. Both converters read their SSSOM table at runtime through the shared sssom_engine.py. The table records which Croissant term maps to which CDIF term and on which class (sc:name means different things on a dataset, a file and a field); a short set of Python shapers, named in the table’s transform column, builds the nested structures. An aliases table normalises variant source IRIs. Because much of the structural work is still in code, a drift-checker (croissant/check_croissant_mappings.py) also runs in CI.

FAIR²

A FAIR² data package (a fair2.json file) is MLCommons Croissant plus an extension namespace, fair2:, for data articles, data archives and portals, method sections and steps, field units and descriptive statistics, contributor roles and changelogs. The converter therefore runs in two passes. The Croissant core goes through the Croissant converter above; what FAIR² adds goes through its own table, fair2-to-cdif.sssom.tsv, read at runtime through sssom_engine.py. A property the FAIR² table claims is withheld from the Croissant pass, so its row decides the output. Method sections and changelog entries become prov:wasGeneratedBy activities following the CDIF provenance pattern; data articles, archives and portals become related links. FAIR² properties with no CDIF term (the full descriptive statistics, isExperimentRelated, citationKey) are kept under the fair2: namespace rather than dropped. A second table, fair2-aliases.sssom.tsv, maps the terms used by older FAIR² exports onto the current ones. The mapping rows are tool-suggested and still under curator review. Questions about the FAIR² specification that came up while building the mapping are in fair-squared/fair2-spec#7.

W3C DCAT

DCAT metadata is typically serialized as a graph: datasets, distributions, agents and locations are separate nodes linked by identifiers, so the converter indexes every node and follows those links before mapping. It is fully table-driven: dcat-to-cdif.sssom.tsv is read at runtime, with a subject_class column saying which class each property belongs to and a transform column naming the shaper for structured values such as agents, qualified attributions, spatial and temporal coverage, and distributions. A second table, dcat-aliases.sssom.tsv, maps IRIs that publishers use by mistake to the ones they meant. Unmapped DCAT properties are preserved in the output. Relations between resources become typed schema:LinkRole related links. A DCAT contact point that names no one — only a web page, such as a contact form or an issue tracker — becomes a related link too, rather than an invented person.

NASA Common Metadata Repository (CMR) Unified Metadata Model-Collections (UMM-C)

UMM-C is the JSON collection metadata model of NASA’s Common Metadata Repository (CMR). The converter follows the DCAT pattern: it reads ummc-to-cdif.sssom.tsv at runtime, where each row gives a UMM-C JSON path, the CDIF target, and a transform naming one of the converter’s shaper functions (for contacts, GCMD keywords, spatial geometries, temporal extents, related URLs and so on). Row order sets precedence. Related URLs that match no mapping row are kept under a ummc: namespace rather than dropped. The mapping rows are tool-suggested and still under curator review.

DDI Codebook 1.2.2 and 2.5

DDI Codebook 2.5 is an element-level superset of 1.2.2, so one data-driven engine handles both. The SSSOM worksheets list every literal-valued element in the DDI Codebook XML Schemas, whether or not CDIF has a target for it, factored into a common table plus version-specific extras. mappings/sync_ddi_mappings.py checks the worksheets against the XSDs and compiles the mapped rows into ddi_mappings.json, which the engine applies. Shapering code builds what a table cannot express, such as variable value domains, shared code lists and summary statistics. Study metadata becomes the CDIF dataset, variables become cdi:InstanceVariable, and data files become distributions; a file’s type goes to schema:encodingFormat when it is a MIME type, and its case and variable counts become schema:additionalProperty values.

DDI-CDI

A DDI-CDI instance is a flat list of typed objects linked by references (basically a graph), with associations represented as objects of their own. The converters index every object by its identifier, resolve the references to rebuild the graph, and then down-shift the verbose DDI-CDI model to the simpler CDIF profile form, following the class and attribute crosswalk maintained in the CDIF ucmism2m project. They are claude-coded. The XML converter was built in six phases: dataset skeleton, value domains and code lists, data structure, physical mappings, provenance, and discovery enrichment. DDI-CDI code lists become CDIF code lists (skos:ConceptScheme) identified by the code list’s DDI URN, and a primary key becomes a cdif:Key that lists its variables in order.

RO-Crate 1.2

RO-Crate and CDIF both use schema.org, but RO-Crate is a flat @graph with unprefixed terms, while CDIF is a nested tree with prefixed terms. These converters use no mapping table: they are pipelines of standard JSON-LD operations (expand, flatten, compact, frame) built on the pyld library, followed by small fix-ups for each format’s conventions.

OHDSI gaiaCatalog

The OHDSI gaiaCatalog, published by the University of Miami Geospatial Digital Special Collections, describes geospatial datasets for health research as schema.org records. Because the source is already schema.org, most of the mapping is property for property, and the converter is hand-coded. The interesting parts are structural: a polygon footprint becomes a bounding box, distribution URLs become downloads with a media type, and each record’s ETL step (the tool and command that produced the data) becomes a prov:wasGeneratedBy activity. The mapping is documented in ohdsi-to-cdif.sssom.tsv, and a drift-checker (OHDSI/check_ohdsi_mappings.py) runs in CI to confirm the table and the converter agree on the corpus.

Harvesting: EarthCube GeoCodes

harvesters/geocodes_harvester.py queries the EarthCube GeoCodes SPARQL catalogue, fetches each dataset’s schema.org JSON-LD from its landing page, and converts it to CDIF core or discovery with the same naming step as the SOSO converter. It differs from the converters above in where its input comes from rather than in what it produces.

How the converters are implemented

A common target shape

Every converter that produces CDIF writes JSON-LD built on schema.org, rooted at a schema:Dataset, with schema:-prefixed property names and a schema:subjectOf catalog record that carries the metadata about the metadata. This is the framed tree that the CDIF JSON Schemas validate. The source formats differ from it mainly in structure: SOSO uses bare schema.org terms and has no catalog record; Croissant nests RecordSet and Field; DCAT is a graph linked by identifiers; DDI uses its own XML vocabularies. Each converter reconciles those structural differences as well as translating terms.

Mapping tables plus shapers

The property-level mappings for each converter are recorded as SSSOM mapping sets in the repository’s mappings/ directory, one per source → target direction. An SSSOM table is a flat list of term-to-term correspondences. The project extends it with a few columns: the class the source property sits on, where the value lands in the CDIF record, and a named transform.

A flat table cannot express all the structural work a conversion needs, such as resolving references between graph nodes, collapsing reified associations, reshaping code lists into skos:ConceptScheme, or creating the catalog record. The converters therefore follow a hybrid pattern:

The DCAT, UMM-C, Croissant, FAIR² and DDI Codebook converters read their table at runtime, so editing the table changes the conversion and the table cannot drift away from the code. For DDI, the worksheets are compiled into a single ddi_mappings.json that the converter reads. The SOSO and OHDSI tables document hand-coded converters rather than drive them, and parts of the Croissant ones do the same. For those three, automated drift-checkers compare the tables with each converter’s actual behaviour on its example corpus and fail the repository’s CI when they disagree.

Declaring conformance from content

A CDIF record declares the profiles it conforms to with dcterms:conformsTo in its catalog record. Rather than hard-coding that list, the converters to CDIF test each record’s actual content using the detect_conformance.py module from the CDIF validation repository: detect_cdif_conformance() (formerly detect_conformance()) works out which profiles the record meets, and apply_conformance() writes them into the catalog record’s dcterms:conformsTo. For each CDIF profile it runs a SPARQL check for the elements the profile introduces, followed by a SHACL validity check. Only the profiles the content actually meets are declared. A record that meets none declares none, because a false claim would select schemas and shapes the record cannot satisfy. A --static-conformance option keeps a converter’s built-in default list instead.

A profile is identified by use of what it adds. Discovery, for example, is declared only when a record uses one of the properties the discovery profile introduces beyond core: schema:measurementTechnique, schema:variableMeasured, schema:spatialCoverage, schema:temporalCoverage or dqv:hasQualityMeasurement. A record with a name, identifier and description but none of these is a core record. Most of the DCAT examples are core records by this rule: of the 289 in the DCAT corpus, 47 also declare discovery.

Validation

Most converters can validate their output (--validate), framing the JSON-LD and checking it against the CDIF JSON Schemas. The validation repository’s FrameAndValidate.py tool does the same for any CDIF record. Its ConformanceValidate.py checks a record against every profile it declares, with both the profile’s JSON Schema and its SHACL shapes, and fails a record that declares a profile its content does not support. The two tools share one framing step, so they validate the same form of a record. Croissant output can be checked with the MLCommons mlcroissant validator, and SOSO output with the SOSO SHACL shapes.

What is in the repository

PathContents
soso2cdif.pyFront end for SOSO → CDIF from a file or URL
sssom_engine.pyShared engine that applies an SSSOM mapping table to a document (used by the Croissant and FAIR² converters)
schemaorg_names.pyShared step that resolves the unprefixed names in a schema.org record through its own @context (used by the SOSO converter and the GeoCodes harvester)
soso/CDIF ↔ SOSO converters, the SOSO mapping drift-checker and an example
croissant/CDIF ↔ Croissant converters, the mapping documents, a drift-checker, and example corpora (Croissant exports harvested from Dataverse and from MLCommons) with their CDIF conversions
FAIR2/The FAIR² converter, the FAIR² specification’s example package and its CDIF conversion
DCAT/The DCAT converter, a regression harness, a profile-coverage workbook, a DCAT-AP vs DCAT-US comparison, and a corpus of about 780 upstream DCAT examples. The 238 that describe a dcat:Dataset, plus the PSDI Resource Catalogue, are converted in cdifOK/ (289 records)
UMM/The UMM-C converter, the UMM-C and UMM-Var JSON Schemas it is written against, and example CMR records with their CDIF conversions
DDI/The DDI flavour dispatcher, the data-driven DDI Codebook engine, the DDI 1.2.2 and Harvard Dataverse converters, the DDI 1.2.2 schema, and examples
DDICodebook/The DDI Codebook 2.5 converter, the 2.5 schema, notes on the 2.5 additions, and examples
DDI-CDI/The DDI-CDI XML and JSON-LD converters, the RepresentedVariable extractor for the CODATA variable index, and examples
ROCrate/The RO-Crate converters and validator, with example RO-Crate and CDIF records
OHDSI/The OHDSI gaiaCatalog converter, its mapping drift-checker, and its 16 source records with their CDIF conversions
harvesters/The EarthCube GeoCodes harvester
mappings/The SSSOM mapping tables (\*.sssom.tsv) and their metadata sidecars (\*.sssom.yml) for every converter, alias tables, and the scripts that keep the sidecars and the DDI mapping file in sync
validation/A git submodule containing the CDIF validation repository: detect_conformance, the CDIF schemas and frame, and validation tools

Each directory has its own README.md with usage and mapping details.

Getting started

git clone https://github.com/Cross-Domain-Interoperability-Framework/converters.git
cd converters
git submodule update --init      # optional: content-based conformance detection and local schemas
pip install -r requirements.txt

python soso2cdif.py https://example.org/dataset -o out.json
python DCAT/dcat_to_cdif.py catalog.jsonld --output ./out --validate

The validation submodule is optional. Without it the converters still run, but they fall back to a default conformsTo list and --validate is skipped.

Licensing

The converter software is licensed under the Apache License 2.0. The mapping tables, documentation and converted example metadata are licensed under CC BY 4.0. Third-party examples and schemas bundled for testing keep their original licenses.

We appreciate constructive feedback. Contact us at cdif-feedback@codata.org or file a GitHub Issue.

Copyright (c) 2022-2026 Committee on Data of the International Science Council (CODATA). The Cross Domain Interoperability Framework (CDIF) is licenced under CC-BY-4.0.

Funded by the EU

CDIF v.1 was supported by the EU Horizon Europe funded WorldFAIR project (GA 10105839). CDIF v.1.1 is is a community effort coordinated by CODATA. Further development will be supported by the EU Horizon Europe funded CDIF4EOSC project (GA 101292473).

Implementation Resources
Mapping from Signposting Relations to CDIF Metadata Elements