FAIR Data Train — General Architecture

1. Introduction

The FAIR Data Train (FDT) is a FAIR-based platform framework that aims at guaranteeing a specific level of interoperability among its participant elements. It defines a set of application types and their expected behaviours to support the findability, accessibility, interoperability and reusability of data following the FAIR principles [FAIR-PRINCIPLES]. The FDT follows the analogy of a train system with two main elements, stations and trains. Data Stations make data (or other kinds of digital objects) available and publish metadata about themselves and about their content. Trains represent analysis or processing algorithms that are sent to Data Stations to process the data where it is: the computation travels to the data, and the data stays. A registry indexes the metadata the stations publish, so that a client can find out which station holds which data before sending anything anywhere.

In this document, data means not only the content of artefacts such as databases, tables and graphs, but also other kinds of digital objects such as controlled vocabularies, ontologies and models; unless otherwise stated the terms are used interchangeably. The FAIR Data Train architecture defines a number of key capabilities:

• provisioning of metadata to describe digital objects;

• a common representation format [RDF11-CONCEPTS] to express the metadata in a machine-actionable manner;

• a common approach to inform clients on how to navigate through the metadata structure;

• a common representation format [SHACL] for the metadata schemas — and, since version 0.2, for what a train requires of a station’s data and what a station’s data conforms to, which is how a station is selected for a train;

• a common way of describing how data sources support interactions with their content, in terms of metadata properties of the station;

• a common way to represent how client applications interact with data: the interfaces and protocols by which a station accepts an incoming train, negotiates the terms under which it may run, runs it, and decides what may leave.

The main goal of the FDT general architecture is to define a set of behaviours, interfaces and protocols to improve interoperability among data sources and data processing services. To fulfil this goal, the FDT specifications help developers build new applications, or extend existing ones, so that they can take part in the FAIR Data Train ecosystem. The envisioned scenario is one with a multitude of trains, stations and client applications, independently created and able to interact with one another because they all follow the same base guidelines — interfaces, protocols and representation formats.

1.1. Purpose

The purpose of this document is to present the general architecture of the FAIR Data Train: its roles, its components, how they interact and the standards they rest on. It is primarily intended as a reference for developers willing to add FDT functionality to their existing applications, or to develop their own implementation of an FDT component. Knowledge of RDF, SHACL, ODRL and REST APIs is required to follow it.

This document is the entry point to a set of specifications. Where a part of the architecture is specified in full elsewhere, this document says what the part is for and points at the specification that governs it. In version 0.2 that is one document: the FDT Visit Protocol [FDT-VISIT-PROTOCOL], which specifies the conversation between a Data Station and a Train Handler for one visit of one train.

1.2. How this document relates to the reference implementation

This section is non-normative.

A reference implementation of the ecosystem exists [FDT-COMMONS] and this version of the architecture was written against it: every component in § 3 Components has a running implementation — of the two services of § 3.6 Identifier translation and vocabulary alignment, identifier translation runs and vocabulary alignment does not yet — the interactions of § 4 Interactions are exercised end to end by its test suites and testbed, and the normative artefacts of the Visit Protocol — JSON Schemas, OpenAPI documents, SHACL shapes and the fixtures every component is tested against — are the contracts that implementation is built on. The architectural decisions behind the text are recorded as Architecture Decision Records [FDT-ADR] and are cited here by number (ADR-nnn) where a reader may want the reasoning rather than the result.

1.3. Changes since version 0.1

This section is non-normative.

Version 0.1 (10 July 2025) described the FDT as it was first conceived. Version 0.2 brings it up to date with what has since been built, and the following names and ideas changed:

The three interface groups of the Data Station are unchanged in name; § 3.1 Data Station now says what each one is.

2. Roles

The parties of an FDT ecosystem, and what each is answerable for. A party may play several roles; a role may be played by an organisation or, where the deployment admits it, by a natural person.

3. Components

Trains are packaged, executable interactions of four families — query, API, script and container — and are assets, not datasets: a train is something a station runs, not something it hosts (ADR-020). A train has an identity and an offer of its own, carries a payload with a digest, and declares the data it needs (as a SHACL model) and the output it produces (as a JSON Schema).

3.1. Data Station

Since the purpose of the FAIR Data Train architecture is to define a set of desired behaviours that applications expose and support, the definition of the Data Station is the definition of its interface. The interface is divided into three groups, as depicted in the figure below: the Metadata Interface, the Station Interface and the Content Interaction Interface. Each group exposes the interfaces of a number of services the station provides, and the interaction with the data happens only through the station’s interaction component, which is the one component that connects to the data storage.

A station hosts digital objects of any kind — "data" in the sense the FAIR principles give the word. Hosted datasets and their parts are the common case; a service that translates identifiers between pseudonym spaces or aligns vocabularies is hosted the same way, described in the same catalogue with the interaction mechanism it is reached by, and visited by a train under the same protocol and its own agreement (§ 3.6 Identifier translation and vocabulary alignment).

3.1.1. Metadata Interface

The Metadata Interface publishes what the station holds and on whose terms, as RDF, following the FAIR Data Point specification [FDP] for the metadata structure and its navigation. A station publishes:

• a catalogue [VOCAB-DCAT-3] of the hosted datasets it serves and of the dataset parts a controller offers in their own right, each with its distributions per interaction mechanism, its dcat:theme, and the ODRL offer [ODRL-MODEL] that states the controller’s access conditions over it;

• for each dataset or part, what its data conforms to: a SHACL shape [SHACL], served by the station alongside the catalogue. This is what makes station selection answerable — a train declares the data it needs as a SHACL model, and a station matches when the shapes it publishes cover that model (ADR-030). Where the station’s data is not natively RDF, the shape is generated from the mapping the dataset declares [RML] and the mapping is published too.

Publishing a catalogue entry grants nothing. A catalogue says what exists and under which terms; every access is still negotiated on arrival against the offer, per visit.

3.1.2. Station Interface

The Station Interface publishes the station’s self-description: its type and the interaction mechanisms it supports, the networks it is a member of and the governance authority of each, the dispatch modes it accepts (push, poll, or both), the protocol versions it implements, and links to its metadata and catalogue. The self-description is what a Train Handler reads to know whether a station will accept a train from a given network and how to reach it: the station’s dispatch endpoint is a dcat:DataService whose dct:conformsTo names the Visit Protocol version.

3.1.3. Content Interaction Interface

The Content Interaction Interface is the Visit Protocol [FDT-VISIT-PROTOCOL]: the conversation between a Train Handler and a Data Station for one visit — one train, at one station, under one agreement. It covers dispatch, negotiation on arrival, the signatures of both parties, every checkpoint decision as an event, execution progress, result delivery and the hand-over of onward state. It is deliberately the only interface a station has to implement to take part in any itinerary pattern. A Data Station conforms to this architecture by conforming to that protocol as a station.

Behind the interface a station enforces policy at three checkpoints, with negotiation between the first two:

3.2. Train Handler

The Train Handler is the consumer’s agent. It resolves a train from a Train Depot, asks the registry which stations hold the data the train needs, composes a plan — the train, its parameter values, an itinerary and the consumer’s purpose as an ODRL request template — and runs it: one run is one execution of a plan, decomposed into visits, each one train execution at one station under one agreement. The Handler dispatches each visit by push or, for a station that accepts no inbound connection, by letting the station poll for it; follows the station’s event stream; signs the assignee’s side of each agreement with the train owner’s key; applies the plan’s failure policy to each outcome; and assembles the run record and the completeness statement — for every intended station, its outcome in the protocol’s own words, its cause and the station’s stated reason.

The Handler holds **Aggregate-class state only** (ADR-015): counts, aggregates, next-hop lists and the names of hand-overs. Pseudonym-class state travels only from station to station, under an agreement; RecordLevel state never leaves the station that hosts it. The rule follows from the Handler’s position — it is a party to no agreement, and the Data Governance Act and the EHDS give an intermediary that holds data a position this architecture keeps the Handler out of — and not from any claim that what it holds is harmless: an aggregate can be personal, which is what the k-anonymity duties of result inspection exist for. This document therefore describes what may reach the Handler by state class and never as non-personal.

3.3. Individual Gateway

The Individual Gateway is one controller’s agent across every station hosting data they control. Through it a controller authors access conditions once and publishes them to each station, reviews the agreements concluded over their data and the slice of each station’s audit trail that touched it, receives approval requests where a person must decide and decides them, and exercises revocation and opt-out. A controller may also act directly at a station’s own console; the Gateway is what makes the same controller’s view coherent across stations.

3.4. FDT Registry

The registry harvests the self-descriptions and catalogues that stations and depots publish and indexes them: stations by interaction mechanism, network, theme, the shapes their data conforms to and the join keys they offer; trains by what they require. Its central question is which stations hold the data this train needs, answered by structural coverage of the train’s data requirement by a station’s published shapes — and, for a near miss, by naming the property a station lacks. A registry is an index: nothing may be granted on the strength of an entry, a station may be indexed by several registries, and a Handler acts on what a station publishes, not on what a registry harvested.

3.5. Train Depot

The Train Depot is the authority for a train. It holds the payload bytes and their digest, the train’s declared parameters, data requirement and output, the creator’s licence over the train as an ODRL offer, and the creator’s public signing keys; it publishes them as a catalogue of train offers. A station resolves a train at the Depot before anything runs and verifies that the payload it was handed is the one the Depot describes. Taking a train from a Depot is itself a two-party agreement — the creator’s licence, signed by the creator’s Gateway and by the would-be owner — and a Depot signs nothing in anybody’s name.

3.6. Identifier translation and vocabulary alignment

A multi-station analysis across organisational boundaries needs two services that no single data-holding station provides: translating identifiers between pseudonym spaces, and aligning vocabularies. Both are hosted by Data Stations like any other digital object and reached by the same protocol under their own agreements; neither is a component of the architecture.

Identifier translation. A station whose interaction mechanism is identifier translation (fdt-inst:IdentifierTranslation) receives a list of pseudonyms in one organisation’s space and returns the corresponding list in another’s — one side only, never the pair, and in an order that discloses no correspondence — so that two stations can be joined at the level of the individual by nobody but the trusted third party that operates it. The station declares the pseudonym spaces and join keys it serves in its catalogue, and an existing trusted third party is wrapped as such a station.

Vocabulary alignment. A station hosting mappings between code systems, units, spatial and administrative divisions, which a train consults or visits as a hop; stations declare per dataset the vocabularies they use, so a Handler can check at planning time that two stations' data can be put together.

The FDT ontology carries fdt-o:LinkageStation and fdt-o:AlignmentStation as subclasses of fdt-o:DataStation: labels a station may add to its self-description so that a registry finds it by what it hosts. They introduce no interface and no requirement that the mechanism declaration does not already carry.

3.7. Networks and governance

An FDT network is a set of participants that accept one governance authority’s rules and trust anchors. A station frequently serves several communities at once, so a station may belong to several networks with different governance (ADR-017). The station’s self-description lists its memberships; policy is evaluated per network, because a controller’s offer may differ per network; every agreement records the network it was concluded under; and identity is issued by network authorities, not by the FDT — in version 1 through federated OpenID Connect providers the network trusts [OPENID-CONNECT-CORE].

A network also states whether a station in it may conclude a negotiation without a person. Where it may not, the station still evaluates the request and produces a recommendation with its evidence, and a person decides; where a machine may not grant, it may not refuse either (ADR-032).

3.8. Shared contracts

Interoperability rests on shared contracts, not on shared code. fdt-commons [FDT-COMMONS] holds the FDT ontology [FDT-O] and the ODRL and run-description profiles; the SHACL shapes that every published document and every arriving request is validated against; the JSON Schemas and OpenAPI documents of the Visit Protocol and of the other component interfaces; the state machines and their mapping to the Dataspace Protocol [DSP]; and the fixtures every implementation is tested against, including counter-examples that must fail. No component redefines a term, shape, schema or endpoint that exists there, and a change to a contract is made there first.

4. Interactions

4.1. Plan, run, visit

A plan names a train, its parameter values, an itinerary — the pattern, the entry stations or a registry query that selects them, phases and dependencies, stop conditions, budget — and the consumer’s purpose as an ODRL request template. A run is one execution of a plan. A visit is one train execution at one station under one agreement. Every itinerary pattern — a single visit, fan-out over many stations, a sequence, a discovery chain that learns where to go next from what it finds, iterative rounds, two-phase selection, composition — decomposes into visits, which is why the Visit Protocol is the only interface a station needs.

4.2. Agreements, per visit, signed by both parties

Every visit — including a visit for identifier translation or vocabulary alignment — is governed by its own ODRL agreement, negotiated on arrival (ADR-014). The station matches the train’s request against the controller’s offer for the visit’s network and the owner’s policy and derives the agreement: it grants exactly the actions requested that the offer permits, carries every usage constraint bounded by what was offered, carries the offer’s prohibitions and duties verbatim, records the eligibility facts it established as evidence rather than as live terms, and pins the offer, the request and the train’s payload by digest. There is no counter-offer: narrowing is always permitted and widening never is.

The agreement is then signed by both parties over the same canonical form: the station signs the assigner’s side — with the controller’s key where they hold one, otherwise as the controller’s agent, and the signature says which — and the Train Handler signs the assignee’s side with the train owner’s key. An agreement only one party has signed is not an agreement, and nothing runs under one. Both parties keep a pinned copy; neither can rewrite history unnoticed by the other, and a third party holding the document can verify it without asking either.

4.3. Dispatch, checkpoints and outcomes

A visit is dispatched by push — the Handler posts a visit descriptor to the station — or by poll — a station that accepts no inbound connection fetches the descriptors waiting for it from the Handler. Beyond the gateway the two are processed identically (ADR-004). The station then runs the visit through its checkpoints (§ 3.1.3 Content Interaction Interface), and every decision it takes is an event with a justification (ADR-027): the ordered event stream is the source of truth for what happened to a visit, for the Handler’s view of the run and for the audit trail a controller reads.

Every visit ends in exactly one of seven outcomes, and the words are part of the contract (ADR-026), because a completeness statement reports them to a data consumer verbatim:

These are not shades of one fact. Reporting a controller’s refusal as a rejection tells a consumer their request failed a check when a person declined it; reporting a timeout as either puts a verdict in the mouth of somebody who never gave one. Implementations keep the vocabulary exact in code, logs and user interfaces.

4.4. State between hops

Three mechanisms carry state from one visit to the next, chosen per plan and constrained by the stations' policies (ADR-016): inside the train — a container or script train carries state, and the origin station’s PEP 3 judges what may leave with it; via the Handler — only Aggregate-class state returns to the Handler and is re-injected as parameters; via an identifier-translation hop — only opaque pseudonyms cross an organisational boundary, translated at a station a trusted third party operates and handed to the next station by location and digest, never by value, so that the Handler holds a name and the bytes stay at the station that produced them.

5. Standards

This section is non-normative.

The FDT is built on open standards, and this table says how each is used — used where the reference implementation reads or writes it as specified, aligned where the FDT is designed to map onto it, declared where the architecture names it and the implementation does not yet exercise it.

6. Conformance

6.1. Conformance classes

This document defines conformance for the two components every itinerary needs, and states what each must do beyond the Visit Protocol. The key words MUST, MUST NOT, SHOULD and MAY are those of [RFC2119]. The protocol-level requirements — dispatch, negotiation, signatures, checkpoints, events, envelopes, hand-over — are the Visit Protocol’s and are collected as numbered criteria there [FDT-VISIT-PROTOCOL]; the criteria below are the architecture’s own and are numbered apart so that a conformance claim can cite both without ambiguity.

Conforming Data Station

A component that conforms to the Visit Protocol as a push-dispatch station or as a poll-dispatch station, and meets every criterion of § 6.3 Criteria for a Data Station — the last four of them where it hosts an identifier-translation or alignment service.

Conforming Train Handler

A component that conforms to the Visit Protocol as a Train Handler, and meets every criterion of § 6.4 Criteria for a Train Handler.

Conformance for the Individual Gateway, the FDT Registry and the Train Depot will be specified in their own documents as those are published; their interfaces exist today only as OpenAPI documents and fixtures in fdt-commons [FDT-COMMONS], and a claim about them is a claim against those artefacts.

6.2. How the criteria are checked

The criteria below are checked in the three ways the Visit Protocol defines: by schema — a SHACL shape or JSON Schema in fdt-commons decides it, and the fixtures named are what an implementation must accept and the counter-examples it must refuse, each with the message the shape carries; by rule — a checker in fdt-commons/tools executes it over the fixtures named; or by harness — a behaviour observable only against a live counterpart, for which no third-party-runnable harness is yet published and the reference implementation’s test suites are the only published evidence. The artefacts are those pinned by the Visit Protocol’s Appendix B.

6.3. Criteria for a Data Station

6.4. Criteria for a Train Handler

6.5. Conformance statement

An implementation claims conformance to this architecture by stating the class claimed, the Visit Protocol conformance statement it rests on, the fdt-commons version and commit it was validated against, and, for each criterion above, that it accepts the fixtures named and refuses the counter-examples named with their recorded messages, or the evidence it offers where only a harness can decide. The reference implementation is the only implementation that can make a third-party-verified claim today, and this document says so rather than imply otherwise.