Associate files without relevant metadata

group/assign sort files using metadata read out of the files themselves (DICOM tags, by default). Some data that belongs in a session doesn't work that way — e.g. raw list-mode or count-rate files from a Siemens PET scanner, which have no readable patient/session metadata of their own, or live in a completely separate export location to the DICOMs. If those files can still be found some other way (a shared naming convention with the primary DICOMs, in the same folder or elsewhere) and you know which scan/resource they belong to from their path, associate can link them into a session that's already been through group.

It can run at any point after 1. Group files into sessions/scans/resources — on group's own output, on assign's, or later — since it only needs the session's own metadata (read from the primary files' own headers), not resolved project/subject/session IDs. The one real ordering constraint is deidentify: run associate before it if GLOB relies on identifying metadata, as in the example below ({PatientName.family_name}) — deidentify replaces PatientName with a fixed placeholder rather than removing it, so the glob would still evaluate afterward, just silently against the wrong (anonymised) name, and never find the real files:

$ xnat-ingest associate /data/staging/assigned /data/staging/associated \
    "medimage/vnd.siemens.syngo-mi.vr20b.count-rate|medimage/vnd.siemens.syngo-mi.vr20b.list-mode" \
    "/data/raw-exports/{PatientName.family_name}_{PatientName.given_name}*.ptd" \
    ".*/[^.]+\.[^.]+\.[^.]+\.(?P<id>\d+)\.[A-Z]+_(?P<resource>[^.]+).*"

  • INPUT_DIR — the directory of sessions to associate files into, from whichever stage you're running this after (XINGEST_INPUT_DIR)

  • OUTPUT_DIR — where sessions are written with the associated files attached (XINGEST_OUTPUT_DIR)

  • DATATYPE — the format of the files being linked in, as one or more |-separated MIME-like identifiers (XINGEST_DATATYPE) — multiple formats can be matched by a single call, as in the example above

  • GLOB — a glob pattern used to find the files, with {field} placeholders filled in from the session's own metadata (e.g. {PatientName.family_name}) (XINGEST_GLOB). A relative pattern is resolved against the directory the primary DICOMs came from; an absolute one (as above) searches a separate location entirely, ignoring where the DICOMs live

  • ID_PATTERN — a regular expression matched against each found file's path (XINGEST_ID_PATTERN), with named groups id (the scan ID to attach it to) and resource (the resource name to file it under), and optionally type (the scan type/description, which otherwise defaults to the scan ID)

Attaching files after cleanup (metadata skeletons)

Associated files sometimes turn up late — after a session's primary DICOMs have already been through assign/deidentify/upload and had their local copies cleaned up. associate doesn't need to know or care whether that's happened: if assign (and/or deidentify) was run with --unlink-source keep-metadata rather than all, the session and scan directories are still there — just without the (now redundant) image data — since --unlink-source keep-metadata only removes each resource's files, never the session's or scan's own __METADATA__.json.

associate loads a session the same way either way. A scan with no resources at all is an ordinary, valid state (it's what a freshly-grouped scan looks like before anything's been added to it), so a metadata-only skeleton reloads and matches files against it exactly as it would for a session whose data is still fully present. The GLOB template is filled in from the session's own metadata (preserved regardless), and ID_PATTERN only ever looks at the new file's path — neither depends on the primary resource data actually being there.

In short: run assign/deidentify with --unlink-source keep-metadata instead of all if you expect associate to need to attach files sometime after the bulk of a session's data has already moved on, and you don't want to keep the full image data around indefinitely just in case.