Attach files without relevant metadata

collate/label sort files using metadata read out of the files themselves (DICOM tags, by default). Some data that belongs in an accession doesn't work that way — e.g. raw list-mode or count-rate files from a Siemens PET scanner, which have no readable patient/session metadata of their own, or live in a completely separate export location to the DICOMs. If those files can still be found some other way (a shared naming convention with the primary DICOMs, in the same folder or elsewhere) and you know which acquisition/resource they belong to from their path, attach can link them into an accession that's already been through collate (see Sources).

It can run at any point after collate — on collate's own output, on label's, or later — since it only needs the accession's own metadata (read from the primary files' own headers), not its labels. The one real ordering constraint is deidentify: run attach before it if GLOB relies on identifying metadata, as in the example below ({PatientName.family_name}) — deidentify replaces PatientName with a fixed placeholder rather than removing it, so the glob would still evaluate afterward, just silently against the wrong (anonymised) name, and never find the real files:

$ accessionflow attach /data/staging/labelled /data/staging/attached \
    "medimage/vnd.siemens.syngo-mi.vr20b.count-rate|medimage/vnd.siemens.syngo-mi.vr20b.list-mode" \
    "/data/raw-exports/{PatientName.family_name}_{PatientName.given_name}*.ptd" \
    ".*/[^.]+\.[^.]+\.[^.]+\.(?P<id>\d+)\.[A-Z]+_(?P<resource>[^.]+).*"
  • INPUT_DIR — the directory of accessions to attach files into, from whichever stage you're running this after (ACFLOW_INPUT_DIR)

  • OUTPUT_DIR — where accessions are written with the attached files attached (ACFLOW_OUTPUT_DIR)

  • DATATYPE — the format of the files being linked in, as one or more |-separated MIME-like identifiers (ACFLOW_DATATYPE) — multiple formats can be matched by a single call, as in the example above

  • GLOB — a glob pattern used to find the files, with {field} placeholders filled in from the accession's own metadata (e.g. {PatientName.family_name}) (ACFLOW_GLOB). A relative pattern is resolved against the directory the primary DICOMs came from; an absolute one (as above) searches a separate location entirely, ignoring where the DICOMs live

  • ID_PATTERN — a regular expression matched against each found file's path (ACFLOW_ID_PATTERN), with named groups id (the acquisition ID to attach it to) and resource (the resource name to file it under), and optionally type (the acquisition type/description, which otherwise defaults to the acquisition ID)

Attaching files after cleanup (metadata skeletons)

Attached files sometimes turn up late — after an accession's primary DICOMs have already been through label/deidentify/store and had their local copies cleaned up. attach doesn't need to know or care whether that's happened: if label (and/or deidentify) was run with --unlink-source keep-metadata rather than all, the accession and acquisition directories are still there — just without the (now redundant) image data — since --unlink-source keep-metadata only removes each resource's files, never the accession's or acquisition's own __METADATA__.json (see Cleaning up previous stages).

attach loads an accession the same way either way. An acquisition with no resources at all is an ordinary, valid state (it's what a freshly-collated acquisition looks like before anything's been added to it), so a metadata-only skeleton reloads and matches files against it exactly as it would for an accession whose data is still fully present. The GLOB template is filled in from the accession's own metadata (preserved regardless), and ID_PATTERN only ever looks at the new file's path — neither depends on the primary resource data actually being there.

In short: run label/deidentify with --unlink-source keep-metadata instead of all if you expect attach to need to attach files sometime after the bulk of an accession's data has already moved on, and you don't want to keep the full image data around indefinitely just in case.