Deidentification

For sites where the assign-ed sessions still contain identifiable data (e.g. a clinical scanner staging DICOMs before upload), insert a deidentify step between assign and upload. It strips patient-identifying fields from each session and writes both the deidentified copy and a re-identification mapping (so the process can be reversed later if needed, e.g. to look up a scan for a clinical follow-up).

$ xnat-ingest deidentify /data/staging/assigned /data/staging/deidentified \
    /etc/xnat-ingest/deid-specs /data/staging/reid

• INPUT_DIR — the assign output directory (XINGEST_INPUT_DIR)

• OUTPUT_DIR — where deidentified sessions are written; point upload at this directory instead of the assign output (XINGEST_OUTPUT_DIR)

• SPEC_DIR — the deidentification specs, one directory per project (see below) (XINGEST_SPEC_DIR)

• REID_DIR — where the re-identification mappings are written, one JSON file per session (XINGEST_REID_DIR)

As with assign, --unlink-source all/--unlink-source keep-metadata clean up the assign output once a session's been deidentified — see 2. Assign project/subject/session IDs for what the two modes do.

Only formats known to carry patient information are touched — DICOM is treated this way by default, while derived formats like NIfTI are assumed already de-identifiable-in-place and are just copied through unchanged. If a session contains a PHI-bearing format with no applicable spec (see below), that session is skipped and logged as an error rather than uploaded with PHI still attached.

Laying out the deidentification specs

SPEC_DIR holds one subdirectory per project, named to match the project ID that assign gave the session, plus an optional __default__ subdirectory used as a fallback for any project without its own:

deid-specs/
    __default__/
        medimage@dicom-series.json
    MYPROJECT/
        medimage@dicom-series.json

Within each project directory, there's one JSON spec file per format, named after the format's MIME-like identifier with / replaced by @ (e.g. medimage/dicom-series becomes medimage@dicom-series.json). A spec also covers more specific sub-formats — e.g. a spec for the broader medimage/dicom-collection format applies to medimage/dicom-series sessions too if there's no more specific match.

Note

The intent is for each spec file's contents to configure how that format is deidentified for that project — e.g. which additional DICOM tags to blank or remap beyond the built-in set, or project-specific replacement values. That per-project customisation isn't wired up yet: today, the mere presence of a spec file is what matters (it tells deidentify a format is handled for that project), while the actual fields removed from DICOM are currently a fixed, built-in list regardless of what the spec file contains. Until the content is read, an empty JSON object ({}) is a reasonable placeholder for each spec file.

Re-identifying data later

Each session's original identifying values (before they were stripped) are written to REID_DIR as <session_id>.json. If --reid-encrypt-key is set to a URL-safe base64-encoded 32-byte key (e.g. from Fernet.generate_key() in the cryptography package), the file is written encrypted instead, as <session_id>.json.enc, and can only be read back with that same key — keep it somewhere separate from REID_DIR itself.