Command-line interface¶
AccessionFlow's command-line interface is a set of subcommands of the
accessionflow command (or its shorter alias, acflow). See Concepts for
how they fit together into an accession flow.
The collate stage has a subcommand for each source, e.g.
accessionflow collate folder, and the label, store and check-store
stages have one for each data store, e.g. accessionflow label xnat.
Most options can also be set by environment variables, listed in
Environment variables.
Sources¶
accessionflow collate folder¶
Collates the files found in folders, on a local path or a mounted file system, into accessions, acquisitions and resources
INPUT_PATHS are either paths to directories containing the files to collate, or glob patterns that select the paths directly
OUTPUT_DIR is the staging directory the files of each accession are collated to, ready to be labelled
Usage
accessionflow collate folder [OPTIONS] [INPUT_PATHS]... OUTPUT_DIR
Options
- --accession <specifier> <datatype>¶
The metadata field whose value identifies the accession each file is collated into, before it is labelled, so it must be unique across all accessions (ACFLOW_ACCESSION env. var). Defaults to 'StudyInstanceUID' for DICOM collections; other fileset types need an explicit spec. Can also be a Python format string over several fields, e.g. '{PatientID}_{StudyDate:%Y%m%d}', to compose one. Not to be confused with DICOM's AccessionNumber, which is just another field that can be used here or to label the accession.
- --acquisition <specifier> <datatype>¶
The keyword of the metadata field to extract the acquisition ID from, which must be unique within its accession, or a Python format string over several fields (see --accession). Defaults to 'SeriesNumber' for DICOM collections (one acquisition per series); for any fileset type not matched by an --acquisition spec the acquisition is named after the resource (ACFLOW_ACQUISITION env. var)
- --resource <specifier> <datatype>¶
The keywords of the metadata field to extract the resource ID from for different datatypes (use generic/file-set as a catch-all if required), or a Python format string over several fields (see --accession). If not given, each resource is labelled with the mime-like rendering of its fileset type name, e.g. 'vectra-export', 'sqlite3-db' (ACFLOW_RESOURCE env. var)
- --datatype <mime-type>¶
The MIME-type(s) (or "MIME-like" see FileFormats docs) of potential datatype(s) of the primary files to to upload, defaults to 'medimage/dicom-series'. Any formats implemented in the FileFormats Python package (https://github.com/ArcanaFramework/fileformats) that implement the 'read_metadata' "extra" are supported, see FF docs on how to add support for new formats.
- --path-metadata-regex <regex> <datatype>¶
Regular expressions to extract "metadata" values from resource file paths as named groups. using Python regular expression syntax. The named groups are used as metadata fields for the resource files, and the extracted values will be used to populate the corresponding metadata fields to complement the metadata read from the file headers.
- --path-metadata <pattern> <datatype>¶
A simpler, glob-like alternative to --path-metadata-regex. '{name}' placeholders mark the parts of the resource's path to extract into the metadata field 'name', e.g. '{subject}/{session}'. '*' matches within a path segment, '**' across segments and '?' a single character; '{{' and '}}' match literal braces. Unless the pattern starts with '/', it only needs to match the trailing segments of the path. Applied before any --path-metadata-regex (ACFLOW_PATH_METADATA env. var)
- --on-resource-clash <policy> <scope>¶
How to handle two filesets resolving to the same acquisition/resource name: a <policy> ('avoid' appends a _2/_3 suffix, 'merge' folds both into one SetOf, 'overwrite' replaces) plus the datatype <scope> it applies to (a mime-like, a '|'-union, or 'all'). Repeatable. A clash is resolved by the first spec whose scope covers BOTH filesets; a clash no spec covers raises. (ACFLOW_ON_RESOURCE_CLASH env. var)
- --allow-unrecognised, --allow-unrecognized <allow_unrecognised>¶
Regular expressions matched against the basename of any input path that no --datatype recognised; matches are skipped instead of raising. Use '.*' to tolerate all unrecognised files. Does not affect recognised filesets. Repeatable; the ACFLOW_ALLOW_UNRECOGNISED env. var takes a ';'-separated list.
- --exclude-path <exclude_paths>¶
Glob patterns matched against each input path relative to <input-dir>, applied before classification so a match is dropped even if a --datatype would claim it (e.g. a vendor thumbnail that is a valid image/png). '*' does not cross '/', '**' does. Repeatable; the ACFLOW_EXCLUDE_PATH env. var takes a ';'-separated list.
- --ignore-datatype <ignore_datatypes>¶
Datatypes expected in the input but not wanted: recognised files of these types are dropped instead of raising, and with --recursive a matching directory is skipped without descending into it. A path matching neither --datatype nor --ignore-datatype nor --allow-unrecognised/--exclude-path still raises. (ACFLOW_IGNORE_DATATYPE env. var)
- --dont-read-metadata <datatype>¶
Datatypes whose files aren't read for metadata, so their only metadata fields are those from --path-metadata/--path-metadata-regex and --metadata-table. Saves reading the header of every file when none of its fields are used, e.g. when the IDs all come from the path. Matched with isinstance, so 'medimage/nifti' also covers 'medimage/nifti-gz'. Repeatable. (ACFLOW_DONT_READ_METADATA env. var)
- --recursive, --not-recursive¶
Whether to recursively search input directories for input files
- --unlink-source <unlink_source>¶
Whether to unlink the source files after they have been successfully collated. 'all' and 'keep-metadata' behave the same here (individual source files are removed one by one) since 'collate's source isn't a directory tree that accessionflow owns and can't have a metadata skeleton left behind in it (ACFLOW_UNLINK_SOURCE env. var)
- Options:
all | keep-metadata
- --collate-resources <mime-type> <collation>¶
Flatten files of the given datatype into the resource directory during collation, regardless of source directory structure. Collation level is one of 'any', 'siblings', or 'adjacent' (default 'siblings').
- --metadata-table <path> <row-frequency> <join-exprs>¶
Specify metadata tables to extract and join metadata from input files (ACFLOW_METADATA_TABLES env. var). The 'path' arg specifies the location of the metadata table file. Its format is auto-detected as CSV or TSV from the file extension; a different format can be forced by appending its mime-type in square brackets, e.g. 'path/to/table.dat[text/csv]'. The "row frequency" arg specifies what each row in the metadata table corresponds to in the data hierarchy, and can be one of 'accession', 'acquisition', 'resource', 'fileset', 'fileset[<mime-type>]'. When one or more mime-types are given in square brackets after 'fileset' they restrict the join to input files of those types (multiple mime-types can be '|'-separated, e.g. 'fileset[image/png|image/jpeg]'); a bare 'fileset' matches any input file. The 'join-exprs' arg is a comma-separated list of '<column-name>=<cell-value>' expressions; a row is a match when every expression holds. The '<cell-value>' is either the name of an existing metadata field or a Python format string over one or more metadata fields, e.g. '{PatientID}_{SessionID}'. All columns of the matched row are then merged into the target's metadata. The example below extracts the relative path of an image file with --path-metadata-regex and uses it to join a table whose 'ImagePath' column holds spreadsheet HYPERLINK() formulas.
- accessionflow collate folder ...
--path-metadata-regex '.*/(?P<relpath>[w-]+/[w-]+.(?:png|jpg))' image/png|image/jpeg --metadata-table path/to/table.csv[text/csv] fileset[image/png|image/jpeg] 'ImagePath=HYPERLINK("{relpath}")'
- --loop <loop>¶
Run the collation continuously every LOOP seconds (ACFLOW_LOOP env. var)
- --wait-period <wait_period>¶
The number of seconds since an accession was last modified at the source before it is collated, so that accessions that are still being transferred aren't collated partially (ACFLOW_WAIT_PERIOD env. var)
- --copy-mode <copy_mode>¶
The method to use for copying files (ACFLOW_COPY_MODE env. var)
- --convert <src-mime-like> <tgt-spec>¶
Convert resources of <src-mime-like> to <tgt-spec> during collation. <tgt-spec> may be either '<tgt-mime-like>' or '<tgt-mime-like>:key=value[,key=value...]', e.g. 'application/zip:compression=ZIP_STORED' (ACFLOW_CONVERT env. var)
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
Additional loggers to capture the logs of, besides 'accessionflow' (e.g. 'xnat') (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
Arguments
- INPUT_PATHS¶
Optional argument(s)
- OUTPUT_DIR¶
Required argument
Environment variables
- ACFLOW_INPUT_PATHS
Provide a default for
INPUT_PATHS
- ACFLOW_OUTPUT_DIR
Provide a default for
OUTPUT_DIR
- ACFLOW_ACCESSION
Provide a default for
--accession
- ACFLOW_ACCESSION_UID
Provide a default for
--accession-uid
- ACFLOW_ACQUISITION
Provide a default for
--acquisition
- ACFLOW_ACQUISITION_ID
Provide a default for
--acquisition-id
- ACFLOW_RESOURCE
Provide a default for
--resource
- ACFLOW_DATATYPES
Provide a default for
--datatype
- ACFLOW_PATH_METADATA_REGEX
Provide a default for
--path-metadata-regex
- ACFLOW_PATH_METADATA
Provide a default for
--path-metadata
- ACFLOW_ON_RESOURCE_CLASH
Provide a default for
--on-resource-clash
- ACFLOW_ALLOW_UNRECOGNISED
Provide a default for
--allow-unrecognised
- ACFLOW_EXCLUDE_PATH
Provide a default for
--exclude-path
- ACFLOW_IGNORE_DATATYPE
Provide a default for
--ignore-datatype
- ACFLOW_DONT_READ_METADATA
Provide a default for
--dont-read-metadata
- ACFLOW_RECURSIVE
Provide a default for
--recursive
- ACFLOW_UNLINK_SOURCE
Provide a default for
--unlink-source
- ACFLOW_COLLATE_RESOURCES
Provide a default for
--collate-resources
- ACFLOW_METADATA_TABLES
Provide a default for
--metadata-table
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_WAIT_PERIOD
Provide a default for
--wait-period
- ACFLOW_COPY_MODE
Provide a default for
--copy-mode
- ACFLOW_CONVERT
Provide a default for
--convert
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
accessionflow collate orthanc¶
Collates the DICOM studies held by an Orthanc server into accessions in a staging directory, ready to be labelled. The files are hardlinked from Orthanc's storage directory, so it must be on the same file system as the staging directory.
URL of the Orthanc instance to connect to (ACFLOW_ORTHANC_URL env. var)
STORE_DIR is the path to Orthanc's StorageDirectory as mounted (ACFLOW_ORTHANC_STORE_DIR env. var)
OUTPUT_DIR is the staging directory the files of each accession are collated to
USER is the Orthanc user (ACFLOW_ORTHANC_USER env. var)
PASSWORD is the password of the Orthanc user (ACFLOW_ORTHANC_PASSWORD env. var)
Usage
accessionflow collate orthanc [OPTIONS] URL STORE_DIR OUTPUT_DIR USER PASSWORD
Options
- --to-process-label <to_process_label>¶
Label applied to Orthanc studies after staging to prevent re-processing. Can be removed via the Orthanc UI (ACFLOW_ORTHANC_TO_PROCESS env. var)
- --processed-label <processed_label>¶
Label applied to Orthanc studies after staging to prevent re-processing. Can be removed via the Orthanc UI (ACFLOW_ORTHANC_PROCESSED env. var)
- --unlink-source <unlink_source>¶
Whether to unlink the source studies in Orthanc after they have been successfully collated. Not yet implemented (ACFLOW_UNLINK_SOURCE env. var)
- Options:
all | keep-metadata
- --loop <loop>¶
Run the collation continuously every LOOP seconds (ACFLOW_LOOP env. var)
- --wait-period <wait_period>¶
The number of seconds since an accession was last modified at the source before it is collated, so that accessions that are still being transferred aren't collated partially (ACFLOW_WAIT_PERIOD env. var)
- --copy-mode <copy_mode>¶
The method to use for copying files (ACFLOW_COPY_MODE env. var)
- --convert <src-mime-like> <tgt-spec>¶
Convert resources of <src-mime-like> to <tgt-spec> during collation. <tgt-spec> may be either '<tgt-mime-like>' or '<tgt-mime-like>:key=value[,key=value...]', e.g. 'application/zip:compression=ZIP_STORED' (ACFLOW_CONVERT env. var)
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
Additional loggers to capture the logs of, besides 'accessionflow' (e.g. 'xnat') (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
Arguments
- URL¶
Required argument
- STORE_DIR¶
Required argument
- OUTPUT_DIR¶
Required argument
- USER¶
Required argument
- PASSWORD¶
Required argument
Environment variables
- ACFLOW_ORTHANC_URL
Provide a default for
URL
- ACFLOW_ORTHANC_STORE_DIR
Provide a default for
STORE_DIR
- ACFLOW_ORTHANC_USER
Provide a default for
USER
- ACFLOW_ORTHANC_PASSWORD
Provide a default for
PASSWORD
- ACFLOW_ORTHANC_TO_PROCESS
Provide a default for
--to-process-label
- ACFLOW_ORTHANC_PROCESSED
Provide a default for
--processed-label
- ACFLOW_UNLINK_SOURCE
Provide a default for
--unlink-source
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_WAIT_PERIOD
Provide a default for
--wait-period
- ACFLOW_COPY_MODE
Provide a default for
--copy-mode
- ACFLOW_CONVERT
Provide a default for
--convert
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
Stages¶
accessionflow attach¶
Stages images found in the input directories into separate directories for each imaging acquisition session
FILES_PATH is either the path to a directory containing the files to upload, or a glob pattern that selects the paths directly
OUTPUT_DIR is the directory that the files for each session are collated to before they are uploaded to XNAT
Usage
accessionflow attach [OPTIONS] INPUT_DIR OUTPUT_DIR DATATYPE GLOB ID_PATTERN
Options
- --copy-mode <copy_mode>¶
The method to use for copying files (ACFLOW_COPY_MODE env. var)
- --unlink-source <unlink_source>¶
Whether to unlink the source files matched by GLOB after they have been attached. 'all' and 'keep-metadata' behave the same here (individual source files are removed one by one), since these files live at an arbitrary external location rather than a directory tree accessionflow owns. If not set, the source files are left in place (ACFLOW_UNLINK_SOURCE env. var)
- Options:
all | keep-metadata
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
The loggers to use for logging. By default just the 'accessionflow' logger is used. But additional loggers can be included (e.g. 'xnat') can be specified here (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
- --spaces-to-underscores, --no-spaces-to-underscores¶
Whether to replace spaces with underscores in the filenames of attached files (ACFLOW_SPACES_TO_UNDERSCORES env. var)
- --loop <loop>¶
Run the staging process continuously every LOOP seconds (ACFLOW_LOOP env. var).
- --on-resource-clash <on_resource_clash>¶
Determines the behavior when a resource with the same name already exists in the scan. Options are 'merge', 'avoid', 'error' (ACFLOW_ON_RESOURCE_CLASH env. var)Default: 'error'
- Options:
error | avoid | merge | overwrite
- --temp-dir <temp_dir>¶
The directory to use for temporary downloads (i.e. from s3)
- --require-manifest, --dont-require-manifest¶
Whether to require manifest files in the staged resources or not (default: required)
Arguments
- INPUT_DIR¶
Required argument
- OUTPUT_DIR¶
Required argument
- DATATYPE¶
Required argument
- GLOB¶
Required argument
- ID_PATTERN¶
Required argument
Environment variables
- ACFLOW_INPUT_DIR
Provide a default for
INPUT_DIR
- ACFLOW_OUTPUT_DIR
Provide a default for
OUTPUT_DIR
- ACFLOW_DATATYPE
Provide a default for
DATATYPE
- ACFLOW_GLOB
Provide a default for
GLOB
- ACFLOW_ID_PATTERN
Provide a default for
ID_PATTERN
- ACFLOW_COPY_MODE
Provide a default for
--copy-mode
- ACFLOW_UNLINK_SOURCE
Provide a default for
--unlink-source
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
- ACFLOW_SPACES_TO_UNDERSCORES
Provide a default for
--spaces-to-underscores
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_ON_RESOURCE_CLASH
Provide a default for
--on-resource-clash
- ACFLOW_TEMPDIR
Provide a default for
--temp-dir
- ACFLOW_REQUIRE_MANIFEST
Provide a default for
--require-manifest
accessionflow deidentify¶
Stages images found in the input directories into separate directories for each imaging acquisition session
INPUT_DIR is the path to the directory containing the labelled session directories to de-identify (the output of the 'label' stage). Each is named after the session's labels, e.g. <project>.<subject>.<session> for XNAT, and contains subdirectories for each acquisition, which in turn contain its resource files.
OUTPUT_DIR is the directory the de-identified sessions are written to, ready for the 'store' stage
RECIPE_DIR is the directory containing the deidentification recipes. Recipes are selected by the session's top-level label (e.g. the XNAT project), so it should contain one subdirectory per top-level label, plus an optional "__default__" subdirectory used as a fallback for those without their own. Within each of these subdirectories, the directory structure mirrors the MIME-like hierarchy: a subdirectory per category containing one recipe file per format (e.g. 'medimage/dicom-series'). Any file extension (or none) is accepted. Side-cars of the recipe format can sit alongside the recipe file, e.g. for DICOM a '<format>.transforms.yaml' file (e.g. 'medimage/dicom-series.transforms.yaml') declaring the values of the 'var:' and 'func:' references in the recipe (see the deid transforms spec in fileformats-medimage), and a '<format>.salt' file with a key the transforms can salt hashed values with (only readable by the user running the deidentification). Formats without a matching recipe file are only deidentified if one is found for a broader/parent format (e.g. a 'medimage/dicom-collection' recipe also covers 'medimage/dicom-series'). Metadata side-cars in the deidentified output retain only __uid__, where present. Other metadata fields are omitted; source metadata is unchanged. __uid__ holds the original session UID (by default the DICOM StudyInstanceUID) and is copied to the data store by the store stage, unless it is replaced with a keyed hash using --hash-uid-key.
The re-identification metadata (a mapping from original to de-identified identifiers for each resource in the session, plus any DICOM tags modified during de-identification) is only written if --reid-dir is given; see that option.
Usage
accessionflow deidentify [OPTIONS] INPUT_DIR OUTPUT_DIR RECIPE_DIR
Options
- --reid-dir <reid_dir>¶
The directory to save the re-identification metadata to, which can be used to re-identify the de-identified data if needed. One JSON file is written per session, named after the session's directory, <labels>.json (or <labels>.json.enc if encrypted with --reid-encrypt-key). If this option is not given, the re-identification metadata is discarded rather than written to disk, which is the safer default when retaining the original<->deidentified mapping would itself be a security concern (ACFLOW_REID_DIR env. var)
- --unlink-source <unlink_source>¶
Whether to unlink the labelled session directories after they have been deidentified. 'all' removes the whole labelled session directory; 'keep-metadata' removes the resource data but leaves the session/scan-level metadata behind, so a lightweight skeleton of the session survives (e.g. for 'attach' to use later). If not set, the labelled directories are left in place (ACFLOW_UNLINK_SOURCE env. var)
- Options:
all | keep-metadata
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
The loggers to use for logging. By default just the 'accessionflow' logger is used. But additional loggers can be included (e.g. 'xnat') can be specified here (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
- --require-manifest, --no-require-manifest¶
Whether to require a MANIFEST.json file in each resource directory. If False, resources are loaded as generic FileSets without checksum validation (ACFLOW_REQUIRE_MANIFEST env. var)
- --copy-mode <copy_mode>¶
The method to use for copying files (ACFLOW_COPY_MODE env. var)
- --loop <loop>¶
Run the staging process continuously every LOOP seconds (ACFLOW_LOOP env. var).
- --on-resource-clash <on_resource_clash>¶
Determines the behavior when a resource with the same name already exists in the scan. Options are 'merge', 'avoid', 'error' (ACFLOW_ON_RESOURCE_CLASH env. var).Default: 'error'
- Options:
error | avoid | merge | overwrite
- --reid-encrypt-key <reid_encrypt_key>¶
An optional encryption key to use for encrypting the re-identification metadata (ACFLOW_REID_ENCRYPT_KEY env. var). This should be a URL-safe base64-encoded 32-byte key, e.g. generated using Fernet.generate_key() from the cryptography package
- --metadata-allowlist <metadata_allowlist>¶
Comma-separated metadata keys to retain in output sidecars (ACFLOW_METADATA_ALLOWLIST env. var). Defaults to __uid__. __uid__, __labels__ and __store_type__ are always retained, as they are needed to identify a session.
- --hash-uid-key <hash_uid_key>¶
Secret key used to replace each session's UID (by default its DICOM StudyInstanceUID) with a keyed hash of it in the output. The original and hashed UIDs are both recorded in the re-identification metadata if --reid-dir is given. Keep the key private and don't change it, as the hash identifies the session if it is uploaded again. Sessions already on XNAT hold their previous UID, see --no-uid-check of the upload stage. If not given, the original UID is kept. Prefer the env. var so that the key isn't visible in the process list (ACFLOW_HASH_UID_KEY env. var)
Arguments
- INPUT_DIR¶
Required argument
- OUTPUT_DIR¶
Required argument
- RECIPE_DIR¶
Required argument
Environment variables
- ACFLOW_INPUT_DIR
Provide a default for
INPUT_DIR
- ACFLOW_OUTPUT_DIR
Provide a default for
OUTPUT_DIR
- ['ACFLOW_RECIPE_DIR', 'ACFLOW_SPEC_DIR']
Provide a default for
RECIPE_DIR
- ACFLOW_REID_DIR
Provide a default for
--reid-dir
- ACFLOW_UNLINK_SOURCE
Provide a default for
--unlink-source
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
- ACFLOW_REQUIRE_MANIFEST
Provide a default for
--require-manifest
- ACFLOW_COPY_MODE
Provide a default for
--copy-mode
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_ON_RESOURCE_CLASH
Provide a default for
--on-resource-clash
- ACFLOW_REID_ENCRYPT_KEY
Provide a default for
--reid-encrypt-key
- ACFLOW_METADATA_ALLOWLIST
Provide a default for
--metadata-allowlist
- ACFLOW_HASH_UID_KEY
Provide a default for
--hash-uid-key
accessionflow package¶
Replace staged scan resources with one or more named conversions.
INPUT_DIR is normally the output of the deidentify command. Every OUTPUT_RESOURCE conversion receives the original matching SOURCE_DATATYPE resource, allowing an archive and a sample to be produced independently without reopening the archive.
Usage
accessionflow package [OPTIONS] INPUT_DIR OUTPUT_DIR SOURCE_DATATYPE
Options
- --output-resource <label> <target-spec>¶
Required Resource label and target datatype to produce from each matching source. May be repeated. Target specs accept converter options after ':', e.g. 'application/zip:allowZip64=false'.
- --copy-mode <copy_mode>¶
How converted and pass-through resources are copied to the output.
- --unlink-source <unlink_source>¶
Remove the input session after successful packaging, either entirely or leaving its session/scan metadata skeleton.
- Options:
all | keep-metadata
- --on-resource-clash <on_resource_clash>¶
How to handle output labels that clash within the same scan.
- Options:
error | avoid | merge | overwrite
- --require-manifest, --dont-require-manifest¶
Whether staged input resources must contain manifests.
- --temp-dir <temp_dir>¶
Directory in which converters create temporary data.
- --loop <loop>¶
Run repeatedly every LOOP seconds; a negative value runs once.
- --logger <logtype> <loglevel> <location>¶
- --additional-logger <additional_loggers>¶
- --raise-errors, --dont-raise-errors¶
Arguments
- INPUT_DIR¶
Required argument
- OUTPUT_DIR¶
Required argument
- SOURCE_DATATYPE¶
Required argument
Environment variables
- ACFLOW_INPUT_DIR
Provide a default for
INPUT_DIR
- ACFLOW_OUTPUT_DIR
Provide a default for
OUTPUT_DIR
- ACFLOW_SOURCE_DATATYPE
Provide a default for
SOURCE_DATATYPE
- ACFLOW_OUTPUT_RESOURCES
Provide a default for
--output-resource
- ACFLOW_COPY_MODE
Provide a default for
--copy-mode
- ACFLOW_UNLINK_SOURCE
Provide a default for
--unlink-source
- ACFLOW_ON_RESOURCE_CLASH
Provide a default for
--on-resource-clash
- ACFLOW_REQUIRE_MANIFEST
Provide a default for
--require-manifest
- ACFLOW_TEMPDIR
Provide a default for
--temp-dir
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
accessionflow annotate¶
Annotates accessions with fields extracted from their metadata, which the data store records alongside them (e.g. as custom variables on XNAT sessions).
Usage
accessionflow annotate [OPTIONS] INPUT_DIR OUTPUT_DIR
Options
- --extract <xnat-field> <specifier>¶
Required Expose session metadata as an XNAT field. May be repeated; separate entries with ';' in ACFLOW_FIELDS.
- --loop <loop>¶
- --logger <logtype> <loglevel> <location>¶
- --additional-logger <additional_loggers>¶
- --raise-errors, --dont-raise-errors¶
Arguments
- INPUT_DIR¶
Required argument
- OUTPUT_DIR¶
Required argument
Environment variables
- ACFLOW_INPUT_DIR
Provide a default for
INPUT_DIR
- ACFLOW_OUTPUT_DIR
Provide a default for
OUTPUT_DIR
- ACFLOW_FIELDS
Provide a default for
--extract
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
Folder store¶
accessionflow label folder¶
Labels accessions for folder data stores with their user-defined (--level) labels, extracted from accession metadata, after they have been collated into acquisitions/resources
INPUT_DIR is the path to the directory containing the collated-but-not-yet-labelled accessions (the output of the 'collate' command)
OUTPUT_DIR is the directory that the labelled accessions will be written to, each in a directory named '<label1>.<label2>...'
If a label can't be resolved from an accession's metadata, it is given a unique 'INVALID_MISSING_<FIELD>_<random>' placeholder instead of being dropped, and the whole accession is saved under an '__invalid__' subdirectory of OUTPUT_DIR instead of alongside normally-labelled ones, so it can be found and manually reviewed/reprocessed rather than silently lost.
Usage
accessionflow label folder [OPTIONS] INPUT_DIR OUTPUT_DIR
Options
- --level <name> <field>¶
Required A level of the folder hierarchy the sessions are labelled with, and the keyword of the metadata field to extract its label from (or a Python format string over several fields, e.g. '{PatientID}_{StudyDate:%Y%m%d}'). Repeat for each level, in order from the top, e.g. '--level site SiteID --level subject PatientID' (ACFLOW_LEVELS env. var, as space-separated pairs)
- --acquisition <acquisition_field>¶
The keyword of the metadata field to extract the description of each acquisition from. Acquisitions for which the field can't be resolved are left without one (ACFLOW_ACQUISITION env. var)
- --constant-label <name> <value>¶
Fix the label of a level given by --level as a constant for all data matched by this command, e.g. '--constant-label site SYDNEY'. May be repeated (ACFLOW_CONSTANT_LABELS env. var)
- --include <mime-type>¶
Only label resources matching this fileformats MIME-like datatype. May be repeated to include multiple datatypes (ACFLOW_INCLUDE env. var)
- --loop <loop>¶
Run the label process continuously every LOOP seconds (ACFLOW_LOOP env. var).
- --copy-mode <copy_mode>¶
The method to use for copying files (ACFLOW_COPY_MODE env. var)
- --unlink-source <unlink_source>¶
Whether to unlink the collated session directories after they have been labelled. 'all' removes the whole collated session directory; 'keep-metadata' removes the resource data but leaves the session/acquisition-level metadata behind, so a lightweight skeleton of the session survives (e.g. for 'attach' to use later). If not set, the collated directories are left in place (ACFLOW_UNLINK_SOURCE env. var)
- Options:
all | keep-metadata
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
The loggers to use for logging. By default just the 'accessionflow' logger is used. But additional loggers can be included (e.g. 'xnat') can be specified here (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
Arguments
- INPUT_DIR¶
Required argument
- OUTPUT_DIR¶
Required argument
Environment variables
- ACFLOW_INPUT_DIR
Provide a default for
INPUT_DIR
- ACFLOW_OUTPUT_DIR
Provide a default for
OUTPUT_DIR
- ACFLOW_LEVELS
Provide a default for
--level
- ACFLOW_ACQUISITION
Provide a default for
--acquisition
- ACFLOW_CONSTANT_LABELS
Provide a default for
--constant-label
- ACFLOW_INCLUDE
Provide a default for
--include
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_COPY_MODE
Provide a default for
--copy-mode
- ACFLOW_UNLINK_SOURCE
Provide a default for
--unlink-source
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
accessionflow store folder¶
Deposits the labelled accessions in a staging directory (as prepared by 'label folder') in a folder data store: nested label folders, with a folder per acquisition and per resource, on a local path, a mount, or a remote file system such as S3 or SFTP.
STAGED is the staging directory, or an S3 bucket to download the accessions from.
DEST is the root of the store, a path or an fsspec URL, e.g. '/mnt/archive', 's3://bucket/archive' or 'sftp://host/archive'. Remote file systems need their fsspec package installed, e.g. 'accessionflow[s3]' for S3 or 'accessionflow[ssh]' for SFTP.
The resources' manifests and the accessions' fields are deposited with them, but not their metadata. A resource already in the store with the same files is skipped, so depositing again only copies what has changed.
Usage
accessionflow store folder [OPTIONS] STAGED DEST
Options
- --storage-option <key> <value>¶
An option for the fsspec file system of the store, e.g. '--storage-option anon true' or '--storage-option key <access-key>'. Values that are JSON are decoded. May be repeated (ACFLOW_STORAGE_OPTIONS env. var, as space-separated pairs)
- --require-manifest, --dont-require-manifest¶
Whether the staged resources must have manifests. If not, the checksums of resources without one are worked out from their files (ACFLOW_REQUIRE_MANIFEST env. var)
- --store-credentials <access-key> <secret-key>¶
Credentials of the S3 bucket the accessions are staged in, if STAGED is one (ACFLOW_STORE_CREDENTIALS env. var)
- --temp-dir <temp_dir>¶
The directory to download accessions staged in S3 to
- --s3-max-workers <s3_max_workers>¶
The number of files to download from an S3 staging bucket at once (ACFLOW_S3_MAX_WORKERS env. var)
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
Additional loggers to capture the logs of, besides 'accessionflow' (e.g. 's3fs') (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
- --overwrite, --no-overwrite¶
Whether to replace a resource that is already in the store with different files. If not, it is reported as an error (ACFLOW_OVERWRITE env. var)
- --wait-period <wait_period>¶
The number of seconds since an accession was last modified before it is deposited (ACFLOW_WAIT_PERIOD env. var)
- --loop <loop>¶
Run continuously, every LOOP seconds (ACFLOW_LOOP env. var)
Arguments
- STAGED¶
Required argument
- DEST¶
Required argument
Environment variables
- ACFLOW_STAGED
Provide a default for
STAGED
- ACFLOW_DEST
Provide a default for
DEST
- ACFLOW_STORAGE_OPTIONS
Provide a default for
--storage-option
- ACFLOW_REQUIRE_MANIFEST
Provide a default for
--require-manifest
- ACFLOW_STORE_CREDENTIALS
Provide a default for
--store-credentials
- ACFLOW_TEMPDIR
Provide a default for
--temp-dir
- ACFLOW_S3_MAX_WORKERS
Provide a default for
--s3-max-workers
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
- ACFLOW_OVERWRITE
Provide a default for
--overwrite
- ACFLOW_WAIT_PERIOD
Provide a default for
--wait-period
- ACFLOW_LOOP
Provide a default for
--loop
accessionflow check-store folder¶
Checks what has been deposited in a folder data store against the staging directory it was deposited from, and logs every resource that is missing, incomplete or different. Exits with a non-zero status if there are any.
STAGED is the staging directory, or an S3 bucket to download the accessions from.
DEST is the root of the store, a path or an fsspec URL, e.g. 's3://bucket/archive'.
Usage
accessionflow check-store folder [OPTIONS] STAGED DEST
Options
- --storage-option <key> <value>¶
An option for the fsspec file system of the store, e.g. '--storage-option anon true' or '--storage-option key <access-key>'. Values that are JSON are decoded. May be repeated (ACFLOW_STORAGE_OPTIONS env. var, as space-separated pairs)
- --require-manifest, --dont-require-manifest¶
Whether the staged resources must have manifests. If not, the checksums of resources without one are worked out from their files (ACFLOW_REQUIRE_MANIFEST env. var)
- --store-credentials <access-key> <secret-key>¶
Credentials of the S3 bucket the accessions are staged in, if STAGED is one (ACFLOW_STORE_CREDENTIALS env. var)
- --temp-dir <temp_dir>¶
The directory to download accessions staged in S3 to
- --s3-max-workers <s3_max_workers>¶
The number of files to download from an S3 staging bucket at once (ACFLOW_S3_MAX_WORKERS env. var)
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
Additional loggers to capture the logs of, besides 'accessionflow' (e.g. 's3fs') (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
- --checksums, --sizes-only¶
Whether to check the files' checksums, which reads every file back from the store, or only compare their sizes, which is much cheaper (ACFLOW_CHECK_CHECKSUMS env. var)
Arguments
- STAGED¶
Required argument
- DEST¶
Required argument
Environment variables
- ACFLOW_STAGED
Provide a default for
STAGED
- ACFLOW_DEST
Provide a default for
DEST
- ACFLOW_STORAGE_OPTIONS
Provide a default for
--storage-option
- ACFLOW_REQUIRE_MANIFEST
Provide a default for
--require-manifest
- ACFLOW_STORE_CREDENTIALS
Provide a default for
--store-credentials
- ACFLOW_TEMPDIR
Provide a default for
--temp-dir
- ACFLOW_S3_MAX_WORKERS
Provide a default for
--s3-max-workers
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
- ACFLOW_CHECK_CHECKSUMS
Provide a default for
--checksums
XNAT¶
accessionflow label xnat¶
Labels accessions for XNAT data stores with their project/subject/session labels, extracted from accession metadata, after they have been collated into scans/resources
INPUT_DIR is the path to the directory containing the collated-but-not-yet-labelled accessions (the output of the 'collate' command)
OUTPUT_DIR is the directory that the labelled accessions will be written to, each in a directory named '<project>.<subject>.<session>'
If a label can't be resolved from an accession's metadata, it is given a unique 'INVALID_MISSING_<FIELD>_<random>' placeholder instead of being dropped, and the whole accession is saved under an '__invalid__' subdirectory of OUTPUT_DIR instead of alongside normally-labelled ones, so it can be found and manually reviewed/reprocessed rather than silently lost.
Usage
accessionflow label xnat [OPTIONS] INPUT_DIR OUTPUT_DIR
Options
- --project <label_0>¶
The keyword of the metadata field to extract the XNAT project ID from, or a Python format string over several fields, e.g. '{PatientID}_{StudyDate:%Y%m%d}', to compose one (ACFLOW_PROJECT env. var)
- --subject <label_1>¶
The keyword of the metadata field to extract the XNAT subject ID from, or a Python format string over several fields, e.g. '{PatientID}_{StudyDate:%Y%m%d}', to compose one (ACFLOW_SUBJECT env. var)
- --session <label_2>¶
The keyword of the metadata field to extract the XNAT session ID from, or a Python format string over several fields, e.g. '{PatientID}_{StudyDate:%Y%m%d}', to compose one (ACFLOW_SESSION env. var)
- --scan <acquisition_field>¶
The keyword of the metadata field to extract the XNAT scan type of each scan from. Scans for which the field can't be resolved are left without one (ACFLOW_SCAN env. var)
- --constant-project-id <constant_top_label>¶
Fix the XNAT project ID as a constant for all data matched by this command (ACFLOW_CONSTANT_PROJECT_ID env. var)
- --include <mime-type>¶
Only label resources matching this fileformats MIME-like datatype. May be repeated to include multiple datatypes (ACFLOW_INCLUDE env. var)
- --loop <loop>¶
Run the label process continuously every LOOP seconds (ACFLOW_LOOP env. var).
- --copy-mode <copy_mode>¶
The method to use for copying files (ACFLOW_COPY_MODE env. var)
- --unlink-source <unlink_source>¶
Whether to unlink the collated session directories after they have been labelled. 'all' removes the whole collated session directory; 'keep-metadata' removes the resource data but leaves the session/acquisition-level metadata behind, so a lightweight skeleton of the session survives (e.g. for 'attach' to use later). If not set, the collated directories are left in place (ACFLOW_UNLINK_SOURCE env. var)
- Options:
all | keep-metadata
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
The loggers to use for logging. By default just the 'accessionflow' logger is used. But additional loggers can be included (e.g. 'xnat') can be specified here (ACFLOW_ADDITIONAL_LOGGERS env. var)
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
Arguments
- INPUT_DIR¶
Required argument
- OUTPUT_DIR¶
Required argument
Environment variables
- ACFLOW_INPUT_DIR
Provide a default for
INPUT_DIR
- ACFLOW_OUTPUT_DIR
Provide a default for
OUTPUT_DIR
- ACFLOW_PROJECT
Provide a default for
--project
- ACFLOW_SUBJECT
Provide a default for
--subject
- ACFLOW_SESSION
Provide a default for
--session
- ACFLOW_SCAN
Provide a default for
--scan
- ACFLOW_CONSTANT_PROJECT_ID
Provide a default for
--constant-project-id
- ACFLOW_INCLUDE
Provide a default for
--include
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_COPY_MODE
Provide a default for
--copy-mode
- ACFLOW_UNLINK_SOURCE
Provide a default for
--unlink-source
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
accessionflow store xnat¶
Deposits the labelled accessions in a staging directory (as prepared by the label xnat sub-command) in an XNAT data store, by uploading them to it.
STAGED is either a directory that the files for each session are collated to before they are uploaded to XNAT or an S3 bucket to download the files from.
SERVER is address of the XNAT server to upload the scans up to. Can alternatively provided by setting the "ACFLOW_HOST" environment variable.
Usage
accessionflow store xnat [OPTIONS] STAGED SERVER
Options
- --user <user>¶
the XNAT user to connect with (alternatively the "ACFLOW_USER" env. variable can be used.
- --password <password>¶
the password for the XNAT user, alternatively "ACFLOW_PASS" env. var
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
The loggers to use for logging. By default just the 'accessionflow' logger is used. But additional loggers can be included (e.g. 'xnat') can be specified here (ACFLOW_ADDITIONAL_LOGGERS env. var).
- -i, --always-include <always_include>¶
Scan types to always include in the upload, regardless of whether they arespecified in a column or not. Specified using the scan types IANA mime-type or fileformats "mime-like" (see https://arcanaframework.github.io/fileformats/), e.g. 'application/json', 'medimage/dicom-series', 'image/jpeg'). Use 'all' to include all file-types in the session (ACFLOW_ALWAYSINCLUDE env. var).
- --date <date_field>¶
Session metadata field to use as the XNAT session date when creating a session. The value must resolve to one calendar date (ACFLOW_DATE env. var).
- --raise-errors, --dont-raise-errors¶
Whether to raise errors instead of logging them (typically for debugging)
- --store-credentials <access-key> <secret-key>¶
Credentials to use to access of data stored in remote stores (e.g. AWS S3) (ACFLOW_STORE_CREDENTIALS env. var)
- --temp-dir <temp_dir>¶
The directory to use for temporary downloads (i.e. from s3)
- --s3-max-workers <s3_max_workers>¶
Maximum concurrent S3 object downloads.
- Default:
10
- --xnat-max-workers <xnat_max_workers>¶
Maximum concurrent XNAT resource uploads.
- Default:
10
- --require-manifest, --dont-require-manifest¶
Whether to require manifest files in the staged resources or not (default: required)
- --verify-ssl, --dont-verify-ssl¶
Whether to verify the SSL certificate of the XNAT server
- --use-curl-jsession, --dont-use-curl-jsession¶
Whether to use CURL to create a JSESSION token to authenticate with XNAT. This is used to work around a strange authentication issue when running within a Kubernetes cluster and targeting the XNAT Tomcat directly (ACFLOW_USE_CURL_JSESSION env. var).
- --method <method> <datatype>¶
The methods to use to upload the file types to XNAT with. Passed through to XNATPy and controls whether directories are tarred and/or gzipped before they are uploaded, by default 'tgz_file' is used (ACFLOW_METHODS env. var).
- --wait-period <wait_period>¶
The number of seconds to wait since the last file modification in sessions in the S3 bucket or source file-system directory before uploading them to avoid uploading partial sessions (ACFLOW_WAIT_PERIOD env. var).
- --loop <loop>¶
Run the staging process continuously every LOOP seconds (ACFLOW_LOOP env. var).
- --num-files-per-batch <num_files_per_batch>¶
When uploading files to XNAT, the number of files to upload in each batch. The number of files that are uploaded in a single batch can be limited to avoid overloading the building of the catalog file. If <= 0 (the default), then all files are uploaded in a single batch (ACFLOW_NUM_FILES_PER_BATCH env. var).
- --check-checksums, --dont-check-checksums¶
Whether to check the checksums of the files in the staged resources against the checksums of both the checksums saved in the manifests and verify after upload to verify that they were uploaded correctly (if checksums are enabled site-wide on XNAT, i.e. enableChecksums is set to true in the XNAT configuration)
- --dry-run, --no-dry-run¶
List the sessions that will be uploaded instead of the actually uploading them
- --modality <modality>¶
Specify the modality of the sessions to upload (ACFLOW_MODALITY env. var).
- --no-uid-check¶
Don't compare a session's UID with the one recorded on an existing XNAT session. Only for the transition to hashed UIDs (see --hash-uid-key of the deidentify stage), as sessions already on XNAT hold their previous UID. The subject of the session is still checked, where it is known, and the UID on XNAT isn't changed (ACFLOW_NO_UID_CHECK env. var)
Arguments
- STAGED¶
Required argument
- SERVER¶
Required argument
Environment variables
- ACFLOW_STAGED
Provide a default for
STAGED
- ACFLOW_HOST
Provide a default for
SERVER
- ACFLOW_USER
Provide a default for
--user
- ACFLOW_PASS
Provide a default for
--password
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
- ACFLOW_ALWAYSINCLUDE
Provide a default for
--always-include
- ACFLOW_DATE
Provide a default for
--date
- ACFLOW_STORE_CREDENTIALS
Provide a default for
--store-credentials
- ACFLOW_TEMPDIR
Provide a default for
--temp-dir
- ACFLOW_S3_MAX_WORKERS
Provide a default for
--s3-max-workers
- ACFLOW_XNAT_MAX_WORKERS
Provide a default for
--xnat-max-workers
- ACFLOW_REQUIRE_MANIFEST
Provide a default for
--require-manifest
- ACFLOW_VERIFY_SSL
Provide a default for
--verify-ssl
- ACFLOW_USE_CURL_JSESSION
Provide a default for
--use-curl-jsession
- ACFLOW_METHODS
Provide a default for
--method
- ACFLOW_WAIT_PERIOD
Provide a default for
--wait-period
- ACFLOW_LOOP
Provide a default for
--loop
- ACFLOW_NUM_FILES_PER_BATCH
Provide a default for
--num-files-per-batch
- ACFLOW_CHECK_CHECKSUMS
Provide a default for
--check-checksums
- ACFLOW_DRY_RUN
Provide a default for
--dry-run
- ACFLOW_MODALITY
Provide a default for
--modality
- ACFLOW_NO_UID_CHECK
Provide a default for
--no-uid-check
accessionflow check-store xnat¶
Checks what has been deposited in an XNAT data store against the staging directory it was deposited from, and logs all files that haven't been
STAGED is either a directory that the files for each session are collated to before they are uploaded to XNAT or an S3 bucket to download the files from.
SERVER is address of the XNAT server to upload the scans up to. Can alternatively provided by setting the "ACFLOW_HOST" environment variable.
Usage
accessionflow check-store xnat [OPTIONS] INPUT_DIR SERVER
Options
- --user <user>¶
the XNAT user to connect with (alternatively the "ACFLOW_USER" env. variable can be used.
- --password <password>¶
the password for the XNAT user, alternatively "ACFLOW_PASS" env. var
- --logger <logtype> <loglevel> <location>¶
Setup handles to capture logs that are generated (ACFLOW_LOGGERS env. var)
- --additional-logger <additional_loggers>¶
The loggers to use for logging. By default just the 'accessionflow' logger is used. But additional loggers can be included (e.g. 'xnat') can be specified here (ACFLOW_ADDITIONAL_LOGGERS env. var).
- -i, --always-include <always_include>¶
Scan types to always include in the upload, regardless of whether they arespecified in a column or not. Specified using the scan types IANA mime-type or fileformats "mime-like" (see https://arcanaframework.github.io/fileformats/), e.g. 'application/json', 'medimage/dicom-series', 'image/jpeg'). Use 'all' to include all file-types in the session (ACFLOW_ALWAYSINCLUDE env. var).
- --store-credentials <access-key> <secret-key>¶
Credentials to use to access of data stored in remote stores (e.g. AWS S3)
- --temp-dir <temp_dir>¶
The directory to use for temporary downloads (i.e. from s3) (ACFLOW_TEMPDIR env. var)
- --s3-max-workers <s3_max_workers>¶
Maximum concurrent S3 object downloads.
- Default:
10
- --xnat-max-workers <xnat_max_workers>¶
Maximum concurrent XNAT checksum requests.
- Default:
10
- --verify-ssl, --dont-verify-ssl¶
Whether to verify the SSL certificate of the XNAT server (ACFLOW_VERIFY_SSL env. var)
- --use-curl-jsession, --dont-use-curl-jsession¶
Whether to use CURL to create a JSESSION token to authenticate with XNAT. This is used to work around a strange authentication issue when running within a Kubernetes cluster and targeting the XNAT Tomcat directly (ACFLOW_USE_CURL_JSESSION env. var).
- --disable-progress, --enable-progress¶
Disable the progress bar
- --no-uid-check¶
Don't compare a session's UID with the one recorded on an existing XNAT session. Only for the transition to hashed UIDs (see --hash-uid-key of the deidentify stage), as sessions already on XNAT hold their previous UID. The subject of the session is still checked, where it is known, and the UID on XNAT isn't changed (ACFLOW_NO_UID_CHECK env. var)
Arguments
- INPUT_DIR¶
Required argument
- SERVER¶
Required argument
Environment variables
- ACFLOW_STAGED
Provide a default for
INPUT_DIR
- ACFLOW_HOST
Provide a default for
SERVER
- ACFLOW_USER
Provide a default for
--user
- ACFLOW_PASS
Provide a default for
--password
- ACFLOW_LOGGERS
Provide a default for
--logger
- ACFLOW_ADDITIONAL_LOGGERS
Provide a default for
--additional-logger
- ACFLOW_ALWAYSINCLUDE
Provide a default for
--always-include
- ACFLOW_STORE_CREDENTIALS
Provide a default for
--store-credentials
- ACFLOW_TEMPDIR
Provide a default for
--temp-dir
- ACFLOW_S3_MAX_WORKERS
Provide a default for
--s3-max-workers
- ACFLOW_XNAT_MAX_WORKERS
Provide a default for
--xnat-max-workers
- ACFLOW_VERIFY_SSL
Provide a default for
--verify-ssl
- ACFLOW_USE_CURL_JSESSION
Provide a default for
--use-curl-jsession
- ACFLOW_DISABLE_PROGRESS
Provide a default for
--disable-progress
- ACFLOW_NO_UID_CHECK
Provide a default for
--no-uid-check