Ga naar hoofdinhoud

Batch Anonymization

Status: Proposed​

Batch anonymization extends the single-document anonymization pipeline to support processing multiple files in one guided session. The workflow follows a state-machine model: upload → extract → review → anonymize → completed.

Overview​

Users can upload up to 100 files (admin-configurable) in a single request. Filinq processes them sequentially, extracting text and entities from each file, then presenting a consolidated entity review before applying anonymization. A CSV audit report is available for download after completion.

Batch state is persisted in Nextcloud ICache with a 2-hour TTL. No batch data is stored permanently; only the anonymized output files are saved to the user's Filinq folder.

Workflow Steps​

  1. Upload — POST /api/anonymization/batch/upload — upload multiple files, receive batchId
  2. Extract — POST /api/anonymization/batch/{batchId}/extract — process one file per call until all are extracted
  3. Review — Entity review — consolidated entity list with toggle controls
  4. Anonymize — POST /api/anonymization/batch/{batchId}/anonymize — apply anonymization with reviewed entity list
  5. Report — GET /api/anonymization/batch/{batchId}/report — download CSV audit report

API Endpoints​

MethodPathDescription
POST/api/anonymization/batch/uploadUpload multiple files; returns batchId
POST/api/anonymization/batch/{batchId}/extractExtract next unprocessed file in batch
GET/api/anonymization/batch/{batchId}/statusPolling endpoint — returns batch status and per-file progress
GET/api/anonymization/batch/{batchId}/entitiesConsolidated entity list for review
POST/api/anonymization/batch/{batchId}/anonymizeApply anonymization with reviewed entity list. Body supports entities, outputFormat, appendBasisSummary, and scope (default dossier).
GET/api/anonymization/batch/{batchId}/reportDownload CSV audit report (post-completion)

Placeholder-numbering scope​

A batch is a folder/dossier, so POST /api/anonymization/batch/{batchId}/anonymize defaults to scope: "dossier": a given entity gets the same scope-local placeholder number across all files in the batch, so the redacted set reads as one unit. Pass scope: "document" to number each file independently instead. See Placeholder-numbering scope on the anonymization page for the full semantics.

Where the anonymised copies go​

Each redacted copy lands in a subfolder next to its original, under the original's name:

/Documents/Bezwaar-2026/
  besluit.pdf             (original, unchanged)
  anonymised/
    besluit.pdf           (anonymised copy)
  • The subfolder is anonymised unless an admin sets another name under Settings, Subfolder for anonymised copies (config key anonymisation.output_subfolder_name).
  • The name takes lowercase letters, digits, hyphens and underscores. Anything else is refused on save, and the error names the characters.
  • A rerun replaces a copy of the same name. Other files in the subfolder stay.
  • A copy of an old _anonymized file loses the suffix: besluit_anonymized.pdf becomes anonymised/besluit.pdf.
  • If the move fails, the copy stays beside the original as <name>_anonymized.<ext>. The file entry then carries warning.code MOVE_FAILED and the path where it stayed.
  • Anonymising a single document is unchanged: its copy stays beside the original with the _anonymized suffix.

Audit Report​

The CSV report includes: fileName, originalFileId, anonymizedFileId, entityCount, replacementCount, status, timestamp. Entity values are excluded (GDPR data minimization, Recital 26).

Standards​

  • GDPR / AVG — Batch state is transient (ICache TTL 2h); entity values excluded from audit report
  • WOO — Anonymization profiles aligned with WOO publication requirements
  • GEMMA Media-behandelingcomponent
  • TEC-DMS-7 (Workflow Management)

Limits​

ParameterDefaultConfig Key
Max files per batch100filinq_batch_max_files (IAppConfig)
Batch TTL2 hoursHardcoded (ICache)