Implementing AI in Records Management - Business perspective

Records and archives institutions are being asked to do something arithmetic will not allow: govern rapidly growing digital corpora with roughly the same number of professionals. The sums are simple enough. An institution holding a 5-terabyte records corpus that grows at 30 percent a year will be sitting on roughly 11 terabytes within three years. That growth rate is a planning assumption until you confirm it against your own storage reports, but the underlying mismatch is real - human capacity to classify, describe, and index records does not grow at the rate of digital storage. AI is one mechanism by which institutions can attempt to keep records governance intact as volumes expand.

This article distils a comprehensive implementation guide I have prepared for archival and records management institutions. It draws on four knowledge sources: my qualitative PhD research into AI/ML-enabled records accessibility on shared drives in a South African state-owned company; design work on the Heratio platform for AI-assisted archival and records workflows; the AHG four-day AI training curriculum; and the regulatory and standards environment - POPIA, PAIA, the NARSSA Act and the ECT Act in South Africa, the GDPR where EU data subjects are involved, and the ISO records and AI-governance standards (ISO 15489, ISO 23081, ISO 30301, ISO/IEC 23894, ISO/IEC 42001), together with the ICA Records in Contexts conceptual model.

One point of intellectual honesty up front, because the guide insists on it throughout. My PhD study was qualitative and exploratory - stakeholder consultation and thematic analysis. It did not empirically test an AI system or benchmark retrieval performance. Where the guide quotes figures such as accuracy targets or efficiency gains, these are illustrative planning targets to be measured against your own baseline, not empirical results. Any vendor or consultant who quotes you universal accuracy percentages without a local baseline should be treated with suspicion.

Where AI actually earns its keep

Across the research, the platform work and the training curriculum, five records operations consistently emerge as the highest-value applications. Automated classification assigns records to categories based on content. Metadata extraction populates fields - date, author, subject, reference number, department - from document text using natural language processing. Semantic search understands query intent rather than matching keywords, which matters when records are described in archival language that differs from user vocabulary: a search for "contract termination" should also return "cancellation of agreement". Retention schedule automation flags records approaching disposal dates, while the disposal decision itself remains a human, authorised act. And anomaly detection surfaces unusual patterns in metadata - missing fields, inconsistent classification, records accessed at odd hours - as indicators of data quality problems or security concerns.

The guide works through an illustrative municipal scenario: a records office managing around 4 million physical records, scoping a bounded pilot of 80,000 to 90,000 records over three months, with every AI suggestion reviewed by a records professional before being committed. The digits are deliberately hypothetical. The method is the point: define a bounded corpus, set measurable targets in advance, keep a human in the loop, measure against a baseline, and decide on expansion from evidence.

None of this replaces the records manager. In a supervised deployment the AI suggests and the human approves or overrides. As accuracy is demonstrated, the human moves from reviewing every record to reviewing only high-risk or uncertain cases, and the role evolves from operator to governor, auditor, and exception handler.

The technology, in the vocabulary you need

Records leaders do not need to build models, but they do need a working vocabulary to evaluate and govern them: machine learning, NLP, large language models, tokens, fine-tuning, and above all hallucination - an LLM generating plausible-sounding but fabricated information, a critical risk in a domain where accuracy is legally required. Retrieval-Augmented Generation (RAG) is the architecture that mitigates it: retrieve relevant records from the actual corpus first, then answer from that retrieved context, grounding the model in real data.

The guide also distinguishes three learning paradigms. Supervised learning needs accurately classified training data in volume; unsupervised learning finds its own patterns and suits early-stage corpora; self-supervised learning uses the data itself as the supervisory signal. Most South African public sector records environments are best served by self-supervised or hybrid approaches initially, because large volumes of consistently labelled training data are rarely available.

For retrieval, the guide proposes two evaluation labels for precision risks that arise specifically in archival systems. "Smear" describes false associations driven by entity-token sharing: two records mention the same pseudonymised officer or case reference, and the embedding model groups them even though they concern unrelated events. "Smudge" describes false positives driven by vocabulary overlap: a query and a record share similar language but concern different topics - an inherent hazard in multi-author, multi-decade archival description. These are proposed labels, not empirically measured defects, and they matter in archives precisely because records legitimately share entities through provenance, and because a false positive in an archival search may surface sensitive personal information from the wrong case.

The regulatory frame: POPIA and GDPR are cousins, not twins

Any AI deployment on records containing personal information sits inside data protection law, and South African institutions frequently have to satisfy both regimes. The comparison rewards care.

On automated decision-making, GDPR Article 22 gives data subjects the right not to be subject to solely automated decisions producing significant effects. POPIA's closest counterpart is section 71, which prohibits decisions with legal consequences, or that affect a person to a substantial degree, based solely on automated processing intended to profile them, subject to the s71(2) exceptions. AI classification that determines records access or disposal without human review engages both provisions - and meaningful human-in-the-loop review is the primary mitigation under each.

On breach notification, the regimes genuinely diverge. GDPR requires notification to the regulator within 72 hours. POPIA's section 22 requires notification "as soon as reasonably possible" after discovery - there is no fixed statutory hour-count, though the Information Regulator's guidance has informally treated around 72 hours as reasonable, and a separate Cybercrimes Act 72-hour SAPS report may apply. Critically, POPIA has no materiality threshold: even potential exposure can trigger the duty. Build one incident workflow, but apply each regime's actual standard rather than assuming identical timelines.

Other load-bearing provisions include POPIA section 72 on cross-border transfer (cloud AI processing records offshore needs a valid transfer ground, mirrored by GDPR Chapter V), section 26 on special personal information (health, biometric, criminal history - common in public sector records and prohibited to process except on limited s27 grounds), and section 55's requirement that the Information Officer be registered and involved in AI deployment decisions. GDPR requires a Data Protection Impact Assessment under Article 35 before high-risk processing; POPIA has no mandatory DPIA equivalent, but section 57 prior authorisation can be triggered. The guide's advice: conduct a DPIA regardless, and check s57. On anonymisation, information falls outside POPIA once the data subject can no longer be identified by reasonably foreseeable means; apply GDPR Recital 26 as the best-practice standard - irreversible de-identification, documented well enough to demonstrate to the Regulator.

The standards layer wraps around the law. ISO 15489 and ISO 23081 mean AI-generated metadata must meet the same reliability, integrity, and authenticity standards as human-generated metadata. ISO 30301 supplies the Plan-Do-Check-Act management wrapper. ISO/IEC 23894:2023 provides AI-specific risk management, naming risk categories directly relevant to records work: misclassification of sensitive records, false-positive and false-negative disclosure risk, unintended disposal recommendations, and model drift. ISO/IEC 42001:2023 - the AI management system standard - is the institutional accountability wrapper, placing AI use inside documented policy, roles, monitoring, and continual improvement rather than ad hoc algorithmic processing. The NIST AI Risk Management Framework (Govern, Map, Measure, Manage) complements rather than replaces them. And in South Africa, the NARSSA Act's sections 13 and 16 remain decisive: AI may flag records as due for disposal, but destruction of public records requires the National Archivist's authorisation. The ECT Act governs the legal recognition and evidential weight of the electronic records AI helps produce and describe.

One provision deserves particular attention from leadership: PAIA section 25 imposes a 30-day duty to respond to access requests. An institution can hold a record in its register yet be in practical PAIA non-compliance because the record cannot be located within the statutory window. Improving discoverability is not merely an efficiency gain; it is a direct compliance control.

Archival theory must drive the design

AI tools for records are often built by technology teams who are not archival specialists, optimising for keyword recall or speed without understanding provenance, context, and chain of custody. The guide's position is firm: Records in Contexts must govern how AI is designed, deployed, and evaluated. RiC's principles - entity differentiation between descriptive and contextual entities, multiplicity of relationships, the fonds-series-file-item provenance hierarchy, and description as evidence in its own right - are exactly the structures naive AI violates. AI-generated summaries are not archival description, and the distinction must be maintained. AI that ignores provenance provides misleading, not improved, access.

Heratio, the AHG platform design for AI-enabled archival workflows, operationalises RiC through four design decisions: entity tracking that maps extracted entities to archival description fields and authority records; provenance-aware retrieval that respects hierarchy; human-in-the-loop metadata review, so the archivist retains authority over description; and local model hosting where configured, so institutional records can remain on the institution's own infrastructure - a genuine reduction of cross-border and data-residency risk under POPIA and GDPR, though a risk reduction is not automatic legal compliance. These are platform design assertions, to be verified against a deployed instance before being cited as compliance evidence.

Privacy first, then governance, then rollout

Anonymisation is a prerequisite for AI evaluation, not a compliance step bolted on at the end. The guide sets out a four-stage pipeline: identifier extraction using named entity recognition; deterministic pseudonymisation, where each unique identifier maps consistently to a token and the mapping table is held separately from the corpus; date coarsening to month-year or year where precision could re-identify; and a manual QA pass by two independent reviewers hunting residual identifiers such as unusual job titles or geography narrowing to a single property. Residual risk is then tested - token frequency analysis, co-occurrence inference, date-location pairs - against a proposed privacy-leak proxy score on a 0-100 scale, with a working target below 10 for evaluation corpora. The guide is explicit that this is an operational protocol, not formal k-anonymity or differential privacy; that requires a specialist audit.

Governance sits inside a matrix assigning each risk dimension - privacy, algorithmic bias, transparency, accountability, security, compliance - a mitigation and a named owner, from the Information Officer to IT security. Human-in-the-loop review rests on three principles: it must be mandatory for any output affecting classification, accessibility, or disposal; it must be meaningful, performed by people with the training and authority to override the AI, otherwise the review is performative; and it must be risk-proportionate, with senior review for sensitive, privileged, or access-restricted records. Every AI-assisted action should be logged - suggestion, confidence score, human decision, override reason, timestamp, user - creating the accountability evidence POPIA and GDPR both demand.

Implementation itself starts with a 20-item readiness audit across data quality, infrastructure, governance, and people, scored out of 100. Above 80, proceed to pilot; below 40, do the foundational work first - AI will not fix a broken records programme. From there the guide lays out a 12-month phased roadmap: foundation (readiness audit, DPIA, classification scheme clean-up), a bounded pilot on two record series with a measured accuracy baseline, expansion to eight to ten series with retention automation and e-discovery, and an optimisation phase assessing whether more autonomous classification is viable. Every phase has a decision gate, and every success metric is a target to be measured, not assumed. Evaluation continues after go-live: a corpus-specific benchmark of at least 100 representative queries judged by at least two expert annotators, because generic IR benchmarks such as TREC and MS MARCO do not represent archival provenance, access restrictions, or local descriptive conventions.

The records manager's role does not shrink through any of this; it moves up a level, into AI governance, exception handling, training-data management, privacy operations, and evaluation. The closing thought of the guide is the one I would leave with any leadership team weighing an AI initiative: AI in records management is not a technology project. It is a records governance project that uses technology. The technology will change, the standards will evolve, the regulatory landscape will shift - but provenance, context, accountability, privacy, and the primacy of the record endure.

The full guide, including the governance matrix, the comparative GDPR-POPIA tables, the readiness audit, the 12-month roadmap, and the training handouts, is available to download on this page.