The overlay principle
Most institutions will not, and should not, replace a working EDRMS to add AI. That single sentence drives everything else in the IT Implementation Guide this article summarises. The right architecture is a governance and intelligence overlay: it reads from and writes back to the existing EDRMS through that system's own API, runs AI processing (extraction, enrichment, retrieval) in a separate scalable tier, routes end-of-active-life records into archival custody in AtoM, and keeps a human-in-the-loop checkpoint before anything is committed to the system of record.
The division of responsibility is deliberate. The EDRMS remains the system of record. The overlay tier holds AI-generated metadata, embeddings and audit trails - derived data that can be rebuilt from the corpus if lost. AtoM is the archival custody endpoint. Heratio, the open-source platform I work on at The Archives and Heritage Group, realises the overlay tier and the archival-integration layer, but the architecture stands independently of any single product.
Network-wise, I recommend four zones. A DMZ carries the public-facing nginx, Cloudflare and TLS termination, and is treated as untrusted. An application zone holds PHP-FPM, Python workers, the AI gateway and the EDRMS connectors. A data zone contains MySQL or Postgres, the vector database and the object store, at restricted trust. The fourth zone - AI/GPU - is isolated: model serving needs GPU memory and high-bandwidth access to the data zone, but it must never be directly addressable from the public internet. Where models are hosted locally for data-sovereignty reasons, this zone never reaches a third-party cloud inference endpoint at all.
Inside those zones, service-to-service traffic runs over mutual TLS with short-lived per-service certificates from an internal CA. Linkerd or Istio with cert-manager gives you encrypted internal traffic, per-call service identity, automatic retry, timeout and circuit-breaking behaviour, and observability essentially for free.
What the platforms already do
Any sponsor or auditor will ask the obvious question first: doesn't SharePoint already do this? Increasingly, yes. SharePoint Premium (formerly Syntex) handles document processing, classification and metadata extraction inside SharePoint libraries. Microsoft Purview provides sensitivity labels, retention labels and policies, and eDiscovery with legal hold. Microsoft 365 Copilot uses Microsoft Graph and the user's existing permissions for natural-language assistance. OpenText Aviator gives AI-assisted search and summarisation across supported OpenText platforms. Alfresco can be extended with Intelligence Services and AI-enabled content services, depending on version and add-ons.
For an institution running an all-Microsoft 365 estate of current office documents, with the budget to license Copilot and Premium, a separate AI overlay adds little value. The right answer there is to switch on the native capability and govern it well. Being plain about that is what makes the case for an overlay credible where one is genuinely warranted.
The native tools share three assumptions: that records are active business documents with a file plan, that they sit in a single vendor's repository, and that processing in the vendor's cloud is acceptable. Across much of the South African public-sector and archival landscape, those assumptions break. Inherited archival collections arrive with no metadata, disturbed original order and partial provenance - an arrangement-and-description task the platforms do not attempt. Hybrid estates span SharePoint plus a legacy OpenText repository plus shared drives, and each vendor's AI sees only its own content. Sovereignty matters too: processing within a vendor's service boundary is not the same as guaranteed in-country processing, and any processing or replication outside South Africa can trigger POPIA section 72. Finally, per-user AI licensing across thousands of public-sector seats is costly, and proprietary AI exposes limited model-level internals when institutions need openly auditable evidence for POPIA access and correction rights.
So the decision compresses to one line: use native platform AI for active documents inside a single vendor's environment where its cloud is acceptable; add an overlay only for inherited archives, content spread across several systems, and processing that must stay in-country and be auditable. Deliberately, the overlay does not re-classify active SharePoint documents that Syntex already handles, does not duplicate the platform's retention engine, and does not replace native eDiscovery. It reads native classifications as input and adds only what the platforms leave open.
Integration patterns per EDRMS
Every connector follows the same contract: read content and existing metadata through the platform's own API, write AI-suggested metadata back only after human approval, and respect the EDRMS's permission model so the AI service never sees more than its service account is entitled to.
For SharePoint and Microsoft 365 - the most common SA public-sector environment - the connector is the Microsoft Graph API (/sites, /drives, /lists) with an app registration scoped to least privilege: Sites.Selected, not tenant-wide. Azure AI services and Azure OpenAI are options only where licensing, region and residency terms suit the data classification; personal-information records requiring in-country processing route to the local model tier instead. Write-back goes through managed metadata and the term store, into a proposed field promoted only on archivist approval. One caution deserves emphasis: SharePoint retention labels and the overlay's retention logic must not both act autonomously. The overlay flags and recommends; disposal follows the EDRMS workflow and the applicable disposal-authority process.
Alfresco connects through its REST API and CMIS, with the Alfresco event gateway (ActiveMQ) enabling near-real-time processing on ingest. It is model-agnostic - point the gateway at local Ollama or vLLM, or a permitted cloud endpoint. Write-back uses aspects and properties, with AI-generated aspects namespaced (ai:suggestedSubject, for example) so they stay visibly distinct from human-asserted metadata.
OpenText Content Manager (formerly HPE/HP TRIM) integrates via the ServiceAPI (REST) or the .NET SDK, respecting record-type and security-caveat models. In practice these deployments are on-prem, so local model hosting is the natural fit and often the compliance requirement. The connector never bypasses the disposal-authority workflow.
Hybrid environments get data federation. The overlay indexes and retrieves across all connected sources - SharePoint, Alfresco, OpenText, shared drives - and builds a single search index, tagging every entry with its source system so a result is always traceable to where the record lives. This is a discoverability layer, not a migration: content is not moved or merged, and provenance is preserved on every result.
The archival path is distinct. Records at the end of their active lifecycle, and material of enduring value, transfer to AtoM for governed custody under the NARSSA Act where applicable to South African public records. Records and accumulated metadata move without manual re-keying and without loss of provenance. The distinction matters: this is archival transfer, not disposal. Archival material never ages out toward deletion; cold storage means infrequent access, never a step toward destruction. Heratio realises this path as open source - AtoM extended with the ahg-retrieval, ahg-statistics, ahg-term-taxonomy, ahg-storage-manage and ahg-user-manage modules, plus Laravel services and local model hosting.
Sizing the overlay tier
Sizing here covers the overlay only; the EDRMS is the vendor's concern and should not be double-counted. Four dimensions drive it. CPU follows concurrent users, at roughly one core per 50-100 concurrent users. RAM runs about 4 GB per core for PHP plus database buffers. Disk is roughly 1.5 times the metadata and index footprint, because the corpus itself stays in the EDRMS or object store. Network needs 1 Gbps minimum, 10 Gbps for the AI gateway.
Three worked examples anchor the scale, all indicative rather than measured. A small municipal records office (50 users, 1M records, light AI) runs on two app servers at 4 vCPU and 16 GB each, one database server at 4 vCPU and 32 GB with 2 TB NVMe, and one AI gateway with an optional 8 GB GPU. A medium government department (500 users, 10M records) steps up to four app servers, a primary-plus-replica database pair at 8 vCPU and 64 GB, and two AI gateways each with a 24 GB GPU. A large national archive (5000+ users, 100M+ records, intensive AI) needs 8-16 app servers, a four-node database cluster at 16 vCPU and 128 GB each, four AI gateways with 80 GB A100s, and a dedicated training and embedding server with two more A100s.
The planning rule: provision for twice current peak load, not current average. A successful pilot generates more traffic than the pilot itself, and over-provisioning at 2x is cheaper than a 2am emergency scale-up.
The runtime baseline
The baseline OS is Ubuntu 24.04 LTS, with standard security maintenance to May 2029, shipping kernel 6.8, PHP 8.3 and Python 3.12 in the release repositories. Kernel tuning for AI workloads raises connection backlogs (net.core.somaxconn to 4096), widens TCP buffers to 16 MB, drops vm.swappiness to 10, and lifts file-handle and conntrack limits substantially. On an 8 vCPU overlay server, PHP-FPM runs dynamic with pm.max_children at 100 and pm.max_requests at 1000. Python AI gateways use gunicorn with two workers and four threads (gthread class) per vCPU, fronted by nginx for TLS and rate limiting - 10 requests per second per IP on the API zone, 1 per second on login.
For the database layer I recommend Postgres 16 over MySQL for the overlay store. The pgvector extension lets embeddings live beside the derived metadata, removing a class of consistency issues, and Postgres brings JSONB, tsvector full-text search and logical replication. (AtoM itself ships on MySQL; the overlay's store is separate and can be Postgres regardless.) The embeddings table carries a source_system column - sharepoint, alfresco, opentext or atom - which is precisely what makes federated retrieval provenance-aware, with an HNSW index (m = 16, ef_construction = 64) over 768-dimension vectors. Beyond roughly 10M embeddings, or when you need hybrid sparse-plus-dense search or sharding, move to Qdrant. Redis covers caching, the Laravel queue backend for AI jobs, and distributed rate limiting.
Model serving offers three runtimes: Ollama for single-server, development and light production (Heratio's default local host); vLLM with PagedAttention where you need the best throughput per GPU; llama.cpp for CPU-only and edge deployments. Local hosting is the sovereignty default for personal-information records - the relevant test is the residency of the processing, not whether data "leaves the tenant". Pin model versions explicitly (qwen2.5:7b-instruct-q5_K_M, never latest), pre-download and cache models, and use queue-based batching: accept a request, wait up to about 50ms for more, batch them into one model call. One operational lesson worth passing on: if the overlay reaches Ollama through a reverse-proxy path, health-check the generation endpoint separately from retrieval. A 404 on generation while retrieval still works means the proxy route or model load has failed, not the search stack.
Security, observability and recovery
Security follows seven layers of defence in depth, from network segmentation and WAF through identity (SSO via OIDC or SAML, MFA, RBAC), application controls, encryption, endpoint hardening, supply chain and operations. Secrets live in HashiCorp Vault or a cloud secrets manager, rotated every 90 days for service accounts and 30 for elevated human users. The build pipeline targets SLSA Level 2, with container images signed via Cosign and Sigstore. RBAC defines roles per context - records officer, archivist, records manager, auditor, admin - and critically, the AI service account is read-only against records and can never exceed what the EDRMS's own permission model grants. Every AI-assisted action is logged append-only with model version, input, output, confidence and the human reviewer, forwarded to a separate store such as Loki or Elasticsearch.
Observability runs on the standard three pillars - Prometheus and Grafana for metrics, Loki or Elasticsearch for logs, Tempo or Jaeger for traces - instrumented through OpenTelemetry, including the EDRMS connector calls. The SLO for records search is 99.5% availability over 30 days with p95 latency under 500ms, and here the guide makes a point I have not seen elsewhere: this is a compliance metric, not just an ops metric. A discoverability deficiency can put an institution in practical PAIA non-compliance even when it holds the record, because the section 25 duty to respond within 30 days is not met if the record cannot be found in time.
Disaster recovery carries the same sovereignty constraint as inference. Replicating personal-information records outside South Africa is a trans-border transfer that must satisfy POPIA section 72 before it is configured; do not enable offshore DR on convenience alone. Institutions on the local-hosting path keep DR in-country by design. RTO and RPO targets tier from one hour and five minutes for records search and the AI gateway down to 24 hours for reporting. Backups follow 3-2-1 with WORM object lock on the corpus bucket, a quarterly restore of a random 100-record sample with integrity checks, and an annual full restore into a clean environment. The corpus is the one asset you can never lose; the overlay database is a derived view, reconstructible from the corpus and source-system metadata, and models are simply replaceable artifacts.
The full IT Implementation Guide, including reference nginx and Postgres configurations, the hardware sizing calculator and the complete discovery question sets, is available to download on this page.
Comments (0)
No comments yet. Be the first to comment.