Eric Mumford
Automation Architect · Quality Engineering & AI Systems
- eric@mumfordengineering.com
- mumfordengineering.com
- Albany, New York
Summary
I have worked in software quality and automation for thirty years, currently as an automation architect at a national mortgage lender. I start from the outcome a business needs, work out what capabilities will get there, and then build, staff, and measure the plan — the same method whether the deliverable is a test platform, a data pipeline, or an engineering team. For the past few years I have used that method to build infrastructure where AI agents carry much of the engineering work themselves — they design, build, review, and deliver software around the clock, and the systems that coordinate them are mine. Everything below is my own implementation, running in production — in Python, Java, C#, F#, Go, and TypeScript, released to PyPI and internal registries.
Leadership & Delivery
- Build & lead teams
- Stand up quality and automation practices where none existed — from headcount plans and hiring through distributed standups, code review, and executive reporting. Built and led functional, test-data, and performance automation teams across regulated enterprises.
- Quality strategy & execution
- Translate business outcomes into capability maps, tool evaluations, and phased multi-year plans, and deliver each phase on schedule.
- Change management
- Led framework migrations and toolchain consolidations end to end in regulated environments, including change-management tracking, sized rollouts, and training plans.
- Architecture
- Designed target test architectures and platforms — a multi-tier test-authoring platform, a multi-agent engineering system, and the graph and telemetry layers beneath them.
- Thought leadership
- Writing a book on quality engineering; developed a documented model for dividing testing work between people and AI agents; published an open-source model-calibration library.
- Hands-on delivery
- Write production code, tests, and CI pipelines directly; the systems below are my own implementations, released to PyPI and internal registries.
unified-comms — coordination infrastructure for fleets of AI agents
When several AI agents work on the same software, they need what any team needs: a way to talk privately, divide up the work, record who decided what, and keep going when a machine fails or the network disappears. I designed and built that backbone, and I run my own engineering on it — a fleet of agents across two machines has coordinated through it continuously for months, including overnight with no one watching. It needs no cloud account and no central server, and it keeps working with no internet at all. The message bill is zero.
- The business problem Cloud message buses charge per call, expose every conversation to a third party, and stop working when the network does. I needed a fleet of coding agents on my own machines to collaborate privately and reliably — offline, at zero marginal cost, with no central service to run, pay for, or trust.
- What it does Two agents pair with a six-digit code, like pairing a device — no accounts, no configuration. From there they find each other automatically and exchange private messages in under a second, on one machine, across a room, or with no network at all. Beyond messaging, agents claim and release units of work, bid for tasks in an auction designed so that exaggerating ability wins nothing, and record decisions in a permanent register. The mesh forgets the way people do: routine chatter fades, claims are archived, decisions are kept.
- Running in production 69 agents registered across two machines; the message ledger spans three months of operation. Background services keep the mesh alive around the clock, and the agents coordinating through it carry work from review to merge while I sleep; an escalation path notifies me when an agent's question goes unanswered too long. At night the mesh switches modes: a review panel settles routine disputes automatically and holds judgment calls for the morning. A live dashboard renders the mesh and its message flow in 3D, so the coordination is visible rather than hidden.
- Under the hood Two transports — filesystem (<1 ms same metal) and WebSocket (<10 ms LAN) — behind one five-method API, with mDNS peer discovery and per-peer offline buffers that replay on reconnect. Security is X25519 ECDH + HKDF-SHA256 key exchange and Ed25519-signed AES-256-GCM envelopes, adversarially reviewed before implementation. The system of record is an append-only write-ahead log modeled as a grow-only CRDT — any two nodes reconcile by set union, no consensus, no coordinator — with hybrid logical clocks ordering events across processes. Thirteen typed speech acts (CLAIM, COMMIT, RELEASE, CONFLICT, DECISION…) carry lease/claim/release work-item semantics; a VCG auctioneer with Clarke-pivot pricing makes the auction strategy-proof — overstating capacity or value wins an agent nothing. Reliability is lock-free circuit breakers, per-sender gap-buffer dedup, and idempotent delivery; a decision register moves PENDING → DISPUTED → RESOLVED under day-shift/night-shift operator modes, and ledger consolidation applies per-message-type exponential decay (FORGET / ARCHIVE / REMEMBER). Eleven MCP tools expose it to any agent runtime; the dashboard is Starlette + HTMX + Three.js. 2,033 tests across 348 files including property-based invariants; 10 accepted ADRs.
- Design notes What's unusual is the pairing: replication without consensus (CRDTs settle the shared record) combined with mechanism design (auctions settle who does the work), in a mesh that treats the network as optional.
Claude Code Skill Platform
A set of composable skills and role-specialized agents I wrote for Claude Code that runs parts of engineering work autonomously — covering product ownership, architecture, and test generation as reusable, versioned capability.
- Product ownership Skills that turn intent into product definitions, business-outcome roadmaps, and architecture decision records — a product-intent steward, a roadmap builder, and crash-resilient file-based planning that survives interruption.
- Architectural structures A system-architect agent and a principal-architect review enforce clean boundaries and ADR-logged decisions; an adversarial reviewer attacks every consequential change before it ships, and architectural fitness functions gate merges.
- Test generation Red-green-refactor TDD builders, multi-dimensional verification matrices, and N-wise combinatorial generation produce failing tests first, then the minimal code to pass — under fail-closed, independent verification, so a model never grades its own homework.
- Autonomous delivery A router classifies intent and dispatches parallel role-specialized agents; the platform carries work from design through review to merge on local models at near-zero API cost.
AI Engineering & Autonomous Systems
A multi-agent system for engineering work: role-specialized agents governed by versioned policy, run by background automation, served mostly by local models, and coordinated over an encrypted mesh — discovery and decision records through design review, planning, build, verification, and delivery.
- .NET 10 automation platform The engineering quality lifecycle as code: 34 modules and 6 shipped artifacts — a test-primitives library, a measurement engine, a vendor-agnostic AI cognition layer, coverage-mining and requirements adapters, Roslyn architecture analyzers, and an ASP.NET Core dashboard. 219 declarative fitness functions (versioned YAML, schema-validated) gate merges against architectural drift; every metric publishes its formula and provenance per ISO/IEC/IEEE 15939. 73 ADRs; 344 test files across 32 projects; OpenTelemetry throughout.
- cognilateral-trust (PyPI, Apache-2.0) Measures the gap between a model's stated and observed accuracy (Brier score, expected calibration error against a 200-scenario benchmark) and routes each decision to act, verify, or require human approval. Every decision lands in a tamper-evident, SHA-256-linked chain; a welfare constraint hard-blocks wellbeing-affecting actions regardless of confidence. Integrates with LangGraph, CrewAI, OpenAI, Anthropic, and MCP.
Quality & Testability Engineering
I encode quality criteria as automated checks that run continuously, and track how the results move over time.
- jidoka (PyPI, Apache-2.0) A testability-assessment engine: eight-tier interrogation, seven scoring dimensions, 31 fitness rules and 40 domain ontologies across seven regulated industries, and eight tree-sitter language parsers feeding a graph model. Runs air-gapped and deterministically and emits JSON, an HTML certificate, and SARIF 2.1.0 for CI gating.
- Test-execution telemetry pipeline Consolidates test-execution data from individual testers and CI runners into datasets for BI. Runs on Alpine Python in CI with no install step, with incremental state tracking and package-registry delivery.
- Coverage & ownership graph A Neo4j graph and telemetry store linking coverage, flakiness, and ownership, so those become measured signals rather than after-the-fact reports.
Automation Strategy & Implementation Planning
Establish the business outcome, define the capabilities it requires, evaluate available tools against them, and deliver a sequenced plan in which each phase produces usable capability.
- A national mortgage lender Designed a three-year automation and continuous-verification roadmap: mapped the existing test estate against a target architecture, evaluated framework options, and ranked a debt-laden Robot Framework codebase by business risk to guide refactor / retire / keep, tooling, and staffing decisions.
- Karate migration (consulting) Assessed a client's legacy test codebase, weighed maintainability against migration cost, authored the migration proposal, then implemented reusable feature files for authentication, data ingestion, and analytics.
- Homesite Insurance Consolidated a fragmented QA toolchain (Quality Center, UFT, JMeter, ad-hoc scripts) onto a single platform, with a sized rollout and training plan.
- QA-factory diagnosis Measured delivery friction across roughly 960 work items using DMAIC and ISO 15939, and identified approval bounce — not defect volume — as the main source of delay.
Legacy Systems & Terminal Automation
I automate testing for legacy systems that lack modern interfaces.
- hti5250j (released) Headless IBM i (AS/400) 5250 terminal automation: removed the Swing/AWT dependency so tests run as a pure library in Docker and CI. Java 21 virtual threads carry 1,000+ concurrent sessions and 300+ workflows/second — roughly 50× throughput — with session pooling, a YAML workflow DSL, 23+ EBCDIC codepages, and property-based (jqwik) plus chaos (resilience4j) testing that needs no live mainframe.
- lom-runtime Deterministic legacy automation in Go: replaced timing-based waits with explicit state contracts — a workflow may not sleep, and every failure must be explainable by observed state — plus PII auto-classification with deterministic, seed-locked masking so masked datasets reproduce exactly in CI.
- Hardware validation (earlier career) Validated cache-coherency for DEC's TSUNAMI chipset behind the Alpha 21264 processor, testing 256-bit memory buses at 83 MHz.
Platform & Data Engineering
- cortical Reconstructs a workflow ontology from issue event streams and reports bottlenecks, rework loops, and structure/architecture misalignment. Falsification-driven: detectors auto-retire when their signal rate fails a Popperian threshold. Neo4j-backed, 192K nodes, 2,827 tests.
Technical Range
- Languages
- Python · Java 21 · C#/.NET · F# · Go · TypeScript · Cypher · Bash
- Frameworks & test
- Playwright · Selenium · Robot Framework · Karate · Cucumber/BDD · pytest · JUnit · property-based (Hypothesis, jqwik) · FastAPI · ASP.NET Core · Spring Boot · React/Next.js
- Data & platform
- PostgreSQL · Neo4j · DuckDB · SQLite · Model Context Protocol · OpenTelemetry · event sourcing · CRDTs · Docker · Kubernetes · AWS · Azure · Terraform · GitHub Actions · GitLab CI · Prometheus · Grafana
Experience
Automation Architect
Oct 2024 – presentA national mortgage lender (Remote)
- Lead QA automation and a multi-year modernization program.
Consultant — Quality Engineering & Automation Platforms
Oct 2023 – Nov 2024Various clients (Remote)
- Quality strategy and automation for distributed web-service teams; led a Karate-framework migration.
Director, Quality Management
Oct 2022 – Jun 2023Broadview Federal Credit Union
- Owned the test program for a regulated Fiserv DNA core-banking migration.
Director, QA & Release Management
Jan 2022 – Oct 2022Data Skrive
- Led QA and release management for an automated content and analytics platform.
Enterprise Architect, Engineering Quality Platforms
Aug 2020 – Jan 2022American Family Insurance
- Led functional, test-data, and performance automation teams.
- Built a cross-platform code-integrity toolkit and a cloud policy-quote load simulator on AWS.
Architect, Director & Senior Engineering roles
1995 – 2020Insurance, fintech, analytics & enterprise hardware
- Homesite Insurance, Gartner, Bridgewater Associates, Sapient, and Digital Equipment Corporation.
- Exemplar-based validation frameworks for financial analytics, and a DB2 change-and-test orchestration system spanning OS/390 mainframe and a Windows mid-tier.
Education & Certifications
- B.S., Electrical Engineering — Rensselaer Polytechnic Institute (RPI)
- Certified Software Quality Engineer (CSQE) — American Society for Quality (ASQ)
- AWS Certified Cloud Practitioner (2020–2023) · Hexawise Certified Professional
- Continuing coursework: Introduction to Quantum Computing, RPI (2025)