FACTS, LIES, AND ALGORITHMIC GOVERNANCE
Quality Thinking for the Agentic AI Era
— 2nd Edition of Facts, Lies, and Quality Management
Dr. Suresh Kumar Krishnan
Strasys Solutions Sdn Bhd, Kuala Lumpur
— 2026
Publisher's Note on This
Edition
The first edition of this
book, Facts, Lies, and Quality Management, was written in 2015 from more than twenty-five
years of experience in quality management systems (QMS), statistical thinking,
and organizational auditing across Malaysian hospitals, government agencies,
vocational institutions, and enterprises. Every principle in that edition —
Q1/Q2/Q3, Productive and Unproductive Variation, Experience Based Outcome, the
DRSK framework — was built on watching systems succeed and fail in front of
real people.
Eleven years later, the
systems doing the failing (and succeeding) increasingly include ones that were
never explicitly programmed for the situation they find themselves in. Large
language models, retrieval pipelines, and autonomous agents now sit inside the
same processes this book originally studied — the hospital counter, the
procurement desk, the training room, the feedback form. The variation hasn't
gone away. It has simply acquired a new source: the model itself, and the
agents built on top of it.
This 2nd edition does not
discard the original framework. It re-centers it. Every part of the book has
been rebuilt around AI governance and agentic AI, while keeping quality
thinking — and the DRSK framework in particular — as the philosophical spine that
connects 2015 to 2026 and beyond. Readers who know the 2008-2015 QMS world will
recognize every skeleton. Readers arriving fresh through AI governance work
will find the skeleton was there all along, waiting for the machines to catch
up to it.
Contents
PART 1 — The Awareness: Quality and Trust in the Age
of Agentic AI
PART 2 — More Awareness: Variation in the Age of
Autonomous Agents
PART 3 — The Realization: From Experience Based
Outcome to Accountable AI
PART 4 — Approaches: Standardizing Governance, Not
Just Process
PART 5 — Successful or Not: Are We Measuring the Right
AI Things Right?
PART 6 — Quality of Teaching and Learning: From AI
User to Corporate AI Entrepreneur
PART 7 — Innovation Reimagined: From the 3Is to
Compound AI Systems
PART 8 — Feedback Management in Agentic Systems:
Closing the Loop
PART 9 — DRSK Framework 2.0: Robust Agentic AI
Solutions at the End and Beyond
PART 1
The Awareness: Quality and
Trust in the Age of Agentic AI
In 2015, this part opened
with a simple provocation: ask ten people what "quality" means and
you will get ten answers, because quality is the degree of satisfaction,
delight, and value a product or service creates relative to what a person actually
needed — not an abstract universal standard. The BMW-versus-Kancil example made
the point that a locally made car is not "low quality" simply because
it costs less; it has its own quality, matched to its own specification and its
own buyer.
In 2026, the same
argument needs only a change of subject. Ask ten people what makes an AI system
"good”, and you will get ten answers — accuracy, speed, cost, safety,
explainability, autonomy, compliance — because different stakeholders need
different things from the same model. A frontier reasoning model is not
automatically "higher quality" than a small, fine-tuned classifier
running on device, any more than a BMW is automatically higher quality than a
Kancil. A bank's fraud detection model that must run in under 50 milliseconds
on commodity hardware has its own quality; a research assistant model with a
two-minute reasoning budget has a different quality. The attributes that
measure quality — latency, groundedness, tool-call precision, jailbreak
resistance, cost per inference, auditability — determine the price and the fit,
exactly as engine performance, comfort, and after-sales service did for the
car.
What has changed
structurally, and this is the heart of the 2026 re-centering, is where Quality
is realized. The original Q1–Q2–Q3 phases still hold, but each now has an
AI-native reading:
• Q1 — Specification and Realization.
In 2015 this meant design inputs, verification and validation, process
controls, and inspection before delivery. In 2026 it means the model or agent's
documented specification: training data provenance, evaluation benchmarks,
red-team results, and the explicit "agent job description” - its tool
radius, memory scope, and autonomy level — bounded before a single token is
generated in production. A model with no documented specification is a product
shipped without an inspection stage.
• Q2 — Presentation and Experience of
Capability. In 2015 this was packaging, branding, and the visualization of
experience — the test drive, the free sample. In 2026 it is the interface layer
of an AI system: the chat window, the agent's tool-call confirmations, the
citations it shows, the confidence signals it gives. This is where a user first
forms an impression of whether the system can be trusted, well before they have
used it enough to know for certain.
• Q3 — Realized Trust. In 2015, Q3 was
the point where the customer's experience touched their Belief System
positively or negatively, and they either came back or didn't. In 2026, Q3 is
the point where a person decides — based on accumulated interactions, not a
single demo — whether an AI system is trustworthy enough to keep delegating
tasks to. This is not a technical metric. It is the same belief-system
transformation the original book described, except the product now talks back,
reasons, and sometimes acts autonomously in the world.
The reframing that this edition
insists on, borrowed directly from the AI Governance Protocol developed
alongside this book, is the rejection of "unbreakable" as a claim and
the adoption of "resilient and accountable" as the design principle.
No AI system — agentic, generative, or otherwise — can be made unbreakable.
Claiming otherwise is itself a quality failure, because it discourages the
ongoing vigilance the system actually needs. Just as the 2015 edition warned
against QMS auditors who mistook a passed inspection for a guarantee of
quality, the 2026 edition warns against mistaking a passed benchmark for a
guarantee of safety. Every control reduces the probability and blast radius of
failure; none eliminates it. That, and not a certificate on a wall, is what
quality thinking now looks like at the frontier of AI.
Figure 1. The Q1–Q2–Q3 trust cycle, re-centered for
agentic AI.
PART 2
More Awareness: Variation
in the Age of Autonomous Agents
The 2015 edition spent
this part teaching statistical thinking — the discipline, inherited from Six
Sigma and Deming of understanding, measuring, controlling, and eliminating
variation to produce consistent output. It introduced the crucial distinction
between Unproductive Variation (UV) - variation that must be eliminated,
reduced, or controlled — and Productive Variation (PV) - variation that must be
deliberately created to solve a problem, such as splitting a hospital pharmacy
counter by patient type.
That distinction turns
out to be one of the most useful lenses available for agentic AI, almost
unchanged. Large language models are structurally, variation-generating
machines: the same prompt can produce different outputs across runs, the same
agent can take different tool-call paths through the same task, and the same
underlying model can be jailbroken by phrasing a request ten different ways. Two
categories of variation now sit inside every AI system, and mistaking one for
the other is the single most common governance failure this edition has
observed:
• Unproductive Variation (UV) in AI
systems — hallucination drift across sessions, prompt injection succeeding on
one phrasing but not another, an agent looping indefinitely on a malformed tool
response, or a fine-tuned model quietly regressing on a safety benchmark after
a routine update. This is the AI-era version of the first-wear shoe blister:
painful, avoidable, and traceable to a root cause once you look for it rather
than reaching for a superstitious remedy (a bigger model, a longer prompt, a
stern system message with no enforcement behind it).
• Productive Variation (PV) in AI
systems — deliberately architected diversity that exists to eliminate or
control the UV above: ensemble models that cross-check each other's outputs, a
stricter model routed in for high-risk requests while a faster model handles
routine ones, multiple independent agents whose outputs are validated against
each other before any is trusted (multi-agent cross-validation), or a
human-in-the-loop gate inserted specifically at the points where autonomous
action would otherwise be irreversible.
This is precisely the
logic behind the seven-stage lifecycle governance gates now used across this
practice's AI Governance Protocol: data classification and consent basis
(controlling UV at the source), least-privilege tool grants and bounded memory
scope (PV that prevents scope-creep UV), adversarial red-teaming before
deployment (measuring whether the PVs actually control the UVs they were
designed for), and full trace logging (the equivalent of the QMS quality
checkpoint, now applied to every reasoning step and tool call an agent makes).
The prompt injection
"kill chain" — Delivery, Execution, Propagation, Persistence/Impact —
described in this practice's Promptware protocols is, at its core, a UV
classification exercise. Untrusted content enters the model's context
(Delivery); the model cannot structurally tell an instruction from data because
both travel through the same channel (this is the AI-native reason variation
cannot simply be inspected away at one checkpoint, exactly as Deming found that
94 percent of manufacturing problems live in the system itself, not in a single
special-cause incident); if the agent has tool access, the UV propagates into
the real world; and if unresolved, it persists across sessions. No single
control closes this — the same lesson the 2015 edition drew from the fast-food
kitchen that keeps consistency not through one great cook but through policies,
procedures, and equipment that perform the same everywhere. An AI system's
consistency depends on controls distributed the same way, across data, model,
harness, tool layer, orchestration, and governance — not concentrated at
deployment.
The practical steps to create the
right experience with an AI system, updated from the 2015 checklist, remain in
the same order: understand the desired outcome and the risk it carries;
identify the Productive Variation needed (which model, which guardrail, which
human checkpoint); identify the Unproductive Variation each PV might itself
introduce (a stricter model that is slower, a human gate that becomes a
bottleneck); and only then standardize through documented, owned, auditable
procedure — never the reverse.
Figure 2. Unproductive vs. Productive Variation, and the
Promptware Kill Chain.
PART 3
The Realization: From
Experience Based Outcome to Accountable AI
Experience Based Outcome
(EBO) was the 2015 edition's answer to a mindset problem: organizations chase
Output (did we ship it; did we close the ticket) when they should be chasing
Outcome (did it create the experience that touches a person's Belief System
positively). A QMS auditor who only checks whether a procedure was followed,
without asking what experience the procedure created for the patient, the
student, or the customer, is measuring Output while missing Outcome entirely.
This distinction is, if
anything, more urgent in an agentic AI system than it ever was in a paper-based
process, because an AI agent can satisfy every Output metric — the ticket is
closed, the API call succeeded, the response was generated within latency
budget — while creating a badly damaging Outcome: a hallucinated answer
delivered with total confidence, a chatbot that quietly enrolled a customer in
the wrong plan because it optimized for "resolved" rather than
"resolved correctly," or an autonomous coding agent that passed its
own unit tests while introducing a security vulnerability nobody asked it to
check for. An AI system, exactly like the QMS documents in the original
edition, can be technically compliant and experientially catastrophic at the
same time.
The 2026 reading of the
Belief System diagram is direct: a person's willingness to keep delegating
tasks to an AI agent is not formed by the model's benchmark scores, which they
will never see. It is formed by the accumulated experience of being right, being
wrong, being caught when wrong, and being told the truth about uncertainty.
Trust — the AI-era name for what the original book called the Belief System
being touched positively — is built the same way loyalty to a brand was built
in 2015: through repeated, holistic experience, not through a single
well-marketed demo (Q2, in the old language) that never survives contact with
production use.
Sustainable Controlled
Condition (SCC) — the 2015 concept that a solution must be kept under control
with the outcome in mind, using real-time information to stay on course — maps
almost verbatim onto what this practice's AI Governance Protocol calls operational
governance at scale: trace logging, observability dashboards, and
human-in-the-loop gates running live across every deployed agent, with
thirty-plus days of clean audit trail as the exit criterion before an
organization can claim SCC over its AI estate. The 2015 example of checking
real-time traffic data before choosing a route is almost unchanged, the 2026
practice of an orchestration layer routing a request to a stricter model or a
human reviewer based on real-time risk signals rather than a static rule
written once and forgotten.
The auditing failure mode this part
warned about in 2015 — the lab technician following a written procedure
correctly while creating a terrible experience for an anxious patient,
invisible to a compliance-only QMS auditor — has a precise AI-era counterpart:
an agent that follows its system prompt correctly, refuses nothing it was told
to refuse, and logs every action, while still producing an outcome nobody
intended, because the specification itself never anticipated the situation. QMS
auditors, the original book argued, needed to evolve from compliance-based
auditing to performance-and-experience-based auditing. AI governance is being
asked to make the identical evolution, on a compressed timeline, at far higher
stakes, with regulators (PDPA 2010, Bank Negara Malaysia, MCMC, and the
emerging National AI Office frameworks) only now catching up to write the
rules.
Figure 3. Output vs. Outcome, and the Sustainable
Controlled Condition (SCC) loop.
PART 4
Approaches: Standardizing
Governance, Not Just Process
The 2015 edition's
auditing case studies — the pharmacy lab that never told anxious patients their
results unless abnormal, the pregnant lecturer's student forced to submit
ultrasound images as medical evidence, the hemodialysis servicing technician
nobody could identify — shared one diagnosis: certification theatre.
Organizations that get ISO 9001 certified "for the sake of being in the
league" without the system ever working for the people inside it. The
procedures were followed to the letter, the auditors found no non-conformance,
and the actual human experience was quietly terrible.
AI governance in 2026 is
at exactly the same fork in the road, and this edition names the risk plainly:
an organization can produce a beautiful model card, pass a superficial bias
audit, publish an AI ethics policy, and still ship an agent that no one has
adversarially tested for prompt injection, whose tool permissions were never
reviewed against least privilege, and whose failures are invisible because
nobody logs its reasoning steps. That organization is ISO-9001/ ISO-42001(AIMS)
-certified in spirit — technically compliant, substantively unaccountable.
The corrective this
edition offers is the same one the AI Governance Protocol already operates:
standardization must attach a mandatory governance gate to every stage of the
AI solution lifecycle, and no stage is "complete" until its gate
question can be answered with evidence, not assurance.
|
Lifecycle Stage |
Governance Gate Question |
QMS/ AIMS Equivalent |
|
Data Ingestion & Types |
Is every source classified, consented, and provenance-tagged? |
Incoming inspection of raw materials |
|
ML Approach Selection |
Does the paradigm match a documented business justification? |
Design-input review |
|
Model & Agent Selection |
Is the agent's tool radius, memory, and autonomy explicitly
bounded? |
Process capability study |
|
Evaluation & Refinement |
Has it been red teamed for injection and jailbreak, not just
accuracy? |
Product qualification testing |
|
Deployment & Integration |
Is there a human checkpoint before irreversible action, and full
logging? |
Final inspection before release |
|
Feedback & Governance |
Can every decision trace to a version, a lineage, and an owner? |
Traceability and lot control |
|
Continuous Improvement |
Are reviews institutionalized, or dependent on one person
remembering? |
Management review cycle |
The role-by-role charter
this edition inherits from the same protocol answers the 2015 complaint that
governance which is "everyone's responsibility" is in practice no
one's. Just as the original book insisted that a QMS auditor's non-transferable
question was whether the system created the right experience — not merely
whether the procedure existed — the 2026 edition assigns each discipline in the
AI stack (Prompt Engineering, Context Engineering, Memory Engineering, Harness
Engineering, Tool Layer Engineering, Orchestration, Agent Design, Safety
Engineering, Observability, Alignment & Policy) a single named owner and a
single non-transferable question, with evidence required, not a signature of
assurance.
The closing insight of
Part 4 in 2015 still stands word for word in 2026: a system nobody must work
for, because it works for them, is the only kind worth standardizing. An AI
agent that a team has to babysit, second-guess, and manually patch every week
is not "innovative" simply because it uses a large language model —
it has recreated the exact certification-theatre burden the original book spent
this entire part warning against, just with a more expensive vendor invoice
attached.
Figure 4. The seven-stage AI governance gate lifecycle.
PART 5
Successful or Not: Are We
Measuring the Right AI Things Right?
Non-Value-Added Activity
(NVAA) was the 2015 edition's term for the procurement officer's habitual
clarification call — an activity nobody designed, that adds no value, that
persists because it became a norm, and that quietly inflates the Hidden Cost of
Quality. It arises either because the activity became irrelevant through
external change, or because a single unresolved Failure Event kept spawning
workaround activity around itself.
Agentic AI systems
generate their own AI-native version of NVAA, and it is easy to miss because it
hides inside an impressively large number of API calls rather than a human
phone call. An agent that makes six redundant tool calls to accomplish what one
well-scoped call could do; a multi-agent pipeline where two agents repeatedly
re-verify the same fact because neither trusts the other's output; a retrieval
step that re-fetches a document already in context because memory scope was
never designed — each of these is Non Value Added Automation, and it carries
the same two causes the 2015 edition identified: irrelevance (the workaround
exists because a model upgrade or a tool-schema change should have removed the
need for it, and nobody revisited the design) and unresolved Failure Event (a
single upstream hallucination that never got root-caused, now permanently
"fixed" by three extra verification calls downstream).
The Hidden Cost of
Quality in 2015 was absorbed quietly into the cost of operations. In 2026 it
shows up as compute spend that never gets scrutinized the way the original book
insisted scrap and rework should be scrutinized: token cost per resolved case,
GPU-hours spent on agent loops that never converge, and — the most
consequential hidden cost of all — the cost of an unmonitored agent's mistakes
that surface downstream, in a customer's inbox or a regulator's inquiry, long
after the agent itself reported success.
This part's 2015 warning
about percentages misleading without a denominator translates directly:
"our AI system resolved 94% of tickets automatically" sounds like the
Motorola Six Sigma standard until you ask, as the original book insisted on the
Kapit-Sarawak and China poverty-line examples, what the actual base and the
actual stakes are. Ninety-four percent of low-stakes queries resolved
automatically is a very different fact from ninety-four percent of all queries
— including the six percent of genuinely high-risk ones — resolved without a
human ever reviewing them. Model uptime, response rate, and even
user-satisfaction thumbs-up counts are, in this edition's language, colorful
graphs that don't add value unless matched against the outcome metric that
actually mattered: was the decision correct, traceable, and accountable.
The corrective, unchanged from 2015:
measure Outcome-based Output. For an AI system this means pairing every
operational metric (latency, cost, automation rate) with a governance metric (traceability,
red-team pass rate, human-override rate on flagged cases) and refusing to
report the first without the second. An organization drowning in dashboards
about its AI system's uptime, with no dashboard about whether its outputs were
ever wrong in a way that mattered, is doing exactly what the 2015 lab and
hospital examples warned against — measuring the things that are easy to
measure, not the things that were promised.
Figure 5. Pairing operational metrics with governance
metrics to measure Outcome-based Output.
PART 6
Quality of Teaching and
Learning: From AI User to Corporate AI Entrepreneur
The 2015 edition's
distinction between competence (the ability, skill, and knowledge a person has
accumulated) and Competency (the skill required to perform a specific job in
its specific present-day context, which can shift even for an experienced
person moved to a new environment) is, unaltered, the single most important
idea for organizations trying to build an AI-literate workforce in 2026. A
person can be highly competent — years of domain expertise — and still lack the
competency to work safely alongside an autonomous agent, precisely as the cryogenic
lab example insisted that even an experienced lab technician needed a
structured eight-week onboarding before working alone with a new piece of
equipment and its new hazards.
This is the entire
premise behind restructuring this practice's flagship 5-Day Masterclass — The
Future of Data: From Analytics to Artificial Intelligence, AI User to Corporate
AI Entrepreneur (CAIE) - as the direct 2026 descendant of this part's Learning
Needs Diagnosis (LND). The 2015 edition insisted that training only produces
Return on Training Investment (ROTI) if it is preceded by a proper diagnosis of
how the learner actually needs to learn; a workforce handed a generic "AI
101" course without first diagnosing whether they need foundational AI
literacy, Compound AI System Solution (CAISS) design skills, or AI governance
fluency will produce the same thin ROTI the original book warned about — training
that benefits a selected few and never becomes organizational capability.
The CAIE program’s
five-day arc mirrors the original Teaching and Learning logic almost
structurally: Day 1 (Foundations of AI and Data Mastery) is the Learning Needs
Diagnosis stage made explicit — establishing statistical thinking and the DRSK
framework as shared vocabulary before anyone touches a tool. Days 2 through 4
(AI Value Chain, CAISS/AIAVC architecture, AI Governance) are
competency-building under real job conditions, the equivalent of the eight-week
structured OJT, not a one-off seminar. Day 5 (Commercialization and Business
Modeling) is the transition this book has always insisted matters most: from a
person who consumes a system correctly to a person who can design, govern, and
monetize one — from AI User to Corporate AI Entrepreneur, which is simply the
2026 vocabulary for what the 2015 edition called moving someone from following
a procedure to building Quality into the process itself.
The 2015 warning about the retiring
employee whose accumulated competence is treated as expired the day they leave
applies without modification to organizations that let their most experienced
people be replaced by an AI agent without first extracting what they know into
the agent's specification, its guardrails, and its escalation paths. Retiring
is not equal to expired, the original book insisted; equally,
replaced-by-an-agent is not equal to institutional-knowledge-preserved, unless
someone deliberately did the Learning Needs Diagnosis on the departing expert
before the agent went live. Malaysia's MD2030 mandate to reskill 700,000
workers and lift the skilled-worker share to 35% is, in this reading, a
national-scale Learning Needs Diagnosis exercise — and it will fail the same
way an untailored in-house training program failed in 2015, if it is delivered
as generic digital literacy rather than diagnosed, competency-specific,
hands-on capability building.
Figure 6. The CAIE five-day arc: from Learning Needs
Diagnosis to Corporate AI Entrepreneur.
PART 7
Innovation Reimagined:
From the 3Is to Compound AI Systems
The 2015 edition's 3Is —
Invention, Improvement, and Innovation, cycling until a product becomes the
fourth I, Irrelevant — used Alexander Graham Bell's telephone to make the point
that innovation is renewal with novelty, not the introduction of something from
nothing. It also punctured the myth that innovation requires a science
background or a large budget: anyone solving a problem with the resources at
hand, at home or at work, is already doing it.
The 3Is map onto the AI
Agent Value Chain (AIAVC) and Compound AI System Solutions (CAISS) with almost
no translation needed. Invention is the foundation model itself — the frontier
lab's breakthrough, analogous to Bell's original handset, that most organizations
will never build and do not need to. Improvement is what almost every
enterprise does: fine-tuning, prompt engineering, retrieval augmentation, and
orchestration layered onto an existing model to make it fit a specific job —
the 2026 equivalent of the decades of incremental redesign that took Bell's
handset to the smartphone without reinventing telephony. Innovation, in this
edition's re-centered reading, is the deliberate combination of several such
improved components into a Compound AI System Solution that solves a business
problem no single model call could — precisely the "renewal with
novelty" the original book defined, applied to systems rather than
devices. And Irrelevance is what happens to any AI deployment, however
impressive at launch, that is never revisited as models, threats, and
regulations move on around it.
The original book's
warning against mistaking hype for innovation — organizations "getting
excited when there's hype in certain initiatives," chasing trend rather
than fit — is, if anything, an understatement of what agentic AI adoption looks
like in 2026. The corrective is unchanged: innovation is the ability to change
even the smallest thing with novelty that creates value, not a mandate to
deploy the largest, most autonomous agent available regardless of whether the
job needs it. A CAISS built from two small, well-governed models and one human
checkpoint can out-innovate a single ungoverned frontier-model agent, exactly
as the Kancil serving its owner's actual commute needs outperforms a BMW parked
outside a train station.
"Thinking outside the box"
gets the same treatment here that it got in 2015: not a call to abandon
structure, but a reminder that the box itself must be identified before anyone
can usefully step outside it. In agentic AI design, the boxes worth naming
explicitly are the ones this practice's Enterprise AI Agent Blueprint already
separates — Foundation Engineering (prompt, context, memory, harness), Agentic
Execution (tools, orchestration, agent design, multi-agent systems), and
Governance (safety, observability, alignment). Innovating inside only one of
these three categories, in isolation from the other two, reproduces the exact
failure mode the original book warned about: a solution generated with
excitement that never survives contact with the people who have to live with
it, because Innovation was never load-bearing across the whole architecture —
only "a fast car with no brakes," to use this practice's own
recurring image for agentic systems built without equal weight of governance.
Figure 7. The 3Is cycle mapped onto the AI Agent Value
Chain, and the Enterprise AI Agent Blueprint's three pillars.

PART 8
Feedback Management in
Agentic Systems: Closing the Loop
The 2015 edition's core
complaint about feedback management was timing and intent, not the absence of
forms. Organizations collected feedback constantly — complaint forms,
suggestion boxes, satisfaction surveys — but frequently at the wrong point in
the process (the father handed a survey moments after his wife's delivery; the
enforcement-agency client rating only the counter experience, with the entire
rest of the process unaccounted for), turning a genuine mechanism for closure
into what the book called a classic blunder: an experience captured at the
input level, mistaken for feedback on the outcome.
Agentic AI systems
reproduce this exact failure at a much larger scale and a much faster cadence,
because feedback in an AI context can mean at least three different things that
are routinely conflated: a user's in-the-moment thumbs-up or thumbs-down on a
single response (Passive Feedback, gathered reactively, exactly like the
original suggestion box); a structured evaluation run against a held-out test
set before deployment (a proactive, Active Feedback mechanism, akin to the
satisfaction survey); and the continuous trace of an agent's real-world tool
calls and outcomes over weeks of deployment (a form of feedback the 2015
edition did not have a category for, because static QMS processes did not
generate it — call it Systemic Feedback, the AI-native third category).
The lesson from the 2015
edition's father-friendly delivery room example applies directly: asking a user
to rate an AI agent's response the instant it is generated captures only the
input-level experience — did the answer sound plausible right now — and says
nothing about the outcome that matters, which may only be knowable days later
(did the advice work, did the code ship without a security hole, did the
medical information hold up). This is precisely why this practice's AI
Governance Protocol treats "AI System Feedback & Governance" as
its own lifecycle stage, separate from deployment, with its own gate question:
can every model decision be traced back to a version, a data lineage, and a
responsible owner, long after the moment of generation has passed?
The 2015 edition's
Passive/Active feedback classification maps onto the current governance
vocabulary almost directly: Passive Feedback is the RLHF-style thumbs-up/down
signal gathered incidentally as users interact; Active Feedback is the
deliberately designed red-team exercise, the structured human evaluation, the
adversarial test suite run before going-live specifically to surface what
casual use never would. Both matter, exactly as the original book insisted
complaints, compliments, and comments (the 3Cs) all mattered — but neither substitute
for the other, and an organization that only gathers Passive Feedback on its AI
system, the way the enforcement agency only surveyed its counter, will discover
its blind spots the hard way: downstream, in the outcome, long after the
input-level scores looked fine.
Closure, the organizing
idea of this part in 2015, still means the same thing in 2026: no system, human
or agentic, can run on a one-way street. Every feedback loop this book has ever
described — from the suggestion box to the RLHF signal to the post-incident
red-team report — exists to feed back into Standardization (Part 4) and the
DRSK cycle (Part 9), so that the next version of the system is provably better
than the last, not merely differently confident.
Figure 8. Passive, Active, and Systemic feedback closing
the loop into Standardization and DRSK.
PART 9
DRSK Framework 2.0: Robust
Agentic AI Solutions at the End and Beyond
This closing part is
where the 2015 edition and its 2026 successor are, in truth, the same document.
The DRSK framework — Desired Outcomes and Risk Assessment, Robust Solutions,
Standardization, and Key Measures and Kick Factors — was built to ensure that
any problem-solving team, from an Innovative and Creative Circle to a Kaizen
group, thinks from every angle before committing to a solution, and keeps
testing that solution's feasibility rather than abandoning good ideas
prematurely or shipping fragile ones. Every letter of DRSK survives the
transition to agentic AI without alteration, only extension.
|
DRSK Element (2015) |
2026 Agentic AI Extension |
|
D — Desired Outcomes: what experience must this solution create? |
What must this agent's users be able to trust it to do, and what
must it never be allowed to do — the explicit "agent job
description" bounding tool radius, memory scope, and autonomy level. |
|
R — Risk Assessment: what could go wrong, and for whom? |
Adversarial red-teaming for prompt injection and jailbreak
resistance; mapping the Promptware kill chain (Delivery, Execution,
Propagation, Persistence/Impact) against this specific deployment before
go-live. |
|
S — Standardization: how do we lock in the solution without
freezing it? |
Least privilege tool grants, instruction-hierarchy design so
operator commands are never equal in authority to retrieved content, and
context-tagging so the model can distinguish trusted instruction from
untrusted data. |
|
K — Key Measures and Kick Factors: how do we know it's working,
and staying working? |
Full trace logging of every reasoning step and tool call;
scheduled (not ad hoc) metric reviews across data, model, and deployment
layers; incident playbooks for the Kick Factors — the undesired outcomes risk
assessment didn't anticipate. |
The 2015 edition's
closing insistence — that a robust solution must never be addressed in
isolation, and that total participation from interested parties is not optional
— is exactly the argument behind this practice's Governance Maturity Roadmap,
staged from Literacy (everyone touching the system can explain, in plain
language, why instruction/data separation matters) through Control Mapping,
Safety Baseline, Operational Governance at Scale, and finally Board-Level
Assurance. Just as the original book warned that problem-solving teams who get
excited about their own solution while forgetting the people who will actually
live with it on a daily basis end up building something that is never truly
adopted, an AI governance program that stops at a red-team report without
reaching quarterly Board-level reporting has not completed the DRSK cycle — it
has stopped at R and never fully closed the loop back to K.
The line this book closed
with in 2015 needs almost no rewriting for 2026: the key to survival in this
very fast-changing world is understanding the holistic dynamism of Quality in
all aspects that touch our life — failing which, it will lead us directly to
extinction. In 2026, add only this: the aspects that touch our life now reason,
act, and sometimes decide on our behalf, faster than any QMS auditor could ever
walk a factory floor. The DRSK framework was built to keep pace with dynamism
wherever it shows up. It was simply waiting, in 2015, for the machines to
become dynamic enough to need it this badly.
True Enterprise AI is
Accountable AI. A system with distributed, independently owned controls at
every lifecycle stage — data to deployment, prompt to policy, agent to audit —
will fail more often at any single point and is for that very reason far less
likely to fail catastrophically or invisibly. That was the realistic,
defensible standard this practice set for its clients, regulators, and boards
in its AI Governance Protocol. It is, read closely, the same standard the first
edition of this book set for hospitals, banks, and training rooms in 2015.
Quality thinking did not need to be reinvented for the agentic AI era. It only
needed, at last, a system fast and autonomous enough to prove it was right all
along.
Figure 9. DRSK Framework 2.0 — the eight-element Robust
AI Solutions map (per the DRSK-AI Solutions framework).
About the Author — 2026
Dr. Suresh Kumar Krishnan
is the Managing Director of Strasys Solutions Sdn Bhd, a Kuala Lumpur-based
consultancy founded in 2008. He works at the intersection of AI advisory,
enterprise transformation, and AI governance education, with a client base
spanning Malaysian GLCs, public sector agencies, enterprises and other countries such as Indonesia, Brunei, China, South Africa, Tanzania and Kingdom of Saudi Arabia. A certified
QMS lead auditor and PRINCE2 practitioner by training, and an AI educator and
facilitator by current practice, Dr. Suresh built the DRSK framework, the AI
Governance Protocol, and the CAIE masterclass on the same foundation: more than
27 years of watching quality succeed or fail in front of real people, now
applied to systems that increasingly act on their own in the AI Era.
No comments:
Post a Comment