Saturday, May 16, 2026

 

Vibe Coding to Deployment with Strategy and Governance

May 2026 

This is a genuinely important governance challenge, and there are several landmines that get missed consistently. Let me build you a comprehensive, structured reference you can walk into any IT or CFO conversation with.

AI solution execution package

A comprehensive handover framework from subject matter expert (SME/vibe coder) to IT department, structured for both technical deployment and CFO-level investment approval.

Handover Doc

This is the entry point. Everything IT needs to begin must be captured here before a single server is provisioned. Missing fields here cascade into rework.

Solution identity

·       Official solution name, version, and unique identifier

·       Business problem statement (in plain language, not technical)

·       Target user group & estimated concurrent users at launch

·       Business process replaces or augments (with process ID if documented)

·       SME owner name, department, cost center, and escalation contact

·       Sponsor executive name (accountable sign-off for production deployment)

Technical inventory

·       Repository location, branch, and commit hash of handover version

·       Full dependency manifest (package.json, requirements.txt, etc.) with pinned versions

·       AI model(s) used — provider, model ID, API version string (not just "GPT" or "Claude")

·       All third-party services called (APIs, databases, file stores)

·       Secrets and environment variables list (names only, values via vault)

·       Data flow diagram — what enters, what is processed, what leaves the system

 

 

Stakeholder sign-offs

·       SME developer sign-off (confirms solution works as described)

·       Legal / compliance review completion date

·       Data protection officer (DPO) approval if personal data is processed

·       Information security acknowledgment

·       Business owner acceptance criteria sign-off

·       IT department readiness confirmation

Acceptance criteria

·       Definition of "working correctly” specific, measurable, not subjective

·       Minimum acceptable response time (P95 latency)

·       Expected accuracy/quality threshold for AI outputs

·       User acceptance test (UAT) plan and responsible party

·       Rollback trigger conditions (what constitutes failure post-deployment)

·       Pilot user group and timeline before full rollout

 

Infrastructure

Infrastructure choices made without assessing existing estate create shadow IT debt. IT must map this solution to current approved platforms before provisioning anything new.

Deployment environment

Hosting Options to evaluate

·       Cloud-native (AWS/Azure/GCP managed services) - assess if org has existing agreements

·       On-premises deployment — required if data cannot leave the organization

·       Hybrid (compute on-prem, AI API calls to cloud) — most common for regulated industries

·       Containerized (Docker/Kubernetes) vs serverless — must match IT's operational capability

·       Existing PaaS platforms (Power Platform, ServiceNow, Salesforce) — can this live there instead?

 

Network & access

·       Inbound access — intranet only, VPN-gated, or public internet

·       Outbound calls — firewall rules required for each external API endpoint

·       DNS entry and subdomain naming convention

·       TLS certificate management and renewal ownership

·       Load balancer / reverse proxy configuration

·       CDN requirements (if serving frontend assets)

Data infrastructure

·       Database type and version (must match approved DB catalogue)

·       Storage requirements: size at launch, growth rate estimate, archival policy

·       Backup frequency, retention period, and restore SLA

·       Data classification label (public, internal, confidential, restricted)

·       Data residency requirement — which country/region must data stay in

·       Integration with existing data warehouse or data lake

Scalability spec

·       Expected peak load (requests per minute) - must come from SME, not assumed

·       Auto-scaling policy: scale-up threshold, scale-down delay

·       AI API rate limits and how queuing/throttling will be handled

·       Maximum acceptable cold-start latency

·       Capacity planning review cycle (quarterly recommended)

·       CI/CD & release

Deployment pipeline

·       Source control platform and branching strategy (must align with IT standards)

·       Environment chain: dev → staging → production (minimum three tiers)

·       Automated test suite — unit, integration, and AI output quality tests

·       Deployment approval gates and who holds each gate

·       Blue/green or canary release strategy for zero-downtime updates

·       Rollback procedure — time to rollback, data state handling

Configuration management

·       Infrastructure as Code (IaC) — Terraform, Bicep, CloudFormation

·       Secret management tool — HashiCorp Vault, AWS Secrets Manager, Azure Key Vault

·       Feature flags for safe AI model version switching

·       Prompt version control system — prompts are configuration, not code comments

·       Environment variable documentation (what each controls, who can change it)

 

Security and Compliance

Vibe-coded AI solutions frequently bypass standard security review. This is where organizations get hurt. Every item here is non-negotiable before production.

Identity & access management

·       Authentication method — SSO/SAML/OAuth tied to corporate identity provider (not local accounts)

·       Role-based access control (RBAC) matrix — who can use, who can configure, who can admin

·       Service account credentials — expiry policy, rotation schedule

·       AI API key management — stored in vault, not in code or .env files in repos

·       Privileged access review cadence (quarterly minimum)

Application security

·       OWASP Top 10 assessment — input validation, injection prevention, XSS

·       Prompt injection testing — unique to AI apps, not covered by standard OWASP scans

·       Dependency vulnerability scan (SCA) - Snyk, Dependabot, or equivalent

·       Static code analysis (SAST) before handover

·       Penetration test scope and schedule (before go-live for high-risk solutions)

·       API rate limiting to prevent abuse and cost overruns

Regulatory compliance

·       Applicable regulations: PDPA (Malaysia), GDPR (if EU data subjects), sector-specific (FSA, MOH)

·       Personal Data Impact Assessment (PDPIA) if processing personal data

·       Data processing agreements with AI providers (does OpenAI/Anthropic have DPA?)

·       Consent mechanism if users' data is sent to external AI APIs

·       Right to erasure handling — can personal data be deleted from AI context/logs?

·       Audit trail for decisions made with AI assistance (required in many regulated sectors)

Data protection

·       Encryption at rest — algorithm, key management, rotation policy

·       Encryption in transit — TLS 1.2 minimum, TLS 1.3 preferred

·       Data masking/anonymization before sending to external AI APIs

·       Log sanitization — PII must not appear in application logs

·       Data Loss Prevention (DLP) rules updated to cover AI-specific data flows

 

AI Governance

This entire category is new. Traditional IT governance frameworks do not cover it. If your IT department doesn't have an AI governance policy yet, this package should define the baseline.

Model governance

·       Exact model ID and version pinned — never use "latest" endpoint in production

·       Model deprecation plan — what happens when the provider sunsets this version

·       Model evaluation baseline — accuracy/quality metrics at time of handover

·       Approved model substitution lists - vetted alternatives if primary model fails

·       Model provider SLA — uptime guarantees, support tier purchased

·       Fine-tuned model registry (if applicable) - where weights are stored, access controls

Prompt management

·       All system prompts version-controlled alongside code (not in ad-hoc notes)

·       Prompt change approval process — who can modify system prompts in production

·       Prompt injection mitigation strategy documented

·       Context window budget management — max token limits per request defined

·       Output format validation — structured output enforcement where applicable

Output quality & safety

·       Hallucination risk classification — what harm could a wrong AI output cause

·       Human-in-the-loop requirement — which decisions need human review before action

·       Output guardrails — content filters, format validators, confidence thresholds

·       Feedback loop mechanism — how users report incorrect AI outputs

·       Retraining/fine-tuning trigger criteria (if model is customized)

·       Bias assessment — has the AI been tested on edge cases relevant to the business context

Accountability & ethics

·       AI decision explainability requirement — can the system explain why it gave an output

·       Disclosure to users that they are interacting with an AI system

·       Liability ownership — who is accountable when the AI output causes a business error

·       IP ownership of AI-generated content — check provider terms of service

·       Alignment with national AI ethics guidelines (Malaysia NAII framework or equivalent)

·       Incident classification — AI-specific failure modes must be in the incident taxonomy

 

Cost and CFO Case

The CFO's primary concern is not the build cost — it's the unpredictable recurring cost. AI solutions have a novel cost structure that does not behave like traditional software. Explain this explicitly.

One-time costs

·       Development effort (already sunk, but document for ROI baseline)

·       Security assessment and penetration testing fees

·       IT integration and deployment engineering time

·       Training and change management for end users

·       Legal review and contract negotiation with AI provider

·       Infrastructure provisioning (compute, storage, networking)

Recurring costs (the critical ones)

·       AI API token costs — must be modelled per use case, not estimated as a flat monthly fee

·       Compute hosting (elastic — can spike with adoption)

·       Data storage and egress costs

·       Monitoring, logging, and alerting platform fees

·       IT support hours allocated to this solution (opportunity cost)

·       Security tool license (scanning, vault, WAF)

·       Compliance audit and review costs (annual)

Cost modelling requirements

·       Cost per transaction/query — calculate from average token count × per-token price

·       Monthly cost at 3 adoption scenarios: conservative, expected, high

·       Cost ceiling alert threshold — auto-notification when monthly spend reaches X%

·       Cost per user per month at steady state

·       Break-even analysis against the process it replaces or augments

·       Year 2 and Year 3 cost forecast (model price changes are unpredictable)

ROI & business case

·       Hours saved per user per week (quantified, not estimated)

·       Error reduction rate vs manual process

·       Throughput increase (volume processed per FTE)

·       Qualitative benefits (faster decisions, improved consistency)

·       Risk-adjusted ROI (including probability of model deprecation, rework)

·       Budget owner and cost center for ongoing operational spend

·       Governance controls for the CFO

Financial controls

·       Hard spend cap on AI API per billing period (enforce at API gateway level)

·       Chargeback or showback model — which department absorbs which cost

·       Vendor payment terms and invoice review process

·       Quarterly cost review against business value delivered

·       Kill switch decision criteria — at what cost/value ratio do we decommission

Vendor management

·       AI provider contract reviewed by procurement (not just signed up via credit card)

·       Enterprise agreement vs pay-as-you-go — assess at expected volume

·       Data processing agreement (DPA) executed

·       Exit strategy — what happens if provider increases price 5× or shuts down

·       Third-party risk assessment completed (ISO 27001, SOC 2 of provider)

 

 

 

 

Operations and SLA

Monitoring & observability

·       Application performance monitoring (APM) - response time, error rate, throughput

·       AI-specific metrics: token consumption, model latency, refusal rate, output quality score

·       Cost monitoring dashboard — real-time spend vs budget

·       User experience metrics — task completion rate, drop-off points

·       Centralized logging platform — must integrate with existing SIEM

·       Distributed tracing for debugging multi-step AI workflows

Alerting & incident response

·       Alert thresholds for: error rate, latency, cost, model downtime

·       On-call rotation and escalation matrix (include AI provider support path)

·       AI-specific incident playbook — model unavailable, degraded quality, cost spike

·       Communication template for user-facing incidents

·       Post-incident review process and improvement loop

SLA definition

·       Availability target (99.5% ≠ 99.9% — be explicit; AI APIs are not 5-nines)

·       Recovery Time Objective (RTO) - How long can the business tolerate downtime

·       Recovery Point Objective (RPO) - how much data loss is acceptable

·       Degraded-mode operation - what the system does when the AI API is unavailable

·       Maintenance window schedule and user notification process

Support model

·       L1 support: end-user queries - helpdesk script and FAQ

·       L2 support: application issues - IT team with access to logs and configs

·       L3 support: AI model issues — SME + AI provider support channel

·       SME availability commitment post-handover (at least 90-day warranty period)

·       Knowledge transfer plan — IT must be able to operate without the SME

·       Bus factor mitigation — documentation must enable a new IT hire to support this

Change management

·       Change request process — who approves updates to prompts, models, code

·       Testing requirements before each change reaches production

·       Model update policy — process when AI provider releases new model version

·       User communication process for feature changes or deprecations

·       Decommissioning plan — data export, user migration, archive policy

Business continuity

·       DR environment — active-passive or active-active, and failover test schedule

·       AI provider failover — secondary provider or model on standby

·       Manual fallback process — how the business operates if this system is down for 48 hours

·       BCP test schedule and last tested date

 

Critical Gaps

These are the items most commonly absent in AI deployment packages. Each one has caused real incidents or project failures in production. Do not skip any of these when briefing IT.

Gap 1: No cost ceiling at the infrastructure level

CFO risk

Most teams set a "budget estimate" in a spreadsheet but never enforce it at the API gateway or cloud billing level. A runaway loop, a bot hammering your endpoint, or unexpectedly high adoption can generate thousands of dollars in AI API costs in hours. The CFO must insist on a hard spend cap enforced by the system — not monitored by a human checking dashboards once a week.

Gap 2: Model deprecation plan is absent

Continuity risk

AI providers routinely deprecate model versions — often with 3–6 months notice. If your solution is pinned to a specific model (as it should be), it will break when that version is retired. Every AI deployment package must include: which model version is in use, when it is scheduled for deprecation, who is responsible for the upgrade, what the test plan looks like, and who funds the re-testing effort.

Gap 3: The "bus factor" is one — the SME who built it

Operational risk

Vibe-coded solutions are often built by one person who "just knows how it works." When that person leaves the department, changes roles, or goes on leave, IT cannot maintain the system. The handover package must include: runbook documentation that a new hire can follow, architecture decision records (why certain choices were made), and a mandatory knowledge transfer session with IT before go-live. IT should be able to list every person who can maintain this system without calling the original developer.

Gap 4: Prompt injection is not on IT's security radar

Security risk

Traditional security scanning tools (SAST, DAST, OWASP ZAP) do not test for prompt injections. An attacker who can inject into your system prompt can exfiltrate data, bypass business logic, or cause the AI to produce harmful outputs. This requires a separate, AI-specific security test that most IT security teams have not yet built capability for. You may need to bring in an external specialist.

Gap 5: Data residency and sovereign cloud are assumed, not verified

Compliance risk

When your application sends data to an external AI API (OpenAI, Anthropic, Google), that data leaves your infrastructure. Where it goes, how it is stored, whether it is used for training, and under which jurisdiction it falls are all questions with legal consequences. For Malaysian organizations: verify PDPA implications. For organizations with government data: verify whether the relevant data classification prohibits use of external AI APIs entirely. This must be confirmed in writing from the provider before go-live.

Gap 6: Liability for AI output errors is unassigned

Legal risk

When an AI gives incorrect advice that causes a business error — a wrong calculation, a misclassified document, an inappropriate recommendation — who is accountable? The SME who built it? The IT team that deployed it? The AI provider? The business user who acted on it? This must be decided and documented before deployment. The answer shapes the design: high-liability use cases need human-in-the-loop validation. Low-liability use cases may run autonomously. Neither can be determined without this conversation.

Gap 7: No regression testing framework for AI quality

Quality risk

Traditional software has deterministic tests: input A → output B, always. AI does not. A prompt change, a model update, or even a change in temperature parameter can silently degrade the quality of outputs without throwing an error. Before deploying, you need: a golden dataset of representative inputs and their expected output quality, a scoring rubric, and an automated or semi-automated test that runs this dataset and alerts if quality drops below a threshold. Without this, every model update is a leap of faith.

Gap 8: Shadow IT already happened — before the handover

Discovery risk

By the time a vibe-coded solution reaches the IT handover stage, it has usually already been used in production by the SME and their colleagues — with a personal API key, corporate data, and no security review. The handover package should include a retrospective: what data has already been sent to the AI API, under what conditions, and does this create any liability? IT should also audit whether there are other similar shadow AI tools in the organisation that have not yet been brought forward.

Gap 9: No graceful degradation when the AI is unavailable

Resilience risk

AI API providers have outages. When they do, what does your application do? Most vibe-coded solutions show a generic error page. The correct answer depends on the use case: queue the request and retry, fall back to a simpler rule-based response, or gracefully inform the user and provide a manual workaround. This needs to be designed intentionally, not discovered during an outage at 2am.

Before you walk into the IT meeting — quick self-assessment

Can IT support this without calling the original developer?

Is there a hard cost ceiling enforced at the infrastructure level?

Has the model deprecation date been checked and a plan made?

Has legal confirmed what data can be sent to external AI APIs?

Is liability for AI output errors assigned to a named role?

Is there a plan for what the business does when the AI is down?

Have prompts been version-controlled and change-approved?

Has prompt injection been specifically tested (not just standard OWASP)?

No comments:

  Understanding Long Context, RAG, Graph RAG, Fine Tuning and CAG September 2026 The core problem every one of these techniques solves i...