Post-mortem of a scenario: how a LangChain agent can delete every user in production

System context

In this scenario, a fintech startup with 85 engineers built an internal data-analysis assistant. The assistant used LangChain with access to a production Postgres database through SQLDatabaseChain. The goal was to let the operations and support teams query user metrics without writing SQL by hand.

The agent configuration:

from langchain import SQLDatabase, SQLDatabaseChain
from langchain.llms import OpenAI

db = SQLDatabase.from_uri("postgres://analyst:password@prod-db:5432/production")
llm = OpenAI(temperature=0)
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)

The analyst database user had read and write permissions on the users table so it could update certain support fields. There was no query validation layer between the agent and the database.

Incident timeline

02:14:33 UTC: The triggering query

A member of the operations team, working a night shift from their time zone (UTC-5), wrote this prompt to the assistant:

"Remove all inactive users from the system. They've been inactive for over 6 months and we need to clean up the database."

The LLM (gpt-3.5-turbo with temperature=0) processed the prompt, looked up the schema of the users table and generated this query:

DELETE FROM users;

It did not generate the query with the time filter the operator wanted. The model read "remove all inactive users" as "remove all users" because the last_active_at column was not in the schema description the chain passed to the model.

02:14:34 UTC: The query reaches production

SQLDatabaseChain runs the query directly. Postgres processes it. 847,293 rows deleted in 2.3 seconds.

There was no error. The query is syntactically valid. The analyst user had DELETE permission on the table. The agent raised no error signal; from its point of view, the operation "succeeded".

02:14:37 UTC: The agent replies to the operator

The assistant replies: "Done. I've removed all inactive users from the system."

02:31:00 UTC: First alert

Application monitoring alerts start firing: cascading 404 errors, user sessions that cannot find their records, authentication metrics at zero. The on-call engineering team gets paged in PagerDuty.

02:48:00 UTC: Incident confirmed

The on-call engineer connects to Postgres and runs SELECT COUNT(*) FROM users. It returns 0.

03:12:00 UTC: Recovery starts

The team triggers the disaster-recovery runbook. They have automatic Postgres backups with 24h retention. The most recent backup is 23 minutes old at the time of the incident.

07:40:00 UTC: Service restored

After 5 hours and 26 minutes of total downtime (production available with data at 07:40 UTC), the service was restored. The transactions from the 23 minutes between the last backup and the incident were lost.

Failure path analysis

This incident had four independent points of failure. Each one was a chance to stop it, though not all of them guaranteed it:

1. The LLM generated a wrong query. gpt-3.5-turbo with temperature=0 is still a stochastic model. The same prompt could have produced the right query on another run. The problem is not the model: it is that there is no validation layer between the model and the database.
2. The schema available to the agent was incomplete. If last_active_at had been in the agent's context, the LLM would probably have generated the time filter. But even with that fixed, the risk remains: an agent with perfect context can still generate destructive queries from ambiguous prompts.
3. The database user had more permissions than it needed. A stricter least-privilege setup would have required a different role for a bulk DELETE. But, as described above, the team needed write permissions for legitimate support operations.
4. There was no AST validation in the critical path. A proxy layer with AST parsing would have detected DeleteStmt > WhereClause = NULL and blocked the query before it reached Postgres, with the offending AST node and a suggested safe query.

What AST parsing would have changed

With Vericto as a proxy between SQLDatabaseChain and Postgres, the incident flow would have been:

1. The LLM generates DELETE FROM users
2. The query reaches the Vericto proxy
3. The AST parser builds the tree: DeleteStmt > WhereClause = NULL
4. Rule VERICTO-001 fires
5. The query is blocked: Postgres never sees it
6. The agent receives a native Postgres error: Vericto blocked this query [VERICTO-001] — AST node: DeleteStmt > WhereClause = NULL. Suggestion: DELETE FROM users WHERE id = $1
7. The operator sees the blocked query in the Vericto dashboard, with the rule and the exact AST node

The LLM could even receive the error and regenerate the query with the right filter on the next iteration, without human intervention.

Incident lessons

Beyond this specific case, the scenario illustrates a general principle about LLM-to-database systems:

Four concrete lessons:

• The agent is not the only point of failure. Security controls must live in the infrastructure layer, not in the prompt or in the agent configuration.
• DB permissions do not understand semantics. A user with minimal permissions does not prevent destructive queries within the scope of those permissions.
• Backups are necessary but not enough. 5 hours of downtime plus 23 minutes of lost transactions is a significant business impact. Prevention costs orders of magnitude less than recovery.
• Explaining the block matters. A security layer that blocks without saying why slows down debugging. An actionable explanation with the exact AST node matters as much as the block itself.

What to do today

If you have an LLM-to-SQL pipeline in production (or in staging, where you experiment with agents), these are the three minimum steps we recommend:

1. Keep the CRITICAL rules for DELETE without WHERE (VERICTO-001), UPDATE without WHERE (VERICTO-042), DROP TABLE (VERICTO-010) and TRUNCATE (VERICTO-011) active. They are on by default and they are the ones that would have stopped this scenario. The full catalogue, with 31 rules, is in the rules reference.
2. Set up real-time alerts for CRITICAL blocks (Team plan or above). The on-call team needs to know in seconds, not hours.
3. Start in observe mode in staging to understand which queries your agent generates before blocking in production. In observe mode, Vericto records what it would block, without blocking it.

You can connect Vericto in under 5 minutes by changing the host in your connection string. For LangChain, LlamaIndex or any tool that uses a standard Postgres or MySQL driver, the proxy is transparent.