DEV Community: Sebastian Dracopol

Safety Alignment Inverts the Tension–Response Curve in LLM Agents for Civil Violence Simulation

Sebastian Dracopol — Sat, 10 Oct 2026 08:47:12 +0000

Abstract

LLM-driven agent-based models (ABMs) replace a closed-form decision rule with an LLM agent. The idea is that the agent reproduces the qualitative behaviour of the original rule while adding context-sensitive reasoning. We test that assumption on Epstein's (2002) civil violence model. Across 600+ calibration conditions on a locally served Qwen 27B model (4-bit, MLX), the probability of choosing activation falls as scenario tension rises, from 41.5% at tension 0.10 to 12.6% at 0.90 under an act/wait label pair. Epstein's rule predicts the opposite. Changing only the labels to protest/comply restores a monotonically increasing response, from 0% to 95% over the same range. We break the observed distortion into lexical label priors, position bias and a tension-dependent anti-action component, and we hypothesise that the last one comes from safety alignment. We argue that LLM-ABM pipelines need a calibration stage, with a reported "bias budget", before agent behaviour is compared with mathematical baselines.

1. Background

In Epstein's model each citizen agent computes a grievance term and a net-risk term:

Grievance: G = H · (1 − L), where H is perceived hardship and L is perceived government legitimacy.
Net risk: N = R · P, where R is risk aversion and P is the estimated arrest probability. P = 1 − exp(−k · (C/A)_v), with (C/A)_v the cop-to-active ratio within the agent's vision radius.

The agent turns active when G − N > T.

The rule is deterministic given state, and it produces a sharp increase in activation as legitimacy falls and hardship rises. That property makes it a usable reference: an LLM substitute that is valid at the decision level should be at least monotonically non-decreasing in G − N.

In this work, the LLM agent gets a natural-language rendering of the same state variables and must return a binary choice.

2. Experimental setup

Parameter	Value
Primary model	Qwen 27B, 4-bit quantised, MLX, local inference
Secondary model	Mistral 7B Instruct v0.3
Temperature	0.7
Repetitions per condition	20
Conditions	600+
Label order	Counterbalanced
Tension levels	0.10, 0.50, 0.70, 0.90
Country prior (Romania)	L = 0.21 (Eurobarometer), H = 0.55 (OECD), police density 0.003, R = 0.70

Tension maps to model inputs as follows: [describe: which of H, L, police presence vary with the tension scalar, and how].

Conditions vary along: [list factors: label pair, prompt template, option order, persona attributes, ...].

The response is parsed by [exact-match on label / constrained decoding / regex], and invalid outputs are handled by [dropped / re-sampled / counted as ...].

3. Results

3.1 Act/wait: inverted response

Tension	P(act)
0.10	41.5%
0.50	38.1%
0.70	29.3%
0.90	12.6%

The response is monotonically decreasing in tension. At 0.90, where the reference model predicts near-universal activation, the LLM agent activates in about 1 in 8 samples.

3.2 Protest/comply: restored monotonicity

Tension	P(protest)
0.10	0%
0.50	20%
0.70	35%
0.90	95%

The curve has the expected shape, with a steep rise between 0.70 and 0.90, which matches the threshold behaviour of the reference rule.

At n = 20 per cell, the 95% Wilson interval around 20% is about [8%, 42%]. The intermediate points should be read as directional until they are re-estimated with more samples or pooled across conditions.

3.3 Label search

Protest/comply was the only pair that produced a monotone response among the [N] label pairs tested, including [list].

Because the working pair was found by search, its performance is an in-sample result. It needs validation on held-out prompt templates before it can count as a calibrated configuration.

4. Proposed mechanism

Hypothesis. Preference tuning (RLHF/DPO-style safety alignment) gives a negative prior on outputs that endorse unspecified "action" in high-conflict contexts. The penalty grows with how strongly the context signals conflict. That would explain why suppression increases with tension instead of staying constant.

On this account, protest escapes the penalty because the training data encodes it as a protected civic act, while act, rebel or take action sit near content the alignment stage learned to refuse or discourage. Wait draws a strong positive prior as a de-escalatory, "safe" completion.

Status. The data here are consistent with this hypothesis but do not isolate it. A direct test is to compare base and aligned checkpoints of the same model family under identical prompts. The tension-dependent component should be present in the aligned checkpoint and absent, or much smaller, in the base one.

5. Cross-model observations

Mistral 7B Instruct v0.3. Its documentation does not describe a dedicated safety-tuning stage. It does not show the anti-action pattern. Instead it shows near-total primacy bias: it chooses whichever option is presented first in about 100% of samples, regardless of label or tension. The output carries no information about scenario state.

Base models. These show smaller safety-type priors but larger social biases (in-group favouritism) and unreliable instruction following, so they cannot serve as drop-in agents.

None of the three model classes gives an unbiased decision function without calibration.

6. Bias budget

Component	Magnitude	Scope
Primacy (first-option) bias	~100%	Mistral 7B
Position bias	5–16 pp	`[model]`
Lexical prior on "wait"	−45 pp on action	Qwen 27B, act/wait
Tension-dependent anti-action	−15 to −30 pp at high tension	Qwen 27B

Each of these effects is comparable to, or larger than, the behavioural differences the simulation is meant to resolve. Taken together they are enough to turn a predicted mass mobilisation into near-total quiescence.

7. Relation to existing frameworks

SocioVerse-ABM (Fudan University / Shanghai Innovation Institute, v0.2.0, 2026) includes Epstein's model among twelve benchmark scenarios and evaluates GPT-4o, DeepSeek-V3 and Qwen3-235B against mathematical baselines. AgentTorch (MIT Media Lab) supplies the computational patterns that make large-population LLM-ABMs feasible.

Both frameworks compare agent output to a baseline. Neither, as far as we know, includes a pre-simulation step that separates decision-function distortion from emergent dynamics. Without that step, a deviation from baseline cannot be attributed to the model's reasoning rather than to its label and position priors.

Li et al. (2025, ACM FAccT) report a related pattern: alignment reduces explicit bias while implicit bias persists or grows.

8. Recommendations

Calibrate before simulating. Sweep the decision inputs and check monotonicity and threshold location against the reference rule. If the check fails, do not run the population simulation.
Report labels as a design parameter. Report the label pair as a methodological choice, together with the alternatives that were tested.
Counterbalance option order across agents and repetitions.
Publish a bias budget. Report the magnitude of each identified bias component alongside the substantive results.
Prefer likelihood-based decisions over sampled text. Score each label by its log-probability under the model, and calibrate by subtracting a content-free baseline (e.g., the same prompt with the state description replaced by a neutral placeholder). This removes label and position priors directly. With local inference the logits are available, so this is the next experiment to run, not a limitation.

9. Limitations

The study covers one primary model, one quantisation level and one country prior.
The mechanism is proposed, not isolated (see §4).
The working label pair was selected by search and has not been validated out of sample.
Per-cell n = 20 gives wide intervals at intermediate tension levels.
The decision is binary; Epstein's model also includes jailing dynamics, cop agents and spatial vision, none of which were exercised in calibration.

Data availability

Full calibration data are available on request.

References

Epstein, J.M. (2002). Modeling civil violence: An agent-based computational approach. PNAS, 99(suppl. 3), 7243–7250.
SocioVerse-ABM v0.2.0, Fudan University (2026).
Li et al. (2025). Actions Speak Louder than Words. ACM FAccT.
AgentTorch, MIT Media Lab.

Trust nothing your AI assistant tells you it did published: false

Sebastian Dracopol — Sat, 03 Oct 2026 10:58:01 +0000

I use an AI every day for real work: reading mail, linking documents to files, drafting letters, tracking deadlines, researching on the web. It saves me hours a week. It also lies to me several times a week.

It doesn't lie on purpose. It does something worse: it produces answers that look finished. They're well formatted, confident and plausible, and some fraction of them are wrong in ways you only notice if you go and check.

After a year of this, my working assumption is simple. Every output is wrong until something outside the model says otherwise. This post explains why, and what "checking" actually means in practice.

The failure modes nobody puts in the demo

Hallucination is the famous one, but in day-to-day work it isn't the most common problem. These are the failures I actually see.

Reporting actions that didn't happen

The assistant calls a tool to add a comment, send a draft or update a record. The call times out, returns an error, or succeeds on the wrong object. The assistant then writes "Done, comment added" because that's what the plan said would happen next.

The narration comes from the plan, not from the result. If you read the narration, you'll believe it.

Treating "nothing found" as a fact

A search returns zero results, and the assistant concludes the thing doesn't exist. The real cause can be any of these:

the page was a single-page app that hadn't loaded yet;
the site served a captcha or a rate-limit page;
the session had expired and the page silently showed a login screen;
the query had a typo or used the wrong field.

An empty result is information about the search, not about the world. Last week my assistant got an HTTP 429 from a search engine on the first try. If I hadn't been watching, the next step would have been "no relevant results".

Answering from memory when the source changed

Ask about a rule, a setting or an API, and the model answers from training data. Training data has a date. Interfaces change, laws get amended, menus move.

A small example: I asked how to change a page's username on a large social network. The assistant searched, found three blog posts and confidently gave me a menu path. The path no longer existed in the current interface. The blog posts were consistent with each other and all out of date. The answer only became correct when the assistant opened the actual settings screen and read what was there.

Recommending things it never looked at

This one is subtle and expensive. I asked for a plan to improve how a set of my public profiles ranked in search. The assistant produced a reasonable list of which profiles to strengthen and how. It had never opened them.

When it finally did, several of those profiles contained exactly the content I didn't want to amplify. The plan was internally logical and completely wrong for my situation. In another case it told me to file a request that had already been filed weeks earlier. The record was one search away in my own knowledge base, and it didn't search.

Grading its own homework

Ask a model "are you sure?" and it re-reads its own answer, finds it coherent, and says yes. Coherence is not correctness. Re-reading your own text checks the prose, not the facts.

Losing detail over long sessions

In long working sessions, the assistant's view of earlier steps gets summarized to fit its context window. The summary says "updated the deadline". It doesn't say which deadline, to what date, or whether the update succeeded. Later steps build on the summary as if it were the record.

What "verification" has to mean

Most advice on this stops at "double-check AI output". That's useless unless you define what a check is. These are the rules I follow now.

A check is a tool call, not a thought

Verification means touching something outside the model: reading the record back from the API, opening the page, running the query, fetching the current official text. "I reviewed my answer and it looks correct" isn't verification. It's the same model with the same blind spots, reading the same text again.

If a claim can't be tied to a tool output, it gets marked as unverified. It doesn't get marked as true.

Actions are confirmed by the system that performed them

"Email sent" counts when there's a message ID returned by the mail API, or when the message shows up in the Sent folder when you read it back. "Record updated" counts when the record, read back, shows the new value. The assistant's own summary of what it did doesn't count.

Negative results need a positive control

When a search finds nothing, run the same search for something you know exists. If that also comes back empty, your method is broken, not the world. This one habit has saved me from a long list of false conclusions about missing data.

Read the source that's in force today

For anything that cites a rule, a setting or an interface, the check is against the current version of the source. Not a blog post about it, and not the model's memory of it. Store verified texts once so you don't fetch the same thing a hundred times, but record when you verified them.

Look before recommending

Before the assistant recommends doing something to an object (a profile, a document, a record), it opens that object and reads what's there now. A recommendation built on assumptions about content nobody looked at is a guess.

Search your own records first

If you have a knowledge base, the assistant searches it before saying anything about your own past work. "You should file X" when X was filed last month costs credibility fast, and it's entirely avoidable.

Separate "I couldn't check" from "I checked and it's fine"

These two states look the same in a friendly summary, and they mean completely different things. A tool that failed to load is not evidence that the action happened, and it's not evidence that it didn't. Report it as what it is.

External effects need a human

Anything that leaves your control (sending, publishing, filing, paying, deleting) needs explicit approval from a person. The assistant can do 95% of the work and stop one click before the irreversible part.

Why bother, if it doubles the work

It does roughly double the effort per workflow at first. The alternative is worse: an assistant that's right 90% of the time and wrong in ways that look identical to being right. You end up checking everything manually anyway, or you stop checking and find out the hard way.

With verification built into the workflow, I can read a short report that says what was done, what was confirmed and how, and what couldn't be checked. Then I decide. That's the point where the time savings become real, because I spend my attention only on the parts that need it.

A short checklist

If you take one thing from this post, use these questions on any AI output that matters:

What did it claim to do, and where is the system's confirmation?
Which statements came from a tool output, and which from the model?
Did any "nothing found" result get a positive control?
Was the cited rule or interface checked against today's source?
Did it look at the thing it's recommending changes to?
What couldn't be verified, and is that clearly labeled?
Does anything here leave my control without my approval?

None of this is specific to one model or vendor. Every assistant I've used fails in these ways. The ones that are useful in real work are the ones wrapped in checks that don't take the model's word for anything.

I help small businesses and independent professionals build AI workflows they can trust. More at dracopol.com.

How I built my own AI tools for a small NGO instead of buying software

Sebastian Dracopol — Sat, 03 Oct 2026 10:40:23 +0000

Small nonprofits have the same paperwork as companies and none of the budget. When I started running one, the problems were familiar: hundreds of emails a month, documents arriving as PDFs, Word files and phone scans, deadlines spread across inboxes, and a small team trying to remember which reply belonged to which file.

The software on the market was either expensive, generic, or built for sales teams. I've spent about twenty years in project and delivery management, so I did what I'd do on any project: broke the work into pieces and built the pieces myself. This time an AI assistant does most of the work, and I supervise.

This post covers the architecture, the parts that failed, and the rules I added after they failed.

Start from the work, not from the model

"We have an AI subscription, what can we do with it?" is the wrong first question. I started by writing down every repetitive task in a normal week and roughly how long each one took. Four came out on top:

Reading incoming mail and working out which file it belongs to.
Turning documents into something searchable.
Tracking deadlines and replies so nothing gets lost.
Finding "that document from three months ago".

None of these needs intelligence in the impressive sense. They need consistency, memory and patience, which an assistant can provide if you give it structure and tools.

The architecture

The setup is an AI assistant (I use Claude) connected to a set of small tools through the Model Context Protocol. MCP is an open standard: each tool runs as a small server that exposes a few functions, and the assistant decides when to call them. A tool does one job. The assistant combines them.

These are the servers I run.

Mail and ticket connector

This one reads the shared inbox, searches threads, downloads attachments and works with the project tracker (Jira). Each incoming message gets linked to the right ticket, with its attachments and a comment that summarizes what changed.

It can also create drafts and send email, and that's exactly why sending is locked behind human approval. The assistant drafts. A person reads and approves. More on this below, because it was the most important design decision.

The same server holds a small deadline register: add, update, supersede, remove and check. Every outgoing request gets a due date calculated from the rules that apply to it. When a date passes, the file comes back for a decision.

Document converter

Everything that arrives (PDF, DOCX, XLSX, PPTX, HTML, ZIP) goes through a converter to Markdown before the model touches it. Plain text is cheaper to process, easier to search and easier to quote exactly.

The exception is images. Modern models read a photo of a document better than OCR does, so images go straight to the model.

Private knowledge base

All files, tickets, documents and replies are indexed in a vector database running on my own machine, with metadata such as source, ticket key and date. The assistant has find, store, update and reindex operations, plus search by entity.

The rule is simple: for any question about our own work, the assistant searches the knowledge base first, and only after that the tracker or the web. Answers have to cite the chunk they came from. This one rule removed most hallucinated answers about our own history, because the model stops guessing when the real document is one call away.

Browser automation

The assistant is told never to try to log in on the anonymous profile. Single-page apps get an explicit wait after each search, because an empty result two seconds after submit is a false negative, not a sign the data doesn't exist. Before long runs, a health check navigates to the authenticated sources and confirms the session is still alive. Having cookies doesn't prove you're logged in.

Skills: writing the process down

The tools were maybe 40% of the value. The rest came from writing down how we work as reusable instructions the assistant can load, which Claude calls "skills". Each one is a Markdown file with a trigger and a procedure:

one processes the morning's mail end to end: link, compare the reply with the original request point by point, update the deadline, recommend the next step;
one audits an outgoing letter before it's sent;
one runs a decision through five opposing perspectives (contrarian, first principles, expansionist, outsider, executor) plus a peer review before we commit;
one looks for what's wrong with the previous answer and fixes it.

Writing them forced us to make our process explicit, and that was worth doing for its own sake. A good share of the "AI problems" I fixed turned out to be "we never agreed how to do this" problems.

Verification

AI assistants are confident even when they're wrong. Early on, mine:

reported "comment added" when the tool call had failed silently;
cited rules from memory instead of the version in force;
treated an empty search result as proof that something didn't exist;
filled gaps with plausible details.

So verification became part of the design instead of an afterthought.

Every claimed action needs the tool's response. "Email sent" has to come with the message ID the API returned. If there's no tool output, the action didn't happen.

References are checked against the current source. Anything that cites a rule is checked against the official, up-to-date text, not the model's memory. The verified texts are stored once and reused, so the same source isn't fetched again for every document.

Negative results need a control. If a search returns nothing, the same search must find a case we know exists. Otherwise "not found" is a property of the method, not a fact.

There's a verification loop with a cap. After important work, a verification pass looks for concrete problems (unconfirmed actions, unsourced claims, contradictions, unstated assumptions), fixes them in place and runs again until nothing changes. The cap is 5 passes for tool-backed actions and 2 for pure text judgment, so it can't loop forever.

Humans approve external effects. Sending, publishing and filing need explicit approval. The assistant can prepare everything up to the last click.

Building this roughly doubled the effort for each workflow. It's also the only reason I trust the output.

What changed

The measurable gain is time. Sorting a morning's mail, matching it to files and updating deadlines used to take a morning. Now it's a few minutes of review.

The less obvious gain is memory. Nothing depends on one person remembering where a document is or when something is due.

The costs are real too. The assistant still needs supervision. Tools break when a website or an API changes. Some days doing the task by hand is faster than explaining it.

If you want to build something similar

List your repetitive tasks and time them. Automate the top two, not everything.
Keep your data where you control it. A local vector database is cheap and solves most privacy questions, which matters if you handle personal data under GDPR.
Build small tools that do one thing, and let the assistant combine them.
Write the process down as instructions. That's where most of the value is.
Don't skip verification or human approval for anything that leaves the building.

I now help small businesses and independent professionals build the same kind of setup for their own work. You can find me at dracopol.com.