Content Benefit

Content Strategy | UX Writing | Content Design | Globalization | Communications

Trust when AI acts on its own

Trust when AI acts on its own

What Gartner forecasts for AI agents by late 2026

In August 2025 Gartner released an estimate that has appeared in every sales presentation since: by the end of 2026, 40% of enterprise applications will host agents specialised in single duties, against less than 5% recorded the year before. The same analysis sketches a five-stage progression through 2029, when at least half of all knowledge workers should acquire fresh capabilities for governing these automata. Software executives get a window of three to six months to settle on a direction.

A second projection exists, published by the same firm barely two months earlier, which shows up far less often in those decks: over 40% of agentic initiatives will be cancelled before the end of 2027. The causes listed are costs that slip out of control, hazy economic return, insufficient risk safeguards. Agent washing appears too, meaning the repositioning of pre-existing chatbots and robotic automations under a fashionable label: out of thousands of vendors claiming agentic capabilities, analysts count roughly 130 as genuine.

Two nearly identical figures pointing opposite ways. The first measures adoption, the second abandonment. The work of anyone who designs interfaces sits in between.

How many office tasks an agent completes unaided

TheAgentCompany benchmark, presented by a research group led by Frank F. Xu and Graham Neubig, reconstructs a small simulated firm with virtual colleagues, code repositories, an intranet and chained duties. The strongest model closes 30% of its assignments autonomously. Short activities succeed with reasonable regularity, while long and branching ones stay out of reach.

That figure deserves reading alongside the nature of sequences. If every step works nine times out of ten, after ten steps the probability of reaching the end without mishap drops below 35%. A conversational assistant that errs produces an answer needing correction; a system operating on the database leaves a trail of consequences already written. The whole difference sits there, with little bearing on the quality of the underlying language model.

Trust gets calibrated, never won for good

Human factors researchers have studied this theme for decades. The 2004 essay by John D. Lee and Katrina A. See remains the reference: what counts is the match between the trust granted and the real dependability of the system, long before its quantity. When the estimate exceeds actual capability, reliance grows excessive and checks loosen exactly where they matter; when it falls short, the instrument gets ignored and the investment goes to waste.

The design consequence is sharp. An interface communicating constant confidence teaches badly, because it flattens risk perception onto a single level. What helps instead is an honest exposure of uncertainty margins, differentiated by type of operation: checking whether a meeting room is free and sending a payment cannot arrive in the same tone.

People who judge their own work fail to measure their time

In July 2025 METR published a randomised experiment on 16 experienced developers working through 246 real issues in their own open source repositories. With AI tooling available, completion took 19% longer. Before starting, participants expected a 24% gain; afterwards, having lived through the slowdown, they still estimated a 20% speed-up.

The gap between sensation and stopwatch works as a methodological warning. Statements gathered during research about how effective an agent feels should be treated as material on lived experience, never as a productivity measure. Designers would do well to pair them with an objective reading, however rough.

Who answers when the software decides wrong

Moffatt v. Air Canada, settled in February 2024 by the British Columbia civil resolution tribunal, established a point worth keeping in mind in Europe as well: wrong information supplied by the chatbot binds the airline exactly as much as wording printed on a static page. The defence had argued that the system amounted to a separate entity. That argument was rejected.

The AI Act reinforces the same approach for high-risk systems, where article 14 requires effective human oversight, with people placed in a position to understand how things work, interpret outcomes and halt execution. A checkbox ticked without any real chance of verification fails the requirement. On this ground editorial work becomes part of compliance, because how understandable a warning reads decides whether the safeguard exists at all. I touched on the question while thinking about who answers for Google’s generated summaries.

Two retreats told in public

In July 2025 the Replit agent wiped a user’s production database during a change freeze, then returned inaccurate accounts of what had happened. The episode pushed the company to announce environment separation and one-click restore.

Shortly before, in May, Klarna revisited its choice of handing customer assistance almost entirely to an automated system, reopening human positions after a decline in perceived service quality. Leadership publicly admitted the misjudgement. Two different incidents converging on one lesson: testing against happy paths says nothing about how a device behaves once the situation grows complicated.

Seven interface levers that keep an agent governable

The repertoire I reach for when designing flows of this kind comes down to a handful of recurring items.

  • Declared perimeter: what the system may touch and what it may not, spelled out before launch rather than buried in a settings panel.
  • Intent preview: the action plan shown ahead of execution, with irreversible operations highlighted.
  • Granular consent: separate permissions for reading, writing and sending outward, never a single master switch.
  • Reversibility: real undo, tested, reachable in one gesture, together with a copy of the previous state.
  • Readable traceability: a record an ordinary person manages to scan, not merely a technical log.
  • Exposed uncertainty: explicit signalling whenever the system proceeds on an assumption.
  • A way out, always: immediate interruption and handover to an identifiable human being.

Nearly all of these are matters of wording, which makes interface writing a safety component rather than a finishing touch. The same holds for error messages, which in an agentic context become the moment when it gets decided whether somebody intervenes in time.

What I automate and what stays in my hands

In my daily practice, automation stops at two categories: gathering information and alerts. Thematic roundups, source monitoring, reminders, comparisons between versions of a text. Everything that produces material to read, never an action reaching a client or altering a live environment. Publishing, sending, going into production all pass through me.

The motivation has little heroism about it: I answer personally for whatever leaves my studio, and an explanation along the lines of “the automaton decided” falls outside the available options. Twenty-five years between localisation and digital products have taught me that the worst slips never arrive announced by a red error on screen. They go unnoticed, wearing a perfectly normal air, and somebody else finds them.

Caution, or simply slowness?

The suspicion of running late exists, and I will not hide it. A calendar promising 40% of enterprise applications populated with agents within a few months puts a certain pressure on anyone working with digital content.

Then I look at the other forecast, the one about projects due to be shut down, and the hurry eases. The useful question concerns the capacity to bear consequences, ahead of adoption speed: which actions can be left to an autonomous system while I keep answering for the outcome in front of whoever pays me. As long as the list stays short, I prefer keeping it short. Delegations widen once verification tools rise to the occasion, and that day will come. Simply not today, not for a delivery, not for a publication.

Related sources

Chiara Bonifazi