The manual review ceiling

Every enterprise scaling agent eventually asks the same question: how many can we run before a human can no longer keep up. This survey put a number on it, and the number is smaller than most AI roadmaps assume.

Among decision-makers using autonomous AI agents, 42% say manual human review becomes operationally or financially impossible at 16 or more deployed agents. For 19%, the ceiling is one to five agents, while 11% say it becomes impossible at 51 or more agents.

Source: Collibra/Harris Poll survey, p.7 (Q5)

That ceiling reflects the hallucination tax showing up as a staffing cost. Independent engineering research, not a Collibra result, offers a useful measure of the underlying gap: One widely cited engineering study found that agents querying a raw, ungoverned data warehouse answered correctly less than 21% of the time, a figure that rose above 95% once the same agents worked from governed definitions, then slid back to roughly 65% within a month once that governance was left unmaintained.1 That gap, between a confident wrong answer and a reliable right one, is close to what enterprises in this survey are already absorbing in review hours.

That pressure is already costing organizations real time. Among decision-makers at organizations using autonomous AI agents, 51% say they spend significant staff hours manually reviewing and correcting agent outputs before they go live, rising to 64% among decision-makers at organizations with $100 million or more in revenue.

The gap between ambition and readiness shows up just as clearly elsewhere. Among decision-makers at organizations using autonomous AI agents, 39% say leadership demands hard financial ROI from AI while the data foundation underneath it is not ready to support that demand. The same proportion (39%) say their governance policy has not kept pace with how quickly agents

are being deployed. Among decision-makers at organizations with $100 million or more in revenue that use autonomous AI agents, ROI pressure exceeding data foundation readiness rises to 52%, and governance lagging deployment reaches 46%.

Among decision-makers at organizations using autonomous AI agents, 29% have discovered agents accessing or surfacing data they should not have had permission to use, rising to 37% among decision-makers at organizations with $100 million or more in revenue. One in five (20%) have scaled back or restricted agent deployments because of ungoverned data retrieval, and 17% have done so because of unpredictable costs.

Source: Collibra/Harris Poll survey, p.7 (Q6)

A useful way to read these numbers together: More than fifteen agents is roughly where manual oversight begins to strain for 11% of decision-makers at organizations that use AI agents. The enterprises ahead of that curve are not the ones with more reviewers on staff. They’re the ones that replaced review with governed context and continuous control before the volume of agents made review impossible.

51% of decision-makers at enterprises that use autonomous AI agents spend significant staff time manually reviewing and correcting agent outputs before launch. Among those at $100M+ enterprises, that number is 64%.

https://claude.com/blog/how-anthropic-enables-self-service-data-analytics-with-claude