philip lelyveld The world of entertainment technology

One year of agentic AI: Six lessons from the people doing the work

...

It’s not about the agent; it’s about the workflow

Achieving business value with agentic AI requires changing workflows. ...

Agents aren’t always the answer

AI agents can do a lot, but they shouldn’t necessarily be used for everything. Too often, leaders don’t look closely enough at the work that needs to be done or ask whether an agent would be the best choice to perform that work. ...

The important thing to remember is not to get trapped in a binary “agent/no agent” mindset. Some agents can do specific tasks well, others can help people do their work better, and in many cases, different technologies altogether might be more appropriate. ...

Stop ‘AI slop’: Invest in evaluations and build trust with users 

... Any efficiency gains achieved through automation can easily be offset by a loss in trust or a decline in quality. ...

Make it easy to track and verify every step

... So when there’s a mistake—and there will always be mistakes as companies scale agents—it’s hard to figure out precisely what went wrong.

Agent performance should be verified at each step of the workflow. Building monitoring and evaluation into the workflow can enable teams to catch mistakes early, refine the logic, and continually improve performance, even after the agents are deployed. ...

The best use case is the reuse case

...

Deciding how much to invest in building reusable agents (versus an agent that executes one specific task) is analogous to the classic IT architecture problem where companies need to build fast but not lock in choices that constrain future capabilities. How to strike that balance often requires a lot of judgment and analysis. ...

Humans remain essential, but their roles and numbers will change

...

People will need to oversee model accuracy, ensure compliance, use judgment, and handle edge cases, for example. And as we discussed earlier, agents will not always be the best answer, so people working with other tools such as machine learning models will be needed. ...

Another big lesson from our experience is that companies should be deliberate in redesigning work so that people and agents can collaborate well together. Without that focus, even the most advanced agentic programs risk silent failures, compounding errors, and user rejection. ...

See the full story here: https://www.mckinsey.com/capabilities/quantumblack/our-insights/one-year-of-agentic-ai-six-lessons-from-the-people-doing-the-work