How Bloomberg scaled their AI Agents

The AI Engineer Summit concluded a few weeks ago in New York. If you missed it, don't worry- I'm here to share key insights on AI agents!

Anju Kambadur from Bloomberg demonstrated how they scaled AI agents at Bloomberg.

Simple Use Cases, Stringent Requirements

One of Bloomberg's first applications of generative AI to their products was the summarization of quarterly earnings calls. Research analysts need to stay updated with these calls, which can number in the tens during earnings season. The aim of AI summarization is to help research analysts save time by avoiding the need to read through entire earnings calls while still grasping the key points.

They started by applying large language models (LLMs) to this challenge, but it was more complex than anticipated. Ensuring accuracy is extremely important, and LLMs are prone to errors, including hallucinations.

Moving to Semi-Agentic Workflows

To enhance the AI application, Bloomberg transitioned to a semi-agentic workflow.

Key principles guiding the design of the semi-agentic workflow include:

• Graceful Error Handling: Given the stochastic nature of language models, which lack the well-defined structure of traditional APIs, the system must handle unexpected errors gracefully.
• Exposed Workflows: Workflows should be visible to facilitate easier checks. For instance, when retrieving the US Consumer Price Index (CPI) for the last five quarters, it's important to ensure that the CPI data corresponds accurately to those quarters.
• Flexible Agent Design: Agent designs can be reorganized over time. Agents may be structured vertically or horizontally, each with its trade-offs. Over time, Bloomberg has identified their optimal structure, specializing agents for specific tasks while maintaining more horizontal and general guardrails.

Iterating Their Way to Success

What stands out to me is the iterative approach Bloomberg took to achieve success, despite their expertise in traditional software. If you're applying AI in your enterprise, it's helpful to remind yourself to iterate continuously rather than expecting a sudden breakthrough.

How has your AI agent design evolved over time? Share your thoughts in the comments below!