A wealth manager had piloted an agentic assistant with 400 advisors. It answered client and portfolio questions by working through research, account data, product documentation and CRM notes, and advisors rated it highly. Each query involved many model calls and a large amount of retrieved context, and the pilot was costing $2.40 per query. At that price a rollout to all 6,000 advisors would have cost more than the business case allowed, and the program was at risk of stopping at the pilot.
Lydatum instrumented the assistant to show what each step of a query cost, then reworked the steps that accounted for the bulk of the spend:
Each change was tested against a set of 1,200 real advisor questions with graded answers, so the team could confirm answer quality held before anything was released. The wealth manager's own platform team took part in every step and now runs the evaluation and cost reviews itself.
Ten weeks after the work began, the cost of a query had fallen from $2.40 to $0.70.
Answer quality on the evaluation set stayed within one point of the pilot baseline. The firm approved the rollout to all 6,000 advisors and completed it in the following quarter, within the budget set for the program. Cost per query is now reported monthly next to adoption, and any new assistant feature has to state its expected unit cost before it is built.
Services Delivered: AI Unit Economics & ROI, Model Routing & Cost Optimization, AI Cost Visibility & Attribution, Agent Evaluation & Observability
Technologies Used: Anthropic Claude on Amazon Bedrock, LangGraph, OpenTelemetry, Langfuse, Snowflake, Python
Schedule a free consultation to discover how AI and Data can drive your business success