Zillow Emphasizes Context in AI Architecture, Glean Partnership
At VB Transform 2026, Zillow's SVP of Engineering Toby Roberts highlighted that persistent context, rather than raw data, is the critical challenge in building effective AI architecture for its customers' complex real estate journeys. Zillow, which impacts about 80% of U.S. real estate transactions, developed its own context layer and uses task-specific models. The company partners with Glean, whose platform centralizes integration work and optimizes costs through model routing and precomputed context, reducing token consumption for repetitive tasks.
Zillow's SVP of Engineering, Toby Roberts, detailed the company's approach to AI architecture at VB Transform 2026, emphasizing the importance of a persistent context layer for customer interactions. Zillow, a real estate technology company, notes that its customers' journeys often span months or years and involve multiple touchpoints, from initial inquiries to loan officers and real estate agents. A single chatbot solution would be insufficient to maintain the necessary context across this extended process.
Roberts, alongside Glean co-founder and CEO Arvind Jain, explained that while establishing a robust data foundation with data mesh, clear lineage, and governance was foundational, the more significant challenge was building a system that remembers a customer's journey and carries that context forward across different interfaces. Zillow opted to build and own this context layer internally, leveraging its 20 years of machine learning experience, including products like Zestimate, by focusing on smaller, task-specific fine-tuned models rather than a single general-purpose model.
Internally, Zillow's custom AI harness operates in conjunction with Glean. Roberts stated that Zillow utilizes thousands of Glean agents in production to manage repetitive tasks, executing tens of thousands of operations across the company. Jain elaborated that Glean's platform centralizes integration work through its MCP gateway, aiming to prevent duplicated efforts across different departments like finance, legal, and marketing.
Glean's platform also functions as a cost-efficiency tool. Jain highlighted two mechanisms: model routing, which directs most tasks to smaller, more economical models instead of default frontier models, and precomputed context. This approach helps agents avoid expending tokens to assemble context from scratch, potentially cutting token consumption by as much as half. Jain underscored that models alone are not sufficient for enterprise AI automation without being connected to enterprise context.
The session offered several practical takeaways for enterprises developing agentic AI systems. These include establishing a measurement baseline before initiating an AI push, centralizing context integration, not relying solely on permission inheritance for regulated data, and recognizing context as a cost lever in addition to a capability.
According to VentureBeat, Roberts attributed a 40% increase in shipped code to AI adoption, crediting a DORA metrics baseline established years prior to the AI rollout itself.
