background image background image

EXCLUSIVE, EARLY RELEASE

Build Secure GenAI With an LLM Mesh

A Practical Guide to Using LLMs in the Enterprise

In this technical guide from O’Reilly and Dataiku, discover how the LLM Mesh addresses key challenges around adopting LLMs at scale, making it easier and safer for IT teams to harness the power of GenAI. 

IT Teams: Get Exclusive Access to Chapters 1 & 2
2017_GARTNER_quadrant-1

46% of Senior Analytics and IT Leaders Plan to Spend More Than $1 Million on GenAI in the Next
12 Months

This data, from a 2024 Dataiku and Cognizant survey of 200 senior analytics and IT leaders, highlights that GenAI is a top priority at the highest level — nearly three-quarters of respondents (73%) will spend over $500,000 on the tech in the next 12 months.

Make sure you can execute fast, at the right cost, with an eye to the future with an LLM Mesh approach.

 

Why an LLM Mesh?

The Dataiku LLM Mesh is the most comprehensive and agnostic LLM gateway offering on the market, partnering with the top Generative AI players for secure access to thousands of LLMs (both as a service or self-managed).

LLM Mesh_Web-Version-GREEN

The Importance of an LLM Mesh Architecture

The Key to Winning With LLMs in the Enterprise

What Exactly Is an
LLM Mesh?

An LLM Mesh is an architecture paradigm for building LLM-powered applications in the enterprise. 

An LLM Mesh should enable you to:

1. Access various LLM-related services through an abstraction layer.

2. Provide federated services for control and analysis.

3. Provide central discovery and documentation for LLM-related objects via a catalog.

When Should You Use an LLM Mesh?

The LLM Mesh is valuable when the total number of LLM-powered applications being developed in an organization begins to grow, more teams start building and using the applications, and more complex LLM-powered applications are being designed and built.

In this context, the LLM Mesh will accelerate the development of the applications, simplify their maintenance, and help to ensure that the applications meet enterprise standards for safety, security, and performance. 

The 7 Objects for Building LLM-Powered Applications

Building an LLM Mesh requires understanding the different types of objects that must interact with one another within an LLM-powered application. The LLM Mesh provides "abstraction" between the high-level object and the underlying, specific service.

Discover the seven types of objects that are used in building LLM-powered applications in the enterprise: applications, agents, prompts, LLM services, LLMs, tools, and retrieval services. 

“A wise LLM strategy, once you have moved beyond initial prototypes, is to use the right model for the right application. This is easier said than done. LLMs are easy to use out of the box. You don’t even need to know how to code! However, using them in production is a different challenge, and it requires careful consideration of security, safety, performance, and cost.”

— Kurt Muehmel and the Dataiku team

“The power of an LLM Mesh architecture is in reducing the cost of introducing an incremental model into production use in an organization. Once set up properly, an LLM Mesh architecture can drive the cost of using an additional LLM in the enterprise to nearly zero. In such a context, an enterprise is free to develop an optimal AI strategy without compromising on performance, cost, safety, or security.”

— Kurt Muehmel and the Dataiku team

“The LLM Mesh represents a pivotal step in AI. At Dataiku, we’re bridging the gap between the promise and reality of using Generative AI in the enterprise. We believe the LLM Mesh provides the structure and control many have sought, paving the way for safer, faster GenAI deployments that deliver real value.” 

— Clément Stenac, Chief Technology Officer and Co-Founder, Dataiku

Read Ch. 1 & 2 of the O'Reilly + Dataiku Guide Today