- Genkit's middleware allows you to intercept and modify the AI generation cycle using model hooks, tools, and general flow.
- It includes native features for automatic retries, fallback between models, file control, and human tool approval.
- It is compatible with TypeScript, Go, and Dart, making it easy to create layers of security and observability without altering the business logic.
If you build AI applications, you know it's not all about writing the perfect prompt. When you move from a prototype to something that actually works in production, you run into obstacles like API timeouts, quota limits, or the need for human validation of critical actions. This is where Genkit Middleware comes in , a Google solution designed to allow developers to add layers of control without having to rewrite all of their agent code.
Basically, we're talking about a system of interceptors that's placed around the generation calls, instead of wrapping each function. generate() In a manual control block, you can define a series of steps that the request must go through. This allows your agents' architecture to be... much cleaner and more modular, separating the application logic from cross-cutting concerns such as system security, observability, and resilience.
How exactly does the interception system work?
To understand this better, think about the lifecycle of a request in Genkit. Typically, the model generates a response, decides whether it needs to use a tool, executes that tool, and reprocesses the result. Genkit's middleware isn't simply a bottleneck; it offers three distinct anchor points for intervening in this process:
- Generate Hook: It runs only once per generation cycle. It's the ideal site for inject instructions into the system or modify the messages before they start flying.
- Hook by Model: It wraps the direct call to the model's API. It is essential for managing automatic retries, implement caches or log how long it takes the LLM to respond.
- Hook Tool: It acts just as a tool is about to run. It's perfect for creating approval systems where a human must give approval before the agent deletes a file or makes a payment.
This structure behaves similarly to the middleware found in frameworks like Express or Koa, functioning like an onion: the request enters through the outer layer, reaches the center (the model), and then exits again, traversing the layers in reverse order. The best part is that you can stack multiple middlewares in an array, thus defining the exact execution order.
Catalog of pre-installed middlewares
So that we don't have to reinvent the wheel, the package @genkit-ai/middleware It already comes with several ready-to-use tools. One of the most useful is... retrieswhich uses an exponential backpedaling algorithm with jitter. This prevents us from overloading the API when there are transient errors such as RESOURCE_EXHAUSTED o UNAVAILABLEmaking the application much more robust in the event of momentary failures.
If your primary model's quota runs out, fallback middleware comes to the rescue. It allows you to configure a list of alternative models so that, if the first one fails, the request is automatically routed to a smaller or cheaper one, preventing the end user from seeing an error. This is a vital strategy for maintaining service continuity in high-demand environments.
Regarding security, we have the middleware of toolapprovalThis system creates a whitelist of safe tools; any action not on that list will trigger an error. ToolInterruptErrorforcing the system to pause until someone confirms the action. This is compounded by the middleware of file systemwhich grants the model access to local files but strictly limited to a root directory to prevent AI from snooping where it shouldn't.
Finally, we have the functionality of skillsThis middleware scans folders for files SKILL.md and injects them into the system prompt. It's a very elegant way to manage agent knowledge, allowing it to upload specific skills on-demand without filling the main prompt with unnecessary text.
Custom Middleware Creation
When native options are insufficient, Genkit allows us to build our own interceptors using the function generateMiddlewareThis tool allows us to define configuration schemes using Zod, ensuring that the parameters we pass are correct and typedWe can create anything from simple loggers that record every entry and exit, to complex systems for writing sensitive data (PII).
A very powerful use case would be to implement a content filter . You could create a model hook that analyzes the response and, if it detects prohibited words or internal pricing data, blocks or cleans the response before it reaches the user. It's also possible to manage quotas per user by querying a database in the generation hook before allowing the call to proceed.
Differences between implementations and ecosystem
Although the concept is universal, there are nuances depending on the language. In TypeScript, the system is very dynamic and allows defining middleware in files. .prompt through the field useIn Go, the implementation is more rigid because it is a compiled language, which has required the creation of a middleware registration to be able to convert text names into functional objects during runtime.
It's important not to confuse Genkit with Google's Agent Development Kit (ADK). While the ADK is designed for complex and resource-intensive multi-agent systems built on GCP infrastructure, Genkit is the ideal option for adding agent capabilities to existing applications (web, mobile, etc.). Furthermore, the entire middleware flow can be monitored from the Genkit Developer UI , allowing you to trace exactly which hook modified the request and the path it took to reach the final result.
The arrival of this system transforms the generation function into a truly composable pipeline . By integrating error handling, file security, and human validation into a separate layer, agent development becomes a much more professional process and less prone to catastrophic errors in production.