Skip to content

feat(py): generate middleware#5253

Draft
huangjeff5 wants to merge 8 commits intojh-dynamic-toolsfrom
jh-mw
Draft

feat(py): generate middleware#5253
huangjeff5 wants to merge 8 commits intojh-dynamic-toolsfrom
jh-mw

Conversation

@huangjeff5
Copy link
Copy Markdown
Contributor

Description here... Help the reviewer by:

  • linking to an issue that includes more details
  • if it's a new feature include samples of how to use the new feature
  • (optional if issue link is provided) if you fixed a bug include basic bug details

Checklist (if applicable):

@github-actions github-actions Bot added docs Improvements or additions to documentation python Python config labels May 7, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a robust middleware system for Genkit, enabling interception and modification of model generation, API calls, and tool executions. It introduces the genkit-plugin-middleware package, providing standard middleware for retries, fallbacks, tool approval, skills, and filesystem operations. Core generation logic was refactored to handle middleware normalization and per-call scoping. Feedback identifies redundant model copies in the asynchronous generation methods and a design conflict between validation tests and the normalization implementation. Furthermore, the reviewer noted blocking synchronous I/O in asynchronous tool implementations and issues with the jitter calculation in the retry middleware that could cause delays to exceed configured maximums.

Comment thread py/packages/genkit/src/genkit/_ai/_aio.py Outdated
Comment thread py/packages/genkit/src/genkit/_ai/_aio.py Outdated
Comment thread py/packages/genkit/tests/genkit/ai/generate_test.py
Comment thread py/plugins/middleware/src/genkit/plugins/middleware/_filesystem.py
Comment thread py/plugins/middleware/src/genkit/plugins/middleware/_retry.py Outdated
Comment thread py/plugins/middleware/src/genkit/plugins/middleware/_skills.py Outdated
Comment thread py/packages/genkit/src/genkit/_core/_plugin.py Outdated
Comment thread py/packages/genkit/src/genkit/_core/_middleware/__init__.py
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new middleware architecture for the Genkit framework, replacing the previous functional middleware approach with a class-based BaseMiddleware system. It adds support for middleware registration via MiddlewareDesc and MiddlewareRef, allowing for more flexible, configuration-driven middleware usage. The changes include the addition of a new genkit-plugin-middleware package containing implementations for Retry, Fallback, ToolApproval, Skills, and Filesystem, along with updates to the core engine to support these new middleware hooks and tool execution flows. No review comments were provided for this pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

config docs Improvements or additions to documentation python Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant