feat(py/plugins/google-genai): Add Virtual Try-On 001 on VertexAI#5181
feat(py/plugins/google-genai): Add Virtual Try-On 001 on VertexAI#5181
Conversation
Adds virtual-try-on-001 image editing to the VertexAI plugin. - New models/virtual_try_on.py module with VirtualTryOnConfig, VirtualTryOnModel, and helper functions mirroring the JS ImagenTryOnConfigSchema and the Go PR #5177 design. - Caller identifies person vs product images via a "type" key in ai.Part metadata (PART_METADATA_TYPE_PERSON_IMAGE and PART_METADATA_TYPE_PRODUCT_IMAGE), matching the cross-runtime convention used by Go. - The google-genai Python SDK does not expose a VTO method, so the model driver issues the :predict call directly through the SDK's authenticated _api_client.async_request, so credentials, quota headers, and tracing come from the plugin's existing client config. - Empty predictions surface as a FinishReason.BLOCKED ModelResponse (matching the Go post-review behaviour), not an exception. - VertexAI plugin registers the hardcoded known-model list so VTO appears in Init and list_actions even when client.models.list() does not surface it. - Adds unit tests for the helpers and generate() flow.
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for Virtual Try-On models in the Google GenAI plugin, including the VirtualTryOnModel driver, configuration schemas, and logic for processing person and product image media parts. Unit tests were also added to ensure correct request formatting and response parsing. Feedback was provided regarding the need for consistent error handling when calling the underlying API client to ensure exceptions are properly mapped to Genkit errors.
| 'genkit:input', | ||
| json.dumps({'model': self._version, 'body': payload}, default=str), | ||
| ) | ||
| http_response = await api_client.async_request('POST', path, payload) |
There was a problem hiding this comment.
The direct call to api_client.async_request lacks the error handling found in other model drivers in this plugin (e.g., GeminiModel). If the API returns an HTTP error (4xx/5xx), the SDK will raise a google.genai.errors.ClientError. This should be caught and re-raised as a genkit.GenkitError with the appropriate status mapping to ensure consistent error reporting within Genkit.
1 participant
Adds
virtual-try-on-001image editing to the VertexAI backend.Callers tag input images with
metadata={'type': 'personImage' | 'productImage'}on theai.Part— same convention as the Go PR #5177. The google-genai Python SDK does not expose a VTO method, so the model driver issues the:predictcall directly through the SDK's authenticated_api_client.async_request, reusing the plugin's existing credentials / quota headers / tracing. Empty predictions surface as aFinishReason.BLOCKEDresponse, not an exception.Test plan
uv run pytest plugins/google-genai/test/models/googlegenai_virtual_try_on_test.py -v— 9 unit tests (helpers + mocked generate flow) pass.uv run ruff check/ruff format --checkclean.GOOGLE_CLOUD_PROJECT,GOOGLE_CLOUD_LOCATION, and two local images, invoke a flow that tags one aspersonImageand one asproductImageand confirm an image MediaPart is returned.