feat: adding AI Gateway support by Kylejeong2 · Pull Request #1627 · browserbase/stagehand

Kylejeong2 · 2026-01-28T03:16:31Z

why

New open source models are released, and don't necessarily have a great inference provider at all times. Seeing as we're already using the AI-SDK, it makes sense to add support for the AI Gateway.

what changed

Added Vercel AI Gateway as a provider to the llm providers list.

test plan

Testing with Evals

Summary by cubic

Adds Vercel AI Gateway as a provider to route model calls across vendors with a single API key; docs now cover gateway/{provider}/{model} usage, AI_GATEWAY_API_KEY setup, examples for Anthropic/Google, and optional baseURL config.

Also enforces JSON outputs for Kimi and DeepSeek, sets Kimi temperature=1, and fixes eval parsing by using the first slash in provider/model strings.

^{Written for commit 514142f. Summary will update on new commits. Review in cubic}

changeset-bot · 2026-01-28T03:16:36Z

⚠️ No Changeset found

Latest commit: 514142f

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

greptile-apps · 2026-01-28T03:19:20Z

Greptile Overview

Greptile Summary

This PR adds Vercel AI Gateway support to enable routing model calls through the gateway, along with model-specific fixes for Kimi and DeepSeek.

Key changes:

Added gateway provider to AISDKProviders and AISDKProvidersWithAPIKey for AI Gateway routing
Enforced temperature=1 for Kimi models (API constraint)
Added explicit JSON schema prompts for DeepSeek and Kimi models to ensure structured output compliance
Fixed model name parsing in evals to handle multiple slashes correctly using substring instead of split
Mirrored changes in both core AISdkClient and eval AISdkClientWrapped for consistency

Concerns:

The gateway import from "ai" package may need verification - check if it should be imported from "@ai-sdk/gateway" instead

Confidence Score: 3/5

Safe to merge after verifying the gateway import source
The changes are well-structured and consistent across files. The main concern is the potentially incorrect import source for gateway/createGateway which could cause runtime issues. Once verified, this should be safe to merge.
packages/core/lib/v3/llm/LLMProvider.ts - verify gateway import source

Important Files Changed

Filename	Overview
packages/core/lib/v3/llm/LLMProvider.ts	Adds AI Gateway support by importing gateway and createGateway from 'ai' package, but import source may be incorrect
packages/core/lib/v3/llm/aisdk.ts	Adds Kimi model support with temperature=1 constraint and forces JSON output for DeepSeek/Kimi models
packages/core/package.json	Adds @ai-sdk/gateway ^3.0.25 as optional dependency

Sequence Diagram

sequenceDiagram
    participant User
    participant LLMProvider
    participant AISdkClient
    participant AIGateway
    participant Model

    User->>LLMProvider: getClient(modelName)
    
    alt modelName contains "/"
        LLMProvider->>LLMProvider: Parse provider/model (substring)
        LLMProvider->>LLMProvider: Check if provider is "gateway"
        
        alt provider is "gateway"
            LLMProvider->>AIGateway: createGateway(clientOptions)
            AIGateway-->>LLMProvider: gateway provider instance
        end
        
        LLMProvider->>AISdkClient: new AISdkClient(model)
        AISdkClient-->>LLMProvider: client instance
    end
    
    User->>AISdkClient: createChatCompletion(options)
    
    alt model is Kimi
        AISdkClient->>AISdkClient: Force temperature=1
    end
    
    alt response_model exists AND (DeepSeek OR Kimi)
        AISdkClient->>AISdkClient: Append JSON schema instruction
    end
    
    AISdkClient->>Model: generateObject/generateText()
    Model-->>AISdkClient: response
    AISdkClient-->>User: formatted response

greptile-apps

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

miguelg719 · 2026-01-29T01:21:03Z

@Kylejeong2 would users have to define gateway/openai/gpt-5 as the model string?

Kylejeong2 · 2026-01-31T06:30:38Z

@Kylejeong2 would users have to define gateway/openai/gpt-5 as the model string?

Yeah that is the current flow, but I can update it to make more ergonomic

greptile-apps

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

cubic-dev-ai

1 issue found across 6 files

Confidence score: 5/5

Only minor wording clarity issue in packages/evals/lib/AISdkClientWrapped.ts; it doesn’t affect runtime behavior and is low severity.
Score reflects low risk since the concern is documentation/terminology rather than functional change.
Pay close attention to packages/evals/lib/AISdkClientWrapped.ts - update the prompt wording to say JSON schema format for clarity.

Prompt for AI agents (all issues)


Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="packages/evals/lib/AISdkClientWrapped.ts">

<violation number="1" location="packages/evals/lib/AISdkClientWrapped.ts:153">
P2: Misleading terminology: the prompt says 'zod schema format' but `toJsonSchema()` produces a JSON Schema, not a Zod schema. Consider changing the wording to 'JSON schema format' for clarity.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant App as App / Eval Engine
    participant LLMP as LLMProvider.ts
    participant Client as AISdkClient
    participant AISDK as Vercel AI SDK
    participant GW as NEW: AI Gateway
    participant Provider as Model Provider (Kimi/DeepSeek)

    Note over App,Provider: Runtime Request Flow with AI Gateway & Model Optimization

    App->>LLMP: getAISDKLanguageModel(provider, model)
    alt NEW: Provider is "gateway"
        LLMP->>AISDK: createGateway(config)
    else Standard Provider
        LLMP->>AISDK: createProvider(config)
    end
    LLMP-->>App: languageModel instance

    App->>Client: createChatCompletion(options)
    
    rect rgb(23, 37, 84)
    Note right of Client: CHANGED: Model-specific Logic
    opt modelId contains "kimi"
        Client->>Client: NEW: Force temperature = 1
    end

    opt modelId contains "deepseek" OR "kimi"
        Client->>Client: NEW: Inject Zod Schema & JSON instructions into prompt
    end
    end

    alt generateObject (Structured Data)
        Client->>AISDK: generateObject(model, schema, messages)
        AISDK->>GW: Request with Gateway Headers
        GW->>Provider: Route to Inference API
        Provider-->>GW: JSON Response
        GW-->>AISDK: Relay Response
        AISDK-->>Client: Typed Object
    else generateText
        Client->>AISDK: generateText(model, messages)
        AISDK->>GW: Request with Gateway Headers
        GW->>Provider: Route to Inference API
        Provider-->>GW: Text Response
        GW-->>AISDK: Relay Response
        AISDK-->>Client: Text
    end

    Client-->>App: LLMResponse

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

cubic-dev-ai · 2026-02-01T00:10:02Z

+
+        formattedMessages.push({
+          role: "user",
+          content: `Respond in this zod schema format:\n${parsedSchema}\n


P2: Misleading terminology: the prompt says 'zod schema format' but toJsonSchema() produces a JSON Schema, not a Zod schema. Consider changing the wording to 'JSON schema format' for clarity.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/evals/lib/AISdkClientWrapped.ts, line 153: <comment>Misleading terminology: the prompt says 'zod schema format' but `toJsonSchema()` produces a JSON Schema, not a Zod schema. Consider changing the wording to 'JSON schema format' for clarity.</comment> <file context> @@ -136,13 +137,30 @@ export class AISdkClientWrapped extends LLMClient { + + formattedMessages.push({ + role: "user", + content: `Respond in this zod schema format:\n${parsedSchema}\n +You must respond in JSON format. respond WITH JSON. Do not include any other text, formatting or markdown in your output. Do not include \`\`\` or \`\`\`json in your response. Only the JSON object itself.`, + }); </file context>

Suggested change

content: `Respond in this zod schema format:\n${parsedSchema}\n

content: `Respond in this JSON schema format:\n${parsedSchema}\n

miguelg719 · 2026-02-05T22:13:30Z

@Kylejeong2 would users have to define gateway/openai/gpt-5 as the model string?

Yeah that is the current flow, but I can update it to make more ergonomic

That's fine let's just cover it in the docs!

Kylejeong2 · 2026-02-05T22:39:50Z

it

Yeah I can add into docs in separate PR, can you approve?

tkattkat · 2026-02-05T22:40:09Z

+    "@ai-sdk/gateway": "^3.0.25",
    "@ai-sdk/google": "^2.0.23",


is this the right version? usually v3 ai sdk packages = only works with v6

im thinking about removing, since the actual package that we're pulling from is "ai" not the /gateway i added this per cubic's recommendation

docs for reference: https://ai-sdk.dev/providers/ai-sdk-providers/ai-gateway

tkattkat · 2026-02-05T22:50:57Z

    "@ai-sdk/cerebras": "^1.0.25",
    "@ai-sdk/deepseek": "^1.0.23",
+    "@ai-sdk/gateway": "^3.0.25",


believe we can also just directly import gateway from ai without needing to add a new dependency

yeah removing this

# why Users need to know how to use the AI gateway provider in stagehand # what changed added docs to the models section to show users how to use the ai gateway # test plan  --- ## Summary by cubic Added first-class support for the Vercel AI Gateway and updated docs so you can use models across providers with a single key and simple model strings. Also improved structured outputs for Kimi/DeepSeek and fixed eval parsing for provider/model names. - **New Features** - Added gateway provider to LLMProvider (use model: "gateway/{provider}/{model}"). - Updated docs with AI Gateway usage, examples, and env var guidance. - For DeepSeek/Kimi, added schema prompt to ensure JSON structured outputs; Kimi uses temperature=1. - Consistent temperature handling in object/text generation. - Improved evals to parse model strings using the first slash. - **Migration** - Set AI_GATEWAY_API_KEY in your environment. - Use model names prefixed with gateway/{provider}/{model}; optional baseURL supported. <sup>Written for commit 560366a. Summary will update on new commits. <a href="https://cubic.dev/pr/browserbase/stagehand/pull/1670">Review in cubic</a></sup>

Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>

greptile-apps Bot reviewed Jan 28, 2026

View reviewed changes

This comment was marked as resolved.

Sign in to view

Kylejeong2 marked this pull request as draft January 28, 2026 20:38

Kylejeong2 marked this pull request as ready for review February 1, 2026 00:04

greptile-apps Bot reviewed Feb 1, 2026

View reviewed changes

Comment thread packages/core/lib/v3/llm/LLMProvider.ts

cubic-dev-ai Bot reviewed Feb 1, 2026

View reviewed changes

tkattkat reviewed Feb 5, 2026

View reviewed changes

Kylejeong2 requested review from miguelg719 and tkattkat February 9, 2026 03:21

Kylejeong2 added 4 commits February 8, 2026 19:43

feat: adding AI Gateway support

ff002b6

updated eval modelname parsing

0b31c76

forcing kimi models to return json from gateway

a81298a

removed the ai gateway dependency

e5cf56d

Kylejeong2 force-pushed the kylejeong/stg-1225-adding-vercel-ai-gateway-support-to-stagehand branch from 42a1c50 to e5cf56d Compare February 9, 2026 04:45

miguelg719 reviewed Feb 10, 2026

View reviewed changes

Comment thread packages/docs/v3/configuration/models.mdx Outdated

miguelg719 reviewed Feb 10, 2026

View reviewed changes

Comment thread packages/docs/v3/configuration/models.mdx Outdated

miguelg719 approved these changes Feb 10, 2026

View reviewed changes

Kylejeong2 and others added 2 commits February 10, 2026 11:34

Update packages/docs/v3/configuration/models.mdx

96efb79

Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>

Update packages/docs/v3/configuration/models.mdx

514142f

Co-authored-by: Miguel <36487034+miguelg719@users.noreply.github.com>

shrey150 approved these changes Feb 10, 2026

View reviewed changes

Kylejeong2 merged commit 5c38f99 into main Feb 10, 2026
28 of 29 checks passed

	content: `Respond in this zod schema format:\n${parsedSchema}\n
	content: `Respond in this JSON schema format:\n${parsedSchema}\n

Conversation

Kylejeong2 commented Jan 28, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

why

what changed

test plan

Summary by cubic

Uh oh!

changeset-bot Bot commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

greptile-apps Bot commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Overview

Greptile Summary

Confidence Score: 3/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This comment was marked as resolved.

Uh oh!

miguelg719 commented Jan 29, 2026

Uh oh!

Kylejeong2 commented Jan 31, 2026

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cubic-dev-ai Bot Feb 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

miguelg719 commented Feb 5, 2026

Uh oh!

Kylejeong2 commented Feb 5, 2026

Uh oh!

tkattkat Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Kylejeong2 Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Kylejeong2 Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

tkattkat Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Kylejeong2 Feb 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Kylejeong2 commented Jan 28, 2026 •

edited by cubic-dev-ai Bot

Loading

changeset-bot Bot commented Jan 28, 2026 •

edited

Loading

greptile-apps Bot commented Jan 28, 2026 •

edited

Loading

cubic-dev-ai Bot Feb 1, 2026 •

edited

Loading