-
Notifications
You must be signed in to change notification settings - Fork 3.3k
feat(ai, openai): expose usage tokens for 'generateImage' function #10128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional Suggestions:
- The
ImageModelV3Usagetype is used in theImageModelV3interface but is not exported from the package, preventing users from importing it for their own type annotations.
View Details
📝 Patch Details
diff --git a/packages/provider/src/image-model/v3/index.ts b/packages/provider/src/image-model/v3/index.ts
index 9eb5613a1..9bb8f0336 100644
--- a/packages/provider/src/image-model/v3/index.ts
+++ b/packages/provider/src/image-model/v3/index.ts
@@ -4,3 +4,4 @@ export type {
} from './image-model-v3';
export type { ImageModelV3CallOptions } from './image-model-v3-call-options';
export type { ImageModelV3CallWarning } from './image-model-v3-call-warning';
+export type { ImageModelV3Usage } from './image-model-v3-usage';
Analysis
Missing export of ImageModelV3Usage type from v3 package
What fails: The ImageModelV3Usage type is used in the ImageModelV3 interface return type (line 108 in image-model-v3.ts for the optional usage field) but is not exported from packages/provider/src/image-model/v3/index.ts, preventing users from importing it for their own type annotations.
How to reproduce:
// This import fails:
import type { ImageModelV3Usage } from '@ai-sdk/provider';
// Or at the v3 level:
import type { ImageModelV3Usage } from '@ai-sdk/provider/v3';Results in TypeScript error: "Module '@ai-sdk/provider' has no exported member 'ImageModelV3Usage'."
Expected behavior: ImageModelV3Usage should be exported from the v3 package index alongside other v3 types (ImageModelV3, ImageModelV3CallOptions, ImageModelV3CallWarning), consistent with the pattern used in the language model package where LanguageModelV3Usage is exported from its v3/index.ts file.
Fix implemented: Added export type { ImageModelV3Usage } from './image-model-v3-usage'; to packages/provider/src/image-model/v3/index.ts
View Details
📝 Patch Details
diff --git a/packages/ai/src/types/index.ts b/packages/ai/src/types/index.ts
index 85c0d7e05..b9ae003fd 100644
--- a/packages/ai/src/types/index.ts
+++ b/packages/ai/src/types/index.ts
@@ -27,4 +27,4 @@ export type {
TranscriptionWarning,
} from './transcription-model';
export type { TranscriptionModelResponseMetadata } from './transcription-model-response-metadata';
-export type { EmbeddingModelUsage, LanguageModelUsage } from './usage';
+export type { EmbeddingModelUsage, ImageModelUsage, LanguageModelUsage } from './usage';
Analysis
Missing export of ImageModelUsage type in packages/ai/src/types/index.ts
What fails: ImageModelUsage is used in the public GenerateImageResult interface (packages/ai/src/generate-image/generate-image-result.ts, line 43) but is not exported from the types index module, making it impossible for users to import the type for their own type annotations.
How to reproduce:
// This fails with TypeScript error
import type { ImageModelUsage } from 'ai/types';
// Error: '"ai/types" has no exported member named 'ImageModelUsage'Result: TypeScript compilation error when users attempt to import ImageModelUsage from the types module, despite it being part of the public API interface returned by generateImage().
Expected: ImageModelUsage should be exported from packages/ai/src/types/index.ts alongside the other usage types (EmbeddingModelUsage and LanguageModelUsage) so users can import and annotate code with the return type of image generation functions.
Note: The type is defined in packages/ai/src/types/usage.ts and is used in the public GenerateImageResult interface, making this an inconsistency in the module exports.
|
for now, only support added to expose usage is for OpenAI provider. support for other providers will be added in future PRs |
can you please create a follow up issue for that so we don't forget? You can create an issue with the https://github.com/vercel/ai/labels/type:batch and add sub issues for each provider that needs the update as well, we can split up the work and invite the community to contribute |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could reasoningTokens and/or cachedInputTokens be relevant for images?
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i had initially added those 2 fields in the spec but updated it since OpenAI didn't return them. For the sake of consistency however, we can still add them but ultimately it would depend on if any provider returns that data
gr2m
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I feel like the token structure for observability should be structured the same across the different model types. We can revise before concluding the v6 beta
Background
experimental_generateImagedoesn't expose the token usage information returned by providers. this PR introduces token usage for OpenAI providerWe also update the ImageProvider spec so as to allow for usage tokens
See #8358
Summary
ImageModelUsageinputTokens,outputTokens,totalTokens]Manual Verification
examples/ai-core/generate-image/openai.tsChecklist
pnpm changesetin the project root)Future Work
Will need support for other providers for which we support image generation
#10150
Related Issues
Fixes #8358