Skip to content

Customized API endpoint might returns "usage": null rather than {"p… #482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
pillsilly wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Customized API endpoint might returns "usage": null rather than {"p… #482

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions api/openai_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -260,10 +260,37 @@ def parse_embedding_response(
r"""Parse the embedding response to a structure Adalflow components can understand.

Should be called in ``Embedder``.
Handles cases where the API returns usage=null (e.g. some gateway proxies like NVDC).
"""
try:
return parse_embedding_response(response)
except Exception as e:
# If the standard parser fails (e.g. usage=None from some gateways),
# fall back to manual parsing that only requires the embeddings data.
if response is not None and hasattr(response, 'data') and response.data:
try:
from adalflow.core.types import Embedding, Usage
embeddings = [
Embedding(embedding=item.embedding, index=item.index)
for item in response.data
if item.embedding
]
# Build usage only if available, otherwise use zeros
if response.usage is not None:
usage = Usage(
prompt_tokens=response.usage.prompt_tokens,
total_tokens=response.usage.total_tokens,
)
else:
usage = Usage(prompt_tokens=0, total_tokens=0)
model = getattr(response, 'model', None)
log.warning(
f"Standard embedding parse failed ({e}), "
f"recovered {len(embeddings)} embeddings manually."
)
return EmbedderOutput(data=embeddings, model=model, usage=usage)
except Exception as e2:
log.error(f"Fallback embedding parsing also failed: {e2}")
log.error(f"Error parsing the embedding response: {e}")
return EmbedderOutput(data=[], error=str(e), raw_response=response)
Comment on lines +292 to 295
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

There is an issue with the error handling here. If the fallback parsing fails and this except block for e2 is entered, the code logs e2 but then proceeds to log the original error e and returns an EmbedderOutput with error=str(e). This is misleading because the more immediate cause of failure is e2.

The returned error should reflect that the fallback mechanism also failed. I suggest modifying this block to return immediately with a more informative error message that includes details from both exceptions.

                except Exception as e2:
                    error_msg = f"Standard parsing failed ('{e}') and fallback parsing also failed ('{e2}')"
                    log.error(error_msg)
                    return EmbedderOutput(data=[], error=error_msg, raw_response=response)
            log.error(f"Error parsing the embedding response: {e}")
            return EmbedderOutput(data=[], error=str(e), raw_response=response)


Expand Down