Skip to content

Conversation

@b3nw
Copy link
Contributor

@b3nw b3nw commented Jan 31, 2026

Description

Fix tool_choice=auto causing HTTP 422 on Dedaluslabs provider

Dedaluslabs API returns HTTP 422 when tool_choice is passed as a string ("auto") instead of an object:

Error code: 422 - {'detail': [{'type': 'model_attributes_type', 'loc': ['body', 'tool_choice'], 
'msg': 'Input should be a valid dictionary or object to extract fields from', 'input': 'auto'}]}

This caused LiteLLM's async_streaming function to catch the UnprocessableEntityError, retry twice, then exit the retry loop without returning a value or raising an exception - implicitly returning None. This resulted in:

AttributeError: 'NoneType' object has no attribute '__aiter__'

Since "auto" is the default tool_choice behavior for OpenAI-compatible APIs, removing it from the request fixes the issue without any functional change.

Root cause investigation notes: The issue was traced through multiple layers - initially suspected to be a LiteLLM bug, but hotpatching debug logging into the running container revealed the actual 422 error being silently caught and swallowed by LiteLLM's retry logic.

Testing Done

  1. Hotpatched running production container with debug logging to trace the actual error
  2. Confirmed fix by hotpatching the transform into production and observing successful requests
  3. Compared with Firmware provider (which also uses Claude Opus 4.5 via a gateway) - Firmware works because its gateway normalizes/accepts tool_choice: "auto" as a string
  4. Verified both providers use identical LiteLLM code paths - the difference is purely in upstream API parameter requirements

Test trace before fix:

DEBUG async_streaming: caught UnprocessableEntityError: Error code: 422 - {'detail': [{'type': 'model_attributes_type', 'loc': ['body', 'tool_choice'], 'msg': 'Input should be a valid dictionary or object to extract fields from', 'input': 'auto'}]}
DEBUG acompletion: init_response type=coroutine, response type=NoneType, is_none=True

After fix - requests complete successfully.

Checklist

  • I have tested these changes locally
  • I have added license headers to new files (LGPL for library, MIT for proxy)
    N/A - modifications to existing file only
  • I have updated documentation (README/DOCUMENTATION.md) if needed
    N/A - internal transform, follows existing pattern
  • Related issue: # N/A

Important

Fixes HTTP 422 error for Dedaluslabs by removing tool_choice='auto' in transforms.py.

  • Behavior:
    • Removes tool_choice='auto' for Dedaluslabs in _transform_dedaluslabs_tool_choice() in transforms.py to prevent HTTP 422 errors.
    • Ensures no functional change as 'auto' is default behavior.
  • Testing:
    • Hotpatched production container to trace and confirm fix.
    • Verified successful requests post-fix.
    • Compared with Firmware provider to confirm issue is specific to Dedaluslabs.

This description was created by Ellipsis for dda8805. You can customize this summary. It will automatically update as commits are pushed.

Dedaluslabs API returns HTTP 422 when tool_choice is passed as a string
('auto') instead of an object. Since 'auto' is the default behavior anyway,
removing it from the request fixes the issue.

This was causing LiteLLM to silently return None after exhausting retries,
which then caused AttributeError: 'NoneType' object has no attribute '__aiter__'
when attempting to iterate the stream.
@b3nw b3nw requested a review from Mirrowel as a code owner January 31, 2026 01:30
@mirrobot-agent
Copy link
Contributor

I'm starting my review of the Dedaluslabs tool_choice fix. Investigating the transforms and will have a detailed report for you soon.

Copy link
Contributor

@mirrobot-agent mirrobot-agent bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall Assessment

This PR effectively addresses a compatibility issue with the Dedaluslabs provider. By introducing a targeted transform to remove tool_choice="auto" when it matches the default behavior, it avoids the HTTP 422 error that previously caused silent failures in LiteLLM's retry logic.

The detailed investigation provided in the description is commendable and ensures the fix is both safe and necessary.

Architectural Feedback

The addition to ProviderTransforms is consistent with existing patterns (e.g., _transform_iflow_stream_options) and correctly isolates provider-specific quirks from the core logic.

Key Suggestions

None. The implementation is concise and correct.

Questions for the Author

None.

This review was generated by an AI assistant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant