Skip to content

Conversation

@Libao-Yang
Copy link
Contributor

LiteLLMServing supports embedding services and implements real embedding calls through the generate_embedding_from_input function. However, an error occurs during the initialization process. The error log is as follows:

During handling of the above exception, another exception occurred:
.venv/lib/python3.11/site-packages/litellm/main.py:3773: in completion
    response = base_llm_http_handler.completion(
.venv/lib/python3.11/site-packages/litellm/llms/custom_httpx/llm_http_handler.py:524: in completion
    response = self._make_common_sync_call(
.venv/lib/python3.11/site-packages/litellm/llms/custom_httpx/llm_http_handler.py:246: in _make_common_sync_call
    raise self._handle_error(e=e, provider_config=provider_config)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.11/site-packages/litellm/llms/custom_httpx/llm_http_handler.py:4507: in _handle_error
    raise provider_config.get_error_class(
E   litellm.llms.ollama.common_utils.OllamaError: {"error":"\"all-minilm:latest\" does not support generate"}

The root cause is that LiteLLMServing.__init__() forcibly executes _validate_setup(), and _validate_setup() always calls litellm.completion(), which leads to a setup error when using an embedding model.

Copy link
Collaborator

@SunnyHaze SunnyHaze left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks!

@SunnyHaze SunnyHaze merged commit 68930ec into OpenDCAI:main Jan 20, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants