LLM output format compatibility issues with reasoning models (DeepSeek, etc.)

[linmengmeng-1314]

Description

When using reasoning models like DeepSeek V4 for HugeGraph-LLM RAG pipeline, several components fail due to LLM output format incompatibility. Reasoning models tend to wrap outputs in markdown code blocks or use different JSON structures than what the parsers expect.

Affected Components

1. Uvicorn reload=True in Docker deployment

• File: hugegraph-llm/src/hugegraph_llm/demo/rag_demo/app.py:205
• reload=True enables file watching which conflicts with Docker bind mounts, causing the service to hang
• Suggestion: Make reload configurable via environment variable (disable by default in production)

2. Keyword extraction parser incompatible with reasoning model output

• File: hugegraph-llm/src/hugegraph_llm/operators/llm_op/keyword_extract.py:146-180
• _extract_keywords_from_response expects KEYWORDS: word:score format, but reasoning models may wrap output in markdown or return different structures
• Workaround: Use KEYWORD_EXTRACT_TYPE=textrank instead of llm
• Suggestion: Improve parser robustness similar to the fix in fix(llm): improve graph JSON parsing robustness for LLM outputs #332 for graph extraction

Related PRs

• fix(graph): resolve edge creation failure due to vertex ID mismatch #331 - Fix edge creation failure due to vertex ID mismatch
• fix(llm): improve graph JSON parsing robustness for LLM outputs #332 - Fix property graph JSON parsing robustness

Environment

• HugeGraph Server: 1.7.0
• LLM: DeepSeek V4 Pro (via OpenAI-compatible API)
• Embedding: SiliconFlow BGE-M3

[dosubot]

Thanks for the detailed report! I looked into the affected components and can confirm your findings:

1. Uvicorn reload=True: Confirmed hardcoded at line 205 with no environment variable or CLI flag to control it [1]. Adding something like --reload to the existing argparse or an UVICORN_RELOAD env var would be a straightforward fix.

2. Keyword extraction parser: The _extract_keywords_from_response method [2] does raw regex splitting on KEYWORDS: tokens and colon/comma delimiters with no markdown code block handling whatsoever. This is in contrast to other parts of the codebase that already handle this — for example, GremlinGenerateSynthesize._extract_response() [3] and SchemaBuilder._extract_schema() [4] both use patterns like re.search(r"```(?:json)?\s*(.*?)```", response, re.DOTALL) to strip markdown fences before parsing.

The fix from PR #332 [5] uses a similar approach — stripping markdown code fences with re.sub(r"```\w*\n?", "", text) before attempting JSON extraction. Applying the same pattern to keyword extraction would be straightforward: strip any markdown fences from the LLM response before searching for the KEYWORDS: token.

A possible patch for keyword_extract.py would look something like:

# Strip markdown code blocks before parsing
import re
response = re.sub(r"```\w*\n?", "", response)
response = re.sub(r"```", "", response)
response = response.strip()

Added before the existing token-based extraction logic.

There might also be value in consolidating these markdown-stripping patterns into a shared utility, since at least 4 different operators implement their own version independently.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.