0.7 alpha 2

.cursor/rules/backend-test-conventions.mdc

@@ -0,0 +1,63 @@
+---
+description: Backend Python test conventions (pytest)
+globs: tests/backend/**/*.py
+alwaysApply: false
+---
+
+# Backend Test Conventions
+
+## File Location & Naming
+
+- Place tests under `tests/backend/unit/`, `tests/backend/integration/`, or `tests/backend/contract/`.
+- Name files `test_<feature_or_module>.py` with descriptive snake_case names.
+- Shared test data goes in `tests/backend/fixtures/`.
+
+## File Structure
+
+```python
+"""One-paragraph summary of what is being tested and why.
+
+Background
+----------
+Brief context about the feature or bug fix these tests validate.
+"""
+from __future__ import annotations
+
+import pytest
+
+pytestmark = [pytest.mark.backend]
+```
+
+## Conventions
+
+- Always add `pytestmark = [pytest.mark.backend]` at module level.
+- Group related tests in classes prefixed with `Test` (e.g. `TestStripImageBlocks`).
+- Use `pytest.mark.parametrize` for data-driven tests instead of writing repetitive cases.
+- Use `pytest.fixture()` for shared setup; keep fixtures close to where they are used.
+- Unit tests must **not** depend on Flask, network, or external services.
+- Mock external calls with `unittest.mock.patch` / `MagicMock`; never make real API calls.
+- Each test function should verify **one** behavior and have a clear name: `test_<what>_<condition>`.
+
+## Running Tests
+
+- When running pytest, use `-q` (quiet mode) instead of `-v` (verbose mode) for cleaner output.
+- **Preferred:** `python -m pytest tests/backend/ -q`
+- **Avoid:** `python -m pytest tests/backend/ -v 2>&1`
+
+## Example
+
+```python
+# ❌ BAD – vague name, no parametrize, no marker
+def test_it_works():
+    assert sanitize("hello world") == "hello_world"
+
+# ✅ GOOD
+pytestmark = [pytest.mark.backend]
+
+@pytest.mark.parametrize("raw,expected", [
+    ("hello world", "hello_world"),
+    ("订单明细", "订单明细"),
+])
+def test_sanitize_preserves_unicode(raw: str, expected: str) -> None:
+    assert sanitize(raw) == expected
+```

.cursor/rules/error-response-safety.mdc

@@ -0,0 +1,61 @@
+---
+description: Prevent information exposure through exception messages in HTTP responses
+globs: py-src/**/*.py
+alwaysApply: false
+---
+
+# Error Response Safety
+
+Never return raw exception text (`str(e)`, `f"...{e}"`) directly in HTTP responses.
+Python exceptions may contain stack traces, file paths, database connection strings,
+API keys, or internal IP addresses — all of which are security risks (CWE-209).
+
+## Rules
+
+1. **5xx errors** — return a fixed generic message; never expose exception details.
+2. **502 errors** — return `"Upstream service unavailable"`; never include upstream error body.
+3. **4xx errors** — run `sanitize_error_message(str(e))` so business-validation messages
+   stay useful while secrets are stripped.
+4. **Logging** — always log the full exception server-side (`logger.warning` / `logger.error`
+   with `exc_info=True` when needed). The client never needs the stack trace.
+
+## How To
+
+For Flask route `except` blocks, use `safe_error_response`:
+
+```python
+from data_formulator.security.sanitize import safe_error_response
+
+except HTTPError as e:
+    return safe_error_response(e, 502, log_message="Upstream call failed")
+except ValueError as e:
+    return safe_error_response(e, 400, log_message="Invalid input")
+except Exception as e:
+    return safe_error_response(e, 500, log_message="Unexpected error")
+```
+
+For non-route contexts (generators, background tasks) where a Flask response
+cannot be returned, use `sanitize_error_message` directly:
+
+```python
+from data_formulator.security.sanitize import sanitize_error_message
+
+except Exception as exc:
+    logger.error("Task failed: %s", exc, exc_info=True)
+    payload = {"status": "error", "message": sanitize_error_message(str(exc))}
+```
+
+## Common Mistakes
+
+```python
+# ❌ BAD — raw exception leaks internal details
+return jsonify({"message": str(e)}), 500
+return jsonify({"message": f"Failed: {e}"}), 502
+
+# ❌ BAD — manual traceback in response
+import traceback
+return jsonify({"message": traceback.format_exc()}), 500
+
+# ✅ GOOD
+return safe_error_response(e, 500, log_message="Operation failed")
+```

.cursor/rules/frontend-test-conventions.mdc

@@ -0,0 +1,64 @@
+---
+description: Frontend TypeScript test conventions (Vitest)
+globs: tests/frontend/**/*.test.{ts,tsx}
+alwaysApply: false
+---
+
+# Frontend Test Conventions
+
+## File Location & Naming
+
+- Place tests under `tests/frontend/unit/` mirroring the `src/` structure:
+  - `tests/frontend/unit/data/` → tests for `src/data/`
+  - `tests/frontend/unit/app/` → tests for `src/app/`
+  - `tests/frontend/unit/views/` → tests for `src/views/`
+- Name files `<functionOrFeature>.test.ts` (or `.test.tsx` for React rendering tests).
+
+## File Structure
+
+```typescript
+import { describe, it, expect } from 'vitest';
+// For React rendering tests:
+// import { render } from '@testing-library/react';
+
+import { myFunction } from '../../../../src/<path>';
+
+describe('myFunction', () => {
+  it('should handle <specific case>', () => {
+    expect(myFunction(input)).toBe(expected);
+  });
+});
+```
+
+## Conventions
+
+- Import `describe`, `it`, `expect` explicitly from `vitest` (globals are enabled but explicit imports improve readability).
+- Use `@testing-library/react` and `@testing-library/jest-dom` for component rendering tests.
+- Prefer testing **exported pure functions** over testing internal component state.
+- When component logic is complex, extract it into an exported helper and test that directly.
+- Group tests with `describe` blocks; use section comments (`// --- Null cases ---`) for clarity.
+- One assertion per `it` block when possible; name tests as `should <expected behavior>`.
+- Do **not** import from `node_modules` internals; only use public API.
+- Keep tests independent — no shared mutable state between `it` blocks.
+
+## Example
+
+```typescript
+// ❌ BAD – no describe, vague test name
+import { expect, test } from 'vitest';
+test('works', () => { expect(fn(1)).toBe(2); });
+
+// ✅ GOOD
+import { describe, it, expect } from 'vitest';
+import { checkIsLikelyTextOnlyModel } from '../../../../src/views/DataLoadingThread';
+
+describe('checkIsLikelyTextOnlyModel', () => {
+  it('returns true for deepseek-chat', () => {
+    expect(checkIsLikelyTextOnlyModel('deepseek-chat')).toBe(true);
+  });
+
+  it('returns false for undefined', () => {
+    expect(checkIsLikelyTextOnlyModel(undefined)).toBe(false);
+  });
+});
+```

.cursor/rules/i18n-no-hardcoded-strings.mdc

@@ -0,0 +1,40 @@
+---
+description: No hardcoded UI strings — use i18n translation keys
+globs: src/**/*.{ts,tsx}
+alwaysApply: false
+---
+
+# i18n: No Hardcoded UI Strings
+
+All user-visible text in the frontend MUST go through the i18n system. Never hardcode Chinese, English, or any other language string directly in components.
+
+## How to Use
+
+```tsx
+import { useTranslation } from 'react-i18next';
+
+const { t } = useTranslation();
+
+// ✅ GOOD
+<Button>{t('common.save')}</Button>
+<Tooltip content={t('chart.noData')} />
+
+// ❌ BAD
+<Button>保存</Button>
+<Button>Save</Button>
+<Tooltip content="No data available" />
+```
+
+## Translation Files
+
+- English: `src/i18n/locales/en/<namespace>.json`
+- Chinese: `src/i18n/locales/zh/<namespace>.json`
+- Namespaces: `common`, `upload`, `chart`, `model`, `encoding`, `messages`, `navigation`
+
+When adding a new key, add it to **both** `en` and `zh` locale files. Pick the namespace that fits; create a new namespace only if none applies.
+
+## What Counts as User-Visible
+
+Must use `t()`: button labels, tooltips, placeholders, error messages, dialog titles, tab names, toast notifications, table headers, empty-state text.
+
+May stay hardcoded: log messages (`console.log`), error messages thrown but never displayed, internal constants, CSS class names, test IDs.

.cursor/rules/language-injection-conventions.mdc

@@ -0,0 +1,21 @@
+---
+description: Language injection conventions for LLM Agent prompts
+globs: py-src/data_formulator/agents/**/*.py,py-src/data_formulator/agent_routes.py
+alwaysApply: false
+---
+
+# Language Injection Conventions
+
+Language flows per-request: `Frontend i18n → Accept-Language header → get_language_instruction() → system prompt`.
+
+## Rules
+
+1. **User-facing LLM output** MUST inject language via `get_language_instruction(mode=...)` in the route handler.
+2. **Mode selection:** `"full"` for text-heavy agents, `"compact"` for code-generation agents and short-text endpoints.
+3. **Inject into system prompt only** — append or insert before a marker, never into user messages.
+4. **Do NOT inject** for non-user-facing calls (health checks, internal tool calls).
+5. **Do NOT duplicate** — if upstream messages already contain language instruction, skip.
+6. **Do NOT** use env vars, global interceptors, or hardcoded language strings (e.g. `"回答请使用中文"`) — always use `build_language_instruction()`.
+7. **New language?** Add to `LANGUAGE_DISPLAY_NAMES` in `agents/agent_language.py` and add locale files in `src/i18n/locales/<lang>/`.
+
+For detailed architecture, code examples, and anti-pattern explanations, see the language-injection skill.

.cursor/rules/package-manager-conventions.mdc

@@ -0,0 +1,12 @@
+---
+description: Use Yarn only, never npm/pnpm
+globs: package.json, yarn.lock
+alwaysApply: true
+---
+
+# Package Manager Rules
+
+- Use Yarn v1.22.22 only - never use npm or pnpm
+- Never manually edit yarn.lock
+- Keep yarn.lock changes minimal when adding deps
+- Registry must be https://registry.yarnpkg.com

.cursor/skills/language-injection/SKILL.md

@@ -0,0 +1,72 @@
+# Language Injection for Agent Prompts
+
+Detailed guide for the language injection system. The short version lives in `.cursor/rules/language-injection-conventions.mdc`.
+
+## Architecture
+
+```
+Frontend i18n.language  →  Accept-Language header  →  get_language_instruction()
+                                                           │
+                                                   build_language_instruction()
+                                                   (agents/agent_language.py)
+                                                           │
+                                              ┌────────────┴────────────┐
+                                              ▼                         ▼
+                                        mode="full"               mode="compact"
+                                    (text-heavy agents)        (code-gen agents)
+```
+
+### Core Modules
+
+| Module | Role |
+|--------|------|
+| `agents/agent_language.py` | `build_language_instruction(lang, mode)` — generates prompt fragments; supports 20 languages; returns `""` for English |
+| `agent_routes.py` → `get_language_instruction()` | Reads `Accept-Language` header, delegates to `build_language_instruction` |
+| `src/app/utils.tsx` → `fetchWithIdentity()` | Sets `Accept-Language` header on every API request from `i18n.language` |
+
+## Code Examples
+
+### Route handler — inject language
+
+```python
+# In a Flask route handler:
+lang_instruction = get_language_instruction(mode="compact")
+lang_suffix = f"\n\n{lang_instruction}" if lang_instruction else ""
+
+messages = [
+    {"role": "system", "content": "You are a helpful assistant." + lang_suffix},
+    {"role": "user", "content": user_input},
+]
+```
+
+### Agent constructor — marker-based insertion
+
+```python
+if language_instruction:
+    marker = "**About the execution environment:**"
+    idx = self.system_prompt.find(marker)
+    if idx > 0:
+        self.system_prompt = (
+            self.system_prompt[:idx]
+            + language_instruction + "\n\n"
+            + self.system_prompt[idx:]
+        )
+    else:
+        self.system_prompt += "\n\n" + language_instruction
+```
+
+## Anti-Patterns (with explanations)
+
+| Pattern | Why it's wrong |
+|---------|---------------|
+| `os.environ.get("DF_DEFAULT_LANGUAGE")` | Process-level — all users get same language; breaks multi-user |
+| Global LLM client interceptor | Hidden behavior; can't distinguish full/compact mode; fragile string detection |
+| New `MessageBuilder` class | Duplicates `agent_language.py`; creates parallel conflicting abstractions |
+| Hardcoded `"回答请使用中文"` in prompts | Not configurable; skips the mode system; breaks for other languages |
+
+## Adding a New Language
+
+1. Add language code + display name to `LANGUAGE_DISPLAY_NAMES` in `agents/agent_language.py`.
+2. Optionally add extra rules to `LANGUAGE_EXTRA_RULES` (e.g. simplified vs traditional Chinese).
+3. Add frontend translations in `src/i18n/locales/<lang>/` — copy an existing locale folder as template.
+4. No Agent code changes needed — the existing flow picks up new languages automatically.

.dockerignore

@@ -0,0 +1,12 @@
+.git
+.gitignore
+node_modules
+__pycache__
+*.pyc
+.env
+.env.*
+*.egg-info
+dist
+build
+.pytest_cache
+.mypy_cache