Skip to content

todo.md

Latest commit

 

History

History
99 lines (79 loc) · 8.89 KB

todo.md

File metadata and controls

99 lines (79 loc) · 8.89 KB

Lot 1 Tasks

User Management (Symfony Security)

  • Implement User entity with email as user identifier, password hash, roles array, status, timestamps; expose getters for UserInterface/PasswordAuthenticatedUserInterface. #auth #security
  • Configure security.yaml (password hashers, user provider, firewall, access control) aligned with Symfony Security defaults. #auth #config
  • Create Admin-only CRUD for users (listing, create, edit roles/status, reset password). #auth #ui #admin
  • Seed an initial Admin user via fixture or console command for first access. #auth #ops

GitHub Sync and Storage

  • Add RepositoryConfig entity (repo name, owner, url, default branch, encrypted token, last sync status/time/message). #github-sync #db
  • Service to validate repo connectivity/token scopes against GitHub API with clear error messages. #github-sync #backend
  • Manual sync job/command to fetch repository tree and persist DocumentNode cache. #github-sync #sync
  • Persist SyncLog entries tying sync runs to triggering user and status. #github-sync #observability
  • Redact tokens after save in UI and encrypt at rest in storage. #security #github-sync

UI

  • Tree view UI backed by cached DocumentNode data, showing path, size, last modified, last sync status/time. #ui #github-sync
  • Authorized action to trigger sync from the UI (roles: Admin/Reviewer). #ui #auth #github-sync
  • Admin UI to configure repository settings and token updates. #ui #admin #github-sync
  • Enforce role-based visibility for Admin/Reviewer/Viewer across pages and actions. #ui #auth

Utils

  • Use Symfony translations instead of raw string in the application (php and twig files) #utils
  • Complete PHPDoc for each functions, class and related items # utils

Lot 2 Tasks

Ingestion queue & storage

  • Create ingestion_queue_item entity (path, branch, sha, size_bytes, storage_path, status queued|processing|indexed|failed|download_failed, source, timestamps, message, FK repo/doc/user) persisted in MySQL. #db #ingestion #rag
  • Create ingestion_log entity linked to queue item (level info|warning|error, message, context json, created_at). #db #logs #ingestion
  • Extend document_node with ingestion_status + last_ingested_at to surface last known ingestion state. #db #ingestion #ui
  • Add rag_shared_dir app config and secure path building for downloaded artifacts. #config #ops #ingestion
  • On enqueue, download file from GitHub default branch and write to rag_shared_dir; mark status queued or download_failed with message. #ingestion #github-sync #fs
  • Enforce duplicate protection (no re-enqueue when queued/processing) and size/extension validation with user-friendly errors. #ingestion #validation #ui

UI (tree/table)

  • Add “Ajouter à la file” action on files (not folders) with toast/flash feedback and role gating. #ui #ingestion #auth
  • Render badges/colors for directory, unindexed, queued, processing, indexed, failed/download_failed states in tree and list views. #ui #ingestion
  • Detail panel shows ingestion status, last attempt, size/branch and links to item logs. #ui #logs #ingestion
  • Provide retry/re-index action for failed/download_failed items (creates/reuses queue item with audit of user). #ui #ingestion

Improvements

  • Create an extension guesser based on the name and / or path of a document to update the document type with accurate value #improvements
  • Fix form errors when creating / editing a user. #improvements

Lot 3 Tasks

Incremental sync and data integrity

  • Implement path+repository upsert for DocumentNode (update metadata if present, create if missing) without purge cycles. #lot3 #github-sync #db
  • Handle GitHub deletions via Doctrine SoftDeletable (no physical DELETE) to keep ingestion_queue_item / ingestion_log references intact. #lot3 #github-sync #db
  • Preserve ingestion fields during resync (no status/timestamp resets); log per-document errors, skip to next item, and avoid dirty partial writes. #lot3 #github-sync #observability
  • Optimize sync diffing with a path hash map to detect create/update/delete and avoid bulk delete/insert loops. #lot3 #github-sync #performance

Tree UI

  • Fix tree layout to avoid horizontal clipping; use ellipsis+tooltip for long paths and keep alignment/scroll within the canvas. #lot3 #ui #ux
  • Make “Add to queue” conditional: show on indexable files in unindexed/failed/download_failed, provide “Retry” for failed states (re-enqueue to queued), hide/disable with tooltip for queued/processing/indexed. #lot3 #ui #ingestion
  • Add tree filters (Indexed, Indexable, Failed, All) with the selection persisted during navigation. #lot3 #ui #ingestion

Cached documents list

  • Add columns: name+extension, path, ingestion status, ingestion date (last attempt/success), size in MB (2 decimals), branch, plus enqueue action with the same state rules as the tree. #lot3 #ui #ingestion
  • Default sort by most recent ingestion; allow sort by size/name; ensure displayed status comes from the latest queue/ingestion state (not overwritten by sync). #lot3 #ui #ingestion #github-sync

Ingestion logs view

  • Create a paginated ingestion logs screen with filters: level (info|warning|error), queue status, date range, document path search, enqueuing user. #lot3 #ui #logs #ingestion
  • Display each log row with timestamp, level, document, queue item status, message, source (user/system), and links to the related queue item/document. #lot3 #ui #logs #ingestion

Lot 4 Tasks

Data model & persistence

  • Add conversation table/entity (id UUID/int, user_id FK, title, created_at/updated_at/last_activity_at, deleted_at nullable) with per-user scoping. #lot4 #db #chat
  • Add conversation_message table/entity (conversation_id FK, role user|assistant|system, content text/json, source_documents json nullable, token_count nullable, status streaming|completed|error, error_message nullable, created_at, streamed_at, finished_at) ordered by created_at. #lot4 #db #chat
  • Add light_rag_request_log table/entity linking conversation/message/user with duration_ms, status success|error|cancelled, error_message, created_at for observability. #lot4 #db #observability

Backend chat flow

  • Expose REST endpoints to create/list conversations (per current user), fetch thread messages, soft delete/archive, and rename title (auto-generate from first prompt fallback). #lot4 #backend #chat
  • Implement send-prompt endpoint: persist user message, load latest N history messages, call LightRag /query/stream, and create assistant message in streaming status. #lot4 #backend #rag
  • Build Symfony SSE endpoint (e.g., /sse/chat/{conversationId}/{messageId}) that consumes LightRag streaming and forwards token|sources|done|error events with keep-alives. #lot4 #backend #sse
  • Handle cancellation: stop downstream stream, mark assistant message as error with reason cancelled_by_user, and emit terminal SSE event. #lot4 #backend #sse
  • Apply per-user rate limiting on prompt submission to prevent API flooding; return friendly error on limit hit. #lot4 #backend #security

UI/UX chat

  • Create chat workspace with left conversation list (title, last activity, unread/error indicator, “Nouvelle discussion” CTA) and right thread view with message bubbles + avatars. #lot4 #ui #chat
  • Display streaming assistant message live; lock send button while in-flight; show token counter/spinner and replace buffer on done. #lot4 #ui #sse
  • Surface errors inline on the thread with retry (resend last prompt) and cancellation control; prevent double-submit (debounce). #lot4 #ui #ux
  • Show citations/sources if provided in SSE sources event; render timestamps on messages. #lot4 #ui #rag
  • Empty state for no conversations with guidance and CTA to start a discussion. #lot4 #ui #ux

Security, isolation, and rules

  • Enforce that users only see their own conversations/messages; gate chat access to authenticated users regardless of role. #lot4 #security
  • Respect soft-delete: exclude archived conversations from list; keep data for audit unless hard delete is added later. #lot4 #backend #ux
  • Enforce ordering and context cap (e.g., last 20 messages) when building history for LightRag to avoid oversized payloads. #lot4 #backend #rag
  • Apply timeout for LightRag stream (e.g., 60s); on timeout, mark message error, persist reason, emit SSE error, and keep last known content. #lot4 #backend #observability

Observability & metrics

  • Log each LightRag request with conversation_id, message_id, user_id, duration, status/error for dashboards. #lot4 #observability
  • Capture token counts/cost metrics if exposed by LightRag and store on assistant messages for later reporting. #lot4 #observability #rag

Prepare prod deployement

  • Configure monolog and other necessary services for prod environment #prod-env !high