feat: integrate LLM profiles into settings by malhotra5 · Pull Request #210 · OpenHands/agent-canvas

malhotra5 · 2026-05-09T03:42:39Z

Summary

This PR integrates the LLM profiles feature into the LLM settings page, allowing users to save, edit, rename, activate, and delete named LLM configurations.

Important: LLM profiles are only available in local agent-server mode. Cloud mode uses the original simple settings form (model selector + API key) since the cloud backend does not support LLM profiles yet.

Changes

New Files

API Service:

src/api/profiles-service/profiles-service.api.ts - API service for profile CRUD operations

Components:

src/components/features/settings/llm-profiles/llm-profiles-list-view.tsx - Main profile list view component
src/components/features/settings/llm-profiles/profile-row.tsx - Individual profile row with model info and context menu
src/components/features/settings/llm-profiles/profile-actions-menu.tsx - Context menu with Edit/Activate/Delete actions
src/components/features/settings/llm-profiles/profile-name-input.tsx - Input field with validation
src/components/features/settings/llm-profiles/rename-profile-modal.tsx - Modal for renaming profiles
src/components/features/settings/llm-profiles/delete-profile-modal.tsx - Confirmation modal for deleting profiles

Hooks:

src/hooks/query/use-llm-profiles.ts - Query hook for listing profiles
src/hooks/mutation/use-save-llm-profile.ts - Mutation hook for saving profiles
src/hooks/mutation/use-delete-llm-profile.ts - Mutation hook for deleting profiles
src/hooks/mutation/use-rename-llm-profile.ts - Mutation hook for renaming profiles
src/hooks/mutation/use-activate-llm-profile.ts - Mutation hook for activating profiles

Utilities:

src/utils/derive-profile-name.ts - Utility to derive profile name from model string

Modified Files

src/routes/llm-settings.tsx - Complete rewrite with cloud/local mode detection:
- Cloud mode: Renders LlmSettingsCloudView (original simple form with model selector + API key)
- Local mode: Renders LlmSettingsLocalView (profile management UI with list and edit views)
- Main LlmSettingsScreen switches between views based on useActiveBackend().backend.kind
src/hooks/query/query-keys.ts - Added LLM_PROFILES_QUERY_KEY
Various i18n translation keys added

Cloud vs Local Mode

The LlmSettingsScreen component detects the active backend mode and renders the appropriate view:

export function LlmSettingsScreen(props: LlmSettingsScreenProps) {
  const { backend } = useActiveBackend();
  const isCloud = backend.kind === "cloud";

  if (isCloud) {
    return <LlmSettingsCloudView {...props} />;
  }

  return <LlmSettingsLocalView {...props} />;
}

Cloud Mode (LlmSettingsCloudView):

Renders the original simple settings form
Model selector with provider/model dropdown
API key input with status indicator
Base URL input (in advanced view)
No profile management features

Local Mode (LlmSettingsLocalView):

Full profile management UI
Profile list view with add/edit/activate/delete
Profile name input for naming configurations
All profile CRUD operations supported

How It Works

Viewing Profiles (Local Only): The profiles list shows all saved LLM configurations with their model info, API key status, and which one is currently active.
Adding a Profile (Local Only): Click "Add LLM Profile" → Enter a profile name → Configure your LLM settings (model, API key, base URL) → Save → The profile is created and available for activation.
Editing a Profile (Local Only): Click the ⋮ menu on any profile → Click "Edit" → Modify the settings → Save → The profile is updated (with API key preserved if not changed).
Activating a Profile (Local Only): Click the ⋮ menu on any profile → Click "Activate" → The profile's LLM settings are applied to the current agent settings. New conversations will use this profile's configuration.
Renaming/Deleting Profiles (Local Only): Use the context menu on any profile row.

Bug Fixes

Profile Editing Issues

Rename creates duplicate instead of renaming: Fixed by calling renameProfile API before saving config when the name changes
Editing without changing API key clears it: Fixed by fetching existing encrypted API key and preserving it in the save request
Active profile not used for new conversations: Fixed by invalidating SettingsService internal cache when activating a profile, ensuring getSettingsForConversation() fetches fresh settings
Changing profile name wipes form values: Fixed by using a stable React.useMemo reference for the empty initial values object, preventing unnecessary form resets on re-render
State reset on error causes data loss: Fixed by moving state reset inside try block to prevent data loss when profile save fails

Accessibility Improvements

Added aria-expanded and aria-haspopup attributes to profile menu trigger
Added keyboard navigation (arrow keys) with focus management for profile actions menu
Added isRequired prop to ProfileNameInput with proper HTML validation

Cache Invalidation

Added SettingsService.invalidateCache() call in delete profile hook for fresh settings after profile deletion

Backend Compatibility

The implementation uses the SDK's /api/profiles endpoints:

GET /api/profiles - List profiles with active_profile indicator
GET /api/profiles/{name} - Get profile details (supports X-Expose-Secrets: encrypted)
POST /api/profiles/{name} - Save/update profile
DELETE /api/profiles/{name} - Delete profile
POST /api/profiles/{name}/rename - Rename profile
POST /api/profiles/{name}/activate - Activate profile (applies LLM config to settings)

Test Coverage

__tests__/routes/llm-settings.test.tsx - UI integration tests including:
- Form value preservation tests
- Cloud vs Local mode tests - Verifies correct view is rendered based on backend kind
__tests__/hooks/mutation/use-activate-llm-profile.test.tsx - Cache invalidation tests
__tests__/api/profiles-service.test.ts - API service tests
__tests__/api/settings-service.test.ts - Settings cache invalidation tests
__tests__/components/settings/llm-profiles/rename-profile-modal.test.tsx - Error handling test for rename modal

This PR was created by an AI agent (OpenHands) on behalf of the user.

- Add LlmProfilesManager component below LLM settings form - Add profiles API service for CRUD operations (/api/profiles) - Add query hook (useLlmProfiles) and mutation hooks (save, delete, rename) - Add ProfileNameInput with placeholder and isOptional props - Add 'Add Profile' flow with optional profile name input - Add 'Edit Profile' flow from profile context menu - Auto-save profiles when saving LLM settings with profile name input shown - Derive profile name from model if not provided - Add SETTINGS$PROFILE_LOADED i18n key with all 15 language translations The implementation uses the SDK's /api/profiles endpoints and does not include activate functionality as the SDK doesn't support it.

vercel · 2026-05-09T03:42:44Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
agent-canvas	Ready	Preview, Comment	May 12, 2026 6:44pm

Add tests for: - derive-profile-name utility (16 tests) - ProfilesService API (8 tests) - useLlmProfiles query hook (6 tests) - useSaveLlmProfile mutation hook (5 tests) - useDeleteLlmProfile mutation hook (4 tests) - useRenameLlmProfile mutation hook (5 tests) - ProfileNameInput component (14 tests) - ProfileRow component (11 tests) - ProfilesBody component (8 tests) - ProfileActionsMenu component (10 tests) - RenameProfileModal component (10 tests) - DeleteProfileModal component (8 tests) - LlmProfilesManager component (13 tests) Total: 118 new tests covering the full LLM profiles feature.

- Add visual wrapper (border + background) around profile name input - Add hint text explaining to configure settings and save - Add scroll behavior to show profile name input when Add Profile is clicked - Add SETTINGS$PROFILE_SAVE_HINT translation key for all 15 languages

Key changes: - Default view now shows profiles list instead of settings form - Settings form only appears when clicking 'Add LLM Profile' button - Add 'Active' badge to indicate which profile matches current settings - Add 'Activate' action in profile menu to switch to a profile - New components: LlmProfilesListView, ProfileListRow, ProfileListActionsMenu - Updated tests to reflect new profiles-first behavior - Added translations for: AVAILABLE_PROFILES, EDIT_LLM_PROFILE, PROFILE_ACTIVATE, PROFILE_EDIT, PROFILE_ACTIVATED, PROFILE_ACTIVE, PROFILE_RENAME, PROFILE_DELETE UI design matches the reference screenshots from openhands-ui.

Key changes: - Add CurrentSettingsRow component to display existing settings as active - Current settings row shows model, API key badge, and Active badge - Current settings appear first in the list when configured - Added onEditCurrentSettings handler to allow editing current config - Pass hasApiKey prop to show API key status - Fixed button styling (removed ml-auto for better layout) - Added CURRENT_CONFIGURATION translation key This ensures users always see their existing LLM configuration even if they haven't saved any profiles yet.

This update assumes SDK support for active_profile tracking (see OpenHands/software-agent-sdk#3172). Key changes: - ProfileListResponse now includes active_profile field - Added activateProfile endpoint and useActivateLlmProfile hook - LlmProfilesListView uses active_profile from API for active state - Removed CurrentSettingsRow workaround (no longer needed) - Simplified LlmProfilesListView props (data comes from useLlmProfiles) - Updated all tests to include active_profile in mocked responses The UI now properly tracks which profile is active via the backend, matching the behavior of the OpenHands main app.

- Add initialValuesOverride prop to SdkSectionPage - When adding a new profile, pass {} to start with blank fields - When editing an existing profile, load values from settings (default) - Fix skeleton display logic to allow empty initialValuesOverride Tests: - Add test for empty profile name when adding new profile - Add test for populated profile name when editing existing profile - Add test for cancel button returning to profiles list

…erve API key on edit - Track original profile name when editing to detect renames - When editing and changing the profile name, call renameProfile API first before saving the updated config instead of creating a new profile - When editing without providing a new API key, fetch the existing profile's encrypted API key and include it in the save request to preserve it - Add ProfileExposeSecretsMode type and exposeSecrets parameter to getProfile for fetching encrypted secrets that can be round-tripped Co-authored-by: openhands <openhands@all-hands.dev>

When activating an LLM profile, the SettingsService internal cache (settingsCache.encrypted) was not being invalidated. This caused getSettingsForConversation() to return stale cached settings instead of fetching fresh settings with the newly activated profile's LLM config. The fix adds a call to SettingsService.invalidateCache() in the useActivateLlmProfile mutation's onSuccess handler, ensuring that conversations started after activating a profile use the active profile's settings rather than whatever profile was last cached. Changes: - Call SettingsService.invalidateCache() in useActivateLlmProfile onSuccess - Add test for useActivateLlmProfile cache invalidation - Add test for getSettingsForConversation cache invalidation behavior - Fix profiles-service tests to expect headers object in getProfile calls - Add test coverage for exposeSecrets parameter in getProfile Co-authored-by: openhands <openhands@all-hands.dev>

When adding a new LLM profile, changing the profile name field would wipe out all other form values (model, API key, base URL, etc). Root cause: The `initialValuesOverride` prop was created as an inline empty object `{}` inside the render function. Every time the profile name changed, the component re-rendered, creating a new object reference. This triggered the `initialValues` useMemo to recompute with a different reference, which triggered the effect that resets all form values. Fix: Use `React.useMemo` to create a stable empty object reference (`emptyInitialValues`) that persists across re-renders. Tests added: - 'preserves form values when changing profile name' - 'preserves form values when changing profile name after selecting model' Co-authored-by: openhands <openhands@all-hands.dev>

Rework the LLM profiles implementation to use the ProfilesClient and types from the OpenHands TypeScript SDK (PR #148) instead of the local axios-based ProfilesService. Changes: - Update @openhands/typescript-client to PR #148 commit (3c78f17) - Add createProfilesClient factory to typescript-client.ts adapter - Re-export SDK profile types (ProfileInfo, ProfileListResponse, etc.) - Rewrite ProfilesService as thin wrapper around SDK's ProfilesClient - Update SaveProfileRequest usage to match SDK LLM type (required model) - Update ActivateProfileResponse to use SDK's llm_applied field - Fix tests to use SDK-compatible types

Replace LlmProfileSummary with SDK's ProfileInfo type directly. No need for the alias since all usages are internal.

- Use SETTINGS_QUERY_KEYS in use-activate-llm-profile - Apply prettier formatting to profile components

- Fix rename-then-save retry bug: Update originalProfileName immediately after successful rename to prevent retry from using non-existent old name - Add meta.disableToast to all mutation hooks to prevent double error toasts: - useSaveLlmProfile - useDeleteLlmProfile - useRenameLlmProfile - useActivateLlmProfile - Use CONFIG_CACHE_OPTIONS in useLlmProfiles for consistent cache settings - Fix misleading test title Co-authored-by: openhands <openhands@all-hands.dev>

malhotra5 · 2026-05-12T17:39:16Z

Fixed critical issues in cf18d4b:

Critical fixes:

Fixed rename-then-save retry bug: originalProfileName is now updated immediately after successful rename, preventing retry from using the old (non-existent) name
Added meta: { disableToast: true } to all mutation hooks to prevent double error toasts

Other fixes:

Using CONFIG_CACHE_OPTIONS in useLlmProfiles for consistent cache settings
Fixed misleading test title

Ready for another look.

all-hands-bot

This PR adds solid LLM profile management, but several critical issues from previous reviews remain unresolved. The cache pollution and race condition bugs are production-breaking, and test coverage for the primary save flows is still missing.

- Add backend identity (backend.id, orgId) to LLM profiles query key to prevent cache pollution when switching between backends - Add race condition fix in handleEditProfile: track latest request and ignore stale responses from concurrent profile edits - Import useActiveBackend in useLlmProfiles hook Co-authored-by: openhands <openhands@all-hands.dev>

malhotra5 · 2026-05-12T17:53:07Z

Addressed additional critical issues in e2e4e36:

Critical fixes:

Cache pollution fix: Added backend identity (backend.id, orgId) to LLM profiles query key to prevent stale data when switching backends
Race condition fix: Track latest edit request in handleEditProfile and ignore stale responses from concurrent profile edits

Ready for another look.

all-hands-bot

This PR adds solid LLM profile management functionality, but several critical issues from previous reviews remain unresolved, and new concurrency bugs have been identified.

Key Blocking Issues:

🔴 Race condition with concurrent save operations (NEW)
🔴 Zero test coverage for primary save flows (unresolved)
🔴 API key status shows global setting instead of profile-specific (unresolved)
🔴 Race condition with concurrent profile edits (unresolved from previous reviews)
🟠 Multiple accessibility issues in modals and menus (NEW)

Important Note: This PR changes agent behavior (LLM settings loading and application). Per repository guidelines, this requires human review after lightweight evaluation runs to ensure benchmark performance is not affected.

See inline comments for details on new issues discovered.

- Fix DeleteProfileModal to prevent close during pending deletion via Escape/backdrop - Fix ProfileActionsMenu click-outside handler to check for trigger button clicks - Consolidate duplicate invalidation tests into comprehensive single tests - Add query key invalidation assertions to activate profile test Co-authored-by: openhands <openhands@all-hands.dev>

Co-authored-by: openhands <openhands@all-hands.dev>

- Add profile name to aria-label in profile-row.tsx for screen reader context - Add Enter key handler for rename modal form submission - Add onKeyDown prop support to SettingsInput and ProfileNameInput components - Add tests for Enter key submission behavior Co-authored-by: openhands <openhands@all-hands.dev>

malhotra5 · 2026-05-12T18:12:49Z

Addressed review feedback in 9fa7336:

Accessibility: Added profile name to aria-label for screen reader context
UX: Added Enter key handler for rename modal form submission
Added tests for Enter key behavior

Test coverage comments (E2E tests, MSW-based integration tests) are acknowledged and better suited for a dedicated follow-up PR.

Ready for another look.

Co-authored-by: openhands <openhands@all-hands.dev>

all-hands-bot

Solid implementation with extensive test coverage. No blocking issues, but flagging for human review due to eval/benchmark risk (changes LLM settings loading). See inline comments for minor improvements.

- Add Tab key handling to close menu per WAI-ARIA practices - Make validation explicit with empty check in rename modal - Return ProfileMutationResponse from mutation hooks for consistency Co-authored-by: openhands <openhands@all-hands.dev>

malhotra5 · 2026-05-12T18:45:19Z

Addressed additional review suggestions in c494052:

Accessibility: Tab key now closes menu per WAI-ARIA practices
Clarity: Made validation explicit with empty check in rename modal
Consistency: Mutation hooks now return ProfileMutationResponse

Other suggestions (client.close error handling, null-model mode consistency) acknowledged with rationale for deferring.

All review threads resolved. Ready for final review.

malhotra5 · 2026-05-12T20:03:45Z

Breaking this PR into smaller ones, with new fixes as well

vercel Bot deployed to Preview May 9, 2026 03:42 View deployment

vercel Bot deployed to Preview May 9, 2026 03:54 View deployment

vercel Bot deployed to Preview May 9, 2026 04:04 View deployment

vercel Bot deployed to Preview May 9, 2026 04:23 View deployment

vercel Bot deployed to Preview May 9, 2026 04:30 View deployment

vercel Bot deployed to Preview May 9, 2026 05:04 View deployment

vercel Bot deployed to Preview May 9, 2026 05:31 View deployment

fix profile creation

97f93a8

vercel Bot deployed to Preview May 9, 2026 06:11 View deployment

vercel Bot deployed to Preview May 9, 2026 06:25 View deployment

vercel Bot deployed to Preview May 9, 2026 06:41 View deployment

vercel Bot deployed to Preview May 9, 2026 06:50 View deployment

vercel Bot had a problem deploying to Preview May 9, 2026 14:20 Failure

refactor: remove LlmProfileSummary backward compat alias

9753406

Replace LlmProfileSummary with SDK's ProfileInfo type directly. No need for the alias since all usages are internal.

vercel Bot had a problem deploying to Preview May 9, 2026 14:34 Failure

fix: lint issues in profile hooks and components

3b44868

- Use SETTINGS_QUERY_KEYS in use-activate-llm-profile - Apply prettier formatting to profile components

vercel Bot had a problem deploying to Preview May 9, 2026 14:42 Failure

malhotra5 marked this pull request as ready for review May 9, 2026 14:45

malhotra5 requested a review from all-hands-bot May 12, 2026 17:39

vercel Bot deployed to Preview May 12, 2026 17:40 View deployment

all-hands-bot reviewed May 12, 2026

View reviewed changes

malhotra5 requested a review from all-hands-bot May 12, 2026 17:53

vercel Bot deployed to Preview May 12, 2026 17:53 View deployment

all-hands-bot reviewed May 12, 2026

View reviewed changes

vercel Bot deployed to Preview May 12, 2026 18:03 View deployment

fix: prevent RenameProfileModal close during pending rename

a41497e

Co-authored-by: openhands <openhands@all-hands.dev>

vercel Bot deployed to Preview May 12, 2026 18:04 View deployment

vercel Bot deployed to Preview May 12, 2026 18:12 View deployment

malhotra5 requested a review from all-hands-bot May 12, 2026 18:12

fix: update profile-row test to expect profile name in aria-label

0ad2530

Co-authored-by: openhands <openhands@all-hands.dev>

vercel Bot deployed to Preview May 12, 2026 18:20 View deployment

all-hands-bot reviewed May 12, 2026

View reviewed changes

fix: address additional review suggestions

c494052

- Add Tab key handling to close menu per WAI-ARIA practices - Make validation explicit with empty check in rename modal - Return ProfileMutationResponse from mutation hooks for consistency Co-authored-by: openhands <openhands@all-hands.dev>

vercel Bot deployed to Preview May 12, 2026 18:44 View deployment

malhotra5 requested a review from all-hands-bot May 12, 2026 18:45

malhotra5 mentioned this pull request May 12, 2026

feat: add LLM profiles API layer and React Query hooks #387

Merged

malhotra5 closed this May 12, 2026

This was referenced May 12, 2026

feat: LLM profiles UI components, modals, and tests (PR B) #389

Merged

feat: LLM profiles route integration (PR C) #393

Open

Conversation

malhotra5 commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

New Files

Modified Files

Cloud vs Local Mode

How It Works

Bug Fixes

Profile Editing Issues

Accessibility Improvements

Cache Invalidation

Backend Compatibility

Test Coverage

Uh oh!

vercel Bot commented May 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

malhotra5 commented May 12, 2026

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

malhotra5 commented May 12, 2026

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

malhotra5 commented May 12, 2026

Uh oh!

all-hands-bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

malhotra5 commented May 12, 2026

Uh oh!

malhotra5 commented May 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

malhotra5 commented May 9, 2026 •

edited

Loading

vercel Bot commented May 9, 2026 •

edited

Loading