Skip to content

[Connector] GitLab Group Connector: Mass Indexing and Integration Support #37

@Anthony-Bible

Description

@Anthony-Bible

Business Requirements

  • Allow administrators to connect the codechunking platform to GitLab, supporting both gitlab.com and self-hosted GitLab instances.
  • Users must be able to authenticate securely (preferably via OAuth 2.0) to grant access to one or more GitLab groups.
  • Administrators can select groups and optionally include subgroups/descendants for recursive indexing.
  • Support repository filtering by name pattern, activity date, and visibility (public/private/internal).
  • API endpoint(s) must allow users to trigger mass indexing jobs for all or filtered subsets of repositories within a group.
  • Connector setup should support storing and revoking OAuth tokens securely, and refreshing tokens as needed.
  • Maintain an audit trail of connections, indexing jobs, and repository discovery outcomes for traceability.
  • Provide clear API responses and error handling for all connector operations (connection, listing, indexing, revocation, etc.).

Technical Requirements

  • Implement OAuth 2.0 authentication flow for GitLab as a first-class integration option. Store tokens securely (encrypted at rest, never log sensitive data).
  • Design connector configuration to support both gitlab.com and arbitrary self-hosted GitLab URLs. Allow API base URL to be configurable.
  • Fetch group/project listing via GitLab's REST API, supporting pagination and deep-nested subgroup traversal.
  • Implement repository filtering logic server-side—accept filter params (name, visibility, last activity, etc.) and apply to the discovery process.
  • Implement connector lifecycle endpoints: connect (initiate OAuth), list (view connected groups), index (trigger mass indexing), disconnect (revoke).
  • Store metadata on discovered repositories (group IDs, project IDs, last indexed timestamp, etc.) for tracking and deduplication.
  • Robust error handling: propagate GitLab and system errors to users with actionable messages.
  • Provide support for automated re-indexing on a schedule or via webhook/event (future work: outline event-driven discovery triggers).
  • Write unit/integration tests covering auth, group discovery, filtering, and mass indexing invocation.

This sub-issue tracks the implementation of a GitLab connector as part of #35. See parent for overall context and next steps.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions