UnitPylot

UnitPylot is a Copilot-enabled Visual Studio Code (VS Code) extension designed to enhance the unit testing experience for Python developers wokring for Brownfield Codebases. Developers can use our extension on an existing codebase to get immediate insights into their testing suite.

It offers a range of features to streamline unit testing, enhance code coverage, and optimise test performance. The interactive dashboard visualises key test metrics and intelligent suggestions are provided to improve test quality.

Table of Contents

1. Features 🌟
2. Installation 📥
3. Usage Instructions 📖
4. Settings Configuration ⚙️
5. List of Commands 🗂
6. Manuals (User and Deployment/Installation) 📁

Features 🌟

📊 Test Performance & Coverage Insights

• Granular Test Metrics Breakdown: displays the structure of the project's test suite within a tree view highlighting,
  • passing/failing test cases,
  • n slowest tests,
  • n most memory intensive tests.
• Line and Branch Coverage Display: provides functionality that highlights untested areas of code within the editor, providing real-time feedback.
• Test History Tracker: tracks test performance with interactive graphs of pass/fail rates and coverage trends.
• Exportable Logs: saves test results and coverage trends into json or markdown formats.

🔁 Automated Test Optimisation & Debugging

UnitPylot offers AI assitance that provides suggestions to improve the following metrics, allowing them to be accepted directly into corresponding files:

• Fix Failing Tests: detects failure points and suggests fixes to improve test reliability.
• Improve Coverage: detects untested code such as edge cases or missed branches and suggests additional test cases.
• Optimise Slowest Tests: detects the n slowest tests and suggests explanations and improved test cases with faster execution time.
• Optimise Memory-intensive Tests: detects the n most memory intensive tests and suggests tests which use lesser memory.

🤖 AI-Powered Enhancements

• Code Insights: highlights vulnerabilities and suggests improvements in test cases to detect bottlenecks and prevent regressions.
• Pydoc Generation: generates documentation for test cases to enhance readability and maintainability.
• AAA Chat Participant: provides guidance on how to follow the best testing practices by adhering to the AAA design pattern.

⚙️ Smart Execution and Customisation

• Customise n: allows user to chose the number of slowest and memory intensive tests to display dynamically.
• Continuous Background Testing: runs necessary tests automatically when changes are detected.
• Refreshing Suite History: allows user to customise whether to periodically save snapshots or track changes based on file changes.
• Selective Test Execution: allows running only relevant tests based on recent changes to shorten feedback loops.

Installation 📥

Prerequisites

• Access to Copilot: To use any GitHub Copilot extension in Visual Studio Code, you need either an active Copilot subscription (such as Copilot Pro, Copilot Enterprise, or Copilot Business).
• Visual Studio Code should be installed.
• Copilot in Visual Studio Code: Follow this if not yet set-up - http://code.visualstudio.com/docs/copilot/setup

Overview

1. Make sure you have the GitHub Copilot activated in VS Code Follow the link above if you need assisstance.
2. Add our extension by searching for "UnitPylot" in the VS Code Extension Marketplace or from Source.
3. Download an example codebase as below to try out our extension!

🔬 For Testing: Clone the example-codebases Repository

git clone https://github.com/ucl-syseng-tools-for-vscode/example-codebases.git

From the Marketplace

1. Open VS Code.
2. Search for UnitPylot in the VS Code Marketplace.
3. Click Install to add the extension.

🛠 Build From Source

To build UnitPylot from source, follow these steps:

1. Clone the repository:

git clone https://github.com/ucl-syseng-tools-for-vscode/UnitPylot.git
cd UnitPylot

2. Install the dependencies:

npm install

3. Compile the extension:

npm run compile

4. Open the project in Visual Studio Code:

code .

🚀 Launch the Extension

To begin using UnitPylot, follow these steps:

1. Press F5 OR open the Command Palette (Shift + Command + P) and run Debug: Start Debugging.
2. Open one of the projects within the example-codebases folder.
3. Run the make.sh file to create a virtual environment (venv) to run the project within OR ensure that you have the necessary dependencies installed by running: pip install pytest pytest-cov pytest-json-report pytest-monitor.

Usage Instructions 📖

🖥️ Dashboard View

• Locate and open the UnitPylot icon on the left-side VSCode navigation bar.
• Access the granular test suite view from the dashboard view under the Tests Overview collapsable view.
• Access the test history graphs to analyse the pass/fail rates and coverage trends over time from the dashboard view under the Graphs & Docs collapsable view.

⏯️ Running Tests

• Open a Python file with tests.
• Use the CodeLens links above each test function to run or debug particular tests.
• OR click the run tests / run all tests button within the dashboard view.

📈 Viewing Test Coverage

• Enable code coverage highlighting in the settings.
• Run your tests to see the coverage data directly in the editor.

✅ Commands for Optimising Tests

• Locate the Code Insights button on the top right next to the run button to generate code insights.
• Right click and navigate to the UnitPylot Commands to find:
  • the fix coverage command when in a src file.
  • the fix failing, optimise slowest, optimise memory, and generate pydoc commands when the current editor is in a test file.
    
    

Settings Configuration ⚙️

Navigate to the UnitPylot settings page by clicking the ⚙️ icon on the top right of the expandable dashboard view. UnitPylot provides several configuration options to customise its behavior. These can be accessed through the VSCode settings:

• unit-pylot.runNecessaryTestsOnly: Run only necessary tests instead of all tests.
• unit-pylot.copilotEnabled: Enable GitHub Copilot integration.
• unit-pylot.codeCoverageHighlighting: Enable code coverage highlighting.
• unit-pylot.numberOfSlowestTests: Number of slowest tests to display.
• unit-pylot.snapshotInterval: Snapshot interval in minutes.
• unit-pylot.runTestsOnSave: Run tests on any file save.
• unit-pylot.runTestsInBackground: Periodically run tests in the background.
• unit-pylot.saveSnapshotOnTestRun: Save snapshots when tests are run.
• unit-pylot.saveSnapshotPeriodically: Periodically save snapshots in the background.
• unit-pylot.customLLM Endpoint: Custom LLM endpoint to use if GitHub Copilot is not enabled.
• unit-pylot.customLLM Model: Custom LLM model to use if GitHub Copilot is not enabled.
• unit-pylot.customLLM APIKey: Custom LLM key to use if GitHub Copilot is not enabled.
• unit-pylot.customLLM MaxTokens: Custom LLM max tokens to use if GitHub Copilot is not enabled.

🔄 Custom, Third-Party LLM Support

This extension should be used with GitHub Copilot, as this provides the best user experience for the developer. However there is the option to use a Third-party or Local LLM if the user wishes.

UnitPylot supports custom language models (LLMs) through an OpenAI-style API. This allows you to integrate your own LLMs for test writing and optimisation. To configure a custom LLM, set the following options in the VSCode settings:

• unit-pylot.customLLM Endpoint: The endpoint URL for your custom LLM API (e.g., http://xxxxxx/v1/chat/completions).
• unit-pylot.customLLM Model: The model name to use with your custom LLM.
• unit-pylot.customLLM APIKey: The API key for authenticating with your custom LLM.
• unit-pylot.customLLM MaxTokens: The maximum number of tokens to use for each request to your custom LLM.

List of Commands 🗂

Below are all the UnitPylot commands also accessible through the command palette:

• UnitPylot: Download Snapshot Report
• UnitPylot: Fix Failing Tests
• UnitPylot: Fix Coverage
• UnitPylot: Focus on Dashboard View
• UnitPylot: Focus on Graphs & Docs View
• UnitPylot: Focus on Tests Overview View
• UnitPylot: Generate Pydoc
• UnitPylot: Optimise Memory Usage of Tests
• UnitPylot: Optimise Slowest Tests
• UnitPylot: Refresh Failing Tests
• UnitPylot: Run All Tests
• UnitPylot: Run Tests
• UnitPylot: Show test pass-fail history graph
• UnitPylot: Show coverage history graph
• UnitPylot: Test Insights

Manuals (User and Deployment/Installation) 📁

Please find our User Manual here: https://drive.google.com/file/d/1vbBsLeV57opwqKB8SWiCx3SLCIn17_t1/view?usp=drive_link

Please find our Deployment/Installation Manual here: https://drive.google.com/file/d/1LNctYDGiqrr84wNwlQs91dk-aHrXxTh_/view?usp=drive_link