Skip to content

v0.9.0

Choose a tag to compare

View all tags
@nina-msft nina-msft released this 09 May 01:42
· 555 commits to main since this release
v0.9.0
f942562

What's Changed

Targets

  • HTTPTarget Improvements that properly parse the HTTP version, automatically calculate the content-length, and make headers case insensitive.
  • FIX: Fixed IndexError with RealtimeTarget to handle responses properly

Datasets

  • Social Engineering (Persuasion and Deception) Scenarios: See datasets/orchestrators/red_teaming/persuasion_deception and datasets/orchestrators/role_play/persuasion_script.yaml
  • Multilingual Vulnerability dataset from "A Framework to Assess Multilingual Vulnerabilities of LLMs"

Converters

  • Enhancements to the AsciiSmugglerConverter by adding support for two methods for encoding hidden data (embedding directly in a Unicode character (default: 😊) and appending hidden data to visible text).
  • ZalgoConverter: Adds Unicode characters to text to make it appear "glitchy"
  • ToxicSentenceGeneratorConverter: Generate toxic sentence starters based on seed prompts
  • FIX: Remove JSON Instructions for TranslationConverter to address intermittent failures due to JSON parsing issues and non-consistent responses from endpoints.

Orchestrators

  • [BREAKING] Rename MultiTurnAttackResult to OrchestratorResult as part of a bigger refactor to tack objectives and results.
  • FIX: Keep Conversation ID in PromptSendingOrchestrator if it is provided
  • FIX: Remove Harm-Specific Prevention from CrescendoOrchestrator

Scorers

  • Generic Scorer with Flexible Inputs: SelfAskGeneralScorer in pyrit/score/general_scorer.py. It can be configured to use different scoring types (e.g. True/False, float) and can format the prompt using a system prompt and a format string.
  • Criteria-Based Scorer (used with SelfAskScaleScorer): Provides evaluation criteria that is specific to a given objective.
  • CompositeScorer: Combines multiple True/False Results into a single True/False Result

Dependencies

  • Moves jupyter and ipykernel from required into an optional [dev] dependency. If you need to use Jupyter notebooks with PyRIT, you'll need to install using methods outlined here.
  • Moves azure-cognitiveservices-speech from required into an optional [speech] dependency.

Other

  • Added custom file name support to allows for saving data (image, audio, video, etc.) to storage under a custom name.
  • Custom Retry Decorator: pyrit_custom_result_retry to retry a function if a certain condition is true. This augments existing retry decorators which retry functions based on exception criteria.
  • Optimizations and various bug fixes to .devcontainer

Full list of changes

New Contributors

Full Changelog: Azure/PyRIT@v0.8.1...v0.9.0

Contributors

afogel, bashirpartovi, and 15 other contributors
Assets 2
Loading