diff --git a/.blackboxrules b/.blackboxrules index 6e236d0..6125caa 100644 --- a/.blackboxrules +++ b/.blackboxrules @@ -1,4 +1,6 @@ +# Pipelex Coding Rules + ## Guide to write or edit pipelines using the Pipelex language in .plx files - Always first write your "plan" in natural language, then transcribe it in pipelex. @@ -39,10 +41,10 @@ ConceptName = "Description of the concept" - Use PascalCase for concept names - Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex - Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents -- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page) +- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON) **Native Concepts:** -Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate. +Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate. **Refining Native Concepts:** To create a concept that specializes a native concept without adding fields: @@ -70,7 +72,7 @@ output = "ConceptName" The pipes will all have at least this base definition. - `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition). So If you have this error: -`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • +`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • variable='['invoice']'`` That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it. @@ -857,21 +859,6 @@ But don't write documentation unless asked explicitly to. ## Guide to execute a pipeline and write example code -### Prerequisites: Virtual Environment - -**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work. - -For standard installations, the virtual environment is named `.venv`. Always check this first: - -```bash -## Activate the virtual environment (standard installation) -source .venv/bin/activate # On macOS/Linux -## or -.venv\Scripts\activate # On Windows -``` - -If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active. - ### Example to execute a pipeline with text output ```python @@ -1146,7 +1133,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp Examples: ```toml llm_to_engineer = { model = "base-claude", temperature = 1 } -llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" } +llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" } ``` The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this: diff --git a/.cursor/rules/llms.mdc b/.cursor/rules/llms.mdc index 7622b33..2035388 100644 --- a/.cursor/rules/llms.mdc +++ b/.cursor/rules/llms.mdc @@ -56,7 +56,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp Examples: ```toml llm_to_engineer = { model = "base-claude", temperature = 1 } -llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" } +llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" } ``` The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this: diff --git a/.cursor/rules/run_pipelex.mdc b/.cursor/rules/run_pipelex.mdc index 78aed74..8387838 100644 --- a/.cursor/rules/run_pipelex.mdc +++ b/.cursor/rules/run_pipelex.mdc @@ -6,21 +6,6 @@ globs: --- # Guide to execute a pipeline and write example code -## Prerequisites: Virtual Environment - -**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work. - -For standard installations, the virtual environment is named `.venv`. Always check this first: - -```bash -# Activate the virtual environment (standard installation) -source .venv/bin/activate # On macOS/Linux -# or -.venv\Scripts\activate # On Windows -``` - -If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active. - ## Example to execute a pipeline with text output ```python diff --git a/.cursor/rules/write_pipelex.mdc b/.cursor/rules/write_pipelex.mdc index b46e4f2..9f23faf 100644 --- a/.cursor/rules/write_pipelex.mdc +++ b/.cursor/rules/write_pipelex.mdc @@ -45,10 +45,10 @@ ConceptName = "Description of the concept" - Use PascalCase for concept names - Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex - Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents -- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page) +- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON) **Native Concepts:** -Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate. +Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate. **Refining Native Concepts:** To create a concept that specializes a native concept without adding fields: @@ -76,7 +76,7 @@ output = "ConceptName" The pipes will all have at least this base definition. - `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition). So If you have this error: -`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • +`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • variable='['invoice']'`` That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it. diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 6e236d0..6125caa 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,4 +1,6 @@ +# Pipelex Coding Rules + ## Guide to write or edit pipelines using the Pipelex language in .plx files - Always first write your "plan" in natural language, then transcribe it in pipelex. @@ -39,10 +41,10 @@ ConceptName = "Description of the concept" - Use PascalCase for concept names - Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex - Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents -- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page) +- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON) **Native Concepts:** -Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate. +Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate. **Refining Native Concepts:** To create a concept that specializes a native concept without adding fields: @@ -70,7 +72,7 @@ output = "ConceptName" The pipes will all have at least this base definition. - `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition). So If you have this error: -`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • +`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • variable='['invoice']'`` That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it. @@ -857,21 +859,6 @@ But don't write documentation unless asked explicitly to. ## Guide to execute a pipeline and write example code -### Prerequisites: Virtual Environment - -**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work. - -For standard installations, the virtual environment is named `.venv`. Always check this first: - -```bash -## Activate the virtual environment (standard installation) -source .venv/bin/activate # On macOS/Linux -## or -.venv\Scripts\activate # On Windows -``` - -If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active. - ### Example to execute a pipeline with text output ```python @@ -1146,7 +1133,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp Examples: ```toml llm_to_engineer = { model = "base-claude", temperature = 1 } -llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" } +llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" } ``` The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this: diff --git a/.github/workflows/github-release.yml b/.github/workflows/github-release.yml new file mode 100644 index 0000000..b91d756 --- /dev/null +++ b/.github/workflows/github-release.yml @@ -0,0 +1,78 @@ +name: Create release + +on: + push: + branches: + - main + +jobs: + github-release: + name: Create GitHub Release + runs-on: ubuntu-latest + permissions: + contents: write # IMPORTANT: mandatory for making GitHub Releases + + steps: + - uses: actions/checkout@v4 + with: + persist-credentials: false + - name: Extract version from pyproject.toml + id: get_version + run: | + VERSION=$(grep -m 1 'version = ' pyproject.toml | cut -d '"' -f 2) + echo "VERSION=$VERSION" >> $GITHUB_ENV + - name: Extract changelog notes for current version + id: get_changelog + run: | + VERSION=${{ env.VERSION }} + echo "Extracting changelog for version v$VERSION" + + # Find the start of the current version section + START_LINE=$(grep -n "## \[v$VERSION\] - " CHANGELOG.md | cut -d: -f1) + + if [ -z "$START_LINE" ]; then + echo "Warning: No changelog entry found for version v$VERSION" + echo "CHANGELOG_NOTES=" >> $GITHUB_ENV + exit 0 + fi + + # Find the start of the next version section (previous version) + NEXT_VERSION_LINE=$(tail -n +$((START_LINE + 1)) CHANGELOG.md | grep -n "^## \[v.*\] - " | head -1 | cut -d: -f1) + + if [ -z "$NEXT_VERSION_LINE" ]; then + # No next version found, extract from current version till end of file + CHANGELOG_CONTENT=$(tail -n +$START_LINE CHANGELOG.md) + else + # Extract content from current version header to before next version + END_LINE=$((START_LINE + NEXT_VERSION_LINE - 1)) + CHANGELOG_CONTENT=$(sed -n "$START_LINE,$((END_LINE - 1))p" CHANGELOG.md) + fi + + # Clean up the content but preserve the blank line after the header + # First, get the header line and add a blank line after it + HEADER_LINE=$(echo "$CHANGELOG_CONTENT" | head -1) + CONTENT_LINES=$(echo "$CHANGELOG_CONTENT" | tail -n +2 | sed '/^$/d' | sed 's/^[[:space:]]*//' | sed 's/[[:space:]]*$//') + + # Combine header + blank line + content + CHANGELOG_CONTENT=$(printf "%s\n\n%s" "$HEADER_LINE" "$CONTENT_LINES") + + # Escape for GitHub Actions + echo "CHANGELOG_NOTES<> $GITHUB_ENV + echo "$CHANGELOG_CONTENT" >> $GITHUB_ENV + echo "EOF" >> $GITHUB_ENV + - name: Create GitHub Release + env: + GITHUB_TOKEN: ${{ github.token }} + run: | + if [ -n "$CHANGELOG_NOTES" ]; then + gh release create "v$VERSION" \ + --repo "$GITHUB_REPOSITORY" \ + --title "v$VERSION" \ + --notes "$CHANGELOG_NOTES" + else + gh release create "v$VERSION" \ + --repo "$GITHUB_REPOSITORY" \ + --title "v$VERSION" \ + --notes "Release v$VERSION" + fi + diff --git a/.github/workflows/version-check.yml b/.github/workflows/version-check.yml index a4e4d17..ebcccc1 100644 --- a/.github/workflows/version-check.yml +++ b/.github/workflows/version-check.yml @@ -4,7 +4,6 @@ on: pull_request: branches: - main - - "release/v[0-9]+.[0-9]+.[0-9]+" jobs: version-check: runs-on: ubuntu-latest @@ -24,17 +23,6 @@ jobs: echo "base_branch=$BASE_BRANCH" >> $GITHUB_OUTPUT echo "source_branch=$SOURCE_BRANCH" >> $GITHUB_OUTPUT - # Check if target is a release branch - if [[ "$BASE_BRANCH" =~ ^release/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then - echo "Target is a release branch" - echo "is_release_target=true" >> $GITHUB_OUTPUT - echo "target_release_version=${BASE_BRANCH#release/v}" >> $GITHUB_OUTPUT - echo "Extracted target release version: ${BASE_BRANCH#release/v}" - else - echo "Target is NOT a release branch" - echo "is_release_target=false" >> $GITHUB_OUTPUT - fi - # Check if source is a release branch if [[ "$SOURCE_BRANCH" =~ ^release/v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then echo "Source is a release branch" @@ -42,8 +30,10 @@ jobs: echo "source_release_version=${SOURCE_BRANCH#release/v}" >> $GITHUB_OUTPUT echo "Extracted source release version: ${SOURCE_BRANCH#release/v}" else - echo "Source is NOT a release branch" + echo "Source is NOT a release branch - skipping version check" echo "is_release_source=false" >> $GITHUB_OUTPUT + echo "This workflow only runs for PRs from release branches to main" + exit 0 fi echo "=======================================" @@ -61,74 +51,24 @@ jobs: echo "version=$VERSION" >> $GITHUB_OUTPUT echo "=======================================" - # MAIN BRANCH CASE - Check that version is bumped - - name: Check version bump for main branch - if: steps.branch_info.outputs.base_branch == 'main' + # Check that version in pyproject.toml matches the source release branch name + - name: Check version matches release branch run: | - echo "====== CHECKING VERSION BUMP FOR MAIN ======" - # Get version from main branch - git checkout ${{ github.event.pull_request.base.ref }} - MAIN_VERSION=$(grep '^version' pyproject.toml | sed -E 's/version = "(.*)"/\1/') - - # Get version from PR branch - git checkout ${{ github.event.pull_request.head.sha }} - PR_VERSION="${{ steps.current_version.outputs.version }}" - - echo "Main version: $MAIN_VERSION" - echo "PR version: $PR_VERSION" - - # Check that PR version is greater than main version - if dpkg --compare-versions "$PR_VERSION" le "$MAIN_VERSION"; then - echo "❌ ERROR: PR version ($PR_VERSION) is not greater than main version ($MAIN_VERSION)." - echo "Please bump the version." - exit 1 - else - echo "✅ Version was properly bumped from $MAIN_VERSION to $PR_VERSION" - fi - echo "=======================================" - - # TARGET RELEASE BRANCH CASE - Check that version matches release branch name - - name: Check version matches target release branch - if: steps.branch_info.outputs.is_release_target == 'true' - run: | - echo "====== CHECKING VERSION MATCH FOR TARGET RELEASE BRANCH ======" - # Extract version from branch name - RELEASE_VERSION="${{ steps.branch_info.outputs.target_release_version }}" - - # Get version from PR branch - PR_VERSION="${{ steps.current_version.outputs.version }}" - - echo "Target release branch version: $RELEASE_VERSION" - echo "PR version: $PR_VERSION" - - # Check if versions match - if [[ "$PR_VERSION" != "$RELEASE_VERSION" ]]; then - echo "❌ ERROR: Version in pyproject.toml ($PR_VERSION) does not match target release branch version ($RELEASE_VERSION)" - exit 1 - else - echo "✅ Version in pyproject.toml matches target release branch version" - fi - echo "=======================================" - - # SOURCE RELEASE BRANCH CASE - Check that version matches release branch name - - name: Check version matches source release branch - if: steps.branch_info.outputs.is_release_source == 'true' - run: | - echo "====== CHECKING VERSION MATCH FOR SOURCE RELEASE BRANCH ======" + echo "====== CHECKING VERSION MATCH FOR RELEASE BRANCH ======" # Extract version from branch name RELEASE_VERSION="${{ steps.branch_info.outputs.source_release_version }}" # Get version from PR branch PR_VERSION="${{ steps.current_version.outputs.version }}" - echo "Source release branch version: $RELEASE_VERSION" - echo "PR version: $PR_VERSION" + echo "Release branch version: $RELEASE_VERSION" + echo "pyproject.toml version: $PR_VERSION" # Check if versions match if [[ "$PR_VERSION" != "$RELEASE_VERSION" ]]; then - echo "❌ ERROR: Version in pyproject.toml ($PR_VERSION) does not match source release branch version ($RELEASE_VERSION)" + echo "❌ ERROR: Version in pyproject.toml ($PR_VERSION) does not match release branch version ($RELEASE_VERSION)" exit 1 else - echo "✅ Version in pyproject.toml matches source release branch version" + echo "✅ Version in pyproject.toml ($PR_VERSION) matches release branch version ($RELEASE_VERSION)" fi - echo "=======================================" + echo "=======================================" \ No newline at end of file diff --git a/.windsurfrules.md b/.windsurfrules.md index 6e236d0..6125caa 100644 --- a/.windsurfrules.md +++ b/.windsurfrules.md @@ -1,4 +1,6 @@ +# Pipelex Coding Rules + ## Guide to write or edit pipelines using the Pipelex language in .plx files - Always first write your "plan" in natural language, then transcribe it in pipelex. @@ -39,10 +41,10 @@ ConceptName = "Description of the concept" - Use PascalCase for concept names - Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex - Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents -- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page) +- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON) **Native Concepts:** -Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate. +Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate. **Refining Native Concepts:** To create a concept that specializes a native concept without adding fields: @@ -70,7 +72,7 @@ output = "ConceptName" The pipes will all have at least this base definition. - `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition). So If you have this error: -`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • +`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • variable='['invoice']'`` That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it. @@ -857,21 +859,6 @@ But don't write documentation unless asked explicitly to. ## Guide to execute a pipeline and write example code -### Prerequisites: Virtual Environment - -**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work. - -For standard installations, the virtual environment is named `.venv`. Always check this first: - -```bash -## Activate the virtual environment (standard installation) -source .venv/bin/activate # On macOS/Linux -## or -.venv\Scripts\activate # On Windows -``` - -If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active. - ### Example to execute a pipeline with text output ```python @@ -1146,7 +1133,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp Examples: ```toml llm_to_engineer = { model = "base-claude", temperature = 1 } -llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" } +llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" } ``` The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this: diff --git a/AGENTS.md b/AGENTS.md index 6e236d0..6125caa 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,4 +1,6 @@ +# Pipelex Coding Rules + ## Guide to write or edit pipelines using the Pipelex language in .plx files - Always first write your "plan" in natural language, then transcribe it in pipelex. @@ -39,10 +41,10 @@ ConceptName = "Description of the concept" - Use PascalCase for concept names - Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex - Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents -- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page) +- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON) **Native Concepts:** -Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate. +Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate. **Refining Native Concepts:** To create a concept that specializes a native concept without adding fields: @@ -70,7 +72,7 @@ output = "ConceptName" The pipes will all have at least this base definition. - `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition). So If you have this error: -`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • +`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • variable='['invoice']'`` That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it. @@ -857,21 +859,6 @@ But don't write documentation unless asked explicitly to. ## Guide to execute a pipeline and write example code -### Prerequisites: Virtual Environment - -**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work. - -For standard installations, the virtual environment is named `.venv`. Always check this first: - -```bash -## Activate the virtual environment (standard installation) -source .venv/bin/activate # On macOS/Linux -## or -.venv\Scripts\activate # On Windows -``` - -If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active. - ### Example to execute a pipeline with text output ```python @@ -1146,7 +1133,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp Examples: ```toml llm_to_engineer = { model = "base-claude", temperature = 1 } -llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" } +llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" } ``` The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this: diff --git a/CHANGELOG.md b/CHANGELOG.md index d046805..8c4d56e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,9 @@ # Changelog +## [v0.6.5] - 2025-11-26 + +- Bump `pipelex` to `v0.17.1`: See `Pipelex` changelog [here](https://docs.pipelex.com/changelog/) + ## [v0.6.4] - 2025-11-18 - Bump `pipelex` to `v0.15.7` to fix `pipelex doctor` diff --git a/CLAUDE.md b/CLAUDE.md index 6e236d0..6125caa 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,4 +1,6 @@ +# Pipelex Coding Rules + ## Guide to write or edit pipelines using the Pipelex language in .plx files - Always first write your "plan" in natural language, then transcribe it in pipelex. @@ -39,10 +41,10 @@ ConceptName = "Description of the concept" - Use PascalCase for concept names - Never use plurals (no "Stories", use "Story") - lists are handled implicitly by Pipelex - Avoid circumstantial adjectives (no "LargeText", use "Text") - focus on the essence of what the concept represents -- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page) +- Don't redefine native concepts (Text, Image, PDF, TextAndImages, Number, Page, JSON) **Native Concepts:** -Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`. Use these directly or refine them when appropriate. +Pipelex provides built-in native concepts: `Text`, `Image`, `PDF`, `TextAndImages`, `Number`, `Page`, `JSON`. Use these directly or refine them when appropriate. **Refining Native Concepts:** To create a concept that specializes a native concept without adding fields: @@ -70,7 +72,7 @@ output = "ConceptName" The pipes will all have at least this base definition. - `inputs`: Dictionary of key being the variable used in the prompts, and the value being the ConceptName. It should ALSO LIST THE INPUTS OF THE INTERMEDIATE STEPS (if PipeSequence) or of the conditional pipes (if PipeCondition). So If you have this error: -`StaticValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • +`PipeValidationError: missing_input_variable • domain='expense_validator' • pipe='validate_expense' • variable='['invoice']'`` That means that the pipe validate_expense is missing the input `invoice` because one of the subpipe is needing it. @@ -857,21 +859,6 @@ But don't write documentation unless asked explicitly to. ## Guide to execute a pipeline and write example code -### Prerequisites: Virtual Environment - -**CRITICAL**: Before running any `pipelex` commands or `pytest`, you MUST activate the appropriate Python virtual environment. Without proper venv activation, these commands will not work. - -For standard installations, the virtual environment is named `.venv`. Always check this first: - -```bash -## Activate the virtual environment (standard installation) -source .venv/bin/activate # On macOS/Linux -## or -.venv\Scripts\activate # On Windows -``` - -If your installation uses a different venv name or location, activate that one instead. All subsequent `pipelex` and `pytest` commands assume the venv is active. - ### Example to execute a pipeline with text output ```python @@ -1146,7 +1133,7 @@ Presets are meant to record the choice of an llm with its hyper parameters (temp Examples: ```toml llm_to_engineer = { model = "base-claude", temperature = 1 } -llm_to_extract_invoice = { model = "claude-3-7-sonnet", temperature = 0.1, max_tokens = "auto" } +llm_to_extract_invoice = { model = "claude-4.5-sonnet", temperature = 0.1, max_tokens = "auto" } ``` The interest is that these presets can be used to set the LLM choice in a PipeLLM, like this: diff --git a/pyproject.toml b/pyproject.toml index 10b2c91..3c0a5c5 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "my-project" -version = "0.6.4" +version = "0.6.5" description = "Replace this with your project description" # authors = [{ name = "Your Name", email = "your.email@example.com" }] license = "MIT" @@ -16,7 +16,7 @@ classifiers = [ ] dependencies = [ - "pipelex[mistralai,anthropic,google,google-genai,bedrock,fal]==0.15.7", + "pipelex[mistralai,anthropic,google,google-genai,bedrock,fal]==0.17.1", ] [tool.setuptools] diff --git a/tests/e2e/conftest.py b/tests/e2e/conftest.py index 7e60b03..223ccd9 100644 --- a/tests/e2e/conftest.py +++ b/tests/e2e/conftest.py @@ -1,9 +1,9 @@ import os -import pipelex.pipelex import pytest from pipelex.pipe_run.pipe_run_mode import PipeRunMode from pipelex.pipe_run.pipe_run_params import FORCE_DRY_RUN_MODE_ENV_KEY +from pipelex.pipelex import Pipelex from rich import print @@ -13,7 +13,7 @@ def reset_pipelex_instance_fixture(): yield # Code to run after each test print("\n[magenta]pipelex instance teardown[/magenta]") - if pipelex_instance := pipelex.pipelex.Pipelex.get_optional_instance(): + if pipelex_instance := Pipelex.get_optional_instance(): pipelex_instance.teardown() diff --git a/tests/integration/conftest.py b/tests/integration/conftest.py index 54ec66f..028da5f 100644 --- a/tests/integration/conftest.py +++ b/tests/integration/conftest.py @@ -1,8 +1,6 @@ import pipelex.config import pipelex.pipelex import pytest -from pipelex import pretty_print -from pipelex.config import get_config from pipelex.system.configuration.config_check import check_is_initialized from rich import print from rich.console import Console @@ -22,9 +20,6 @@ def reset_pipelex_config_fixture(): print("\n[magenta]pipelex setup[/magenta]") try: pipelex_instance = pipelex.pipelex.Pipelex.make() - config = get_config() - pretty_print(config, title="Test config") - assert isinstance(config, pipelex.config.PipelexConfig) except Exception as exc: Console().print(Traceback()) pytest.exit(f"Critical Pipelex setup error: {exc}") diff --git a/tests/integration/test_fundamentals.py b/tests/integration/test_fundamentals.py index 9917b94..066dfef 100644 --- a/tests/integration/test_fundamentals.py +++ b/tests/integration/test_fundamentals.py @@ -1,7 +1,8 @@ +from pathlib import Path + import pytest -from pipelex.hub import get_pipes +from pipelex.hub import get_library_manager, get_pipes, set_current_library from pipelex.pipe_run.dry_run import DryRunStatus, dry_run_pipes -from pipelex.pipelex import Pipelex # We use gha_disabled here because those tests are called directly and explicitly by the tests-check.yml file before the rest of the tests. @@ -12,11 +13,13 @@ def test_boot(self): # Therefore this test will fail if Pipelex.make() fails. pass - def test_validate_libraries(self): - Pipelex.get_instance().validate_libraries() - @pytest.mark.asyncio(loop_scope="class") async def test_dry_run_all_pipes(self): + library_manager = get_library_manager() + library_id, _ = library_manager.open_library() + set_current_library(library_id) + library_manager.load_libraries(library_id=library_id, library_dirs=[Path("my_project")]) + results = await dry_run_pipes(pipes=get_pipes(), raise_on_failure=False) # Check if there were any failures diff --git a/uv.lock b/uv.lock index 6672f1b..6639d26 100644 --- a/uv.lock +++ b/uv.lock @@ -1037,7 +1037,7 @@ wheels = [ [[package]] name = "my-project" -version = "0.6.4" +version = "0.6.5" source = { virtual = "." } dependencies = [ { name = "pipelex", extra = ["anthropic", "bedrock", "fal", "google", "google-genai", "mistralai"] }, @@ -1064,7 +1064,7 @@ dev = [ requires-dist = [ { name = "boto3-stubs", marker = "extra == 'dev'", specifier = ">=1.35.24" }, { name = "mypy", marker = "extra == 'dev'", specifier = ">=1.11.2" }, - { name = "pipelex", extras = ["mistralai", "anthropic", "google", "google-genai", "bedrock", "fal"], specifier = "==0.15.7" }, + { name = "pipelex", extras = ["mistralai", "anthropic", "google", "google-genai", "bedrock", "fal"], specifier = "==0.17.1" }, { name = "pyright", marker = "extra == 'dev'", specifier = ">=1.1.405" }, { name = "pytest", marker = "extra == 'dev'", specifier = ">=9.0.1" }, { name = "pytest-asyncio", marker = "extra == 'dev'", specifier = ">=0.24.0" }, @@ -1350,7 +1350,7 @@ wheels = [ [[package]] name = "pipelex" -version = "0.15.7" +version = "0.17.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "aiofiles" }, @@ -1380,9 +1380,9 @@ dependencies = [ { name = "typing-extensions" }, { name = "yattag" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/11/64/5fdd37e2cae3d5d4ade7fd510c41e5d335da8a2f5135c0f9e0e1bba37f6f/pipelex-0.15.7.tar.gz", hash = "sha256:5758f7c70d20efe4d2ac704be923a6da1744f170e29cf4816012c41bce637c97", size = 360004, upload-time = "2025-11-18T16:14:40.488Z" } +sdist = { url = "https://files.pythonhosted.org/packages/09/41/431acc4ed3877e2dea125b7d52344ebe0f8f3d17cadeaed9004e13cc49a4/pipelex-0.17.1.tar.gz", hash = "sha256:e7107a7a6e7acf179ed43230a4ac4eb4a0e1af5dbd0322962be3f6054f5f21fc", size = 367468, upload-time = "2025-11-27T16:55:56.191Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/8c/c4/c7393a44a19167fd2016547454e5085e2885ff63491656d87f2572e86a8c/pipelex-0.15.7-py3-none-any.whl", hash = "sha256:3a4a81e1a0dc2b3186990214be6d75c5e8daef81e0c89087869af820639c4d0a", size = 563314, upload-time = "2025-11-18T16:14:38.78Z" }, + { url = "https://files.pythonhosted.org/packages/20/f9/ac53af3828e5cd2dff151b7fe5a4de4aebbe3fc812f8f0240e42fc103055/pipelex-0.17.1-py3-none-any.whl", hash = "sha256:60e1f6a4f1191ce7f7f227fb754dc17c6f10034ff24410ed49ebe267e0dcb017", size = 566837, upload-time = "2025-11-27T16:55:54.241Z" }, ] [package.optional-dependencies]