Skip to content
Open
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions cep-recipe-extensions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Extensions to the recipe spec

This document describes extensions to the recipe specification as merged in cep-14.

## The `build.post_process` section

We became aware that the `conda_build_config.yaml` has ways to specify post-processing steps. These are regex replacements that are performed on any new files.

Instead of adding these instructions to the `conda_build_config.yaml`, we decided to add a new section to the recipe spec: `build.post_process`. This section is a list of dictionaries, each with the following keys:

- `files`: globs to select files to process
- `regex`: the regex to apply
- `replacement`: the replacement string

The regex specification follows Rust `regex` syntax. Most notably, the replacement string can refer to capture groups with `$1`, `$2`, etc.
This also means that replacement strings need to escape `$` with `$$`.

Internally, we use `replace_all` from the `regex` crate. This means that the regex is applied to the entire file, not line by line.

### Example

```yaml
build:
post_process:
- files:
- "*.txt"
regex: "foo"
replacement: "bar"
- files:
- "*.cmake"
regex: "/sysroot/"
replacement: "$${PREFIX}/"
```

## Globs, positive and negative
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have to do with recipe post-processing? 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. This really feels like it should just be incorporated into the build script(s).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The positive/negative globs should be part of the recipe. Using the build scripts just to shuffle files around into separate output is a lot of core for little gain, especially because it's a well-defined operation and doesn't overload the recipe syntax IMO.

C.f. also conda/conda-build#5216

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No this doesn't really have anything to do with post-processing per se. The idea is that any list of globs can also include negative globs.

List of globs appear in multiple places in the recipe.


Following some community discussion we would like to extend the recipe format in such a way that anywhere where a list of globs is accepted, we alternatively accept a dictionary with `include` and `exclude` keys. The values of these keys are lists of globs that are included or excluded respectively.

For example:

```yaml
files:
include:
- "*.txt"
exclude:
- "foo.txt"
```

The evaluation would go as follows:

- first record all matches for the `include` globs
- then remove all matches for the `exclude` globs

In conda-build, this is discussed here: [Link to PR 5216](https://github.com/conda/conda-build/pull/5216)