Add git derivation to PATH of uv binary

[juliankuners]

The haskell-backend uses evm-semantics and kontrol for profiling. For this purpose, the script scripts/performance-tests-kontrol.sh runs the develop shell with the --ignore-environment for increased reproducibility. This in turn causes git to be missing from the environment, which is an optional runtime dependency for uv that is required for us, because we reference git repositories in the pyproject.toml.

This pull request wraps uv and adds the git derivation to the PATH of uv. This will make the --ignore-environment kontrol nix develop shell in haskell-backend work reproducibly.

[juliankuners]

An upstream pull request at uv2nix was closed by the maintainer: pyproject-nix/uv2nix#195.

[jberthold]

I tried this branch today with our performance test script

haskell-backend$ KONTROL_VERSION=uv-wrapped-with-git scripts/performance-test-kontrol.sh

The issue with the missing git is addressed, but unfortunately all tests fail with a permission error

The error looks like the python process does not have access to its sub-process `kore-rpc-booster` via `psutil`

Details

__________________ ERROR at setup of test_foundry_refute_node __________________
[gw1] linux -- Python 3.10.14 /tmp/tmp.MD1gdaAkMg/kontrol/.venv/bin/python

self = <psutil._pslinux.Process object at 0x7fffefc09c10>, args = ('inet',)
kwargs = {}

    @functools.wraps(fun)
    def wrapper(self, *args, **kwargs):
        try:
>           return fun(self, *args, **kwargs)

.venv/lib/python3.10/site-packages/psutil/_pslinux.py:1714: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.10/site-packages/psutil/_pslinux.py:2344: in connections
    ret = _connections.retrieve(kind, self.pid)
.venv/lib/python3.10/site-packages/psutil/_pslinux.py:1028: in retrieve
    inodes = self.get_proc_inodes(pid)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <psutil._pslinux.Connections object at 0x7ffff58f7a30>, pid = 1442872

    def get_proc_inodes(self, pid):
        inodes = defaultdict(list)
>       for fd in os.listdir("%s/%s/fd" % (self._procfs_path, pid)):
E       PermissionError: [Errno 13] Permission denied: '/proc/1442872/fd'

.venv/lib/python3.10/site-packages/psutil/_pslinux.py:849: PermissionError

During handling of the above exception, another exception occurred:

foundry = <kontrol.foundry.Foundry object at 0x7fffef7ac160>

    @pytest.fixture(scope='module')
    def server(foundry: Foundry) -> Iterator[KoreServer]:
        llvm_definition_dir = foundry.out / 'kompiled' / 'llvm-library'
        kore_rpc_command = ('kore-rpc-booster',)
    
>       yield kore_server(
            definition_dir=foundry.kevm.definition_dir,
            llvm_definition_dir=llvm_definition_dir,
            module_name=foundry.kevm.main_module,
            command=kore_rpc_command,
            smt_timeout=500,
            smt_retry_limit=10,
            fallback_on=None,
            interim_simplification=None,
            no_post_exec_simplify=None,
        )

src/tests/integration/conftest.py:40: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.10/site-packages/pyk/kore/rpc.py:1394: in kore_server
    return BoosterServer(booster_args)
.venv/lib/python3.10/site-packages/pyk/kore/rpc.py:1316: in __init__
    super().__init__(args)
.venv/lib/python3.10/site-packages/pyk/kore/rpc.py:1121: in __init__
    self.start()
.venv/lib/python3.10/site-packages/pyk/kore/rpc.py:1153: in start
    host, port = self._get_host_and_port(pid)
.venv/lib/python3.10/site-packages/pyk/kore/rpc.py:1252: in _get_host_and_port
    while not proc.connections():
.venv/lib/python3.10/site-packages/psutil/__init__.py:1218: in connections
    return self._proc.connections(kind)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <psutil._pslinux.Process object at 0x7fffefc09c10>, args = ('inet',)
kwargs = {}

    @functools.wraps(fun)
    def wrapper(self, *args, **kwargs):
        try:
            return fun(self, *args, **kwargs)
        except PermissionError:
>           raise AccessDenied(self.pid, self._name)
E           psutil.AccessDenied: (pid=1442872)

.venv/lib/python3.10/site-packages/psutil/_pslinux.py:1716: AccessDenied

while trying to call self._get_host_and_port during server start-up.

The test without `--ignore-environment` works as before. Might there be some other tools missing in the derivation?

[juliankuners]

I tried this branch today with our performance test script

haskell-backend$ KONTROL_VERSION=uv-wrapped-with-git scripts/performance-test-kontrol.sh

The issue with the missing git is addressed, but unfortunately all tests fail with a permission error
The error looks like the python process does not have access to its sub-process kore-rpc-booster via psutil

while trying to call self._get_host_and_port during server start-up.
The test without --ignore-environment works as before. Might there be some other tools missing in the derivation?

This behaviour sounds really odd. Unfortunately, I cannot reproduce this, because I am running into OOM errors during kompile. I will have to increase my swap partition before I can move forward.

Could you please try to update nix and repeat the test?

[jberthold]

I tried this branch today with our performance test script

haskell-backend$ KONTROL_VERSION=uv-wrapped-with-git scripts/performance-test-kontrol.sh

The issue with the missing git is addressed, but unfortunately all tests fail with a permission error
The error looks like the python process does not have access to its sub-process kore-rpc-booster via psutil
while trying to call self._get_host_and_port during server start-up.
The test without --ignore-environment works as before. Might there be some other tools missing in the derivation?

This behaviour sounds really odd. Unfortunately, I cannot reproduce this, because I am running into OOM errors during kompile. I will have to increase my swap partition before I can move forward.

Could you please try to update nix and repeat the test?

I updated nix to 2.28.4 (that's the version I got when following the Linux upgrade instructions from the nix manual) but the result remains the same: when running the performance script with this PR's branch, all tests fail on a permission error during server setup.

[juliankuners]

Git overlay works. The other encountered issue with --ignore-environment is a separate issue.