Skip to content

[FEATURE] Improve setup_models.py to build engines #108

@eliteprox

Description

@eliteprox

Feature type

Build

Describe the feature

setup_models.py should both download and build engines

Motivation

Currently there is a loose dependency between ai-runner and comfystream with models.yaml and dl_checkpoints.sh. A change to models.yaml can break dl_checkpoints and the ai-runner CI.

dl_checkpoints already reuses setup_models.py to download the models https://github.com/livepeer/ai-runner/blob/9e21e45480bf3112388f5b4b6bfb11df3ea51522/runner/dl_checkpoints.sh#L101-L111, however it doesn't compile them.

dl_checkpoints separately compiles tensorrt engines and assumes the model files are present. https://github.com/livepeer/ai-runner/blob/9e21e45480bf3112388f5b4b6bfb11df3ea51522/runner/dl_checkpoints.sh#L136-L145

models.yaml can be updated to include the necessary parameters to compile the engines so that build_trt is only ran for models which were also downloaded by the config.

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions