Skip to content

fix(server): make uvicorn workers>1 work (factory + spawn for CUDA)#1141

Merged
Stardust-minus merged 3 commits intofishaudio:mainfrom
vladlearns:main
Mar 23, 2026
Merged

fix(server): make uvicorn workers>1 work (factory + spawn for CUDA)#1141
Stardust-minus merged 3 commits intofishaudio:mainfrom
vladlearns:main

Conversation

@vladlearns
Copy link
Copy Markdown
Contributor

@vladlearns vladlearns commented Jan 8, 2026

FIXES:
#1142
#1143

Bugs sum up

  1. On linux, default multiprocessing method is fork(). CUDA ctxs can't be inherited by forked child - when multiple workers try to init CUDA after forking, they crash. Only one worker survives /w uv retries;
  2. Got an issues running /w workers > 1 in 1 container, each worker starts as a separate process and imports the module fresh - cli args parsed in __main__ only exist in the parent process, so workers would miss that config and fall back to defaults/break.

What I changed

  • Set multiprocessing to spawn before starting uv, so each worker starts fresh = no fork = no CUDA init crashes;
  • Parent now dumps parsed args to json and stores them in var before starting uv;
  • Workers calls the factory, it reads the var, loads the json, and rebuilds the namespace, so every worker gets the same config.

QA

  • Solo-worker - same;
  • Multi-workers - config is passed to all workers;
  • Tested in Docker with 3 workers on 4090.

@vladlearns vladlearns changed the title fix: pass cli args to uv workers via env vars fix: support uvicorn multi-worker Jan 8, 2026
@vladlearns vladlearns changed the title fix: support uvicorn multi-worker fix(server): make uvicorn workers>1 work (factory + spawn for CUDA) Jan 9, 2026
@vladlearns
Copy link
Copy Markdown
Contributor Author

@Juanma-t, this should help you

@github-actions
Copy link
Copy Markdown
Contributor

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale label Feb 28, 2026
@vladlearns vladlearns marked this pull request as ready for review March 19, 2026 16:25
@github-actions github-actions Bot removed the stale label Mar 20, 2026
@Stardust-minus Stardust-minus merged commit 46806ae into fishaudio:main Mar 23, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants