Skip to content

--request-rate is not respected in multi-turn mode #486

@ajcasagrande

Description

@ajcasagrande

Currently a credit is for a whole conversation, meaning additional turns are not gated by the request rate, causing a higher rate than expected.

Fix is to provide the ability to have per-turn credits.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions