Skip to content

Latest commit

 

History

History

examples

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
api
api
 
 
inference
inference
 
 
logits
logits
 
 
slurm_dependency
slurm_dependency
 
 
README.md
README.md
 
 

README.md

Examples

  • inference: Examples for sending inference requests
    • llm/chat_completions.py: Python example of sending chat completion requests to OpenAI compatible server
    • llm/completions.py: Python example of sending completion requests to OpenAI compatible server
    • llm/completions.sh: Bash example of sending completion requests to OpenAI compatible server, supports JSON mode
    • text_embedding/embeddings.py: Python example of sending text embedding requests to OpenAI compatible server
    • vlm/vision_completions.py: Python example of sending chat completion requests with image attached to prompt to OpenAI compatible server for vision language models
  • logits: Example for logits generation
    • logits.py: Python example of getting logits from hosted model.
  • api: Examples for using the Python API
    • basic_usage.py: Basic Python example demonstrating the Vector Inference API
  • slurm_dependency: Example of launching a model with vec-inf and running a downstream SLURM job that waits for the server to be ready before sending a request.