vultr-inference

Generate images and text using Vultr's Inference API.

Setup

Uses the same API key as Vultr Cloud API. Store it at:

~/.config/vultr/api_key

Image Generation

Available Models

Generate Image

curl -X POST "https://api.vultrinference.com/v1/images/generations" \
  -H "Authorization: Bearer $VULTR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "flux.1-schnell",
    "prompt": "a hedgehog eating a burger in Amsterdam",
    "n": 1,
    "size": "1024x1024"
  }'

Parameters

Response

{
  "created": 1734567890,
  "data": [
    {
      "url": "https://ewr.vultrobjects.com/vultrinference-images/tmp_xxx.png"
    }
  ]
}

Text Generation (Chat Completions)

Available Models

• llama-3.1-405b-instruct - Meta Llama 3.1 405B
• llama-3.1-70b-instruct - Meta Llama 3.1 70B
• llama-3.1-8b-instruct - Meta Llama 3.1 8B
• mixtral-8x7b-32768 - Mixtral 8x7B
• qwen-2-72b-instruct - Qwen 2 72B

Chat Completion

curl -X POST "https://api.vultrinference.com/v1/chat/completions" \
  -H "Authorization: Bearer $VULTR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "llama-3.1-70b-instruct",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ],
    "max_tokens": 100
  }'

Parameters

Python Example

import os
import requests

API_KEY = open(os.path.expanduser("~/.config/vultr/api_key")).read().strip()

# Generate image
response = requests.post(
    "https://api.vultrinference.com/v1/images/generations",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "model": "flux.1-schnell",
        "prompt": "a hedgehog eating a burger",
        "size": "512x512",
        "n": 1
    }
)

result = response.json()
image_url = result["data"][0]["url"]
print(f"Image URL: {image_url}")

# Download image
img_response = requests.get(image_url)
with open("generated_image.png", "wb") as f:
    f.write(img_response.content)

List Available Models

curl -s "https://api.vultrinference.com/v1/models" \
  -H "Authorization: Bearer $VULTR_API_KEY" | jq

Troubleshooting

401 Unauthorized

• Check API key is valid
• Ensure key has inference permissions

400 Bad Request

• Check model name is correct
• Check size is valid (256x256, 512x512, 1024x1024)
• Check prompt is not empty

Rate Limits

• Default: 60 requests per minute
• Contact support for higher limits