API overview

Model hosting API

Our hosted models speak the OpenAI API, so most clients work as-is. Base URL for everything below:

https://api.gpus4all.dev/v1

You get an API key when you create an account. Send it as a Bearer token.

The deploy model screen — Pick a public model or upload your own, set a spending cap, and we give you an endpoint.

Authentication

Pass your key on every request.

$ curl https://api.gpus4all.dev/v1/models \
  -H "Authorization: Bearer g4a_live_xxxxxxxxxxxxxxxxxxxx"

Keys start with g4a_live_. Test keys are not available yet; the sandbox is the same API with a $5 monthly cap.

List models

Get the model ids you can call.

GET /v1/models

Model id	Class	Context
gpus4all/llama-3.1-8b-instruct	7B	128k
gpus4all/qwen-2.5-7b-instruct	7B	32k
gpus4all/mistral-7b-instruct	7B	32k
gpus4all/llama-3.3-70b-instruct	70B	128k
gpus4all/deepseek-r1-distill-70b	70B	64k

We add models on request. If you need something specific, point us at the checkpoint and we usually have it up within a day.

Chat completions

$ curl https://api.gpus4all.dev/v1/chat/completions \
  -H "Authorization: Bearer g4a_live_xxxxxxxxxxxxxxxxxxxx" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpus4all/llama-3.1-8b-instruct",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

Standard parameters are supported: temperature, top_p, max_tokens, stream, stop. Responses match the OpenAI shape, so LangChain, openai-python, and other clients work without changes.

Health

A liveness check that does not count against your quota.

$ curl https://api.gpus4all.dev/health

Returns healthy with a 200. Our status page mirrors the same check and posts a note to the mailing list when something is down.

Limits

Rate limit is 60 requests per minute per key by default. Raise it on request for paid plans.
Max context is 128k tokens on the 70B models, 32k on the 7B ones.
Each deployment has a monthly spending cap you set yourself.
Timeouts: 60s to first token, 300s total per request.