Skip to main content
Add model fallback by keeping a small ordered list of model IDs in your app and trying the next model only when the failure is retryable or model-specific. Do not retry malformed requests; fix 400, 401, and invalid request errors before you call another model.

​
Implement ordered fallback

The following Python example tries model IDs in order and stops on non-retryable errors: 
import os
import time
from openai import OpenAI, APIError, RateLimitError

client = OpenAI(
    api_key=os.environ["COMETAPI_KEY"],
    base_url="https://api.cometapi.com/v1",
)

models = ["your-primary-model-id", "your-fallback-model-id"]

for model in models:
    try:
        completion = client.chat.completions.create(
            model=model,
            messages=[
                {
                    "role": "user",
                    "content": "Summarize CometAPI in one sentence.",
                }
            ],
        )
        print(completion.choices[0].message.content)
        break
    except RateLimitError:
        time.sleep(2)
        continue
    except APIError as error:
        status_code = getattr(error, "status_code", None)
        if status_code in {500, 503, 504, 524}:
            time.sleep(2)
            continue
        raise
else:
    raise RuntimeError("All configured model fallbacks failed.")
The successful response comes from the first model that completes: 
{
  "model": "your-fallback-model-id",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "CometAPI gives developers one API surface for multiple model providers."
      }
    }
  ]
}

​
Choose fallback models

The following request lists available model IDs that you can evaluate for fallback: 
curl https://api.cometapi.com/api/models
The response contains model records: 
{
  "success": true,
  "page": 0,
  "page_size": 0,
  "total": 301,
  "data": [
    {
      "id": "deepseek-v4-pro",
      "provider": "DeepSeek",
      "model_type": "text",
      "features": [
        "text-to-text"
      ],
      "pricing": {
        "currency": "USD / M Tokens",
        "input": 0.416,
        "output": 0.832,
        "per_request": null,
        "per_second": null
      },
      "api_doc_url": "https://apidoc.cometapi.com/api/text/chat"
    }
  ]
}

​
Common errors

ErrorFix
Fallback hides invalid requestsDo not fallback on 400, 401, or invalid request errors.
Models have different output formatsNormalize the response in your app before returning it to users.
Fallback increases costEstimate cost for each model before adding it to the fallback list.
Too many parallel fallbacksTry fallbacks sequentially unless your product requires parallel racing.
Last modified on May 28, 2026