> ## Documentation Index
> Fetch the complete documentation index at: https://apidoc.cometapi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Handle rate limits and concurrency

> Handle CometAPI rate limits by capping request concurrency, retrying 429 responses with jitter, and monitoring usage by model and route.

Handle rate limits by controlling concurrency before requests leave your app. When CometAPI returns `429`, retry with exponential backoff and jitter, then lower burst traffic if repeated retries occur.

## Limit concurrency

The following Python example caps concurrent chat requests with an async semaphore:

```python theme={null}
import asyncio
import os
from openai import AsyncOpenAI

client = AsyncOpenAI(
    api_key=os.environ["COMETAPI_KEY"],
    base_url="https://api.cometapi.com/v1",
)

semaphore = asyncio.Semaphore(5)

async def ask(prompt):
    async with semaphore:
        completion = await client.chat.completions.create(
            model="your-model-id",
            messages=[{"role": "user", "content": prompt}],
        )
        return completion.choices[0].message.content

async def main():
    prompts = ["Say hello.", "Write a title.", "Return one JSON key."]
    results = await asyncio.gather(*(ask(prompt) for prompt in prompts))
    print(results)

asyncio.run(main())
```

The result is an array of model outputs:

```json theme={null}
[
  "Hello.",
  "A concise title",
  "{\"key\":\"value\"}"
]
```

## Retry rate limits

The following JavaScript example retries `429` responses with jitter:

```javascript theme={null}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.COMETAPI_KEY,
  baseURL: "https://api.cometapi.com/v1",
});

async function sleep(milliseconds) {
  return new Promise((resolve) => setTimeout(resolve, milliseconds));
}

async function createCompletion() {
  for (let attempt = 0; attempt < 5; attempt += 1) {
    try {
      return await client.chat.completions.create({
        model: "your-model-id",
        messages: [{ role: "user", content: "Say hello." }],
      });
    } catch (error) {
      if (error.status !== 429 || attempt === 4) {
        throw error;
      }

      const delay = Math.min(30000, 1000 * 2 ** attempt);
      await sleep(delay + Math.random() * 1000);
    }
  }
}

const completion = await createCompletion();
console.log(completion.choices[0].message.content);
```

The successful response contains a normal chat completion:

```json theme={null}
{
  "choices": [
    {
      "message": {
        "content": "Hello."
      }
    }
  ]
}
```

## Common errors

| Error                       | Fix                                                        |
| --------------------------- | ---------------------------------------------------------- |
| Unlimited parallel requests | Add a semaphore, queue, or worker pool.                    |
| Retrying all failures       | Retry only `429` and temporary server failures.            |
| No per-model metrics        | Log route, model ID, status, and latency for each request. |
| Retry storm                 | Add jitter and cap the maximum retry delay.                |

## Related links

* [Error codes and retry strategy](/guides/error-codes-and-retry-strategy)
* [Error Codes & Handling](/errors/error-codes-handling)
* [Models page](/overview/models)
* [Model directory](https://www.cometapi.com/models/)
* [Pricing](https://www.cometapi.com/pricing/)
* [CometAPI quickstart](/overview/quick-start)

<script type="application/ld+json">
  {`
    {
    "@context": "https://schema.org",
    "@graph": [
      {
        "@type": "TechArticle",
        "@id": "https://apidoc.cometapi.com/guides/rate-limits-and-concurrency",
        "headline": "How do I handle rate limits and concurrency?",
        "description": "Handle CometAPI rate limits by capping concurrency, retrying 429 responses with jitter, and monitoring usage by model and route.",
        "url": "https://apidoc.cometapi.com/guides/rate-limits-and-concurrency",
        "author": {
          "@type": "Organization",
          "name": "CometAPI"
        },
        "publisher": {
          "@type": "Organization",
          "name": "CometAPI",
          "url": "https://www.cometapi.com"
        }
      },
      {
        "@type": "BreadcrumbList",
        "itemListElement": [
          {
            "@type": "ListItem",
            "position": 1,
            "name": "CometAPI Docs",
            "item": "https://apidoc.cometapi.com/"
          },
          {
            "@type": "ListItem",
            "position": 2,
            "name": "Guides",
            "item": "https://apidoc.cometapi.com/guides/use-cometapi-with-openai-sdk"
          },
          {
            "@type": "ListItem",
            "position": 3,
            "name": "How do I handle rate limits and concurrency?",
            "item": "https://apidoc.cometapi.com/guides/rate-limits-and-concurrency"
          }
        ]
      }
    ]
    }
    `}
</script>