> ## Documentation Index
> Fetch the complete documentation index at: https://apidoc.cometapi.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Use Promptfoo with CometAPI

> Use this guide to configure Promptfoo with CometAPI by setting the base URL, API key, and model or provider options.

[Promptfoo](https://www.promptfoo.dev/) is a framework for prompt evaluation, regression testing, and AI red teaming. Promptfoo includes a `cometapi` provider that calls CometAPI through OpenAI-compatible endpoints.

## Prerequisites

* Node.js 18+
* Promptfoo installed locally or run through `npx`
* A CometAPI account with an active API key — [get yours in the dashboard](https://www.cometapi.com/console/token)
* A model ID from the [CometAPI Models page](/overview/models)

## Configure the integration

<Steps>
  <Step title="Set your CometAPI API key">
    Set `COMETAPI_KEY` before you run evaluations:

    ```bash theme={null}
    read -rsp "CometAPI API key: " COMETAPI_KEY
    printf '\n'
    export COMETAPI_KEY
    ```
  </Step>

  <Step title="Choose a provider format">
    Promptfoo uses the following provider ID format for CometAPI:

    ```yaml theme={null}
    providers:
      - cometapi:<type>:<model>
    ```

    Use these provider types:

    | Type         | Use case                                         |
    | ------------ | ------------------------------------------------ |
    | `chat`       | Chat completions, vision, and multimodal prompts |
    | `completion` | Text completion models                           |
    | `embedding`  | Text embedding evaluations                       |
    | `image`      | Image generation evaluations                     |

    You can also use `cometapi:your-model-id` for the default chat mode.
  </Step>

  <Step title="Run a CLI evaluation">
    The following command runs a one-prompt chat evaluation:

    ```bash theme={null}
    npx promptfoo@latest eval \
      --prompts "Write a concise release note for {{feature}}" \
      --vars feature="a unified API dashboard" \
      -r cometapi:chat:your-model-id
    ```
  </Step>

  <Step title="Create a Promptfoo config file">
    The following `promptfooconfig.yaml` evaluates the same prompt against a CometAPI model:

    ```yaml theme={null}
    prompts:
      - "Classify this support request: {{message}}"

    providers:
      - id: cometapi:chat:your-model-id
        config:
          temperature: 0.2
          max_tokens: 256

    tests:
      - vars:
          message: "The API key works locally but fails in production."
        assert:
          - type: contains-any
            value:
              - authentication
              - configuration
    ```

    Run the config file with Promptfoo:

    ```bash theme={null}
    npx promptfoo@latest eval -c promptfooconfig.yaml
    ```
  </Step>
</Steps>

## Provider examples

Use a provider type that matches the model behavior that you want to evaluate:

```yaml theme={null}
providers:
  - cometapi:chat:your-model-id
  - cometapi:completion:your-model-id
  - cometapi:embedding:your-model-id
  - cometapi:image:your-model-id
```

For multimodal tests, keep the `chat` provider type and pass image variables through Promptfoo test variables.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Promptfoo cannot find the API key">
    Confirm that `COMETAPI_KEY` is exported in the same shell session that runs `promptfoo eval`.
  </Accordion>

  <Accordion title="The provider type does not match the model">
    Use `chat` for conversational and multimodal models, `embedding` for embedding models, and `image` for image generation models.
  </Accordion>

  <Accordion title="The model ID fails">
    Replace `your-model-id` with an exact model ID from the [CometAPI Models page](/overview/models).
  </Accordion>
</AccordionGroup>

## Related resources

* [Promptfoo CometAPI provider](https://www.promptfoo.dev/docs/providers/cometapi/)
* [CometAPI quick start](/overview/quick-start)
* [CometAPI Models page](/overview/models)

<script type="application/ld+json">
  {`
    {
    "@context": "https://schema.org",
    "@graph": [
      {
        "@type": "HowTo",
        "@id": "https://apidoc.cometapi.com/integrations/promptfoo#howto",
        "name": "Use Promptfoo with CometAPI",
        "description": "Use this guide to configure Promptfoo with CometAPI by setting the base URL, API key, and model or provider options.",
        "step": [
          {
            "@type": "HowToStep",
            "@id": "https://apidoc.cometapi.com/integrations/promptfoo#step-1",
            "position": 1,
            "name": "Set your CometAPI API key",
            "text": "Store your CometAPI API key in the environment variable or settings field used by the integration."
          },
          {
            "@type": "HowToStep",
            "@id": "https://apidoc.cometapi.com/integrations/promptfoo#step-2",
            "position": 2,
            "name": "Choose a provider format",
            "text": "Complete the Choose a provider format step in the Use Promptfoo with CometAPI guide."
          },
          {
            "@type": "HowToStep",
            "@id": "https://apidoc.cometapi.com/integrations/promptfoo#step-3",
            "position": 3,
            "name": "Run a CLI evaluation",
            "text": "Complete the Run a CLI evaluation step in the Use Promptfoo with CometAPI guide."
          },
          {
            "@type": "HowToStep",
            "@id": "https://apidoc.cometapi.com/integrations/promptfoo#step-4",
            "position": 4,
            "name": "Create a Promptfoo config file",
            "text": "Complete the Create a Promptfoo config file step in the Use Promptfoo with CometAPI guide."
          }
        ]
      },
      {
        "@type": "BreadcrumbList",
        "itemListElement": [
          {
            "@type": "ListItem",
            "position": 1,
            "name": "CometAPI Docs",
            "item": "https://apidoc.cometapi.com/"
          },
          {
            "@type": "ListItem",
            "position": 2,
            "name": "Integrations",
            "item": "https://apidoc.cometapi.com/integrations"
          },
          {
            "@type": "ListItem",
            "position": 3,
            "name": "Use Promptfoo with CometAPI",
            "item": "https://apidoc.cometapi.com/integrations/promptfoo"
          }
        ]
      }
    ]
    }
    `}
</script>