Skip to main content

Documentation Index

Fetch the complete documentation index at: https://apidoc.cometapi.com/llms.txt

Use this file to discover all available pages before exploring further.

Langfuse provides LLM observability, prompt management, playgrounds, and evaluation workflows. Configure a Langfuse LLM Connection with CometAPI when you want Langfuse Playground or LLM-as-a-Judge evaluations to call CometAPI models.

​
Prerequisites

​
Configure the integration

1

Create a Langfuse LLM connection

In Langfuse, open Project Settings → LLM Connections. Start the flow for adding an LLM API key.
2

Choose the OpenAI adapter

Select OpenAI as the provider. Langfuse uses this adapter for model providers that support the OpenAI API schema.
3

Enter CometAPI credentials

Configure the connection with these values:
FieldValue
API keyYour CometAPI API key
Base URLhttps://api.cometapi.com/v1
Custom model IDsOne or more model IDs from the CometAPI Models page
If Langfuse exposes the base URL field under Advanced Settings, expand that section before saving the connection.
4

Test in Playground

In Langfuse Playground, select the CometAPI LLM connection and one of the configured model IDs. Send a short prompt to confirm that Langfuse receives a response.

​
Use CometAPI for LLM-as-a-Judge

Langfuse LLM-as-a-Judge evaluators can use the same LLM connection. Select the CometAPI connection in the evaluator configuration, then choose a model ID that supports tool calling if your scoring prompt requires structured extraction. Use provider options only when the selected CometAPI model supports those request fields. For model discovery, use the CometAPI Models page or the /v1/models API.

​
Troubleshooting

Add the exact CometAPI model ID to the custom model IDs list in the LLM connection. Langfuse does not discover every model automatically for custom OpenAI-compatible providers.
Use https://api.cometapi.com/v1 as the base URL. Langfuse appends the OpenAI-compatible path for the selected adapter.
Confirm that the selected model supports the request features used by the evaluator, such as tool calling or JSON output.