Skip to main content

Generation settings

Generation settings define how CogSol interacts with the language model at each step of an agent flow.

Depending on the use case, you can configure:

  • The provider
  • The model
  • Temperature and other generation parameters
  • Whether streaming is enabled
  • Different settings for different stages of the interaction

This allows you to optimize each step independently. For example, one stage may use a faster model for classification, while another may use a more capable model for final answer generation.

Supported provider types​

CogSol supports multiple generation providers. In addition to built-in providers, you can also use a custom provider exposed through your own URL.

Generator configurations must use the OpenAI-compatible SDK format. Older configurations not using this format are no supported and must be updated.

Custom provider with OpenAI SDK compatibility​

You can configure a custom generation service if it exposes an API compatible with the OpenAI SDK.

This is useful when:

  • You want to route generation through your own gateway
  • You use a managed provider with an OpenAI-compatible interface
  • You need to keep a provider-specific endpoint outside the default built-in integrations

When using a custom provider, the configuration includes:

  • Service: Custom
  • API mode: choose the compatible API style:
    • chat_completions
    • responses
  • Secret alias: a reference used to resolve the service credentials and base URL securely

The custom service must provide:

  • A valid HTTPS base URL
  • An API key
  • Compatibility with the selected OpenAI API mode

How secrets are resolved​

For a custom provider, CogSol resolves the connection settings from secrets associated with the selected alias.

In practice, the alias points to:

  • The provider base URL
  • The provider API key

This keeps sensitive values out of the generation configuration itself.

Example: Azure Foundry​

Azure Foundry can be used as a custom provider when exposed through an OpenAI-compatible endpoint.

In that case, you can:

  • Select Custom as the service
  • Choose the API mode supported by your endpoint
  • Configure a secret alias that resolves to the Azure Foundry base URL and API key

This approach lets you use Azure Foundry through the same generation configuration model used for other providers.

When to use a custom provider​

Use a custom provider when you need endpoint flexibility beyond the built-in providers, while keeping the same agent configuration flow inside CogSol.

Cost of generation​

The provider and model you select determine how much each response costs: every provider prices its models differently, and options such as reasoning consume additional tokens. CogSol calculates the cost of every response and accumulates it per conversation, so you can see the impact of a generation setting on real usage.

For built-in providers, prices are kept up to date automatically. For a custom provider, such as Azure Foundry, you tell us the price that applies to your models and we load it manually; costs are calculated from that moment on.

See Conversation and message costs to review these values in the portal, and Usage to follow the aggregated spend by agent, model or provider.