Custom Embedding Model

CogSol allows configuring a custom embedding model at the tenant level, replacing the default model with one hosted externally or on-premise.

This is useful for deployments where the embedding model runs outside of CogSol's infrastructure, either as part of an on-premise installation or as a separately managed service accessible via API.

When to Use This Configuration

• On-premise deployments: CogSol is deployed within a client's own infrastructure and the embedding model is hosted internally.
• External embedding service: The client has a dedicated embedding model (e.g., a fine-tuned or domain-specific model) hosted on their own servers or a third-party provider.

Configuration Parameters

The following parameters are set at the tenant level:

Parameter	Required	Description
embedding_url	Yes	Base URL of the embedding service (e.g., http://localhost:7997)
embedding_model	Yes	Name of the model as expected by the embedding service
embedding_api_key	No	API key for authenticating with the embedding service, if required

API Compatibility Requirement

The embedding service must expose an endpoint compatible with the OpenAI Embeddings API format.

Specifically, it must accept POST requests at /v1/embeddings with the following structure:

Request

POST /v1/embeddings
{
  "input": "text to embed",
  "model": "your-model-name"
}

Response

{
  "object": "list",
  "data": [
    {
      "object": "embedding",
      "index": 0,
      "embedding": [0.0023, -0.0093, ...]
    }
  ],
  "model": "your-model-name",
  "usage": {
    "prompt_tokens": 8,
    "total_tokens": 8
  }
}

Servers such as Infinity or LocalAI implement this interface and can be used to serve open-source embedding models in a compatible way.

How It Works

When a custom embedding model is configured for a tenant, all embedding operations — both during document ingestion and semantic search — will use that model instead of the platform default.

This means:

• Document blocks are vectorized using the custom model when content is uploaded or re-processed.
• Semantic searches use the same model to generate the query embedding, ensuring consistency between stored vectors and search queries.

caution

If the custom model is changed after documents have already been ingested, existing embeddings will be incompatible with new query vectors. A full re-ingestion of content is required to maintain search quality.

Additional Considerations

• The embedding service must be reachable from CogSol's backend at the configured URL.
• The model name must match exactly what the embedding service expects in its API.
• If an API key is provided, it will be sent as a Bearer token in the Authorization header of each request.
• This configuration is applied per tenant and does not affect other tenants on the same platform instance.