Chat (LLM) Provider Options

Chat (LLM) providers include AI language models that generate responses to user queries based on the provided context.

Built-in Providers

Cohere

• Availability: Built-in, always available

• Name: cohere

• Options:

  • model (required): The name of the Cohere language model to use.

  • timeout (default: 15): The request timeout in seconds.

  • temperature (default: 0): The sampling temperature for generating responses.

  • frequency_penalty (default: 0): The frequency penalty to apply to generated tokens.

  • presence_penalty (default: 0): The presence penalty to apply to generated tokens.

OpenAI

• Availability: Built-in, always available

• Name: openai

• Options:

  • model (required): The name of the OpenAI language model to use.

  • timeout (default: 15): The request timeout in seconds.

  • max_tokens (default: 0): The maximum number of tokens to generate in the response.

  • organization_id (default: ''): The ID of your OpenAI organization. Leave empty if not applicable.

  • base_url (default: 'https://api.openai.com/v1'): The base URL for the OpenAI API. You can use any OpenAI-compatible API by changing this value.

  • stop (default: empty list): A list of strings where the API will stop generating further tokens.

  • temperature (default: 0): The sampling temperature for generating responses.

  • max_retries (default: 3): The maximum number of retries for failed requests.

  • seed (default: -1): The random seed for generating responses. Set to -1 for no seed.

  • frequency_penalty (default: 0): The frequency penalty to apply to generated tokens.

  • presence_penalty (default: 0): The presence penalty to apply to generated tokens.

Addon Providers

Anthropic

• Availability: Anthropic Addon

• Name: anthropic

• Options:

  • model (required): The name of the Anthropic language model to use.

  • timeout (default: 15): The request timeout in seconds.

  • max_tokens (default: 0): The maximum number of tokens to generate in the response.

  • temperature (default: 0): The sampling temperature for generating responses.

  • max_retries (default: 3): The maximum number of retries for failed requests.

  • stop (default: empty list): A list of strings where the API will stop generating further tokens.

Azure OpenAI

• Availability: Azure OpenAI Addon

• Name: azure-openai

• Options:

  • deployment_name (required): The name of the Azure OpenAI deployment (model) to use.

  • timeout (default: 15): The request timeout in seconds.

  • max_tokens (default: 0): The maximum number of tokens to generate in the response.

  • max_retries (default: 3): The maximum number of retries for failed requests.

  • stop (default: empty list): A list of strings where the API will stop generating further tokens.

Google AI Studio

• Availability: Google AI Studio Addon

• Name: google-aistudio

• Options:

  • model (required): The name of the Google AI Studio language model to use.

  • timeout (default: 15): The request timeout in seconds.

  • max_tokens (default: 0): The maximum number of tokens to generate in the response.

  • temperature (default: 0): The sampling temperature for generating responses.

  • max_retries (default: 3): The maximum number of retries for failed requests.

HuggingFace

• Availability: HuggingFace Addon

• Name: huggingface

• Options:

  • model (required): The name of the HuggingFace language model to use.

  • timeout (default: 15): The request timeout in seconds.

  • max_tokens (default: 0): The maximum number of tokens to generate in the response.

  • temperature (default: 0): The sampling temperature for generating responses.

  • max_retries (default: 3): The maximum number of retries for failed requests.

Mistral AI

• Availability: Mistral AI Addon

• Name: mistralai

• Options:

  • model (required): The name of the Mistral AI language model to use.

  • timeout (default: 15): The request timeout in seconds.

  • max_tokens (default: 0): The maximum number of tokens to generate in the response.

  • temperature (default: 0): The sampling temperature for generating responses.

  • max_retries (default: 3): The maximum number of retries for failed requests.

  • base_url (default: 'https://api.mistral.ai/v1'): The base URL for the Mistral AI API. You can use any Mistral AI-compatible API by changing this value.

Please note that the list of addon providers above may not include all available providers, as developers can create their own custom addons using the ServerAssistantAI API.