Free Models
Last updated
Last updated
While paid models often provide superior performance and capabilities, there are also several free models available that can be used with ServerAssistantAI. Many of these models are open-source and can be a good starting point for servers.
Embedding models are used to convert text data into numerical representations called embeddings. These embeddings capture the semantic meaning and relationships between different pieces of text. When the documents/ directory is updated, the content is sent to the embedding API. The resulting embeddings are saved to the cache/ directory, allowing the AI to find relevant context efficiently without reprocessing or making new API requests for each query.
| Provider | Model Name | MTEB | Embedding Dimensions |
|---|---|---|---|
Large Language Models (LLMs) are powerful AI models that can understand and generate human-like text based on the input they receive. In ServerAssistantAI, when a user asks a question, the system retrieves relevant cached context from the embedding API results. This context, along with the user's question, is sent to the LLM to generate accurate and context-aware responses.
| Provider | Model Name | ELO | Speed (tokens per second) | Service uses responses to improve model? |
|---|---|---|---|---|
Open-source free LLMs have a higher chance of hallucinating compared to paid models.
64.68
1024
64.47
1024
64.41
1024
64.23
1024
64.01
1024
1260
57.2 TPS
Yes
1249
300+! TPS
No
1249
250+! TPS
No
1227
133.3 TPS
Yes
1210
47.7 TPS
No
1206
300+! TPS
No
1190
43.16 TPS
No
1176
94.52 TPS
No
meta-llama/Meta-Llama-3-8B-Instruct
1172
N/A
No
01-ai/Yi-1.5-34B-Chat
1157
N/A
No
mistralai/Mixtral-8x7B-Instruct-v0.1
1148
N/A
No
google/gemma-1.1-7b-it
1086
N/A
No
NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO
1083
N/A
No
mistralai/Mistral-7B-Instruct-v0.3
1074
N/A
No