While creating a customized agent, you will need to select an LLM (Large Language Model), which will take care of a given task, such as retrieving information from the web or a data source to generate an answer relevant to the user’s query, analyzing data, summarizing content, creating content and much more.
Therefore it is crucial to choose the right LLM for your use case. This page will give you some insights to help you select a model from the ones we offer.
Available LLMs
Below is the list of available LLMs with their main characteristics :
Model name | Provider | Context window (Max number of tokens) |
|---|---|---|
GPT 4o (EU) | Azure OpenAI | 128K |
GPT 4o (East US) | Azure OpenAI | 128k |
GPT 4o (South US) | Azure OpenAI | 128k |
GPT 4o mini (EU) | Azure OpenAI | 128k |
GPT 4o mini (East US) | Azure OpenAI | 128k |
GPT 4o mini (South US) | Azure OpenAI | 128k |
Mistral Mini | Mistral | 128k |
Mistral Large | Mistral | 128k |
Mistral Small | Mistral | 128k |
Groq - Llama 3.2 11B | Groq | 128k |
Groq - Llama 3.2 90B | Groq | 128k |
Claude Haiku | Anthropic | 200k |
Claude Sonnet | Anthropic | 200k |
Gemini Flash | 1 mil | |
Gemini Pro | 2 mil |
Selection criteria
need more info from comparison tests (Alban will do them when he has time)
Selecting an LLM for an agent depends on the following criteria.
the type of task done by the agent. Some LLMs excel in specific tasks like …. for example.
the context window of the LLM. This corresponds to the total number of tokens in input and output that can be processed by the LLM.
Some LLMs have larger context windows than others, which might be interesting if you intend to process substantial documents. On the opposite, if you want to process smaller documents, you might want to choose an LLM with a smaller context window.
The size of the context window also matters if you want your agent to retain more information from earlier in the conversation.
the price for the processed tokens. Some LLMs are more expensive than others.
the response time of the LLM. Some LLMs answer faster than others.
Best performing models
Below are our insights on the best-performing models. (might be outdated)
Overall,
GPT 4o is the best-performing model.
GPT 4o mini has good performance and is very fast and cheap.
Mistral Large is slightly worse than GPT 4o but slightly cheaper.
The Groq models are ideal when the text to generate is long.