Documentation
Local Engines
Ollama

Ollama

Integrate Ollama with Jan

Ollama provides you with large language models that you can run locally. There are two methods to integrate Ollama with Jan:

  1. Integrate the Ollama server with Jan.
  2. Migrate the downloaded model from Ollama to Jan.

To integrate Ollama with Jan, follow the steps below:

This tutorial will show how to integrate Ollama with Jan using the first method. We will use the llama2 (opens in a new tab) model as an example.

Step 1: Server Setup

According to the Ollama documentation on OpenAI compatibility (opens in a new tab), you can connect to the Ollama server using the web address http://localhost:11434/v1/chat/completions. To do this, follow the steps below:

  1. Navigate to the Settings > Extensions.
  2. In the OpenAI Inference Engine section, add the full web address of the Ollama server.

Server Setup

Leave the API Key field blank.

Step 2: Download Model

  1. Navigate to the Hub.
  2. Download the Ollama model, for example, Llama 2 Chat 7B Q4.

Download Model

Step 3: Start the Model

  1. Navigate to the Threads.
  2. Select the Llama 2 Chat 7B Q4 model and configure the model parameters.
  3. Start chatting with the model.

Start Model

LM StudioOpenAI API