Documentation
Installation
Server

Server Installation

To install Jan Server, follow the steps below:

Step 1: Prepare Environment

  • Choose a machine with at least 16GB RAM, 8 CPU cores, and 100GB storage.
  • For better performance, you can use NVIDIA GPU.

AMD GPU/ Intel Arc GPU are not supported yet.

Step 2: Get Jan Server

Pre-requisites

To enable GPU support, you will need:

  • NVIDIA GPU with CUDA Toolkit 11.7 or higher
  • NVIDIA driver 470.63.01 or higher

Jan runs in Docker mode on Linux:


curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh ./get-docker.sh --dry-run

  • Download Jan docker-compose.yml file using the following command:

curl https://raw.githubusercontent.com/janhq/jan/dev/docker-compose.yml -o docker-compose.yml

  • If you intend to run Jan in GPU mode, install nvidia-driver and nvidia-docker2. Follow the instructions here (opens in a new tab) for installation.

Docker Configuration

The available Docker Compose profile and the environment variables are listed below:

Docker compose ProfileDescription
cpu-fsRun Jan in CPU mode with the default file system
cpu-s3fsRun Jan in CPU mode with S3 file system
gpu-fsRun Jan in GPU mode with the default file system
gpu-s3fsRun Jan in GPU mode with S3 file system
Environment VariableDescription
S3_BUCKET_NAMES3 bucket name - leave blank for default file system
AWS_ACCESS_KEY_IDAWS access key ID - leave blank for default file system
AWS_SECRET_ACCESS_KEYAWS secret access key - leave blank for default file system
AWS_ENDPOINTAWS endpoint URL - leave blank for default file system
AWS_REGIONAWS region - leave blank for default file system
API_BASE_URLJan Server URL, please modify it as your public IP address or domain name default http://localhost:1377 (opens in a new tab)
  • Option 1: Run Jan in CPU mode


    # cpu mode with default file system
    docker compose --profile cpu-fs up -d
    # cpu mode with S3 file system
    docker compose --profile cpu-s3fs up -d

  • Option 2: Run Jan in GPU mode

    • Step 1: Check CUDA compatibility with your NVIDIA driver by running nvidia-smi and check the CUDA version in the output


      nvidia-smi
      # Output
      +---------------------------------------------------------------------------------------+
      | NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
      |-----------------------------------------+----------------------+----------------------+
      | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
      | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
      | | | MIG M. |
      |=========================================+======================+======================|
      | 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
      | 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
      | | | N/A |
      +-----------------------------------------+----------------------+----------------------+
      | 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
      | 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
      | | | N/A |
      +-----------------------------------------+----------------------+----------------------+
      | 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
      | 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
      | | | N/A |
      +-----------------------------------------+----------------------+----------------------+
      +---------------------------------------------------------------------------------------+
      | Processes: |
      | GPU GI CI PID Type Process name GPU Memory |
      | ID ID Usage |
      |=======================================================================================|

    • Step 2: Visit NVIDIA NGC Catalog (opens in a new tab) and find the smallest minor version of the image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)

    • Step 3: Update the Dockerfile.gpu line number 5 with the latest minor version of the image tag from step 2 (e.g., change FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base to FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base)

    • Step 4: Run command to start Jan in GPU mode


      # GPU mode with default file system
      docker compose --profile gpu-fs up -d
      # GPU mode with S3 file system
      docker compose --profile gpu-s3fs up -d

    • Step 5: This will start the web server, and you can access it in Jan at http://localhost:3000 (opens in a new tab).

RAG feature is not yet supported in Docker mode with s3fs.