On-Premise Installation
To install Jan Server, follow the steps below:
Step 1: Prepare Environment
- Choose a machine with at least 16GB RAM, 8 CPU cores, and 100GB storage.
- For better performance, you can use NVIDIA GPU.
AMD GPU/ Intel Arc GPU are not supported yet.
Step 2: Get Jan Server
-
Before installing the Jan server, ensure that you have the following requirements:
- NVIDIA GPU with CUDA Toolkit 11.7 or higher
- NVIDIA driver 470.63.01 or higher
- NVIDIA Container Toolkit (opens in a new tab)
-
Install Docker Engine and Docker Compose on Linux using the following command:
To install Docker Engine on Ubuntu, follow the instructions here (opens in a new tab).
curl -fsSL https://get.docker.com -o get-docker.shsudo sh ./get-docker.sh --dry-run
- Download Jan
docker-compose.yml
file using the following command:curl https://raw.githubusercontent.com/janhq/jan/dev/docker-compose.yml -o docker-compose.yml
Step 3: Docker Configuration
Once you have installed Docker Engine and Docker Compose, you must set up the docker profile and environment variables.
The available Docker Compose profile and the environment variables are listed below:
Docker compose Profile | Description |
---|---|
cpu-fs | Run Jan in CPU mode with the default file system |
cpu-s3fs | Run Jan in CPU mode with S3 file system |
gpu-fs | Run Jan in GPU mode with the default file system |
gpu-s3fs | Run Jan in GPU mode with S3 file system |
Environment Variable | Description |
---|---|
S3_BUCKET_NAME | S3 bucket name - leave blank for default file system |
AWS_ACCESS_KEY_ID | AWS access key ID - leave blank for default file system |
AWS_SECRET_ACCESS_KEY | AWS secret access key - leave blank for default file system |
AWS_ENDPOINT | AWS endpoint URL - leave blank for default file system |
AWS_REGION | AWS region - leave blank for default file system |
API_BASE_URL | Jan Server URL, please modify it as your public IP address or domain name default http://localhost:1377 (opens in a new tab) |
Step 4: Run Jan Server
You can run the Jan server in two modes:
- CPU
- GPU
Run Jan in CPU Mode
Run Jan in CPU mode by using the following command:
# cpu mode with default file systemdocker compose --profile cpu-fs up -d# cpu mode with S3 file systemdocker compose --profile cpu-s3fs up -d
Run Jan in GPU mode
- Check CUDA compatibility with your NVIDIA driver by running
nvidia-smi
and check the CUDA version in the output:
nvidia-smi# Output+---------------------------------------------------------------------------------------+| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 ||-----------------------------------------+----------------------+----------------------+| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. || | | MIG M. ||=========================================+======================+======================|| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A || 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default || | | N/A |+-----------------------------------------+----------------------+----------------------+| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A || 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default || | | N/A |+-----------------------------------------+----------------------+----------------------+| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A || 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default || | | N/A |+-----------------------------------------+----------------------+----------------------++---------------------------------------------------------------------------------------+| Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage ||=======================================================================================|
-
Visit NVIDIA NGC Catalog (opens in a new tab) and find the smallest minor version of the image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
-
Update the
Dockerfile.gpu
line number 5 with the latest minor version of the image tag from step 2 (e.g., changeFROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base
toFROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base
) -
Run the following command to start Jan in GPU mode:
# GPU mode with default file systemdocker compose --profile gpu-fs up -d# GPU mode with S3 file systemdocker compose --profile gpu-s3fs up -d
Step 5: Access the Jan Server
Once the Jan server runs, you can access it in Jan at http://localhost:3000
.
RAG feature is not yet supported in Docker mode with s3fs
.