Azure Installation
To install Jan Server, follow the steps below:
Step 1: Prepare Environment
- Go to Azure console ->
Service
->Virtual machines
. - Choose an instance with at least
Standard_F8s_v2
for CPU only orStandard_NC4as_T4_v3
for NVIDIA GPU support. - Add an Azure Disk or Azure Blob Storage volume with at least 100GB.
- Configure network security group rules to allow inbound traffic on port
1377
.
Step 2: Get Jan Server
- Before installing the Jan server, ensure that you have the following requirements:
- Windows 10 or higher is required to run Jan.
- WSL2 must run in Windows in Jan. Follow the instructions here (opens in a new tab) to install it.
- To enable GPU support, you will need:
- NVIDIA GPU with CUDA Toolkit 11.7 or higher
- NVIDIA driver 470.63.01 or higher
- NVIDIA Container Toolkit (opens in a new tab)
- Install Docker Engine and Docker Compose on your Azure VM using the following command:
To install Docker Engine on Ubuntu, follow the instructions here (opens in a new tab).
curl -fsSL https://get.docker.com -o get-docker.shsudo sh ./get-docker.sh --dry-run
- Download Jan
docker-compose.yml
file onto your Azure VM using the following command:
curl https://raw.githubusercontent.com/janhq/jan/dev/docker-compose.yml -o docker-compose.yml
Step 3: Docker Configuration
Once you have installed Docker Engine and Docker Compose in your Azure VM, you need to set up the docker profile and environment variables.
The available Docker Compose profile and the environment variables are listed below:
Docker compose Profile | Description |
---|---|
cpu-fs | Run Jan in CPU mode with the default file system |
cpu-s3fs | Run Jan in CPU mode with S3 file system |
gpu-fs | Run Jan in GPU mode with the default file system |
gpu-s3fs | Run Jan in GPU mode with S3 file system |
Environment Variable | Description |
---|---|
S3_BUCKET_NAME | S3 bucket name - leave blank for default file system |
AZURE_ACCESS_KEY_ID | AZURE access key ID - leave blank for default file system |
AZURE_SECRET_ACCESS_KEY | AZURE secret access key - leave blank for default file system |
AZURE_ENDPOINT | AZURE endpoint URL - leave blank for default file system |
AZURE_REGION | AZURE region - leave blank for default file system |
API_BASE_URL | Jan Server URL, please modify it as your public IP address or domain name default http://localhost:1377 (opens in a new tab) |
Step 4: Run Jan Server
You can run the Jan server in two modes:
- CPU
- GPU
Run Jan in CPU Mode
Run Jan in CPU mode by using the following command:
# cpu mode with default file systemdocker compose --profile cpu-fs up -d# cpu mode with S3 file systemdocker compose --profile cpu-s3fs up -d
Run Jan in GPU mode
- Check CUDA compatibility with your NVIDIA driver by running
nvidia-smi
and check the CUDA version in the output:
nvidia-smi# Output+---------------------------------------------------------------------------------------+| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 ||-----------------------------------------+----------------------+----------------------+| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC || Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. || | | MIG M. ||=========================================+======================+======================|| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A || 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default || | | N/A |+-----------------------------------------+----------------------+----------------------+| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A || 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default || | | N/A |+-----------------------------------------+----------------------+----------------------+| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A || 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default || | | N/A |+-----------------------------------------+----------------------+----------------------++---------------------------------------------------------------------------------------+| Processes: || GPU GI CI PID Type Process name GPU Memory || ID ID Usage ||=======================================================================================|
-
Visit NVIDIA NGC Catalog (opens in a new tab) and find the smallest minor version of the image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)
-
Update the
Dockerfile.gpu
line number 5 with the latest minor version of the image tag from step 2 (e.g., changeFROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base
toFROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base
) -
Run the following command to start Jan in GPU mode:
# GPU mode with default file systemdocker compose --profile gpu-fs up -d# GPU mode with S3 file systemdocker compose --profile gpu-s3fs up -d
Step 5: Access the Jan Server
Once the Jan server is running on your Azure VM, you can access it using the public IP address or domain name of your VM.
- Open a web browser and navigate to the Jan Server URL, typically
http://<VM_public_IP>:3000
orhttp://<domain_name>:3000
.
RAG feature is not yet supported in Docker mode with s3fs
.