Documentation
Server Installation
GCP

GCP Installation

To install Jan Server, follow the steps below:

Step 1: Prepare Environment

  1. Go to GCP console -> Compute instance.
  2. Choose an instance with at least c2-standard-8 for CPU only or g2-standard-4 for NVIDIA GPU support.
  3. Add a Persistent Disk volume with at least 100GB.
  4. Configure network security group rules to allow inbound traffic on port 1377.

Step 2: Get Jan Server

  1. Before installing the Jan server, ensure that you have the following requirements:
  1. Install Docker Engine and Docker Compose on your GCP Instance using the following command:

To install Docker Engine on Ubuntu, follow the instructions here (opens in a new tab).


curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh ./get-docker.sh --dry-run

  1. Download Jan docker-compose.yml file onto your GCP Instance using the following command:

curl https://raw.githubusercontent.com/janhq/jan/dev/docker-compose.yml -o docker-compose.yml

Step 3: Docker Configuration

Once you have installed Docker Engine and Docker Compose in your GCP Instance, you need to set up the docker profile and environment variables.

The available Docker Compose profile and the environment variables are listed below:

Docker compose ProfileDescription
cpu-fsRun Jan in CPU mode with the default file system
cpu-s3fsRun Jan in CPU mode with S3 file system
gpu-fsRun Jan in GPU mode with the default file system
gpu-s3fsRun Jan in GPU mode with S3 file system
Environment VariableDescription
S3_BUCKET_NAMES3 bucket name - leave blank for default file system
GCP_ACCESS_KEY_IDGCP access key ID - leave blank for default file system
GCP_SECRET_ACCESS_KEYGCP secret access key - leave blank for default file system
GCP_ENDPOINTGCP endpoint URL - leave blank for default file system
GCP_REGIONGCP region - leave blank for default file system
API_BASE_URLJan Server URL, please modify it as your public IP address or domain name default http://localhost:1377 (opens in a new tab)

Step 4: Run Jan Server

You can run the Jan server in two modes:

  • CPU
  • GPU

Run Jan in CPU Mode

Run Jan in CPU mode by using the following command:


# cpu mode with default file system
docker compose --profile cpu-fs up -d
# cpu mode with S3 file system
docker compose --profile cpu-s3fs up -d

Run Jan in GPU mode

  1. Check CUDA compatibility with your NVIDIA driver by running nvidia-smi and check the CUDA version in the output:

nvidia-smi
# Output
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 531.18 Driver Version: 531.18 CUDA Version: 12.1 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 4070 Ti WDDM | 00000000:01:00.0 On | N/A |
| 0% 44C P8 16W / 285W| 1481MiB / 12282MiB | 2% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 1 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:02:00.0 Off | N/A |
| 0% 49C P8 14W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
| 2 NVIDIA GeForce GTX 1660 Ti WDDM | 00000000:05:00.0 Off | N/A |
| 29% 38C P8 11W / 120W| 0MiB / 6144MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|

  1. Visit NVIDIA NGC Catalog (opens in a new tab) and find the smallest minor version of the image tag that matches your CUDA version (e.g., 12.1 -> 12.1.0)

  2. Update the Dockerfile.gpu line number 5 with the latest minor version of the image tag from step 2 (e.g. change FROM nvidia/cuda:12.2.0-runtime-ubuntu22.04 AS base to FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base)

  3. Run the following command to start Jan in GPU mode:


# GPU mode with default file system
docker compose --profile gpu-fs up -d
# GPU mode with S3 file system
docker compose --profile gpu-s3fs up -d

Step 5: Access the Jan Server

Once the Jan server runs on your GCP Instance, you can access it using your Instance's public IP address or domain name.

  1. Open a web browser and navigate to the Jan Server URL, typically http://<INSTANCE_public_IP>:3000 or http://<domain_name>:3000.

RAG feature is not yet supported in Docker mode with s3fs.

AWSAzure