Documentation
Remote APIs
Triton-TRT-LLM

Triton-TRT-LLM

How to Integrate Triton-TRT-LLM with Jan

This guide provides step-by-step instructions on integrating Triton-TRT-LLM with Jan, enabling users to leverage Triton-TRT-LLM's capabilities within Jan's conversational interface.

Triton-TRT-LLM extension is only supported on Jan version 0.5.1 or later.

Before proceeding, ensure you have the following:

  • Access to the Jan application
  • Triton-TRT-LLM credentials

Integration Steps

Step 1: Configure Triton-TRT-LLM API Key

  1. Run Triton-TRT-LLM (opens in a new tab) to obtain the API Keys.
  2. Copy your Triton-TRT-LLM Key.
  3. Navigate to the Jan app > Gear Icon (⚙️) > Triton-TRT-LLM section under Model Providers.
  4. Insert your Triton-TRT-LLM API Key.

Step 2: Start Chatting with the Model

  1. Select the model you want to use.
  2. Specify the model's parameters.
  3. Start the conversation with the model.

Troubleshooting

If you encounter any issues during the integration process or while using Triton-TRT-LLM with Jan, consider the following troubleshooting steps:

  • Double-check your API credentials to ensure they are correct.
  • Check for error messages or logs that may provide insight into the issue.
  • Reach out to Triton-TRT-LLM support for assistance if needed.
NVIDIA NIM APIOverview