Documentation
Remote Engines
OpenRouter

OpenRouter

Integrate OpenRouter with Jan

OpenRouter (opens in a new tab) is a tool that gathers AI models. Developers can utilize its API to engage with diverse large language models, generative image models, and generative 3D object models.

To connect Jan with OpenRouter for accessing remote Large Language Models (LLMs) through OpenRouter, you can follow the steps below:

Step 1: Configure OpenRouter API Key

  1. Find your API Key in the OpenRouter API Key (opens in a new tab).
  2. Set the OpenRouter API Key in the ~jan/settings/@janhq/inference-openai-extension/settings.json file.

Step 2: Model Configuration

  1. Go to the directory ~/jan/models.
  2. Make a new folder called openrouter-(modelname), like openrouter-dolphin-mixtral-8x7b.
  3. Inside the folder, create a model.json file with the following settings:
  • Set the id property to the model ID obtained from OpenRouter.
  • Set the format property to api.
  • Set the engine property to openai.
  • Ensure the state property is set to ready.

{
"sources": [
{
"filename": "openrouter",
"url": "https://openrouter.ai/"
}
],
"id": "cognitivecomputations/dolphin-mixtral-8x7b",
"object": "model",
"name": "Dolphin 2.6 Mixtral 8x7B",
"version": "1.0",
"description": "This is a 16k context fine-tune of Mixtral-8x7b. It excels in coding tasks due to extensive training with coding data and is known for its obedience, although it lacks DPO tuning. The model is uncensored and is stripped of alignment and bias. It requires an external alignment layer for ethical use. Users are cautioned to use this highly compliant model responsibly, as detailed in a blog post about uncensored models at erichartford.com/uncensored-models.",
"format": "api",
"settings": {},
"parameters": {},
"metadata": {
"tags": ["General", "Big Context Length"]
},
"engine": "openai"
}

For more details regarding the model.json settings and parameters fields, please see here.

Step 3: Start the Model

  1. Restart Jan and go to the Hub.
  2. Find your model and click on the Use button.

Troubleshooting

If you encounter any issues during the integration process or while using OpenAI with Jan, consider the following troubleshooting steps:

  • Double-check your API credentials to ensure they are correct.
  • Check for error messages or logs that may provide insight into the issue.
  • Reach out to OpenRouter API support for assistance if needed.