Groq

Ultra-fast inference with Groq’s LPU™ Inference Engine

Access ultra-fast inference powered by Groq’s LPU™ (Language Processing Unit) Inference Engine for various open-source models.

Configuration

Add to ~/.factory/settings.json:

{
  "customModels": [
    {
      "model": "moonshotai/kimi-k2-instruct-0905",
      "displayName": "Kimi K2 [Groq]",
      "baseUrl": "https://api.groq.com/openai/v1",
      "apiKey": "YOUR_GROQ_KEY",
      "provider": "generic-chat-completion-api",
      "maxOutputTokens": 16384
    }
  ]
}

Getting Started

1. Sign up at groq.com
2. Get your API key from console.groq.com
3. Browse available models in the Groq documentation
4. Add desired models to your configuration

Notes

• Base URL format: https://api.groq.com/openai/v1
• Groq uses the generic-chat-completion-api provider type
• Known for extremely fast inference speeds thanks to LPU architecture
• Supports streaming responses for compatible models