Model Providers
20+ supported LLM providers with streaming, tool calling, and extended reasoning
k8s-autopilot works with 20+ model providers through LangChain's provider abstraction. You can switch models mid-conversation without restarting, and the agent automatically detects the right provider from the model name and available API keys.
Model Format
Models are referenced using the standard provider:model format:
google_genai:gemini-3.7-flash
anthropic:claude-opus-4-7
openai:gpt-5.4
deepseek:deepseek-reasoner
ollama:llama3.1
You can set the model in several ways:
- UI Model Picker: Click the model selector in the chat interface to switch instantly.
- Settings UI: Go to Settings → Auth & Keys to add provider keys, then select models directly in chat.
- Environment Variable: Set
MODEL=gemini-3.7-flashin your.envfile. - Provider + Model: Set both
MODELandMODEL_PROVIDERfor explicit control.
If you just set the model name without specifying a provider, k8s-autopilot auto-detects the provider. For example, MODEL=gpt-4o automatically routes to OpenAI, and MODEL=gemini-3.7-flash routes to Google GenAI.
Supported Providers
| Provider | ID | Auth Env Var | Highlights |
|---|---|---|---|
| Google GenAI | google_genai | GOOGLE_API_KEY | Gemini 3.8/3.7/3.6/3.5 Flash, Gemini 3.1 Pro, 1M context, Native Thinking |
| Anthropic | anthropic | ANTHROPIC_API_KEY | Claude 3.7 Sonnet, Claude Opus 4.7/5, Extended Thinking budgets, Vision |
| OpenAI | openai | OPENAI_API_KEY | GPT-5.5, GPT-5.4, o3/o3-mini Reasoning, Vision, Tool calling |
| Google Vertex AI | google_vertexai | ADC (GOOGLE_CLOUD_PROJECT) | Enterprise Vertex AI endpoints with Workload Identity |
| Azure OpenAI | azure_openai | AZURE_OPENAI_API_KEY | Sovereign enterprise Azure-hosted OpenAI models |
| AWS Bedrock | bedrock | AWS IAM Roles / ADC | Anthropic Claude and Amazon Nova via AWS private endpoints |
| Groq | groq | GROQ_API_KEY | Ultra-low latency LPU inference with Llama 3.3 70B |
| DeepSeek | deepseek | DEEPSEEK_API_KEY | DeepSeek V3 and DeepSeek R1 reasoning models |
| Together AI | together | TOGETHER_API_KEY | Open-source foundation models hosted on fast clusters |
| Fireworks AI | fireworks | FIREWORKS_API_KEY | High-speed function calling and quantized open weights |
| OpenRouter | openrouter | OPENROUTER_API_KEY | Multi-provider routing gateway (Claude, Gemini, Kimi K3) |
| Mistral AI | mistralai | MISTRAL_API_KEY | Mistral Large, Codestral, Pixtral |
| NVIDIA NIM | nvidia | NVIDIA_API_KEY | Accelerated NIM microservice endpoints |
| Perplexity | perplexity | PPLX_API_KEY | Search-augmented Sonar models |
| Cohere | cohere | COHERE_API_KEY | Command R / Command R+ |
| IBM watsonx | ibm | WATSONX_APIKEY | Enterprise Granite and Llama deployments |
| HuggingFace | huggingface | HUGGINGFACEHUB_API_TOKEN | Dedicated Inference Endpoints |
| LiteLLM | litellm | LITELLM_API_KEY | Unified proxy for internal enterprise gateways |
| xAI | xai | XAI_API_KEY | Grok 4.5 and Grok 2 models |
| Baseten | baseten | BASETEN_API_KEY | Custom deployed foundation models |
| Ollama | ollama | Optional / None | 100% offline local inference on CPU or GPU |
Special Authentication Cases
- Google Vertex AI and AWS Bedrock use implicit cloud authentication (Application Default Credentials or IAM Role bindings). No explicit API key is needed.
- Ollama requires no API key at all. Point the agent to the Ollama endpoint and go.
Auto-Detection Rules
When you set MODEL without specifying MODEL_PROVIDER, the agent infers the provider from the model name:
| Model Name Prefix | Detected Provider |
|---|---|
gpt-, o1, o3, o4, chatgpt | openai |
claude-, sonnet, opus, haiku | anthropic |
gemini | google_genai |
deepseek | deepseek |
llama | ollama |
If the name doesn't match any prefix, the agent checks which providers have API keys configured, using this priority order:
Google GenAI ──► Anthropic ──► OpenAI ──► Groq ──► DeepSeek ──► OpenRouter
Popular Models
These models have specialized capability and token profiles built into k8s-autopilot:
| Model | Provider | Context Window | Reasoning Controls | Tool Calling |
|---|---|---|---|---|
| Gemini 3.8 Flash | google_genai / google_vertexai | 1M tokens | ✅ low / medium / high | ✅ |
| Gemini 3.7 Flash | google_genai / google_vertexai | 1M tokens | ✅ low / medium / high | ✅ |
| Gemini 3.6 Flash | google_genai | 1M tokens | ✅ low / medium / high | ✅ |
| Gemini 3.5 Flash | google_genai | 1M tokens | ✅ low / medium / high | ✅ |
| Gemini 3.1 Pro | google_genai | 1M tokens | ✅ low / high | ✅ |
| Gemini 2.5 Pro | google_genai | 1M tokens | ✅ low / medium / high | ✅ |
| Claude Opus 4.7 / 4.8 / 5 | anthropic | 200K tokens | ✅ low / med / high / max | ✅ |
| Claude 3.7 Sonnet | anthropic | 200K tokens | ✅ low / med / high / max | ✅ |
| Claude Sonnet 4.5 / 4.6 / 5 | anthropic | 200K tokens | ✅ low / med / high / max | ✅ |
| Claude Haiku 4.5 | anthropic | 200K tokens | — | ✅ |
| Claude 3.5 Sonnet | anthropic | 200K tokens | — | ✅ |
| GPT-5.5 / 5.5 Pro | openai | 400K tokens | ✅ low / medium / high | ✅ |
| GPT-5.4 / 5.4 mini | openai | 400K tokens | ✅ low / medium / high | ✅ |
| o3 / o3-mini | openai | 200K tokens | ✅ low / medium / high | ✅ |
| GPT-4o / 4o mini | openai | 128K tokens | — | ✅ |
| DeepSeek V3 | deepseek | 64K tokens | — | ✅ |
| DeepSeek R1 | deepseek | 64K tokens | ✅ | ✅ |
| GLM 5.2 | fireworks / baseten / openrouter | 128K tokens | ✅ | ✅ |
| Grok 4.5 / Grok 2 | xai | 131K tokens | ✅ | ✅ |
| Llama 3.3 70B | groq / together | 128K tokens | — | ✅ |
| Kimi K3 / K2.7 | openrouter / baseten | 1M tokens | ✅ low / medium / high | ✅ |
Any model from any supported provider works — unlisted models simply use safe framework defaults.
Extended Thinking / Reasoning
k8s-autopilot supports extended reasoning for models with native thinking capabilities. The reasoning effort controls how much thinking the model performs before generating output — higher effort yields more thorough operational diagnostics but consumes more tokens.
Setting the Effort
From .env:
REASONING_EFFORT=high # Options: low, medium, high, max
From the UI: Go to Settings → Runtime Config to change the reasoning effort.
How It Works Per Provider
The REASONING_EFFORT setting is translated into each provider's native parameters:
| Provider | Native Parameter | Example for high |
|---|---|---|
| Google GenAI | thinking_level, thinking_budget | thinking_level="HIGH", thinking_budget=8192 |
| Anthropic | thinking.type, thinking.budget_tokens | thinking={"type": "enabled", "budget_tokens": 8192} |
| OpenAI | reasoning.effort | reasoning={"effort": "high"} |
Claude models support max effort while most other providers cap at high. The agent automatically clamps to the nearest supported level.
Custom Base URLs
For self-hosted, proxied, or enterprise deployments, you can override the API endpoint for any provider:
| Provider | Base URL Variable(s) |
|---|---|
| OpenAI | OPENAI_BASE_URL, OPENAI_API_BASE |
| Anthropic | ANTHROPIC_BASE_URL, ANTHROPIC_API_URL |
| Google GenAI | GOOGLE_GEMINI_BASE_URL |
| Azure OpenAI | AZURE_OPENAI_ENDPOINT |
| Groq | GROQ_BASE_URL, GROQ_API_BASE |
| DeepSeek | DEEPSEEK_API_BASE |
| Fireworks | FIREWORKS_BASE_URL, FIREWORKS_API_BASE |
| Together | TOGETHER_API_BASE |
| OpenRouter | OPENROUTER_API_BASE |
| Mistral | MISTRAL_BASE_URL |
| NVIDIA | NVIDIA_BASE_URL |
| Perplexity | PERPLEXITY_BASE_URL |
| xAI | XAI_API_BASE |
| Baseten | BASETEN_BASE_URL, BASETEN_API_BASE |
| Cohere | CO_API_URL |
| HuggingFace | HF_INFERENCE_ENDPOINT |
| IBM watsonx | WATSONX_URL |
You can also set custom base URLs per provider directly from the UI by expanding Advanced Settings in the API key modal.
Google Vertex AI
To use Google's enterprise Vertex AI instead of Google AI Studio:
GOOGLE_GENAI_USE_VERTEXAI=true
GOOGLE_CLOUD_PROJECT=your-gcp-project-id
GOOGLE_CLOUD_LOCATION=us-central1
Authentication uses Application Default Credentials (ADC). Ensure you have run gcloud auth application-default login or configured a service account key.
Local Models with Ollama
Run 100% offline and air-gapped with no API keys:
# Pull a model locally
ollama pull llama3.1
# Configure k8s-autopilot to use it
MODEL=llama3.1
MODEL_PROVIDER=ollama
By default, k8s-autopilot connects to Ollama at http://localhost:11434. To target a remote Ollama server, set the OLLAMA_HOST environment variable or configure it in Settings → Auth & Keys → Ollama → Advanced Settings.
Tool calling and structured outputs work best with models of 13B parameters or larger. Smaller models may struggle with complex multi-step plans.
Next Steps
- Configuration Reference — Full settings reference including all env vars.
- Quickstart Guide — First-time setup and connecting your first provider.
- Approval Modes & Governance — Control agent autonomy across models.