---
name: azure-aigateway
description: "Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway."
license: MIT
metadata:
  author: Microsoft
  version: "3.2.1"
compatibility: Requires Azure CLI (az) for configuration and testing

---

## 功能说明

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway.

## 快速安装

```bash
npx skills add https://github.com/microsoft/azure-skills --skill azure-aigateway
```

### 交给 AI 助手使用

以下是页面提示词的中文转述，命令保持原样。

```text
运行 npx skills use "https://github.com/microsoft/azure-skills" --skill "azure-aigateway"，完整阅读输出并按 Skill 说明操作。必要时先将输出保存到临时文件；相对路径以输出提供的支持文件目录为准。
```

## 补充说明

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI...

---

# Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

> **To deploy APIM**, use the **azure-prepare** skill. See [APIM deployment guide](https://learn.microsoft.com/azure/api-management/get-started-create-service-instance).

## When to Use This Skill

| Category | Triggers |
|----------|----------|
| **Model Governance** | "semantic caching", "token limits", "load balance AI", "track token usage" |
| **Tool Governance** | "rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP" |
| **Agent Governance** | "content safety", "jailbreak detection", "filter harmful content" |
| **Configuration** | "add Azure OpenAI backend", "configure my model", "add AI Foundry model" |
| **Testing** | "test AI gateway", "call OpenAI through gateway" |

---

## Quick Reference

| Policy | Purpose | Details |
|--------|---------|---------|
| `azure-openai-token-limit` | Cost control | [Model Policies](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/policies.md#token-rate-limiting) |
| `azure-openai-semantic-cache-lookup/store` | 60-80% cost savings | [Model Policies](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/policies.md#semantic-caching) |
| `azure-openai-emit-token-metric` | Observability | [Model Policies](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/policies.md#token-metrics) |
| `llm-content-safety` | Safety & compliance | [Agent Policies](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/policies.md#content-safety) |
| `rate-limit-by-key` | MCP/tool protection | [Tool Policies](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/policies.md#request-rate-limiting) |

---

## Get Gateway Details

```bash
# Get gateway URL
az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name <apim-name> --resource-group <rg> \
  --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
  --service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>
```

---

## Test AI Endpoint

```bash
GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
  -H "Content-Type: application/json" \
  -H "Ocp-Apim-Subscription-Key: <key>" \
  -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'
```

---

## Common Tasks

### Add AI Backend

See [references/patterns.md](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/patterns.md#pattern-1-add-ai-model-backend) for full steps.

```bash
# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name <apim> --resource-group <rg> \
  --backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"

# Grant access (managed identity)
az role assignment create --assignee <apim-principal-id> \
  --role "Cognitive Services User" --scope <aoai-resource-id>
```

### Apply AI Governance Policy

Recommended policy order in `<inbound>`:

1. **Authentication** - Managed identity to backend
2. **Semantic Cache Lookup** - Check cache before calling AI
3. **Token Limits** - Cost control
4. **Content Safety** - Filter harmful content
5. **Backend Selection** - Load balancing
6. **Metrics** - Token usage tracking

See [references/policies.md](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/policies.md#combining-policies) for complete example.

---

## Troubleshooting

| Issue | Solution |
|-------|----------|
| Token limit 429 | Increase `tokens-per-minute` or add load balancing |
| No cache hits | Lower `score-threshold` to 0.7 |
| Content false positives | Increase category thresholds (5-6) |
| Backend auth 401 | Grant APIM "Cognitive Services User" role |

See [references/troubleshooting.md](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/troubleshooting.md) for details.

---

## References

- [**Detailed Policies**](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/policies.md) - Full policy examples
- [**Configuration Patterns**](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/patterns.md) - Step-by-step patterns
- [**Troubleshooting**](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/troubleshooting.md) - Common issues
- [AI-Gateway Samples](https://github.com/Azure-Samples/AI-Gateway)
- [GenAI Gateway Docs](https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities)

## SDK Quick References

- **Content Safety**: [Python](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/sdk/azure-ai-contentsafety-py.md) | [TypeScript](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/sdk/azure-ai-contentsafety-ts.md)
- **API Management**: [Python](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/sdk/azure-mgmt-apimanagement-py.md) | [.NET](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/references/sdk/azure-mgmt-apimanagement-dotnet.md)

## 来源与许可

microsoft/azure-skills 提供此 Skill，适用 MIT。本版本追加了安装和访问入口，并将相对文档链接解析为 GitHub 地址；原始文件可从下方原文入口获取。

```text
MIT License

Copyright 2025 (c) Microsoft Corporation.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
```

[在 GitHub 查看原文](https://github.com/microsoft/azure-skills/blob/main/skills/azure-aigateway/SKILL.md)

[查看更多详情](https://9527.codes)
