# SkillPatch skill: agent-platform-inference

This skill provides detailed instructions for authenticating and connecting to Google Cloud Agent Platform to perform inference with Generative AI models, including First-Party Gemini models and Third-Party OpenMaaS models (Llama, DeepSeek, Qwen, etc.). It covers SDK usage (GenAI SDK, OpenAI SDK, legacy Agent Platform SDK), environment setup, base URL configuration, and troubleshooting common errors like 429 Resource Exhausted, 400 User Validation, and 404 Not Found. It also enforces safety confirmation tiers before executing inference commands.

You (the agent) were given this URL and asked to use this skill. This is a **public** skill — no SkillPatch account, API key, or prior setup is required. Two ways to use it:

**1. Use it right now** — the skill's complete file tree (SKILL.md + all reference files) is inlined below; read `SKILL.md` first, then follow it, consulting the other files as it directs.

**2. Install the exact package onto disk** (recommended if you can run a shell — this reproduces the skill byte-for-byte, including any binary assets that can't be inlined):

```bash
mkdir -p .claude/skills/agent-platform-inference
curl -sSL https://skillpatch.dev/install_skill/agent-platform-inference | tar -xz -C .claude/skills/
```

(`.claude/skills/` is Claude Code's convention; use whatever directory your agent loads skills from.)


---

## Skill files (9)

- `SKILL.md`
- `scripts/gemini_genai_sdk.py`
- `scripts/gemini_openai_sdk.py`
- `scripts/gemini_vertexai_sdk.py`
- `scripts/openmaas_genai_sdk.py`
- `scripts/openmaas_openai_sdk.py`
- `scripts/openmaas_vertexai_sdk.py`
- `scripts/requirements.txt`
- `scripts/verify_all.sh`


### `SKILL.md`

````markdown
---
name: agent-platform-inference
metadata:
  category: AiAndMachineLearning
description: >-
  Connects to and performs inference with Google Cloud Agent Platform GenAI
  models, including First-Party Gemini models and Third-Party OpenMaaS models
  (Llama, DeepSeek, Qwen, etc.). Use when you need to generate code for calling
  Gemini or OpenMaaS models, authenticate with GenAI SDK, OpenAI SDK, or legacy
  Agent Platform SDK, configure base URLs and global/regional endpoints, or troubleshoot
  429 Resource Exhausted (DSQ), 400 User Validation, or 404 Not Found errors.
  Don't use for deploying models to endpoints or for running model evaluations.
---

# Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google
Cloud Agent Platform to use Generative AI models. It covers both First-Party
(Gemini) and Third-Party (OpenMaaS) models.

## Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere
to the following safety tiers based on the action requested. (The skill is
read-only; other safety tiers are omitted):

1.  **Tier R: Read-only / Inference (`client.models.generate_content`, `client.chat.completions.create`, `client.completions.create`, `client.embeddings.create`)**
    *   Requires **interactive confirmation** with 'Yes'/
    'No' options before executing model inference on
    behalf of the user, to prevent unexpected cost or
    quota consumption. The confirmation prompt must
    clearly explain the proposed inference execution and
    its key parameters (e.g., target model ID, SDK
    choice, input prompt). Natural-language paraphrases
    without specifying the parameters are NOT sufficient.
    *   **Same-turn restriction**: Do not execute the
    inference scripts or commands in the same turn as
    presenting the confirmation prompt. Stop and wait
    for the user's reply; only execute after explicit
    'Yes' / approval.
    *   **Gold Standard Example**:
        > I will perform model inference with the following parameters. Please
        > confirm this information before I proceed:
        > *   **Model ID**: `deepseek-ai/deepseek-v3.2-maas`
        > *   **SDK**: OpenAI SDK (via Vertex AI Endpoint)
        > *   **Input Prompt**: "Explain the concept of quantum computing..."
        > Do you confirm? [Yes/No]

## Phase 0: Environment Setup

**CRITICAL**: Before running any of the Python sample scripts in the `scripts/`
directory (e.g., `scripts/openmaas_openai_sdk.py`), you MUST ensure the
environment is correctly initialized by following these steps:

1.  **Google Cloud Authentication**: Authenticate with your Google Cloud
    credentials and configure active Application Default Credentials (ADC) for
    Agent Platform access:
    
    ```bash
    gcloud auth login
    gcloud auth application-default login
    ```
2.  **Enable API** (if not already enabled):
    
    ```bash
    gcloud services enable aiplatform.googleapis.com
    ```
3.  **Virtual Environment**: Create and activate a dedicated local virtual
    environment:
    
    ```bash
    python3 -m venv .venv
    source .venv/bin/activate
    ```
4.  **Install Dependencies**: Install the required SDKs:
    
    ```bash
    pip install -r scripts/requirements.txt
    ```
5.  **Verify Setup (Optional)**: Run all sample scripts at once to verify the
    environment is working end-to-end:
    
    ```bash
    ./scripts/verify_all.sh
    ```
6.  **Execution**: Advise the user that every time they execute a Python
    snippet from this skill, they must ensure this virtual environment is
    activated first.

<!-- disableFinding(LINE_OVER_80) -->

> [!IMPORTANT]
> **CRITICAL: Model IDs & Availability**
> *   **Gemini Models**: See [Gemini Models][gemini-models-docs] for valid
>     Model IDs and Regions.
> *   **OpenMaaS Models**: See [Use Open Models on Agent Platform]
>     (https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)
>     for Llama, DeepSeek, Qwen, etc.
> *   **Incomplete Lists**: The Model IDs listed in this skill are **examples
>     only** and may be incomplete or outdated.
> *   **Action**: Always verify the Model ID and Region using the links above
>     before generating code.
>
> [gemini-models-docs]: https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate
<!-- enableFinding(LINE_OVER_80) -->
## Workflow Decision Tree

1.  **Model Family Identification**: Has the user specified whether they want
    to call a **Gemini** (First-Party) model or an **OpenMaaS** (Third-Party,
    e.g. Llama, DeepSeek, Qwen) model?

    *   **No** -> Ask the user which model family they want to use. If they
        provide a specific model name, infer the family from the name.
    *   **Yes** -> Proceed to Step 2.

2.  **SDK Choice**: Which SDK does the user want to use?

    *   **Gemini + GenAI SDK** (preferred for Gemini) -> Proceed to
        [1. Gemini Models].
    *   **Gemini + legacy Vertex AI SDK** -> Proceed to [1. Gemini Models].
    *   **OpenMaaS + OpenAI SDK** (preferred for OpenMaaS) -> Proceed to
        [2. OpenMaaS Models].
    *   **OpenMaaS + GenAI SDK** -> Proceed to [2. OpenMaaS Models].
    *   **Unsure** -> Default to the preferred SDK for the chosen family.

3.  **Troubleshooting**: Is the user reporting an error (429 Resource
    Exhausted, 400 User Validation, 404 Not Found, etc.)?

    *   **Yes** -> Proceed to [3. Troubleshooting & Common Error Codes].
    *   **No** -> Proceed with the SDK choice from Step 2.

## 1. Gemini Models

For Gemini models (e.g., `gemini-2.5-pro`, `gemini-3-flash-preview`), the
**GenAI SDK** (`google-genai`) is the **PREFERRED** method. The legacy
`vertexai` SDK is still supported but GenAI SDK is recommended for new projects.

> [!IMPORTANT]
> **Preview Models (including Gemini 3.1)** are often **ONLY** available in the
> `global` region. Stable models are available in `us-central1` and other
> regions.

### Choosing the Right SDK

*   **Gemini Models**: **GenAI SDK** (`google-genai`) is **PREFERRED**. Use OpenAI SDK for compatibility, or Legacy SDK (`vertexai`) if needed.
*   **OpenMaaS Models**: **OpenAI SDK** is **HIGHLY RECOMMENDED**. Use GenAI SDK or Legacy SDK if you have specific infrastructure requirements.

### Installation

```bash
pip install google-genai
```

### Python Example (GenAI SDK - Preferred)

See [`scripts/gemini_genai_sdk.py`](scripts/gemini_genai_sdk.py) for the
complete code.

### Alternative: OpenAI SDK (Chat Completions)

Use the standard OpenAI SDK with the Agent Platform endpoint. This is great for
cross-compatibility.

See [`scripts/gemini_openai_sdk.py`](scripts/gemini_openai_sdk.py) for the
complete code.

### Legacy: Agent Platform SDK

The legacy `vertexai` SDK is still widely used but `google-genai` is preferred
for new Gemini projects.

See [`scripts/gemini_vertexai_sdk.py`](scripts/gemini_vertexai_sdk.py) for the
complete code.

**Documentation**: [Google GenAI SDK](https://github.com/googleapis/python-genai)

**Documentation**: [Agent Platform Gemini Models](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/google-models)

## 2. OpenMaaS Models (Llama, DeepSeek, Qwen, etc.)

For OpenMaaS (Model-as-a-Service) models, the **HIGHLY RECOMMENDED** approach is
to use the standard **OpenAI SDK** with a specific Vertex AI endpoint.

> [!WARNING]
> While `GenerativeModel` *can* support some OpenMaaS models, it is
**discouraged**. Use the OpenAI SDK for best compatibility (especially for Chat
Completions).

### Installation

```bash
pip install openai google-auth
```

### Authentication for OpenAI SDK

You **MUST** use a Google Cloud OAuth access token as the API key for the OpenAI
SDK.

```python
import google.auth
from google.auth.transport.requests import Request

def get_gcp_access_token():
    creds, _ = google.auth.default()
    creds.refresh(Request())
    return creds.token
```

> [!NOTE]
> Google Cloud access tokens typically expire after 1 hour. The
> `get_gcp_access_token()` function above retrieves a *fresh* token at the time
> it is called.
<!-- disableFinding(LINE_OVER_80) -->
> For long-running applications, you implement a refresh mechanism. See [Refresh the access token](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate/openai/auth-and-credentials?hl=en#refresh_your_credentials) for details.
<!-- enableFinding(LINE_OVER_80) -->

### Configuration (Base URL)

<!-- disableFinding(LINE_OVER_80) -->

-   **Global Endpoint** (Recommended for most models requiring global
    availability):
    `https://aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/global/endpoints/openapi`
-   **Regional Endpoint**:
    `https://{REGION}-aiplatform.googleapis.com/v1/projects/{PROJECT_ID}/locations/{REGION}/endpoints/openapi`
    <!-- enableFinding(LINE_OVER_80) -->

### Python Example (OpenMaaS - Chat Completions)

See [`scripts/openmaas_openai_sdk.py`](scripts/openmaas_openai_sdk.py) for the
complete code.

> [!TIP]
> **Alternative: Environment Variables**
> You can set environment variables in your shell instead of updating the code.
>
> ```bash
> export OPENAI_BASE_URL="https://aiplatform.googleapis.com/v1/projects/YOUR_PROJECT_ID/locations/global/endpoints/openapi"
> export OPENAI_API_KEY="$(gcloud auth print-access-token)"
> ```
> Then initialize the client without arguments: `client = OpenAI()`

### Python Example (OpenMaaS - Completions API)

The following models support the legacy Completions API: `zai-org/glm-5-maas`,
`moonshotai/kimi-k2-thinking-maas`, `minimaxai/minimax-m2-maas`,
`deepseek-ai/deepseek-v3.1-maas`, and `deepseek-ai/deepseek-v3.2-maas`.

```python
response = client.completions.create(
    model="deepseek-ai/deepseek-v3.2-maas",
    prompt="Once upon a time",
    max_tokens=100
)
print(response.choices[0].text)
```

### Python Example (OpenMaaS - Embeddings)

```python
# Verify specific Embedding Model ID on Model Garden (e.g., intfloat/multilingual-e5-small)
response = client.embeddings.create(
    model="intfloat/multilingual-e5-large-maas",
    input="The quick brown fox jumps over the lazy dog",
)
print(response.data[0].embedding)
```

### Alternative: GenAI SDK

The `google-genai` SDK can also access OpenMaaS models via the `vertexai`
backend.

See [`scripts/openmaas_genai_sdk.py`](scripts/openmaas_genai_sdk.py) for the
complete code.

> [!IMPORTANT]
> **Model ID Format**: For GenAI SDK with OpenMaaS, you **MUST** use the full
> path: `publishers/PUBLISHER/models/MODEL` (e.g.,
> `publishers/zai-org/models/glm-5-maas`).

### Legacy: Agent Platform SDK (OpenMaaS)

For OpenMaaS, you can also use `GenerativeModel` (if supported).

See [`scripts/openmaas_vertexai_sdk.py`](scripts/openmaas_vertexai_sdk.py) for
the complete code.

> [!IMPORTANT]
> **Model ID Format**: For Agent Platform SDK with OpenMaaS, you **MUST** use the
> full path: `publishers/PUBLISHER/models/MODEL`.

### Model Reference & Availability

**Documentation**: [Use Open Models on Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)

> [!TIP]
> **Self-Deployment for Control**: If you need **dedicated hardware**
> (GPUs/TPUs), **guaranteed capacity**, or **specific regional placement** not
> offered by MaaS, you can **Self-Deploy** these models to Agent Platform
> Endpoints. Search for the model in Model Garden and click "Deploy" to select
> your machine type.

> [!IMPORTANT]
> **Finding Inference Examples**: The list above is a starting point. For the
> **definitive** inference snippets (especially for Chat Completions payload
> structure):
> 1.  Consult the [Use Open Models on Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)
>     list.
> 2.  Click the link for your specific model (e.g., "DeepSeek-V3") to visit its
>     **Model Garden** page.
> 3.  Look for the **"Sample Code"** or **"Use this model"** button on the Model
>     Garden page to get the exact `curl` or Python code for that specific model
>     version.

> [!NOTE]
> This list is **INCOMPLETE**. See [Use Open Models on Agent Platform]
> (https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models)
> for the full list of supported models.

| Model Family | Model ID Examples | Location | Notes |
| :--- | :--- | :--- | :--- |
| **Llama 4** | `meta/llama-4-maverick-17b-128e-instruct-maas` | `us-east5` | |
| **Llama 4** | `meta/llama-4-scout-17b-16e-instruct-maas` | `us-east5` | |
| **Llama 3.3** | `meta/llama-3.3-70b-instruct-maas` | `us-central1` | |
| **DeepSeek** | `deepseek-ai/deepseek-v3.2-maas` | `global` | Global ONLY |
| **DeepSeek** | `deepseek-ai/deepseek-v3.1-maas` | `us-west2` | US-West2 ONLY |
| **DeepSeek** | `deepseek-ai/deepseek-r1-0528-maas` | `us-central1` | |
| **Qwen 3** | `qwen/qwen3-coder-480b-a35b-instruct-maas` | `global` | |
| **Qwen 3** | `qwen/qwen3-next-80b-a3b-instruct-maas` | `global` | |
| **Kimi** | `moonshotai/kimi-k2-thinking-maas` | `global` | |
| **MiniMax** | `minimaxai/minimax-m2-maas` | `global` | |
| **GLM** | `zai-org/glm-4.7-maas`, `zai-org/glm-5-maas` | `global` | |

## 3. Troubleshooting & Common Error Codes

### 429: Resource Exhausted

*   **Cause**: OpenMaaS and Gemini models use **Dynamic Shared Quota (DSQ)**.
    Resources are pooled and allocated dynamically based on availability. A 429
    error indicates the shared pool is temporarily exhausted, not necessarily
    that *your* specific project quota is hit (though it can be).
*   **Solution**: Implement strict **exponential backoff and retry** strategies.
*   **High Throughput**: For production workloads requiring high throughput or guaranteed capacity, consider **Provisioned Throughput (PT)**.
*   **Important**: Quota increases through normal cloud processes (Cloud Console) are **NOT** applicable for DSQ constraints.
*   **Documentation**: [Quotas and limits (DSQ)](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/quotas)

### 400: User Validation Error

*   **Cause**: Invalid request format, unsupported parameter, or incorrect Model ID.
*   **Action**: Double-check your request payload and parameters. Verify the Model ID and Region are correct.

### 404: Not Found / Model Not Available

*   **Cause**: The model is not enabled, or not available in the specified project or region.
*   **Action**:
    1.  **Check Location Availability**:
        *   **OpenMaaS**: Verify the model is available in your region. See [Model Availability by Location](https://docs.cloud.google.com/gemini-enterprise-agent-platform/resources/locations#genai-open-models).
        *   **Gemini**:
            <!-- disableFinding(LINE_OVER_80) -->
            *   **Source of Truth**: Always check [Gemini Model Locations](https://docs.cloud.google.com/gemini-enterprise-agent-platform/resources/locations#google-models) for the authoritative list.
            <!-- enableFinding(LINE_OVER_80) -->
            *   **Preview Models**: All Preview models (e.g., Gemini 3.1, experimental versions) are often **ONLY** available in the `us-central1` or `global` regions.
            *   **Stable Models**: (e.g., Gemini 2.5 Pro) Available in `us-central1`, `europe-west4`, and many other regions.
            *   **Important**: If you get a 404/400 error, try switching your client location to `us-central1` or `global`.
    2.  **Enable Llama Models**: For **Llama 3.3** and **Llama 4**, you **MUST**
        enable the model in Model Garden before use. Go to the [Model Garden]
        (https://console.cloud.google.com/agent-platform/model-garden), search
        for the model card (e.g., "Llama 3.3 API Service"), and click
        **Enable**. Only then can you make inference requests.

````


### `scripts/gemini_genai_sdk.py`

```
"""Example of using the GenAI SDK with Gemini on Vertex AI."""

from google import genai
import google.auth

# Get default project ID from environment
_, project_id = google.auth.default()

# Initialize GenAI Client with Vertex AI backend
# Use location="global" for Preview models (Gemini 2.0)
client = genai.Client(enterprise=True, project=project_id, location="us-central1")

response = client.models.generate_content(
    model="gemini-2.5-pro", contents="Why is the sky blue?"
)
print(response.text)

```


### `scripts/gemini_openai_sdk.py`

```
"""Example of using the OpenAI SDK with Gemini on Vertex AI."""

import google.auth
import google.auth.transport.requests
import openai


def get_gcp_access_token():
  creds, _ = google.auth.default()
  # Refresh credentials using a Request transport object to obtain a fresh
  # OAuth access token for the OpenAI client authorization header.
  creds.refresh(google.auth.transport.requests.Request())
  return creds.token


# Get default project ID from environment
_, project_id = google.auth.default()

client = openai.OpenAI(
    base_url=f"https://aiplatform.googleapis.com/v1/projects/{project_id}/locations/us-central1/endpoints/openapi",
    api_key=get_gcp_access_token(),
)

response = client.chat.completions.create(
    model="google/gemini-2.5-pro",
    messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(response.choices[0].message.content)

```


### `scripts/gemini_vertexai_sdk.py`

```
"""Example of using the Agent Platform SDK with Gemini on Vertex AI."""
# TODO: replace with rename of Vertex
import google.auth
import vertexai
from vertexai.generative_models import GenerativeModel

# Get default project ID from environment
_, project_id = google.auth.default()

vertexai.init(project=project_id, location="us-central1")

model = GenerativeModel("gemini-2.5-pro")
response = model.generate_content("Why is the sky blue?")
print(response.text)

```


### `scripts/openmaas_genai_sdk.py`

```
"""Example of using the GenAI SDK with OpenMaaS on Vertex AI."""

from google import genai
import google.auth

# Get default project ID from environment
_, project_id = google.auth.default()

client = genai.Client(
    enterprise=True,
    project=project_id,
    location="global",  # OpenMaaS models are often global
)

# Note: For GenAI SDK/Vertex with OpenMaaS, you MUST use the full path:
# `publishers/PUBLISHER/models/MODEL`
response = client.models.generate_content(
    model="publishers/zai-org/models/glm-5-maas",
    contents="Explain quantum computing in simple terms.",
)
print(response.text)

```


### `scripts/openmaas_openai_sdk.py`

```
"""Example of using the OpenAI SDK with OpenMaaS on Vertex AI."""

import google.auth
import google.auth.transport.requests
import openai


def get_gcp_access_token():
  creds, _ = google.auth.default()
  creds.refresh(google.auth.transport.requests.Request())
  return creds.token


# Get default project ID from environment
_, project_id = google.auth.default()

client = openai.OpenAI(
    base_url=f"https://aiplatform.googleapis.com/v1/projects/{project_id}/locations/global/endpoints/openapi",
    api_key=get_gcp_access_token(),
)

# NOTE: For OpenMaaS models, you MUST use the format: `publisher/model`
# Example: `deepseek-ai/deepseek-v3.2-maas`, `zai-org/glm-5-maas` etc.

response = client.chat.completions.create(
    model="zai-org/glm-5-maas",
    messages=[{
        "role": "user",
        "content": "Explain quantum computing in simple terms.",
    }],
)

print(response.choices[0].message.content)

```


### `scripts/openmaas_vertexai_sdk.py`

```
"""Example of using the Agent Platform SDK with OpenMaaS on Vertex AI."""
# TODO: replace with rename of Vertex
import google.auth
import vertexai
from vertexai.generative_models import GenerativeModel

# Get default project ID from environment
_, project_id = google.auth.default()

vertexai.init(project=project_id, location="global")

# Important: Use the full resource path: `publishers/PUBLISHER/models/MODEL`
model = GenerativeModel("publishers/zai-org/models/glm-5-maas")
response = model.generate_content("Explain quantum computing.")
print(response.text)

```


### `scripts/requirements.txt`

```
google-genai
google-cloud-aiplatform
openai
google-auth

```


### `scripts/verify_all.sh`

```
#!/bin/bash
set -e

# Create a temporary directory for the virtual environment
VENV_DIR=$(mktemp -d -t venv_verify.XXXXXX)
python3 -m venv "$VENV_DIR"
source "$VENV_DIR/bin/activate"

# Trap to ensure cleanup happens on exit
cleanup() {
  echo "Cleaning up virtual environment..."
  deactivate 2>/dev/null || true
  rm -rf "$VENV_DIR"
}
trap cleanup EXIT

echo "Installing requirements..."
pip install -q -r scripts/requirements.txt

echo "Running verification tests..."
FAILED=0

# Iterate directly over the files in the scripts directory
for script in scripts/*.py; do
  echo "Running $script..."
  if python3 "$script" > /dev/null 2>&1; then
    echo "  PASS: $script"
  else
    echo "  FAIL: $script"
    python3 "$script" # Run again to show output
    FAILED=1
  fi
done

if [[ $FAILED -eq 0 ]]; then
  echo "All scripts passed verification!"
  exit 0
else
  echo "Some scripts failed verification."
  exit 1
fi

```