# SkillPatch skill: azure-ai-projects-ts

This skill provides comprehensive guidance for building AI applications using the Azure AI Projects SDK for JavaScript/TypeScript (@azure/ai-projects). It covers authentication, client setup, and working with agents, connections, deployments, datasets, indexes, evaluations, and OpenAI clients within Azure AI Foundry projects. The skill includes detailed TypeScript code examples for common workflows like creating agents with various tools (code interpreter, file search, web search, Azure AI Search, and function tools).

You (the agent) were given this URL and asked to use this skill. This is a **public** skill — no SkillPatch account, API key, or prior setup is required. Two ways to use it:

**1. Use it right now** — the skill's complete file tree (SKILL.md + all reference files) is inlined below; read `SKILL.md` first, then follow it, consulting the other files as it directs.

**2. Install the exact package onto disk** (recommended if you can run a shell — this reproduces the skill byte-for-byte, including any binary assets that can't be inlined):

```bash
mkdir -p .claude/skills/azure-ai-projects-ts
curl -sSL https://skillpatch.dev/install_skill/azure-ai-projects-ts | tar -xz -C .claude/skills/
```

(`.claude/skills/` is Claude Code's convention; use whatever directory your agent loads skills from.)


---

## Skill files (3)

- `SKILL.md`
- `references/connections.md`
- `references/evaluations.md`


### `SKILL.md`

````markdown
---
name: azure-ai-projects-ts
description: Build AI applications using Azure AI Projects SDK for JavaScript (@azure/ai-projects). Use when working with Foundry project clients, agents, connections, deployments, datasets, indexes, evaluations, or getting OpenAI clients.
license: MIT
metadata:
  author: Microsoft
  version: "1.0.0"
  package: '@azure/ai-projects'
---

# Azure AI Projects SDK for TypeScript

High-level SDK for Azure AI Foundry projects with agents, connections, deployments, and evaluations.

## Installation

```bash
npm install @azure/ai-projects @azure/identity
```

For tracing:
```bash
npm install @azure/monitor-opentelemetry @opentelemetry/api
```

## Environment Variables

```bash
AZURE_AI_PROJECT_ENDPOINT=https://<resource>.services.ai.azure.com/api/projects/<project>
MODEL_DEPLOYMENT_NAME=gpt-4o
AZURE_TOKEN_CREDENTIALS=prod # Required only if DefaultAzureCredential is used in production
```

## Authentication

```typescript
import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential, ManagedIdentityCredential } from "@azure/identity";

// Local dev: DefaultAzureCredential. Production: set AZURE_TOKEN_CREDENTIALS=prod or AZURE_TOKEN_CREDENTIALS=<specific_credential>
const credential = new DefaultAzureCredential({requiredEnvVars: ["AZURE_TOKEN_CREDENTIALS"]});
// Or use a specific credential directly in production:
// See https://learn.microsoft.com/javascript/api/overview/azure/identity-readme?view=azure-node-latest#credential-classes
// const credential = new ManagedIdentityCredential();

const client = new AIProjectClient(
  process.env.AZURE_AI_PROJECT_ENDPOINT!,
  credential
);
```

## Operation Groups

| Group | Purpose |
|-------|---------|
| `client.agents` | Create and manage AI agents |
| `client.connections` | List connected Azure resources |
| `client.deployments` | List model deployments |
| `client.datasets` | Upload and manage datasets |
| `client.indexes` | Create and manage search indexes |
| `client.evaluators` | Manage evaluation metrics |
| `client.memoryStores` | Manage agent memory |

## Getting OpenAI Client

```typescript
const openAIClient = await client.getOpenAIClient();

// Use for responses
const response = await openAIClient.responses.create({
  model: "gpt-4o",
  input: "What is the capital of France?"
});

// Use for conversations
const conversation = await openAIClient.conversations.create({
  items: [{ type: "message", role: "user", content: "Hello!" }]
});
```

## Agents

### Create Agent

```typescript
const agent = await client.agents.createVersion("my-agent", {
  kind: "prompt",
  model: "gpt-4o",
  instructions: "You are a helpful assistant."
});
```

### Agent with Tools

```typescript
// Code Interpreter
const agent = await client.agents.createVersion("code-agent", {
  kind: "prompt",
  model: "gpt-4o",
  instructions: "You can execute code.",
  tools: [{ type: "code_interpreter", container: { type: "auto" } }]
});

// File Search
const agent = await client.agents.createVersion("search-agent", {
  kind: "prompt",
  model: "gpt-4o",
  tools: [{ type: "file_search", vector_store_ids: [vectorStoreId] }]
});

// Web Search
const agent = await client.agents.createVersion("web-agent", {
  kind: "prompt",
  model: "gpt-4o",
  tools: [{
    type: "web_search_preview",
    user_location: { type: "approximate", country: "US", city: "Seattle" }
  }]
});

// Azure AI Search
const agent = await client.agents.createVersion("aisearch-agent", {
  kind: "prompt",
  model: "gpt-4o",
  tools: [{
    type: "azure_ai_search",
    azure_ai_search: {
      indexes: [{
        project_connection_id: connectionId,
        index_name: "my-index",
        query_type: "simple"
      }]
    }
  }]
});

// Function Tool
const agent = await client.agents.createVersion("func-agent", {
  kind: "prompt",
  model: "gpt-4o",
  tools: [{
    type: "function",
    function: {
      name: "get_weather",
      description: "Get weather for a location",
      strict: true,
      parameters: {
        type: "object",
        properties: { location: { type: "string" } },
        required: ["location"]
      }
    }
  }]
});

// MCP Tool
const agent = await client.agents.createVersion("mcp-agent", {
  kind: "prompt",
  model: "gpt-4o",
  tools: [{
    type: "mcp",
    server_label: "my-mcp",
    server_url: "https://mcp-server.example.com",
    require_approval: "always"
  }]
});
```

### Run Agent

```typescript
const openAIClient = await client.getOpenAIClient();

// Create conversation
const conversation = await openAIClient.conversations.create({
  items: [{ type: "message", role: "user", content: "Hello!" }]
});

// Generate response using agent
const response = await openAIClient.responses.create(
  { conversation: conversation.id },
  { body: { agent: { name: agent.name, type: "agent_reference" } } }
);

// Cleanup
await openAIClient.conversations.delete(conversation.id);
await client.agents.deleteVersion(agent.name, agent.version);
```

## Connections

```typescript
// List all connections
for await (const conn of client.connections.list()) {
  console.log(conn.name, conn.type);
}

// Get connection by name
const conn = await client.connections.get("my-connection");

// Get connection with credentials
const connWithCreds = await client.connections.getWithCredentials("my-connection");

// Get default connection by type
const defaultAzureOpenAI = await client.connections.getDefault("AzureOpenAI", true);
```

## Deployments

```typescript
// List all deployments
for await (const deployment of client.deployments.list()) {
  if (deployment.type === "ModelDeployment") {
    console.log(deployment.name, deployment.modelName);
  }
}

// Filter by publisher
for await (const d of client.deployments.list({ modelPublisher: "OpenAI" })) {
  console.log(d.name);
}

// Get specific deployment
const deployment = await client.deployments.get("gpt-4o");
```

## Datasets

```typescript
// Upload single file
const dataset = await client.datasets.uploadFile(
  "my-dataset",
  "1.0",
  "./data/training.jsonl"
);

// Upload folder
const dataset = await client.datasets.uploadFolder(
  "my-dataset",
  "2.0",
  "./data/documents/"
);

// Get dataset
const ds = await client.datasets.get("my-dataset", "1.0");

// List versions
for await (const version of client.datasets.listVersions("my-dataset")) {
  console.log(version);
}

// Delete
await client.datasets.delete("my-dataset", "1.0");
```

## Indexes

```typescript
import { AzureAISearchIndex } from "@azure/ai-projects";

const indexConfig: AzureAISearchIndex = {
  name: "my-index",
  type: "AzureSearch",
  version: "1",
  indexName: "my-index",
  connectionName: "search-connection"
};

// Create index
const index = await client.indexes.createOrUpdate("my-index", "1", indexConfig);

// List indexes
for await (const idx of client.indexes.list()) {
  console.log(idx.name);
}

// Delete
await client.indexes.delete("my-index", "1");
```

## Key Types

```typescript
import {
  AIProjectClient,
  AIProjectClientOptionalParams,
  Connection,
  ModelDeployment,
  DatasetVersionUnion,
  AzureAISearchIndex
} from "@azure/ai-projects";
```

## Best Practices

1. **Use getOpenAIClient()** - For responses, conversations, files, and vector stores
2. **Version your agents** - Use `createVersion` for reproducible agent definitions
3. **Clean up resources** - Delete agents, conversations when done
4. **Use connections** - Get credentials from project connections, don't hardcode
5. **Filter deployments** - Use `modelPublisher` filter to find specific models

````


### `references/connections.md`

````markdown
# Connections Reference

Working with Azure AI Foundry project connections to access linked Azure resources.

## Overview

Connections represent linked Azure resources (Azure OpenAI, AI Search, Storage, etc.) configured in your Foundry project. The SDK provides methods to list, retrieve, and access credentials for these connections.

## Connection Types

| Type | Description | Use Case |
|------|-------------|----------|
| `AzureOpenAI` | Azure OpenAI Service | Chat completions, embeddings |
| `AzureAISearch` | Azure AI Search | Vector search, RAG |
| `AzureBlob` | Blob Storage | File storage for agents |
| `AzureAIServices` | Cognitive Services | Speech, Vision, etc. |
| `Custom` | Custom connections | External APIs |

## List Connections

```typescript
import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

const client = new AIProjectClient(
  process.env.AZURE_AI_PROJECT_ENDPOINT!,
  new DefaultAzureCredential()
);

// List all connections
for await (const connection of client.connections.list()) {
  console.log(`Name: ${connection.name}`);
  console.log(`Type: ${connection.type}`);
  console.log(`---`);
}

// Filter by category
for await (const conn of client.connections.list({ 
  category: "AzureOpenAI" 
})) {
  console.log(`OpenAI Connection: ${conn.name}`);
}
```

## Get Connection by Name

```typescript
// Get connection metadata (no credentials)
const connection = await client.connections.get("my-openai-connection");
console.log(`Endpoint: ${connection.target}`);
console.log(`Type: ${connection.type}`);

// Get connection with credentials
const connWithCreds = await client.connections.getWithCredentials(
  "my-openai-connection"
);

// Access credentials based on auth type
if (connWithCreds.credentials.type === "ApiKey") {
  console.log(`API Key: ${connWithCreds.credentials.key}`);
} else if (connWithCreds.credentials.type === "AAD") {
  // Use DefaultAzureCredential for AAD-based connections
  console.log("Uses Entra ID authentication");
}
```

## Get Default Connection

```typescript
// Get default connection of a specific type
const defaultOpenAI = await client.connections.getDefault(
  "AzureOpenAI",
  true // withCredentials
);

const defaultSearch = await client.connections.getDefault(
  "AzureAISearch",
  true
);

// Use the connection endpoint
console.log(`OpenAI Endpoint: ${defaultOpenAI.target}`);
console.log(`Search Endpoint: ${defaultSearch.target}`);
```

## Connection Interface

```typescript
interface Connection {
  /** Connection name */
  name: string;
  
  /** Connection type (e.g., "AzureOpenAI", "AzureAISearch") */
  type: string;
  
  /** Target endpoint URL */
  target: string;
  
  /** Authentication type */
  authType: "ApiKey" | "AAD" | "SAS" | "CustomKeys";
  
  /** Additional metadata */
  metadata?: Record<string, string>;
}

interface ConnectionWithCredentials extends Connection {
  credentials: ApiKeyCredentials | AADCredentials | SASCredentials;
}

interface ApiKeyCredentials {
  type: "ApiKey";
  key: string;
}

interface AADCredentials {
  type: "AAD";
  // Use DefaultAzureCredential to get tokens
}
```

## Using Connections with Agents

```typescript
// Get Search connection for agent tool
const searchConn = await client.connections.getWithCredentials("my-search");

// Create agent with Azure AI Search tool
const agent = await client.agents.createVersion("search-agent", {
  kind: "prompt",
  model: "gpt-4o",
  tools: [{
    type: "azure_ai_search",
    azure_ai_search: {
      indexes: [{
        project_connection_id: searchConn.name,
        index_name: "my-index",
        query_type: "vector_semantic_hybrid"
      }]
    }
  }]
});
```

## Using Connections for Direct SDK Access

```typescript
// Get Azure OpenAI connection
const openAIConn = await client.connections.getWithCredentials("my-openai");

// Create Azure OpenAI client directly
import { AzureOpenAI } from "openai";

const openAIClient = new AzureOpenAI({
  endpoint: openAIConn.target,
  apiKey: openAIConn.credentials.type === "ApiKey" 
    ? openAIConn.credentials.key 
    : undefined,
  // Or use credential for AAD
  azureADTokenProvider: openAIConn.credentials.type === "AAD"
    ? () => getAccessToken() 
    : undefined,
});

// Get AI Search connection
const searchConn = await client.connections.getWithCredentials("my-search");

// Create Search client directly
import { SearchClient, AzureKeyCredential } from "@azure/search-documents";

const searchClient = new SearchClient(
  searchConn.target,
  "my-index",
  new AzureKeyCredential(searchConn.credentials.key)
);
```

## Error Handling

```typescript
import { RestError } from "@azure/core-rest-pipeline";

try {
  const conn = await client.connections.get("non-existent");
} catch (error) {
  if (error instanceof RestError) {
    if (error.statusCode === 404) {
      console.log("Connection not found");
    } else if (error.statusCode === 403) {
      console.log("Not authorized to access connection");
    }
  }
  throw error;
}
```

## Best Practices

1. **Use `getDefault()` for standard resources** — Avoids hardcoding connection names
2. **Cache connections** — Connection metadata rarely changes; cache to reduce API calls
3. **Use AAD when possible** — Prefer `AAD` auth over `ApiKey` for better security
4. **Never log credentials** — Avoid logging `getWithCredentials()` responses
5. **Validate connection type** — Check `type` before casting credentials

## See Also

- [AIProjectClient Reference](../SKILL.md)
- [Agents with Tools](../../agents/references/tools.md)
- [Azure OpenAI Integration](https://learn.microsoft.com/azure/ai-services/openai/)

````


### `references/evaluations.md`

````markdown
# Evaluations Reference

Running AI evaluations and metrics analysis using Azure AI Foundry project SDK.

## Overview

Evaluations allow you to assess the quality of AI model outputs using various metrics like groundedness, relevance, coherence, and custom evaluators.

## Evaluator Types

| Evaluator | Measures | Use Case |
|-----------|----------|----------|
| `groundedness` | Response factual accuracy vs context | RAG applications |
| `relevance` | Response relevance to query | Search, Q&A |
| `coherence` | Response logical consistency | Content generation |
| `fluency` | Language quality | All text generation |
| `similarity` | Semantic similarity | Paraphrasing, translation |
| `f1_score` | Token overlap | Classification, NER |

## List Available Evaluators

```typescript
import { AIProjectClient } from "@azure/ai-projects";
import { DefaultAzureCredential } from "@azure/identity";

const client = new AIProjectClient(
  process.env.AZURE_AI_PROJECT_ENDPOINT!,
  new DefaultAzureCredential()
);

// List all evaluators in the project
for await (const evaluator of client.evaluators.list()) {
  console.log(`Name: ${evaluator.name}`);
  console.log(`Type: ${evaluator.type}`);
  console.log(`Description: ${evaluator.description}`);
  console.log("---");
}
```

## Run Evaluation

```typescript
// Prepare evaluation data
const evaluationData = [
  {
    query: "What is the capital of France?",
    context: "France is a country in Europe. Paris is the capital of France.",
    response: "The capital of France is Paris.",
    ground_truth: "Paris"
  },
  {
    query: "What is machine learning?",
    context: "Machine learning is a subset of AI that enables systems to learn from data.",
    response: "Machine learning is a type of AI where computers learn from data without explicit programming.",
    ground_truth: "Machine learning is a subset of AI that learns from data."
  }
];

// Run evaluation with built-in evaluators
const evaluationResult = await client.evaluations.create({
  displayName: "RAG Evaluation - v1",
  description: "Evaluating RAG pipeline quality",
  data: evaluationData,
  evaluators: {
    groundedness: {
      type: "builtin",
      name: "groundedness"
    },
    relevance: {
      type: "builtin", 
      name: "relevance"
    },
    coherence: {
      type: "builtin",
      name: "coherence"
    }
  }
});

console.log(`Evaluation ID: ${evaluationResult.id}`);
console.log(`Status: ${evaluationResult.status}`);
```

## Poll for Results

```typescript
// Poll until evaluation completes
let evaluation = await client.evaluations.get(evaluationResult.id);

while (evaluation.status === "Running" || evaluation.status === "Queued") {
  console.log(`Status: ${evaluation.status}...`);
  await new Promise(resolve => setTimeout(resolve, 5000));
  evaluation = await client.evaluations.get(evaluationResult.id);
}

if (evaluation.status === "Completed") {
  console.log("Evaluation completed!");
  console.log("Metrics:", evaluation.metrics);
} else {
  console.error("Evaluation failed:", evaluation.error);
}
```

## Access Evaluation Results

```typescript
// Get detailed results
const evaluation = await client.evaluations.get(evaluationId);

// Overall metrics
console.log("Overall Metrics:");
for (const [metric, value] of Object.entries(evaluation.metrics || {})) {
  console.log(`  ${metric}: ${value}`);
}

// Per-row results
if (evaluation.results) {
  console.log("\nPer-Row Results:");
  for (const row of evaluation.results) {
    console.log(`Query: ${row.query}`);
    console.log(`Groundedness: ${row.groundedness}`);
    console.log(`Relevance: ${row.relevance}`);
    console.log("---");
  }
}
```

## Evaluation with Dataset

```typescript
// Upload dataset first
const dataset = await client.datasets.uploadFile(
  "evaluation-data",
  "1.0",
  "./data/eval_samples.jsonl"
);

// Run evaluation on dataset
const evaluationResult = await client.evaluations.create({
  displayName: "Dataset Evaluation",
  datasetId: dataset.id,
  datasetVersion: dataset.version,
  evaluators: {
    groundedness: { type: "builtin", name: "groundedness" },
    relevance: { type: "builtin", name: "relevance" }
  },
  // Map dataset columns to evaluator inputs
  columnMapping: {
    query: "question",
    context: "retrieved_context",
    response: "model_answer",
    ground_truth: "expected_answer"
  }
});
```

## Custom Evaluator

```typescript
// Define custom evaluator with prompt template
const customEvaluator = await client.evaluators.create({
  name: "custom-toxicity",
  displayName: "Toxicity Check",
  description: "Checks response for toxic content",
  type: "prompt",
  model: "gpt-4o",
  promptTemplate: `
    You are evaluating AI response quality.
    
    Response to evaluate: {{response}}
    
    Rate the toxicity of this response on a scale of 1-5:
    1 = Not toxic at all
    5 = Highly toxic
    
    Return only the numeric score.
  `,
  outputType: "number"
});

// Use custom evaluator
const result = await client.evaluations.create({
  displayName: "Toxicity Evaluation",
  data: testData,
  evaluators: {
    toxicity: {
      type: "custom",
      id: customEvaluator.id
    }
  }
});
```

## Evaluation Interfaces

```typescript
interface EvaluationConfig {
  /** Display name for the evaluation run */
  displayName: string;
  
  /** Optional description */
  description?: string;
  
  /** Inline data to evaluate */
  data?: EvaluationRow[];
  
  /** Or reference a dataset */
  datasetId?: string;
  datasetVersion?: string;
  
  /** Evaluators to run */
  evaluators: Record<string, EvaluatorConfig>;
  
  /** Column mapping for dataset */
  columnMapping?: Record<string, string>;
}

interface EvaluatorConfig {
  type: "builtin" | "custom";
  name?: string; // For builtin
  id?: string;   // For custom
}

interface EvaluationResult {
  id: string;
  displayName: string;
  status: "Queued" | "Running" | "Completed" | "Failed";
  metrics?: Record<string, number>;
  results?: EvaluationRowResult[];
  error?: EvaluationError;
  createdAt: Date;
  completedAt?: Date;
}

interface EvaluationRowResult {
  [key: string]: unknown;
  // Contains original data plus evaluator scores
}
```

## List Evaluation Runs

```typescript
// List all evaluations
for await (const evaluation of client.evaluations.list()) {
  console.log(`${evaluation.displayName}: ${evaluation.status}`);
}

// Filter by status
for await (const evaluation of client.evaluations.list({ 
  status: "Completed" 
})) {
  console.log(`${evaluation.displayName}: ${evaluation.metrics?.groundedness}`);
}
```

## Delete Evaluation

```typescript
await client.evaluations.delete(evaluationId);
```

## Best Practices

1. **Use multiple evaluators** — Combine groundedness, relevance, and coherence for comprehensive assessment
2. **Provide ground truth** — Include expected answers for more accurate evaluation
3. **Use datasets for scale** — Upload JSONL files for large-scale evaluations
4. **Monitor costs** — Evaluations use model inference; large datasets incur costs
5. **Version evaluations** — Use descriptive names to track evaluation iterations
6. **Compare baselines** — Run evaluations on baseline vs improved models

## Common Evaluation Patterns

### RAG Quality Assessment

```typescript
const ragEvaluation = await client.evaluations.create({
  displayName: "RAG Pipeline v2",
  data: ragTestData,
  evaluators: {
    groundedness: { type: "builtin", name: "groundedness" },
    relevance: { type: "builtin", name: "relevance" },
    coherence: { type: "builtin", name: "coherence" }
  }
});
```

### A/B Model Comparison

```typescript
// Evaluate Model A
const modelAResults = await client.evaluations.create({
  displayName: "Model A Evaluation",
  data: testData.map(d => ({ ...d, response: modelAResponses[d.id] })),
  evaluators: { quality: { type: "builtin", name: "relevance" } }
});

// Evaluate Model B
const modelBResults = await client.evaluations.create({
  displayName: "Model B Evaluation", 
  data: testData.map(d => ({ ...d, response: modelBResponses[d.id] })),
  evaluators: { quality: { type: "builtin", name: "relevance" } }
});

// Compare
console.log(`Model A: ${modelAResults.metrics?.quality}`);
console.log(`Model B: ${modelBResults.metrics?.quality}`);
```

## See Also

- [Datasets Reference](./datasets.md)
- [Azure AI Evaluation](https://learn.microsoft.com/azure/ai-studio/concepts/evaluation-approach-gen-ai)
- [Built-in Evaluators](https://learn.microsoft.com/azure/ai-studio/how-to/evaluate-generative-ai-app)

````