Adding AI to .NET Apps with Microsoft.Extensions.AI, Semantic Kernel and Azure OpenAI

Almost every client conversation I have now includes AI: "Can we let users ask questions about their data?", "Can we summarise these documents automatically?", "Can support tickets be triaged by a model?" The good news for .NET teams is that you don't need to switch to Python to build these features. Modern .NET has first-class building blocks for working with large language models (LLMs). This article shows how to add AI features to an ASP.NET Core application with Microsoft.Extensions.AI, Semantic Kernel and Azure OpenAI, and the architecture decisions that matter more than the code.

The .NET AI building blocks

• Microsoft.Extensions.AI: a set of provider-neutral abstractions (IChatClient, IEmbeddingGenerator) with middleware for logging, caching, telemetry and tool calling. It's the equivalent of ILogger for AI: your code depends on the interface, and you can switch between Azure OpenAI, OpenAI, Ollama or other providers in configuration.
• Semantic Kernel: Microsoft's orchestration SDK on top of those abstractions. It adds plugins (your C# methods exposed as tools the model can call), prompt templates, memory and vector store connectors, and agent support.
• Azure OpenAI / Azure AI Foundry: hosted models with enterprise features: data stays in your Azure tenant, private networking, content filtering, managed identity and regional deployments.
• Vector stores: Azure AI Search, PostgreSQL with pgvector, Cosmos DB, Qdrant and others, used to store embeddings for retrieval.

Package names and APIs in this space still evolve quickly, so check the current versions when you start. The concepts below are stable.

Your first chat call with IChatClient

Register a chat client once, using managed identity rather than an API key:

// Packages: Azure.AI.OpenAI, Azure.Identity, Microsoft.Extensions.AI.OpenAI
builder.Services.AddChatClient(sp =>
        new AzureOpenAIClient(new Uri(builder.Configuration["AI:Endpoint"]!), new DefaultAzureCredential())
            .GetChatClient(builder.Configuration["AI:Deployment"]!)   // your model deployment name
            .AsIChatClient())
    .UseOpenTelemetry()
    .UseLogging();

Then inject IChatClient wherever you need it:

app.MapPost("/api/tickets/{id:guid}/summary", async (Guid id, IChatClient ai, AppDbContext db, CancellationToken ct) =>
{
    var ticket = await db.Tickets.AsNoTracking().FirstOrDefaultAsync(t => t.Id == id, ct);
    if (ticket is null) return Results.NotFound();

    var response = await ai.GetResponseAsync(
    [
        new(ChatRole.System, "You summarise customer support tickets in 3 bullet points for an engineer. Be factual. Do not invent details."),
        new(ChatRole.User, ticket.Body)
    ], cancellationToken: ct);

    return Results.Ok(new { summary = response.Text });
}).RequireAuthorization();

That's a production-shaped AI feature in about fifteen lines, with logging and OpenTelemetry traces included.

Getting structured data back

For classification and extraction you want typed results, not prose. Microsoft.Extensions.AI can ask the model for JSON matching a C# type:

public sealed record TicketTriage(string Category, string Priority, bool NeedsHuman, string Reason);

var triage = await ai.GetResponseAsync<TicketTriage>(
    $"Classify this support ticket. Categories: Billing, Bug, FeatureRequest, Account.\n\n{ticket.Body}",
    cancellationToken: ct);

if (triage.TryGetResult(out var result))
{
    ticket.ApplyTriage(result.Category, result.Priority, result.NeedsHuman);
}

Always validate the result before acting on it. A model can return a category that doesn't exist, so treat its output like any other untrusted input.

Letting the model call your code (tools and plugins)

The most useful AI features combine the model's language skills with your own data and business logic. With Semantic Kernel you expose ordinary C# methods as plugins:

public sealed class OrderPlugin(IOrderQueries orders, ICurrentUser currentUser)
{
    [KernelFunction, Description("Gets the status and expected delivery date of one of the current customer's orders.")]
    public async Task<OrderStatusDto?> GetOrderStatus([Description("The order number, e.g. SO-10023")] string orderNumber)
        => await orders.GetStatusForCustomerAsync(currentUser.CustomerId, orderNumber);
}

var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, new DefaultAzureCredential());
kernelBuilder.Plugins.AddFromType<OrderPlugin>();
var kernel = kernelBuilder.Build();

var settings = new OpenAIPromptExecutionSettings { FunctionChoiceBehavior = FunctionChoiceBehavior.Auto() };
var answer = await kernel.InvokePromptAsync("Where is my order SO-10023?", new(settings));

The model decides to call GetOrderStatus, Semantic Kernel executes it, and the model writes the answer using the real data. Notice that the plugin itself enforces that customers can only see their own orders. Never rely on the prompt for security.

Answering questions about your documents (RAG)

Retrieval-augmented generation (RAG) is how you make a model answer from your content, such as policies, manuals or past tickets, instead of its training data. The pipeline:

1. Ingest: split documents into chunks of a few hundred tokens, generate an embedding for each chunk with IEmbeddingGenerator, and store chunk text, embedding and metadata (source, permissions) in a vector store.
2. Retrieve: embed the user's question, run a vector (ideally hybrid vector + keyword) search, filtered by what this user is allowed to see.
3. Generate: send the top chunks to the chat model with instructions to answer only from the provided context and cite sources.

var queryEmbedding = await embeddings.GenerateVectorAsync(question, cancellationToken: ct);
var chunks = await search.FindRelevantChunksAsync(queryEmbedding, userId, top: 5, ct);   // your vector store query

var context = string.Join("\n\n", chunks.Select(c => $"[{c.Source}] {c.Text}"));
var reply = await ai.GetResponseAsync(
[
    new(ChatRole.System, "Answer only using the context below. If the answer isn't there, say you don't know. Cite sources in [brackets].\n\n" + context),
    new(ChatRole.User, question)
], cancellationToken: ct);

In my experience, the quality of a RAG feature depends mostly on chunking, metadata and search quality, not on the choice of model. Spend your time there.

Making it production-ready

• Security: treat user input as potentially malicious (prompt injection). Keep authorisation in your code and tools, never in the prompt. Don't send data to a model that the current user isn't allowed to see.
• Privacy and compliance: with Azure OpenAI your data stays in your tenant and isn't used to train models. Document what data goes to which model, and choose the region carefully for GDPR.
• Cost control: log token usage per feature and per tenant, cache repeated answers, use smaller, cheaper models for classification and summarisation, and set rate limits.
• Latency: stream responses to the UI (GetStreamingResponseAsync) so users see output immediately.
• Quality: build an evaluation set of real questions with expected answers, and run it whenever you change prompts, chunking or models. The Microsoft.Extensions.AI.Evaluation libraries help automate this.
• Resilience: models time out and get throttled. Add retries with backoff, timeouts and a graceful fallback in the UI.
• Human in the loop: for anything with consequences (refunds, emails to customers, data changes), let the model propose and a person approve.

Where to start

Pick one narrow, valuable use case with a clear way to measure success: summarising long tickets, drafting replies, extracting fields from invoices or searching internal documentation. Build it behind a feature flag, measure quality and cost with real users, then expand. That approach delivers value in weeks, while "AI strategy" projects that try to do everything at once rarely ship.

I help companies add practical AI features to their .NET products, from a first proof of concept to a secure, observable production rollout. Read more about my AI integration service or get in touch.