Azure AI Foundry And OpenAI Deployments For A Chatbot App

More random rambelings trying to use Azure Cognative Account - thats what they call a deployemnt which gives access to a endpoint, the endpoint then talks to a OpenAI AI model.

In my case, I needed one Azure OpenAI resource that could do two jobs for the chatbot app: generate chat responses and create embeddings for vector search. I also wanted the deployment names to be explicit, so the app could use them from configuration without any code changes.

This is the setup I used, and it matches what is in the app today.

1. Create the Azure OpenAI resource and deployments

The Terraform for this part is straightforward. I created a cognitive account, then deployed two models: one for embeddings and one for chat.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# foundry.tf
resource "azurerm_cognitive_account" "cb-aca" {
  name                = "chatbot-openai-${var.env_id}"
  location            = azurerm_resource_group.cb-rg.location
  resource_group_name = azurerm_resource_group.cb-rg.name

  kind     = "OpenAI"
  sku_name = "S0"

  custom_subdomain_name = "chatbot-0000000001-${var.env_id}"
}

resource "azurerm_cognitive_deployment" "embeddings" {
  name                 = "embeddings"
  cognitive_account_id = azurerm_cognitive_account.cb-aca.id

  model {
    format  = "OpenAI"
    name    = "text-embedding-3-small"
    version = "1"
  }

  sku {
    name     = "GlobalStandard"
    capacity = 10
  }
}

resource "azurerm_cognitive_deployment" "chat" {
  name                 = "chat"
  cognitive_account_id = azurerm_cognitive_account.cb-aca.id
  rai_policy_name      = "Microsoft.DefaultV2"

  model {
    format  = "OpenAI"
    name    = "gpt-5"
    version = "2025-08-07"
  }

  sku {
    name     = "GlobalStandard"
    capacity = 10
  }
}

The important bit here is that the deployment names are stable. I use those exact names in configuration so the app does not need to know anything about the underlying model version.

Docs: Azure OpenAI resource • Deploy models in Azure OpenAI

2. Expose the endpoint and deployment names

Once the deployments are created, I output the endpoint and the deployment names so they can be consumed elsewhere.

1
2
3
4
5
6
7
8
9
10
11
output "openai_endpoint" {
  value = azurerm_cognitive_account.cb-aca.endpoint
}

output "chat_deployment" {
  value = azurerm_cognitive_deployment.chat.name
}

output "embedding_deployment" {
  value = azurerm_cognitive_deployment.embeddings.name
}

That gives me the values I need for the app configuration:

  • Endpoint
  • ChatDeployment
  • EmbeddingDeployment

3. Put the values into app settings

In the web app, I keep these values under an AzureOpenAI section.

1
2
3
4
5
6
7
8
9
{
  "AzureOpenAI": {
    "Endpoint": "https://chatbot-00000000001-uat.openai.azure.com/",
    "ApiKey": "<your-key>",
    "EmbeddingDeployment": "embeddings",
    "EmbeddingDimensions": 512,
    "ChatDeployment": "chat"
  }
}

The ApiKey is the secret that the app needs to authenticate to the Azure OpenAI resource. In practice, I set that from environment variables or deployment secrets rather than hard coding it.

In this app, the secret is supplied as:

1
AzureOpenAI__ApiKey

That is the value the app reads from configuration when it creates the client.

Docs: Configuration in ASP.NET Core • Azure OpenAI authentication

4. Use the chat deployment in the C# app

The chatbot service uses the configured endpoint and the chat deployment to send prompts to Azure OpenAI.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
public sealed class AzureOpenAIChatCompletionService(
    IOptions<AzureOpenAIOptions> options,
    ISystemPromptProvider systemPromptProvider,
    VectorSearchService vectorSearchService,
    ILogger<AzureOpenAIChatCompletionService> logger) : IChatCompletionService
{
    private readonly AzureOpenAIOptions _options = options.Value;

    private ChatClient CreateChatClient()
    {
        var endpoint = new Uri(_options.Endpoint);
        var credential = new AzureKeyCredential(_options.ApiKey);
        var client = new AzureOpenAIClient(endpoint, credential);
        return client.GetChatClient(_options.ChatDeployment);
    }
}

That is the critical part: the app creates an AzureOpenAIClient using the endpoint and API key, then asks for the chat client for the configured deployment name.

The same Azure OpenAI resource is also used for embeddings. The vector search layer creates an embedding client from the same endpoint and API key, but with the embedding deployment instead.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public sealed class VectorSearchService : IAsyncDisposable
{
    private readonly EmbeddingClient _embeddingClient;

    public VectorSearchService(IOptions<AzureOpenAIOptions> options, IConfiguration configuration)
    {
        var azureOpenAiOptions = options.Value;
        var client = new AzureOpenAIClient(
            new Uri(azureOpenAiOptions.Endpoint),
            new AzureKeyCredential(azureOpenAiOptions.ApiKey));

        _embeddingClient = client.GetEmbeddingClient(azureOpenAiOptions.EmbeddingDeployment);
    }
}

This is how the app turns a user query into an embedding, then compares that embedding against stored vectors in PostgreSQL using the pgvector extension.

Docs: Azure AI Inference client library • Azure.AI.OpenAI client library

6. Configure the secrets the app expects

The app expects a few Azure OpenAI values to be present before it will run the chat flow.

The relevant options class looks like this:

1
2
3
4
5
6
7
8
9
10
public sealed class AzureOpenAIOptions
{
    public const string SectionName = "AzureOpenAI";

    public string Endpoint { get; init; } = string.Empty;
    public string ApiKey { get; init; } = string.Empty;
    public string EmbeddingDeployment { get; init; } = "embeddings";
    public int EmbeddingDimensions { get; init; } = 512;
    public string ChatDeployment { get; init; } = "chat";
}

In the deployment pipeline, the secret is passed in as an environment variable so it does not live in source control.

1
AzureOpenAI__ApiKey

That is the one I would make sure is present in every environment where the app runs. Without it, the app reports that the chat service is not configured.

7. What this gives us

The result is a very simple mental model:

  1. Terraform creates the Azure OpenAI account and the two deployments.
  2. The app reads the endpoint, deployment names, and API key from configuration.
  3. The chat flow uses the chat deployment.
  4. The vector search flow uses the embedding deployment.
  5. The same Azure OpenAI resource powers both experiences.

That is the approach I used here, and it keeps the infrastructure and the application wiring aligned.

Docs: Azure OpenAI deployments • Azure AI Foundry overview