Add initial version of Aspire.Hosting.Azure.Kusto #10935

MattKotsenas · 2025-08-12T20:58:33Z

Description

This pull request introduces initial support for Kusto (Azure Data Explorer) as a Aspire Hosting integration. It adds a new Aspire.Hosting.Kusto library, and wires up the Kustainer emulator to allow for local development and testing. The resource is currently always excluded from the manifest as deployment is not yet supported.

Contributes to #8233

Checklist

Is this feature complete?
- Yes. Ready to ship.
- No. Follow-up changes expected.
  - I believe this can / should ship in the next version, but this PR does not implement any client integrations, client health checks, etc.
Are you including unit tests for the changes and scenario tests if relevant?
- Yes
- No
Did you add public API?
- Yes
  - If yes, did you have an API Review for it?
    - Yes
    - No
  - Did you add <remarks /> and <code /> elements on your triple slash comments?
    - Yes
    - No
- No
Does the change make any security assumptions or guarantees?
- Yes
  - If yes, have you done a threat model and had a security review?
    - Yes
    - No
- No
Does the change require an update in our Aspire docs?
- Yes
  - Link to aspire-docs issue (consider using one of the following templates):
- No

MattKotsenas · 2025-08-12T20:59:16Z

This is my first non-trivial contribution to Aspire, so I'm sure I've done some part of the process incorrectly. Any guidance is greatly appreciated. Thanks!

Co-authored-by: Copilot <[email protected]>

mitchdenny

This is a good first step but there are a few things that will need to be done before we could accept this. We would want this package to be named Aspire.Hosting.Azure.Kusto since Kusto is realistically an Azure resource - and when you do AddKusto(...) you would want it to deploy an actual Kusto cluster (unless you added RunAsEmulator(...)). This means we'll need to ccordinate with the Azure SDK team on getting an Azure.Provisioning package set up for Kusto.

We need to add database support as well. Also what can we do about client integration packages here. Kusto has client libraries too - we probably need to figure out how to take them and plug them into the Aspire model.

src/Aspire.Hosting.Kusto/KustoResourceBuilderExtensions.cs

src/Aspire.Hosting.Kusto/Aspire.Hosting.Kusto.csproj

mitchdenny · 2025-08-12T23:50:23Z

src/Aspire.Hosting.Kusto/KustoResourceBuilderExtensions.cs

+    /// <param name="builder">The <see cref="IDistributedApplicationBuilder"/>.</param>
+    /// <param name="name">The name of the resource. This name will be used as the connection string name when referenced in a dependency.</param>
+    /// <returns>A reference to the <see cref="IResourceBuilder{T}"/>.</returns>
+    public static IResourceBuilder<KustoResource> AddKusto(this IDistributedApplicationBuilder builder, [ResourceName] string name = "kusto")


We have this method AddKusto(...) which creates a Kusto resource, and then an extension method RunAsEmulator(...). What is the plan if they want to use a real Kusto resource deployed in Azure? We would need to add support for deploying the Kusto cluster here I think to match the pattern for other Azure based resources.

We'll probably need to work with the Azure SDK team to get the Azure.Provisioning libraries expanded to cover Kusto clusters.

Agreed. I modelled the resources so that we could add Azure Provisioning once it exists. Until then, is there a way to ship, even perhaps as a preview?

My hope is to allow folks to start using the emulator to allow for tests and local dev, while working / waiting on the other pieces.

Let me know how best to allow for working incrementally and in the open, while also following the required process 🙂

src/Aspire.Hosting.Kusto/KustoResourceBuilderExtensions.cs

mitchdenny · 2025-08-13T00:29:01Z

@tg-msft sent you a message about this PR. We are looking for Azure.Provisioning.Kusto support...

davidfowl · 2025-08-13T02:16:25Z

@mitchdenny That would be @ArcturusZhang

src/Aspire.Hosting.Azure.Kusto/Aspire.Hosting.Azure.Kusto.csproj

Co-authored-by: Copilot <[email protected]>

Copilot

Pull Request Overview

Introduces initial support for Azure Kusto (Azure Data Explorer) as an Aspire Hosting integration. This PR adds the foundational components to run Kusto as an emulator using the Kustainer container for local development, with database creation capabilities.

Key changes:

Adds new Aspire.Hosting.Azure.Kusto library with server and database resources
Implements emulator support using the Kustainer container
Includes health checks and database creation capabilities

Reviewed Changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/Aspire.Hosting.Azure.Kusto/Aspire.Hosting.Azure.Kusto.csproj	Project file for the new Azure Kusto hosting library
src/Aspire.Hosting.Azure.Kusto/KustoServerResource.cs	Core server resource definition with connection string support
src/Aspire.Hosting.Azure.Kusto/KustoDatabaseResource.cs	Database resource with parent-child relationship
src/Aspire.Hosting.Azure.Kusto/KustoResourceBuilderExtensions.cs	Extension methods for adding Kusto resources and emulator configuration
src/Aspire.Hosting.Azure.Kusto/KustoEmulatorResource.cs	Emulator-specific resource wrapper for container deployment
src/Aspire.Hosting.Azure.Kusto/KustoHealthCheck.cs	Health check implementation for Kusto connectivity
tests/Aspire.Hosting.Azure.Kusto.Tests/*.cs	Comprehensive test coverage including functional and API tests
Directory.Packages.props	Adds Microsoft.Azure.Kusto.Data package dependency
Aspire.slnx	Solution file updates to include new projects

src/Aspire.Hosting.Azure.Kusto/KustoResourceBuilderExtensions.cs

src/Aspire.Hosting.Azure.Kusto/KustoDatabaseResource.cs

mitchdenny · 2025-08-14T22:30:17Z

We should create a playground in /playground for this integration which includes usage of client libraries to ingest and query data from the emulator.

…sistence

MattKotsenas · 2025-08-14T23:42:46Z

src/Aspire.Hosting.Azure.Kusto/KustoResourceBuilderExtensions.cs

+
+        return resourceBuilder
+            .WithHealthCheck(healthCheckKey)
+            .ExcludeFromManifest();


/cc @mitchdenny flagging here as a reminder if we want this to happen automatically, or if we want users to have to specify this in the AppHost.

MattKotsenas · 2025-08-15T00:32:56Z

Added a skeleton playground that just shows getting an admin and query client. Will update to add some sample ingestion and queries tomorrow.

mitchdenny · 2025-08-15T07:26:36Z

playground/AzureKusto/AzureKusto.AppHost/Program.cs

+
+var builder = DistributedApplication.CreateBuilder(args);
+
+var kusto = builder.AddKusto("kusto")


For consistency we'll need to rename this to AddAzureKusto(...)

mitchdenny · 2025-08-15T07:29:50Z

src/Aspire.Hosting.Azure.Kusto/KustoCreateDatabaseScriptAnnotation.cs

+    /// </param>
+    public KustoCreateDatabaseScriptAnnotation(string script)
+    {
+        Script = script;


ArgumentNullException.ThrowIfNull(script);

mitchdenny · 2025-08-15T07:31:50Z

src/Aspire.Hosting.Azure.Kusto/KustoEmulatorContainerImageTags.cs

+    /// The tag for the Kusto emulator container image.
+    /// </summary>
+    /// <remarks>latest</remarks>
+    public static string Tag { get; } = "latest";


Will need to find out from the Kusto team whether it is better to use latest here or some other tag. Generally with emulators we've found teams like the use latest because it aligns to service versions (although not perfectly sometimes).

mitchdenny · 2025-08-15T07:35:31Z

src/Aspire.Hosting.Azure.Kusto/KustoServerResource.cs

+/// <summary>
+/// A resource that represents a Kusto cluster.
+/// </summary>
+public class KustoServerResource : Resource, IResourceWithConnectionString, IResourceWithEndpoints


We should rename this to AzureKustoClusterResource, and it will need to derive from AzureProvisioningResource at some point. For the database resource we should rename it to AzureKustoDatabaseResource.

mitchdenny · 2025-08-15T07:36:22Z

src/Aspire.Hosting.Azure.Kusto/KustoResourceBuilderExtensions.cs

+    /// <param name="builder">The <see cref="IDistributedApplicationBuilder"/>.</param>
+    /// <param name="name">The name of the resource. This name will be used as the connection string name when referenced in a dependency.</param>
+    /// <returns>A reference to the <see cref="IResourceBuilder{T}"/>.</returns>
+    public static IResourceBuilder<KustoServerResource> AddKusto(this IDistributedApplicationBuilder builder, [ResourceName] string name)


Maybe AddAzureKustoCluster ....

MattKotsenas added 11 commits August 11, 2025 18:54

Add Aspire.Hosting.Kusto and associated tests

dc6edef

Clean up tests to follow conventions

bba25bc

Add container tests and fix container annotation

65b9bec

Enable packaging and disable package validation

cd22a4a

Clean up connection string on surroage

4d25faa

Fix manifest exclusion test

6d8ae81

Update to latest version of Kusto.Data package

ee8819c

Update README

c0f6653

Clean up tests

a7cfd1e

Always use strong consistency when seeding to make seeding more reliable

ddfc1e6

Add additional docs

69c2284

Copilot AI review requested due to automatic review settings August 12, 2025 20:58

github-actions bot added the area-integrations Issues pertaining to Aspire Integrations packages label Aug 12, 2025

dotnet-policy-service bot added the community-contribution Indicates that the PR has been added by a community member label Aug 12, 2025

This comment was marked as outdated.

Sign in to view

Fix xmldocs from code review

38580a7

Co-authored-by: Copilot <[email protected]>

MattKotsenas requested a review from Copilot August 12, 2025 21:01

This comment was marked as outdated.

Sign in to view

Fix passing memory to container

4ccc698

mitchdenny reviewed Aug 12, 2025

View reviewed changes

MattKotsenas added 3 commits August 13, 2025 14:23

Rename Aspire.Hosting.Kusto -> Aspire.Hosting.Azure.Kusto

f350309

Do not set default resource name

7ab5891

Add KustoDatabaseResource

9735961

davidfowl changed the title ~~Add initial version of Aspire.Hosting.Kusto~~ Add initial version of Aspire.Hosting.Azure.Kusto Aug 14, 2025

mitchdenny reviewed Aug 14, 2025

View reviewed changes

src/Aspire.Hosting.Azure.Kusto/Aspire.Hosting.Azure.Kusto.csproj Show resolved Hide resolved

MattKotsenas added 2 commits August 14, 2025 13:37

Wire up KustoDatabaseResource

b899896

Clean up health check registration and add missing tests

e3602b9

Fix remarks for documentation reference

64fd559

MattKotsenas requested review from mitchdenny and Copilot August 14, 2025 21:42

This comment was marked as outdated.

Sign in to view

Fix typo in comment

3ad4543

Co-authored-by: Copilot <[email protected]>

MattKotsenas requested a review from Copilot August 14, 2025 21:43

Copilot AI reviewed Aug 14, 2025

View reviewed changes

src/Aspire.Hosting.Azure.Kusto/KustoResourceBuilderExtensions.cs Show resolved Hide resolved

src/Aspire.Hosting.Azure.Kusto/KustoDatabaseResource.cs Show resolved Hide resolved

Code review cleanup

242a7bc

Add functional test for bind mount + creation script to test data per…

1446fb0

…sistence

MattKotsenas commented Aug 14, 2025

View reviewed changes

Add skeleton playground app for Kusto

9bc3442

mitchdenny reviewed Aug 15, 2025

View reviewed changes


		var builder = DistributedApplication.CreateBuilder(args);

		var kusto = builder.AddKusto("kusto")

Add initial version of Aspire.Hosting.Azure.Kusto #10935

Are you sure you want to change the base?

Add initial version of Aspire.Hosting.Azure.Kusto #10935

Conversation

MattKotsenas commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

MattKotsenas commented Aug 12, 2025

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

mitchdenny left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mitchdenny commented Aug 13, 2025

Uh oh!

davidfowl commented Aug 13, 2025

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

mitchdenny commented Aug 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MattKotsenas commented Aug 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MattKotsenas commented Aug 12, 2025 •

edited

Loading