Create an app to run management commands

  • Article

Applies to: ✅ Azure Data Explorer

In this article, you learn how to:

Prerequisites

Set up your development environment to use the Kusto client library.

Run a management command and process the results

In your preferred IDE or text editor, create a project or file named management commands using the convention appropriate for your preferred language. Then add the following code:

  1. Create a client app that connects your cluster. Replace the <your_cluster_uri> placeholder with your cluster name.

    Note

    For management commands, you'll use the CreateCslAdminProvider client factory method.

    using Kusto.Data;
using Kusto.Data.Net.Client;

namespace ManagementCommands {
  class ManagementCommands {
    static void Main(string[] args) {
      var clusterUri = "<your_cluster_uri>";
      var kcsb = new KustoConnectionStringBuilder(clusterUri)
          .WithAadUserPromptAuthentication();

      using (var kustoClient = KustoClientFactory.CreateCslAdminProvider(kcsb)) {
      }
    }
  }
}

  2. Define a function that prints the command being run and its resultant tables. This function unpacks the column names in the result tables and prints each name-value pair on a new line.

    static void PrintResultsAsValueList(string command, IDataReader response) {
  while (response.Read()) {
    Console.WriteLine("\n{0}\n", new String('-', 20));
    Console.WriteLine("Command: {0}", command);
    Console.WriteLine("Result:");
    for (int i = 0; i < response.FieldCount; i++) {
      Console.WriteLine("\t{0} - {1}", response.GetName(i), response.IsDBNull(i) ? "None" : response.GetString(i));
    }
  }
}

  3. Define the command to run. The command creates a table called MyStormEvents and defines the table schema as a list of column names and types. Replace the <your_database> placeholder with your database name.

    string database = "<your_database>";
string table = "MyStormEvents";

// Create a table named MyStormEvents
// The brackets contain a list of column Name:Type pairs that defines the table schema
string command = @$".create table {table}
                  (StartTime:datetime,
                   EndTime:datetime,
                   State:string,
                   DamageProperty:int,
                   DamageCrops:int,
                   Source:string,
                   StormSummary:dynamic)";

  4. Run the command and print the result using the previously defined function.

    Note

    You'll use the ExecuteControlCommand method to run the command.

    using (var response = kustoClient.ExecuteControlCommand(database, command, null)) {
  PrintResultsAsValueList(command, response);
}

The complete code should look like this:

using Kusto.Data;
using Kusto.Data.Net.Client;

namespace ManagementCommands {
  class ManagementCommands {
    static void Main(string[] args) {
      string clusterUri = "https://<your_cluster_uri>";
      var kcsb = new KustoConnectionStringBuilder(clusterUri)
          .WithAadUserPromptAuthentication();

      using (var kustoClient = KustoClientFactory.CreateCslAdminProvider(kcsb)) {
        string database = "<your_database>";
        string table = "MyStormEvents";

        // Create a table named MyStormEvents
        // The brackets contain a list of column Name:Type pairs that defines the table schema
        string command = @$".create table {table} 
                          (StartTime:datetime,
                           EndTime:datetime,
                           State:string,
                           DamageProperty:int,
                           DamageCrops:int,
                           Source:string,
                           StormSummary:dynamic)";

        using (var response = kustoClient.ExecuteControlCommand(database, command, null)) {
          PrintResultsAsValueList(command, response);
        }
      }
    }

    static void PrintResultsAsValueList(string command, IDataReader response) {
      while (response.Read()) {
        Console.WriteLine("\n{0}\n", new String('-', 20));
        Console.WriteLine("Command: {0}", command);
        Console.WriteLine("Result:");
        for (int i = 0; i < response.FieldCount; i++) {
          Console.WriteLine("\t{0} - {1}", response.GetName(i), response.IsDBNull(i) ? "None" : response.GetString(i));
        }
      }
    }
  }
}

Run your app

In a command shell, use the following command to run your app:

# Change directory to the folder that contains the management commands project
dotnet run .

You should see a result similar to the following:

--------------------

Command: .create table MyStormEvents 
                 (StartTime:datetime,
                  EndTime:datetime,
                  State:string,
                  DamageProperty:int,
                  Source:string,
                  StormSummary:dynamic)
Result:
   TableName - MyStormEvents
   Schema - {"Name":"MyStormEvents","OrderedColumns":[{"Name":"StartTime","Type":"System.DateTime","CslType":"datetime"},{"Name":"EndTime","Type":"System.DateTime","CslType":"datetime"},{"Name":"State","Type":"System.String","CslType":"string"},{"Name":"DamageProperty","Type":"System.Int32","CslType":"int"},{"Name":"Source","Type":"System.String","CslType":"string"},{"Name":"StormSummary","Type":"System.Object","CslType":"dynamic"}]}
   DatabaseName - MyDatabaseName
   Folder - None
   DocString - None

Change the table level ingestion batching policy

You can customize the ingestion batching behavior for tables by changing the corresponding table policy. For more information, see IngestionBatching policy.

Note

If you don't specify all parameters of a PolicyObject, the unspecified parameters will be set to default values. For example, specifying only "MaximumBatchingTimeSpan" will result in "MaximumNumberOfItems" and "MaximumRawDataSizeMB" being set to default.

For example, you can modify the app to change the ingestion batching policy timeout value to 30 seconds by altering the ingestionBatching policy for the MyStormEvents table using the following command:

// Reduce the default batching timeout to 30 seconds
command = @$".alter-merge table {table} policy ingestionbatching '{{ ""MaximumBatchingTimeSpan"":""00:00:30"" }}'";

using (var response = kustoClient.ExecuteControlCommand(database, command, null))
{
  PrintResultsAsValueList(command, response);
}

When you add the code to your app and run it, you should see a result similar to the following:

--------------------

Command: .alter-merge table MyStormEvents policy ingestionbatching '{ "MaximumBatchingTimeSpan":"00:00:30" }'
Result:
   PolicyName - IngestionBatchingPolicy
   EntityName - [YourDatabase].[MyStormEvents]
   Policy - {
  "MaximumBatchingTimeSpan": "00:00:30",
  "MaximumNumberOfItems": 500,
  "MaximumRawDataSizeMB": 1024
}
   ChildEntities - None
   EntityType - Table

Show the database level retention policy

You can use management commands to display a database's retention policy.

For example, you can modify the app to display your database's retention policy using the following code. The result is curated to project away two columns from the result:

// Show the database retention policy (drop some columns from the result)
command = @$".show database {database} policy retention | project-away ChildEntities, EntityType";

using (var response = kustoClient.ExecuteControlCommand(database, command, null)) {
  PrintResultsAsValueList(command, response);
}

When you add the code to your app and run it, you should see a result similar to the following:

--------------------

Command: .show database YourDatabase policy retention | project-away ChildEntities, EntityType
Result:
   PolicyName - RetentionPolicy
   EntityName - [YourDatabase]
   Policy - {
  "SoftDeletePeriod": "365.00:00:00",
  "Recoverability": "Enabled"
}

Next step

Create an app to get data using queued ingestion