System Center Me – Blog by Jonathan Andersson

Using parse in Azure Log Analytics to create fields in queries

This is an example of how to grab information from a string in Azure Log Analytics and create a field to be used later in a query. In the example I will show how to collect the SubjectUserName, SubjectDomainName, ObjectName and AccessMask from EventData in a SecurityEvent.

This is the query I have created with the command parse to collect the SubjectUserName, SubjectDomainName, ObjectName and AccessMask.

SecurityEvent
| parse EventData with * '<Data Name="SubjectUserName">' SubjectUserName "</Data>" *
| parse EventData with * '<Data Name="SubjectDomainName">' SubjectDomainName "</Data>" *
| parse EventData with * '<Data Name="ObjectName">' ObjectName "</Data>" *
| parse EventData with * '<Data Name="AccessMask">' AccessMask "</Data>" *
| where EventID == "4656"
| where ObjectName like "."
| where ObjectName like @"F:\shares"
| where AccessMask == "0x10080"
| sort by TimeGenerated 
| project TimeGenerated, UserName = SubjectUserName, DomainName = SubjectDomainName, File = ObjectName

SecurityEvent

| parse EventData with * '<Data Name="SubjectUserName">' SubjectUserName "</Data>" *

| parse EventData with * '<Data Name="SubjectDomainName">' SubjectDomainName "</Data>" *

| parse EventData with * '<Data Name="ObjectName">' ObjectName "</Data>" *

| parse EventData with * '<Data Name="AccessMask">' AccessMask "</Data>" *

| where EventID == "4656"

| where ObjectName like "."

| where ObjectName like @"F:\shares"

| where AccessMask == "0x10080"

| sort by TimeGenerated

| project TimeGenerated, UserName = SubjectUserName, DomainName = SubjectDomainName, File = ObjectName

Below are some EventData sample from SecurityEvent and under the sample are the results from the query.

<EventData xmlns=”http://schemas.microsoft.com/win/2004/08/events/event”> <Data Name=”SubjectUserSid”>S-1-5-21-1225449125-278733945-481909518-72603</Data> <Data Name=”SubjectUserName”>duck</Data> <Data Name=”SubjectDomainName”>CORP</Data> <Data Name=”SubjectLogonId”>0x1dac58</Data> <Data Name=”ObjectServer”>Security</Data> <Data Name=”ObjectType”>File</Data> <Data Name=”ObjectName”>F:\shares\1234\Folder2\myfile.txt</Data> <Data Name=”HandleId”>0xeac</Data> <Data Name=”TransactionId”>{00000000-0000-0000-0000-000000000000}</Data> <Data Name=”AccessList”>%%1537 %%4423 </Data> <Data Name=”AccessReason”>%%1537: %%1801 D:(A;ID;FA;;;BA) %%4423: %%1801 D:(A;ID;FA;;;BA) </Data> <Data Name=”AccessMask”>0x10080</Data> <Data Name=”PrivilegeList”>-</Data> <Data Name=”RestrictedSidCount”>0</Data> <Data Name=”ProcessId”>0x4</Data> <Data Name=”ProcessName”></Data> <Data Name=”ResourceAttributes”>-</Data> </EventData>

Results from the query.

Send Email when Azure Site Recovery is done or manual step is needed

This post is about sending an email when a Azure Site Recovery (ASR) failover is done or before a manual step in the ASR failover plan. In the example I have used an Azure Automation runbook in the ASR plan to send an email through the service SendGrid. SendGrid can off course be changed to another solution but in my case, I find it easy to use.

If you want to try it, start by creating a SendGrid account in Azure.

Make a note of the username and your password, you will need it later.

Create or import an Azure Automation Runbook that will send the email. This is the Runbook I used: SendEmail Runbook. Read the information in the description of the Runbook to get it working.

In the example Runbook above, an Azure Automation credential is needed. This is how it should look like. Add the username and password from the SendGrid account.

Edit the variables in the Runbook script and publish it.

Go to the Recovery Services vault and add the Runbook to the ASR plan.

Clone VMs after ASR Test Failover

If you want to clone a production environment on-prem to Azure and then, for example, test an upgrade or do new development on those servers, here is one way to do it.

My solution is using Azure Site Recovery (ASR) and a PowerShell script. It does not have any impact on the on-prem environment because I am using Test Failover in ASR which is starting the servers on a separate VNet in Azure which is not having any connectivity back on-prem. The Test Failover feature in ASR will make a clone of the on-prem servers in Azure and will not shut them down.

In ASR, as of today, you can only do one Test Failover at the time. This means that if you have done one Test Failover you cannot do another one while the first one is running. Because of this I am using a script in ASR to clone the Test Failover VMs so you can do more than one environment for testing.

Here is how I did it!

1. First step is to install and configure ASR to replicate the servers that is going to be cloned https://docs.microsoft.com/en-us/azure/site-recovery. When that is done, control that the servers are using Managed Disks. It they are not, change so they do.

2. Next step is to create an Azure Automation Account and a Runbook for cloning the servers. Here is the script I use: My GitHub. If you are using my script, change the variables to fit your needs.

If you have issues with the script, update the Azure Automation Account modules and import the modules that are needed. Here is a screenshot of the modules I have tested the script with.

3. When the servers have been replicated with ASR, create an ASR Recovery Plan and add the servers to a Group. Add the earlier created Runbook from Azure Automation as a Recovery Plan post step on the Group with the servers.

When this is done, do a Test Failover to test you Recovery Plan and clone of the servers.

Clone VMs in an Azure Resource Group

There are many ways to clone VMs from one Azure Resource Group to another. Here is one example that are using Azure Snapshots. The VMs, to clone, has to belong to the same Azure Region as where the copy should be created and has to use managed disks.

https://github.com/ajonathan/powershell/blob/master/AzureResourceGroupVMsCopy.ps1

To get going with the script, download or copy it from my GitHub account. It works perfect in Azure Cloud Shell for PowerShell as well as in a Azure Automation Runbook. Just remember to upgrade the modules before running it.

Here is a screenshot from a ready clone from Resource Group myvms-rg.

Use an Azure Function to add information to Log Analytics

This is an example how to add information from an Azure Function to Log Analytics with C#. I have used the C# code from the Log Analytics documentation and made some changes to fit my needs.

In this example I have chosen to use an http trigger and let the Function to take an json input. The input data is then added into the Log Analytics workspace.

Create an Azure Function as a HTTP trigger.

Paste the following code and replace the variables LogAnalyticsWorkspaceId and LogAnalyticsWorkspaceKey.

#r "Newtonsoft.Json"
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using Newtonsoft.Json;

// The input json
public class InputObject {
     public String stringValue1 {get; set;}
     public String stringValue2 {get; set;}
     public String stringValue3 {get; set;}
     public String stringValue4 {get; set;}
}

// Class for what should be sent to Log Analytics
public class InputJson
{
    public string StringField1 { get; set; }
    public string StringField2 { get; set; }
    public string StringField3 { get; set; }
    public string StringField4 { get; set; }
}

public static void Run(InputObject req, TraceWriter log)
{
    log.Info("Start function to add information to Log Analytics");

    // Update customerId to your Operations Management Suite workspace ID
    string customerId = "LogAnalyticsWorkspaceId";
    // For sharedKey, use either the primary or the secondary Connected Sources client authentication key   
    string sharedKey = "LogAnalyticsWorkspaceKey
    // LogName is name of the event type that is being submitted to Log Analytics
    string LogName = "AzureFunctionLog";

    // Creates the JSON object, with key/value pairs
    InputJson jsonObj = new InputJson();
    jsonObj.StringField1 = req?.stringValue1;
    jsonObj.StringField2 = req?.stringValue2;
    jsonObj.StringField3 = req?.stringValue3;
    jsonObj.StringField4 = req?.stringValue4;
    // Convert object to json
    var json = JsonConvert.SerializeObject(jsonObj);

    log.Info("json file sent to Log Analytics: " + json);
    
    // Create a hash for the API signature
    var datestring = DateTime.UtcNow.ToString("r");
    string stringToHash = "POST\n" + json.Length + "\napplication/json\n" + "x-ms-date:" + datestring + "\n/api/logs";
    string hashedString = BuildSignature(stringToHash, sharedKey);
    string signature = "SharedKey " + customerId + ":" + hashedString;

    PostData(signature, datestring, json, customerId, LogName);
}

// Build the API signature
public static string BuildSignature(string message, string secret)
{
    var encoding = new System.Text.ASCIIEncoding();
    byte[] keyByte = Convert.FromBase64String(secret);
    byte[] messageBytes = encoding.GetBytes(message);
    using (var hmacsha256 = new HMACSHA256(keyByte))
    {
        byte[] hash = hmacsha256.ComputeHash(messageBytes);
        return Convert.ToBase64String(hash);
    }
}

// Send a request to the POST API endpoint
public static void PostData(string signature, string date, string json, string customerId, string LogName)
{
    // You can use an optional field to specify the timestamp from the data. If the time field is not specified, Log Analytics assumes the time is the message ingestion time
    string TimeStampField = "";
    try
    {
        string url = "https://" + customerId + ".ods.opinsights.azure.com/api/logs?api-version=2016-04-01";

        System.Net.Http.HttpClient client = new System.Net.Http.HttpClient();
        client.DefaultRequestHeaders.Add("Accept", "application/json");
        client.DefaultRequestHeaders.Add("Log-Type", LogName);
        client.DefaultRequestHeaders.Add("Authorization", signature);
        client.DefaultRequestHeaders.Add("x-ms-date", date);
        client.DefaultRequestHeaders.Add("time-generated-field", TimeStampField);

        System.Net.Http.HttpContent httpContent = new StringContent(json, Encoding.UTF8);
        httpContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
        Task<System.Net.Http.HttpResponseMessage> response = client.PostAsync(new Uri(url), httpContent);

        System.Net.Http.HttpContent responseContent = response.Result.Content;
        string result = responseContent.ReadAsStringAsync().Result;
        Console.WriteLine("Return Result: " + result);
    }
    catch (Exception excep)
    {
        Console.WriteLine("API Post Exception: " + excep.Message);
    }
}

100

#r "Newtonsoft.Json"

using System;

using System.Net;

using System.Net.Http;

using System.Net.Http.Headers;

using System.Security.Cryptography;

using System.Text;

using System.Threading.Tasks;

using Newtonsoft.Json;

// The input json

public class InputObject {

public String stringValue1 {get; set;}

public String stringValue2 {get; set;}

public String stringValue3 {get; set;}

public String stringValue4 {get; set;}

}

// Class for what should be sent to Log Analytics

public class InputJson

{

public string StringField1 { get; set; }

public string StringField2 { get; set; }

public string StringField3 { get; set; }

public string StringField4 { get; set; }

}

public static void Run(InputObject req, TraceWriter log)

{

log.Info("Start function to add information to Log Analytics");

// Update customerId to your Operations Management Suite workspace ID

string customerId = "LogAnalyticsWorkspaceId";

// For sharedKey, use either the primary or the secondary Connected Sources client authentication key

string sharedKey = "LogAnalyticsWorkspaceKey

// LogName is name of the event type that is being submitted to Log Analytics

string LogName = "AzureFunctionLog";

// Creates the JSON object, with key/value pairs

InputJson jsonObj = new InputJson();

jsonObj.StringField1 = req?.stringValue1;

jsonObj.StringField2 = req?.stringValue2;

jsonObj.StringField3 = req?.stringValue3;

jsonObj.StringField4 = req?.stringValue4;

// Convert object to json

var json = JsonConvert.SerializeObject(jsonObj);

log.Info("json file sent to Log Analytics: " + json);

// Create a hash for the API signature

var datestring = DateTime.UtcNow.ToString("r");

string stringToHash = "POST\n" + json.Length + "\napplication/json\n" + "x-ms-date:" + datestring + "\n/api/logs";

string hashedString = BuildSignature(stringToHash, sharedKey);

string signature = "SharedKey " + customerId + ":" + hashedString;

PostData(signature, datestring, json, customerId, LogName);

}

// Build the API signature

public static string BuildSignature(string message, string secret)

{

var encoding = new System.Text.ASCIIEncoding();

byte[] keyByte = Convert.FromBase64String(secret);

byte[] messageBytes = encoding.GetBytes(message);

using (var hmacsha256 = new HMACSHA256(keyByte))

{

byte[] hash = hmacsha256.ComputeHash(messageBytes);

return Convert.ToBase64String(hash);

}

// Send a request to the POST API endpoint

public static void PostData(string signature, string date, string json, string customerId, string LogName)

{

// You can use an optional field to specify the timestamp from the data. If the time field is not specified, Log Analytics assumes the time is the message ingestion time

string TimeStampField = "";

try

{

string url = "https://" + customerId + ".ods.opinsights.azure.com/api/logs?api-version=2016-04-01";

System.Net.Http.HttpClient client = new System.Net.Http.HttpClient();

client.DefaultRequestHeaders.Add("Accept", "application/json");

client.DefaultRequestHeaders.Add("Log-Type", LogName);

client.DefaultRequestHeaders.Add("Authorization", signature);

client.DefaultRequestHeaders.Add("x-ms-date", date);

client.DefaultRequestHeaders.Add("time-generated-field", TimeStampField);

System.Net.Http.HttpContent httpContent = new StringContent(json, Encoding.UTF8);

httpContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");

Task<System.Net.Http.HttpResponseMessage> response = client.PostAsync(new Uri(url), httpContent);

System.Net.Http.HttpContent responseContent = response.Result.Content;

string result = responseContent.ReadAsStringAsync().Result;

Console.WriteLine("Return Result: " + result);

}

catch (Exception excep)

{

Console.WriteLine("API Post Exception: " + excep.Message);

}

Add test json code in the “Request body” and test the Function by clicking Run.

{
    "stringValue1": "Azure1",
    "stringValue2": "Azure2",
    "stringValue3": "Azure3",
    "stringValue4": "Azure4"
}

{

"stringValue1": "Azure1",

"stringValue2": "Azure2",

"stringValue3": "Azure3",

"stringValue4": "Azure4"

}