Loading...

Azure VMware Solution - Using Log Analytics With NSX-T Firewall Logs

Azure VMware Solution - Using Log Analytics With NSX-T Firewall Logs

Azure VMware Solution

How To Series: Monitoring Azure VMware Solution

 

Overview

Requirements

Lab Environment

Tagging & Groups

Kusto for NSX-T Logs

 

Overview

Many Azure VMware Solution customers are new to NSX-T.  This blog uses an example to show how to configure NSX-T firewall rules and then use Log Analytics for troubleshooting and alerting. The focus is on restricting NSX-T East-West traffic by creating tags and groups and using these in the development of rules that we can then easily query in Azure Log Analytics.

 

Requirements

  • Access to the NSX-T console, in this case we use the default cloudadmin account.
  • A log analytics workspace to store the logs.
  • The diagnostics settings enabled for Azure VMware Solution and configured with the log analytics workspace name and 'All logs' selected.  (Audit logs are the actions that privileged users take on the Azure VMware Solution private cloud).

MarkGodfrey_1-1714761665853.png

Figure 1: Azure VMware Solution Diagnostics Settings

 

Lab Environment

The routes between vLAN segments in the Azure VMware Solution (AVS) are by default open; both between each other (East – West) and for traffic into and out of AVS.   Depending on your approach, you can either set rules to 'Allow', 'Drop', or 'Reject' (application layer).  Once the rule set is built the default layer 3 rule will be changed to 'Drop'.

 

The lab environment consists of 4 network segments. The network segments VLAN-1 and VLAN-2 contain the web app, segment VLAN-3 contains the application servers, and segment VLAN-4 contains the backend servers. The NSX-T Firewall will be configured to allow traffic from servers in vLAN-1 and vLAN-2 to servers in vLAN-3.  vLAN-3 must be able to connect to vLAN-4 and not be able to connect and servers in vLAN-1 or vLAN-2.  vLAN-4 should not be able to connect to vLAN-1, vLAN-2 or vLAN3.  .

MarkGodfrey_5-1714763708220.png

Figure 2:  NSX-T logging lab environment

 

The firewall traffic matrix looks like this:

Segment

AVS-VLAN-1

AVS-VLAN-2

AVS-VLAN-3

AVS-VLAN-4

AVS-VLAN-1

n/a

Block

Allow

Block

AVS-VLAN-2

Block

n/a

Allow

Block

AVS-VLAN-3

Block

Block

n/a

Allow

AVS-VLAN-4

Block Block

Block

n/a

 

Tagging & Groups

Step 1: Create tags for each of the servers

Login to the NSX-T console and click on Inventory then select Tags, and then ADD TAG. Enter the name of the server ‘AppServer-01’  (we are keeping it simple here), set the Scope as ‘Server Type’  and then click on ‘Set Virtual Machines’.

Note: ‘Tag scope’ is not required for this demo.  This field is analogous to a key and ‘tag name is analogous to a value’. For example, let us say, you want to label all virtual machines based on their operating system (Windows, Mac, Linux). You can create three tags, such as Windows, Linux, and Mac, and set the scope of each tag to OS. Other examples of tag scope can be tenant, owner, name, and so on.  Finally click on the 'Set Virtual Machines' and add the tag to a specific machine.  Then click Save.

MarkGodfrey_6-1714764575181.png

Figure 3:  Adding tags to servers

 

For example, click-on ‘Set Virtual Machines’ , select the virtual machine AppServer-01, then click ‘APPLY’ in the bottom right hand corner.

 

MarkGodfrey_8-1714765207643.pngFigure 4: Attaching a tag to a virtual machine

 

Now click on Save to complete the process to attach a tag to a virtual machine.

 

MarkGodfrey_9-1714765427845.png

Figure 5: Completing the tag attach to a virtual machine.

 

The process is repeated for the other 5 machines so we end up with the following.

MarkGodfrey_3-1714837189621.png

Figure 6:  Tags created and assigned to servers

 

Step 2: Create Groups based on tags

Select Inventory then select Groups and click on ‘ADD GROUP’.  Create a group with the name  “WebServers” and a description  “Contains servers from vLAN-1 and vLAN-2” , then click on ‘Set’.

 

MarkGodfrey_4-1714837656587.png

Figure 7: Creating groups

 

In the Set Members page, leave the ‘Select Group Type’ as Generic and click on “+ Add Criterion” and select ‘Tag’ , the select 'Contains' and then enter Web.  Next click on 'APPLY'.

 

MarkGodfrey_0-1714837917490.png

Figure 8:  Creating Groups based on server the server Tag

 

Repeat the process for the AppServers and BackEndServers.  Once complete the following groups should exist.

Note: Use the 'View Members' link to check that each group contains the correct servers.

MarkGodfrey_5-1714838637888.png

Figure 9: Groups containing relevant servers.

 

Step 3: Create rules based on groups

Click on ‘Security’ and ‘Distributed Firewall’,  'Category Specific Rules' , select the 'Infrastructure' tab, and then “ADD POLICY”.    A ‘New Policy’ will appear as a new line, rename the policy My-App-Policy, then click on the 3 ellipses, and select ‘Add Rule’.

MarkGodfrey_3-1714839939085.png

MarkGodfrey_1-1714839658376.png

Figure 10: Add a new policy and add a rule

 

Rename the New Rule to ‘Allow-WebServers-To-AppServers’ , then fill in the Sources ‘Webservers’, Destination ‘AppServers’ and Services ‘HTTPS’ + ICMPv4-All’ (ICMP is just for testing).  Leave the Action as ‘Allow’ , the click on the gear icon on the far right.

 

MarkGodfrey_4-1714839951998.png

Figure 11:  Creating a new rule under the My-App-Policy

 

In the rule Setting screen, enable logging, and enter the name of the rule in the log label.  This is the label we will see later in the Log Analytics results when we run a Kusto query.

MarkGodfrey_5-1714840125254.png

Figure 12:  Setting the Log Label for the rule.

 

Repeat the process for each rule required to meet the connectivity matrix requirements defined above, be sure to add name of the rule to the Log Label.  The end result should look like the following.  Note that for the WebServers-to-BackEndServers the action has been set to 'Reject',  the 'Drop' option can also be used.  There is a difference in the ping response.

Click on 'PUBLISH' to write the policy and it's rule set.

MarkGodfrey_2-1714840478609.png

Figure 13: Final rule set configuration

 

Switch to the APPLICATION tab and open up the Default Layer 3 Section.  Note this rule is always on, so when the NSX-T is first installed, the default is to allow all layer 3 traffic.  Switch the Action to 'Drop' and then click on 'PUBLISH' to update the policy.

 

MarkGodfrey_3-1714845771637.png

Figure 14: Changing the default Application layer rule to drop.

 

Step 3: Testing Connectivity

Now test the connections between the servers to ensure that connectivity is behaving as expected.

WebServers servers to other servers.  Note that each machine in this case has the following Hosts file additions, rather than using specific IP addresses.

 

Host File

192.168.101.10     webserver-01
192.168.102.10     webserver-02

192.168.103.13     appserver-01
192.168.103.17     appserver-02

192.168.104.13     backend-01
192.168.104.17     backend-02

Webserver-01 to WebServer-02, AppServers and to BackEndServers groups.

MarkGodfrey_1-1714847606627.png

Figure 15:  Expected behavior of traffic from WebServer-01 (Note: Webserver-01 and -02 are on different segments)

 

AppServer-01 to WebServer-01 and to BackEndServers group

MarkGodfrey_2-1714848176478.png

Figure 16:  Expected behavior from AppServer-01 to WebServers and BackEndServers groups


BackEndServer-01 to WebServers and BackEndServers groups

MarkGodfrey_3-1714848541878.png

Figure 17:  Expected behavior from the BackEndServer-01 to the WebServers and AppServers groups

 

 

Kusto for NSX-T Logs

Open up the log analytics space selected in step 1 and add the following Kusto query to a new tab.

 

AVSSyslog
| where AppName == "FIREWALL"
| where Message contains "192.168.101.10" and Message contains "PASS"
| sort by TimeGenerated
| take 10

 

This will show the most recent 10 logs that have a message in the syslog that contains both the source and IP address.  This simple query is very powerful and clearly demonstrates why the tags are important in identifying which rule has been triggered.  The syslog 'Message' contains the PASS effect of the rule (PASS or DROP or REJECT), the protocol used, the IP source and destination and the rule that triggered the log.

MarkGodfrey_1-1714849796126.png

Figure 18:  Kusto query example 1; PASS - allowing traffic from WebServer-01 to AppServer-01

 

In the screenshot below, we can see that the WebServer-01 to BackEndServer-01 the rule 'WebServers-to-BackEnd Servers has a 'REJECT' that is blocking traffic (as expected).  However for the WebServer-01 to WebServer-02, this traffic is being blocked by the Default layer 3 rule (found under the Applications tab).  This is because unlike WebServers-to-BackEndServers rule, there is no explicit rule blocking traffic between WebServer-01 and WebServer-02

 

MarkGodfrey_3-1714850594620.png

Figure 19:  Kusto query example 2; DROP, traffic blocked between WebServer-01 to WebServer-02 and the BackEndServer-01

 

 

 

Concluding Remarks

The key message here is to use tagging and log labelling to facilitate the use of NSX-T firewall rules and simplify the ability to find which rules are being triggered using Kusto. Note that Kusto can be used to create more simplified specific data outputs rather than the raw data shown here.

 

Tips & Tricks

  • Ensure that when you change the default layer 3 rule to 'Drop' that you have created a rule to enable DHCP so your client VM's can get an IP address.        MarkGodfrey_4-1714846954543.png
  • Whenever you create a rule, add the log label.
  • Turn on logging when you need to trouble shoot, otherwise a large amount of data will be stored in log analytics, and that can become expensive.

 

References:

Configure VMware syslogs for Azure VMware Solution: https://learn.microsoft.com/en-us/azure/azure-vmware/configure-vmware-syslogs

NSX-T Log messages and error codes:  https://docs.vmware.com/en/VMware-NSX/4.1/administration/GUID-406AF9C3-E8F7-447A-8E3D-92AFB9D5E973.html

Tags in NSX-T: https://docs.vmware.com/en/VMware-NSX-T-Data-Center/3.2/administration/GUID-358DF469-75C8-48C4-B0E2-279E55C7BB3E.html

 

 

 

 

 

 

 

 

 

Published on:

Learn more
Azure Migration and Modernization Blog articles
Azure Migration and Modernization Blog articles

Azure Migration and Modernization Blog articles

Share post:

Related posts

General Availability of SQL FCI and AG Features SQL Server Enabled by Azure Arc

We have good news. Two business continuity features for SQL Server enabled by Azure Arc are now generally available: View Failover Cluster In...

23 hours ago

Azure VMware Solution using a public IP down to the NSX-T Edge; configure SNAT, No-SNAT & DNAT

Azure VMware Solution How To Series: Configuring NSX-T SNAT, No-SNAT & DNAT rules   Overview Requirements Lab Environment NAT Rules K...

1 day ago

App attach for Azure Virtual Desktop now generally available

App attach for Azure Virtual Desktop allows IT admins to dynamically attach applications from an application package to a user session in Azur...

2 days ago

A Closer Look at Azure WAF’s Data Masking Capabilities for Azure Front Door

The Azure Web Application Firewall (WAF) on Azure Front Door offers centralized protection for your web applications against vulnerabilities a...

2 days ago

Azure Virtual Network Manager (AVNM) Mesh and Direct Connectivity are Generally Available!

Azure Virtual Network Manager's (AVNM) mesh connectivity configuration and direct connectivity option in the hub and spoke connectivity config...

2 days ago

Azure pricing: How to calculate costs of Azure products and services

In our previous blogs we explained the Azure pricing structure and how customers can estimate their project costs when migrating to Azure or b...

2 days ago

Azure Pricing: How to estimate Azure project costs

In the previous blog we explained how you can learn about Azure pricing with free services and a pay-as-you-go model. Next, lets understand ho...

2 days ago

Azure Pricing: How to navigate Azure pricing options and resources

In this blog we discussed customer pricing needs and how they match different phases of the cloud journey, and we provided various tools and r...

2 days ago

Azure pricing: How to optimize costs for your Azure workloads

In our previous blogs we explained the Azure pricing structure, how customers can calculate their costs when migrating or building in Azure, a...

2 days ago
Stay up to date with latest Microsoft Dynamics 365 and Power Platform news!
* Yes, I agree to the privacy policy