Azure Network Watcher Explained: A Beginner’s Guide

By /

Azure Network Watcher Explained: A Comprehensive Beginner’s Guide

The cloud revolution, spearheaded by platforms like Microsoft Azure, offers unprecedented flexibility, scalability, and power. However, managing resources in the cloud introduces new challenges, especially when it comes to networking. Unlike traditional on-premises networks where you often have physical access and familiar tools, the virtualized nature of cloud networking requires a different approach to monitoring, diagnostics, and visibility. This is where Azure Network Watcher steps in.

For anyone building or managing applications and services in Azure, understanding network behavior is critical. Is your web server accessible from the internet? Can your application server connect to the database? Are your security rules blocking legitimate traffic? Why is latency high between two virtual machines? Answering these questions quickly and accurately is essential for maintaining application availability, performance, and security.

Azure Network Watcher is Microsoft’s native suite of tools designed specifically to provide the insight needed to monitor, diagnose, and manage Azure virtual networks effectively. It’s not a single product but rather a collection of features integrated into the Azure platform, empowering administrators and developers to gain deep visibility into their network infrastructure.

This guide is designed for beginners who are new to Azure networking or Network Watcher itself. We will explore what Network Watcher is, why it’s indispensable, how to enable it, and delve deep into its core features with practical explanations and use cases. By the end of this guide, you’ll have a solid understanding of how to leverage Network Watcher to keep your Azure networks healthy, secure, and performing optimally.

Table of Contents

  1. What is Azure Network Watcher?
    • The Need for Network Visibility in Azure
    • Network Watcher as the Solution
    • Key Objectives of Network Watcher
  2. Why Use Azure Network Watcher? The Benefits
    • Proactive Monitoring and Early Detection
    • Faster Troubleshooting and Root Cause Analysis
    • Enhanced Network Security Posture
    • Improved Performance Optimization
    • Simplified Compliance and Auditing
    • End-to-End Network Visibility
  3. Core Concepts and Architecture
    • Regional Service
    • Enabling Network Watcher
    • The Network Watcher Agent (VM Extension)
    • Permissions and RBAC
  4. Getting Started: Enabling Network Watcher
    • Automatic Enablement (Default)
    • Manual Enablement via Azure Portal
    • Enabling via Azure PowerShell
    • Enabling via Azure CLI
  5. Deep Dive into Network Watcher Features
    • Monitoring Features:
      • Topology
      • Connection Monitor
    • Diagnostic Tools:
      • IP Flow Verify
      • Next Hop
      • Effective Security Rules
      • VPN Troubleshoot (Gateway & Connection Diagnostics)
      • Packet Capture
      • Connection Troubleshoot
    • Logging and Traffic Analysis:
      • NSG Flow Logs
      • Traffic Analytics
      • Diagnostic Logs (for other network resources)
    • Usage and Quotas:
      • Network Resource Usage and Limits
  6. Integrating Network Watcher with Other Azure Services
    • Azure Monitor (Log Analytics, Alerts)
    • Azure Security Center
    • Azure Policy
    • Azure Sentinel
  7. Best Practices for Using Azure Network Watcher
    • Enable Network Watcher Proactively
    • Understand Agent Requirements
    • Leverage NSG Flow Logs and Traffic Analytics
    • Use Connection Monitor for Critical Paths
    • Integrate with Azure Monitor for Alerting
    • Regularly Review Diagnostic Results
    • Utilize RBAC for Secure Access
    • Automate Where Possible
  8. Azure Network Watcher Pricing
    • Free Features
    • Paid Features (Usage-Based)
  9. Conclusion: Mastering Network Visibility in Azure

1. What is Azure Network Watcher?

Imagine trying to navigate a complex city without a map, GPS, or road signs. That’s akin to managing an Azure network without proper monitoring tools. Azure Network Watcher provides that essential map, GPS, and diagnostic toolkit for your virtual networks.

The Need for Network Visibility in Azure:

Azure Virtual Networks (VNets) form the foundation of your private network space in the cloud. Within these VNets, you deploy Virtual Machines (VMs), databases, load balancers, firewalls, and other services. These components communicate with each other, with on-premises networks via VPNs or ExpressRoute, and with the public internet. This intricate web of connections can become complex quickly. Problems can arise anywhere:

  • Connectivity Issues: A VM might suddenly lose internet access, or two VMs within the same VNet might fail to communicate.
  • Security Misconfigurations: A Network Security Group (NSG) rule might be incorrectly configured, either blocking necessary traffic or allowing unwanted access.
  • Performance Degradation: Users might experience high latency when accessing an application, potentially due to network path issues.
  • Compliance Requirements: Organizations often need to audit network traffic flows for security and regulatory compliance.

Without visibility, diagnosing these issues becomes a time-consuming guessing game.

Network Watcher as the Solution:

Azure Network Watcher is a regional Azure service that provides a suite of tools specifically designed to address these challenges. It allows you to:

  • Monitor: Keep an eye on network health and performance proactively.
  • Diagnose: Pinpoint the root cause of network connectivity and configuration issues.
  • View: Visualize your network topology and understand traffic flows.
  • Log: Capture detailed network traffic data for analysis and auditing.

It acts as a central hub for network monitoring and diagnostic capabilities within Azure.

Key Objectives of Network Watcher:

  • Provide Network Topology Visualization: Understand the relationships between your Azure resources.
  • Monitor Connectivity and Performance: Check reachability and measure latency/packet loss between endpoints.
  • Diagnose Traffic Filtering Problems: Verify if NSG rules are allowing or denying traffic as expected.
  • Determine Network Routing: Identify the path traffic takes to its destination.
  • Capture Network Traffic: Perform packet captures on VMs for deep-dive analysis.
  • Log Network Traffic Flows: Record detailed metadata about IP traffic flowing through NSGs.
  • Analyze Traffic Patterns: Gain insights into traffic sources, destinations, protocols, and potential threats.

2. Why Use Azure Network Watcher? The Benefits

Integrating Network Watcher into your Azure operational strategy offers significant advantages:

  • Proactive Monitoring and Early Detection: Tools like Connection Monitor allow you to continuously check connectivity and performance along critical network paths. This helps you detect potential issues before they impact users or applications, moving from a reactive to a proactive stance.
  • Faster Troubleshooting and Root Cause Analysis: When network problems occur, time is critical. Network Watcher provides targeted diagnostic tools (IP Flow Verify, Next Hop, Connection Troubleshoot) that drastically reduce the time needed to identify the source of the problem – whether it’s an NSG rule, a routing issue, or a platform problem.
  • Enhanced Network Security Posture: Understanding traffic flow is crucial for security. NSG Flow Logs and Traffic Analytics provide detailed insights into who is talking to whom, over which ports, and whether traffic is being allowed or denied. This helps identify misconfigured security rules, detect anomalous traffic patterns, and validate your network segmentation strategy. Effective Security Rules view helps understand the cumulative effect of all applied NSGs.
  • Improved Performance Optimization: High latency or packet loss can cripple application performance. Connection Monitor helps measure these metrics between various endpoints (Azure VMs, on-premises, URLs). By identifying bottlenecks or suboptimal routing paths, you can take corrective actions to improve user experience.
  • Simplified Compliance and Auditing: Many regulatory frameworks (like PCI-DSS, HIPAA) require detailed logging and auditing of network traffic. NSG Flow Logs provide the necessary data, capturing source/destination IPs, ports, protocols, and NSG rule actions. This data can be archived and analyzed to meet compliance mandates.
  • End-to-End Network Visibility: From visualizing your network layout with Topology to tracking specific flows with Connection Troubleshoot, Network Watcher provides a comprehensive view across your Azure network landscape, bridging the visibility gap often encountered in cloud environments.

In essence, Network Watcher empowers you with the information and tools needed to build, operate, and secure reliable and performant networks in Azure.

3. Core Concepts and Architecture

Understanding a few fundamental concepts is key to using Network Watcher effectively:

  • Regional Service: Network Watcher is deployed and operates on a per-region basis. When you enable Network Watcher for a specific Azure region, you gain access to its monitoring and diagnostic capabilities for the resources within that region. If you have resources spread across multiple regions, you need to enable Network Watcher in each of those regions.
  • Enabling Network Watcher: By default, Network Watcher is automatically enabled in your subscription for all available regions when you create or update a virtual network within that region. However, you can also manually enable or disable it per region if needed. Enabling it creates a hidden Network Watcher resource group (NetworkWatcherRG) in each region to store associated resources.
  • The Network Watcher Agent (VM Extension): Some Network Watcher features, particularly those requiring interaction from within a Virtual Machine (like Packet Capture, Connection Troubleshoot, and certain Connection Monitor tests), require the Network Watcher Agent VM extension to be installed on the target VM. This agent facilitates communication between the Network Watcher service and the VM’s operating system. It’s available for both Windows and Linux VMs. If the agent isn’t installed, these specific features won’t work for that VM. Other features, like IP Flow Verify or Next Hop, operate at the Azure platform level and do not require the agent.
  • Permissions and RBAC (Role-Based Access Control): Like all Azure services, access to Network Watcher features is controlled by Azure RBAC. Users need appropriate permissions (like Microsoft.Network/networkWatchers/* actions) to use the tools. Standard built-in roles like Network Contributor typically include these permissions. You can also create custom roles for more granular control.

4. Getting Started: Enabling Network Watcher

As mentioned, Network Watcher is often enabled automatically. However, it’s good practice to verify its status and know how to enable it manually if required.

Automatic Enablement (Default):

  • When you create or update a VNet in a region where Network Watcher wasn’t previously enabled for your subscription, Azure typically enables it automatically.
  • You can check the status in the Azure portal.

Manual Enablement via Azure Portal:

  1. Navigate to the Azure portal (portal.azure.com).
  2. In the search bar at the top, type “Network Watcher” and select it from the results.
  3. On the Network Watcher overview page, you’ll see a list of your subscriptions and the regions within them.
  4. The map or the list below it will show the enablement status for each region.
  5. If a region is shown as “Disabled”, you can enable it by:
    • Clicking the ellipsis (...) next to the region name.
    • Selecting “Enable Network Watcher”.
  6. Confirm the action. Azure will then provision the necessary Network Watcher resources for that region within the NetworkWatcherRG resource group.

Enabling via Azure PowerShell:

Ensure you have the Azure PowerShell module installed and are connected to your Azure account (Connect-AzAccount) and have selected the correct subscription (Set-AzContext -SubscriptionId "your-subscription-id").

“`powershell

Get a list of all locations where Network Watcher is supported

$locations = Get-AzLocation | Where-Object { $_.Providers -contains “Microsoft.Network” }

Loop through each location and enable Network Watcher if it’s not already enabled

foreach ($location in $locations) {
$networkWatcher = Get-AzNetworkWatcher -Location $location.Location -ErrorAction SilentlyContinue
if ($null -eq $networkWatcher) {
New-AzNetworkWatcher -Location $location.Location -ResourceGroupName NetworkWatcherRG
Write-Host “Enabled Network Watcher in region: $($location.Location)”
} else {
Write-Host “Network Watcher already enabled in region: $($location.Location)”
}
}
“`

Enabling via Azure CLI:

Ensure you have the Azure CLI installed and are logged in (az login) and have set the correct subscription (az account set --subscription "your-subscription-id").

“`bash

Get a list of all locations supporting Microsoft.Network provider

locations=$(az provider show –namespace Microsoft.Network –query “resourceTypes[?resourceType==’virtualNetworks’].locations” -o tsv)

Loop through each location and enable Network Watcher

for location in $locations; do
echo “Checking Network Watcher status in region: $location”
# The configure command enables Network Watcher for the specified region(s)
az network watcher configure –locations $location –enabled true –resource-group NetworkWatcherRG
done

echo “Network Watcher enablement process completed for all supported regions.”
“`

Once enabled in the desired regions, you can start using the various features.

5. Deep Dive into Network Watcher Features

Network Watcher offers a rich set of tools categorized generally into Monitoring, Diagnostics, and Logging/Analytics. Let’s explore the most important ones.

5.1 Monitoring Features

These features provide ongoing visibility into your network’s structure and health.

a) Topology

  • What is it? The Topology feature provides a visual representation of your network resources within a specific Virtual Network (VNet), resource group, or across your entire subscription. It displays resources like VNets, subnets, Network Interfaces (NICs), Public IPs, NSGs, Load Balancers, VPN Gateways, etc., and the relationships between them.
  • What problem does it solve? Understanding the layout of complex Azure networks can be challenging. Topology gives you a clear diagram, making it easier to:
    • Visualize network architecture.
    • Identify how resources are interconnected.
    • Understand dependencies.
    • Spot potential configuration errors (e.g., a VM not associated with the correct subnet or NSG).
  • How does it work? Network Watcher queries Azure Resource Manager (ARM) for resource information within the selected scope and generates an interactive diagram.
  • How to use it?
    1. Go to Network Watcher in the Azure portal.
    2. Select “Topology” under the “Monitoring” section in the left-hand menu.
    3. Choose your Subscription, Resource Group, and the specific Virtual Network you want to visualize.
    4. Click “Generate topology”.
    5. The portal displays the diagram. You can pan, zoom, and click on resources to see more details. You can also download the topology diagram as an SVG file.
  • Outputs/Results: An interactive diagram showing network resources and their associations.
  • Limitations/Considerations: The diagram focuses on Azure networking resources. It might not show all intricate details of application-level connections. The complexity can grow significantly in large environments.

b) Connection Monitor

  • What is it? Connection Monitor provides unified, end-to-end connection monitoring. It allows you to set up tests to check reachability, latency, and packet loss between various endpoints. It’s the successor to the older Connection Monitor (Classic) and Network Performance Monitor (NPM), combining and enhancing their capabilities.
  • What problem does it solve? Ensures critical network paths are healthy and performant. It helps answer questions like:
    • Can my web server reach the backend database?
    • Is there high latency between my Azure VM and my on-premises server over ExpressRoute/VPN?
    • Can my application access a specific external URL?
    • Are there specific hops in the network path causing delays?
  • How does it work? You define a Connection Monitor resource, specifying:
    • Test Groups: Logical containers for tests.
    • Endpoints: Source and destination points for your tests. These can be Azure VMs, Azure VNets, Azure Subnets, on-premises machines (requires Azure Arc agent with Network Monitor extension), or external URLs/IPs.
    • Test Configurations: Define the protocol (TCP, ICMP, HTTP/S), success criteria (latency thresholds, packet loss percentage), test frequency, and ports.
    • Network Watcher then performs regular checks based on your configuration, collecting data on reachability, round-trip time (latency), and packet loss. For HTTP/S tests, it also checks the response code. It can also perform traceroute analysis to identify the hop-by-hop path and latency at each hop.
  • How to use it?
    1. Go to Network Watcher -> Connection Monitor.
    2. Click “+ Create”.
    3. Define the basic settings (Name, Subscription, Region). Choose a Log Analytics workspace to store the monitoring data.
    4. Define Test Groups: Give the group a name.
    5. Add Sources: Select Azure endpoints (VMs, VNets, Subnets) or Non-Azure endpoints (requires Arc agent) or External Addresses.
    6. Add Test Configurations: Choose protocol (TCP, ICMP, HTTP), specify ports, test frequency, and success thresholds (e.g., latency < 100ms, packet loss < 1%).
    7. Add Destinations: Select target endpoints similar to sources.
    8. Review and Create the Connection Monitor.
    9. Once created and running, you can view dashboards showing test status, latency charts, packet loss trends, and hop-by-hop topology views.
  • Outputs/Results: Dashboards and alerts (if configured via Azure Monitor) showing connection status (% checks failed), round-trip time (ms), hop-by-hop latency, and topology diagrams highlighting problematic links. Data is stored in Log Analytics for further querying.
  • Limitations/Considerations:
    • Requires a Log Analytics workspace.
    • Testing from/to on-premises requires the Azure Arc agent and Network Monitor extension.
    • Agentless monitoring is available for some scenarios but agent-based provides more detailed insights (like hop-by-hop).
    • There are costs associated with the number of tests and data ingestion into Log Analytics.

5.2 Diagnostic Tools

These tools help you pinpoint the cause of specific network issues on demand.

a) IP Flow Verify

  • What is it? A quick diagnostic tool to check if a specific network flow (defined by source/destination IP and port, and protocol) is allowed or denied to or from a Virtual Machine. It specifically checks the effective Network Security Group (NSG) rules applied to the VM’s network interface (NIC).
  • What problem does it solve? Quickly determines if network connectivity is being blocked by an NSG rule. Useful when:
    • A VM cannot reach another resource (or vice-versa).
    • You suspect an NSG rule is misconfigured.
    • You need to validate that a newly created NSG rule is working as intended.
  • How does it work? You provide the details of the VM, its NIC, the traffic direction (Inbound/Outbound), protocol (TCP/UDP/ICMP), local IP/port, and remote IP/port. Network Watcher simulates this flow against the effective NSG rules for the specified NIC and tells you whether the traffic would be allowed or denied, and importantly, which specific NSG rule caused that decision.
  • How to use it?
    1. Go to Network Watcher -> IP Flow Verify (under “Network Diagnostic Tools”).
    2. Select the Subscription, Resource Group, and the Virtual Machine.
    3. Select the Network Interface on the VM.
    4. Specify the Protocol (TCP, UDP, ICMP).
    5. Choose the Direction (Outbound or Inbound).
    6. Enter the Local IP address (usually the private IP of the VM’s NIC) and Local Port.
    7. Enter the Remote IP address and Remote Port you are trying to connect to or from.
    8. Click “Check”.
  • Outputs/Results: A clear “Allowed” or “Denied” status. If denied, it explicitly names the NSG and the specific security rule responsible for blocking the traffic. If allowed, it indicates which rule permitted it (often a default rule if no specific custom rule matched).
  • Limitations/Considerations:
    • Only checks NSG rules. It does not check OS firewall rules, Azure Firewall rules, routing issues, or application-level problems.
    • It’s a point-in-time check based on the current configuration.

b) Next Hop

  • What is it? This tool helps determine the next network hop (the next routing device) for traffic leaving a specific VM towards a given destination IP address.
  • What problem does it solve? Diagnoses routing problems. Useful when:
    • A VM cannot reach a destination, and you suspect a routing issue rather than an NSG block.
    • You need to verify if traffic is being routed correctly (e.g., through a Network Virtual Appliance (NVA), VPN gateway, or directly to the internet).
    • You are troubleshooting User Defined Routes (UDRs).
  • How does it work? You specify the source VM, its NIC, and the destination IP address. Network Watcher consults the effective route table for that VM’s subnet (considering system routes and UDRs) and identifies the type and IP address of the next hop the traffic will be sent to.
  • How to use it?
    1. Go to Network Watcher -> Next Hop.
    2. Select the Subscription, Resource Group, and the Virtual Machine.
    3. Select the Network Interface on the VM.
    4. Enter the Destination IP address you want to check the route for.
    5. Click “Next hop”.
  • Outputs/Results: Provides the “Next hop type” (e.g., VirtualNetworkGateway, VnetLocal, Internet, VirtualAppliance, None) and the “Next hop IP address” (if applicable). For example, if routing to the internet, the type might be Internet with no IP. If routing through an NVA defined in a UDR, it will show VirtualAppliance and the NVA’s private IP.
  • Limitations/Considerations:
    • Shows only the next hop from the perspective of the Azure fabric, not the entire path.
    • Doesn’t guarantee reachability to the final destination, only where the traffic is initially sent.

c) Effective Security Rules

  • What is it? This feature displays the final, aggregated set of security rules applied to a specific Network Interface (NIC) or a subnet. Since NSGs can be associated at both the NIC and subnet levels, and include default rules, this tool calculates the combined effect.
  • What problem does it solve? Helps understand the complete security posture for a VM or subnet. Useful when:
    • You have NSGs applied at both subnet and NIC levels and need to see the consolidated rules.
    • You want to verify which default rules are active.
    • You need a clear overview for security audits or troubleshooting complex NSG interactions.
  • How does it work? Network Watcher gathers all NSGs associated with the selected resource (NIC or subnet), including default rules, processes their priorities, and presents the final list of rules that are actually in effect.
  • How to use it?
    1. Go to Network Watcher -> Effective Security Rules.
    2. Select the Subscription and Resource Group.
    3. Choose the resource type: either a Virtual Machine (which then lets you select a NIC) or a Subnet.
    4. Select the specific VM/NIC or Subnet.
    5. The effective rules are displayed, showing the rule name, protocol, source/destination, action (Allow/Deny), and whether the rule originates from the NIC-level NSG or the Subnet-level NSG.
  • Outputs/Results: A comprehensive table listing all effective security rules, their properties, and their origin (NIC or Subnet NSG).
  • Limitations/Considerations: Provides a snapshot of the rules at the time of query. It doesn’t dynamically monitor changes.

d) VPN Troubleshoot (Gateway & Connection Diagnostics)

  • What is it? A diagnostic tool specifically for troubleshooting Azure VPN Gateways and their Connections. It runs a series of checks on the gateway and specific connections to identify configuration issues or platform problems.
  • What problem does it solve? Helps diagnose common issues with VPN tunnels, such as:
    • VPN connection failing to establish or being unstable.
    • Connectivity problems between Azure and on-premises networks over the VPN.
    • Performance issues with the VPN connection.
  • How does it work? You select a VPN Gateway and optionally a specific Connection resource linked to it. You also specify a storage account to store the detailed diagnostic logs. Network Watcher then runs checks covering gateway health, configuration mismatches (e.g., pre-shared keys, IPsec/IKE parameters), route table issues, tunnel status, and platform health.
  • How to use it?
    1. Go to Network Watcher -> VPN Troubleshoot.
    2. Select the Subscription and Resource Group containing the gateway.
    3. Choose the VPN Gateway.
    4. Optionally, select a specific Connection to diagnose.
    5. Select or create a Storage Account to store the results. Provide a container name.
    6. Click “Start troubleshooting”. The process can take several minutes.
    7. Once complete, Network Watcher provides a summary of the findings (Health Status: Healthy, Unhealthy, Degraded) and a link to download the detailed logs from the storage account.
  • Outputs/Results: A health summary in the portal and detailed log files in the specified storage account containing diagnostics information about gateway configuration, BGP status (if applicable), tunnel status, and potential issues identified.
  • Limitations/Considerations: Focuses on the Azure side of the VPN. Issues on the on-premises VPN device still require investigation using vendor-specific tools. Log analysis might require familiarity with VPN concepts.

e) Packet Capture

  • What is it? Allows you to remotely start and stop network packet captures on an Azure Virtual Machine. The captured data (in .cap format) can be stored in an Azure Storage account, on the VM’s local disk, or both.
  • What problem does it solve? Provides deep-level network traffic analysis for complex troubleshooting scenarios where metadata logs (like NSG Flow Logs) are insufficient. Useful for:
    • Analyzing specific network protocol behavior (TCP handshakes, HTTP requests/responses).
    • Diagnosing intermittent connectivity issues.
    • Investigating application-specific network errors.
    • Security incident response and forensic analysis.
  • How does it work? Requires the Network Watcher Agent VM extension installed on the target VM. You define a packet capture session, specifying the target VM, filters (source/destination IP/port, protocol), capture duration or size limits, and storage location. Network Watcher instructs the agent on the VM to start capturing packets matching the filters using the OS’s native capture capabilities (like tcpdump on Linux or netsh on Windows). Once the capture stops (manually or based on limits), the .cap file is uploaded to the specified storage.
  • How to use it?
    1. Ensure the Network Watcher Agent extension is installed on the target VM.
    2. Go to Network Watcher -> Packet Capture.
    3. Click “+ Add”.
    4. Select the Subscription, Resource Group, and Target virtual machine.
    5. Provide a Packet capture name.
    6. Choose the Storage location (Storage account, local file path on VM, or both). Select/configure the storage account if needed.
    7. Optionally set limits: Maximum bytes per packet (truncation), Maximum bytes per session, Time limit (seconds).
    8. Define Filters (optional but highly recommended to reduce capture size): Specify Local IP, Remote IP, Local Port, Remote Port, and Protocol (TCP, UDP, ICMP, Any). You can add multiple filters.
    9. Click “Start”. The capture session begins.
    10. You can monitor the status (Starting, Running, Stopping, Stopped, Error).
    11. Click “Stop” to manually end the capture, or it will stop based on the defined limits.
    12. Once stopped, the .cap file will be available in the configured storage location. You can download it and analyze it using tools like Wireshark or tcpdump.
  • Outputs/Results: A .cap file containing the captured network packets, ready for analysis with standard network protocol analyzers.
  • Limitations/Considerations:
    • Requires the Network Watcher Agent on the VM.
    • Can consume significant CPU/memory resources on the VM during capture.
    • Can generate large files, impacting storage costs and VM disk space if stored locally.
    • Filtering is crucial to manage capture size and focus on relevant traffic.
    • Captures are performed within the VM’s OS, after NSGs have been processed for inbound traffic and before they are processed for outbound traffic.

f) Connection Troubleshoot

  • What is it? A powerful tool that performs a point-in-time check of connectivity between a source (VM, Application Gateway, Bastion Host) and a destination (VM, FQDN, URI, IP Address). It combines checks similar to IP Flow Verify and Next Hop, and can also perform latency checks and path analysis if the Network Watcher agent is installed.
  • What problem does it solve? Provides a unified check for diagnosing connectivity issues between two endpoints, considering both security rules (NSGs) and routing. It helps answer: “Can VM A reach VM B on port 443, and if not, why?”
  • How does it work? You specify the source resource, the destination (IP, FQDN, or resource ID), and the destination port/protocol.
    • Without Agent: It checks NSG rules and routes (similar to IP Flow Verify + Next Hop).
    • With Agent (on source VM): It performs the above checks and attempts a live connection test (TCP ping) from the source VM, measures latency, performs a traceroute to identify hops, and reports detailed path information.
  • How to use it?
    1. Go to Network Watcher -> Connection Troubleshoot.
    2. Select the Subscription and Resource Group of the source resource.
    3. Choose the Source type (Virtual machine, Application Gateway, Bastion Host) and select the specific resource.
    4. Specify the Destination:
      • Manually enter an IP address, FQDN (e.g., www.microsoft.com), or URI.
      • Or, select a destination Virtual machine.
    5. Enter the Destination Port.
    6. Choose the Protocol (TCP, ICMP, HTTP, HTTPS – options depend on source/destination and agent presence).
    7. Optionally, specify Source Port and Preferred IP version (IPv4/IPv6).
    8. Click “Check”. The test runs, potentially taking a minute or two, especially with the agent.
  • Outputs/Results: Provides a clear status: “Reachable” or “Unreachable”.
    • If Unreachable, it identifies the bottleneck: usually an NSG rule (SecurityRule) or a routing issue (UserDefinedRoute, SystemRoute).
    • If Reachable (especially with the agent), it provides:
      • Min/Avg/Max Latency (ms).
      • Packet loss percentage.
      • Number of probes sent/failed.
      • A detailed Hop-by-hop path view, showing IP addresses and latency at each step between source and destination.
  • Limitations/Considerations:
    • Full capabilities (latency, path analysis) require the Network Watcher Agent on the source VM.
    • Checks firewalls within the OS only if the agent is installed and depending on the test protocol (e.g., TCP ping might be blocked by Windows Firewall even if NSG allows it).
    • It’s a point-in-time diagnostic, not continuous monitoring (use Connection Monitor for that).

5.3 Logging and Traffic Analysis

These features focus on recording and analyzing network traffic patterns over time.

a) NSG Flow Logs

  • What is it? A feature that enables logging of metadata about IP traffic flowing through a Network Security Group (NSG). It records information about allowed and denied traffic based on NSG rules.
  • What problem does it solve? Provides essential visibility into network traffic for:
    • Security Auditing: Understanding who is accessing what resources and whether access was permitted or blocked.
    • Compliance: Meeting regulatory requirements for traffic logging.
    • Troubleshooting: Identifying unexpected allowed or denied flows.
    • Network Forensics: Investigating security incidents by examining historical traffic patterns.
    • Capacity Planning: Understanding traffic volumes and patterns.
  • How does it work? You enable NSG Flow Logs on a specific NSG. Azure then captures metadata for all IP flows passing through the NSG rules. This data includes:
    • Timestamp
    • Source & Destination IP Address
    • Source & Destination Port
    • Protocol (TCP/UDP/ICMP)
    • NSG Rule that processed the traffic
    • Action taken (Allow/Deny)
    • Traffic direction (Inbound/Outbound)
    • Flow state (for TCP flows – beginning, ongoing, ending)
    • Bytes and Packets transferred
    • (Version 2 adds) Throughput and Flow state information.
      The logs are written in JSON format to an Azure Storage account that you specify. You can choose the retention period for these logs.
  • How to use it?
    1. Go to Network Watcher -> NSG Flow Logs.
    2. Find the NSG you want to enable logs for (you can filter by subscription/resource group).
    3. Click on the NSG name.
    4. In the settings pane that appears, click “+ Create”.
    5. Configure the settings:
      • Basics: Select Subscription, NSG.
      • Configuration:
        • Set Flow Logs Status to “On”.
        • Select Flow Logs version (Version 2 is recommended for more data and Traffic Analytics compatibility).
        • Select an existing Azure Storage account or create a new one to store the logs.
        • Set the Retention period (days). Set to 0 to retain forever (or manage deletion manually/via lifecycle policies).
      • Traffic Analytics (Optional but Recommended):
        • Set Traffic Analytics Status to “Enabled”.
        • Choose a Processing Interval (e.g., every 1 hour or every 10 mins).
        • Select a Log Analytics Workspace to store the processed analytics data.
    6. Review and Create. It might take a few minutes for logging to start.
  • Outputs/Results: JSON log files stored in the designated Azure Storage account blob container (insights-logs-networksecuritygroupflowevent). The files are organized by subscription, resource group, NSG name, date, and time.
  • Limitations/Considerations:
    • Logs metadata, not the actual packet content (use Packet Capture for that).
    • Generates potentially large amounts of data, impacting storage costs. Plan storage account capacity and lifecycle management.
    • Raw JSON logs can be difficult to analyze manually for large volumes. Requires tools or services like Traffic Analytics or custom scripts/SIEM integration.
    • There’s a slight delay (a few minutes) between traffic flow and log appearance.

b) Traffic Analytics

  • What is it? An Azure-native service that processes raw NSG Flow Logs, enriches them with threat intelligence and topology information, and provides rich visualizations and insights within a Log Analytics workspace.
  • What problem does it solve? Transforms the raw, verbose NSG Flow Log data into actionable intelligence. It helps you:
    • Visualize network activity on maps (Geo Map).
    • Understand traffic patterns (Top Talkers, Frequent Conversations, Traffic Distribution).
    • Identify network security risks (Malicious IPs, Open Ports, Traffic to/from risky regions).
    • Analyze traffic across VNets, subnets, VMs, and applications.
    • Monitor VPN gateway usage and traffic distribution.
    • Optimize network deployments based on traffic insights.
  • How does it work? When you enable Traffic Analytics (usually during NSG Flow Log setup), you specify a Log Analytics workspace and a processing interval. Azure periodically fetches the NSG Flow Logs from the configured storage account, processes them, correlates them with Azure topology data, Microsoft threat intelligence feeds, and GeoIP databases, and then ingests the enriched data into your Log Analytics workspace. You can then explore this data using pre-built dashboards in Network Watcher or by writing custom Kusto Query Language (KQL) queries in Log Analytics.
  • How to use it?
    1. Ensure NSG Flow Logs (Version 2 recommended) are enabled for the NSGs you want to analyze.
    2. Enable Traffic Analytics either during NSG Flow Log setup or by editing the flow log settings. Select a Log Analytics workspace and processing interval.
    3. Wait for the processing interval to complete (e.g., 10 mins or 1 hour).
    4. Go to Network Watcher -> Traffic Analytics.
    5. Select the Log Analytics Workspace you configured.
    6. Explore the dashboards: Geo Map, Virtual Network Topology, Traffic Distribution, Application Ports, Malicious Flows, etc. Use the time range selector and filters (Subscription, Resource Group, NSG, VM, etc.) to narrow down the view.
    7. For deeper analysis, click the links that take you to Log Analytics, where you can view the underlying KQL queries or write your own.
  • Outputs/Results: Interactive dashboards and visualizations within the Network Watcher portal. Enriched log data stored in the specified Log Analytics workspace (in the AzureNetworkAnalytics_CL table) queryable via KQL.
  • Limitations/Considerations:
    • Requires NSG Flow Logs (Version 2 recommended).
    • Requires a Log Analytics workspace.
    • Incurs costs based on the volume of log data processed and ingested into Log Analytics. Processing intervals affect cost and data freshness (more frequent = higher cost, fresher data).
    • Relies on the accuracy and completeness of NSG Flow Logs.

c) Diagnostic Logs (for other network resources)

  • What is it? While NSG Flow Logs are specific to NSGs, Azure allows enabling diagnostic logging for many other network resources like Azure Firewall, Load Balancer, Application Gateway, VPN Gateway, ExpressRoute Circuit, Public IP Addresses, etc. These logs capture operational events, health status, rule processing (for Firewall), request/response data (for App Gateway), etc.
  • What problem does it solve? Provides operational and diagnostic insights specific to the resource type, complementing the flow-level data from NSGs. Helps troubleshoot resource-specific issues, monitor health, and audit configurations.
  • How does it work? Similar to NSG Flow Logs, you configure Diagnostic Settings for the specific resource. You choose which log categories and metrics you want to capture and where to send them:
    • Log Analytics workspace (for analysis with KQL, integration with Azure Monitor)
    • Azure Storage account (for archival)
    • Azure Event Hubs (for streaming to external SIEMs or real-time processing)
  • How to use it?
    1. Navigate to the specific network resource (e.g., your Azure Firewall or Load Balancer) in the Azure portal.
    2. In the resource’s menu, find “Diagnostic settings” under the “Monitoring” section.
    3. Click “+ Add diagnostic setting”.
    4. Give the setting a name.
    5. Select the log categories (e.g., AzureFirewallApplicationRuleLog, AzureFirewallNetworkRuleLog for Firewall; LoadBalancerProbeHealthStatus for Load Balancer) and metrics you want to collect.
    6. Choose the destination(s) (Log Analytics, Storage Account, Event Hub) and configure them.
    7. Save the diagnostic setting.
  • Outputs/Results: Log data and metrics sent to the configured destinations, available for analysis using tools appropriate for the destination (KQL in Log Analytics, file access in Storage, stream processing for Event Hubs).
  • Limitations/Considerations: Log schema and content vary significantly depending on the resource type. Costs are associated with data storage (Storage Account), ingestion/retention (Log Analytics), or throughput (Event Hubs).

5.4 Usage and Quotas

a) Network Resource Usage and Limits

  • What is it? An informational feature within Network Watcher that displays your current usage of specific network resources within a subscription against the established Azure subscription limits.
  • What problem does it solve? Helps you track resource consumption and avoid hitting subscription quotas unexpectedly. Allows for better capacity planning. Shows usage for resources like VNets, Static Public IPs, Load Balancers, Application Gateways, NSGs, Route Tables, etc.
  • How does it work? Queries ARM to get the count of deployed network resources per region within the selected subscription and compares it to the documented limits.
  • How to use it?
    1. Go to Network Watcher -> Usage + quotas (under “Settings”).
    2. Select the Subscription you want to check.
    3. Optionally, filter by Region or Resource Type.
    4. The portal displays a table showing the Resource type, Region, Current usage count, and the Limit.
  • Outputs/Results: A table summarizing network resource usage against limits per region.
  • Limitations/Considerations: Reflects current usage based on ARM data. Limits can sometimes be increased by contacting Azure support.

6. Integrating Network Watcher with Other Azure Services

Network Watcher doesn’t operate in isolation. Its value is significantly amplified when integrated with other Azure monitoring, security, and automation services:

  • Azure Monitor: This is the primary integration point.
    • Log Analytics: NSG Flow Logs (processed by Traffic Analytics), Connection Monitor results, and Diagnostic Logs from various network resources are typically sent to a Log Analytics workspace. This allows for powerful querying using KQL, creating custom visualizations, and correlating network data with application and infrastructure logs.
    • Alerts: You can configure alerts in Azure Monitor based on Network Watcher data. For example, create alerts when Connection Monitor detects high latency or packet loss, when specific NSG rules are hit frequently (from flow logs), or when diagnostic logs indicate resource health issues.
    • Azure Monitor for Networks (Network Insights): This curated monitoring experience within Azure Monitor leverages Network Watcher data (Topology, Connection Monitor, NSG Flow Logs, Diagnostics) to provide a unified view of network health, performance, and dependencies across your subscriptions. It offers pre-built workbooks and visualizations.
  • Azure Security Center: Traffic Analytics data, particularly findings related to malicious IPs or risky open ports, can feed into Security Center recommendations and alerts, contributing to your overall security posture assessment. NSG configurations are also assessed by Security Center.
  • Azure Policy: You can use Azure Policy to enforce Network Watcher configurations, such as ensuring NSG Flow Logs are enabled on all critical NSGs or mandating the installation of the Network Watcher Agent on specific VMs.
  • Azure Sentinel (or other SIEMs): NSG Flow Logs and other diagnostic logs can be streamed via Event Hubs or ingested directly from Log Analytics into Azure Sentinel (Microsoft’s cloud-native SIEM) or third-party SIEMs for advanced threat detection, correlation with other security signals, and incident response workflows.

7. Best Practices for Using Azure Network Watcher

To maximize the benefits of Network Watcher, consider these best practices:

  1. Enable Network Watcher Proactively: Ensure Network Watcher is enabled in all Azure regions where you have network resources. Since it’s often enabled automatically, verify its status.
  2. Understand Agent Requirements: Be aware of which features (Packet Capture, Connection Troubleshoot details, Connection Monitor agent-based tests) require the Network Watcher Agent VM extension and ensure it’s deployed where needed, potentially using Azure Policy for automation.
  3. Leverage NSG Flow Logs and Traffic Analytics: Enable NSG Flow Logs (Version 2) for critical NSGs and utilize Traffic Analytics for insightful visualization and analysis. This provides invaluable security and operational visibility. Plan for storage and Log Analytics costs.
  4. Use Connection Monitor for Critical Paths: Set up Connection Monitor tests for essential communication pathways: between application tiers, from frontend to backend, hybrid connections (VPN/ExpressRoute), and critical external dependencies. Define meaningful thresholds for latency and packet loss.
  5. Integrate with Azure Monitor for Alerting: Don’t just collect data; act on it. Configure alerts in Azure Monitor based on Connection Monitor failures, critical diagnostic log entries, or specific patterns identified in Traffic Analytics/NSG Flow Logs.
  6. Regularly Review Diagnostic Results: Periodically use tools like Effective Security Rules and Topology to validate your network configuration and architecture. Use IP Flow Verify and Next Hop as your first line of defense when troubleshooting connectivity.
  7. Utilize RBAC for Secure Access: Assign appropriate permissions using Azure RBAC. Grant diagnostic capabilities to operational teams but restrict configuration changes to authorized personnel. Use built-in roles like Network Contributor or Reader or create custom roles as needed.
  8. Automate Where Possible: Use Azure CLI, PowerShell, or ARM templates/Bicep to automate the deployment and configuration of Network Watcher features (enabling flow logs, creating connection monitors, deploying agents) as part of your infrastructure-as-code practices.

8. Azure Network Watcher Pricing

Network Watcher’s pricing model involves both free and paid components:

  • Free Features: Many core diagnostic tools are generally free to use, including:
    • Topology view
    • IP Flow Verify
    • Next Hop
    • Effective Security Rules
    • VPN Troubleshoot (running the diagnostic itself is free, but log storage incurs costs)
    • Usage + Quotas view
  • Paid Features (Usage-Based): Costs are typically associated with data generation, storage, and processing:
    • NSG Flow Logs: Charged based on the volume (GB) of logs generated and stored in Azure Storage. Standard storage account pricing applies.
    • Traffic Analytics: Charged based on the volume (GB) of NSG Flow Log data processed. Data ingestion and retention in the Log Analytics workspace also incur standard Log Analytics pricing.
    • Connection Monitor: Charged based on the number of connection tests performed per month. Data ingestion and retention in Log Analytics also apply.
    • Packet Capture: Charged based on the amount of data stored in Azure Storage. Standard storage account pricing applies.
    • Diagnostic Logs (for other resources): Costs depend on the destination – Log Analytics ingestion/retention, Storage account storage, or Event Hubs throughput/capture.

Important: Always refer to the official Azure Pricing page for Network Watcher for the most current and detailed pricing information, as costs can vary by region and specific usage patterns.

9. Conclusion: Mastering Network Visibility in Azure

Azure Network Watcher is an indispensable service for anyone operating workloads in Azure. It transforms the potentially opaque world of cloud networking into a visible, manageable, and diagnosable environment. From visualizing topology and proactively monitoring critical connections with Connection Monitor to performing deep diagnostics with Packet Capture and gaining security insights with NSG Flow Logs and Traffic Analytics, Network Watcher provides a comprehensive toolkit.

By understanding its core concepts, enabling it in your regions, and learning how to leverage its diverse features, you move from reactive troubleshooting to proactive management. Integrating Network Watcher with Azure Monitor and other services further enhances its capabilities, allowing for automated alerting and correlation with broader application and security data.

While the cloud offers immense benefits, effective monitoring and diagnostics are non-negotiable for ensuring reliability, performance, and security. Azure Network Watcher provides the essential capabilities to achieve this, empowering beginners and experienced professionals alike to confidently build and operate robust network infrastructure in Azure. Start exploring its features today, integrate them into your operational workflows, and gain the network visibility you need to succeed in the cloud.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top