Connectivity from IaaS(VM) and PaaS service(ADF) to ADLS account with public network access disabled
In this blog post, we will investigate the impacts of disabling public network access in the storage account and learn how to establish connections to an Azure Data Lake Storage account from both IaaS (Virtual Machine) and PaaS services (Azure Data Factory), when public network access is disabled. Additionally, we will address common troubleshooting issues that may arise due to this configuration.
Introduction
Azure storage accounts provide a scalable and secure way to store and access data in the cloud. This model enables you to control the level of access to your storage accounts that your applications and enterprise environments demand, based on the type and subset of networks or resources that you use. However, some scenarios may require you to disable public network access to your storage accounts, such as compliance, security, or cost reasons. This means that your storage accounts can only be accessed from a private endpoint within your virtual network or from a service that has a private link to your storage account which helps to enhance your data protection and isolation.
Effects of Disabling Public Network Access
When you disable public network access to your storage accounts, you are essentially blocking all requests that do not originate from the Virtual Network that has a private endpoint or from services that do not have a private link to your storage account. You will not be able to access your storage account from the Azure portal, Azure CLI, Azure PowerShell, Azure SDKs, or any other tool that uses the public endpoint of your storage account. You will need to use a private endpoint or a service with a private link to connect to your storage account.
By default, storage accounts accept connections from clients on any network. You can limit access to selected networks or prevent traffic from all networks and permit access only through a private endpoint.
Demonstration and Troubleshooting
1) How to disable public network access.
Step1. Navigate to your Azure storage account on the Azure Portal. Locate the Networking settings under Security + networking blade.
Step2. Click on “Networking” to open the Azure Storage networking/firewall settings.
Step3. To block traffic from all networks, select “Disabled” under the “Public network access” setting in the storage account firewall and save the changes.
2) Connecting to storage account from an IaaS Service – Azure VM
In this section, we will illustrate the process of establishing a connection to an Azure storage account with public network access disabled, from an Azure Virtual machine.
Step1. Create a virtual machine within the same region as the storage account and configure a new virtual Network with a resource subnet.
Step2. Use the virtual machine you created in the previous steps to connect to the storage account using Microsoft Azure Storage Explorer. The request failed with 403 - Authorization error while accessing the storage account due to the restricted access on the storage account.
Please note that Microsoft Azure Storage Explorer is a standalone app that makes it easy to work with Azure Storage data on Windows, macOS, and Linux. Click here to download.
Azure monitoring logs, enabled for the storage account and stored in the log analytics workspace, provide more information about the same error. To check the failed request in the logs, navigate to the Monitoring blade of your storage account, click on “Logs”, and access the storage logs stored in Azure Log Analytics using Azure Monitor log queries.
This indicates that the request originates from the private IP address of the Azure virtual machine.
StorageBlobLogs
| where CorrelationId == "xxxxx-xxxx-xxxx-xxxxx000000"
| project TimeGenerated, StatusCode, StatusText, CallerIpAddress, CorrelationId, UserAgentHeader, Uri
Quick Tip:
There is an alternative approach to perform a nslookup on the storage account. This should result in resolving to a private IP, and you can verify this by checking the IP assigned to the FQDN in the private endpoint configuration.
Step3. Please note that when public network access is disabled in the storage account, connectivity to the storage account can only be established via a private endpoint. Hence, private endpoint connections need to be set up for both blob and dfs endpoint of the storage account if you have an ADLS Gen2 account or when utilizing a DFS endpoint on a blob account, otherwise private endpoint configuration on DFS endpoint is not required for non-ADLS Gen2 account.
Please configure the private endpoint of the storage account in the VNet named "vnetforstorage," ensuring it is in the same virtual network as the virtual machine. However different subnets can be used for private endpoint connections for blob and dfs endpoints of the storage account.
Navigate to your Azure storage account on the Azure Portal. Locate the Networking settings under Security + networking blade. Click on “+ Private endpoint” to create a private endpoint connection.
Following the steps mentioned here, create a private endpoint connection for the blob and dfs endpoint of the storage account.
Further, you will see that private endpoint connections have been created for both the Blob Storage Account private endpoint and the ADLS Gen2 private endpoint.
Kindly note that when creating a Private Endpoint, the related A record will automatically be created in the target Private DNS Zone with the private IP address of the network interface associated to the Private Endpoint and the name of the Azure resource referenced by the Private Endpoint.
Quick Tips:
- If you missed to configure private endpoint for both the blob and dfs endpoints of the storage account, particularly in the case of an ADLS Gen2 account or when using a DFS endpoint on a blob account then certain features might work as expected while others might encounter issues. This is because several APIs work on specific endpoints. Therefore, we recommend you set up private endpoints for both endpoints of the storage account.
- You can also test connectivity from a client to a storage account’s blob and dfs endpoints using the following PowerShell command on Windows.
OR the following Telnet command on Linux.
If the private endpoint is configured correctly, but the client is still unable to resolve the storage endpoint to the correct private IP, there could be a potential DNS resolution issue or a blocker in the network components between the client and server, such as a firewall or proxy, causing the problem. Refer to Azure Private Endpoint private DNS zone values | Microsoft Learn and Azure Private Endpoint DNS integration | Microsoft Learn for DNS related issues.
Step4. Again, try to access the storage account container within the virtual machine using Azure storage explorer to verify connectivity. The connection has been successfully established.
Step5. The CallerIpAddress field in the storage logs shown below also indicates that connectivity has been established from the virtual machine, which has a private IP address within the same virtual network.
3) Connecting to storage account from an Azure PaaS Service – ADF
In the following section, we will outline the steps for connecting to an Azure storage account that has public network access disabled, specifically from an Azure Platform as a Service (PaaS) offering such as Azure Data Factory.
Step1. Navigate to your Azure Data Factory instance and Launch Studio.
Step2. Create a new linked service from ADF, search for Azure Data Lake Storage Gen2 and select the Azure Data Lake Storage Gen2 connector.
Step3. Configure the linked service details such as Name, Integration Runtime, select Data Lake Storage Account and test the connection. The test connection failed, showing an 'ADLSGen2ForbiddenError' error.
If Azure monitoring logs are enabled for the storage account, you can filter Storage logs stored in the Log Analytics workspace for the storage request ID mentioned in the error message, thereby confirming the occurrence of the same error.
There could be two potential root causes for this error:
- The integration runtime is blocked by network access in Azure storage account firewall settings.
- The service principal or managed identity doesn’t have enough permission to access the data.
Step4. Further, upon checking the Azure storage account network/firewall settings, we identified that the issue was caused due to restricted network access on the storage account. Public network access is disabled in the storage account, and when this setting is disabled, the solution is to use a managed virtual network integration runtime and create a private endpoint to access to the storage account from Azure Data Factory.
Step5. Create an Azure, self-hosted integration runtime within a managed virtual network.
There are two ways to enable managed virtual network in your data factory:
- Enable managed virtual network during the creation of data factory. https://learn.microsoft.com/en-us/azure/data-factory/tutorial-deploy-ssis-virtual-network#configure-a-virtual-network
- Enable managed virtual network in existing integration runtime. https://learn.microsoft.com/en-us/azure/data-factory/tutorial-managed-virtual-network-migrate
Creating an integration runtime within a managed virtual network ensures the data integration process is isolated and secure.
Step6. Create a new Managed private endpoint. Managed private endpoints are private endpoints created in the Data Factory managed virtual network that establishes a private link to access Azure platform as a service (PaaS) services like Azure Storage, Azure Cosmos DB, and Azure Synapse Analytics.
Follow the steps Managed virtual network and managed private endpoints - Azure Data Factory | Microsoft Learn to configure a managed private endpoint.
When a managed private endpoint in Data Factory is created, it is created in a Pending state and an approval workflow is initiated for the Data Lake Storage account. The private link resource owner needs to approve the connection to establish the private link. Otherwise, the private link won't be established.
Navigate to the private endpoint connections in your storage account network settings and approve the connection established through ADF.
Step7. Edit Linked service and connect via integration runtime within a managed virtual network and test connection. The connection was successfully established.
Conclusion
Disabling public network access to your storage accounts can help you enhance your data security and compliance. In this blog post, we have discussed the effects and troubleshooting tips for using storage accounts with disabled public network access. We hope that this guide will help you manage your storage accounts more effectively and securely.
Hope this helps!!