Connect to a Linux VM using Bastion and Key Vault without a private key file

In enterprise environments, requirements often demand the use of private networks, resulting in VMs not being assigned Public IP addresses. In such cases, Azure Bastion offers an extremely useful feature for accessing these VMs without complexity. When logging into Linux VMs on Azure via Azure Bastion using an SSH Private Key, the key is often managed as a local file. Managing SSH Private Key files on individual devices poses risks such as potential key leakage. Therefore, centralized management, including proper role assignment, is preferable from a security standpoint. A common need in such scenarios is to manage SSH Private Keys using Key Vault. This article provides such a comprehensive step-by-step guide.

There are two huge benefits here.

SSH Private Key is no longer required to store as a local file Local file by storing it in Azure Key Vault.
Access to Linux VM can be be controlled using RBAC assignment for Azure Key Vault.

Regarding the first point, storing the SSH Private Key locally after creating it through the Azure Portal can be issues such as loss the file or difficulty in identifying which VM it is for. Additionally, it has the benefit of reducing the risk of file leakage if a developer leaves the company or the project.
Regarding the second point, using Key Vault for access management allows you to control VM access through Key Vault. You can manage access control to specific VMs.

Here is the architecture diagram for the current setup. When creating a Linux VM, the SSH Private Key generated through the Azure portal is registered in Key Vault. This SSH Private Key is then used to access the Linux VM via Azure Bastion.

Prerequisites

Before following this article, you have to create Azure resources as follows:

Virtual Network
Key Vault
Azure Bastion

Setup

First, we create a new SSH Key when creating the Linux VM through the Azure portal. While it is possible to create an SSH Private Key when creating a Linux VM through the Azure portal, you can also create it separately. Once you input the parameters and create the Linux VM, you can download the SSH Private Key and save it as a local file.

If you choose to create the SSH Private Key separately, note that you need to select "Reset password" from the VM menu on Azure Portal and use "Add SSH public key" to set the SSH Key information for the Linux VM. The first link in the References section contains relevant information.

Next, you have to acquire proper RBAC role to manage Azure Key Vault. It's not enough just having subscription Owner role. In this article, we assign Key Vault Administrator role.

We can use Key Vault Secrets Officer role for developers who access your Linux VM. Please refer to the article as follows to assign proper RBA roles.

Provide access to Key Vault keys, certificates, and secrets with an Azure role-based access control

Then, we register the SSH Private Key in the secrets of Azure Key Vault (you cannot select Keys or Certificates with Azure Bastion). There is a tip here that the SSH Private Key contains a multi-line file, but the Key Vault portal does not support the registration of multi-line strings. Therefore, we have to execute this via command line as follows:

az keyvault secret set --name "your-secret-name" --vault-name "your-key-name" --file "my-ssh-private-key.pem"

If the command works well, you can find your SSH Private Key on the Azure Key Vault.

Then, we can access the Linux VM via Azure Bastion. Select the Linux VM, choose "SSH Private Key from Azure Key Vault" from the Authentication Type options, and use the the SSH Private Key you have registered.