How do AKS and AKS on Azure Stack HCI compare?

Kubernetes Management Cluster/AKS host

AKS on Azure Stack HCI and Windows Server is a Cluster API based hosted Kubernetes offering. A management Kubernetes cluster is used to manage Kubernetes workload clusters. The management Kubernetes cluster runs in customer datacenters and is managed by the infrastructure administrator.

AKS is a managed Kubernetes offering. AKS control plane is hosted and managed by Microsoft. AKS worker nodes are created in customer subscriptions.

Kubernetes Target Cluster

(lifecycle operations)  

Cloud Native Computing Foundation (CNCF) certification

Yes

Yes

Who manages the cluster?

Managed by you

Managed by you

Where is the cluster located?

In your datacenter alongside your AKS hybrid management cluster.

Azure Stack HCI 21H2

Windows Server 2019 Datacenter

Windows Server 2022 Datacenter

Windows 10/11 IoT Enterprise*
Windows 10/11 Enterprise*
Windows 10/11 Pro*

Azure cloud

K8s cluster lifecycle management tools (create, scale, update and delete clusters)

PowerShell (PS)

Windows Admin Center (WAC)

Az CLI*

Azure Portal*

ARM templates*

Az CLI

Az PowerShell

Azure Portal

Bicep

ARM templates

Can you use kubectl and other open-source Kubernetes tools?

Yes

Yes

Workload cluster updates

K8s version upgrade through PowerShell or WAC. Initiated by you.

Node OS image update initiated by you;

Updates in a target cluster happen at the cluster level – control plane nodes + node pools updated.

Azure CLI, Azure PS, Portal, ARM templates, GitHub Actions;

OS image patch upgrade;

Automatic upgrades;

Planned maintenance windows;

Kubernetes versions

Continuous updates to supported Kubernetes versions. For latest version support, visit AKS hybrid releases on GitHub.

Continuous updates to supported Kubernetes versions. For latest version support, run az aks get-versions.

Can you start/stop K8s clusters to save costs?

Yes, by stopping the underlying failover cluster

Yes

Azure Fleet Manager integration

Not yet.

Yes*

Terraform support

Not yet.

Yes

Node Pools

Do you support running Linux and Windows node pools in the same cluster?

Yes!

Linux nodes: CBL-Mariner

Windows nodes:

Windows Server 2019 Datacenter, Windows Server 2022 Datacenter

Yes.

Linux nodes: Ubuntu 18.04, CBL-Mariner

Windows nodes:

Windows Server 2019 Datacenter

Windows Server 2022 Datacenter

What’s your container runtime?

Linux nodes: containerd

Windows nodes: containerd

Linux nodes: containerd

Windows nodes: containerd

Can you scale node pools?

Manually

Cluster autoscaler

Vertical pod autoscalar

Manually

Cluster autoscaler

Vertical pod autoscalar

Horizontal pod autoscalar

Yes

Yes

What about virtual nodes?

Azure container instance

No

Yes

Can you upgrade a node pool?

We do not support upgrading individual node pools. All upgrades happen at the K8s cluster level.

You can perform node pool specific upgrades in an AKS cluster.

GPU enabled node pools

Yes*

Yes 

Azure Container Registry

Yes

Yes

KEDA support

Not yet

Yes*

Networking

Who creates and manages the networks?

All networks (for both the management cluster and target K8s clusters) are created and managed by you

By default, Azure creates the virtual network and subnet for you. You can also choose an existing virtual network to create your AKS clusters

What type of network options are supported?

DHCP networks with/without VLAN ID

Static IP networks with/without VLAN ID

SDN support for AKS on Azure Stack HCI 

Bring your own Azure virtual network for AKS clusters.

Load balancers

HAProxy (default) runs in a separate VM in the target K8s cluster

kubeVIP – runs as a K8s service in the control plane K8s node

Bring your own load balancer

Load balancers are always given sIP addresses from a customer vip pool to ensure application and K8s cluster availability.

You can create multiple instances of a LB (active-passive) for high availability

Azure load balancer – Basic SKU or Standard SKU

Can also use internal load balancer

By default, load balancer IP address is tied to load balancer ARM resource. You can also assign a static public IP address directly to your Kubernetes service

CNI/Network plugin

Calico (default)

Note: Network policies are covered in the Security and Authentication section.

Azure CNI

Calico

Azure CNI Overlay

Bring your own CNI

Note: Network policies are covered in the Security and Authentication section.

Ingress controllers

No but you can use 3^rd party addons – Nginx. 3^rd party addons are not supported by Microsoft’s support policy.

Support for Nginx with web app routing addon.

Egress controls

Egress is controlled by Network policies, by default all outbound traffic from pods is blocked. You can deploy additional egress controls and policies.

You can use Azure Policy and NSGs to control network flow or use Calico policies. You can also use Azure FW and Azure Security Groups.

Egress types

Egress types and options depend on your network architecture.

Azure load balancer, managed NAT gateway and user defined routes are the supported egress types.

Customize CoreDNS

Allowed

Allowed

Service Mesh

Yes, Open Service Mesh (OSM) through Azure Arc enabled Kubernetes.

3^rd party addons – Istio, etc. 3^rd party addons are not supported by Microsoft’s support policy.

Open Service Mesh

Marketplace offering available for Istio

Storage

Where is the storage provisioned?

On-premises

Azure Storage.

Azure Files and Azure Disk premium CSI drivers deployed by default. You can also deploy any custom storage class.

What types of persistent volumes are supported?

Read Write Once

Read Write Many

Read Write Once

Read Write Many

Do the storage drivers support Container Storage Interface (CSI)?

Yes

Yes

Is dynamic provisioning supported?

Yes

Yes

Is volume resizing supported?

Yes

Yes

Are volume snapshots supported?

No

Yes

Security and Authentication

How do you access your Kubernetes cluster?

Certificate based kubeconfig (default)

AD based kubeconfig

Azure AD and Kubernetes RBAC

Azure AD and Azure RBAC*

Certificate based kubeconfig (default)

Azure AD and Kubernetes RBAC

Azure AD and Azure RBAC

Network Policies

Yes, we support Calico network policies

Yes, we support Calico and Azure CNI network policies

Limit source networks that can access API server

Yes, by using VIP pools.

Yes, by using the “-api-server-authorized-ip-ranges” parameter and private clusters.

Certificate rotation and secrets encryption

Yes

Yes

Support for private cluster

Not supported yet

Yes! You can create private AKS clusters

Secrets store CSI driver

Yes

Yes

Support for disk encryption

Yes, via bitlocker

Disks are encrypted on the storage side with platform managed keys and with support for customer provided keys.

Hosts and locally attached disks can also be encrypted with encryption at host.

gMSA v2 support for Windows containers

Yes

Yes

Azure Policy

Yes, through Azure Arc enabled K8s

Yes

Azure Defender

Yes, through Azure Arc enabled K8s*

Yes

Monitoring and Logging

Collect logs

Yes, through PS and WAC. All logs – management cluster, control plane nodes, target K8s clusters are collected.

Yes, through Azure Portal, Az CLI, etc

Support for Azure Monitor

Yes, through Azure Arc enabled K8s.

Yes

3^rd party addons for monitoring and logging

AKS works with Azure managed Prometheus* and Azure managed Grafana*

Subscribe to Azure Event Grid Events

Yes, via Azure Arc enabled Kubernetes*

Yes

Develop and run applications

Azure App service

Yes, through Azure Arc enabled K8s*

Yes

Azure Functions

Yes, through Azure Arc enabled K8s*

Yes

Azure Logic Apps

Yes, through Azure Arc enabled K8s*

You can directly create App Service, Functions, Logic Apps on Azure instead of creating on AKS

Develop applications using Helm

Yes

Yes

Develop applications using Dapr

Yes, through Azure Arc enabled K8s*

Yes

DevOps

Azure DevOps via Azure Arc enabled K8s.

GitHub Actions via Azure Arc enabled K8s.

GitOps Flux v2 via Azure Arc enabled K8s.

3^rd party addon: ArgoCD. 3^rd party addons are not supported by Microsoft’s support policy.

GitOps Flux v2 through Azure Arc enabled Kubernetes is free for AKS-HCI customers.

Azure DevOps

GitHub Actions

GitOps Flux v2

Product Pricing

Product pricing

If you have Azure Hybrid Benefit, you can use AKS-HCI at no additional cost.

If you do not have Azure Hybrid Benefit pricing based on number of workload cluster vCPUs. Management cluster, control plane nodes, load balancers are free.

Unlimited free clusters, pay for on-demand compute of the worker nodes.

Paid tier available with uptime SLA, support for 5k nodes.

Azure Support

AKS-HCI is supported out of the Windows Server support organization aligned with Arc for Kubernetes and Azure Stack HCI. You can open support requests through the Azure portal and other support channels like Premier Support.

AKS in Azure is supported through enterprise class support in the Azure team. You can open support requests in the Azure portal.

SLA

We do not offer SLAs since AKS-HCI runs in your environment.

Paid uptime SLA clusters for production with fixed cost on the API + worker node compute, storage and networking costs.