AKS on Azure Stack HCI and Windows Server 2023-01-30 Update

Hello everyone,

I'm so excited to share our first release of 2023!  This release is jam-packed with new pre-checks before install, improvements to Windows Admin Center flows, and we published a ton of great documentation over the winter holiday.  In case you were wondering about the release name, we're moving to year-month-day release names to align with AKS in Azure more closely.

Before getting into the update details, we have a few Announcements:

• If you shut down your AKS clusters over the holiday or skipped the November update, you may have internal certificates that expired. Follow these steps to recover your AKS cluster from expired certificates.
• We are retiring AKS hybrid builds that are more than a year old.  Please upgrade to our new release (or at least something newer than March '22) and remember to update cluster Kubernetes versions.  Supported Kubernetes versions for AKS hybrid 

Ok!  On to new features and things to check out.

As always, you can try AKS on Azure Stack HCI or Windows Server any time using our get-started guide.  If you do not have the hardware handy to evaluate AKS on physical hardware you can use our eval guide to set up AKS on a Windows Server Azure VM.

Install pre-checks for AKS on Azure Stack HCI and AKS on Windows Server

We have heard your feedback - you told us that you have seen some specific problems when installing AKS hybrid on your system. To address those problems, in the last 2 releases, we have been adding validation tests as part of  Set-AksHciConfig,  Set-AksHciRegistration, and Install-AksHci to ensure that your system and configuration values will work prior to proceeding with the installation.

Now you will see the following tests prior to install:

• MOC Host Internet Connectivity
• MOC Host Limits
• MOC Host Remoting
• MOC Network Configuration
• MOC SDN Configuration
• MOC directories
• Failover Cluster Health
• Failover Cluster HCI Registration
• VM Creation in Hyper-V
• Availability of a Switch in Hyper-V
• AKS Management cluster configuration
• Azure user permissions
• AD permissions
• Proxy configuration

We plan to continue adding validation tests in the coming releases.

Windows Admin Center UI Improvements

There are three great updates in Windows Admin Center this month!

First, you can now edit the details of existing node pool configurations in the Kubernetes cluster creation wizard. 

Second, we updated DNS server field on the Host configuration page to include a field by default (before you needed to click “Add” for the first field to appear even though it's required).

Last but not least, Resource group and Azure region fields on the Azure registration pages are easier to get right.

• New Resource Groups now default to EastUS (rather than blank)
• The Resource Group and Azure Region fields changed to the “ComboBox” element, which means you can search if you have a ton of different Resource Groups to choose from.

Documentation updates.  Tons of doc updates.

Seriously, we have so many updates to documentation and net new documentation I hope you'll find useful.

We published the Azure security baseline for Azure Kubernetes

This security baseline applies guidance from the Microsoft cloud security benchmark version 1.0 to Azure Kubernetes Service on Azure Stack HCI and Azure Kubernetes Service on Widows Server. The Microsoft cloud security benchmark provides recommendations on how you can secure your cloud solutions on Azure.

Azure security baseline for Azure Kubernetes Service on Azure Stack HCI | Microsoft Learn

New guidance for recovering AKS hybrid from management cluster failure.

In AKS on Azure Stack HCI or Windows Server, the management cluster is deployed as a single standalone virtual machine (VM) per deployment, making it a single point of failure. That said, a management cluster outage has no impact on applications running in the workload clusters. When the management cluster VM fails, the workload clusters (and workloads) continue running, but you won't be able to perform day-2 operations until the VM is restored. In addition, the management cluster is a VM protected by Windows failover clustering so it is also resilient to host-level disruptions. In other words, during a host machine failure, Windows failover clustering restarts the VM on a healthy host machine.

We have published the process for restoring AKS hybrid clusters from a disaster article to outline restoring AKS on new hardware (could be a new site) and how to recover from corruption of the management cluster.

New and updated content:

• New doc with an overview of AKS hybrid install pre-checks - Validation tests in AKS hybrid
• New reference to help with disaster recovery - Process for restoring AKS hybrid clusters from a disaster

• New - Azure security baseline for Azure Kubernetes Service on Azure Stack HCI
• There have been updates to the repair certificates article to be more inclusive of more certs with different lifetimes - Certificates and tokens in AKS hybrid
• Updated Adapt applications for use in mixed-OS Kubernetes clusters in AKS hybrid, Use multiple node pools in AKS hybrid, to use the -osSKU in all of the examples parameters

Troubleshooting guide updates:

• kubectl logs return "error: You must be logged in to the server (the server has asked for the client to provide credentials)"
• Get-AksHciCredential fails with "cannot find the path specified" error
• Cluster auto-scaling fails
• Target cluster pod logs are not accessible - remote error: tls: internal error
• Arc-connected clusters have empty JSON "distribution" property

Bug fixes:

• If you're using SDN for AKS hybrid
• you can now upgrade your AKS clusters this month.   
• we have resolved the 80-character limit bug for some names

• Fix for CVE-2022-32149 for kube-state-metrics and ghosttunnel updates
• Fix for GPU-enabled node pools described in GitHub Issue #272
• PowerShell will now propagate PS config when adding a new node. 
• Improvements around recurring certificate errors caused by environmental Variables not being set before KVA commands which causes the old certificate being used.

Once you have downloaded and installed the AKS on Azure Stack HCI or Windows Server Update – you can report any issues you encounter and track future feature work on our GitHub Project at  https://github.com/Azure/aks-hci.

We look forward to hearing from you all!

Cheers,

Sarah