by Since the introduction of vSAN Encryption in vSAN 6.6 some of the routine questions that get asked include:
- Do vSAN hosts get encryption key information from vCenter?
- What if vCenter is offline, is vSAN encryption impacted?
- What if vCenter is removed/replaced/not available, is vSAN impacted?
The answers to these are pretty straightforward and easy to answer. To better understand the answers to those questions, it is important to understand the requirements and workflow of setting up vSAN Encryption.
Key Management Solution (KMS)
vSAN Encryption requires a KMS that is compliant with the KMIP 1.1 protocol. As of today, 9 different vendors offering 21 different solutions have been certified for use with vSAN Encryption. Those can be found here. Other KMS’es that are compliant with the KMIP 1.1 protocol are supported though.
The KMS must have connectivity with the vCenter Server and each of the vSAN hosts that will have encryption enabled.
vCenter
vCenter Server must be a version that is supported for use with the vSAN version. vSAN Encryption is available in vSAN 6.6 and vSAN 6.7. Refer to the VMware Product Interoperability Matrices to ensure that the vCenter and vSAN versions are compatible.
The creation of a Key Management Server profile in vCenter Server creates connection information used by vSAN hosts. As part of the profile creation process, a trust is established between the vCenter Server and the Key Management Solution. The Key Management Server profile allows vSAN Encryption the ability to use keys from the Key Management Solution.
When vSAN Encryption is enabled, vCenter requests two keys from the Key Management Solution for use by the encrypted vSAN Cluster. The Key Encryption Key (KEK) is used to encrypt each Data Encryption Key (DEK) on each vSAN storage device, and the Host Key is used by each host to encrypt core dumps.
vSAN Hosts
The Host Key and KEK are not stored on vSAN hosts, but rather stored in the key cache after being requested by the vSAN host when vSAN Encryption is enabled. When a vSAN host reboots, these keys are discarded. When a vSAN host reboots, because the Host Key and KEK are not present, they must be requested directly from the Key Management Server. The Key Management Server profile, Host Key Id, and KEK Id information stored in /etc/vmware/esx.conf is used to request the Host Key and KEK.
Here are some of those values found in /etc/vmware/esx.conf for vSAN Encryption:
When vSAN Encryption is enabled, or when a deep rekey operation is invoked, the vSAN host creates a unique DEK (XTS-AES-256) for each device, and it is encrypted with the KEK. A shallow rekey operation swaps out the KEK and rewraps each DEK.
When a host with vSAN Encryption enabled attempts to mount a vSAN Disk Group, the DEK is unwrapped using the KEK, allowing vSAN to mount and then use the vSAN Disk Group.
The Boot Process
So what happens if a vSAN Cluster is completely offline? How does the boot process work, and how are VMs brought back online when vSAN Encryption is in place?
Notice in this illustration that hosts are powered off as well as all of the virtual machines, including vCenter. The KMS Server is online and external to the vSAN Cluster.
vSAN hosts are then booted. Disks are not immediately mounted, and VMs are offline (including vCenter).
After a vSAN host boots, it will read the values in /etc/vmware/esx.conf and request the KEK and Host Key from the KMS using the KEK Id and Host Key Id respectively, directly from the KMS. 
The KEK and Host Key are placed in memory in the key cache. These keys are not persistently stored on the vSAN hosts. The KEK is then used to mount the vSAN Disk Groups and VMs may be powered on.
Something to consider
If the KMS were residing on top of vSAN, a circular dependency would occur. This is because the KMS is needed to provide the KEK to the vSAN hosts. This is covered in the Understanding vSAN Encryption – KMS Server Accessibility post here:
https://blogs.vmware.com/virtualblocks/2018/05/17/vsan-enc-kms-accessibility/
Summary
As long as vSAN hosts using vSAN Encryption have connectivity to their configured KMS, they have no issue booting, even when vCenter is offline. The boot process is not dependent on vCenter to unlock and mount vSAN Disk Groups.
And what happens if you lose access to vCenter long term?
Well Dave Morera has a post on how to restore vCenter that manages a vSAN Encrypted cluster here: https://greatwhitetec.com/2017/06/06/replacing-vcenter-with-vsan-encryption-enabled/
This was originally posted on the VMware Virtual Blocks site: https://blogs.vmware.com/virtualblocks/2018/07/13/understanding-ve-booting-w-vc-unavailable/
