In this chapter, we will discuss the following topics, with a recipe for each:
Using the VMware Compatibility Guides
Selecting a server platform
Selecting a storage controller
Selecting a solid-state drive (SSD) for the cache tier
Selecting hard disk drives/magnetic disks
Deciding on a network standard
Choosing a VSAN Ready Node (an alternative option)
VSAN can only be as good as the hardware on which it runs. VSAN has special requirements for its hardware, and so elements of your VSAN hardware will need to be selected against a stricter subset of the overall VMware Compatibility Guide. Each element of your future VSAN node will need to be compliant with the vSphere and VSAN Compatibility Guides to be production-ready and ensure that the configuration will be supported by VMware.
Depending on your specific needs, you may find that the VSAN Ready Node will better fit your needs. VSAN Ready Nodes are preconfigured systems built by the VMware hardware partners to be VSAN ready, so the machines can simply be purchased and deployed.
You will need to decide whether specifying your own system configuration, or selecting a VSAN Ready Node, is the best choice for your infrastructure. This chapter will prepare you for either choice. VSAN Ready Nodes are covered in the last recipe in this chapter.
While most system vendors offer hardware that is compatible with VMware vSphere, only a subset of the hardware in the VMware Compatibility Guide is applicable for use for VSAN. It is important to make sure that all applicable hardware exists in both guides. The storage-specific components like hard-disk controllers and disks must meet the more-exhaustive performance and reliability requirements for VSAN.
You should have some idea of your hardware requirements. This includes an estimate of your needs for system memory (RAM), along with processor power, networking needs, and storage requirements in terms of capacity and performance.
Go to your web browser and navigate to both the regular VMware vSphere Compatibility Guide at http://www.vmware.com/resources/compatibility/search.php and the VMware VSAN Compatibility Guide at http://www.vmware.com/resources/compatibility/search.php?deviceCategory=vsan.
The first link will take you to the standard VMware Compatibility Guide, and the second will take you to the VSAN sub-guide. By default, you will land on the system/server page, to help you select a server platform.
For VSAN, the only requirement in terms of the server platform is that it needs to appear in the regular VMware Compatibility Guide for vSphere 5.5 or 6.0, as applicable to your deployment. Any compatible/certified server is acceptable for use with VSAN.
You will need to filter your selection to restrict output to only the relevant results. To do this, carry out the following steps:
Within Product Release Version:, select the most recent vSphere ESXi 5.5 or ESXi 6.0 update release.
If you have a brand preference, select it within Partner Name.
While most systems in the Compatibility Guide are rack mount systems, if you have other form-factor needs, select those within System Type.
If you need dedicated or expanded functionality (such as graphics acceleration for VMware Horizon View or I/O redirection), make that selection within Features.
Select your CPU socket/core and/or brand needs within the Sockets:, Max Cores per Socket:, and CPU Series sections.
VSAN requires at least two CPU cores across one or more sockets. Any field left blank assumes an inclusive search and all subcategories will be included.
Once you have selected all of your requirements, click on Update and View Results to see a list of all the compatible systems meeting your criteria.
Example of the compatibility guide once filters are applied:
VSAN does also impose additional resource needs. Depending on the size of the disks you choose and how many, VSAN will consume additional compute and memory resources. If you typically run your systems close to the margins, in terms of compute/memory resources (or if you are planning to with your new cluster), consider specifying a modest amount of additional CPU and memory in the configuration. VSAN is capped at 10 percent resource utilization for CPU and memory as a maximum, so adding at least 10 percent to your assumed consumption should leave you with comfortable operating margins.
Additionally, VSAN requires a local boot device, either an SD card, or other low-performance solid-state memory, or a dedicated hard disk drive, or SSD. The boot device should be at least 16GB, in accordance with standard VMware recommendations. If your server configuration equals/exceeds 512 GB of system RAM, then you must use a hard disk drive or SSD as your boot device to ensure supportability, otherwise, a core dump cannot be written in the event of a system crash.
Selecting an appropriate storage controller is one of the most important decisions you will make when creating a VSAN server configuration. The storage controller has tremendous weight in terms of I/O performance and reliability. Because of the importance of the storage controller on the overall performance and reliability of your VSAN cluster the storage controller must be selected from the VSAN-specific subset of the overall VMware Compatibility Guide.
The initial landing page for the VSAN Compatibility Guide will allow you to browse through VSAN Ready Node configurations. As we will be discussing the case of VSAN Ready Nodes later in this chapter, for now please click the "Build Your Own based on Certified Components" link from the main page of the VSAN Compatibility Guide:
Unlike the regular VMware Compatibility Guide, the VSAN-specific component guide is laid out as a branching tree that you can use to drill down to your desired configuration, after which time hardware choices will be displayed.
In the leftmost pane, select I/O Controller.
In the next pane, select the most-recent vSphere 5.5 or 6.0 update release.
In the next pane, select a brand name if desired.
In the next pane, specify whether you wish to view the SAS, SAS-RAID, or SAS/SATA RAID controller types.
Tip
Some controllers for VSAN support the pass-through (JBOD) mode, and some require you to create single-disk RAID-0 sets. Pass-through controllers tend to be easier to configure and make drive-replacement simpler. RAID-0 controllers are more common and typically have larger queue depths. When selecting controllers and disks, please keep in mind the performance and reliability differences between drive technologies. Generally speaking, SAS and Nearline SAS (NL-SAS) disks have deeper queue depths and are higher-performance. SAS disks are typically more reliable than their SATA and NL-SAS counterparts.
After making your selections, choose Update and View Results to get a list of hardware that matches your specifications.
Aside from simply ensuring that you select a supported storage controller, the VSAN-specific Compatibility Guide also provides you with additional detail that will be crucial to selecting your storage controller.
This view will tell you whether or not the controller supports the pass-through/JBOD mode and, vitally, it will tell you the controller's command queue depth. The controller's queue depth is a vital consideration. If each node is relatively small with one disk group, any queue depth above 256 commands is acceptable. If you have two disk groups, please consider 512 commands to be the minimum queue depth.
For large or high-performance configurations with several disk groups and extremely large capacities, and/or very high-performance SSDs, you should choose the highest queue depth you can. Standard controllers with maximum queue depths usually have around 1000 commands, like in the preceding example.
Queue depth can have a tremendous effect on the performance of streaming data. Opting for controllers and disks with deeper queues will provide better overall performance, particularly when VSAN is reconfiguring or resynchronizing data.
In combination with the storage controller, the SSD for the cache tier is the most important choice you can make in terms of the long-term reliability and performance of your VSAN deployment. The cache-tier SSD is used for caching reads and writes in VSAN hybrid configurations, and for caching writes only in all-flash configurations. SSDs are graded in many ways, and these grades for performance, write resiliency, and fabrication technology will all affect your selection. The VSAN Compatibility Guide gives you an overview of all of these factors to help you make the best choice.
Note
For the SSD:
The write-performance class is on a scale of A-F, with the F class being the fastest (class A is technically deprecated).
Write-resiliency is on a scale of A-D, with D being the most resilient.
Fabrication technology is either multi-layer cell (MLC), or single-layer cell (SLC). SLC is usually more performant and resilient, at the cost of lower capacities and higher price.
SSDs come in either SATA, SAS, or PCIe connections. PCIe SSD cards are usually at the top end in terms of performance. Given that SAS and PCIe interfaces permit significantly higher device command queues, it is a good idea to use SAS or PCIe type SSDs, especially if you are using a pass-through/JBOD controller that directly leverages device-level command queues.
The Compatibility Guide for SSD is navigated in the same way as for the I/O controller.
In the leftmost pane, select SSD.
In the next pane, select the most recent vSphere 5.5 or 6.0 update release.
In the next pane, select a brand name if desired.
Specify whether are searching for All Flash or Hybrid cache tier.
In the various other fields and drop-down menus, you can select the interface type, manufacturer, performance class, capacity, and form-factor.
After making your selections, choose Update and View Results to get a list of hardware that matches your specifications:
Determining the size of your capacity-tier (magnetic or SSD) storage will directly affect the sizing decision for your cache-tier SSD. In general, the SSD should be 10 percent of the size of the magnetic disks in the underlying disk group, for example, if your disk group will consist of four 1.5 TB capacity-tier disks, you will have 6 TB of bulk storage. To accommodate this, the SSD should be about 600 GB. Alternatively, you may choose to opt for two disk groups consisting of three 1TB capacity disks each. In this case, each disk group should have a cache-tier SSD of 300GB. The ratio of cache-tier SSD to capacity-tier disks or SSDs should be approximately 1:10.
For additional guidance regarding how to appropriately size the VSAN capacity and cache tiers, please see the section Chapter 1 – VSAN Capacity Planning of Appendix A, Chapter-specific Expansions.
The magnetic disks or SSDs you choose will be used for storage capacity and persistent data that is destaged from cache. This is the capacity tier within VSAN, whereas the caching tier SSD will act as the performance caching layer.
In general, you will want to select magnetic disks or SSDs that have adequate capacity to fit your needs. For highly dynamic workloads where data will be frequently destaged from the SSD write buffer and fetched into the SSD read cache, HDD performance is important and you may wish to go with faster disks and/or SAS disks. Only SAS and SATA disks are supported for use with VSAN.
The Compatibility Guide for SSD is navigated in the same way as for the I/O controller
In the leftmost pane, select HDD or SSD.
In the next pane, select the most recent vSphere 5.5 or 6.0 update release.
In the next pane, select a brand name if desired.
In the various other fields and drop-down menus, you can select the interface type, manufacturer, disk speed (RPM), capacity, and form-factor.
If we are pursuing an all-flash VSAN configuration, ensure that we select Virtual SAN All Flash Capacity Tier from the Tier: pane.
After making your selections, choose Update and View Results to get a list of hardware that matches your specifications.
In general, SAS disks outperform SATA disks of equivalent capacities and/or rotational speeds because SAS drives use more robust recording technique, deeper queues or both. When cost is a concern, slower SAS drives (typically 7200 RPM; also called near-line SAS or NL-SAS) are usually built on cheaper SATA platforms but include enterprise-grade features like deeper command queues, error-correction, dual-channel connections and native SCSI support. Low-end SAS drives are typically better than high-end SATA drives despite the shared technology platform and costs are usually not significantly higher. NL-SAS is a great alternative to SATA for building out a cost-conscious capacity tier when HDD performance is a factor.
Before settling on a Cache + Capacity disk combination, please review the Chapter 1 – VSAN Capacity Planning section of Appendix A, Chapter-specific Expansions for a verbose description of the capacity expectations and recommended maximums to help you build your VSAN cluster to an appropriate scale.
For smaller clusters, network speed is typically only a forefront concern during times of data reconstruction in the event of a node/disk failure, rebalancing, or user-invoked configuration changes. These are generally fairly rare operations and, for the most part, 1GbE networking will be adequate for clusters with fewer than 5 nodes. 10GbE networking is recommended as a VMware best practice for all clusters. 10GbE networking should be considered mandatory for larger clusters, especially clusters of >8 nodes. If 1GbE network interfaces are being used for VSAN, those interfaces need to be dedicated to use by VSAN. 10GbE interfaces can be shared between VSAN and other workloads.
Tip
Unlike disks and storage controllers, there is no special compatibility guide for network controllers. Any vSphere-compatible network controller is acceptable for use with VSAN.
Within Product Release Version:, select the most recent vSphere 5.5 or 6.0 update release.
If you have a brand preference, select it within Brand Name.
Select Network, within I/O Device Type.
After making your selections, choose Update and View Results to get a list of hardware that matches your specifications.
Before settling on a networking standard, please take a look at a detailed description of the VSAN networking options and recommendations in the VSAN Network Considerations section of Appendix B, Additional VSAN Information.
To help simplify the hardware selection process, numerous VMware partners have preconfigured "VSAN Ready Nodes" that are compliant with the various aspects of the vSphere and VSAN compatibility guides. With VSAN Ready Nodes, the sizing decisions, disk/SSD technology, network standards, and so on, have already been determined by the systems vendor. You may find that a VSAN Ready Node will simplify your purchasing and designing decisions for new VSAN build-outs and/or if you have a preferred system vendor.
Note
VSAN Ready Nodes are configured by the manufacturer and are designed to target specific infrastructure scales, or for specific use cases, such as virtual desktops (VDI). These scales are known as Ready Node Profiles.
A matrix that defines these profiles can be found by clicking on the Virtual SAN Hardware Quick Reference Guide link at the top of the VSAN Compatibility Guide page.
The VSAN Compatibility Guide for VSAN Ready Nodes is navigated in a similar manner to the component-oriented Compatibility Guide. The guide is presented as a series of drill-down categories.
Select the parameters that fit your needs from the Ready Node type (All Flash or Hybrid), preferred manufacturer, performance/scale profile, form-factor, etc.
Once your choices have been selected, click the "Update and View Results" button to populate a list of compliant ready-node configurations:
The initial output will be a collapsed list of the various Ready Node configurations that are compliant with your selected filters. The output is initially grouped by vendor.
To get additional information about the specific Ready Node configuration, twirl down the arrow next to the vendor that you wish to use and then you can expand configuration information about the specific server platform that is recommended:
You can then use the SKU number (pictured above) when you contact your preferred vendor to get pricing and ordering information for the Ready Node.
As with user-specified build-outs, please review the Chapter 1 – VSAN Capacity Planning section of Appendix A, Chapter-specific Expansions for a verbose description of the capacity expectations and recommended maximums, to help you select an appropriate VSAN Ready Node.
