For most of the last two decades, a server room had three kinds of box: servers, a storage array, and the switches joining them. Hyperconverged infrastructure, usually shortened to HCI, collapses the first two. Every node in the cluster carries its own CPUs, memory and disks, and a software layer pools the disks across nodes into one shared storage system. The array disappears.
What “compute and storage in the same chassis” means
In a traditional design, a virtual machine runs on one host but its disk lives on a separate array reached over a storage network. The array is a single, expensive, dual-controller device, and its performance ceiling is the performance ceiling of the whole environment.
In an HCI cluster, the virtual machine still runs on one host, but its disk is split into chunks and written to several nodes at once. Add a node and you add compute, storage capacity and storage throughput together. Lose a node and the other nodes already hold copies of its data. The hypervisor on each node and the storage daemons on each node are peers; there is no controller to become the bottleneck or the outage.
That is the promise. It is real, but it comes with a condition: the software doing the distribution must be configured for the hardware it is running on, and the network between nodes becomes part of the storage path.
How distributed storage spreads the risk
Open-source distributed storage, the kind that underpins most serious HCI, works by treating every physical disk as an object storage daemon, shortened to OSD. Data is broken into objects, objects are grouped into placement groups, and placement groups are mapped across OSDs according to rules that describe your failure domains: disk, host, rack, room.
The standard approach keeps three replicas of each object on three different hosts, so that any single host can fail without data loss and without interrupting service. Erasure coding is an alternative for capacity-heavy, performance-light data, trading CPU for disk efficiency. Either way, recovery from a failure is automatic: the cluster notices a missing OSD, and the surviving nodes rebuild the missing copies in the background.
Why default installs disappoint
Most people’s first HCI cluster is built from a quick-start guide, and the result is often slower than the old array it replaced. That is not a flaw in the concept. It is the consequence of defaults chosen to work everywhere rather than to work well anywhere.
The problems we see most often:
- one flat network carrying VM traffic, client storage traffic and replication traffic together, so a rebuild after a disk failure starves the virtual machines
- consumer or QLC SSDs behind the OSDs, which look fast in a benchmark and collapse under sustained mixed writes
- OSD metadata and write-ahead logs left on the same slow device as the data they describe
- placement group counts left at defaults that make no sense for the actual disk count
- memory and CPU not reserved for the storage daemons, so they compete with the workloads they serve
Each is fixable. None is fixed by default.
The tuning that matters
The single most important decision is separating the cluster network from the client network. Replication and recovery traffic goes over a dedicated back-end fabric, as fast as the budget allows and with jumbo frames enabled, while the hypervisors reach the storage over a separate front-end. When a disk fails in the small hours and the cluster starts moving terabytes around, the virtual machines do not notice.
The second is OSD layout. Each OSD should sit on a device with power-loss protection and honest sustained write performance. Where spinning disks or slower flash are used for capacity, the metadata database and write-ahead log for those OSDs belong on a small, fast NVMe partition. Getting this split right often matters more than the raw disk count.
Then there are the quieter settings: placement group sizing matched to the OSD count and the expected growth, CPU pinning so the storage daemons and the management domain are not fighting the guests, and recovery throttles set so a rebuild finishes in hours without flattening production.
Sizing and failure domains
An HCI cluster should be sized for what it looks like after a failure, not before. Three nodes is the minimum for three-way replication, but a three-node cluster with one node down has nowhere to rebuild and is one more failure away from losing data. Four or five nodes gives the cluster room to heal. Memory should be planned with the storage daemons’ own footprint included, which is substantial and grows with the number of disks.
Failure domains should reflect physical reality. If two nodes share a power feed or a top-of-rack switch, the placement rules should know that, so replicas are not all sitting behind the same single point of failure. In our experience this is the step most often skipped, and it is the one that decides whether a cluster survives its first real hardware fault gracefully.
Where Bizix fits
Bizix builds hyperconverged private cloud on an open-source hypervisor and distributed storage stack, with the network separation, OSD layout and failure-domain rules described here designed in from the start. Our Operator UI sits on top so the cluster can be run day to day without a storage specialist on every shift.