What is the difference between Redundancy Factor and Replication Factor on Nutanix

What is the difference between the Redundancy Factor and the Replication Factor on Nutanix.

Redundancy Factor (aka FT – Fault Tolerance),  allows a Nutanix cluster to withstand the failure of two nodes or drives in different blocks at the same time. Nutanix Acropolis supports FT=1 (default) and FT=2.

Redundancy Factor 3 requirements:

• min 5 nodes in the cluster
• CVM with 32GB RAM configured
• For guest VM to tolerate a simultaneous failure of 2 nodes or 2 disks in different blocks, VM data must be stored on a container with replication factor 3

NOTE: Nutanix cluster with FT2 enabled, can host storage containers with RF=2 and RF=3.

Redundancy Factor 2 requirements:

• min 3 nodes in the cluster
• CVM with 24GB RAM configured

Nutanix cluster components.

Cassandra

• Key Role: Distributed metadata store
• Description: Cassandra stores and manages all of the cluster metadata in a distributed ring-like manner based upon a heavily modified Apache Cassandra.  The Paxos algorithm is utilized to enforce strict consistency.  This service runs on every node in the cluster.  The Cassandra is accessed via an interface called Medusa.
  Source: Nutanix Bible by Steven Poitras

Zookeeper

• Key Role: Cluster configuration manager
• Description: Zookeeper stores all of the cluster configurations including hosts, IPs, state, etc. and is based upon Apache Zookeeper.  This service runs on three nodes in the cluster, one of which is elected as a leader.  The leader receives all requests and forwards them to their peers.  If the leader fails to respond, a new leader is automatically elected.   Zookeeper is accessed via an interface called Zeus.
  Source: Nutanix Bible by Steven Poitras

So, what is an impact when you change Redundancy Factor from 2 to 3. With  Redundancy Factor 2 cluster keeps 3 copies of metadata (Cassandra) and Zookeeper. With Redundancy factor 3 enabled, cluster keeps 5 copies of Metadata and Zookeeper (configuration data).

How to change Nutanix cluster Redundancy Factor from FT2 to FT3.

$  ncli cluster get-redundancy-state           
    Current Redundancy Factor : 2
    Desired Redundancy Factor : 2
    Redundancy Factor Status  : kCassandraPrepareDone=true;kZookeeperPrepareDone=true
$

Go to Prism Element, click on the Gear button –> Redundancy State (from left pane).

Redundancy state

From a drop-down menu choose Redundancy Factor 3 and Save configuration

It may take some time to update the cluster to RF3 as the system has to increase the number of metadata replicas from 3 to 5. You can watch progress in Tasks on Prism Central or Prism Element

Update metadata

Verify if the cluster Redundancy factor changed via ncli

$  ncli cluster get-redundancy-state           
    Current Redundancy Factor : 3
    Desired Redundancy Factor : 3
    Redundancy Factor Status  : kCassandraPrepareDone=true;kZookeeperPrepareDone=true
$

NOTE: Changing the Redundancy Factor from 2 to 3 does not affect storage capacity on the cluster.

What is the Replication factor (RF)?

Nutanix Replication Factor states for a number of data copies on the Nutanix cluster (VM data and Oplog).  For example, if you have Replication Factor 2 (RF=2) set on the container, meaning every VM data block has 2 copies on the Nutanix cluster (including data in Oplog). With RF=3, all VM data has 3 copies of data on different blocks.

NOTE #1: You can have a container with RF=2 and another container with RF=3 configured on the same Nutanix cluster.
NOTE #2: Changing the container replication Factor from RF=2 to RF=3, consumes more storage space on the Nutanix cluster (because systems have to keep 3 copies of the VM data which are on containers with RF=3).

To read more about VM data placement on containers with RF=2 vs RF=3 and Zookeeper and Metadata placement see Nutanix Official documentation 

The below table should bring more clarity on what is supported and what is the impact on the clusters