Upgrade Apache Cassandra

Learn how to upgrade your Cassandra environment from version 2.2.8 or 2.2.12 to version 3.11.11 with no downtime.

7 minute read

The upgrade process differs according to your environment:

  • For single-node environments, you can upgrade directly from version 2.2.8/2.2.12 to version 3.11.11.
  • For multi-node or multi-DC environments, you must first upgrade each node and data center individually to version 2.2.19, then to the target version of 3.11.11.

Before you start

  • You must upgrade your API Gateway to the August 2021 release prior to upgrading your Cassandra environment to 3.11.11. This is because in this release we’ve implemented changes to the Cassandra client driver configuration to facilitate the upgrade process.
  • In multi-datacenter clusters:
    • Upgrade every node in one datacenter before upgrading another datacenter.
    • Upgrade and restart the nodes one at a time.
    • Other nodes in the cluster continue to operate at the earlier version until all nodes are upgraded.
  • When upgrading a cluster on a single-datacenter or multi-datacenter setup, you must avoid any schema changes until the entire cluster has been upgraded to the same version.
  • Running nodetool repair on a Cassandra node affects performance on a system running live traffic. We recommend that you perform the Cassandra upgrade in the evening or during a maintenance window when the load is minimal.

Upgrade Cassandra in a single-node

In a single-node environment, you can upgrade directly to version 3.11.11. To get started you must download Cassandra 3.11.11, unzip the downloaded package, and copy the installation directory to the target Cassandra server node in an appropriate directory, for example, /home/cassandra-31111/

Now you can proceed to the next sections to complete your installation.

Step 1 - Backup your old Cassandra data

Perform the backup of your 2.2.8/2.2.12 data.

  1. Use the cqlsh command to find your Cassandra keyspaces to back up.

    SELECT * FROM system_schema.keyspaces;
    

    In the following example, xxx_group_2 is an API Management keyspace:

    keyspace_name                                  | durable_writes | replication
    -----------------------------------------------+----------------+-------------------------------------------------------------------------------------------
    system_auth                                    |           True | {'class': 'org.apache.cassandra.locator.NetworkTopologyStrategy', 'dc1': '3', 'dc2': '3'}
    xb9076241_7cf8_4963_a3ac_375c53f21b7d_group_2  |           True | {'class': 'org.apache.cassandra.locator.NetworkTopologyStrategy', 'dc1': '3', 'dc2': '3'}
    system_schema                                  |           True | {'class': 'org.apache.cassandra.locator.LocalStrategy'}
    system_distributed                             |           True | {'class': 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '3'}
    system                                         |           True | {'class': 'org.apache.cassandra.locator.LocalStrategy'}
    system_traces                                  |           True | {'class': 'org.apache.cassandra.locator.SimpleStrategy', 'replication_factor': '2'}
    
  2. Copy the apigw-backup-tool folder, located at install_dir/apigateway/tools/ to your Cassandra node.

  3. Update the /conf/apigw-backup-tool.ini file to configure your backup. For more information, see Apache Cassandra backup and restore.

  4. After the configuration is set and Cassandra is running, run the following command to validate the configuration:

    apigw-backup-tool validateConfig
    
  5. While Cassandra is running, run the following command to create a backup in the backup_root_dir folder. The folder is configured in the /conf/apigw-backup-tool.ini file:

    apigw-backup-tool backup -k <keyspace name> -s <snapshot name>
    
  6. Backup your Cassandra configuration.

    In addition to backing up your data in Apache Cassandra keyspaces, you must also back up your Apache Cassandra configuration and API Gateway configuration. You must back up the CASSANDRA_HOME/conf directory on the Cassandra node.

Step 2 - Update Cassandra configuration files

Compare the configuration in CASSANDRA_HOME/conf of your current Cassandra installation to the configuration in the new Cassandra 3.11.11 installation, and apply any custom changes to the relevant configuration file in your new Cassandra installation.

Step 3 - Copy SSL certificates

If you have SSL certificates in your old Cassandra installation, you must copy them to your new Cassandra installation.

Copy the following files to the CASSANDRA_HOME/conf/ folder in your new installation:

  • CASSANDRA_HOME/conf/.truststore
  • CASSANDRA_HOME/conf/.keystore

Step 4 - Stop API Gateway

Stop each API Gateway instance, and run the following command from the /posix/bin directory of your gateway installation:

./startinstance -g "my_group" -n "my_server" -k

Step 5 - Shutdown your old Cassandra installation

To shutdown your old Cassandra installation, follow these steps:

  1. Run Cassandra’s nodetool drain command in CASSANDRA_HOME/bin to flush any data in memory and write it to disk:

    $./nodetool drain
    
  2. Kill the Cassandra process:

    $ps auwx | grep cassandra
    $sudo kill pid #Stop Cassandra
    

Step 6 - Copy data between Cassandra installations

Copy the CASSANDRA_HOME/data directory from your old Cassandra installation to the corresponding directory in your new installation (for example, /home/cassandra-31111/cassandra/).

The Cassandra 3.11.11 data directory should then have the following subdirectories:

commitlog
data
saved_caches

Step 7 - Start Cassandra 3.11.11

Run the following command from the bin directory of the Cassandra 3.11.11 installation to start the Cassandra instance:

$cd /home/cassandra-31111/cassandra/bin
$./cassandra

Step 8 - Repair and upgrade your tables

Run the following commands to repair and upgrade your tables:

  1. Repair tables on your new installation:

    $cd /home/cassandra-31111/cassandra/bin
    $./nodetool repair
    
  2. Rewrite SSTables that are not on the current version and upgrade them to the 3.11.11 version:

    $cd /home/cassandra-31111/cassandra/bin
    $./nodetool upgradesstables
    

Step 9 - Restart API Gateway

Restart each API Gateway instance. Open a shell at the /posix/bin directory of your gateway installation and run the startinstance command. For example:

startinstance -n "my_server" -g "my_group"

Upgrade Cassandra in a multi-node single datacenter

This section provides an example of how to upgrade Cassandra in a multi-node environment. The upgrade is a two-stage process:

  1. Stage 1 - Upgrade Cassandra from version 2.2.8/2.2.12 to version 2.2.19.
  2. Stage 2 - Upgrade Cassandra from version 2.2.19 to version 3.11.11.

Stage 1 - Upgrade Cassandra 2.2.8/2.2.12 to 2.2.19

You must upgrade each node in your Cassandra cluster individually before moving to the next node in the cluster. The entire cluster must be upgraded before moving to Stage 2.

Update Cassandra’s driver configuration in API Gateway

Before upgrading the Cassandra cluster, you must set Cassandra’s driver protocol to version V3 in your gateways:

  1. Create a file called jvm.xml in the following location:

    INSTALL_DIR/apigateway/groups/GROUP_ID/INSTANCE_ID/conf
    
  2. Edit the jvm.xml as follows:

    <ConfigurationFragment>
        <VMArg name="-DCASSANDRA_PROTOCOL_VERSION=3" />
    </ConfigurationFragment>
    
  3. Restart API Gateway.

Install Cassandra 2.2.19

To start upgrading your old installation:

  1. Download Cassandra 2.2.19.

  2. Unzip the downloaded package, and copy the installation directory to the target Cassandra server node in an appropriate directory, for example, /home/cassandra-2219/.

Stage 2 - Upgrade Cassandra 2.2.19 to 3.11.11

To start upgrading your old installation:

  1. Download Cassandra 3.11.11

  2. Unzip the downloaded package, and copy the installation directory to the target Cassandra server node in an appropriate directory, for example, /home/cassandra-31111/.

Revert Cassandra’s driver configuration in API Gateway

Update API Gateway back to use Cassandra’s V4 protocol.

  1. To reconfigure the protocol version on each instance of API Gateway, edit the INSTALL_DIR/apigateway/groups/GROUP_ID/INSTANCE_ID/conf/jvm.xml file, and remove the previously set JVM argument.

    <ConfigurationFragment>
        <VMArg name="-DCASSANDRA_PROTOCOL_VERSION=4" />
    </ConfigurationFragment>
    

    This reverts the gateway to use Cassandra’s V4 protocol.

  2. Restart API Gateway.

Upgrade Cassandra in a multi-node multi-datacenter

To upgrade Apache Cassandra in a multi-node multi-datacenter environment, follow the same steps as for the Multi-node single-datacenter procedure, which follows a two-stage process: upgrade Cassandra from version 2.2.8/2.2.12 to version 2.2.19, then upgrade from version 2.2.19 to version 3.11.11.

Attention to the following:

  • Repeat the steps on each node in the cluster.
  • Upgrade all of the nodes in one datacenter before moving to the next one.
  • Only reconfigure the API Gateway to use Cassandra protocol V4 after all datacenter have been upgraded.

Upgrade multi-node multi-datacenter directly to 3.11.11

To upgrade Apache Cassandra directly to version 3.11.11 in a multi-node or multi-datacenter environment, follow the same steps as for Single-node) procedure.

Attention to the following:

  • Repeat the steps on each node in the cluster.
  • Upgrade all of the nodes in one DC before moving to the next DC.
  • Only restart the API Gateway once all nodes and DCs have been upgraded.

Troubleshooting

For more information on problems you might encounter when running Cassandra with API Gateway, see Cassandra troubleshooting.