Scenario 4: One service at a time with live Compute upgrade

For most services this scenario is identical to scenario 2, with the exception of the compute service. This scenario takes advantage of work done for the Icehouse release that permits you to run Havana compute nodes in parallel with Icehouse compute nodes.

Upgrade Nova controllers

  1. Follow the procedure for scenario 2, but stop after completing the Horizon upgrade (do not upgrade Nova).

    You will want to run the final package upgrade on systems that are not running Nova services.

  1. Upgrade the Nova controller services. On each controller node, stop the running Nova services:

    openstack-service stop nova
    

    and upgrade the Nova packages:

    yum -y upgrade \*nova\*
    

    Note the Configuration changes section of the Nova upgade documentation.

At this point, you will want to examine any *.rpmnew files installed by the packages and update your existing configuration files appropriately.

  1. On one of the controller nodes, run the database upgrade script:

    openstack-db --service nova --update
    
  2. Cap the compute RPC API at a version that will still be understood by your Havana compute nodes. Look for the [upgrade_levels] section in /etc/nova/nova.conf and set the compute option like this:

    [upgrade_levels]
    compute = icehouse-compat
    
  3. Restart the controller services. On each controller node:

    openstack-service start nova
    

Upgrade Neutron

If you are running Neutron, follow the Neutron upgrade instructions.

Upgrade Nova compute nodes

At this point, your controller nodes are all running the Icehouse version of Nova and your compute nodes are still running the Havana version.

For each compute node:

  1. Mark the nova-compute service as disabled to prevent Nova from scheduling any new servers on this node:

    nova service-disable --reason upgrade myhost nova-compute
    

    Where myhost is the name of the compute host as it is known to Nova. You can see a list of compute hosts using the nova service-list command:

    nova service-list --binary nova-compute
    
  2. Stop the nova-compute service on the node:

    openstack-service stop nova
    
  3. Upgrade all the Nova packages:

    yum upgrade \*nova\*
    
  4. Restart Nova services:

    openstack-service start nova
    
  5. Re-enable the compute service:

    nova service-enable myhost nova-compute
    

Migrating instances

If you need to perform hardware or operating system upgrades on the compute nodes, you can migrate running instances off the nodes as part of the upgrade process.

If you will be migrating instances between Havana and Icehouse compute nodes, you will need to ensure that your Icehouse nodes have the same upgrade_levels setting as your controllers:

   [upgrade_levels]
   compute = icehouse-compat

Live migration

If you have configured live migration in your environment, you can use the nova live-migration command to move instances to another compute host with zero downtime:

nova live-migration myinstance

Where myinstance is the name or UUID of a running server.

Cold migration

If live migration is not available, you can use cold migration to move instances to another node with minimal downtime:

nova migrate myinstance

Post-upgrade tasks

After you have complete the upgrade to all of your Nova compute nodes, you should remove the RPC API version limit from your Nova configuration. In /etc/nova/nova.conf on your controllers and compute nodes, comment out the compute setting in the upgrade_levels section:

   [upgrade_levels]
   # compute = icehouse-compat

You will need to restart Nova services on each host where you make this change:

openstack-service restart nova

At this point, you should perform a final package upgrade on your compute and controller nodes to ensure that all your installed packages are at the latest version.