To upgrade to Newton, the following steps are performed:
- Stop the daemons of the configuration management tool (Puppet, Chef, Quattor, ...) to ensure that it will not interfere with the upgrade procedure. We are using Quattor at IPHC. Two daemons need to be stopped:
[root@controller ~]# service cdp-listend stop - Stop the OpenStack services and ensure with the systemctl command that they are effectily stopped:
[root@controller ~]# for service in "nova neutron cinder glance"; do \
service openstack-${service} stop \
done
[root@controller ~]# service httpd stop
Note: we have created startup scripts, like /etc/init.d/openstack-nova, that manage all related daemons. - We took advantage of this upgrade to perform some database cleanup:
- Backup the database using mysql_dump
- [root@controller ~]# keystone-manage token_flush
- On our test infrastructure, we were not able to update the keystone database. This issue was caused by a UTF-8 charset problem. To fix this, we had to set correctly the charset (utf8/utf8_general_ci) of each table and database using the following script. This step is probably not required if your installation of OpenStack if younger than Juno.
- Replace the RDO Mitaka repo by the Newton repo and update the RPMs:
[root@controller ~]# cat /etc/yum.repos.d/newton.repo
[x86_64]
name=OpenStack Newton Repository
baseurl=http://mirror.centos.org/centos/7/cloud/x86_64/openstack-newton/
enabled=1
skip_if_unavailable=0
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Cloud
priority=98
[root@controller ~]# rm /etc/yum.repos.d/mitaka.repo - Install the configuration files for the new version. We are using our configuration management tools, in manual mode):
[root@controller ~]# ccm-fetch
[root@controller ~]# ncm-ncd --config filecopy
[root@controller ~]# ncm-ncd --config mysql - Once all OpenStack components are configured, each database needs to be updated to the current schema:
- Keystone
[root@controller ~]# su -s /bin/sh -c "keystone-manage db_sync" keystone - Glance
[root@controller ~]# su -s /bin/sh -c "glance-manage db_sync" glance - Cinder
[root@controller ~]# su -s /bin/sh -c "cinder-manage db sync" cinder
With cinder, we hit the following issue:
ERROR oslo_service.service ServiceTooOld: One of the services is in Liberty vers
ion. We do not provide backward compatibility with Liberty now, you need to upgra
de to Mitaka first.
If you check the enabled services, you can see that a cinder-backup service is registered:
[root@controller ~]# cinder-manage service list
Binary Host Zone Status State Updated At RPC Version Object Version Cluster
cinder-scheduler controller nova enabled XXX 2017-07-27 17:38:25 3.0 1.11
cinder-volume controller nova enabled XXX 2017-07-27 17:38:25 3.0 1.11
cinder-backup controller nova enabled XXX 2014-07-15 05:49:40 None None
The solution is to remove the out-dated service:
[root@controller ~]# cinder-manage service remove cinder-backup controller - Neutron
[root@controller ~]# su -s /bin/sh -c "neutron-db-manage upgrade heads" neutron - Nova
First, if your configuration management tool do not create database, manually create the nova_api database and give the nova user access to it.
[root@controller ~]# su -s /bin/sh -c "nova-manage api_db sync" nova
[root@controller ~]# su -s /bin/sh -c "nova-manage db sync" nova - Heat
[root@controller ~]# su -s /bin/sh -c "heat-manage db_sync" heat - Magnum
[root@controller ~]# su -s /bin/sh -c "magnum-db-manage upgrade heads" magnum
- Keystone
- The /etc/keystone/credential-keys/ directory has to be created. It is owned by the keystone user.
- The use of Keystone was really slow after the upgrade. To return to a normal state, be sure that Keystone is using memcache and that your token get flushed regularly (we are using a cron file).
- The upgrade is completed, restart the OpenStack services, as well as the configuration management tool daemon(s). Look at the OpenStack log files for any errors and test your services using your favorite probe platform.
- After this update, the creation of new flavor fails with the following error:
not all flavors have been migrated to the API databaseThis is caused by a known bug. To revolve this issue, we have run:nova-manage db online_data_migrations
