Statistics
| Branch: | Tag: | Revision:

root / UPGRADE @ c7ec6a25

History | View | Annotate | Download (7.8 kB)

1
Upgrade notes
2
=============
3

    
4
.. highlight:: shell-example
5

    
6
This document details the steps needed to upgrade a cluster to newer versions
7
of Ganeti.
8

    
9
As a general rule the node daemons need to be restarted after each software
10
upgrade; if using the provided example init.d script, this means running the
11
following command on all nodes::
12

    
13
    $ /etc/init.d/ganeti restart
14

    
15

    
16
2.1 and above
17
-------------
18

    
19
Starting with Ganeti 2.0, upgrades between revisions (e.g. 2.1.0 to 2.1.1)
20
should not need manual intervention. As a safety measure, minor releases (e.g.
21
2.1.3 to 2.2.0) require the ``cfgupgrade`` command for changing the
22
configuration version. Below you find the steps necessary to upgrade between
23
minor releases.
24

    
25
To run commands on all nodes, the `distributed shell (dsh)
26
<http://www.netfort.gr.jp/~dancer/software/dsh.html.en>`_ can be used, e.g.
27
``dsh -M -F 8 -f /var/lib/ganeti/ssconf_online_nodes gnt-cluster --version``.
28

    
29
#. Ensure no jobs are running (master node only)::
30

    
31
    $ gnt-job list
32

    
33
#. Stop all daemons on all nodes::
34

    
35
    $ /etc/init.d/ganeti stop
36

    
37
#. Backup old configuration (master node only)::
38

    
39
    $ tar czf /var/lib/ganeti-$(date +\%FT\%T).tar.gz -C /var/lib ganeti
40

    
41
#. Install new Ganeti version on all nodes
42
#. Run cfgupgrade on the master node::
43

    
44
    $ /usr/lib/ganeti/tools/cfgupgrade --verbose --dry-run
45
    $ /usr/lib/ganeti/tools/cfgupgrade --verbose
46

    
47
   (``cfgupgrade`` supports a number of parameters, run it with
48
   ``--help`` for more information)
49

    
50
#. Upgrade the directory permissions on all nodes::
51

    
52
    $ /usr/lib/ganeti/ensure-dirs --full-run
53

    
54
#. Restart daemons on all nodes::
55

    
56
    $ /etc/init.d/ganeti restart
57

    
58
#. Re-distribute configuration (master node only)::
59

    
60
    $ gnt-cluster redist-conf
61

    
62
#. If you use file storage, check that the ``/etc/ganeti/file-storage-paths``
63
   is correct on all nodes. For security reasons it's not copied
64
   automatically, but it can be copied manually via::
65

    
66
   $ gnt-cluster copyfile /etc/ganeti/file-storage-paths
67

    
68
#. Restart daemons again on all nodes::
69

    
70
    $ /etc/init.d/ganeti restart
71

    
72
#. Verify cluster (master node only)::
73

    
74
    $ gnt-cluster verify
75

    
76

    
77
2.0 releases
78
------------
79

    
80
2.0.3 to 2.0.4
81
~~~~~~~~~~~~~~
82

    
83
No changes needed except restarting the daemon; but rollback to 2.0.3 might
84
require configuration editing.
85

    
86
If you're using Xen-HVM instances, please double-check the network
87
configuration (``nic_type`` parameter) as the defaults might have changed:
88
2.0.4 adds any missing configuration items and depending on the version of the
89
software the cluster has been installed with, some new keys might have been
90
added.
91

    
92
2.0.1 to 2.0.2/2.0.3
93
~~~~~~~~~~~~~~~~~~~~
94

    
95
Between 2.0.1 and 2.0.2 there have been some changes in the handling of block
96
devices, which can cause some issues. 2.0.3 was then released which adds two
97
new options/commands to fix this issue.
98

    
99
If you use DRBD-type instances and see problems in instance start or
100
activate-disks with messages from DRBD about "lower device too small" or
101
similar, it is recoomended to:
102

    
103
#. Run ``gnt-instance activate-disks --ignore-size $instance`` for each
104
   of the affected instances
105
#. Then run ``gnt-cluster repair-disk-sizes`` which will check that
106
   instances have the correct disk sizes
107

    
108
1.2 to 2.0
109
----------
110

    
111
Prerequisites:
112

    
113
- Ganeti 1.2.7 is currently installed
114
- All instances have been migrated from DRBD 0.7 to DRBD 8.x (i.e. no
115
  ``remote_raid1`` disk template)
116
- Upgrade to Ganeti 2.0.0~rc2 or later (~rc1 and earlier don't have the needed
117
  upgrade tool)
118

    
119
In the below steps, replace :file:`/var/lib` with ``$libdir`` if Ganeti was not
120
installed with this prefix (e.g. :file:`/usr/local/var`). Same for
121
:file:`/usr/lib`.
122

    
123
Execution (all steps are required in the order given):
124

    
125
#. Make a backup of the current configuration, for safety::
126

    
127
    $ cp -a /var/lib/ganeti /var/lib/ganeti-1.2.backup
128

    
129
#. Stop all instances::
130

    
131
    $ gnt-instance stop --all
132

    
133
#. Make sure no DRBD device are in use, the following command should show no
134
   active minors::
135

    
136
    $ gnt-cluster command grep cs: /proc/drbd | grep -v cs:Unconf
137

    
138
#. Stop the node daemons and rapi daemon on all nodes (note: should be logged
139
   in not via the cluster name, but the master node name, as the command below
140
   will remove the cluster ip from the master node)::
141

    
142
    $ gnt-cluster command /etc/init.d/ganeti stop
143

    
144
#. Install the new software on all nodes, either from packaging (if available)
145
   or from sources; the master daemon will not start but give error messages
146
   about wrong configuration file, which is normal
147
#. Upgrade the configuration file::
148

    
149
    $ /usr/lib/ganeti/tools/cfgupgrade12 -v --dry-run
150
    $ /usr/lib/ganeti/tools/cfgupgrade12 -v
151

    
152
#. Make sure ``ganeti-noded`` is running on all nodes (and start it if
153
   not)
154
#. Start the master daemon::
155

    
156
    $ ganeti-masterd
157

    
158
#. Check that a simple node-list works::
159

    
160
    $ gnt-node list
161

    
162
#. Redistribute updated configuration to all nodes::
163

    
164
    $ gnt-cluster redist-conf
165
    $ gnt-cluster copyfile /var/lib/ganeti/known_hosts
166

    
167
#. Optional: if needed, install RAPI-specific certificates under
168
   :file:`/var/lib/ganeti/rapi.pem` and run::
169

    
170
    $ gnt-cluster copyfile /var/lib/ganeti/rapi.pem
171

    
172
#. Run a cluster verify, this should show no problems::
173

    
174
    $ gnt-cluster verify
175

    
176
#. Remove some obsolete files::
177

    
178
    $ gnt-cluster command rm /var/lib/ganeti/ssconf_node_pass
179
    $ gnt-cluster command rm /var/lib/ganeti/ssconf_hypervisor
180

    
181
#. Update the xen pvm (if this was a pvm cluster) setting for 1.2
182
   compatibility::
183

    
184
    $ gnt-cluster modify -H xen-pvm:root_path=/dev/sda
185

    
186
#. Depending on your setup, you might also want to reset the initrd parameter::
187

    
188
    $ gnt-cluster modify -H xen-pvm:initrd_path=/boot/initrd-2.6-xenU
189

    
190
#. Reset the instance autobalance setting to default::
191

    
192
    $ for i in $(gnt-instance list -o name --no-headers); do \
193
       gnt-instance modify -B auto_balance=default $i; \
194
      done
195

    
196
#. Optional: start the RAPI demon::
197

    
198
    $ ganeti-rapi
199

    
200
#. Restart instances::
201

    
202
    $ gnt-instance start --force-multiple --all
203

    
204
At this point, ``gnt-cluster verify`` should show no errors and the migration
205
is complete.
206

    
207
1.2 releases
208
------------
209

    
210
1.2.4 to any other higher 1.2 version
211
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
212

    
213
No changes needed. Rollback will usually require manual edit of the
214
configuration file.
215

    
216
1.2.3 to 1.2.4
217
~~~~~~~~~~~~~~
218

    
219
No changes needed. Note that going back from 1.2.4 to 1.2.3 will require manual
220
edit of the configuration file (since we added some HVM-related new
221
attributes).
222

    
223
1.2.2 to 1.2.3
224
~~~~~~~~~~~~~~
225

    
226
No changes needed. Note that the drbd7-to-8 upgrade tool does a disk format
227
change for the DRBD metadata, so in theory this might be **risky**. It is
228
advised to have (good) backups before doing the upgrade.
229

    
230
1.2.1 to 1.2.2
231
~~~~~~~~~~~~~~
232

    
233
No changes needed.
234

    
235
1.2.0 to 1.2.1
236
~~~~~~~~~~~~~~
237

    
238
No changes needed. Only some bugfixes and new additions that don't affect
239
existing clusters.
240

    
241
1.2.0 beta 3 to 1.2.0
242
~~~~~~~~~~~~~~~~~~~~~
243

    
244
No changes needed.
245

    
246
1.2.0 beta 2 to beta 3
247
~~~~~~~~~~~~~~~~~~~~~~
248

    
249
No changes needed. A new version of the debian-etch-instance OS (0.3) has been
250
released, but upgrading it is not required.
251

    
252
1.2.0 beta 1 to beta 2
253
~~~~~~~~~~~~~~~~~~~~~~
254

    
255
Beta 2 switched the config file format to JSON. Steps to upgrade:
256

    
257
#. Stop the daemons (``/etc/init.d/ganeti stop``) on all nodes
258
#. Disable the cron job (default is :file:`/etc/cron.d/ganeti`)
259
#. Install the new version
260
#. Make a backup copy of the config file
261
#. Upgrade the config file using the following command::
262

    
263
    $ /usr/share/ganeti/cfgupgrade --verbose /var/lib/ganeti/config.data
264

    
265
#. Start the daemons and run ``gnt-cluster info``, ``gnt-node list`` and
266
   ``gnt-instance list`` to check if the upgrade process finished successfully
267

    
268
The OS definition also need to be upgraded. There is a new version of the
269
debian-etch-instance OS (0.2) that goes along with beta 2.
270

    
271
.. vim: set textwidth=72 :
272
.. Local Variables:
273
.. mode: rst
274
.. fill-column: 72
275
.. End: