Statistics
| Branch: | Tag: | Revision:

root / UPGRADE @ grnet-2.4-ippool-devel

History | View | Annotate | Download (7.3 kB)

1
Upgrade notes
2
=============
3

    
4
.. highlight:: sh
5

    
6
This document details the steps needed to upgrade a cluster to newer versions
7
of Ganeti.
8

    
9
As a general rule the node daemons need to be restarted after each software
10
upgrade; if using the provided example init.d script, this means running the
11
following command on all nodes::
12

    
13
  /etc/init.d/ganeti restart
14

    
15

    
16
2.1 and above
17
-------------
18

    
19
Starting with Ganeti 2.0, upgrades between revisions (e.g. 2.1.0 to 2.1.1)
20
should not need manual intervention. As a safety measure, minor releases (e.g.
21
2.1.3 to 2.2.0) require the ``cfgupgrade`` command for changing the
22
configuration version. Below you find the steps necessary to upgrade between
23
minor releases.
24

    
25
To run commands on all nodes, the `distributed shell (dsh)
26
<http://www.netfort.gr.jp/~dancer/software/dsh.html.en>`_ can be used, e.g.
27
``dsh -M -F 8 -f /var/lib/ganeti/ssconf_online_nodes gnt-cluster --version``.
28

    
29
#. Ensure no jobs are running (master node only)::
30

    
31
     gnt-job list
32

    
33
#. Stop all daemons on all nodes::
34

    
35
     /etc/init.d/ganeti stop
36

    
37
#. Backup old configuration (master node only)::
38

    
39
     tar czf /var/lib/ganeti-$(date +%FT%T).tar.gz -C /var/lib ganeti
40

    
41
#. Install new Ganeti version on all nodes
42
#. Run cfgupgrade on the master node::
43

    
44
     /usr/lib/ganeti/tools/cfgupgrade --verbose --dry-run
45
     /usr/lib/ganeti/tools/cfgupgrade --verbose
46

    
47
   (``cfgupgrade`` supports a number of parameters, run it with
48
   ``--help`` for more information)
49

    
50
#. Restart daemons on all nodes::
51

    
52
     /etc/init.d/ganeti restart
53

    
54
#. Re-distribute configuration (master node only)::
55

    
56
    gnt-cluster redist-conf
57

    
58
#. Restart daemons again on all nodes::
59

    
60
   /etc/init.d/ganeti restart
61

    
62
#. Verify cluster (master node only)::
63

    
64
     gnt-cluster verify
65

    
66

    
67
2.0 releases
68
------------
69

    
70
2.0.3 to 2.0.4
71
~~~~~~~~~~~~~~
72

    
73
No changes needed except restarting the daemon; but rollback to 2.0.3 might
74
require configuration editing.
75

    
76
If you're using Xen-HVM instances, please double-check the network
77
configuration (``nic_type`` parameter) as the defaults might have changed:
78
2.0.4 adds any missing configuration items and depending on the version of the
79
software the cluster has been installed with, some new keys might have been
80
added.
81

    
82
2.0.1 to 2.0.2/2.0.3
83
~~~~~~~~~~~~~~~~~~~~
84

    
85
Between 2.0.1 and 2.0.2 there have been some changes in the handling of block
86
devices, which can cause some issues. 2.0.3 was then released which adds two
87
new options/commands to fix this issue.
88

    
89
If you use DRBD-type instances and see problems in instance start or
90
activate-disks with messages from DRBD about "lower device too small" or
91
similar, it is recoomended to:
92

    
93
#. Run ``gnt-instance activate-disks --ignore-size $instance`` for each
94
   of the affected instances
95
#. Then run ``gnt-cluster repair-disk-sizes`` which will check that
96
   instances have the correct disk sizes
97

    
98
1.2 to 2.0
99
----------
100

    
101
Prerequisites:
102

    
103
- Ganeti 1.2.7 is currently installed
104
- All instances have been migrated from DRBD 0.7 to DRBD 8.x (i.e. no
105
  ``remote_raid1`` disk template)
106
- Upgrade to Ganeti 2.0.0~rc2 or later (~rc1 and earlier don't have the needed
107
  upgrade tool)
108

    
109
In the below steps, replace :file:`/var/lib` with ``$libdir`` if Ganeti was not
110
installed with this prefix (e.g. :file:`/usr/local/var`). Same for
111
:file:`/usr/lib`.
112

    
113
Execution (all steps are required in the order given):
114

    
115
#. Make a backup of the current configuration, for safety::
116

    
117
    cp -a /var/lib/ganeti /var/lib/ganeti-1.2.backup
118

    
119
#. Stop all instances::
120

    
121
    gnt-instance stop --all
122

    
123
#. Make sure no DRBD device are in use, the following command should show no
124
   active minors::
125

    
126
    gnt-cluster command grep cs: /proc/drbd \| grep -v cs:Unconf
127

    
128
#. Stop the node daemons and rapi daemon on all nodes (note: should be logged
129
   in not via the cluster name, but the master node name, as the command below
130
   will remove the cluster ip from the master node)::
131

    
132
    gnt-cluster command /etc/init.d/ganeti stop
133

    
134
#. Install the new software on all nodes, either from packaging (if available)
135
   or from sources; the master daemon will not start but give error messages
136
   about wrong configuration file, which is normal
137
#. Upgrade the configuration file::
138

    
139
    /usr/lib/ganeti/tools/cfgupgrade12 -v --dry-run
140
    /usr/lib/ganeti/tools/cfgupgrade12 -v
141

    
142
#. Make sure ``ganeti-noded`` is running on all nodes (and start it if
143
   not)
144
#. Start the master daemon::
145

    
146
    ganeti-masterd
147

    
148
#. Check that a simple node-list works::
149

    
150
    gnt-node list
151

    
152
#. Redistribute updated configuration to all nodes::
153

    
154
    gnt-cluster redist-conf
155
    gnt-cluster copyfile /var/lib/ganeti/known_hosts
156

    
157
#. Optional: if needed, install RAPI-specific certificates under
158
   :file:`/var/lib/ganeti/rapi.pem` and run::
159

    
160
    gnt-cluster copyfile /var/lib/ganeti/rapi.pem
161

    
162
#. Run a cluster verify, this should show no problems::
163

    
164
    gnt-cluster verify
165

    
166
#. Remove some obsolete files::
167

    
168
    gnt-cluster command rm /var/lib/ganeti/ssconf_node_pass
169
    gnt-cluster command rm /var/lib/ganeti/ssconf_hypervisor
170

    
171
#. Update the xen pvm (if this was a pvm cluster) setting for 1.2
172
   compatibility::
173

    
174
    gnt-cluster modify -H xen-pvm:root_path=/dev/sda
175

    
176
#. Depending on your setup, you might also want to reset the initrd parameter::
177

    
178
    gnt-cluster modify -H xen-pvm:initrd_path=/boot/initrd-2.6-xenU
179

    
180
#. Reset the instance autobalance setting to default::
181

    
182
    for i in $(gnt-instance list -o name --no-headers); do \
183
      gnt-instance modify -B auto_balance=default $i; \
184
    done
185

    
186
#. Optional: start the RAPI demon::
187

    
188
    ganeti-rapi
189

    
190
#. Restart instances::
191

    
192
    gnt-instance start --force-multiple --all
193

    
194
At this point, ``gnt-cluster verify`` should show no errors and the migration
195
is complete.
196

    
197
1.2 releases
198
------------
199

    
200
1.2.4 to any other higher 1.2 version
201
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
202

    
203
No changes needed. Rollback will usually require manual edit of the
204
configuration file.
205

    
206
1.2.3 to 1.2.4
207
~~~~~~~~~~~~~~
208

    
209
No changes needed. Note that going back from 1.2.4 to 1.2.3 will require manual
210
edit of the configuration file (since we added some HVM-related new
211
attributes).
212

    
213
1.2.2 to 1.2.3
214
~~~~~~~~~~~~~~
215

    
216
No changes needed. Note that the drbd7-to-8 upgrade tool does a disk format
217
change for the DRBD metadata, so in theory this might be **risky**. It is
218
advised to have (good) backups before doing the upgrade.
219

    
220
1.2.1 to 1.2.2
221
~~~~~~~~~~~~~~
222

    
223
No changes needed.
224

    
225
1.2.0 to 1.2.1
226
~~~~~~~~~~~~~~
227

    
228
No changes needed. Only some bugfixes and new additions that don't affect
229
existing clusters.
230

    
231
1.2.0 beta 3 to 1.2.0
232
~~~~~~~~~~~~~~~~~~~~~
233

    
234
No changes needed.
235

    
236
1.2.0 beta 2 to beta 3
237
~~~~~~~~~~~~~~~~~~~~~~
238

    
239
No changes needed. A new version of the debian-etch-instance OS (0.3) has been
240
released, but upgrading it is not required.
241

    
242
1.2.0 beta 1 to beta 2
243
~~~~~~~~~~~~~~~~~~~~~~
244

    
245
Beta 2 switched the config file format to JSON. Steps to upgrade:
246

    
247
#. Stop the daemons (``/etc/init.d/ganeti stop``) on all nodes
248
#. Disable the cron job (default is :file:`/etc/cron.d/ganeti`)
249
#. Install the new version
250
#. Make a backup copy of the config file
251
#. Upgrade the config file using the following command::
252

    
253
    /usr/share/ganeti/cfgupgrade --verbose /var/lib/ganeti/config.data
254

    
255
#. Start the daemons and run ``gnt-cluster info``, ``gnt-node list`` and
256
   ``gnt-instance list`` to check if the upgrade process finished successfully
257

    
258
The OS definition also need to be upgraded. There is a new version of the
259
debian-etch-instance OS (0.2) that goes along with beta 2.