root / man / ganeti.rst @ e715a6d6
History | View | Annotate | Download (15.1 kB)
1 |
ganeti(7) Ganeti | Version @GANETI_VERSION@ |
---|---|
2 |
=========================================== |
3 |
|
4 |
Name |
5 |
---- |
6 |
|
7 |
ganeti - cluster-based virtualization management |
8 |
|
9 |
Synopsis |
10 |
-------- |
11 |
|
12 |
:: |
13 |
|
14 |
# gnt-cluster init cluster1.example.com |
15 |
# gnt-node add node2.example.com |
16 |
# gnt-instance add -n node2.example.com \ |
17 |
> -o debootstrap --disk 0:size=30g \ |
18 |
> -t plain instance1.example.com |
19 |
|
20 |
|
21 |
DESCRIPTION |
22 |
----------- |
23 |
|
24 |
The Ganeti software manages physical nodes and virtual instances of a |
25 |
cluster based on a virtualization software. The current version (2.3) |
26 |
supports Xen 3.x and KVM (72 or above) as hypervisors, and LXC as an |
27 |
experimental hypervisor. |
28 |
|
29 |
Quick start |
30 |
----------- |
31 |
|
32 |
First you must install the software on all the cluster nodes, either |
33 |
from sources or (if available) from a package. The next step is to |
34 |
create the initial cluster configuration, using **gnt-cluster init**. |
35 |
|
36 |
Then you can add other nodes, or start creating instances. |
37 |
|
38 |
Cluster architecture |
39 |
-------------------- |
40 |
|
41 |
In Ganeti 2.0, the architecture of the cluster is a little more |
42 |
complicated than in 1.2. The cluster is coordinated by a master daemon |
43 |
(**ganeti-masterd**\(8)), running on the master node. Each node runs |
44 |
(as before) a node daemon, and the master has the RAPI daemon running |
45 |
too. |
46 |
|
47 |
Node roles |
48 |
~~~~~~~~~~ |
49 |
|
50 |
Each node can be in one of the following states: |
51 |
|
52 |
master |
53 |
Only one node per cluster can be in this role, and this node is the |
54 |
one holding the authoritative copy of the cluster configuration and |
55 |
the one that can actually execute commands on the cluster and |
56 |
modify the cluster state. See more details under |
57 |
*Cluster configuration*. |
58 |
|
59 |
master_candidate |
60 |
The node receives the full cluster configuration (configuration |
61 |
file and jobs) and can become a master via the |
62 |
**gnt-cluster master-failover** command. Nodes that are not in this |
63 |
state cannot transition into the master role due to missing state. |
64 |
|
65 |
regular |
66 |
This the normal state of a node. |
67 |
|
68 |
drained |
69 |
Nodes in this state are functioning normally but cannot receive |
70 |
new instances, because the intention is to set them to *offline* |
71 |
or remove them from the cluster. |
72 |
|
73 |
offline |
74 |
These nodes are still recorded in the Ganeti configuration, but |
75 |
except for the master daemon startup voting procedure, they are not |
76 |
actually contacted by the master. This state was added in order to |
77 |
allow broken machines (that are being repaired) to remain in the |
78 |
cluster but without creating problems. |
79 |
|
80 |
|
81 |
Node flags |
82 |
~~~~~~~~~~ |
83 |
|
84 |
Nodes have two flags which govern which roles they can take: |
85 |
|
86 |
master_capable |
87 |
The node can become a master candidate, and furthermore the master |
88 |
node. When this flag is disabled, the node cannot become a |
89 |
candidate; this can be useful for special networking cases, or less |
90 |
reliable hardware. |
91 |
|
92 |
vm_capable |
93 |
The node can host instances. When enabled (the default state), the |
94 |
node will participate in instance allocation, capacity calculation, |
95 |
etc. When disabled, the node will be skipped in many cluster checks |
96 |
and operations. |
97 |
|
98 |
|
99 |
Node Parameters |
100 |
~~~~~~~~~~~~~~~ |
101 |
|
102 |
The ``ndparams`` refer to node parameters. These can be set as defaults |
103 |
on cluster and node group levels, but they take effect for nodes only. |
104 |
|
105 |
Currently we support the following node parameters: |
106 |
|
107 |
oob_program |
108 |
Path to an executable used as the out-of-band helper as described in |
109 |
the `Ganeti Node OOB Management Framework <design-oob.rst>`_ design |
110 |
document. |
111 |
|
112 |
spindle_count |
113 |
This should reflect the I/O performance of local attached storage |
114 |
(e.g. for "file", "plain" and "drbd" disk templates). It doesn't |
115 |
have to match the actual spindle count of (any eventual) mechanical |
116 |
hard-drives, its meaning is site-local and just the relative values |
117 |
matter. |
118 |
|
119 |
exclusive_storage |
120 |
When this Boolean flag is enabled, physical disks on the node are |
121 |
assigned to instance disks in an exclusive manner, so as to lower I/O |
122 |
interference between instances. See the `Partitioned Ganeti |
123 |
<design-partitioned.rst>`_ design document for more details. This |
124 |
parameter cannot be set on individual nodes, as its value must be |
125 |
the same within each node group. |
126 |
|
127 |
ovs |
128 |
When this Boolean flag is enabled, OpenvSwitch will be used as the |
129 |
network layer. This will cause the initialization of OpenvSwitch on |
130 |
the nodes when added to the cluster. Per default this is not enabled. |
131 |
|
132 |
ovs_name |
133 |
When ovs is enabled, this parameter will represent the name of the |
134 |
OpenvSwitch to generate and use. This will default to `switch1`. |
135 |
|
136 |
ovs_link |
137 |
When ovs is enabled, a OpenvSwitch will be initialized on new nodes |
138 |
and will have this as its connection to the outside. This parameter |
139 |
is not set per default, as it depends very much on the specific |
140 |
setup. |
141 |
|
142 |
ssh_port |
143 |
The port used for SSH connections to nodes belonging to a group. The user |
144 |
is responsible for properly configuring the ports of SSH daemons on |
145 |
machines prior to adding them as Ganeti nodes or when modifying the |
146 |
parameter value of an existing group. Note that using non-standard SSH |
147 |
ports and downgrading to an older Ganeti version that doesn't support |
148 |
``ssh_port`` will break the cluster. |
149 |
|
150 |
|
151 |
Hypervisor State Parameters |
152 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
153 |
|
154 |
Using ``--hypervisor-state`` you can set hypervisor specific states as |
155 |
pointed out in ``Ganeti Resource Model <design-resource-model.rst>``. |
156 |
|
157 |
The format is: ``hypervisor:option=value``. |
158 |
|
159 |
Currently we support the following hypervisor state values: |
160 |
|
161 |
mem_total |
162 |
Total node memory, as discovered by this hypervisor |
163 |
mem_node |
164 |
Memory used by, or reserved for, the node itself; note that some |
165 |
hypervisors can report this in an authoritative way, other not |
166 |
mem_hv |
167 |
Memory used either by the hypervisor itself or lost due to instance |
168 |
allocation rounding; usually this cannot be precisely computed, but |
169 |
only roughly estimated |
170 |
cpu_total |
171 |
Total node cpu (core) count; usually this can be discovered |
172 |
automatically |
173 |
cpu_node |
174 |
Number of cores reserved for the node itself; this can either be |
175 |
discovered or set manually. Only used for estimating how many VCPUs |
176 |
are left for instances |
177 |
|
178 |
Note that currently this option is unused by Ganeti; values will be |
179 |
recorded but will not influence the Ganeti operation. |
180 |
|
181 |
|
182 |
Disk State Parameters |
183 |
~~~~~~~~~~~~~~~~~~~~~ |
184 |
|
185 |
Using ``--disk-state`` you can set disk specific states as pointed out |
186 |
in ``Ganeti Resource Model <design-resource-model.rst>``. |
187 |
|
188 |
The format is: ``storage_type/identifier:option=value``. Where we |
189 |
currently just support ``lvm`` as storage type. The identifier in this |
190 |
case is the LVM volume group. By default this is ``xenvg``. |
191 |
|
192 |
Currently we support the following hypervisor state values: |
193 |
|
194 |
disk_total |
195 |
Total disk size (usually discovered automatically) |
196 |
disk_reserved |
197 |
Reserved disk size; this is a lower limit on the free space, if such a |
198 |
limit is desired |
199 |
disk_overhead |
200 |
Disk that is expected to be used by other volumes (set via |
201 |
``reserved_lvs``); usually should be zero |
202 |
|
203 |
Note that currently this option is unused by Ganeti; values will be |
204 |
recorded but will not influence the Ganeti operation. |
205 |
|
206 |
|
207 |
Cluster configuration |
208 |
~~~~~~~~~~~~~~~~~~~~~ |
209 |
|
210 |
The master node keeps and is responsible for the cluster |
211 |
configuration. On the filesystem, this is stored under the |
212 |
``@LOCALSTATEDIR@/ganeti/lib`` directory, and if the master daemon is |
213 |
stopped it can be backed up normally. |
214 |
|
215 |
The master daemon will replicate the configuration database called |
216 |
``config.data`` and the job files to all the nodes in the master |
217 |
candidate role. It will also distribute a copy of some configuration |
218 |
values via the *ssconf* files, which are stored in the same directory |
219 |
and start with a ``ssconf_`` prefix, to all nodes. |
220 |
|
221 |
Jobs |
222 |
~~~~ |
223 |
|
224 |
All cluster modification are done via jobs. A job consists of one |
225 |
or more opcodes, and the list of opcodes is processed serially. If |
226 |
an opcode fails, the entire job is failed and later opcodes are no |
227 |
longer processed. A job can be in one of the following states: |
228 |
|
229 |
queued |
230 |
The job has been submitted but not yet processed by the master |
231 |
daemon. |
232 |
|
233 |
waiting |
234 |
The job is waiting for for locks before the first of its opcodes. |
235 |
|
236 |
canceling |
237 |
The job is waiting for locks, but is has been marked for |
238 |
cancellation. It will not transition to *running*, but to |
239 |
*canceled*. |
240 |
|
241 |
running |
242 |
The job is currently being executed. |
243 |
|
244 |
canceled |
245 |
The job has been canceled before starting execution. |
246 |
|
247 |
success |
248 |
The job has finished successfully. |
249 |
|
250 |
error |
251 |
The job has failed during runtime, or the master daemon has been |
252 |
stopped during the job execution. |
253 |
|
254 |
|
255 |
Common command line features |
256 |
---------------------------- |
257 |
|
258 |
Options |
259 |
~~~~~~~ |
260 |
|
261 |
Many Ganeti commands provide the following options. The |
262 |
availability for a certain command can be checked by calling the |
263 |
command using the ``--help`` option. |
264 |
|
265 |
| **gnt-...** *command* [\--dry-run] [\--priority {low | normal | high}] |
266 |
| [\--submit] [\--print-job-id] |
267 |
|
268 |
The ``--dry-run`` option can be used to check whether an operation |
269 |
would succeed. |
270 |
|
271 |
The option ``--priority`` sets the priority for opcodes submitted |
272 |
by the command. |
273 |
|
274 |
The ``--submit`` option is used to send the job to the master daemon but |
275 |
not wait for its completion. The job ID will be shown so that it can be |
276 |
examined using **gnt-job info**. |
277 |
|
278 |
The ``--print-job-id`` option makes the command print the job id as first |
279 |
line on stdout, so that it is easy to parse by other programs. |
280 |
|
281 |
Defaults |
282 |
~~~~~~~~ |
283 |
|
284 |
For certain commands you can use environment variables to provide |
285 |
default command line arguments. Just assign the arguments as a string to |
286 |
the corresponding environment variable. The format of that variable |
287 |
name is **binary**_*command*. **binary** is the name of the ``gnt-*`` |
288 |
script all upper case and dashes replaced by underscores, and *command* |
289 |
is the command invoked on that script. |
290 |
|
291 |
Currently supported commands are ``gnt-node list``, ``gnt-group list`` |
292 |
and ``gnt-instance list``. So you can configure default command line |
293 |
flags by setting ``GNT_NODE_LIST``, ``GNT_GROUP_LIST`` and |
294 |
``GNT_INSTANCE_LIST``. |
295 |
|
296 |
Debug options |
297 |
~~~~~~~~~~~~~ |
298 |
|
299 |
If the variable ``FORCE_LUXI_SOCKET`` is set, it will override the |
300 |
socket used for LUXI connections by command-line tools |
301 |
(``gnt-*``). This is useful mostly for debugging, and some operations |
302 |
won't work at all if, for example, you point this variable to the |
303 |
confd-supplied query socket and try to submit a job. |
304 |
|
305 |
If the variable is set to the value ``master``, it will connect to the |
306 |
correct path for the master daemon (even if, for example, split |
307 |
queries are enabled and this is a query operation). If set to |
308 |
``query``, it will always (try to) connect to the query socket, even |
309 |
if split queries are disabled. Otherwise, the value is taken to |
310 |
represent a filesystem path to the socket to use. |
311 |
|
312 |
Field formatting |
313 |
---------------- |
314 |
|
315 |
Multiple ganeti commands use the same framework for tabular listing of |
316 |
resources (e.g. **gnt-instance list**, **gnt-node list**, **gnt-group |
317 |
list**, **gnt-debug locks**, etc.). For these commands, special states |
318 |
are denoted via a special symbol (in terse mode) or a string (in |
319 |
verbose mode): |
320 |
|
321 |
\*, (offline) |
322 |
The node in question is marked offline, and thus it cannot be |
323 |
queried for data. This result is persistent until the node is |
324 |
de-offlined. |
325 |
|
326 |
?, (nodata) |
327 |
Ganeti expected to receive an answer from this entity, but the |
328 |
cluster RPC call failed and/or we didn't receive a valid answer; |
329 |
usually more information is available in the node daemon log (if |
330 |
the node is alive) or the master daemon log. This result is |
331 |
transient, and re-running command might return a different result. |
332 |
|
333 |
-, (unavail) |
334 |
The respective field doesn't make sense for this entity; |
335 |
e.g. querying a down instance for its current memory 'live' usage, |
336 |
or querying a non-vm_capable node for disk/memory data. This |
337 |
result is persistent, and until the entity state is changed via |
338 |
ganeti commands, the result won't change. |
339 |
|
340 |
??, (unknown) |
341 |
This field is not known (note that this is different from entity |
342 |
being unknown). Either you have mis-typed the field name, or you |
343 |
are using a field that the running Ganeti master daemon doesn't |
344 |
know. This result is persistent, re-running the command won't |
345 |
change it. |
346 |
|
347 |
Key-value parameters |
348 |
~~~~~~~~~~~~~~~~~~~~ |
349 |
|
350 |
Multiple options take parameters that are of the form |
351 |
``key=value,key=value,...`` or ``category:key=value,...``. Examples |
352 |
are the hypervisor parameters, backend parameters, etc. For these, |
353 |
it's possible to use values that contain commas by escaping with via a |
354 |
backslash (which needs two if not single-quoted, due to shell |
355 |
behaviour):: |
356 |
|
357 |
# gnt-instance modify -H kernel_path=an\\,example instance1 |
358 |
# gnt-instance modify -H kernel_path='an\,example' instance1 |
359 |
|
360 |
Query filters |
361 |
~~~~~~~~~~~~~ |
362 |
|
363 |
Most commands listing resources (e.g. instances or nodes) support filtering. |
364 |
The filter language is similar to Python expressions with some elements from |
365 |
Perl. The language is not generic. Each condition must consist of a field name |
366 |
and a value (except for boolean checks), a field can not be compared to another |
367 |
field. Keywords are case-sensitive. |
368 |
|
369 |
Examples (see below for syntax details): |
370 |
|
371 |
- List webservers:: |
372 |
|
373 |
gnt-instance list --filter 'name =* "web*.example.com"' |
374 |
|
375 |
- List instances with three or six virtual CPUs and whose primary |
376 |
nodes reside in groups starting with the string "rack":: |
377 |
|
378 |
gnt-instance list --filter |
379 |
'(be/vcpus == 3 or be/vcpus == 6) and pnode.group =~ m/^rack/' |
380 |
|
381 |
- Nodes hosting primary instances:: |
382 |
|
383 |
gnt-node list --filter 'pinst_cnt != 0' |
384 |
|
385 |
- Nodes which aren't master candidates:: |
386 |
|
387 |
gnt-node list --filter 'not master_candidate' |
388 |
|
389 |
- Short version for globbing patterns:: |
390 |
|
391 |
gnt-instance list '*.site1' '*.site2' |
392 |
|
393 |
Syntax in pseudo-BNF:: |
394 |
|
395 |
<quoted-string> ::= /* String quoted with single or double quotes, |
396 |
backslash for escaping */ |
397 |
|
398 |
<integer> ::= /* Number in base-10 positional notation */ |
399 |
|
400 |
<re> ::= /* Regular expression */ |
401 |
|
402 |
/* |
403 |
Modifier "i": Case-insensitive matching, see |
404 |
http://docs.python.org/library/re#re.IGNORECASE |
405 |
|
406 |
Modifier "s": Make the "." special character match any character, |
407 |
including newline, see http://docs.python.org/library/re#re.DOTALL |
408 |
*/ |
409 |
<re-modifiers> ::= /* empty */ | i | s |
410 |
|
411 |
<value> ::= <quoted-string> | <integer> |
412 |
|
413 |
<condition> ::= |
414 |
{ /* Value comparison */ |
415 |
<field> { == | != | < | <= | >= | > } <value> |
416 |
|
417 |
/* Collection membership */ |
418 |
| <value> [ not ] in <field> |
419 |
|
420 |
/* Regular expressions (recognized delimiters |
421 |
are "/", "#", "^", and "|"; backslash for escaping) |
422 |
*/ |
423 |
| <field> { =~ | !~ } m/<re>/<re-modifiers> |
424 |
|
425 |
/* Globbing */ |
426 |
| <field> { =* | !* } <quoted-string> |
427 |
|
428 |
/* Boolean */ |
429 |
| <field> |
430 |
} |
431 |
|
432 |
<filter> ::= |
433 |
{ [ not ] <condition> | ( <filter> ) } |
434 |
[ { and | or } <filter> ] |
435 |
|
436 |
Operators: |
437 |
|
438 |
*==* |
439 |
Equality |
440 |
*!=* |
441 |
Inequality |
442 |
*<* |
443 |
Less than |
444 |
*<=* |
445 |
Less than or equal |
446 |
*>* |
447 |
Greater than |
448 |
*>=* |
449 |
Greater than or equal |
450 |
*=~* |
451 |
Pattern match using regular expression |
452 |
*!~* |
453 |
Logically negated from *=~* |
454 |
*=\** |
455 |
Globbing, see **glob**\(7), though only * and ? are supported |
456 |
*!\** |
457 |
Logically negated from *=\** |
458 |
*in*, *not in* |
459 |
Collection membership and negation |
460 |
|
461 |
|
462 |
Common daemon functionality |
463 |
--------------------------- |
464 |
|
465 |
All Ganeti daemons re-open the log file(s) when sent a SIGHUP signal. |
466 |
**logrotate**\(8) can be used to rotate Ganeti's log files. |
467 |
|
468 |
.. vim: set textwidth=72 : |
469 |
.. Local Variables: |
470 |
.. mode: rst |
471 |
.. fill-column: 72 |
472 |
.. End: |