root / man / ganeti.rst @ 708f8301
History | View | Annotate | Download (14.7 kB)
1 |
ganeti(7) Ganeti | Version @GANETI_VERSION@ |
---|---|
2 |
=========================================== |
3 |
|
4 |
Name |
5 |
---- |
6 |
|
7 |
ganeti - cluster-based virtualization management |
8 |
|
9 |
Synopsis |
10 |
-------- |
11 |
|
12 |
:: |
13 |
|
14 |
# gnt-cluster init cluster1.example.com |
15 |
# gnt-node add node2.example.com |
16 |
# gnt-instance add -n node2.example.com \ |
17 |
> -o debootstrap --disk 0:size=30g \ |
18 |
> -t plain instance1.example.com |
19 |
|
20 |
|
21 |
DESCRIPTION |
22 |
----------- |
23 |
|
24 |
The Ganeti software manages physical nodes and virtual instances of a |
25 |
cluster based on a virtualization software. The current version (2.3) |
26 |
supports Xen 3.x and KVM (72 or above) as hypervisors, and LXC as an |
27 |
experimental hypervisor. |
28 |
|
29 |
Quick start |
30 |
----------- |
31 |
|
32 |
First you must install the software on all the cluster nodes, either |
33 |
from sources or (if available) from a package. The next step is to |
34 |
create the initial cluster configuration, using **gnt-cluster init**. |
35 |
|
36 |
Then you can add other nodes, or start creating instances. |
37 |
|
38 |
Cluster architecture |
39 |
-------------------- |
40 |
|
41 |
In Ganeti 2.0, the architecture of the cluster is a little more |
42 |
complicated than in 1.2. The cluster is coordinated by a master daemon |
43 |
(**ganeti-masterd**\(8)), running on the master node. Each node runs |
44 |
(as before) a node daemon, and the master has the RAPI daemon running |
45 |
too. |
46 |
|
47 |
Node roles |
48 |
~~~~~~~~~~ |
49 |
|
50 |
Each node can be in one of the following states: |
51 |
|
52 |
master |
53 |
Only one node per cluster can be in this role, and this node is the |
54 |
one holding the authoritative copy of the cluster configuration and |
55 |
the one that can actually execute commands on the cluster and |
56 |
modify the cluster state. See more details under |
57 |
*Cluster configuration*. |
58 |
|
59 |
master_candidate |
60 |
The node receives the full cluster configuration (configuration |
61 |
file and jobs) and can become a master via the |
62 |
**gnt-cluster master-failover** command. Nodes that are not in this |
63 |
state cannot transition into the master role due to missing state. |
64 |
|
65 |
regular |
66 |
This the normal state of a node. |
67 |
|
68 |
drained |
69 |
Nodes in this state are functioning normally but cannot receive |
70 |
new instances, because the intention is to set them to *offline* |
71 |
or remove them from the cluster. |
72 |
|
73 |
offline |
74 |
These nodes are still recorded in the Ganeti configuration, but |
75 |
except for the master daemon startup voting procedure, they are not |
76 |
actually contacted by the master. This state was added in order to |
77 |
allow broken machines (that are being repaired) to remain in the |
78 |
cluster but without creating problems. |
79 |
|
80 |
|
81 |
Node flags |
82 |
~~~~~~~~~~ |
83 |
|
84 |
Nodes have two flags which govern which roles they can take: |
85 |
|
86 |
master_capable |
87 |
The node can become a master candidate, and furthermore the master |
88 |
node. When this flag is disabled, the node cannot become a |
89 |
candidate; this can be useful for special networking cases, or less |
90 |
reliable hardware. |
91 |
|
92 |
vm_capable |
93 |
The node can host instances. When enabled (the default state), the |
94 |
node will participate in instance allocation, capacity calculation, |
95 |
etc. When disabled, the node will be skipped in many cluster checks |
96 |
and operations. |
97 |
|
98 |
|
99 |
Node Parameters |
100 |
~~~~~~~~~~~~~~~ |
101 |
|
102 |
The ``ndparams`` refer to node parameters. These can be set as defaults |
103 |
on cluster and node group levels, but they take effect for nodes only. |
104 |
|
105 |
Currently we support the following node parameters: |
106 |
|
107 |
oob_program |
108 |
Path to an executable used as the out-of-band helper as described in |
109 |
the `Ganeti Node OOB Management Framework <design-oob.rst>`_ design |
110 |
document. |
111 |
|
112 |
spindle_count |
113 |
This should reflect the I/O performance of local attached storage |
114 |
(e.g. for "file", "plain" and "drbd" disk templates). It doesn't |
115 |
have to match the actual spindle count of (any eventual) mechanical |
116 |
hard-drives, its meaning is site-local and just the relative values |
117 |
matter. |
118 |
|
119 |
exclusive_storage |
120 |
When this Boolean flag is enabled, physical disks on the node are |
121 |
assigned to instance disks in an exclusive manner, so as to lower I/O |
122 |
interference between instances. See the `Partitioned Ganeti |
123 |
<design-partitioned.rst>`_ design document for more details. This |
124 |
parameter cannot be set on individual nodes, as its value must be |
125 |
the same within each node group. |
126 |
|
127 |
ovs |
128 |
When this Boolean flag is enabled, OpenvSwitch will be used as the |
129 |
network layer. This will cause the initialization of OpenvSwitch on |
130 |
the nodes when added to the cluster. Per default this is not enabled. |
131 |
|
132 |
ovs_name |
133 |
When ovs is enabled, this parameter will represent the name of the |
134 |
OpenvSwitch to generate and use. This will default to `switch1`. |
135 |
|
136 |
ovs_link |
137 |
When ovs is enabled, a OpenvSwitch will be initialized on new nodes |
138 |
and will have this as its connection to the outside. This parameter |
139 |
is not set per default, as it depends very much on the specific |
140 |
setup. |
141 |
|
142 |
|
143 |
Hypervisor State Parameters |
144 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
145 |
|
146 |
Using ``--hypervisor-state`` you can set hypervisor specific states as |
147 |
pointed out in ``Ganeti Resource Model <design-resource-model.rst>``. |
148 |
|
149 |
The format is: ``hypervisor:option=value``. |
150 |
|
151 |
Currently we support the following hypervisor state values: |
152 |
|
153 |
mem_total |
154 |
Total node memory, as discovered by this hypervisor |
155 |
mem_node |
156 |
Memory used by, or reserved for, the node itself; note that some |
157 |
hypervisors can report this in an authoritative way, other not |
158 |
mem_hv |
159 |
Memory used either by the hypervisor itself or lost due to instance |
160 |
allocation rounding; usually this cannot be precisely computed, but |
161 |
only roughly estimated |
162 |
cpu_total |
163 |
Total node cpu (core) count; usually this can be discovered |
164 |
automatically |
165 |
cpu_node |
166 |
Number of cores reserved for the node itself; this can either be |
167 |
discovered or set manually. Only used for estimating how many VCPUs |
168 |
are left for instances |
169 |
|
170 |
Note that currently this option is unused by Ganeti; values will be |
171 |
recorded but will not influence the Ganeti operation. |
172 |
|
173 |
|
174 |
Disk State Parameters |
175 |
~~~~~~~~~~~~~~~~~~~~~ |
176 |
|
177 |
Using ``--disk-state`` you can set disk specific states as pointed out |
178 |
in ``Ganeti Resource Model <design-resource-model.rst>``. |
179 |
|
180 |
The format is: ``storage_type/identifier:option=value``. Where we |
181 |
currently just support ``lvm`` as storage type. The identifier in this |
182 |
case is the LVM volume group. By default this is ``xenvg``. |
183 |
|
184 |
Currently we support the following hypervisor state values: |
185 |
|
186 |
disk_total |
187 |
Total disk size (usually discovered automatically) |
188 |
disk_reserved |
189 |
Reserved disk size; this is a lower limit on the free space, if such a |
190 |
limit is desired |
191 |
disk_overhead |
192 |
Disk that is expected to be used by other volumes (set via |
193 |
``reserved_lvs``); usually should be zero |
194 |
|
195 |
Note that currently this option is unused by Ganeti; values will be |
196 |
recorded but will not influence the Ganeti operation. |
197 |
|
198 |
|
199 |
Cluster configuration |
200 |
~~~~~~~~~~~~~~~~~~~~~ |
201 |
|
202 |
The master node keeps and is responsible for the cluster |
203 |
configuration. On the filesystem, this is stored under the |
204 |
``@LOCALSTATEDIR@/ganeti/lib`` directory, and if the master daemon is |
205 |
stopped it can be backed up normally. |
206 |
|
207 |
The master daemon will replicate the configuration database called |
208 |
``config.data`` and the job files to all the nodes in the master |
209 |
candidate role. It will also distribute a copy of some configuration |
210 |
values via the *ssconf* files, which are stored in the same directory |
211 |
and start with a ``ssconf_`` prefix, to all nodes. |
212 |
|
213 |
Jobs |
214 |
~~~~ |
215 |
|
216 |
All cluster modification are done via jobs. A job consists of one |
217 |
or more opcodes, and the list of opcodes is processed serially. If |
218 |
an opcode fails, the entire job is failed and later opcodes are no |
219 |
longer processed. A job can be in one of the following states: |
220 |
|
221 |
queued |
222 |
The job has been submitted but not yet processed by the master |
223 |
daemon. |
224 |
|
225 |
waiting |
226 |
The job is waiting for for locks before the first of its opcodes. |
227 |
|
228 |
canceling |
229 |
The job is waiting for locks, but is has been marked for |
230 |
cancellation. It will not transition to *running*, but to |
231 |
*canceled*. |
232 |
|
233 |
running |
234 |
The job is currently being executed. |
235 |
|
236 |
canceled |
237 |
The job has been canceled before starting execution. |
238 |
|
239 |
success |
240 |
The job has finished successfully. |
241 |
|
242 |
error |
243 |
The job has failed during runtime, or the master daemon has been |
244 |
stopped during the job execution. |
245 |
|
246 |
|
247 |
Common command line features |
248 |
---------------------------- |
249 |
|
250 |
Options |
251 |
~~~~~~~ |
252 |
|
253 |
Many Ganeti commands provide the following options. The |
254 |
availability for a certain command can be checked by calling the |
255 |
command using the ``--help`` option. |
256 |
|
257 |
| **gnt-...** *command* [\--dry-run] [\--priority {low | normal | high}] |
258 |
| [\--submit] [\--print-job-id] |
259 |
|
260 |
The ``--dry-run`` option can be used to check whether an operation |
261 |
would succeed. |
262 |
|
263 |
The option ``--priority`` sets the priority for opcodes submitted |
264 |
by the command. |
265 |
|
266 |
The ``--submit`` option is used to send the job to the master daemon but |
267 |
not wait for its completion. The job ID will be shown so that it can be |
268 |
examined using **gnt-job info**. |
269 |
|
270 |
The ``--print-job-id`` option makes the command print the job id as first |
271 |
line on stdout, so that it is easy to parse by other programs. |
272 |
|
273 |
Defaults |
274 |
~~~~~~~~ |
275 |
|
276 |
For certain commands you can use environment variables to provide |
277 |
default command line arguments. Just assign the arguments as a string to |
278 |
the corresponding environment variable. The format of that variable |
279 |
name is **binary**_*command*. **binary** is the name of the ``gnt-*`` |
280 |
script all upper case and dashes replaced by underscores, and *command* |
281 |
is the command invoked on that script. |
282 |
|
283 |
Currently supported commands are ``gnt-node list``, ``gnt-group list`` |
284 |
and ``gnt-instance list``. So you can configure default command line |
285 |
flags by setting ``GNT_NODE_LIST``, ``GNT_GROUP_LIST`` and |
286 |
``GNT_INSTANCE_LIST``. |
287 |
|
288 |
Debug options |
289 |
~~~~~~~~~~~~~ |
290 |
|
291 |
If the variable ``FORCE_LUXI_SOCKET`` is set, it will override the |
292 |
socket used for LUXI connections by command-line tools |
293 |
(``gnt-*``). This is useful mostly for debugging, and some operations |
294 |
won't work at all if, for example, you point this variable to the |
295 |
confd-supplied query socket and try to submit a job. |
296 |
|
297 |
If the variable is set to the value ``master``, it will connect to the |
298 |
correct path for the master daemon (even if, for example, split |
299 |
queries are enabled and this is a query operation). If set to |
300 |
``query``, it will always (try to) connect to the query socket, even |
301 |
if split queries are disabled. Otherwise, the value is taken to |
302 |
represent a filesystem path to the socket to use. |
303 |
|
304 |
Field formatting |
305 |
---------------- |
306 |
|
307 |
Multiple ganeti commands use the same framework for tabular listing of |
308 |
resources (e.g. **gnt-instance list**, **gnt-node list**, **gnt-group |
309 |
list**, **gnt-debug locks**, etc.). For these commands, special states |
310 |
are denoted via a special symbol (in terse mode) or a string (in |
311 |
verbose mode): |
312 |
|
313 |
\*, (offline) |
314 |
The node in question is marked offline, and thus it cannot be |
315 |
queried for data. This result is persistent until the node is |
316 |
de-offlined. |
317 |
|
318 |
?, (nodata) |
319 |
Ganeti expected to receive an answer from this entity, but the |
320 |
cluster RPC call failed and/or we didn't receive a valid answer; |
321 |
usually more information is available in the node daemon log (if |
322 |
the node is alive) or the master daemon log. This result is |
323 |
transient, and re-running command might return a different result. |
324 |
|
325 |
-, (unavail) |
326 |
The respective field doesn't make sense for this entity; |
327 |
e.g. querying a down instance for its current memory 'live' usage, |
328 |
or querying a non-vm_capable node for disk/memory data. This |
329 |
result is persistent, and until the entity state is changed via |
330 |
ganeti commands, the result won't change. |
331 |
|
332 |
??, (unknown) |
333 |
This field is not known (note that this is different from entity |
334 |
being unknown). Either you have mis-typed the field name, or you |
335 |
are using a field that the running Ganeti master daemon doesn't |
336 |
know. This result is persistent, re-running the command won't |
337 |
change it. |
338 |
|
339 |
Key-value parameters |
340 |
~~~~~~~~~~~~~~~~~~~~ |
341 |
|
342 |
Multiple options take parameters that are of the form |
343 |
``key=value,key=value,...`` or ``category:key=value,...``. Examples |
344 |
are the hypervisor parameters, backend parameters, etc. For these, |
345 |
it's possible to use values that contain commas by escaping with via a |
346 |
backslash (which needs two if not single-quoted, due to shell |
347 |
behaviour):: |
348 |
|
349 |
# gnt-instance modify -H kernel_path=an\\,example instance1 |
350 |
# gnt-instance modify -H kernel_path='an\,example' instance1 |
351 |
|
352 |
Query filters |
353 |
~~~~~~~~~~~~~ |
354 |
|
355 |
Most commands listing resources (e.g. instances or nodes) support filtering. |
356 |
The filter language is similar to Python expressions with some elements from |
357 |
Perl. The language is not generic. Each condition must consist of a field name |
358 |
and a value (except for boolean checks), a field can not be compared to another |
359 |
field. Keywords are case-sensitive. |
360 |
|
361 |
Examples (see below for syntax details): |
362 |
|
363 |
- List webservers:: |
364 |
|
365 |
gnt-instance list --filter 'name =* "web*.example.com"' |
366 |
|
367 |
- List instances with three or six virtual CPUs and whose primary |
368 |
nodes reside in groups starting with the string "rack":: |
369 |
|
370 |
gnt-instance list --filter |
371 |
'(be/vcpus == 3 or be/vcpus == 6) and pnode.group =~ m/^rack/' |
372 |
|
373 |
- Nodes hosting primary instances:: |
374 |
|
375 |
gnt-node list --filter 'pinst_cnt != 0' |
376 |
|
377 |
- Nodes which aren't master candidates:: |
378 |
|
379 |
gnt-node list --filter 'not master_candidate' |
380 |
|
381 |
- Short version for globbing patterns:: |
382 |
|
383 |
gnt-instance list '*.site1' '*.site2' |
384 |
|
385 |
Syntax in pseudo-BNF:: |
386 |
|
387 |
<quoted-string> ::= /* String quoted with single or double quotes, |
388 |
backslash for escaping */ |
389 |
|
390 |
<integer> ::= /* Number in base-10 positional notation */ |
391 |
|
392 |
<re> ::= /* Regular expression */ |
393 |
|
394 |
/* |
395 |
Modifier "i": Case-insensitive matching, see |
396 |
http://docs.python.org/library/re#re.IGNORECASE |
397 |
|
398 |
Modifier "s": Make the "." special character match any character, |
399 |
including newline, see http://docs.python.org/library/re#re.DOTALL |
400 |
*/ |
401 |
<re-modifiers> ::= /* empty */ | i | s |
402 |
|
403 |
<value> ::= <quoted-string> | <integer> |
404 |
|
405 |
<condition> ::= |
406 |
{ /* Value comparison */ |
407 |
<field> { == | != | < | <= | >= | > } <value> |
408 |
|
409 |
/* Collection membership */ |
410 |
| <value> [ not ] in <field> |
411 |
|
412 |
/* Regular expressions (recognized delimiters |
413 |
are "/", "#", "^", and "|"; backslash for escaping) |
414 |
*/ |
415 |
| <field> { =~ | !~ } m/<re>/<re-modifiers> |
416 |
|
417 |
/* Globbing */ |
418 |
| <field> { =* | !* } <quoted-string> |
419 |
|
420 |
/* Boolean */ |
421 |
| <field> |
422 |
} |
423 |
|
424 |
<filter> ::= |
425 |
{ [ not ] <condition> | ( <filter> ) } |
426 |
[ { and | or } <filter> ] |
427 |
|
428 |
Operators: |
429 |
|
430 |
*==* |
431 |
Equality |
432 |
*!=* |
433 |
Inequality |
434 |
*<* |
435 |
Less than |
436 |
*<=* |
437 |
Less than or equal |
438 |
*>* |
439 |
Greater than |
440 |
*>=* |
441 |
Greater than or equal |
442 |
*=~* |
443 |
Pattern match using regular expression |
444 |
*!~* |
445 |
Logically negated from *=~* |
446 |
*=\** |
447 |
Globbing, see **glob**\(7), though only * and ? are supported |
448 |
*!\** |
449 |
Logically negated from *=\** |
450 |
*in*, *not in* |
451 |
Collection membership and negation |
452 |
|
453 |
|
454 |
Common daemon functionality |
455 |
--------------------------- |
456 |
|
457 |
All Ganeti daemons re-open the log file(s) when sent a SIGHUP signal. |
458 |
**logrotate**\(8) can be used to rotate Ganeti's log files. |
459 |
|
460 |
.. vim: set textwidth=72 : |
461 |
.. Local Variables: |
462 |
.. mode: rst |
463 |
.. fill-column: 72 |
464 |
.. End: |