root / man / ganeti.rst @ d6cd74dd
History | View | Annotate | Download (14.1 kB)
1 |
ganeti(7) Ganeti | Version @GANETI_VERSION@ |
---|---|
2 |
=========================================== |
3 |
|
4 |
Name |
5 |
---- |
6 |
|
7 |
ganeti - cluster-based virtualization management |
8 |
|
9 |
Synopsis |
10 |
-------- |
11 |
|
12 |
:: |
13 |
|
14 |
# gnt-cluster init cluster1.example.com |
15 |
# gnt-node add node2.example.com |
16 |
# gnt-instance add -n node2.example.com \ |
17 |
> -o debootstrap --disk 0:size=30g \ |
18 |
> -t plain instance1.example.com |
19 |
|
20 |
|
21 |
DESCRIPTION |
22 |
----------- |
23 |
|
24 |
The Ganeti software manages physical nodes and virtual instances of a |
25 |
cluster based on a virtualization software. The current version (2.3) |
26 |
supports Xen 3.x and KVM (72 or above) as hypervisors, and LXC as an |
27 |
experimental hypervisor. |
28 |
|
29 |
Quick start |
30 |
----------- |
31 |
|
32 |
First you must install the software on all the cluster nodes, either |
33 |
from sources or (if available) from a package. The next step is to |
34 |
create the initial cluster configuration, using **gnt-cluster init**. |
35 |
|
36 |
Then you can add other nodes, or start creating instances. |
37 |
|
38 |
Cluster architecture |
39 |
-------------------- |
40 |
|
41 |
In Ganeti 2.0, the architecture of the cluster is a little more |
42 |
complicated than in 1.2. The cluster is coordinated by a master daemon |
43 |
(**ganeti-masterd**\(8)), running on the master node. Each node runs |
44 |
(as before) a node daemon, and the master has the RAPI daemon running |
45 |
too. |
46 |
|
47 |
Node roles |
48 |
~~~~~~~~~~ |
49 |
|
50 |
Each node can be in one of the following states: |
51 |
|
52 |
master |
53 |
Only one node per cluster can be in this role, and this node is the |
54 |
one holding the authoritative copy of the cluster configuration and |
55 |
the one that can actually execute commands on the cluster and |
56 |
modify the cluster state. See more details under |
57 |
*Cluster configuration*. |
58 |
|
59 |
master_candidate |
60 |
The node receives the full cluster configuration (configuration |
61 |
file and jobs) and can become a master via the |
62 |
**gnt-cluster master-failover** command. Nodes that are not in this |
63 |
state cannot transition into the master role due to missing state. |
64 |
|
65 |
regular |
66 |
This the normal state of a node. |
67 |
|
68 |
drained |
69 |
Nodes in this state are functioning normally but cannot receive |
70 |
new instances, because the intention is to set them to *offline* |
71 |
or remove them from the cluster. |
72 |
|
73 |
offline |
74 |
These nodes are still recorded in the Ganeti configuration, but |
75 |
except for the master daemon startup voting procedure, they are not |
76 |
actually contacted by the master. This state was added in order to |
77 |
allow broken machines (that are being repaired) to remain in the |
78 |
cluster but without creating problems. |
79 |
|
80 |
|
81 |
Node flags |
82 |
~~~~~~~~~~ |
83 |
|
84 |
Nodes have two flags which govern which roles they can take: |
85 |
|
86 |
master_capable |
87 |
The node can become a master candidate, and furthermore the master |
88 |
node. When this flag is disabled, the node cannot become a |
89 |
candidate; this can be useful for special networking cases, or less |
90 |
reliable hardware. |
91 |
|
92 |
vm_capable |
93 |
The node can host instances. When enabled (the default state), the |
94 |
node will participate in instance allocation, capacity calculation, |
95 |
etc. When disabled, the node will be skipped in many cluster checks |
96 |
and operations. |
97 |
|
98 |
|
99 |
Node Parameters |
100 |
~~~~~~~~~~~~~~~ |
101 |
|
102 |
The ``ndparams`` refer to node parameters. These can be set as defaults |
103 |
on cluster and node group levels, but they take effect for nodes only. |
104 |
|
105 |
Currently we support the following node parameters: |
106 |
|
107 |
oob_program |
108 |
Path to an executable used as the out-of-band helper as described in |
109 |
the `Ganeti Node OOB Management Framework <design-oob.rst>`_ design |
110 |
document. |
111 |
|
112 |
spindle_count |
113 |
This should reflect the I/O performance of local attached storage |
114 |
(e.g. for "file", "plain" and "drbd" disk templates). It doesn't |
115 |
have to match the actual spindle count of (any eventual) mechanical |
116 |
hard-drives, its meaning is site-local and just the relative values |
117 |
matter. |
118 |
|
119 |
exclusive_storage |
120 |
When this Boolean flag is enabled, physical disks on the node are |
121 |
assigned to instance disks in an exclusive manner, so as to lower I/O |
122 |
interference between instances. See the `Partitioned Ganeti |
123 |
<design-partitioned.rst>`_ design document for more details. This |
124 |
parameter cannot be set on individual nodes, as its value must be |
125 |
the same within each node group. |
126 |
|
127 |
|
128 |
Hypervisor State Parameters |
129 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
130 |
|
131 |
Using ``--hypervisor-state`` you can set hypervisor specific states as |
132 |
pointed out in ``Ganeti Resource Model <design-resource-model.rst>``. |
133 |
|
134 |
The format is: ``hypervisor:option=value``. |
135 |
|
136 |
Currently we support the following hypervisor state values: |
137 |
|
138 |
mem_total |
139 |
Total node memory, as discovered by this hypervisor |
140 |
mem_node |
141 |
Memory used by, or reserved for, the node itself; note that some |
142 |
hypervisors can report this in an authoritative way, other not |
143 |
mem_hv |
144 |
Memory used either by the hypervisor itself or lost due to instance |
145 |
allocation rounding; usually this cannot be precisely computed, but |
146 |
only roughly estimated |
147 |
cpu_total |
148 |
Total node cpu (core) count; usually this can be discovered |
149 |
automatically |
150 |
cpu_node |
151 |
Number of cores reserved for the node itself; this can either be |
152 |
discovered or set manually. Only used for estimating how many VCPUs |
153 |
are left for instances |
154 |
|
155 |
Note that currently this option is unused by Ganeti; values will be |
156 |
recorded but will not influence the Ganeti operation. |
157 |
|
158 |
|
159 |
Disk State Parameters |
160 |
~~~~~~~~~~~~~~~~~~~~~ |
161 |
|
162 |
Using ``--disk-state`` you can set disk specific states as pointed out |
163 |
in ``Ganeti Resource Model <design-resource-model.rst>``. |
164 |
|
165 |
The format is: ``storage_type/identifier:option=value``. Where we |
166 |
currently just support ``lvm`` as storage type. The identifier in this |
167 |
case is the LVM volume group. By default this is ``xenvg``. |
168 |
|
169 |
Currently we support the following hypervisor state values: |
170 |
|
171 |
disk_total |
172 |
Total disk size (usually discovered automatically) |
173 |
disk_reserved |
174 |
Reserved disk size; this is a lower limit on the free space, if such a |
175 |
limit is desired |
176 |
disk_overhead |
177 |
Disk that is expected to be used by other volumes (set via |
178 |
``reserved_lvs``); usually should be zero |
179 |
|
180 |
Note that currently this option is unused by Ganeti; values will be |
181 |
recorded but will not influence the Ganeti operation. |
182 |
|
183 |
|
184 |
Cluster configuration |
185 |
~~~~~~~~~~~~~~~~~~~~~ |
186 |
|
187 |
The master node keeps and is responsible for the cluster |
188 |
configuration. On the filesystem, this is stored under the |
189 |
``@LOCALSTATEDIR@/ganeti/lib`` directory, and if the master daemon is |
190 |
stopped it can be backed up normally. |
191 |
|
192 |
The master daemon will replicate the configuration database called |
193 |
``config.data`` and the job files to all the nodes in the master |
194 |
candidate role. It will also distribute a copy of some configuration |
195 |
values via the *ssconf* files, which are stored in the same directory |
196 |
and start with a ``ssconf_`` prefix, to all nodes. |
197 |
|
198 |
Jobs |
199 |
~~~~ |
200 |
|
201 |
All cluster modification are done via jobs. A job consists of one |
202 |
or more opcodes, and the list of opcodes is processed serially. If |
203 |
an opcode fails, the entire job is failed and later opcodes are no |
204 |
longer processed. A job can be in one of the following states: |
205 |
|
206 |
queued |
207 |
The job has been submitted but not yet processed by the master |
208 |
daemon. |
209 |
|
210 |
waiting |
211 |
The job is waiting for for locks before the first of its opcodes. |
212 |
|
213 |
canceling |
214 |
The job is waiting for locks, but is has been marked for |
215 |
cancellation. It will not transition to *running*, but to |
216 |
*canceled*. |
217 |
|
218 |
running |
219 |
The job is currently being executed. |
220 |
|
221 |
canceled |
222 |
The job has been canceled before starting execution. |
223 |
|
224 |
success |
225 |
The job has finished successfully. |
226 |
|
227 |
error |
228 |
The job has failed during runtime, or the master daemon has been |
229 |
stopped during the job execution. |
230 |
|
231 |
|
232 |
Common command line features |
233 |
---------------------------- |
234 |
|
235 |
Options |
236 |
~~~~~~~ |
237 |
|
238 |
Many Ganeti commands provide the following options. The |
239 |
availability for a certain command can be checked by calling the |
240 |
command using the ``--help`` option. |
241 |
|
242 |
| **gnt-...** *command* [\--dry-run] [\--priority {low | normal | high}] |
243 |
| [\--submit] [\--print-job-id] |
244 |
|
245 |
The ``--dry-run`` option can be used to check whether an operation |
246 |
would succeed. |
247 |
|
248 |
The option ``--priority`` sets the priority for opcodes submitted |
249 |
by the command. |
250 |
|
251 |
The ``--submit`` option is used to send the job to the master daemon but |
252 |
not wait for its completion. The job ID will be shown so that it can be |
253 |
examined using **gnt-job info**. |
254 |
|
255 |
The ``--print-job-id`` option makes the command print the job id as first |
256 |
line on stdout, so that it is easy to parse by other programs. |
257 |
|
258 |
Defaults |
259 |
~~~~~~~~ |
260 |
|
261 |
For certain commands you can use environment variables to provide |
262 |
default command line arguments. Just assign the arguments as a string to |
263 |
the corresponding environment variable. The format of that variable |
264 |
name is **binary**_*command*. **binary** is the name of the ``gnt-*`` |
265 |
script all upper case and dashes replaced by underscores, and *command* |
266 |
is the command invoked on that script. |
267 |
|
268 |
Currently supported commands are ``gnt-node list``, ``gnt-group list`` |
269 |
and ``gnt-instance list``. So you can configure default command line |
270 |
flags by setting ``GNT_NODE_LIST``, ``GNT_GROUP_LIST`` and |
271 |
``GNT_INSTANCE_LIST``. |
272 |
|
273 |
Debug options |
274 |
~~~~~~~~~~~~~ |
275 |
|
276 |
If the variable ``FORCE_LUXI_SOCKET`` is set, it will override the |
277 |
socket used for LUXI connections by command-line tools |
278 |
(``gnt-*``). This is useful mostly for debugging, and some operations |
279 |
won't work at all if, for example, you point this variable to the |
280 |
confd-supplied query socket and try to submit a job. |
281 |
|
282 |
If the variable is set to the value ``master``, it will connect to the |
283 |
correct path for the master daemon (even if, for example, split |
284 |
queries are enabled and this is a query operation). If set to |
285 |
``query``, it will always (try to) connect to the query socket, even |
286 |
if split queries are disabled. Otherwise, the value is taken to |
287 |
represent a filesystem path to the socket to use. |
288 |
|
289 |
Field formatting |
290 |
---------------- |
291 |
|
292 |
Multiple ganeti commands use the same framework for tabular listing of |
293 |
resources (e.g. **gnt-instance list**, **gnt-node list**, **gnt-group |
294 |
list**, **gnt-debug locks**, etc.). For these commands, special states |
295 |
are denoted via a special symbol (in terse mode) or a string (in |
296 |
verbose mode): |
297 |
|
298 |
\*, (offline) |
299 |
The node in question is marked offline, and thus it cannot be |
300 |
queried for data. This result is persistent until the node is |
301 |
de-offlined. |
302 |
|
303 |
?, (nodata) |
304 |
Ganeti expected to receive an answer from this entity, but the |
305 |
cluster RPC call failed and/or we didn't receive a valid answer; |
306 |
usually more information is available in the node daemon log (if |
307 |
the node is alive) or the master daemon log. This result is |
308 |
transient, and re-running command might return a different result. |
309 |
|
310 |
-, (unavail) |
311 |
The respective field doesn't make sense for this entity; |
312 |
e.g. querying a down instance for its current memory 'live' usage, |
313 |
or querying a non-vm_capable node for disk/memory data. This |
314 |
result is persistent, and until the entity state is changed via |
315 |
ganeti commands, the result won't change. |
316 |
|
317 |
??, (unknown) |
318 |
This field is not known (note that this is different from entity |
319 |
being unknown). Either you have mis-typed the field name, or you |
320 |
are using a field that the running Ganeti master daemon doesn't |
321 |
know. This result is persistent, re-running the command won't |
322 |
change it. |
323 |
|
324 |
Key-value parameters |
325 |
~~~~~~~~~~~~~~~~~~~~ |
326 |
|
327 |
Multiple options take parameters that are of the form |
328 |
``key=value,key=value,...`` or ``category:key=value,...``. Examples |
329 |
are the hypervisor parameters, backend parameters, etc. For these, |
330 |
it's possible to use values that contain commas by escaping with via a |
331 |
backslash (which needs two if not single-quoted, due to shell |
332 |
behaviour):: |
333 |
|
334 |
# gnt-instance modify -H kernel_path=an\\,example instance1 |
335 |
# gnt-instance modify -H kernel_path='an\,example' instance1 |
336 |
|
337 |
Query filters |
338 |
~~~~~~~~~~~~~ |
339 |
|
340 |
Most commands listing resources (e.g. instances or nodes) support filtering. |
341 |
The filter language is similar to Python expressions with some elements from |
342 |
Perl. The language is not generic. Each condition must consist of a field name |
343 |
and a value (except for boolean checks), a field can not be compared to another |
344 |
field. Keywords are case-sensitive. |
345 |
|
346 |
Examples (see below for syntax details): |
347 |
|
348 |
- List webservers:: |
349 |
|
350 |
gnt-instance list --filter 'name =* "web*.example.com"' |
351 |
|
352 |
- List instances with three or six virtual CPUs and whose primary |
353 |
nodes reside in groups starting with the string "rack":: |
354 |
|
355 |
gnt-instance list --filter |
356 |
'(be/vcpus == 3 or be/vcpus == 6) and pnode.group =~ m/^rack/' |
357 |
|
358 |
- Nodes hosting primary instances:: |
359 |
|
360 |
gnt-node list --filter 'pinst_cnt != 0' |
361 |
|
362 |
- Nodes which aren't master candidates:: |
363 |
|
364 |
gnt-node list --filter 'not master_candidate' |
365 |
|
366 |
- Short version for globbing patterns:: |
367 |
|
368 |
gnt-instance list '*.site1' '*.site2' |
369 |
|
370 |
Syntax in pseudo-BNF:: |
371 |
|
372 |
<quoted-string> ::= /* String quoted with single or double quotes, |
373 |
backslash for escaping */ |
374 |
|
375 |
<integer> ::= /* Number in base-10 positional notation */ |
376 |
|
377 |
<re> ::= /* Regular expression */ |
378 |
|
379 |
/* |
380 |
Modifier "i": Case-insensitive matching, see |
381 |
http://docs.python.org/library/re#re.IGNORECASE |
382 |
|
383 |
Modifier "s": Make the "." special character match any character, |
384 |
including newline, see http://docs.python.org/library/re#re.DOTALL |
385 |
*/ |
386 |
<re-modifiers> ::= /* empty */ | i | s |
387 |
|
388 |
<value> ::= <quoted-string> | <integer> |
389 |
|
390 |
<condition> ::= |
391 |
{ /* Value comparison */ |
392 |
<field> { == | != | < | <= | >= | > } <value> |
393 |
|
394 |
/* Collection membership */ |
395 |
| <value> [ not ] in <field> |
396 |
|
397 |
/* Regular expressions (recognized delimiters |
398 |
are "/", "#", "^", and "|"; backslash for escaping) |
399 |
*/ |
400 |
| <field> { =~ | !~ } m/<re>/<re-modifiers> |
401 |
|
402 |
/* Globbing */ |
403 |
| <field> { =* | !* } <quoted-string> |
404 |
|
405 |
/* Boolean */ |
406 |
| <field> |
407 |
} |
408 |
|
409 |
<filter> ::= |
410 |
{ [ not ] <condition> | ( <filter> ) } |
411 |
[ { and | or } <filter> ] |
412 |
|
413 |
Operators: |
414 |
|
415 |
*==* |
416 |
Equality |
417 |
*!=* |
418 |
Inequality |
419 |
*<* |
420 |
Less than |
421 |
*<=* |
422 |
Less than or equal |
423 |
*>* |
424 |
Greater than |
425 |
*>=* |
426 |
Greater than or equal |
427 |
*=~* |
428 |
Pattern match using regular expression |
429 |
*!~* |
430 |
Logically negated from *=~* |
431 |
*=\** |
432 |
Globbing, see **glob**\(7), though only * and ? are supported |
433 |
*!\** |
434 |
Logically negated from *=\** |
435 |
*in*, *not in* |
436 |
Collection membership and negation |
437 |
|
438 |
|
439 |
Common daemon functionality |
440 |
--------------------------- |
441 |
|
442 |
All Ganeti daemons re-open the log file(s) when sent a SIGHUP signal. |
443 |
**logrotate**\(8) can be used to rotate Ganeti's log files. |
444 |
|
445 |
.. vim: set textwidth=72 : |
446 |
.. Local Variables: |
447 |
.. mode: rst |
448 |
.. fill-column: 72 |
449 |
.. End: |