root / man / ganeti.rst @ 354c4f62
History | View | Annotate | Download (13.2 kB)
1 |
ganeti(7) Ganeti | Version @GANETI_VERSION@ |
---|---|
2 |
=========================================== |
3 |
|
4 |
Name |
5 |
---- |
6 |
|
7 |
ganeti - cluster-based virtualization management |
8 |
|
9 |
Synopsis |
10 |
-------- |
11 |
|
12 |
:: |
13 |
|
14 |
# gnt-cluster init cluster1.example.com |
15 |
# gnt-node add node2.example.com |
16 |
# gnt-instance add -n node2.example.com \ |
17 |
> -o debootstrap --disk 0:size=30g \ |
18 |
> -t plain instance1.example.com |
19 |
|
20 |
|
21 |
DESCRIPTION |
22 |
----------- |
23 |
|
24 |
The Ganeti software manages physical nodes and virtual instances of a |
25 |
cluster based on a virtualization software. The current version (2.3) |
26 |
supports Xen 3.x and KVM (72 or above) as hypervisors, and LXC as an |
27 |
experimental hypervisor. |
28 |
|
29 |
Quick start |
30 |
----------- |
31 |
|
32 |
First you must install the software on all the cluster nodes, either |
33 |
from sources or (if available) from a package. The next step is to |
34 |
create the initial cluster configuration, using **gnt-cluster init**. |
35 |
|
36 |
Then you can add other nodes, or start creating instances. |
37 |
|
38 |
Cluster architecture |
39 |
-------------------- |
40 |
|
41 |
In Ganeti 2.0, the architecture of the cluster is a little more |
42 |
complicated than in 1.2. The cluster is coordinated by a master daemon |
43 |
(**ganeti-masterd**\(8)), running on the master node. Each node runs |
44 |
(as before) a node daemon, and the master has the RAPI daemon running |
45 |
too. |
46 |
|
47 |
Node roles |
48 |
~~~~~~~~~~ |
49 |
|
50 |
Each node can be in one of the following states: |
51 |
|
52 |
master |
53 |
Only one node per cluster can be in this role, and this node is the |
54 |
one holding the authoritative copy of the cluster configuration and |
55 |
the one that can actually execute commands on the cluster and |
56 |
modify the cluster state. See more details under |
57 |
*Cluster configuration*. |
58 |
|
59 |
master_candidate |
60 |
The node receives the full cluster configuration (configuration |
61 |
file and jobs) and can become a master via the |
62 |
**gnt-cluster master-failover** command. Nodes that are not in this |
63 |
state cannot transition into the master role due to missing state. |
64 |
|
65 |
regular |
66 |
This the normal state of a node. |
67 |
|
68 |
drained |
69 |
Nodes in this state are functioning normally but cannot receive |
70 |
new instances, because the intention is to set them to *offline* |
71 |
or remove them from the cluster. |
72 |
|
73 |
offline |
74 |
These nodes are still recorded in the Ganeti configuration, but |
75 |
except for the master daemon startup voting procedure, they are not |
76 |
actually contacted by the master. This state was added in order to |
77 |
allow broken machines (that are being repaired) to remain in the |
78 |
cluster but without creating problems. |
79 |
|
80 |
|
81 |
Node flags |
82 |
~~~~~~~~~~ |
83 |
|
84 |
Nodes have two flags which govern which roles they can take: |
85 |
|
86 |
master_capable |
87 |
The node can become a master candidate, and furthermore the master |
88 |
node. When this flag is disabled, the node cannot become a |
89 |
candidate; this can be useful for special networking cases, or less |
90 |
reliable hardware. |
91 |
|
92 |
vm_capable |
93 |
The node can host instances. When enabled (the default state), the |
94 |
node will participate in instance allocation, capacity calculation, |
95 |
etc. When disabled, the node will be skipped in many cluster checks |
96 |
and operations. |
97 |
|
98 |
|
99 |
Node Parameters |
100 |
~~~~~~~~~~~~~~~ |
101 |
|
102 |
The ``ndparams`` refer to node parameters. These can be set as defaults |
103 |
on cluster and node group levels, but they take effect for nodes only. |
104 |
|
105 |
Currently we support the following node parameters: |
106 |
|
107 |
oob_program |
108 |
Path to an executable used as the out-of-band helper as described in |
109 |
the `Ganeti Node OOB Management Framework <design-oob.rst>`_ design |
110 |
document. |
111 |
|
112 |
spindle_count |
113 |
This should reflect the I/O performance of local attached storage |
114 |
(e.g. for "file", "plain" and "drbd" disk templates). It doesn't |
115 |
have to match the actual spindle count of (any eventual) mechanical |
116 |
hard-drives, its meaning is site-local and just the relative values |
117 |
matter. |
118 |
|
119 |
exclusive_storage |
120 |
When this Boolean flag is enabled, physical disks on the node are |
121 |
assigned to instance disks in an exclusive manner, so as to lower I/O |
122 |
interference between instances. See the `Partitioned Ganeti |
123 |
<design-partitioned.rst>`_ design document for more details. |
124 |
|
125 |
|
126 |
Hypervisor State Parameters |
127 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
128 |
|
129 |
Using ``--hypervisor-state`` you can set hypervisor specific states as |
130 |
pointed out in ``Ganeti Resource Model <design-resource-model.rst>``. |
131 |
|
132 |
The format is: ``hypervisor:option=value``. |
133 |
|
134 |
Currently we support the following hypervisor state values: |
135 |
|
136 |
mem_total |
137 |
Total node memory, as discovered by this hypervisor |
138 |
mem_node |
139 |
Memory used by, or reserved for, the node itself; note that some |
140 |
hypervisors can report this in an authoritative way, other not |
141 |
mem_hv |
142 |
Memory used either by the hypervisor itself or lost due to instance |
143 |
allocation rounding; usually this cannot be precisely computed, but |
144 |
only roughly estimated |
145 |
cpu_total |
146 |
Total node cpu (core) count; usually this can be discovered |
147 |
automatically |
148 |
cpu_node |
149 |
Number of cores reserved for the node itself; this can either be |
150 |
discovered or set manually. Only used for estimating how many VCPUs |
151 |
are left for instances |
152 |
|
153 |
Note that currently this option is unused by Ganeti; values will be |
154 |
recorded but will not influence the Ganeti operation. |
155 |
|
156 |
|
157 |
Disk State Parameters |
158 |
~~~~~~~~~~~~~~~~~~~~~ |
159 |
|
160 |
Using ``--disk-state`` you can set disk specific states as pointed out |
161 |
in ``Ganeti Resource Model <design-resource-model.rst>``. |
162 |
|
163 |
The format is: ``storage_type/identifier:option=value``. Where we |
164 |
currently just support ``lvm`` as storage type. The identifier in this |
165 |
case is the LVM volume group. By default this is ``xenvg``. |
166 |
|
167 |
Currently we support the following hypervisor state values: |
168 |
|
169 |
disk_total |
170 |
Total disk size (usually discovered automatically) |
171 |
disk_reserved |
172 |
Reserved disk size; this is a lower limit on the free space, if such a |
173 |
limit is desired |
174 |
disk_overhead |
175 |
Disk that is expected to be used by other volumes (set via |
176 |
``reserved_lvs``); usually should be zero |
177 |
|
178 |
Note that currently this option is unused by Ganeti; values will be |
179 |
recorded but will not influence the Ganeti operation. |
180 |
|
181 |
|
182 |
Cluster configuration |
183 |
~~~~~~~~~~~~~~~~~~~~~ |
184 |
|
185 |
The master node keeps and is responsible for the cluster |
186 |
configuration. On the filesystem, this is stored under the |
187 |
``@LOCALSTATEDIR@/ganeti/lib`` directory, and if the master daemon is |
188 |
stopped it can be backed up normally. |
189 |
|
190 |
The master daemon will replicate the configuration database called |
191 |
``config.data`` and the job files to all the nodes in the master |
192 |
candidate role. It will also distribute a copy of some configuration |
193 |
values via the *ssconf* files, which are stored in the same directory |
194 |
and start with a ``ssconf_`` prefix, to all nodes. |
195 |
|
196 |
Jobs |
197 |
~~~~ |
198 |
|
199 |
All cluster modification are done via jobs. A job consists of one |
200 |
or more opcodes, and the list of opcodes is processed serially. If |
201 |
an opcode fails, the entire job is failed and later opcodes are no |
202 |
longer processed. A job can be in one of the following states: |
203 |
|
204 |
queued |
205 |
The job has been submitted but not yet processed by the master |
206 |
daemon. |
207 |
|
208 |
waiting |
209 |
The job is waiting for for locks before the first of its opcodes. |
210 |
|
211 |
canceling |
212 |
The job is waiting for locks, but is has been marked for |
213 |
cancellation. It will not transition to *running*, but to |
214 |
*canceled*. |
215 |
|
216 |
running |
217 |
The job is currently being executed. |
218 |
|
219 |
canceled |
220 |
The job has been canceled before starting execution. |
221 |
|
222 |
success |
223 |
The job has finished successfully. |
224 |
|
225 |
error |
226 |
The job has failed during runtime, or the master daemon has been |
227 |
stopped during the job execution. |
228 |
|
229 |
|
230 |
Common command line features |
231 |
---------------------------- |
232 |
|
233 |
Options |
234 |
~~~~~~~ |
235 |
|
236 |
Many Ganeti commands provide the following options. The |
237 |
availability for a certain command can be checked by calling the |
238 |
command using the ``--help`` option. |
239 |
|
240 |
| **gnt-...** *command* [\--dry-run] [\--priority {low | normal | high}] |
241 |
| [\--submit] |
242 |
|
243 |
The ``--dry-run`` option can be used to check whether an operation |
244 |
would succeed. |
245 |
|
246 |
The option ``--priority`` sets the priority for opcodes submitted |
247 |
by the command. |
248 |
|
249 |
The ``--submit`` option is used to send the job to the master daemon but |
250 |
not wait for its completion. The job ID will be shown so that it can be |
251 |
examined using **gnt-job info**. |
252 |
|
253 |
Defaults |
254 |
~~~~~~~~ |
255 |
|
256 |
For certain commands you can use environment variables to provide |
257 |
default command line arguments. Just assign the arguments as a string to |
258 |
the corresponding environment variable. The format of that variable |
259 |
name is **binary**_*command*. **binary** is the name of the ``gnt-*`` |
260 |
script all upper case and dashes replaced by underscores, and *command* |
261 |
is the command invoked on that script. |
262 |
|
263 |
Currently supported commands are ``gnt-node list``, ``gnt-group list`` |
264 |
and ``gnt-instance list``. So you can configure default command line |
265 |
flags by setting ``GNT_NODE_LIST``, ``GNT_GROUP_LIST`` and |
266 |
``GNT_INSTANCE_LIST``. |
267 |
|
268 |
Field formatting |
269 |
---------------- |
270 |
|
271 |
Multiple ganeti commands use the same framework for tabular listing of |
272 |
resources (e.g. **gnt-instance list**, **gnt-node list**, **gnt-group |
273 |
list**, **gnt-debug locks**, etc.). For these commands, special states |
274 |
are denoted via a special symbol (in terse mode) or a string (in |
275 |
verbose mode): |
276 |
|
277 |
\*, (offline) |
278 |
The node in question is marked offline, and thus it cannot be |
279 |
queried for data. This result is persistent until the node is |
280 |
de-offlined. |
281 |
|
282 |
?, (nodata) |
283 |
Ganeti expected to receive an answer from this entity, but the |
284 |
cluster RPC call failed and/or we didn't receive a valid answer; |
285 |
usually more information is available in the node daemon log (if |
286 |
the node is alive) or the master daemon log. This result is |
287 |
transient, and re-running command might return a different result. |
288 |
|
289 |
-, (unavail) |
290 |
The respective field doesn't make sense for this entity; |
291 |
e.g. querying a down instance for its current memory 'live' usage, |
292 |
or querying a non-vm_capable node for disk/memory data. This |
293 |
result is persistent, and until the entity state is changed via |
294 |
ganeti commands, the result won't change. |
295 |
|
296 |
??, (unknown) |
297 |
This field is not known (note that this is different from entity |
298 |
being unknown). Either you have mis-typed the field name, or you |
299 |
are using a field that the running Ganeti master daemon doesn't |
300 |
know. This result is persistent, re-running the command won't |
301 |
change it. |
302 |
|
303 |
Key-value parameters |
304 |
~~~~~~~~~~~~~~~~~~~~ |
305 |
|
306 |
Multiple options take parameters that are of the form |
307 |
``key=value,key=value,...`` or ``category:key=value,...``. Examples |
308 |
are the hypervisor parameters, backend parameters, etc. For these, |
309 |
it's possible to use values that contain commas by escaping with via a |
310 |
backslash (which needs two if not single-quoted, due to shell |
311 |
behaviour):: |
312 |
|
313 |
# gnt-instance modify -H kernel_path=an\\,example instance1 |
314 |
# gnt-instance modify -H kernel_path='an\,example' instance1 |
315 |
|
316 |
Query filters |
317 |
~~~~~~~~~~~~~ |
318 |
|
319 |
Most commands listing resources (e.g. instances or nodes) support filtering. |
320 |
The filter language is similar to Python expressions with some elements from |
321 |
Perl. The language is not generic. Each condition must consist of a field name |
322 |
and a value (except for boolean checks), a field can not be compared to another |
323 |
field. Keywords are case-sensitive. |
324 |
|
325 |
Examples (see below for syntax details): |
326 |
|
327 |
- List webservers:: |
328 |
|
329 |
gnt-instance list --filter 'name =* "web*.example.com"' |
330 |
|
331 |
- List instances with three or six virtual CPUs and whose primary |
332 |
nodes reside in groups starting with the string "rack":: |
333 |
|
334 |
gnt-instance list --filter |
335 |
'(be/vcpus == 3 or be/vcpus == 6) and pnode.group =~ m/^rack/' |
336 |
|
337 |
- Nodes hosting primary instances:: |
338 |
|
339 |
gnt-node list --filter 'pinst_cnt != 0' |
340 |
|
341 |
- Nodes which aren't master candidates:: |
342 |
|
343 |
gnt-node list --filter 'not master_candidate' |
344 |
|
345 |
- Short version for globbing patterns:: |
346 |
|
347 |
gnt-instance list '*.site1' '*.site2' |
348 |
|
349 |
Syntax in pseudo-BNF:: |
350 |
|
351 |
<quoted-string> ::= /* String quoted with single or double quotes, |
352 |
backslash for escaping */ |
353 |
|
354 |
<integer> ::= /* Number in base-10 positional notation */ |
355 |
|
356 |
<re> ::= /* Regular expression */ |
357 |
|
358 |
/* |
359 |
Modifier "i": Case-insensitive matching, see |
360 |
http://docs.python.org/library/re#re.IGNORECASE |
361 |
|
362 |
Modifier "s": Make the "." special character match any character, |
363 |
including newline, see http://docs.python.org/library/re#re.DOTALL |
364 |
*/ |
365 |
<re-modifiers> ::= /* empty */ | i | s |
366 |
|
367 |
<value> ::= <quoted-string> | <integer> |
368 |
|
369 |
<condition> ::= |
370 |
{ /* Value comparison */ |
371 |
<field> { == | != | < | <= | >= | > } <value> |
372 |
|
373 |
/* Collection membership */ |
374 |
| <value> [ not ] in <field> |
375 |
|
376 |
/* Regular expressions (recognized delimiters |
377 |
are "/", "#", "^", and "|"; backslash for escaping) |
378 |
*/ |
379 |
| <field> { =~ | !~ } m/<re>/<re-modifiers> |
380 |
|
381 |
/* Globbing */ |
382 |
| <field> { =* | !* } <quoted-string> |
383 |
|
384 |
/* Boolean */ |
385 |
| <field> |
386 |
} |
387 |
|
388 |
<filter> ::= |
389 |
{ [ not ] <condition> | ( <filter> ) } |
390 |
[ { and | or } <filter> ] |
391 |
|
392 |
Operators: |
393 |
|
394 |
*==* |
395 |
Equality |
396 |
*!=* |
397 |
Inequality |
398 |
*<* |
399 |
Less than |
400 |
*<=* |
401 |
Less than or equal |
402 |
*>* |
403 |
Greater than |
404 |
*>=* |
405 |
Greater than or equal |
406 |
*=~* |
407 |
Pattern match using regular expression |
408 |
*!~* |
409 |
Logically negated from *=~* |
410 |
*=\** |
411 |
Globbing, see **glob**\(7), though only * and ? are supported |
412 |
*!\** |
413 |
Logically negated from *=\** |
414 |
*in*, *not in* |
415 |
Collection membership and negation |
416 |
|
417 |
|
418 |
Common daemon functionality |
419 |
--------------------------- |
420 |
|
421 |
All Ganeti daemons re-open the log file(s) when sent a SIGHUP signal. |
422 |
**logrotate**\(8) can be used to rotate Ganeti's log files. |
423 |
|
424 |
.. vim: set textwidth=72 : |
425 |
.. Local Variables: |
426 |
.. mode: rst |
427 |
.. fill-column: 72 |
428 |
.. End: |