Statistics
| Branch: | Tag: | Revision:

root / man / gnt-cluster.sgml @ 7d713344

History | View | Annotate | Download (23.8 kB)

1
<!doctype refentry PUBLIC "-//OASIS//DTD DocBook V4.1//EN" [
2

    
3
  <!-- Fill in your name for FIRSTNAME and SURNAME. -->
4
  <!-- Please adjust the date whenever revising the manpage. -->
5
  <!ENTITY dhdate      "<date>February 12, 2009</date>">
6
  <!-- SECTION should be 1-8, maybe w/ subsection other parameters are
7
       allowed: see man(7), man(1). -->
8
  <!ENTITY dhsection   "<manvolnum>8</manvolnum>">
9
  <!ENTITY dhucpackage "<refentrytitle>gnt-cluster</refentrytitle>">
10
  <!ENTITY dhpackage   "gnt-cluster">
11

    
12
  <!ENTITY debian      "<productname>Debian</productname>">
13
  <!ENTITY gnu         "<acronym>GNU</acronym>">
14
  <!ENTITY gpl         "&gnu; <acronym>GPL</acronym>">
15
  <!ENTITY footer SYSTEM "footer.sgml">
16
]>
17

    
18
<refentry>
19
  <refentryinfo>
20
    <copyright>
21
      <year>2006</year>
22
      <year>2007</year>
23
      <year>2008</year>
24
      <year>2009</year>
25
      <holder>Google Inc.</holder>
26
    </copyright>
27
    &dhdate;
28
  </refentryinfo>
29
  <refmeta>
30
    &dhucpackage;
31

    
32
    &dhsection;
33
    <refmiscinfo>ganeti 2.0</refmiscinfo>
34
  </refmeta>
35
  <refnamediv>
36
    <refname>&dhpackage;</refname>
37

    
38
    <refpurpose>ganeti administration, cluster-wide</refpurpose>
39
  </refnamediv>
40
  <refsynopsisdiv>
41
    <cmdsynopsis>
42
      <command>&dhpackage; </command>
43

    
44
      <arg choice="req">command</arg>
45
      <arg>arguments...</arg>
46
    </cmdsynopsis>
47
  </refsynopsisdiv>
48
  <refsect1>
49
    <title>DESCRIPTION</title>
50

    
51
    <para>
52
      The <command>&dhpackage;</command> is used for cluster-wide
53
      administration in the ganeti system.
54
    </para>
55

    
56
  </refsect1>
57
  <refsect1>
58
    <title>COMMANDS</title>
59

    
60
    <refsect2>
61
      <title>ADD-TAGS</title>
62

    
63
      <cmdsynopsis>
64
        <command>add-tags</command>
65
        <arg choice="opt">--from <replaceable>file</replaceable></arg>
66
        <arg choice="req"
67
        rep="repeat"><replaceable>tag</replaceable></arg>
68
      </cmdsynopsis>
69

    
70
      <para>
71
        Add tags to the cluster. If any of the tags contains invalid
72
        characters, the entire operation will abort.
73
      </para>
74

    
75
      <para>
76
        If the <option>--from</option> option is given, the list of
77
        tags will be extended with the contents of that file (each
78
        line becomes a tag). In this case, there is not need to pass
79
        tags on the command line (if you do, both sources will be
80
        used). A file name of - will be interpreted as stdin.
81
      </para>
82
    </refsect2>
83

    
84
    <refsect2>
85
      <title>COMMAND</title>
86

    
87
      <cmdsynopsis>
88
        <command>command</command>
89
        <arg>-n <replaceable>node</replaceable></arg>
90
        <arg choice="req"><replaceable>command</replaceable></arg>
91
      </cmdsynopsis>
92

    
93
      <para>
94
        Executes a command on all nodes. If the option
95
        <option>-n</option> is not given, the command will be executed
96
        on all nodes, otherwise it will be executed only on the
97
        node(s) specified. Use the option multiple times for running
98
        it on multiple nodes, like:
99

    
100
        <screen>
101
          # gnt-cluster command -n node1.example.com -n node2.example.com date
102
        </screen>
103

    
104
      </para>
105

    
106
      <para>
107
        The command is executed serially on the selected nodes. If the
108
        master node is present in the list, the command will be
109
        executed last on the master. Regarding the other nodes, the
110
        execution order is somewhat alphabetic, so that
111
        node2.example.com will be earlier than node10.example.com but
112
        after node1.example.com.
113
      </para>
114

    
115
      <para>
116
        So given the node names node1, node2, node3, node10, node11,
117
        with node3 being the master, the order will be: node1, node2,
118
        node10, node11, node3.
119
      </para>
120

    
121
      <para>
122
        The command is constructed by concatenating all other command
123
        line arguments. For example, to list the contents of the
124
        <filename class="directory">/etc</filename> directory on all
125
        nodes, run:
126

    
127
        <screen>
128
          # gnt-cluster command ls -l /etc
129
        </screen>
130

    
131
        and the command which will be executed will be
132
        <computeroutput>"ls -l /etc"</computeroutput>
133
      </para>
134
    </refsect2>
135

    
136
    <refsect2>
137
      <title>COPYFILE</title>
138

    
139
      <cmdsynopsis>
140
        <command>copyfile</command>
141
        <arg>-n <replaceable>node</replaceable></arg>
142
        <arg choice="req"><replaceable>file</replaceable></arg>
143
      </cmdsynopsis>
144

    
145
      <para>
146
        Copies a file to all or to some nodes. The argument specifies
147
        the source file (on the current system), the
148
        <option>-n</option> argument specifies the target node, or
149
        nodes if the option is given multiple times. If
150
        <option>-n</option> is not given at all, the file will be
151
        copied to all nodes.
152

    
153
        Example:
154
        <screen>
155
          # gnt-cluster -n node1.example.com -n node2.example.com copyfile /tmp/test
156
        </screen>
157

    
158
        This will copy the file <filename>/tmp/test</filename> from
159
        the current node to the two named nodes.
160
      </para>
161
    </refsect2>
162

    
163
    <refsect2>
164
      <title>DESTROY</title>
165

    
166
      <cmdsynopsis>
167
        <command>destroy</command>
168
        <arg choice="req">--yes-do-it</arg>
169
      </cmdsynopsis>
170

    
171
      <para>
172
        Remove all configuration files related to the cluster, so that
173
        a <command>gnt-cluster init</command> can be done again
174
        afterwards.
175
      </para>
176

    
177
      <para>
178
        Since this is a dangerous command, you are required to pass
179
        the argument <replaceable>--yes-do-it.</replaceable>
180
      </para>
181
    </refsect2>
182

    
183
    <refsect2>
184
      <title>GETMASTER</title>
185

    
186
      <cmdsynopsis>
187
        <command>getmaster</command>
188
      </cmdsynopsis>
189

    
190
      <para>
191
        Displays the current master node.
192
      </para>
193
    </refsect2>
194

    
195
    <refsect2>
196
      <title>INFO</title>
197

    
198
      <cmdsynopsis>
199
        <command>info</command>
200
      </cmdsynopsis>
201

    
202
      <para>
203
        Shows runtime cluster information: cluster name, architecture
204
        (32 or 64 bit), master node, node list and instance list.
205
      </para>
206
    </refsect2>
207

    
208
    <refsect2>
209
      <title>INIT</title>
210

    
211
      <cmdsynopsis>
212
        <command>init</command>
213
        <sbr>
214
        <arg>-s <replaceable>secondary_ip</replaceable></arg>
215
        <sbr>
216
        <arg>-g <replaceable>vg-name</replaceable></arg>
217
        <sbr>
218
        <arg>--master-netdev <replaceable>vg-name</replaceable></arg>
219
        <sbr>
220
        <arg>-m <replaceable>mac-prefix</replaceable></arg>
221
        <sbr>
222
        <arg>--no-lvm-storage</arg>
223
        <sbr>
224
        <arg>--file-storage-dir <replaceable>dir</replaceable></arg>
225
        <sbr>
226
        <arg>--enabled-hypervisors <replaceable>hypervisors</replaceable></arg>
227
        <sbr>
228
        <arg>-t <replaceable>hypervisor name</replaceable></arg>
229
        <sbr>
230
        <arg>--hypervisor-parameters <replaceable>hypervisor</replaceable>:<replaceable>hv-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>hv-param</replaceable>=<replaceable>value</replaceable></arg></arg>
231
        <sbr>
232
        <arg>--backend-parameters <replaceable>be-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>be-param</replaceable>=<replaceable>value</replaceable></arg></arg>
233
        <sbr>
234
        <arg>--nic-parameters <replaceable>nic-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>nic-param</replaceable>=<replaceable>value</replaceable></arg></arg>
235
        <sbr>
236
        <arg choice="req"><replaceable>clustername</replaceable></arg>
237
      </cmdsynopsis>
238

    
239
      <para>
240
        This commands is only run once initially on the first node of
241
        the cluster. It will initialize the cluster configuration and
242
        setup ssh-keys and more.
243
      </para>
244

    
245
      <para>
246
        Note that the <replaceable>clustername</replaceable> is not
247
        any random name. It has to be resolvable to an IP address
248
        using DNS, and it is best if you give the fully-qualified
249
        domain name. This hostname must resolve to an IP address
250
        reserved exclusively for this purpose.
251
      </para>
252

    
253
      <para>
254
        The cluster can run in two modes: single-home or
255
        dual-homed. In the first case, all traffic (both public
256
        traffic, inter-node traffic and data replication traffic) goes
257
        over the same interface. In the dual-homed case, the data
258
        replication traffic goes over the second network. The
259
        <option>-s</option> option here marks the cluster as
260
        dual-homed and its parameter represents this node's address on
261
        the second network. If you initialise the cluster with
262
        <option>-s</option>, all nodes added must have a secondary IP
263
        as well.
264
      </para>
265

    
266
      <para>
267
        Note that for Ganeti it doesn't matter if the secondary
268
        network is actually a separate physical network, or is done
269
        using tunneling, etc. For performance reasons, it's
270
        recommended to use a separate network, of course.
271
      </para>
272

    
273
      <para>
274
        The <option>-g</option> option will let you specify a volume group
275
        different than 'xenvg' for ganeti to use when creating instance disks.
276
        This volume group must have the same name on all nodes. Once the
277
        cluster is initialized this can be altered by using the
278
        <command>modify</command> command. If you don't want to use lvm
279
        storage at all use the <option>--no-lvm-storage</option> option.
280
        Once the cluster is initialized you can change this setup with the
281
        <command>modify</command> command.
282
      </para>
283

    
284
      <para>
285
        The <option>--master-netdev</option> option is useful for specifying a
286
        different interface on which the master will activate its IP address.
287
        It's important that all nodes have this interface because you'll need
288
        it for a master failover.
289
      </para>
290

    
291
      <para>
292
        The <option>-m</option> option will let you specify a three byte prefix
293
        under which the virtual MAC addresses of your instances will be
294
        generated. The prefix must be specified in the format XX:XX:XX and the
295
        default is aa:00:00.
296
      </para>
297

    
298
      <para>
299
        The <option>--no-lvm-storage</option> allows you to initialize the
300
        cluster without lvm support. This means that only instances using
301
        files as storage backend will be possible to create. Once the cluster
302
        is initialized you can change this setup with the
303
        <command>modify</command> command.
304
      </para>
305

    
306
      <para>
307
        The <option>--file-storage-dir</option> option allows you
308
        set the directory to use for storing the instance disk
309
        files when using file storage as backend for instance disks.
310
      </para>
311

    
312
      <para>
313
        The <option>--enabled-hypervisors</option> option allows you
314
        to set the list of hypervisors that will be enabled for
315
        this cluster. Instance hypervisors can only be choosen from
316
        the list of enabled hypervisors. Currently, the following
317
        hypervisors are available:
318
      </para>
319

    
320
      <para>
321
        <variablelist>
322
          <varlistentry>
323
            <term>xen-pvm</term>
324
            <listitem>
325
              <para>
326
                Xen PVM hypervisor
327
              </para>
328
            </listitem>
329
          </varlistentry>
330
          <varlistentry>
331
            <term>xen-hvm</term>
332
            <listitem>
333
              <para>
334
                Xen HVM hypervisor
335
              </para>
336
            </listitem>
337
          </varlistentry>
338
          <varlistentry>
339
            <term>kvm</term>
340
            <listitem>
341
              <para>
342
                Linux KVM hypervisor
343
              </para>
344
            </listitem>
345
          </varlistentry>
346
          <varlistentry>
347
            <term>fake</term>
348
            <listitem>
349
              <para>
350
                fake hypervisor for development/testing
351
              </para>
352
            </listitem>
353
          </varlistentry>
354
        </variablelist>
355
      </para>
356

    
357
      <para>
358
        Either a single hypervisor name or a comma-separated list of
359
        hypervisor names can be specified. If this option is not
360
        specified, only the xen-pvm hypervisor is enabled by default.
361
      </para>
362

    
363
      <para>
364
        With the <option>-t</option> option, the default hypervisor
365
        can be set. It has to be a member of the list of enabled
366
        hypervisors. If not specified, the first entry on the list of
367
        enabled hypervisors will be used by default.
368
      </para>
369

    
370
      <para>
371
        The <option>--backend-parameters</option> option allows you to set
372
        the default backend parameters for the cluster. The parameter
373
        format is a comma-separated list of key=value pairs with the
374
        following supported keys:
375
      </para>
376

    
377
      <para>
378
        <variablelist>
379
          <varlistentry>
380
            <term>vcpus</term>
381
            <listitem>
382
              <para>
383
                Number of VCPUs to set for an instance by default, must
384
                be an integer, will be set to 1 if no specified.
385
              </para>
386
            </listitem>
387
          </varlistentry>
388
          <varlistentry>
389
            <term>memory</term>
390
            <listitem>
391
              <para>
392
                Amount of memory to allocate for an instance by default,
393
                can be either an integer or an integer followed by a
394
                unit (M for mebibytes and G for gibibytes are
395
                supported), will be set to 128M if not specified.
396
              </para>
397
            </listitem>
398
          </varlistentry>
399
          <varlistentry>
400
            <term>auto_balance</term>
401
            <listitem>
402
              <para>
403
                Value of the auto_balance flag for instances to use by
404
                default, will be set to true if not specified.
405
              </para>
406
            </listitem>
407
          </varlistentry>
408
        </variablelist>
409
      </para>
410

    
411
      <para>
412
        The <option>--hypervisor-parameters</option> option allows you
413
        to set default hypervisor specific parameters for the
414
        cluster. The format of this option is the name of the
415
        hypervisor, followed by a colon and a comma-separated list of
416
        key=value pairs. The keys available for each hypervisors are
417
        detailed int the <citerefentry>
418
        <refentrytitle>gnt-instance</refentrytitle>
419
        <manvolnum>8</manvolnum> </citerefentry> man page, in the
420
        <command>add</command> command.
421
      </para>
422

    
423
      <para>
424
        The <option>--nic-parameters</option> option allows you to set
425
        the default nic parameters for the cluster. The parameter
426
        format is a comma-separated list of key=value pairs with the
427
        following supported keys:
428
        <variablelist>
429
          <varlistentry>
430
            <term>mode</term>
431
            <listitem>
432
              <para>
433
                The default nic mode, 'routed' or 'bridged'.
434
              </para>
435
            </listitem>
436
          </varlistentry>
437
          <varlistentry>
438
            <term>link</term>
439
            <listitem>
440
              <para>
441
                In bridged mode the default NIC bridge. In routed mode
442
                it represents an hypervisor-vif-script dependent value
443
                to allow different instance groups (perhaps via multiple
444
                routing tables)
445
              </para>
446
            </listitem>
447
          </varlistentry>
448
        </variablelist>
449
      </para>
450

    
451
    </refsect2>
452

    
453
    <refsect2>
454
      <title>LIST-TAGS</title>
455

    
456
      <cmdsynopsis>
457
        <command>list-tags</command>
458
      </cmdsynopsis>
459

    
460
      <para>List the tags of the cluster.</para>
461
    </refsect2>
462

    
463
    <refsect2>
464
      <title>MASTERFAILOVER</title>
465

    
466
      <cmdsynopsis>
467
        <command>masterfailover</command>
468
      </cmdsynopsis>
469

    
470
      <para>
471
        Failover the master role to the current node.
472
      </para>
473
    </refsect2>
474

    
475
    <refsect2>
476
      <title>MODIFY</title>
477

    
478
      <cmdsynopsis>
479
        <command>modify</command>
480
        <sbr>
481
        <arg choice="opt">-g <replaceable>vg-name</replaceable></arg>
482
        <sbr>
483
        <arg choice="opt">--no-lvm-storage</arg>
484
        <sbr>
485
        <arg choice="opt">--enabled-hypervisors
486
        <replaceable>hypervisors</replaceable></arg>
487
        <sbr>
488
        <arg choice="opt">--hypervisor-parameters <replaceable>hypervisor</replaceable>:<replaceable>hv-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>hv-param</replaceable>=<replaceable>value</replaceable></arg></arg>
489
        <sbr>
490
        <arg choice="opt">--backend-parameters <replaceable>be-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>be-param</replaceable>=<replaceable>value</replaceable></arg></arg>
491
        <sbr>
492
        <arg choice="opt">--nic-parameters <replaceable>nic-param</replaceable>=<replaceable>value</replaceable><arg rep="repeat" choice="opt">,<replaceable>nic-param</replaceable>=<replaceable>value</replaceable></arg></arg>
493
        <sbr>
494
        <arg>-C <replaceable>candidate_pool_size</replaceable></arg>
495

    
496
      </cmdsynopsis>
497

    
498
        <para>
499
          Modify the options for the cluster.
500
        </para>
501

    
502
        <para>
503
          The <option>-g</option>, <option>--no-lvm-storarge</option>,
504
          <option>--enabled-hypervisors</option>,
505
          <option>--hypervisor-parameters</option>,
506
          <option>--backend-parameters</option> and
507
          <option>--nic-parameters</option> and options are
508
          described in the <command>init</command> command.
509
        </para>
510

    
511
      <para>
512
        The <option>-C</option> options specifies the
513
        <varname>candidate_pool_size</varname> cluster parameter. This
514
        is the number of nodes that the master will try to keep as
515
        <literal>master_candidates</literal>. For more details about
516
        this role and other node roles, see the <citerefentry>
517
        <refentrytitle>ganeti</refentrytitle><manvolnum>7</manvolnum>
518
        </citerefentry>. If you increase the size, the master will
519
        automatically promote as many nodes as required and possible
520
        to reach the intended number.
521
      </para>
522
    </refsect2>
523

    
524
    <refsect2>
525
      <title>QUEUE</title>
526

    
527
      <cmdsynopsis>
528
        <command>queue</command>
529
        <arg choice="opt">drain</arg>
530
        <arg choice="opt">undrain</arg>
531
        <arg choice="opt">info</arg>
532

    
533
      </cmdsynopsis>
534

    
535
      <para>
536
        Change job queue properties.
537
      </para>
538

    
539
      <para>
540
        The <option>drain</option> option sets the drain flag on the
541
        job queue. No new jobs will be accepted, but jobs already in
542
        the queue will be processed.
543
      </para>
544

    
545
      <para>
546
        The <option>undrain</option> will unset the drain flag on the
547
        job queue. New jobs will be accepted.
548
      </para>
549

    
550
      <para>
551
        The <option>info</option> option shows the properties of the
552
        job queue.
553
      </para>
554
    </refsect2>
555

    
556
    <refsect2>
557
      <title>redist-conf</title>
558
      <cmdsynopsis>
559
        <command>redist-conf</command>
560
          <arg>--submit</arg>
561
      </cmdsynopsis>
562

    
563
      <para>
564
        This command forces a full push of configuration files from
565
        the master node to the other nodes in the cluster. This is
566
        normally not needed, but can be run if the
567
        <command>verify</command> complains about configuration
568
        mismatches.
569
      </para>
570

    
571
      <para>
572
        The <option>--submit</option> option is used to send the job
573
        to the master daemon but not wait for its completion. The job
574
        ID will be shown so that it can be examined via
575
        <command>gnt-job info</command>.
576
      </para>
577

    
578
    </refsect2>
579
    <refsect2>
580
      <title>REMOVE-TAGS</title>
581

    
582
      <cmdsynopsis>
583
        <command>remove-tags</command>
584
        <arg choice="opt">--from <replaceable>file</replaceable></arg>
585
        <arg choice="req"
586
        rep="repeat"><replaceable>tag</replaceable></arg>
587
      </cmdsynopsis>
588

    
589
      <para>
590
        Remove tags from the cluster. If any of the tags are not
591
        existing on the cluster, the entire operation will abort.
592
      </para>
593

    
594
      <para>
595
        If the <option>--from</option> option is given, the list of
596
        tags will be extended with the contents of that file (each
597
        line becomes a tag). In this case, there is not need to pass
598
        tags on the command line (if you do, both sources will be
599
        used). A file name of - will be interpreted as stdin.
600
      </para>
601
    </refsect2>
602

    
603
    <refsect2>
604
      <title>RENAME</title>
605

    
606
      <cmdsynopsis>
607
        <command>rename</command>
608
        <arg>-f</arg>
609
        <arg choice="req"><replaceable>name</replaceable></arg>
610
      </cmdsynopsis>
611

    
612
      <para>
613
        Renames the cluster and in the process updates the master IP
614
        address to the one the new name resolves to. At least one of
615
        either the name or the IP address must be different, otherwise
616
        the operation will be aborted.
617
      </para>
618

    
619
      <para>
620
        Note that since this command can be dangerous (especially when
621
        run over SSH), the command will require confirmation unless
622
        run with the <option>-f</option> option.
623
      </para>
624
    </refsect2>
625

    
626
    <refsect2>
627
      <title>SEARCH-TAGS</title>
628

    
629
      <cmdsynopsis>
630
        <command>search-tags</command>
631
        <arg choice="req"><replaceable>pattern</replaceable></arg>
632
      </cmdsynopsis>
633

    
634
      <para>
635
        Searches the tags on all objects in the cluster (the cluster
636
        itself, the nodes and the instances) for a given pattern. The
637
        pattern is interpreted as a regular expression and a search
638
        will be done on it (i.e. the given pattern is not anchored to
639
        the beggining of the string; if you want that, prefix the
640
        pattern with <literal>^</literal>).
641
      </para>
642

    
643
      <para>
644
        If no tags are matching the pattern, the exit code of the
645
        command will be one. If there is at least one match, the exit
646
        code will be zero. Each match is listed on one line, the
647
        object and the tag separated by a space. The cluster will be
648
        listed as <filename>/cluster</filename>, a node will be listed
649
        as
650
        <filename>/nodes/<replaceable>name</replaceable></filename>,
651
        and an instance as
652
        <filename>/instances/<replaceable>name</replaceable></filename>.
653
        Example:
654
      </para>
655
<screen>
656
# gnt-cluster search-tags time
657
/cluster ctime:2007-09-01
658
/nodes/node1.example.com mtime:2007-10-04
659
</screen>
660
    </refsect2>
661

    
662
    <refsect2>
663
      <title>VERIFY</title>
664

    
665
      <cmdsynopsis>
666
        <command>verify</command>
667
        <arg choice="opt">--no-nplus1-mem</arg>
668
      </cmdsynopsis>
669

    
670
      <para>
671
        Verify correctness of cluster configuration. This is safe with
672
        respect to running instances, and incurs no downtime of the
673
        instances.
674
      </para>
675

    
676
      <para>
677
        If the <option>--no-nplus1-mem</option> option is given, ganeti won't
678
        check whether if it loses a node it can restart all the instances on
679
        their secondaries (and report an error otherwise).
680
      </para>
681
    </refsect2>
682

    
683
    <refsect2>
684
      <title>VERIFY-DISKS</title>
685

    
686
      <cmdsynopsis>
687
        <command>verify-disks</command>
688
      </cmdsynopsis>
689

    
690
      <para>
691
        The command checks which instances have degraded DRBD disks
692
        and activates the disks of those instances.
693
      </para>
694

    
695
      <para>
696
        This command is run from the <command>ganeti-watcher</command>
697
        tool, which also has a different, complementary algorithm for
698
        doing this check. Together, these two should ensure that DRBD
699
        disks are kept consistent.
700
      </para>
701
    </refsect2>
702

    
703
    <refsect2>
704
      <title>VERSION</title>
705

    
706
      <cmdsynopsis>
707
        <command>version</command>
708
      </cmdsynopsis>
709

    
710
      <para>
711
        Show the cluster version.
712
      </para>
713
    </refsect2>
714

    
715
  </refsect1>
716

    
717
  &footer;
718

    
719
</refentry>
720

    
721
<!-- Keep this comment at the end of the file
722
Local variables:
723
mode: sgml
724
sgml-omittag:t
725
sgml-shorttag:t
726
sgml-minimize-attributes:nil
727
sgml-always-quote-attributes:t
728
sgml-indent-step:2
729
sgml-indent-data:t
730
sgml-parent-document:nil
731
sgml-default-dtd-file:nil
732
sgml-exposed-tags:nil
733
sgml-local-catalogs:nil
734
sgml-local-ecat-files:nil
735
End:
736
-->