Revision 6499c5b8 doc/design-ovf-support.rst

b/doc/design-ovf-support.rst
77 77
  OpenStack
78 78

  
79 79
In our implementation of the OVF we plan to allow a choice between
80
raw, cow and vmdk disk formats for both import and export. The
81
justification is the following:
80
raw, cow and vmdk disk formats for export. We will not limit the import
81
formats in any way, but the used format has to be supported by qemu-img.
82
The justification is the following:
82 83

  
83 84
- Raw format is supported as it is the main format of disk images used
84 85
  in Ganeti, thus it is effortless to provide support for this format
85
- Cow is used in Qemu, [TODO: ..why do we support it, again? That is,
86
  if we do?]
86
- Cow is used in Qemu
87 87
- Vmdk is most commonly supported in virtualization software, it also
88 88
  has the advantage of producing relatively small disk images, which
89 89
  is extremely important advantage when moving instances.
......
119 119
            <gnt:VersionId/>
120 120
            <gnt:AutoBalance/>
121 121
            <gnt:Tags></gnt:Tags>
122
            <gnt:OSParameters></gnt:OSParameters>
122
            <gnt:DiskTemplate</gnt:DiskTemplate>
123
            <gnt:OperatingSystem>
124
                <gnt:Name/>
125
                <gnt:Parameters>
126
                </gnt:Parameters>
127
            </gnt:OperatingSystem>
123 128
            <gnt:Hypervisor>
124
                <gnt:HypervisorParameters>
125
                </gnt:HypervisorParameters>
129
                <gnt:Name/>
130
                <gnt:Parameters>
131
                </gnt:Parameters>
126 132
            </gnt:Hypervisor>
133
            <gnt:Network>
134
            <gnt:Mode/>
135
            <gnt:MACAddress/>
136
            <gnt:Link/>
137
            <gnt:IPAddress/>
138
            </gnt:Network>
127 139
        </gnt:GanetiSection>
128 140
    </Envelope>
129 141

  
......
137 149
where will the data be in OVF format::
138 150

  
139 151
  [instance]
140
      disk0_dump = filename     => References
141
      disk0_ivname = name       => ignored
142
      disk0_size = size_in_mb   => DiskSection
143
      disk_count = number       => ignored
144
      disk_template = disk_type => References
145
      hypervisor = hyp-name     => gnt:HypervisorSection
152
      disk0_dump = filename     => File in References
153
      disk0_ivname = name       => generated automatically
154
      disk0_size = size_in_mb   => calculated after conversion to RAW
155
      disk_count = number       => generated automatically
156
      disk_template = disk_type => gnt:DiskTemplate
157
      hypervisor = hyp-name     => gnt:Name in gnt:Hypervisor
146 158
      name = inst-name          => Name in VirtualSystem
147
      nic0_ip = ip              => Item in VirtualHardwareSection
148
      nic0_link = link          => Item in VirtualHardwareSection
149
      nic0_mac = mac            => Item in VirtualHardwareSection
150
      nic0_mode = mode          => Network in NetworkSection
151
      nic_count = number        => ignored
159
      nic0_ip = ip              => gnt:IPAddress in gnt:Network
160
      nic0_link = link          => gnt:Link in gnt:Network
161
      nic0_mac = mac            => gnt:MACAddress in gnt:Network or
162
                                   Item in VirtualHardwareSection
163
      nic0_mode = mode          => gnt:Mode in gnt:Network
164
      nic_count = number        => generated automatically
152 165
      tags                      => gnt:Tags
153 166

  
154 167
  [backend]
......
157 170
      vcpus = number            => Item in VirtualHardwareSection
158 171

  
159 172
  [export]
160
      compression               => DiskSection
161
      os                        => OperatingSystemSection
173
      compression		=> ignored
174
      os                        => gnt:Name in gnt:OperatingSystem
162 175
      source                    => ignored
163 176
      timestamp                 => ignored
164
      version                   => gnt:VersionId
177
      version                   => gnt:VersionId or
178
                                   constants.EXPORT_VERSION
165 179

  
166
  [os]                          => gnt:OSParameters
180
  [os]                          => gnt:Parameters in gnt:OperatingSystem
167 181

  
168
  [hypervisor]                  => gnt:HypervisorParameters
182
  [hypervisor]                  => gnt:Parameters in gnt:Hypervisor
169 183

  
170 184
In case of multiple networks/disks used by an instance, they will
171 185
all be saved in appropriate sections as specified above for the first
......
178 192
import may be missing. Most often it will happen that such OVF package
179 193
will lack the ``gnt:GanetiSection``.
180 194

  
181
If this happens, the tool will simply ask for all the necessary
182
information or otherwise you can specify all the missing parameters in
183
the command line. For the latter, please refer to [TODO: reference to
184
command line options]
195
If this happens you can specify all the missing parameters in
196
the command line. Please refer to `Command Line`_ section.
197

  
198
In the `user's manual <TODO: link to manual>`_ we provide examples of
199
options when converting from VirtualBox, VMWare and OpenSuseStudio.
185 200

  
186 201
Export to other virtualization software
187 202
---------------------------------------
......
194 209
cause to skip the non-standard information.
195 210

  
196 211
2. Manually remove the gnt:GanetiSection from the ``.ovf`` file. You
197
will also have to recompute sha1 sum (``sha1sum`` command) and update
198
your ``.mf`` file with new value.
212
will also have to recompute sha1 sum (``sha1sum`` command) of the .ovf
213
file and update your ``.mf`` file with new value.
199 214

  
200 215
.. note::
201 216
    Manual change option is only recommended when you have exported your
......
209 224
The limitations regarding import of the OVF instances generated
210 225
outside Ganeti will be (in general) the same, as limitations for
211 226
Ganeti itself.  The desired behavior in case of encountering
212
unsupported element will be to ignore this element's tag and inform
213
the user on console output, if possible - without interruption of the
214
import process.
227
unsupported element will be to ignore this element's tag without
228
interruption of the import process.
215 229

  
216 230
Package
217 231
-------
......
241 255

  
242 256
We plan to support compression both for import and export - in tar.gz
243 257
format. There is also a possibility to provide virtual disk in chunks
244
of equal size.
258
of equal size. The latter will not be implemented in the first version,
259
but we do plan to support it eventually.
260

  
261
The ``ovf:format`` tag is not used in our case. Instead we use
262
``qemu-img info``, which provides enough information for our purposes
263
and is better standardized.
245 264

  
246
When no ``ovf:format`` tag is provided during import, we assume that
247
the disk is to be created on import and proceed accordingly.
265
Please note, that due to security reasons we require the disk image to
266
be in the same directory as the ``.ovf`` description file.
267

  
268
In order to completely ignore disk-related information in resulting
269
config file, please use ``--disk-template=diskless`` option.
248 270

  
249 271
Network
250 272
-------
251 273

  
252
There are no known limitations regarding network support.
274
Ganeti provides support for routed and bridged mode for the networks.
275
Since the standard OVF format does not contain any information regarding
276
used network type, we add our own source of such information in
277
``gnt:GanetiSection``. In case this additional information is not
278
present, we perform a simple check - if network name specified in
279
``NetworkSection`` contains words ``bridged`` or ``routed``, we consider
280
this to be the network type. Otherwise option ``auto`` is chosen, in
281
which case the clusters' default value for that field will be used when
282
importing. This provides a safe fallback in case of NAT networks usage,
283
which are commonly used e.g. in VirtualBox.
253 284

  
254 285
Hardware
255 286
--------
256 287

  
257
TODO
288
The supported hardware is limited to virtual CPUs, RAM memory, disks and
289
networks. In particular, no USB support is currently provided, as Ganeti
290
does not support them.
258 291

  
259 292
Operating Systems
260 293
-----------------
261 294

  
262
TODO
295
Support for different operating systems depends solely on their
296
accessibility for Ganeti instances. List of installed OSes can be
297
checked using ``gnt-os list`` command.
263 298

  
264 299
Other
265 300
-----
266 301

  
302
The instance name (``gnt:VirtualSystem\gnt:Name``) has to be resolvable
303
in order for successful import using ``gnt-backup import``.
304

  
305
_`Command Line`
306
===============
307

  
308
The basic usage of the ovf tool is one of the following::
309

  
310
    ovfconverter import filename
311
    ovfconverter export filename
312

  
313
This will result in a conversion based solely on the content of provided
314
file. In case some information required to make the conversion is
315
missing, an error will occur.
316

  
317
If output directory should be different than the standard Ganeti export
318
directory (usually ``/srv/ganeti/export``), option ``--output-dir``
319
can be used.
320

  
321
If name of resulting entity should be different than the one read from
322
the file, use ``--name`` option.
323

  
324
Import options
325
--------------
326

  
327
Import options that ``ovfconverter`` supports include options for
328
backend, disks, hypervisor, networks and operating system. If an option
329
is given, it overrides the values provided in the OVF file.
330

  
331
Backend
332
^^^^^^^
333
``--backend=option=value`` can be used to set auto balance, number of
334
vcpus and amount of RAM memory.
335

  
336
Please note that when you do not provide full set of options, the
337
omitted ones will be set to cluster defaults (``auto``).
338

  
339
Disks
340
^^^^^
341
``--disk-template=diskless`` causes the converter to ignore all other
342
disk option - both from .ovf file and the command line.
343

  
344
``--disk=number:size=value`` causes to create disks instead of
345
converting them from OVF package; numbers should start with ``0`` and be
346
consecutive.
347

  
348
Hypervisor
349
^^^^^^^^^^
350
``-H hypervisor_name`` and ``-H hypervisor_name:option=value``
351
provide options for hypervisor.
352

  
353
Network
354
^^^^^^^
355
``--no-nics`` option causes converter to ignore any network information
356
provided.
357

  
358
``--network=number:option=value`` sets network information according to
359
provided data, ignoring the OVF package configuration.
360

  
361
Operating System
362
^^^^^^^^^^^^^^^^
363
``--os-type=type`` sets os type accordingly, this option is **required**
364
when importing from OVF instance not created from Ganeti config file.
365

  
366
``--os-parameters`` provides options for chosen operating system.
367

  
368
Tags
369
^^^^
370
``--tags=tag1,tag2,tag3`` is a means of providing tags specific for the
371
instance.
372

  
373
After the conversion is completed, you may use ``gnt-backup import`` to
374
import the instance into Ganeti.
375

  
376
Example::
377

  
378
	ovfconverter file.ovf --disk-template=diskless \
379
                        --os-type=lenny-image \
380
                        --backend=vcpus=1,memory=512,auto_balance \
381
                        -H:xen-pvm \
382
                        --net=0:mode=bridged,link=xen-br0 \
383
                        --name=xen.i1 \
384
                        -v
385
	[output]
386
	gnt-backup import xen.i1
387
	[output]
388
	gnt-instance list
389

  
390
Export options
391
--------------
392
Export options include choice of disk formats to convert the disk image
393
(``--format``; default=``raw`` with no conversion) and compression of
394
the disk into tar.gz format (``--compress``).
395
User has also the choice of allowing to skip the Ganeti-specific part of
396
the OVF document (``--external``).
397

  
398
By default, exported OVF package will not be contained in the OVA
399
package, but this may be changed by adding ``--ova`` option.
400

  
401
[TODO: examples of usage]
402

  
403
Please note that in order to create an OVF package, it is first
404
required that you export your VM using ``gnt-backup export``.
405

  
406
[TODO: complete example of export]
267 407

  
268 408
Implementation details
269 409
======================
270 410

  
271
TODO
411
Disk conversion
412
---------------
413

  
414
Disk conversion for both import and export is done using external tool
415
called qemu-tools. The same tool is used to determine the type of disks.
416

  
417

  
418
Import
419
------
420

  
421
Import functionality is implemented using two classes - OVFReader and
422
OVFImporter.
423

  
424
OVFReader class is used to read the contents of the ``.ovf`` file. Every
425
action that requires ``.ovf`` file access is done through that class.
426
It also performs validation of manifest, if one is present.
427

  
428
The result of reading some part of file is typically a dictionary or a
429
string, containing options which correspond to the ones in
430
``config.ini`` file. Only in case of disks, the resulting value is
431
different - it is then a list of disk names. The reason for that is the
432
need for conversion.
433

  
434
OVFImporter class performs all the command-line-like tasks, such as
435
unpacking OVA package, removing temporary directory, converting disk
436
file to raw format or saving the configuration file on disk.
437
It also contains a set of functions that read the options provided in
438
the command line.
439

  
440

  
441
Typical workflow for the import is very simple:
442

  
443
- read the ``.ovf`` file info memory
444
- verify manifest
445
- parse each element of the configuration file: name, disk template,
446
  hypervisor, operating system, backend parameters, network and disks
447

  
448
    - check if option for the element can be read from command line
449
      options
450

  
451
		- if yes: parse options from command line
452

  
453
		- otherwise: read the appropriate portion of ``.ovf`` file
454

  
455
- save gathered information in ``config.ini`` file
456

  
457
ToDo
458
====
459

  
460
This lists functionalities for import that are not yet implemented, but
461
should be before the initial release:
462

  
463
- Support for compressed disks
464

  
272 465

  
273 466
.. vim: set textwidth=72 :
274 467
.. Local Variables:

Also available in: Unified diff