Bump version to 0.2.7

[snf-image-creator] / docs / usage.rst

diff --git a/docs/usage.rst b/docs/usage.rst
index 6fee241..c43fb29 100644 (file)
--- a/docs/usage.rst
+++ b/docs/usage.rst
@@ -2,9 +2,14 @@ Usage
 ^^^^^
 
 snf-image-creator comes in 2 variants:
 ^^^^^
 
 snf-image-creator comes in 2 variants:

+

  * snf-image-creator: A non-interactive command line program
  * snf-mkimage: A user-friendly dialog-based program
 
  * snf-image-creator: A non-interactive command line program
  * snf-mkimage: A user-friendly dialog-based program
 

+Both expect the input media as first argument. The input media may be a local
+file, a block device or *"/"* if you want to create an image out of the running
+system (see `host bundling operation`_).
+

 Non-interactive version
 =======================
 
 Non-interactive version
 =======================
 

@@ -13,58 +18,58 @@ snf-image-creator receives the following options:
 .. code-block:: console
 
  $ snf-image-creator --help
 .. code-block:: console
 
  $ snf-image-creator --help

-

  Usage: snf-image-creator [options] <input_media>
 
  Options:
  Usage: snf-image-creator [options] <input_media>
 
  Options:

-  --version             show program's version number and exit
-  -h, --help            show this help message and exit
-  -o FILE, --outfile=FILE
-                        dump image to FILE
-  -f, --force           overwrite output files if they exist
-  -s, --silent          silent mode, only output errors
-  -u FILENAME, --upload=FILENAME
-                        upload the image to pithos with name FILENAME
-  -r IMAGENAME, --register=IMAGENAME
-                        register the image with ~okeanos as IMAGENAME
-  -a ACCOUNT, --account=ACCOUNT
-                        Use this ACCOUNT when uploading/registering images
-                        [Default: None]
-  -m KEY=VALUE, --metadata=KEY=VALUE
-                        Add custom KEY=VALUE metadata to the image
-  -t TOKEN, --token=TOKEN
-                        Use this token when uploading/registering images
-                        [Default: None]
-  --print-sysprep       print the enabled and disabled system preparation
-                        operations for this input media
-  --enable-sysprep=SYSPREP
-                        run SYSPREP operation on the input media
-  --disable-sysprep=SYSPREP
-                        prevent SYSPREP operation from running on the input
-                        media
-  --no-sysprep          don't perform system preparation
-  --no-shrink           don't shrink any partition
-
-
-Most input options are self-describing. If you want to save a local copy for
-the image you create, you specify *-o* option. To upload the image to
-*pithos+*, you specify valid credentials with *-a* and *-t* options and a
-filename using *-u*. If you want to register the image with *~okeanos*,
-in addition to *-u* specify a registration name using *-r*.
-
-By default snf-image-creator will perform a number of system preparation
-operations on the snapshot of the media and will shrink the last partition
-found, before extracting the image. Both can be disabled by specifying
+   --version             show program's version number and exit
+   -h, --help            show this help message and exit
+   -o FILE, --outfile=FILE
+                         dump image to FILE
+   -f, --force           overwrite output files if they exist
+   -s, --silent          output only errors
+   -u FILENAME, --upload=FILENAME
+                         upload the image to pithos with name FILENAME
+   -r IMAGENAME, --register=IMAGENAME
+                         register the image with ~okeanos as IMAGENAME
+   -m KEY=VALUE, --metadata=KEY=VALUE
+                         add custom KEY=VALUE metadata to the image
+   -t TOKEN, --token=TOKEN
+                         use this token when uploading/registering images
+                         [Default: None]
+   --print-sysprep       print the available enabled and disabled system
+                         preparation operations for this input media
+   --enable-sysprep=SYSPREP
+                         run SYSPREP operation on the input media
+   --disable-sysprep=SYSPREP
+                         prevent SYSPREP operation from running on the input
+                         media
+   --no-sysprep          don't perform any system preparation operation
+   --no-shrink           don't shrink the image
+   --public              register image with cyclades as public
+   --tmpdir=DIR          create large temporary image files under DIR
+
+Most input options are self-describing. If you want to save a local copy of
+the image you create, provide a filename using the *-o* option. To upload the
+image to *pithos+*, provide a valid authentication token using *-t* and a
+filename using *-u*. If you also want to register the image with *~okeanos*, in
+addition to *-u* provide a registration name using *-r*. All images are
+registered as *private*. Only the user that registers the image can create
+VM's out of it. If you want the image to be visible by other user too, use the
+*--public* option.
+
+By default, before extracting the image, snf-image-creator will perform a
+number of system preparation operations on the snapshot of the media and will
+shrink the last partition found. Both actions can be disabled by specifying

 *--no-sysprep* and *--no-shrink* respectively.
 
 *--no-sysprep* and *--no-shrink* respectively.
 

-If *--print-sysprep* is defined, the program will exit after outputing a
+If *--print-sysprep* is defined, the program will exit after printing a

 list of enabled and disabled system preparation operation applicable to this
 list of enabled and disabled system preparation operation applicable to this

-media source. The user can enable or disable specific *syspreps* when creating
-an image, using *-{enable,disable}-sysprep* options. You can specify those
-options multiple times to enable or disable multiple *syspreps*.
+input media. The user can enable or disable specific *syspreps*, using
+*-{enable,disable}-sysprep* options. The user may specify those options
+multiple times.

 
 Running *snf-image-creator* with *--print-sysprep* on a raw file that hosts a
 
 Running *snf-image-creator* with *--print-sysprep* on a raw file that hosts a

-debian system, we get the following output:
+debian system, we print the following output:

 
 .. _sysprep:
 
 
 .. _sysprep:
 

@@ -126,8 +131,8 @@ debian system, we get the following output:
 
    cleaning up...
 
 
    cleaning up...
 

-If we want the image to have all normal user accounts and all mail files
-removed, we can create it specifying *--enable-sysprep* option like this:
+If you want the image to have all normal user accounts and all mail files
+removed, you should use *--enable-sysprep* option like this:

 
 .. code-block:: console
 
 
 .. code-block:: console
 

@@ -140,18 +145,27 @@ Dialog-based version
 
 .. code-block:: console
 
 
 .. code-block:: console
 

-   $ Usage: snf-mkimage [options] [<input_media>]
+ $ snf-mkimage --help
+ Usage: snf-mkimage [options] [<input_media>]

 
 

-   Options:
-     --version             show program's version number and exit
-     -h, --help            show this help message and exit
-     -l FILE, --logfile=FILE
-                            log all messages to FILE
+ Options:
+   --version             show program's version number and exit
+   -h, --help            show this help message and exit
+   -l FILE, --logfile=FILE
+                         log all messages to FILE
+   --tmpdir=DIR          create large temporary image files under DIR

 
 If the input media is not specified in the command line, in the first dialog
 
 If the input media is not specified in the command line, in the first dialog

-box the user will be asked to specify it. After the input media is examined and
-the program is initialized, the user will be given the choice to run
-*snf-mkimage* in *wizard* or *expert* mode.
+box the user will be asked to specify it:
+
+.. image:: /snapshots/select_media.png
+
+The user can select a file (regular or block device) or use the *Bundle Host*
+button to create an image out of the running system (see
+`Host bundling operation`_).
+
+After the input media is examined and the program is initialized, the user will
+be given the choice to run *snf-mkimage* in *wizard* or *expert* mode.

 
 Wizard mode
 -----------
 
 Wizard mode
 -----------

@@ -160,20 +174,22 @@ When *snf-mkimage* runs in *wizard* mode, the user is just asked to provide the
 following basic information:
 
  * Name: A short name for the image (ex. "Slackware")
 following basic information:
 
  * Name: A short name for the image (ex. "Slackware")

- * Description: An one-line description for the image (ex. "Slackware Linux 14.0 with KDE")
- * Account: An *~okeanos* account email
- * Token: A token corresponding to the account defined previously
+ * Description: An one-line description for the image
+   (ex. "Slackware Linux 14.0 with KDE")
+ * Registration Type: Private or Public
+ * Account: The authentication token for an *~okeanos* account

 
 After confirming, the image will be extracted, uploaded to *pithos+* and
 
 After confirming, the image will be extracted, uploaded to *pithos+* and

-registered to *~okeanos*. The user will also be given the choice to keep a local
-copy of it. For most users the functionality this mode provides should be
-sufficient.
+registered with *~okeanos*. The user will also be given the choice to keep a
+local copy of it.
+
+For most users the functionality this mode provides should be sufficient.

 
 Expert mode
 -----------
 
 Expert mode allows the user to have better control on the image creation
 
 Expert mode
 -----------
 
 Expert mode allows the user to have better control on the image creation

-process. In the picture below the main menu can be seen:
+process. The main menu can be seen in the picture below:

 
 .. image:: /snapshots/main_menu.png
 
 
 .. image:: /snapshots/main_menu.png
 

@@ -186,14 +202,27 @@ In the *Customize* sub-menu the user can control:
 
 In the *Register* sub-menu the user can provide:
 
 
 In the *Register* sub-menu the user can provide:
 

- * The credentials to login to *~okeanos*
- * A pithos filename for the uploaded *diskdump* image
- * A name for the image to be registered to *~okeanos* with
+ * The credentials (authentication token) to use when authenticating
+   to *~okeanos*
+ * A *pithos+* filename for the uploaded *diskdump* image
+ * A name for the image to use when registering it with *~okeanos*, as well as
+   the registration type (*private* or *public*)

 
 

-By choosing the *Extract* menu entry the user can dump the image to the local
-file system and finally, if the user selects *Reset*, the system will ignore
+By choosing the *Extract* menu entry, the user can dump the image to the local
+file system. Finally, if the user selects *Reset*, the system will ignore

 all changes made so far and will start the image creation process again.
 
 all changes made so far and will start the image creation process again.
 

+Host bundling operation
+=======================
+
+As a new feature in *v0.2*, snf-image-creator can create images out of the host
+system that runs the program. This is done either by specifying / as input
+media or by using the *Bundle Host* button in the media selection dialog of
+snf-mkimage. During this operation, the files of the disk are copied into a
+temporary image file, which means that the file system that will host the
+temporary image needs to have a lot of free space (see `large temporary files`_
+for more information).
+

 Creating a new image
 ====================
 
 Creating a new image
 ====================
 

@@ -202,16 +231,16 @@ disk from the Internet:
 
 .. code-block:: console
 
 
 .. code-block:: console
 

-   $ wget http://ubuntureleases.tsl.gr/12.04.1/ubuntu-12.04.1-server-amd64.iso
+   $ wget http://ubuntureleases.tsl.gr/12.04.2/ubuntu-12.04.2-server-amd64.iso

 
 Verify that it has been downloaded correctly:
 
 .. code-block:: console
 
 
 Verify that it has been downloaded correctly:
 
 .. code-block:: console
 

-   $ echo 'a8c667e871f48f3a662f3fbf1c3ddb17  ubuntu-12.04.1-server-amd64.iso' > check.md5
+   $ echo 'a8c667e871f48f3a662f3fbf1c3ddb17  ubuntu-12.04.2-server-amd64.iso' > check.md5

    $ md5sum -c check.md5
 
    $ md5sum -c check.md5
 

-Create a 2G sparce file to host the new system:
+Create a 2G sparse file to host the new system:

 
 .. code-block:: console
 
 
 .. code-block:: console
 

@@ -222,45 +251,55 @@ And install the Ubuntu system on this file:
 .. code-block:: console
 
    $ sudo kvm -boot d -drive file=ubuntu_hd.raw,format=raw,cache=none,if=virtio \
 .. code-block:: console
 
    $ sudo kvm -boot d -drive file=ubuntu_hd.raw,format=raw,cache=none,if=virtio \

-     -m 1000 -cdrom ubuntu-12.04.1-server-amd64.iso
+     -m 1G -cdrom ubuntu-12.04.2-server-amd64.iso

 
 

-.. note::
+.. warning::

 
 

-   During the installation, you will be asked about the partition scheme. Since
-   snf-image-creator does not support LVM partitions, you are advised to create
-   regular partitions.
+   During the installation, you will be asked about the partition scheme. Don't 
+   use LVM partitions. They are not supported by snf-image-creator.

 
 

-When the installation is complete, you can close the QEMU window. You
-will be able to boot your installed OS and make any changes you want to it
+You will be able to boot your installed OS and make any changes you want

 (e.g. install openssh-server) using the following command::
 
 (e.g. install openssh-server) using the following command::
 

-   $ sudo kvm -m 1000 -drive file=linuxmint.raw,format=raw,cache=none,if=virtio
+   $ sudo kvm -m 1G -boot c -drive file=ubuntu_hd.raw,format=raw,cache=none,if=virtio

 
 

-After you're done, become root, activate the virtual environment you have
-installed snf-image-creator in, and use *snf-mkimage* to create and upload the
+After you're done, you may use *snf-mkimage* as root to create and upload the

 image:
 
 .. code-block:: console
 
    $ sudo -s
 image:
 
 .. code-block:: console
 
    $ sudo -s

-   $ source /path/to/snf-image-env/bin/activate

    $ snf-mkimage ubuntu_hd.raw
 
 In the first screen you will be asked to choose if you want to run the program
 in *Wizard* or *Expert* mode. Choose *Wizard*.
 
    $ snf-mkimage ubuntu_hd.raw
 
 In the first screen you will be asked to choose if you want to run the program
 in *Wizard* or *Expert* mode. Choose *Wizard*.
 

-.. image:: /snapshots/01_wizard.png
+.. image:: /snapshots/wizard.png

 
 

-Then you will be asked to provide a name, a description, an *~okeanos* account
-and the token corresponding to this account. After that you will be asked to
-confirm the provided data.
+Then you will be asked to provide a name, a description, a registration type
+(*private* or *public*) and the authentication token corresponding to your
+*~okeanos* account. Finally, you'll be asked to confirm the provided data.

 
 

-.. image:: /snapshots/06_confirm.png
+.. image:: /snapshots/confirm.png

 
 

-Choosing *YES* will create the image and upload it to your *~okeanos* account.
+Choosing *YES* will create and upload the image to your *~okeanos* account.

 
 

-Some caveats on image creation
-==============================
+Limitations
+===========
+
+Supported operating systems
+---------------------------
+
+*snf-image-creator* can only fully function on input media hosting *Linux*
+systems. The program will detect the needed metadata and you may use it to
+upload and register other *Unix* or *Windows* images, but you cannot use it to
+shrink them or perform system preparation operations.
+
+Logical Volumes
+---------------
+
+The program cannot work on LVM partitions [#f1]_. The input media may only
+contain primary or logical partitions.

 
 Para-virtualized drivers
 ------------------------
 
 Para-virtualized drivers
 ------------------------

@@ -271,28 +310,58 @@ Ethernet cards on the VM instances are para-virtualized and need special
 version 2.6.25 and are shipped with all the popular Linux distributions. The
 problem is that if the driver for the para-virtualized disk I/O controller is
 built as module, it needs to be preloaded using an initial ramdisk, otherwise
 version 2.6.25 and are shipped with all the popular Linux distributions. The
 problem is that if the driver for the para-virtualized disk I/O controller is
 built as module, it needs to be preloaded using an initial ramdisk, otherwise

-the VM will not be able to boot.
-
-In the image creation demonstration above, we initially installed the Ubuntu
-system on a hard disk (*ubuntu_hd.raw*) that was connected on a
-para-virtualized interface (pay attention to the *if=virtio* option of the kvm
-line). Ubuntu and Debian create a generic initial ramdisk file that contains
-many different modules, including the VirtIO drivers. In many distros this is
-not the case. In Arch Linux for example, the user needs to manually add
-*virtio_blk* and *virtio_pci* drivers in */etc/mkinitcpio.conf* and rebuild the
-initial ramdisk [#f1]_ to make the virtio drivers get preloaded during boot.
-For now, *snf-image-creator* cannot resolve this kind of problems and it's left
-to the user to do it.
-
-Swap partitions
----------------
+the VM won't be able to boot.
+
+Many popular Linux distributions, like Ubuntu and Debian, will automatically
+create a generic initial ramdisk file that contains many different modules,
+including the VirtIO drivers. Others that target more experienced users, like
+Slackware, won't do that [#f2]_. *snf-image-creator* cannot resolve this kind
+of problems and it's left to the user to do so. Please refer to your
+distribution's documentation for more information on this. You can always check
+if a system can boot with para-virtualized disk controller by launching it with
+kvm using the *if=virtio* option (see the kvm command in the
+`Creating a new image`_ section).
+
+Some caveats on image creation
+==============================
+
+Image partition schemes and shrinking
+-------------------------------------
+
+When image shrinking is enabled, *snf-image-creator* will shrink the last
+partition on the disk. If this is a swap partition, it will remove it, save
+enough information to recreate it during image deployment and shrink the
+partition that lays just before that. This will make the image smaller which
+speeds up the deployment process.
+
+During image deployment, the last partition is enlarged to occupy the available
+space in the VM's hard disk and a swap partition is added at the end if a SWAP
+image property is present.
+
+Keep this in mind when creating images. It's always better to have your swap
+partition placed as the last partition on the disk and have your largest
+partition (*/* or */home*) just before that.
+
+Large temporary files
+---------------------
+
+*snf-image-creator* may create large temporary files when running:
+
+ * During image shrinking, the input media snapshot file may reach the size of
+   the original media.
+ * When bundling the host system, the temporary image file may became as large
+   as the rest of the disk files altogether.

 
 

-If you want your image to have a swap partition, make sure this is the last
-partition on the disk. If snf-image-creator detects a swap partition in the end
-of the input media, it will remove the partition when shrinking and will save
-enough information to be able to recreate it during image deployment. This will
-make the image smaller and will speed up the deployment process.
+*/tmp* directory is not a good place for hosting large files. In many systems
+the contents of */tmp* are stored in volatile memory and the size they may occupy
+is limited. By default, *snf-image-creator* will use a heuristic approach to
+determine where to store large temporary files. It will examine the free space
+under */var/tmp*, the user's home directory and */mnt* and will pick the one
+with the most available space. The user may overwrite this behaviour and
+indicate a different directory using the *tmpdir* option. This option is
+supported by both *snf-image-creator* and *snf-mkimage*.

 
 .. rubric:: Footnotes
 
 
 .. rubric:: Footnotes
 

-.. [#f1] https://wiki.archlinux.org/index.php/KVM#Paravirtualized_guests_.28virtio.29
+.. [#f1] http://sourceware.org/lvm2/
+.. [#f2] http://mirrors.slackware.com/slackware/slackware-14.0/README.initrd