^^^^^
snf-image-creator comes in 2 variants:
+
* snf-image-creator: A non-interactive command line program
* snf-mkimage: A user-friendly dialog-based program
+Both expect the input media as first argument. The input media may be a local
+file, a block device or *"/"* if you want to create an image out of the running
+system (see `host bundling operation`_).
+
Non-interactive version
=======================
.. code-block:: console
$ snf-image-creator --help
-
Usage: snf-image-creator [options] <input_media>
Options:
- --version show program's version number and exit
- -h, --help show this help message and exit
- -o FILE, --outfile=FILE
- dump image to FILE
- -f, --force overwrite output files if they exist
- -s, --silent silent mode, only output errors
- -u FILENAME, --upload=FILENAME
- upload the image to pithos with name FILENAME
- -r IMAGENAME, --register=IMAGENAME
- register the image with ~okeanos as IMAGENAME
- -a ACCOUNT, --account=ACCOUNT
- Use this ACCOUNT when uploading/registering images
- [Default: None]
- -m KEY=VALUE, --metadata=KEY=VALUE
- Add custom KEY=VALUE metadata to the image
- -t TOKEN, --token=TOKEN
- Use this token when uploading/registering images
- [Default: None]
- --print-sysprep print the enabled and disabled system preparation
- operations for this input media
- --enable-sysprep=SYSPREP
- run SYSPREP operation on the input media
- --disable-sysprep=SYSPREP
- prevent SYSPREP operation from running on the input
- media
- --no-sysprep don't perform system preparation
- --no-shrink don't shrink any partition
-
-
-Most input options are self-describing. If you want to save a local copy for
-the image you create, you specify *-o* option. To upload the image to
-*pithos+*, you specify valid credentials with *-a* and *-t* options and a
-filename using *-u*. If you want to register the image with *~okeanos*,
-in addition to *-u* specify a registration name using *-r*.
-
-By default snf-image-creator will perform a number of system preparation
-operations on the snapshot of the media and will shrink the last partition
-found, before extracting the image. Both can be disabled by specifying
+ --version show program's version number and exit
+ -h, --help show this help message and exit
+ -o FILE, --outfile=FILE
+ dump image to FILE
+ -f, --force overwrite output files if they exist
+ -s, --silent output only errors
+ -u FILENAME, --upload=FILENAME
+ upload the image to pithos with name FILENAME
+ -r IMAGENAME, --register=IMAGENAME
+ register the image with ~okeanos as IMAGENAME
+ -m KEY=VALUE, --metadata=KEY=VALUE
+ add custom KEY=VALUE metadata to the image
+ -t TOKEN, --token=TOKEN
+ use this token when uploading/registering images
+ [Default: None]
+ --print-sysprep print the available enabled and disabled system
+ preparation operations for this input media
+ --enable-sysprep=SYSPREP
+ run SYSPREP operation on the input media
+ --disable-sysprep=SYSPREP
+ prevent SYSPREP operation from running on the input
+ media
+ --no-sysprep don't perform any system preparation operation
+ --no-shrink don't shrink the image
+ --public register image with cyclades as public
+ --tmpdir=DIR create large temporary image files under DIR
+
+Most input options are self-describing. If you want to save a local copy of
+the image you create, provide a filename using the *-o* option. To upload the
+image to *pithos+*, provide a valid authentication token using *-t* and a
+filename using *-u*. If you also want to register the image with *~okeanos*, in
+addition to *-u* provide a registration name using *-r*. All images are
+registered as *private*. Only the user that registers the image can create
+VM's out of it. If you want the image to be visible by other user too, use the
+*--public* option.
+
+By default, before extracting the image, snf-image-creator will perform a
+number of system preparation operations on the snapshot of the media and will
+shrink the last partition found. Both actions can be disabled by specifying
*--no-sysprep* and *--no-shrink* respectively.
-If *--print-sysprep* is defined, the program will exit after outputing a
-list of enabled and disabled system preparation operation appliable to this
-media source. The user can enable or disable specific *syspreps* when creating
-an image, using *-{enable,disable}-sysprep* options. You can specify those
-options multiple times to enable or disable multiple *syspreps*.
+If *--print-sysprep* is defined, the program will exit after printing a
+list of enabled and disabled system preparation operation applicable to this
+input media. The user can enable or disable specific *syspreps*, using
+*-{enable,disable}-sysprep* options. The user may specify those options
+multiple times.
Running *snf-image-creator* with *--print-sysprep* on a raw file that hosts a
-debian system, we get the following output:
+debian system, we print the following output:
.. _sysprep:
cleaning up...
-If we want the image to have all normal user accounts and all mail files
-removed, we can create it specifying *--enable-sysprep* option like this:
+If you want the image to have all normal user accounts and all mail files
+removed, you should use *--enable-sysprep* option like this:
.. code-block:: console
.. code-block:: console
- $ Usage: snf-mkimage [options] [<input_media>]
+ $ snf-mkimage --help
+ Usage: snf-mkimage [options] [<input_media>]
- Options:
- --version show program's version number and exit
- -h, --help show this help message and exit
- -l FILE, --logfile=FILE
- log all messages to FILE
+ Options:
+ --version show program's version number and exit
+ -h, --help show this help message and exit
+ -l FILE, --logfile=FILE
+ log all messages to FILE
+ --tmpdir=DIR create large temporary image files under DIR
If the input media is not specified in the command line, in the first dialog
-box the user will be asked to specify it. After the input media is examined and
-the program is initialized, the user will be given the choice to run
-*snf-mkimage* in *wizard* or *expert* mode.
+box the user will be asked to specify it:
+
+.. image:: /snapshots/select_media.png
+
+The user can select a file (regular or block device) or use the *Bundle Host*
+button to create an image out of the running system (see
+`Host bundling operation`_).
+
+After the input media is examined and the program is initialized, the user will
+be given the choice to run *snf-mkimage* in *wizard* or *expert* mode.
Wizard mode
-----------
following basic information:
* Name: A short name for the image (ex. "Slackware")
- * Description: An one-line description for the image (ex. "Slackware Linux 14.0 with KDE")
- * Account: An *~okeanos* account email
- * Token: A token corresponding to the account defined previously
+ * Description: An one-line description for the image
+ (ex. "Slackware Linux 14.0 with KDE")
+ * Registration Type: Private or Public
+ * Account: The authentication token for an *~okeanos* account
After confirming, the image will be extracted, uploaded to *pithos+* and
-registered to *~okeanos*. The user will also be given the choice to keep a local
-copy of it. For most users the functionality this mode provides should be
-sufficient.
+registered with *~okeanos*. The user will also be given the choice to keep a
+local copy of it.
+
+For most users the functionality this mode provides should be sufficient.
Expert mode
-----------
Expert mode allows the user to have better control on the image creation
-process. In the picture below the main menu can be seen:
+process. The main menu can be seen in the picture below:
.. image:: /snapshots/main_menu.png
In the *Register* sub-menu the user can provide:
- * The credentials to login to *~okeanos*
- * A pithos filename for the uploaded *diskdump* image
- * A name for the image to be registered to *~okeanos* with
+ * The credentials (authentication token) to use when authenticating
+ to *~okeanos*
+ * A *pithos+* filename for the uploaded *diskdump* image
+ * A name for the image to use when registering it with *~okeanos*, as well as
+ the registration type (*private* or *public*)
-By choosing the *Extract* menu entry the user can dump the image to the local
-file system and finally, if the user selects *Reset*, the system will ignore
+By choosing the *Extract* menu entry, the user can dump the image to the local
+file system. Finally, if the user selects *Reset*, the system will ignore
all changes made so far and will start the image creation process again.
-Usage example
-=============
-
-Supposing you have snf-image-creator installed on a machine (hereafter referred
-to as your **host machine**), you can follow the next steps to upload and
-register an image of an OS of your preference (hereafter referred to as your
-**guest OS**) to your synnefo deployment.
-
- * `Step 1: Install the guest OS`_
- * `Step 2: Create and upload an image of the guest OS`_
- * `Step 3: Create your VM`_
-
-
-Step 1: Install the guest OS
------------------------------
-
-The guest OS has to be installed on a media such as a block device or a regular
-raw file, that can be **accessible** by your host machine.
-
-But why is accessible empasized? Well, because you don't need to do the
-installation of the guest OS on your host machine. You can just as well install
-it on a raw file, upload it on Pithos+, download it on your host machine and
-use it for Step 2.
-
-*Note: If you have a guest OS already installed, you may want to skip the
-next step. However, be sure to check out the* `Caveats`_ *section, where
-some requirements about the guest OS are presented.*
-
-Installation example
-""""""""""""""""""""
+Host bundling operation
+=======================
-To simplify things a bit, we will install the guest OS on the host machine
-where snf-image-creator is installed. We will assume that the host machine is
-an Ubuntu 12.04 ~okeanos VM, built with max specifications (4 CPUs, 2GB of
-ram, 40GB of disk space at the time of writing this).
+As a new feature in *v0.2*, snf-image-creator can create images out of the host
+system that runs the program. This is done either by specifying / as input
+media or by using the *Bundle Host* button in the media selection dialog of
+snf-mkimage. During this operation, the files of the disk are copied into a
+temporary image file, which means that the file system that will host the
+temporary image needs to have a lot of free space (see `large temporary files`_
+for more information).
-*Note: Since the installation of the guest OS will take place on your host
-VM, you must be able to connect to it graphically. This is covered by our*
-`connection guide <https://okeanos.grnet.gr/support/user-guide/cyclades-how-do-
-i-connect-to-a-vm/#windows-linux-host-to-linux-guest-graphical>`_.
+Creating a new image
+====================
-For our guest OS, we will choose, Linux Mint, which is the most hyped Linux
-OS according to Distrowatch. A new version has just been released, so
-this seems like a fine choice. ::
+Suppose you want to create a new Ubuntu server image. Download the installation
+disk from the Internet:
- Warning: The installation might take a long time (~1 hour) and a bit of
- lagginess due to nested virtualization is to be expected.
+.. code-block:: console
-Fire up your terminal, go to your home directory and type the following to
-download the Linux Mint live cd::
+ $ wget http://ubuntureleases.tsl.gr/12.04.2/ubuntu-12.04.2-server-amd64.iso
- $ wget http://ftp.ntua.gr/pub/linux/linuxmint//stable/14/linuxmint-14-mate-dvd-64 bit.iso
+Verify that it has been downloaded correctly:
-Verify that it has been downloaded correctly. If the following command
-prints "OK". then you are good to go::
+.. code-block:: console
- $ echo 'b370ac59d1ac6362f1662cfcc22a489c linuxmint-14-mate-dvd-64bit.iso' > check.md5
+ $ echo 'a8c667e871f48f3a662f3fbf1c3ddb17 ubuntu-12.04.2-server-amd64.iso' > check.md5
$ md5sum -c check.md5
-Allocate a few gigs of space to create a sparse file::
-
- $ truncate -s 7G linuxmint.raw
-
-Use `kvm` to boot the Live CD::
-
- $ sudo kvm -m 1200 -smp 4 -boot d -drive \
- file=linuxmint.raw,format=raw,cache=none,if=virtio \
- -cdrom linuxmint-14-mate-dvd-64bit.iso
-
- At a glance, let's see what the above options do:
- -m 1200: Use 1200MB of RAM for the guest OS. You should
- allocate as much as possible
- -smp 4: Simulate an SMP system with 4 CPUs for the
- guest OS to use.
- -boot d: Place cdrom first in the boot order
- file=opensuse.raw Use this raw file as the "hard disk" for the
- installation
- if=virtio: Inform the OS that it should preload the
- VirtIO drivers (more on that on `Caveats`_
- section)
- -cdrom linuxmint-14-mate-dvd-64bit.iso:
- "Insert" Linux Mint's live cd in the cdrom
- drive
-
-Wait a few seconds and then a new screen with the Linux Mint logo should
-appear. You don't have to press any key since it will boot automatically to
-Linux Mint's live desktop after a few minutes.
-
-|qemu-live|
-
-Choose "Install Linux Mint". The installation process should be pretty
-straightforward. Just keep two things in mind:
-
-* The username you choose will also be used later on, when you create a VM
- from this OS image. The password, however, will be removed and you will
- be given a new one.
-* The installed OS must have no more than one primary partition and
- optionally a swap partition. You can read more in the `Caveats`_
- section below. You don't have to worry about it in this installation
- however, since the default option takes care of that.
-
- |qemu-partition|
-
-After the installation is complete, you can close the QEMU window. You
-will be able to boot your installed OS and make any changes you want to it
-using the following command::
+Create a 2G sparse file to host the new system:
- $ sudo kvm -m 1200 -smp 4 -boot d -drive \
- file=linuxmint.raw,format=raw,cache=none,if=virtio
-
-At the very least, you should install OpenSSH server to connect to your VM
-properly. You can install OpenSSH server using the following command::
-
- $ sudo apt-get install openssh-server
-
-Bear in mind that once the OS image has been uploaded to your synnefo
-deployment, you will not be able to make changes to it. Since you can only
-apply changes to your raw file, you are advised to do so now and then proceed
-to Step 2.
-
-Caveats
-"""""""
-This is a list of restrictions you must have in mind while installing the
-guest OS:
-
-**Partitions**
-
-The installation must consist of no more than one primary partition. It
-can have a swap partition though, which should ideally - but not
-necessarily - be located at the end of the media. In this case, the
-uploaded image will be much smaller and the VM deployment process much
-faster.
-
-**VirtIO drivers**
-
-Depending on your synnefo-deployment, you may need to use para-virtualized
-drivers for your storage and network needs.
-
-~okeanos uses the VirtIO framework which is essential for the ~okeanos VMs
-to work properly since their disk I/O controller and Ethernet cards are
-para-virtualized.
-
-Fortunately, you will not need to deal with the installation of VirtIO drivers
-directly, since they are included in Linux kernel since version 2.6.25 and
-shipped with all the modern Linux distributions. However, if the VirtIO drivers
-are built as a module (and most modern OSes do so), they need to be preloaded
-using an initial ramdisk (initramfs), otherwise the VM you create from this OS
-image will not be able to boot.
-
-Debian derivatives will create an initial ramdisk with VirtIO included if
-they are connected during the installation on a para-virtualized interface
-(the "if=virtio" option in the Linux Mint example).
-
-In many other distros though, this is not the case. In Arch Linux for
-example, the user needs to manually add virtio_blk and virtio_pci drivers
-in /etc/mkinitcpio.conf and rebuild the initial ramdisk to make the
-virtio drivers get preloaded during boot. You can read more in the `Arch
-Linux wiki <https://wiki.archlinux.org/index.php/KVM#Paravirtualized_
-guests_.28virtio.29>`_ on how to do it.
-
-For now, snf-image-creator cannot resolve this kind of problems and it's
-left to the user to do it.
-
-Step 2: Create and upload an image of the guest OS
---------------------------------------------------
-
-This is the step on which we use snf-image-creator. There are actually two
-variants of snf-image-creator, `snf-image-creator`_ and `snf-mkimage`_, both
-achieving the same results but suited for different needs. Their scope is
-documented at the start of the `Usage`_ section of this document.
-
-*Note: Both tools take a snapshot of the installed OS on the media
-provided to them. So, any changes they apply do not affect the OS
-installed on the original media.*
-
-Let's see both tools in action. We will use them to create an image of the
-Linux Mint 14 OS we installed in Step 2.
-
-snf-mkimage
-"""""""""""
-
-In order to use snf-mkimage, simply type::
-
- $ sudo snf-mkimage linuxmint.raw
-
-snf-mkimage will initially check if the media is indeed a single disk
-partition or a raw file representing a hard disk. Then, it will use
-heuristics to understand which OS has been installed on the media. After
-that, you will be asked which mode you prefer.
-
-|mkimage-wizard|
-
-* Wizard mode is intuitive and consists of 4 simple steps.
-* Expert mode has an abundance of options but requires a bit of knowledge
- of the inner workings of Cyclades from your part. You can learn more on the
- `Expert Mode`_ section of snf-mkimage.
-
-For our tutorial, we will use Wizard mode. So, choose "Wizard" and then provide
-a name for the image.
-
-|mkimage1|
-
-This name will appear on Pithos+ and on the Public Images section of Cyclades.
+.. code-block:: console
-Then, provide a description for the image.
+ $ truncate -s 2G ubuntu_hd.raw
-|mkimage2|
+And install the Ubuntu system on this file:
-This will appear under the chosen image name on the Public Images section of
-cyclades.
+.. code-block:: console
-Next, add your account e-mail
+ $ sudo kvm -boot d -drive file=ubuntu_hd.raw,format=raw,cache=none,if=virtio \
+ -m 1G -cdrom ubuntu-12.04.2-server-amd64.iso
-|mkimage3|
+.. warning::
-... your account token...
+ During the installation, you will be asked about the partition scheme. Don't
+ use LVM partitions. They are not supported by snf-image-creator.
-|mkimage4|
+You will be able to boot your installed OS and make any changes you want
+(e.g. install openssh-server) using the following command::
-...and you're done! A list operations will appear on your console.
+ $ sudo kvm -m 1G -boot c -drive file=ubuntu_hd.raw,format=raw,cache=none,if=virtio
-|mkimage-results|
+After you're done, you may use *snf-mkimage* as root to create and upload the
+image:
-We will briefly comment on the above output.
+.. code-block:: console
-* **Sysprep:** Operations from 1/9 to 9/9 are part of the system
- preparation operations and are best explained in the snf-image-creator's
- `sysprep`_ section.
-* **Shrinking:** When shrinking the image, we check if a swap partition
- exists at the end of the media. If this is the case, it will be removed
- and re-inserted upon the deployment process of the VM. Alternatively, if
- the swap partition lies at the start of the media, it will be left
- untouched. On both cases, the primary partition will be shrunken as much
- as possible. On this example, we can see that the final size is 3.5GB,
- whereas the orginal size was 7GB. This means that the image was reduced
- by half, a pretty impressive feat.
-* **MD5SUM:** The md5sum of the image is used later on to verify that the
- image has been uploaded successfully.
-* **Uploading:** Everytime you upload an OS image, every block is hashed,
- checked against existing blocks in Pithos+ and finally uploaded, if no
- other block has the same hash.
+ $ sudo -s
+ $ snf-mkimage ubuntu_hd.raw
- *Consider this example: You have just uploaded a Gentoo Linux image but
- had forgotten to install a necessary package. In this case, you would
- probably edit the OS in the raw file and then use snf-mkimage to upload
- the new image. However, since there is an almost identical image already
- uploaded on Pithos+, you can just as well upload only the blocks that
- differentiate those two images. This is both time and space efficient.*
+In the first screen you will be asked to choose if you want to run the program
+in *Wizard* or *Expert* mode. Choose *Wizard*.
-Finally, after the image has been uploaded successfully, you will be asked
-whether you want to save a local copy of the **shrunken** image. This is
-just a copy of the diskdump that has been uploaded to Pithos+ and, in case
-you are confused, the original OS installed on the media (linuxmint.raw in
-our example) remains intact.
+.. image:: /snapshots/wizard.png
-snf-image-creator
-"""""""""""""""""
+Then you will be asked to provide a name, a description, a registration type
+(*private* or *public*) and the authentication token corresponding to your
+*~okeanos* account. Finally, you'll be asked to confirm the provided data.
-snf-image-creator is the command-line equivalent of snf-mkimage. All the
-info provided in the steps above are given now as options, which makes it
-ideal for scripting purposes. The full set of options can be found in the
-`Usage section <#non-interactive-version>`_ of snf-image-creator's
-documentation.
+.. image:: /snapshots/confirm.png
-This tool is most commonly used with the following set of options::
+Choosing *YES* will create and upload the image to your *~okeanos* account.
- $ sudo snf-image-creator linuxmint.raw -a user@email.com \
- -t hUudl4DEIlomlnvWnv7Rlw== -u linuxmint.diskdump -r "Linux Mint 14 Nadia"
+Limitations
+===========
-As you can see, these options are exactly what snf-mkimage's steps
-translate to. You can also see that the output is nearly identical:
+Supported operating systems
+---------------------------
-|image-creator|
+*snf-image-creator* can only fully function on input media hosting *Linux*
+systems. The program will detect the needed metadata and you may use it to
+upload and register other *Unix* or *Windows* images, but you cannot use it to
+shrink them or perform system preparation operations.
-Step 3: Create your VM
-----------------------
+Logical Volumes
+---------------
-Creating a VM out of an uploaded custom image is a fairly simple task.
-Just select "New Machine", go to "My Images" section and select your
-image.
+The program cannot work on LVM partitions [#f1]_. The input media may only
+contain primary or logical partitions.
-|custom-vm|
+Para-virtualized drivers
+------------------------
-Alternatively, if you want to create a VM from another user's custom
-image, you can go to the "Public Images" section.
+*~Okeanos* uses the *VirtIO* framework. The disk I/O controller and the
+Ethernet cards on the VM instances are para-virtualized and need special
+*VirtIO* drivers. Those drivers are included in the Linux Kernel mainline since
+version 2.6.25 and are shipped with all the popular Linux distributions. The
+problem is that if the driver for the para-virtualized disk I/O controller is
+built as module, it needs to be preloaded using an initial ramdisk, otherwise
+the VM won't be able to boot.
-.. |qemu-live| image:: /snapshots/qemu-live.png
+Many popular Linux distributions, like Ubuntu and Debian, will automatically
+create a generic initial ramdisk file that contains many different modules,
+including the VirtIO drivers. Others that target more experienced users, like
+Slackware, won't do that [#f2]_. *snf-image-creator* cannot resolve this kind
+of problems and it's left to the user to do so. Please refer to your
+distribution's documentation for more information on this. You can always check
+if a system can boot with para-virtualized disk controller by launching it with
+kvm using the *if=virtio* option (see the kvm command in the
+`Creating a new image`_ section).
-.. |qemu-partition| image:: /snapshots/qemu-partition.png
+Some caveats on image creation
+==============================
-.. |mkimage-wizard| image:: /snapshots/mkimage-wizard.png
+Image partition schemes and shrinking
+-------------------------------------
-.. |mkimage1| image:: /snapshots/mkimage1.png
+When image shrinking is enabled, *snf-image-creator* will shrink the last
+partition on the disk. If this is a swap partition, it will remove it, save
+enough information to recreate it during image deployment and shrink the
+partition that lays just before that. This will make the image smaller which
+speeds up the deployment process.
-.. |mkimage2| image:: /snapshots/mkimage2.png
+During image deployment, the last partition is enlarged to occupy the available
+space in the VM's hard disk and a swap partition is added at the end if a SWAP
+image property is present.
-.. |mkimage3| image:: /snapshots/mkimage3.png
+Keep this in mind when creating images. It's always better to have your swap
+partition placed as the last partition on the disk and have your largest
+partition (*/* or */home*) just before that.
-.. |mkimage4| image:: /snapshots/mkimage4.png
+Large temporary files
+---------------------
-.. |mkimage-fin| image:: /snapshots/mkimage-fin.png
+*snf-image-creator* may create large temporary files when running:
-.. |mkimage-results| image:: /snapshots/mkimage-results.png
+ * During image shrinking, the input media snapshot file may reach the size of
+ the original media.
+ * When bundling the host system, the temporary image file may became as large
+ as the rest of the disk files altogether.
-.. |image-creator| image:: /snapshots/image-creator.png
+*/tmp* directory is not a good place for hosting large files. In many systems
+the contents of */tmp* are stored in volatile memory and the size they may occupy
+is limited. By default, *snf-image-creator* will use a heuristic approach to
+determine where to store large temporary files. It will examine the free space
+under */var/tmp*, the user's home directory and */mnt* and will pick the one
+with the most available space. The user may overwrite this behaviour and
+indicate a different directory using the *tmpdir* option. This option is
+supported by both *snf-image-creator* and *snf-mkimage*.
-.. |custom-vm| image:: /snapshots/custom-vm.png
+.. rubric:: Footnotes
+.. [#f1] http://sourceware.org/lvm2/
+.. [#f2] http://mirrors.slackware.com/slackware/slackware-14.0/README.initrd
projects / snf-image-creator / blobdiff