Wiki

Version 10 (Vangelis Koukis, 10/10/2011 05:36 pm)

1 1 Constantinos Venetsanopoulos
h1. snf-image
2 1 Constantinos Venetsanopoulos
3 10 Vangelis Koukis
h2. Introduction - Features
4 10 Vangelis Koukis
5 3 Constantinos Venetsanopoulos
snf-image is a Ganeti OS Provider primarly used by Synnefo.
6 2 Constantinos Venetsanopoulos
7 7 Vangelis Koukis
It is written from scratch and allows Ganeti to launch instances from predefined or untrusted custom Images. The whole process of deploying an Image onto the block device, as provided by Ganeti, is done in complete isolation from the physical host, enhancing robustness and security.
8 1 Constantinos Venetsanopoulos
9 7 Vangelis Koukis
There are also additional hooks that can be enabled at image deployment. They allow for:
10 1 Constantinos Venetsanopoulos
11 7 Vangelis Koukis
 * changing the password of root or arbitrary users
12 7 Vangelis Koukis
 * injecting files at arbitrary locations inside the filesystem, e.g., SSH keys
13 7 Vangelis Koukis
 * setting a custom hostname
14 7 Vangelis Koukis
 * re-creating SSH host keys to ensure the image uses unique keys
15 1 Constantinos Venetsanopoulos
16 7 Vangelis Koukis
snf-image has been used successfully to deploy many major Linux distributions (Debian, Ubuntu/Kubuntu, CentOS, Fedora), as well as Windows 2008 R2.  
17 7 Vangelis Koukis
18 7 Vangelis Koukis
snf-image requires ganeti-os-interface v20 to operate.
19 7 Vangelis Koukis
It introduces the following OS parameters:
20 7 Vangelis Koukis
21 8 Vangelis Koukis
 * @img_id@ (_required_): the unique id of the image as known by the storage backend
22 8 Vangelis Koukis
 * @img_format@ (_required_): the image format (extdump and ntfsdump [dd] currently supported)
23 8 Vangelis Koukis
 * @img_passwd@ (_required_): the passwd to be injected inside the image
24 8 Vangelis Koukis
 * @img_personality@ (_optional_): files to be injected into the image filesystem. It is a JSON-encoded list of files to be injected: every file is defined by its path and base64-encoded data. This format follows the notation proposed by the "OpenStack Compute API v1.1":http://docs.openstack.org/api/openstack-compute/1.1/content/CreateServers.html. ["more...":http://docs.openstack.org/api/openstack-compute/1.1/content/Server_Personality-d1e2543.html ] for defining server personalities.
25 1 Constantinos Venetsanopoulos
26 1 Constantinos Venetsanopoulos
The snf-image Ganeti OS Provider is released under a [[Licence|2-clause BSD Licence]].
27 1 Constantinos Venetsanopoulos
28 1 Constantinos Venetsanopoulos
h2. Architecture
29 8 Vangelis Koukis
30 8 Vangelis Koukis
snf-image is split in two components: A part running on the Ganeti host, with full root privilege (@snf-image-host@), and a part running inside an unprivileged, helper VM (@snf-image-helper@).
31 8 Vangelis Koukis
32 8 Vangelis Koukis
h3. snf-image-host
33 8 Vangelis Koukis
34 8 Vangelis Koukis
This part implements the Ganeti OS interface. It extracts the Image onto the Ganeti-provided block device, using streaming block I/O (@dd@ with @oflag=direct@), then passes control to @snf-image-helper@ running inside a helper VM. The helper VM is created using KVM, runs as an unprivileged user, @nobody@ by default.
35 8 Vangelis Koukis
36 8 Vangelis Koukis
There is no restriction on the distribution running inside the helper VM, as long as it executes the @snf-image-helper@ component automatically upon bootup. The @snf-image-update-helper@ script is provided with @snf-image-host@ to automate the creation of a helper VM image based on Debian Stable, using @debootstrap@.
37 8 Vangelis Koukis
38 8 Vangelis Koukis
The @snf-image-helper@ component is spawned inside a specific hardware environment:
39 8 Vangelis Koukis
40 8 Vangelis Koukis
* The VM features a virtual floppy, containing an @ext2@ filesystem with all parameters needed for image customization.
41 8 Vangelis Koukis
* The hard disk of the VM being deployed is accessible as the first @virtio@ hard disk.
42 8 Vangelis Koukis
* All kernel/console output is redirected to the first virtual serial console, and eventually finds its way into the OS provider log files that Ganeti maintains.
43 8 Vangelis Koukis
* The helper VM is expected to output "SUCCESS" to its second serial port if image customization was successful inside the VM.
44 8 Vangelis Koukis
  In any other case, execution of the helper VM or @snf-image-helper@ has failed.
45 8 Vangelis Koukis
* The helper VM is expected to shutdown automatically once it is done. Its execution is time-limited; if it has not terminated after a number of seconds, configurable via @/etc/default/snf-image@, it is sent a @SIGTERM@ and/or a @SIGKILL@.
46 8 Vangelis Koukis
47 8 Vangelis Koukis
KVM is currently a dependency for @snf-image@, meaning it is needed to spawn the helper VM. There is no restriction on the hypervisor used for the actual Ganeti instances. This is not a strict requirement; KVM could be replaced by @qemu@, doing full CPU virtualization without any kernel support for spawning the helper VM.
48 8 Vangelis Koukis
49 8 Vangelis Koukis
h3. snf-image-helper
50 8 Vangelis Koukis
51 8 Vangelis Koukis
This part runs inside the helper VM and undertakes customization of the VM being deployed using a number of hooks, or _tasks_.
52 8 Vangelis Koukis
The tasks run in an environment, specified by rules found in a virtual floppy, placed there by the @snf-image-host@ component.
53 8 Vangelis Koukis
@snf-image-helper@ uses @runparts@ to run tasks found under @/usr/lib/snf-image-helper/tasks@ by default.
54 3 Constantinos Venetsanopoulos
55 1 Constantinos Venetsanopoulos
h2. Download
56 1 Constantinos Venetsanopoulos
57 10 Vangelis Koukis
The latest Debian packages are posted under:
58 10 Vangelis Koukis
https://code.grnet.gr/projects/snf-image/files
59 10 Vangelis Koukis
60 10 Vangelis Koukis
To download the latest development version, use @git@ to clone the snf-image repository:
61 10 Vangelis Koukis
git clone https://code.grnet.gr/git/snf-image
62 10 Vangelis Koukis
63 1 Constantinos Venetsanopoulos
h2. Installation
64 1 Constantinos Venetsanopoulos
65 9 Vangelis Koukis
Before installing snf-image be sure to have a working Ganeti installation in your cluster. The installation process should take place in *all* ganeti nodes. Here we will describe the installation in a single node. The process is identical for all nodes and should be repeated manually or automatically, e.g., with puppet.
66 5 Constantinos Venetsanopoulos
67 5 Constantinos Venetsanopoulos
# Download the snf-image-host debian package as described in the download section.
68 5 Constantinos Venetsanopoulos
# Install the snf-image-host debian package:
69 5 Constantinos Venetsanopoulos
<pre>
70 5 Constantinos Venetsanopoulos
 # dpkg -i snf-image-host_version.deb
71 5 Constantinos Venetsanopoulos
</pre>
72 5 Constantinos Venetsanopoulos
# If the dependencies are not met, install all the dependencies using @apt-get install@
73 5 Constantinos Venetsanopoulos
# Download the snf-image-helper debian package as described in the download section and store it in a handy location.
74 5 Constantinos Venetsanopoulos
# *Do NOT install the snf-image-helper debian package* in the Ganeti node (the package should be present but NOT installed in any node).
75 6 Constantinos Venetsanopoulos
# Configure the packages, as described in the next section, before you can start using the new OS Provider.
76 5 Constantinos Venetsanopoulos
77 5 Constantinos Venetsanopoulos
h2. Configuration
78 5 Constantinos Venetsanopoulos
79 5 Constantinos Venetsanopoulos
Once you have installed snf-image-host in the Ganeti node and also snf-image-helper is present, proceed with the following configuration:
80 9 Vangelis Koukis
81 5 Constantinos Venetsanopoulos
# Edit @/etc/default/snf-image@ and set the @HELPER_DIR@ variable to a directory in which all snf-helper related stuff are going to be stored (the directory should be able to store at least 700MB of data)
82 9 Vangelis Koukis
# Move the snf-image-helper debian package inside this directory and edit the @HELPER_PKG@ variable in @/etc/default/snf-image@ accordingly:
83 1 Constantinos Venetsanopoulos
<pre>
84 10 Vangelis Koukis
 $ mv /path/to/handy/location/snf-image-helper_version.deb $HELPER_DIR/
85 5 Constantinos Venetsanopoulos
</pre>
86 5 Constantinos Venetsanopoulos
# Use snf-image-update-helper to create a Debian stable-based helper VM image and install $HELPER_PKG in it
87 9 Vangelis Koukis
<pre>
88 1 Constantinos Venetsanopoulos
 $ /usr/bin/snf-image-update-helper
89 1 Constantinos Venetsanopoulos
</pre>
90 1 Constantinos Venetsanopoulos
91 1 Constantinos Venetsanopoulos
h2. Community & Support
92 1 Constantinos Venetsanopoulos
93 9 Vangelis Koukis
* Bug reports - feedback - support: synnefo@lists.grnet.gr