Wiki
Version 81 (Nikos Skalkotos, 01/17/2013 07:19 pm)
1 | 21 | Constantinos Venetsanopoulos | h2. snf-image |
---|---|---|---|
2 | 21 | Constantinos Venetsanopoulos | |
3 | 21 | Constantinos Venetsanopoulos | !snf-image-logo.png! |
4 | 1 | Constantinos Venetsanopoulos | |
5 | 10 | Vangelis Koukis | h2. Introduction - Features |
6 | 10 | Vangelis Koukis | |
7 | 54 | Constantinos Venetsanopoulos | snf-image is a "Ganeti":http://code.google.com/p/ganeti/ OS Definition, primarily used by "Synnefo":https://code.grnet.gr/projects/synnefo . |
8 | 2 | Constantinos Venetsanopoulos | |
9 | 66 | Constantinos Venetsanopoulos | It is rewritten from scratch and allows Ganeti to launch instances from predefined or untrusted custom Images. The whole process of deploying an Image onto the block device, as provided by Ganeti, is done in complete isolation from the physical host, enhancing robustness and security (currently supports only KVM based ganeti clusters). |
10 | 1 | Constantinos Venetsanopoulos | |
11 | 7 | Vangelis Koukis | There are also additional hooks that can be enabled at image deployment. They allow for: |
12 | 1 | Constantinos Venetsanopoulos | |
13 | 7 | Vangelis Koukis | * changing the password of root or arbitrary users |
14 | 7 | Vangelis Koukis | * injecting files at arbitrary locations inside the filesystem, e.g., SSH keys |
15 | 7 | Vangelis Koukis | * setting a custom hostname |
16 | 7 | Vangelis Koukis | * re-creating SSH host keys to ensure the image uses unique keys |
17 | 1 | Constantinos Venetsanopoulos | |
18 | 72 | Nikos Skalkotos | snf-image has been used successfully to deploy many major Linux distributions (Debian, Ubuntu/Kubuntu, CentOS, Fedora), as well as Windows 2008 R2 & Windows Server 2012. |
19 | 7 | Vangelis Koukis | |
20 | 40 | Constantinos Venetsanopoulos | The snf-image Ganeti OS Definition is released under [[Licence|GNU Generic Public Lisence version 2]]. |
21 | 35 | Nikos Skalkotos | |
22 | 35 | Nikos Skalkotos | h2. Ganeti OS Interface |
23 | 35 | Nikos Skalkotos | |
24 | 7 | Vangelis Koukis | snf-image requires ganeti-os-interface v20 to operate. |
25 | 7 | Vangelis Koukis | It introduces the following OS parameters: |
26 | 7 | Vangelis Koukis | |
27 | 69 | Nikos Skalkotos | * @img_id@ (_required if @config_url@ is missing_): the URI used to identify the image (see [[Image IDs and Storage Back Ends|here]]) |
28 | 63 | Constantinos Venetsanopoulos | * @img_format@ (_required if @config_url@ is missing_): the image format type (see [[Image Format|here]]) |
29 | 63 | Constantinos Venetsanopoulos | * @img_passwd@ (_required if @config_url@ is missing_): the passwd to be injected inside the image |
30 | 8 | Vangelis Koukis | * @img_properties@ (_optional_): additional image properties used to customize the image ([[Image_Format#Image-Properties|details]]) |
31 | 1 | Constantinos Venetsanopoulos | * @img_personality@ (_optional_): files to be injected into the image filesystem ([[Personality OS Parameter|details]]). |
32 | 62 | Constantinos Venetsanopoulos | * @config_url@ (_optional_): the url to download configuration data from |
33 | 1 | Constantinos Venetsanopoulos | |
34 | 1 | Constantinos Venetsanopoulos | h2. Architecture |
35 | 8 | Vangelis Koukis | |
36 | 8 | Vangelis Koukis | snf-image is split in two components: A part running on the Ganeti host, with full root privilege (@snf-image-host@), and a part running inside an unprivileged, helper VM (@snf-image-helper@). |
37 | 8 | Vangelis Koukis | |
38 | 8 | Vangelis Koukis | h3. snf-image-host |
39 | 8 | Vangelis Koukis | |
40 | 8 | Vangelis Koukis | This part implements the Ganeti OS interface. It extracts the Image onto the Ganeti-provided block device, using streaming block I/O (@dd@ with @oflag=direct@), then passes control to @snf-image-helper@ running inside a helper VM. The helper VM is created using KVM, runs as an unprivileged user, @nobody@ by default. |
41 | 8 | Vangelis Koukis | |
42 | 8 | Vangelis Koukis | There is no restriction on the distribution running inside the helper VM, as long as it executes the @snf-image-helper@ component automatically upon bootup. The @snf-image-update-helper@ script is provided with @snf-image-host@ to automate the creation of a helper VM image based on Debian Stable, using @debootstrap@. |
43 | 8 | Vangelis Koukis | |
44 | 8 | Vangelis Koukis | The @snf-image-helper@ component is spawned inside a specific hardware environment: |
45 | 8 | Vangelis Koukis | |
46 | 8 | Vangelis Koukis | * The VM features a virtual floppy, containing an @ext2@ filesystem with all parameters needed for image customization. |
47 | 8 | Vangelis Koukis | * The hard disk of the VM being deployed is accessible as the first @virtio@ hard disk. |
48 | 40 | Constantinos Venetsanopoulos | * All kernel/console output is redirected to the first virtual serial console, and eventually finds its way into the OS definition log files that Ganeti maintains. |
49 | 8 | Vangelis Koukis | * The helper VM is expected to output "SUCCESS" to its second serial port if image customization was successful inside the VM. |
50 | 8 | Vangelis Koukis | In any other case, execution of the helper VM or @snf-image-helper@ has failed. |
51 | 8 | Vangelis Koukis | * The helper VM is expected to shutdown automatically once it is done. Its execution is time-limited; if it has not terminated after a number of seconds, configurable via @/etc/default/snf-image@, it is sent a @SIGTERM@ and/or a @SIGKILL@. |
52 | 8 | Vangelis Koukis | |
53 | 8 | Vangelis Koukis | KVM is currently a dependency for @snf-image@, meaning it is needed to spawn the helper VM. There is no restriction on the hypervisor used for the actual Ganeti instances. This is not a strict requirement; KVM could be replaced by @qemu@, doing full CPU virtualization without any kernel support for spawning the helper VM. |
54 | 8 | Vangelis Koukis | |
55 | 8 | Vangelis Koukis | h3. snf-image-helper |
56 | 8 | Vangelis Koukis | |
57 | 43 | Nikos Skalkotos | This part runs inside the helper VM and undertakes customization of the VM being deployed using a number of hooks, or _[[Configuration Tasks|tasks]]_. The tasks run in an environment, specified by rules found in a virtual floppy, placed there by the @snf-image-host@ component. @snf-image-helper@ uses @runparts@ to run tasks found under @/usr/lib/snf-image-helper/tasks@ by default. |
58 | 3 | Constantinos Venetsanopoulos | |
59 | 14 | Vangelis Koukis | The architecture is presented below: |
60 | 11 | Constantinos Venetsanopoulos | |
61 | 12 | Constantinos Venetsanopoulos | h3. Graphical Representation |
62 | 11 | Constantinos Venetsanopoulos | |
63 | 11 | Constantinos Venetsanopoulos | !snf-image_arch.png! |
64 | 11 | Constantinos Venetsanopoulos | |
65 | 47 | Nikos Skalkotos | h3. Image Names |
66 | 47 | Nikos Skalkotos | |
67 | 47 | Nikos Skalkotos | snf-image assumes all image files have the following format: |
68 | 47 | Nikos Skalkotos | <pre><image_id>.<image_format></pre> |
69 | 47 | Nikos Skalkotos | |
70 | 47 | Nikos Skalkotos | In order to create an instance with ganeti using an image with filename @debian_base-6.0-5-x86_64.extdump@, you need to specify the following OS parameters: |
71 | 47 | Nikos Skalkotos | <pre>img_format=extdump |
72 | 47 | Nikos Skalkotos | img_id=debian_base-6.0-5-x86_64</pre> |
73 | 47 | Nikos Skalkotos | |
74 | 1 | Constantinos Venetsanopoulos | h2. Download |
75 | 1 | Constantinos Venetsanopoulos | |
76 | 79 | Nikos Skalkotos | h3. Source |
77 | 1 | Constantinos Venetsanopoulos | |
78 | 79 | Nikos Skalkotos | The latest source packages for snf-image (v0.7.5) can be found here: |
79 | 79 | Nikos Skalkotos | * "@snf-image-host_0.7.5.orig.tar.gz@":https://code.grnet.gr/attachments/download/2147/snf-image-host_0.7.5.orig.tar.gz |
80 | 79 | Nikos Skalkotos | * "@snf-image-helper_0.7.5.orig.tar.gz@":https://code.grnet.gr/attachments/download/2150/snf-image-helper_0.7.5.orig.tar.gz |
81 | 79 | Nikos Skalkotos | |
82 | 79 | Nikos Skalkotos | h3. Packages (for debian squeeze) |
83 | 79 | Nikos Skalkotos | |
84 | 79 | Nikos Skalkotos | The latest debian package for snf-image-host (v0.7.5) can be found here: |
85 | 77 | Nikos Skalkotos | * "@snf-image-host@":https://code.grnet.gr/attachments/download/2154/snf-image-host_0.7.5-1_all.deb |
86 | 10 | Vangelis Koukis | |
87 | 1 | Constantinos Venetsanopoulos | Previous versions and source tarballs can be found here: |
88 | 10 | Vangelis Koukis | https://code.grnet.gr/projects/snf-image/files |
89 | 1 | Constantinos Venetsanopoulos | |
90 | 73 | Nikos Skalkotos | You can also download the package from our @apt@ repo: |
91 | 64 | Constantinos Venetsanopoulos | |
92 | 78 | Nikos Skalkotos | http://apt2.dev.grnet.gr/stable/ |
93 | 64 | Constantinos Venetsanopoulos | |
94 | 80 | Nikos Skalkotos | h3. Fixed snf-image-helper image |
95 | 80 | Nikos Skalkotos | |
96 | 80 | Nikos Skalkotos | People who installed snf-image-host using the provided package can create the helper VM by launching `snf-image-update-helper' command. People who prefer installing the program from source may use the fixed helper VM image provided below: |
97 | 80 | Nikos Skalkotos | |
98 | 80 | Nikos Skalkotos | * "@snf_image_helper_image-0.7.5-1.tar.xz@":https://pithos.okeanos.grnet.gr/public/87r3q |
99 | 80 | Nikos Skalkotos | |
100 | 41 | Constantinos Venetsanopoulos | h3. Sample Images |
101 | 41 | Constantinos Venetsanopoulos | |
102 | 41 | Constantinos Venetsanopoulos | While developing snf-image, we created and tested a number of images. The following images are basic installations of some popular Linux distributions, that have been tested with snf-image and provided here for testing purposes: |
103 | 41 | Constantinos Venetsanopoulos | |
104 | 70 | Nikos Skalkotos | * Debian Squeeze Base System ["diskdump":https://pithos.okeanos.grnet.gr/public/66ke3] ["md5sum":https://pithos.okeanos.grnet.gr/public/pcwxc] ["metadata":https://pithos.okeanos.grnet.gr/public/4hd8n] |
105 | 71 | Nikos Skalkotos | * Debian Desktop ["diskdump":https://pithos.okeanos.grnet.gr/public/9avb8] ["md5sum":https://pithos.okeanos.grnet.gr/public/2p8mq] ["metadata":https://pithos.okeanos.grnet.gr/public/ggk6r] |
106 | 61 | Nikos Skalkotos | * CentOS 6.0 ["diskdump":https://pithos.okeanos.grnet.gr/public/mw8vg] ["md5sum":https://pithos.okeanos.grnet.gr/public/85k8d] ["metadata":https://pithos.okeanos.grnet.gr/public/cm6tz] |
107 | 60 | Nikos Skalkotos | * Fedora Desktop 17 ["diskdump":https://pithos.okeanos.grnet.gr/public/wf6c7] ["md5sum":https://pithos.okeanos.grnet.gr/public/j76kv] ["metadata":https://pithos.okeanos.grnet.gr/public/yss35] |
108 | 60 | Nikos Skalkotos | * Ubuntu Desktop 12.04 ["diskdump":https://pithos.okeanos.grnet.gr/public/vjkyy] ["md5sum":https://pithos.okeanos.grnet.gr/public/z4wgs] ["metadata":https://pithos.okeanos.grnet.gr/public/mejnd] |
109 | 60 | Nikos Skalkotos | * Kubuntu Desktop 12.04 ["diskdump":https://pithos.okeanos.grnet.gr/public/8xd6f] ["md5sum":https://pithos.okeanos.grnet.gr/public/cvkzx] ["metadata":https://pithos.okeanos.grnet.gr/public/wpwh6] |
110 | 41 | Constantinos Venetsanopoulos | |
111 | 41 | Constantinos Venetsanopoulos | After getting familiar with snf-image you can build your own Images ([[Image Format|more...]]), or build upon the ones listed above. |
112 | 33 | Vangelis Koukis | |
113 | 1 | Constantinos Venetsanopoulos | h2. Installation |
114 | 1 | Constantinos Venetsanopoulos | |
115 | 5 | Constantinos Venetsanopoulos | Before installing snf-image be sure to have a working Ganeti installation in your cluster. The installation process should take place in *all* ganeti nodes. Here we will describe the installation in a single node. The process is identical for all nodes and should be repeated manually or automatically, e.g., with puppet. |
116 | 5 | Constantinos Venetsanopoulos | |
117 | 81 | Nikos Skalkotos | h3. Installing snf-image using binary packages |
118 | 81 | Nikos Skalkotos | |
119 | 73 | Nikos Skalkotos | # Download the snf-image-host debian package as described in the download section. |
120 | 1 | Constantinos Venetsanopoulos | # Install the snf-image-host debian package:<pre>$ dpkg -i snf-image-host_version.deb</pre> |
121 | 81 | Nikos Skalkotos | # If the dependencies are not met, install all the dependencies using @apt-get -f install@ (or install @snf-image-host@ with @apt-get@). |
122 | 1 | Constantinos Venetsanopoulos | # Create the helper VM using the following command:<pre>$ snf-image-update-helper</pre>This command will automatically download all the packages needed to create the helper VM (including snf-image-helper). |
123 | 81 | Nikos Skalkotos | |
124 | 81 | Nikos Skalkotos | h3. Installing snf-image from source |
125 | 81 | Nikos Skalkotos | |
126 | 81 | Nikos Skalkotos | # Download the provided source package:<pre>$ wget https://code.grnet.gr/attachments/download/2147/snf-image-host_0.7.5.orig.tar.gz</pre> |
127 | 81 | Nikos Skalkotos | # Untar, configure and compile the source:<pre>$ tar -xvf snf-image-host_0.7.5.orig.tar.gz |
128 | 81 | Nikos Skalkotos | $ cd snf-image-host-0.7.5 |
129 | 81 | Nikos Skalkotos | $ ./autoget.sh |
130 | 81 | Nikos Skalkotos | $ ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var |
131 | 81 | Nikos Skalkotos | $ make</pre> |
132 | 81 | Nikos Skalkotos | # Install snf-image-host:<pre>$ make install |
133 | 81 | Nikos Skalkotos | $ install -Dm600 /etc/defaults/snf-image</pre> |
134 | 81 | Nikos Skalkotos | # Download the fixed snf-image-helper image |
135 | 81 | Nikos Skalkotos | # Extract it's content under @/var/lib/snf-image/helper/@:<pre>$ mkdir -p /var/lib/snf-image/helper |
136 | 81 | Nikos Skalkotos | $ cd /var/lib/snf-image/helper |
137 | 81 | Nikos Skalkotos | $ tar -xvf /place/where/you/stored/snf_image_helper_image-0.7.5-1.tar.xz</pre> |
138 | 45 | Nikos Skalkotos | |
139 | 45 | Nikos Skalkotos | h2. Configuration |
140 | 1 | Constantinos Venetsanopoulos | |
141 | 45 | Nikos Skalkotos | In this section, we assume that the image file you want to deploy resides under a local directory. |
142 | 73 | Nikos Skalkotos | Once you have installed snf-image-host in the Ganeti node, you may need to overwrite the value of the following variables in @/etc/default/snf-image@: |
143 | 55 | Constantinos Venetsanopoulos | # *IMAGE_DIR*: This should point to the location where the image files are stored [default: @/var/lib/snf-image@]. |
144 | 31 | Vangelis Koukis | |
145 | 47 | Nikos Skalkotos | h2. Sample Usage |
146 | 1 | Constantinos Venetsanopoulos | |
147 | 65 | Constantinos Venetsanopoulos | h3. @diskdump@ images |
148 | 65 | Constantinos Venetsanopoulos | |
149 | 65 | Constantinos Venetsanopoulos | If you want to deploy an image of type @diskdump@, you need to provide the corresponding @img_properties@ as described in the [[Image_Format#OS-parameter-img_properties|Image Format]] section. If using a diskdump found in the Sample Images list, use the @img_properties@ described in the image's metadata file. For example: |
150 | 65 | Constantinos Venetsanopoulos | |
151 | 65 | Constantinos Venetsanopoulos | <pre>gnt-instance add -o snf-image+default --os-parameters img_passwd=SamplePassw0rd,img_format=diskdump,img_id=debian_base-6.0-7-x86_64,img_properties='{"OSFAMILY":"linux"\,"ROOT_PARTITION":"1"}' -t plain --disk=0:size=10G --no-name-check --no-ip-check --no-nics test1</pre> |
152 | 65 | Constantinos Venetsanopoulos | |
153 | 56 | Constantinos Venetsanopoulos | h3. @extdump@/@ntfsdump@ images |
154 | 1 | Constantinos Venetsanopoulos | |
155 | 56 | Constantinos Venetsanopoulos | To create a ganeti instance with snf-image using @debian_base-6.0-5-x86_64.extdump@ hosted under @IMAGE_DIR@, use a command like the one below: |
156 | 56 | Constantinos Venetsanopoulos | <pre>gnt-instance add -o snf-image+default --os-parameters img_passwd=SamplePassw0rd,img_format=extdump,img_id=debian_base-6.0-5-x86_64 -t plain --disk=0:size=10G --no-name-check --no-ip-check --no-nics test1</pre> |
157 | 1 | Constantinos Venetsanopoulos | |
158 | 65 | Constantinos Venetsanopoulos | h2. Advanced Usage |
159 | 1 | Constantinos Venetsanopoulos | |
160 | 65 | Constantinos Venetsanopoulos | If you want to learn more and use the advanced features of snf-image please refer to following sections: |
161 | 1 | Constantinos Venetsanopoulos | |
162 | 76 | Nikos Skalkotos | # [[Configuration Tasks|Image Configuration Tasks]] |
163 | 76 | Nikos Skalkotos | # [[Image Format|Supported Image formats]] |
164 | 76 | Nikos Skalkotos | # [[Image Store|Supported Image store locations]] |
165 | 76 | Nikos Skalkotos | # [[PMI|Progress Monitor Interface]] |
166 | 22 | Constantinos Venetsanopoulos | |
167 | 13 | Constantinos Venetsanopoulos | h2. Developers |
168 | 46 | Nikos Skalkotos | |
169 | 45 | Nikos Skalkotos | If you are a developer or want to use the latest source code you can download the latest development version by cloning the snf-image git repository: |
170 | 18 | Vangelis Koukis | <pre>$ git clone https://code.grnet.gr/git/snf-image</pre> |
171 | 18 | Vangelis Koukis | The _master_ branch contains the latest development version |
172 | 18 | Vangelis Koukis | The _debian_ branch contains the _master_ plus the debian packaging directories. |
173 | 24 | Constantinos Venetsanopoulos | |
174 | 27 | Nikos Skalkotos | For more information on how to configure, install or package from source, see the [[Developer|Developer's page]]. |
175 | 19 | Vangelis Koukis | |
176 | 1 | Constantinos Venetsanopoulos | h2. Community & Support |
177 | 13 | Constantinos Venetsanopoulos | |
178 | 13 | Constantinos Venetsanopoulos | Your help is very important. Any contributions and bug reports will be highly appreciated. You can contact the team at: |
179 | 1 | Constantinos Venetsanopoulos | |
180 | 75 | Nikos Skalkotos | * Bug reports - feedback - support: synnefo@googlegroups.com |