root / doc / design-glusterfs-ganeti-support.rst @ ab6536ba
History | View | Annotate | Download (4.3 kB)
1 | 50119242 | Weiwei Jia | ======================== |
---|---|---|---|
2 | 50119242 | Weiwei Jia | GlusterFS Ganeti support |
3 | 50119242 | Weiwei Jia | ======================== |
4 | 50119242 | Weiwei Jia | |
5 | 50119242 | Weiwei Jia | This document describes the plan for adding GlusterFS support inside Ganeti. |
6 | 50119242 | Weiwei Jia | |
7 | 50119242 | Weiwei Jia | .. contents:: :depth: 4 |
8 | 50119242 | Weiwei Jia | .. highlight:: shell-example |
9 | 50119242 | Weiwei Jia | |
10 | 50119242 | Weiwei Jia | Objective |
11 | 50119242 | Weiwei Jia | ========= |
12 | 50119242 | Weiwei Jia | |
13 | 50119242 | Weiwei Jia | The aim is to let Ganeti support GlusterFS as one of its backend storage. |
14 | 50119242 | Weiwei Jia | This includes three aspects to finish: |
15 | 50119242 | Weiwei Jia | |
16 | 50119242 | Weiwei Jia | - Add Gluster as a storage backend. |
17 | 50119242 | Weiwei Jia | - Make sure Ganeti VMs can use GlusterFS backends in userspace mode (for |
18 | 50119242 | Weiwei Jia | newer QEMU/KVM which has this support) and otherwise, if possible, through |
19 | 50119242 | Weiwei Jia | some kernel exported block device. |
20 | 50119242 | Weiwei Jia | - Make sure Ganeti can configure GlusterFS by itself, by just joining |
21 | 50119242 | Weiwei Jia | storage space on new nodes to a GlusterFS nodes pool. Note that this |
22 | 50119242 | Weiwei Jia | may need another design document that explains how it interacts with |
23 | 50119242 | Weiwei Jia | storage pools, and that the node might or might not host VMs as well. |
24 | 50119242 | Weiwei Jia | |
25 | 50119242 | Weiwei Jia | Background |
26 | 50119242 | Weiwei Jia | ========== |
27 | 50119242 | Weiwei Jia | |
28 | 50119242 | Weiwei Jia | There are two possible ways to implement "GlusterFS Ganeti Support". One is |
29 | 50119242 | Weiwei Jia | GlusterFS as one of external backend storage, the other one is realizing |
30 | 50119242 | Weiwei Jia | GlusterFS inside Ganeti, that is, as a new disk type for Ganeti. The benefit |
31 | 50119242 | Weiwei Jia | of the latter one is that it would not be opaque but fully supported and |
32 | 50119242 | Weiwei Jia | integrated in Ganeti, which would not need to add infrastructures for |
33 | 50119242 | Weiwei Jia | testing/QAing and such. Having it internal we can also provide a monitoring |
34 | 50119242 | Weiwei Jia | agent for it and more visibility into what's going on. For these reasons, |
35 | 50119242 | Weiwei Jia | GlusterFS support will be added directly inside Ganeti. |
36 | 50119242 | Weiwei Jia | |
37 | 50119242 | Weiwei Jia | Implementation Plan |
38 | 50119242 | Weiwei Jia | =================== |
39 | 50119242 | Weiwei Jia | |
40 | 50119242 | Weiwei Jia | Ganeti Side |
41 | 50119242 | Weiwei Jia | ----------- |
42 | 50119242 | Weiwei Jia | |
43 | 50119242 | Weiwei Jia | To realize an internal storage backend for Ganeti, one should realize |
44 | 50119242 | Weiwei Jia | BlockDev class in `ganeti/lib/storage/base.py` that is a specific |
45 | 50119242 | Weiwei Jia | class including create, remove and such. These functions should be |
46 | 50119242 | Weiwei Jia | realized in `ganeti/lib/storage/bdev.py`. Actually, the differences |
47 | 50119242 | Weiwei Jia | between implementing inside and outside (external) Ganeti are how to |
48 | 50119242 | Weiwei Jia | finish these functions in BlockDev class and how to combine with Ganeti |
49 | 50119242 | Weiwei Jia | itself. The internal implementation is not based on external scripts |
50 | 50119242 | Weiwei Jia | and combines with Ganeti in a more compact way. RBD patches may be a |
51 | 50119242 | Weiwei Jia | good reference here. Adding a backend storage steps are as follows: |
52 | 50119242 | Weiwei Jia | |
53 | 50119242 | Weiwei Jia | - Implement the BlockDev interface in bdev.py. |
54 | 50119242 | Weiwei Jia | - Add the logic in cmdlib (eg, migration, verify). |
55 | 50119242 | Weiwei Jia | - Add the new storage type name to constants. |
56 | 50119242 | Weiwei Jia | - Modify objects.Disk to support GlusterFS storage type. |
57 | 50119242 | Weiwei Jia | - The implementation will be performed similarly to the RBD one (see |
58 | 50119242 | Weiwei Jia | commit 7181fba). |
59 | 50119242 | Weiwei Jia | |
60 | 50119242 | Weiwei Jia | GlusterFS side |
61 | 50119242 | Weiwei Jia | -------------- |
62 | 50119242 | Weiwei Jia | |
63 | 50119242 | Weiwei Jia | GlusterFS is a distributed file system implemented in user space. |
64 | 50119242 | Weiwei Jia | The way to access GlusterFS namespace is via FUSE based Gluster native |
65 | 50119242 | Weiwei Jia | client except NFS and CIFS. The efficiency of this way is lower because |
66 | 50119242 | Weiwei Jia | the data would be pass the kernel space and then come to user space. |
67 | 50119242 | Weiwei Jia | Now, there are two specific enhancements: |
68 | 50119242 | Weiwei Jia | |
69 | 50119242 | Weiwei Jia | - A new library called libgfapi is now available as part of GlusterFS |
70 | 50119242 | Weiwei Jia | that provides POSIX-like C APIs for accessing Gluster volumes. |
71 | 50119242 | Weiwei Jia | libgfapi support will be available from GlusterFS-3.4 release. |
72 | 50119242 | Weiwei Jia | - QEMU/KVM (starting from QEMU-1.3) will have GlusterFS block driver that |
73 | 50119242 | Weiwei Jia | uses libgfapi and hence there is no FUSE overhead any longer when QEMU/KVM |
74 | 50119242 | Weiwei Jia | works with VM images on Gluster volumes. |
75 | 50119242 | Weiwei Jia | |
76 | ffde7fb6 | Weiwei Jia | Proposed implementation |
77 | ffde7fb6 | Weiwei Jia | ----------------------- |
78 | ffde7fb6 | Weiwei Jia | |
79 | ffde7fb6 | Weiwei Jia | QEMU/KVM includes support for GlusterFS and Ganeti could support GlusterFS |
80 | ffde7fb6 | Weiwei Jia | through QEMU/KVM. However, this way could just let VMs of QEMU/KVM use GlusterFS |
81 | ffde7fb6 | Weiwei Jia | backend storage but not other VMs like XEN and such. There are two parts that need |
82 | ffde7fb6 | Weiwei Jia | to be implemented for supporting GlusterFS inside Ganeti so that it can not only |
83 | ffde7fb6 | Weiwei Jia | support QEMU/KVM VMs, but also XEN and other VMs. One part is GlusterFS for XEN VM, |
84 | ffde7fb6 | Weiwei Jia | which is similar to sharedfile disk template. The other part is GlusterFS for |
85 | ffde7fb6 | Weiwei Jia | QEMU/KVM VM, which is supported by the GlusterFS driver for QEMU/KVM. After |
86 | ffde7fb6 | Weiwei Jia | ``gnt-instance add -t gluster instance.example.com`` command is executed, the added |
87 | ffde7fb6 | Weiwei Jia | instance should be checked. If the instance is a XEN VM, it would run the GlusterFS |
88 | ffde7fb6 | Weiwei Jia | sharedfile way. However, if the instance is a QEMU/KVM VM, it would run the |
89 | ffde7fb6 | Weiwei Jia | QEMU/KVM + GlusterFS way. For the first part (GlusterFS for XEN VMs), sharedfile |
90 | ffde7fb6 | Weiwei Jia | disk template would be a good reference. For the second part (GlusterFS for QEMU/KVM |
91 | ffde7fb6 | Weiwei Jia | VMs), RBD disk template would be a good reference. The first part would be finished |
92 | ffde7fb6 | Weiwei Jia | at first and then the second part would be completed, which is based on the first |
93 | ffde7fb6 | Weiwei Jia | part. |
94 | 50119242 | Weiwei Jia | |
95 | 50119242 | Weiwei Jia | .. vim: set textwidth=72 : |
96 | 50119242 | Weiwei Jia | .. Local Variables: |
97 | 50119242 | Weiwei Jia | .. mode: rst |
98 | 50119242 | Weiwei Jia | .. fill-column: 72 |
99 | 50119242 | Weiwei Jia | .. End: |