root / doc / design-glusterfs-ganeti-support.rst @ 333bd799
History | View | Annotate | Download (12.6 kB)
1 | 50119242 | Weiwei Jia | ======================== |
---|---|---|---|
2 | 50119242 | Weiwei Jia | GlusterFS Ganeti support |
3 | 50119242 | Weiwei Jia | ======================== |
4 | 50119242 | Weiwei Jia | |
5 | 50119242 | Weiwei Jia | This document describes the plan for adding GlusterFS support inside Ganeti. |
6 | 50119242 | Weiwei Jia | |
7 | 50119242 | Weiwei Jia | .. contents:: :depth: 4 |
8 | 50119242 | Weiwei Jia | .. highlight:: shell-example |
9 | 50119242 | Weiwei Jia | |
10 | cdea7fa8 | Santi Raffa | Gluster overview |
11 | cdea7fa8 | Santi Raffa | ================ |
12 | cdea7fa8 | Santi Raffa | |
13 | cdea7fa8 | Santi Raffa | Gluster is a "brick" "translation" service that can turn a number of LVM logical |
14 | cdea7fa8 | Santi Raffa | volume or disks (so-called "bricks") into an unified "volume" that can be |
15 | cdea7fa8 | Santi Raffa | mounted over the network through FUSE or NFS. |
16 | cdea7fa8 | Santi Raffa | |
17 | cdea7fa8 | Santi Raffa | This is a simplified view of what components are at play and how they |
18 | cdea7fa8 | Santi Raffa | interconnect as data flows from the actual disks to the instances. The parts in |
19 | cdea7fa8 | Santi Raffa | grey are available for Ganeti to use and included for completeness but not |
20 | cdea7fa8 | Santi Raffa | targeted for implementation at this stage. |
21 | cdea7fa8 | Santi Raffa | |
22 | cdea7fa8 | Santi Raffa | .. digraph:: "gluster-ganeti-overview" |
23 | cdea7fa8 | Santi Raffa | |
24 | cdea7fa8 | Santi Raffa | graph [ spline=ortho ] |
25 | cdea7fa8 | Santi Raffa | node [ shape=rect ] |
26 | cdea7fa8 | Santi Raffa | |
27 | cdea7fa8 | Santi Raffa | { |
28 | cdea7fa8 | Santi Raffa | |
29 | cdea7fa8 | Santi Raffa | node [ shape=none ] |
30 | cdea7fa8 | Santi Raffa | _volume [ label=volume ] |
31 | cdea7fa8 | Santi Raffa | |
32 | cdea7fa8 | Santi Raffa | bricks -> translators -> _volume |
33 | cdea7fa8 | Santi Raffa | _volume -> network [label=transport] |
34 | cdea7fa8 | Santi Raffa | network -> instances |
35 | cdea7fa8 | Santi Raffa | } |
36 | cdea7fa8 | Santi Raffa | |
37 | cdea7fa8 | Santi Raffa | { rank=same; brick1 [ shape=oval ] |
38 | cdea7fa8 | Santi Raffa | brick2 [ shape=oval ] |
39 | cdea7fa8 | Santi Raffa | brick3 [ shape=oval ] |
40 | cdea7fa8 | Santi Raffa | bricks } |
41 | cdea7fa8 | Santi Raffa | { rank=same; translators distribute } |
42 | cdea7fa8 | Santi Raffa | { rank=same; volume [ shape=oval ] |
43 | cdea7fa8 | Santi Raffa | _volume } |
44 | cdea7fa8 | Santi Raffa | { rank=same; instances instanceA instanceB instanceC instanceD } |
45 | cdea7fa8 | Santi Raffa | { rank=same; network FUSE NFS QEMUC QEMUD } |
46 | cdea7fa8 | Santi Raffa | |
47 | cdea7fa8 | Santi Raffa | { |
48 | cdea7fa8 | Santi Raffa | node [ shape=oval ] |
49 | cdea7fa8 | Santi Raffa | brick1 [ label=brick ] |
50 | cdea7fa8 | Santi Raffa | brick2 [ label=brick ] |
51 | cdea7fa8 | Santi Raffa | brick3 [ label=brick ] |
52 | cdea7fa8 | Santi Raffa | } |
53 | cdea7fa8 | Santi Raffa | |
54 | cdea7fa8 | Santi Raffa | { |
55 | cdea7fa8 | Santi Raffa | node [ shape=oval ] |
56 | cdea7fa8 | Santi Raffa | volume |
57 | cdea7fa8 | Santi Raffa | } |
58 | cdea7fa8 | Santi Raffa | |
59 | cdea7fa8 | Santi Raffa | brick1 -> distribute |
60 | cdea7fa8 | Santi Raffa | brick2 -> distribute |
61 | cdea7fa8 | Santi Raffa | brick3 -> distribute -> volume |
62 | cdea7fa8 | Santi Raffa | volume -> FUSE [ label=<TCP<br/><font color="grey">UDP</font>> |
63 | cdea7fa8 | Santi Raffa | color="black:grey" ] |
64 | cdea7fa8 | Santi Raffa | |
65 | cdea7fa8 | Santi Raffa | NFS [ color=grey fontcolor=grey ] |
66 | cdea7fa8 | Santi Raffa | volume -> NFS [ label="TCP" color=grey fontcolor=grey ] |
67 | cdea7fa8 | Santi Raffa | NFS -> mountpoint [ color=grey fontcolor=grey ] |
68 | cdea7fa8 | Santi Raffa | |
69 | cdea7fa8 | Santi Raffa | mountpoint [ shape=oval ] |
70 | cdea7fa8 | Santi Raffa | |
71 | cdea7fa8 | Santi Raffa | FUSE -> mountpoint |
72 | cdea7fa8 | Santi Raffa | |
73 | cdea7fa8 | Santi Raffa | instanceA [ label=instances ] |
74 | cdea7fa8 | Santi Raffa | instanceB [ label=instances ] |
75 | cdea7fa8 | Santi Raffa | |
76 | cdea7fa8 | Santi Raffa | mountpoint -> instanceA |
77 | cdea7fa8 | Santi Raffa | mountpoint -> instanceB |
78 | cdea7fa8 | Santi Raffa | |
79 | cdea7fa8 | Santi Raffa | mountpoint [ shape=oval ] |
80 | cdea7fa8 | Santi Raffa | |
81 | cdea7fa8 | Santi Raffa | QEMUC [ label=QEMU ] |
82 | cdea7fa8 | Santi Raffa | QEMUD [ label=QEMU ] |
83 | cdea7fa8 | Santi Raffa | |
84 | cdea7fa8 | Santi Raffa | { |
85 | cdea7fa8 | Santi Raffa | instanceC [ label=instances ] |
86 | cdea7fa8 | Santi Raffa | instanceD [ label=instances ] |
87 | cdea7fa8 | Santi Raffa | } |
88 | cdea7fa8 | Santi Raffa | |
89 | cdea7fa8 | Santi Raffa | volume -> QEMUC [ label=<TCP<br/><font color="grey">UDP</font>> |
90 | cdea7fa8 | Santi Raffa | color="black:grey" ] |
91 | cdea7fa8 | Santi Raffa | volume -> QEMUD [ label=<TCP<br/><font color="grey">UDP</font>> |
92 | cdea7fa8 | Santi Raffa | color="black:grey" ] |
93 | cdea7fa8 | Santi Raffa | QEMUC -> instanceC |
94 | cdea7fa8 | Santi Raffa | QEMUD -> instanceD |
95 | cdea7fa8 | Santi Raffa | |
96 | cdea7fa8 | Santi Raffa | brick: |
97 | cdea7fa8 | Santi Raffa | The unit of storage in gluster. Typically a drive or LVM logical volume |
98 | cdea7fa8 | Santi Raffa | formatted using, for example, XFS. |
99 | cdea7fa8 | Santi Raffa | |
100 | cdea7fa8 | Santi Raffa | distribute: |
101 | cdea7fa8 | Santi Raffa | One of the translators in Gluster, it assigns files to bricks based on the |
102 | cdea7fa8 | Santi Raffa | hash of their full path inside the volume. |
103 | cdea7fa8 | Santi Raffa | |
104 | cdea7fa8 | Santi Raffa | volume: |
105 | cdea7fa8 | Santi Raffa | A filesystem you can mount on multiple machines; all machines see the same |
106 | cdea7fa8 | Santi Raffa | directory tree and files. |
107 | cdea7fa8 | Santi Raffa | |
108 | cdea7fa8 | Santi Raffa | FUSE/NFS: |
109 | cdea7fa8 | Santi Raffa | Gluster offers two ways to mount volumes: through FUSE or a custom NFS server |
110 | cdea7fa8 | Santi Raffa | that is incompatible with other NFS servers. FUSE is more compatible with |
111 | cdea7fa8 | Santi Raffa | other services running on the storage nodes; NFS gives better performance. |
112 | cdea7fa8 | Santi Raffa | For now, FUSE is a priority. |
113 | cdea7fa8 | Santi Raffa | |
114 | cdea7fa8 | Santi Raffa | QEMU: |
115 | cdea7fa8 | Santi Raffa | QEMU 1.3 has the ability to use Gluster volumes directly in userspace without |
116 | cdea7fa8 | Santi Raffa | the need for mounting anything. Ganeti still needs kernelspace access at disk |
117 | cdea7fa8 | Santi Raffa | creation and OS install time. |
118 | cdea7fa8 | Santi Raffa | |
119 | cdea7fa8 | Santi Raffa | transport: |
120 | cdea7fa8 | Santi Raffa | FUSE and QEMU allow you to connect using TCP and UDP, whereas NFS only |
121 | cdea7fa8 | Santi Raffa | supports TCP. Those protocols are called transports in Gluster. For now, TCP |
122 | cdea7fa8 | Santi Raffa | is a priority. |
123 | cdea7fa8 | Santi Raffa | |
124 | cdea7fa8 | Santi Raffa | It is the administrator's duty to set up the bricks, the translators and thus |
125 | cdea7fa8 | Santi Raffa | the volume as they see fit. Ganeti will take care of connecting the instances to |
126 | cdea7fa8 | Santi Raffa | a given volume. |
127 | cdea7fa8 | Santi Raffa | |
128 | cdea7fa8 | Santi Raffa | .. note:: |
129 | cdea7fa8 | Santi Raffa | |
130 | cdea7fa8 | Santi Raffa | The gluster mountpoint must be whitelisted by the administrator in |
131 | cdea7fa8 | Santi Raffa | ``/etc/ganeti/file-storage-paths`` for security reasons in order to allow |
132 | cdea7fa8 | Santi Raffa | Ganeti to modify the filesystem. |
133 | cdea7fa8 | Santi Raffa | |
134 | cdea7fa8 | Santi Raffa | Why not use a ``sharedfile`` disk template? |
135 | cdea7fa8 | Santi Raffa | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
136 | cdea7fa8 | Santi Raffa | |
137 | cdea7fa8 | Santi Raffa | Gluster volumes `can` be used by Ganeti using the generic shared file disk |
138 | cdea7fa8 | Santi Raffa | template. There is a number of reasons why that is probably not a good idea, |
139 | cdea7fa8 | Santi Raffa | however: |
140 | cdea7fa8 | Santi Raffa | |
141 | cdea7fa8 | Santi Raffa | * Shared file, being a generic solution, cannot offer userspace access support. |
142 | cdea7fa8 | Santi Raffa | * Even with userspace support, Ganeti still needs kernelspace access in order to |
143 | cdea7fa8 | Santi Raffa | create disks and install OSes on them. Ganeti can manage the mounting for you |
144 | cdea7fa8 | Santi Raffa | so that the Gluster servers only have as many connections as necessary. |
145 | cdea7fa8 | Santi Raffa | * Experiments showed that you can't trust ``mount.glusterfs`` to give useful |
146 | cdea7fa8 | Santi Raffa | return codes or error messages. Ganeti can work around its oddities so |
147 | cdea7fa8 | Santi Raffa | administrators don't have to. |
148 | cdea7fa8 | Santi Raffa | * The shared file folder scheme (``../{instance.name}/disk{disk.id}``) does not |
149 | cdea7fa8 | Santi Raffa | work well with Gluster. The ``distribute`` translator distributes files across |
150 | cdea7fa8 | Santi Raffa | bricks, but directories need to be replicated on `all` bricks. As a result, if |
151 | cdea7fa8 | Santi Raffa | we have a dozen hundred instances, that means a dozen hundred folders being |
152 | cdea7fa8 | Santi Raffa | replicated on all bricks. This does not scale well. |
153 | cdea7fa8 | Santi Raffa | * This frees up the shared file disk template to use a different, unsupported |
154 | cdea7fa8 | Santi Raffa | replication scheme together with Gluster. (Storage pools are the long term |
155 | cdea7fa8 | Santi Raffa | solution for this, however.) |
156 | cdea7fa8 | Santi Raffa | |
157 | cdea7fa8 | Santi Raffa | So, while gluster `is` a shared file disk template, essentially, Ganeti can |
158 | cdea7fa8 | Santi Raffa | provide better support for it than that. |
159 | cdea7fa8 | Santi Raffa | |
160 | cdea7fa8 | Santi Raffa | Implementation strategy |
161 | cdea7fa8 | Santi Raffa | ======================= |
162 | cdea7fa8 | Santi Raffa | |
163 | cdea7fa8 | Santi Raffa | Working with GlusterFS in kernel space essentially boils down to: |
164 | cdea7fa8 | Santi Raffa | |
165 | cdea7fa8 | Santi Raffa | 1. Ask FUSE to mount the Gluster volume. |
166 | cdea7fa8 | Santi Raffa | 2. Check that the mount succeeded. |
167 | cdea7fa8 | Santi Raffa | 3. Use files stored in the volume as instance disks, just like sharedfile does. |
168 | cdea7fa8 | Santi Raffa | 4. When the instances are spun down, attempt unmounting the volume. If the |
169 | cdea7fa8 | Santi Raffa | gluster connection is still required, the mountpoint is allowed to remain. |
170 | cdea7fa8 | Santi Raffa | |
171 | cdea7fa8 | Santi Raffa | Since it is not strictly necessary for Gluster to mount the disk if all that's |
172 | cdea7fa8 | Santi Raffa | needed is userspace access, however, it is inappropriate for the Gluster storage |
173 | cdea7fa8 | Santi Raffa | class to inherit from FileStorage. So the implementation should resort to |
174 | cdea7fa8 | Santi Raffa | composition rather than inheritance: |
175 | cdea7fa8 | Santi Raffa | |
176 | cdea7fa8 | Santi Raffa | 1. Extract the ``FileStorage`` disk-facing logic into a ``FileDeviceHelper`` |
177 | cdea7fa8 | Santi Raffa | class. |
178 | cdea7fa8 | Santi Raffa | |
179 | cdea7fa8 | Santi Raffa | * In order not to further inflate bdev.py, Filestorage should join its helper |
180 | cdea7fa8 | Santi Raffa | functions in filestorage.py (thus reducing their visibility) and add Gluster |
181 | cdea7fa8 | Santi Raffa | to its own file, gluster.py. Moving the other classes to their own files |
182 | cdea7fa8 | Santi Raffa | like it's been done in ``lib/hypervisor/``) is not addressed as part of this |
183 | cdea7fa8 | Santi Raffa | design. |
184 | cdea7fa8 | Santi Raffa | |
185 | cdea7fa8 | Santi Raffa | 2. Use the ``FileDeviceHelper`` class to implement a ``GlusterStorage`` class in |
186 | cdea7fa8 | Santi Raffa | much the same way. |
187 | cdea7fa8 | Santi Raffa | 3. Add Gluster as a disk template that behaves like SharedFile in every way. |
188 | cdea7fa8 | Santi Raffa | 4. Provide Ganeti knowledge about what a ``GlusterVolume`` is and how to mount, |
189 | cdea7fa8 | Santi Raffa | unmount and reference them. |
190 | cdea7fa8 | Santi Raffa | |
191 | cdea7fa8 | Santi Raffa | * Before attempting a mount, we should check if the volume is not mounted |
192 | cdea7fa8 | Santi Raffa | already. Linux allows mounting partitions multiple times, but then you also |
193 | cdea7fa8 | Santi Raffa | have to unmount them as many times as you mounted them to actually free the |
194 | cdea7fa8 | Santi Raffa | resources; this also makes the output of commands such as ``mount`` less |
195 | cdea7fa8 | Santi Raffa | useful. |
196 | cdea7fa8 | Santi Raffa | * Every time the device could be released (after instance shutdown, OS |
197 | cdea7fa8 | Santi Raffa | installation scripts or file creation), a single unmount is attempted. If |
198 | cdea7fa8 | Santi Raffa | the device is still busy (e.g. from other instances, jobs or open |
199 | cdea7fa8 | Santi Raffa | administrator shells), the failure is ignored. |
200 | cdea7fa8 | Santi Raffa | |
201 | cdea7fa8 | Santi Raffa | 5. Modify ``GlusterStorage`` and customize the disk template behavior to fit |
202 | cdea7fa8 | Santi Raffa | Gluster's needs. |
203 | cdea7fa8 | Santi Raffa | |
204 | cdea7fa8 | Santi Raffa | Directory structure |
205 | cdea7fa8 | Santi Raffa | ~~~~~~~~~~~~~~~~~~~ |
206 | cdea7fa8 | Santi Raffa | |
207 | cdea7fa8 | Santi Raffa | In order to address the shortcomings of the generic shared file handling of |
208 | cdea7fa8 | Santi Raffa | instance disk directory structure, Gluster uses a different scheme for |
209 | cdea7fa8 | Santi Raffa | determining a disk's logical id and therefore path on the file system. |
210 | cdea7fa8 | Santi Raffa | |
211 | cdea7fa8 | Santi Raffa | The naming scheme is:: |
212 | cdea7fa8 | Santi Raffa | |
213 | cdea7fa8 | Santi Raffa | /ganeti/{instance.uuid}.{disk.id} |
214 | cdea7fa8 | Santi Raffa | |
215 | cdea7fa8 | Santi Raffa | ...bringing the actual path on a node's file system to:: |
216 | cdea7fa8 | Santi Raffa | |
217 | cdea7fa8 | Santi Raffa | /var/run/ganeti/gluster/ganeti/{instance.uuid}.{disk.id} |
218 | cdea7fa8 | Santi Raffa | |
219 | cdea7fa8 | Santi Raffa | This means Ganeti only uses one folder on the Gluster volume (allowing other |
220 | cdea7fa8 | Santi Raffa | uses of the Gluster volume in the meantime) and works better with how Gluster |
221 | cdea7fa8 | Santi Raffa | distributes storage over its bricks. |
222 | cdea7fa8 | Santi Raffa | |
223 | cdea7fa8 | Santi Raffa | Changes to the storage types system |
224 | cdea7fa8 | Santi Raffa | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
225 | cdea7fa8 | Santi Raffa | |
226 | cdea7fa8 | Santi Raffa | Ganeti has a number of storage types that abstract over disk templates. This |
227 | cdea7fa8 | Santi Raffa | matters mainly in terms of disk space reporting. Gluster support is improved by |
228 | cdea7fa8 | Santi Raffa | a rethinking of how disk templates are assigned to storage types in Ganeti. |
229 | cdea7fa8 | Santi Raffa | |
230 | cdea7fa8 | Santi Raffa | This is the summary of the changes: |
231 | cdea7fa8 | Santi Raffa | |
232 | cdea7fa8 | Santi Raffa | +--------------+---------+---------+-------------------------------------------+ |
233 | cdea7fa8 | Santi Raffa | | Disk | Current | New | Does it report storage information to... | |
234 | cdea7fa8 | Santi Raffa | | template | storage | storage +-------------+----------------+------------+ |
235 | cdea7fa8 | Santi Raffa | | | type | type | ``gnt-node | ``gnt-node | iallocator | |
236 | cdea7fa8 | Santi Raffa | | | | | list`` | list-storage`` | | |
237 | cdea7fa8 | Santi Raffa | +==============+=========+=========+=============+================+============+ |
238 | cdea7fa8 | Santi Raffa | | File | File | File | Yes. | Yes. | Yes. | |
239 | cdea7fa8 | Santi Raffa | +--------------+---------+---------+-------------+----------------+------------+ |
240 | cdea7fa8 | Santi Raffa | | Shared file | File | Shared | No. | Yes. | No. | |
241 | cdea7fa8 | Santi Raffa | +--------------+---------+ file | | | | |
242 | cdea7fa8 | Santi Raffa | | Gluster (new)| N/A | (new) | | | | |
243 | cdea7fa8 | Santi Raffa | +--------------+---------+---------+-------------+----------------+------------+ |
244 | cdea7fa8 | Santi Raffa | | RBD (for | RBD | No. | No. | No. | |
245 | cdea7fa8 | Santi Raffa | | reference) | | | | | |
246 | cdea7fa8 | Santi Raffa | +--------------+-------------------+-------------+----------------+------------+ |
247 | cdea7fa8 | Santi Raffa | |
248 | cdea7fa8 | Santi Raffa | Gluster or Shared File should not, like RBD, report storage information to |
249 | cdea7fa8 | Santi Raffa | gnt-node list or to IAllocators. Regrettably, the simplest way to do so right |
250 | cdea7fa8 | Santi Raffa | now is by claiming that storage reporting for the relevant storage type is not |
251 | cdea7fa8 | Santi Raffa | implemented. An effort was made to claim that the shared storage type did support |
252 | cdea7fa8 | Santi Raffa | disk reporting while refusing to provide any value, but it was not successful |
253 | cdea7fa8 | Santi Raffa | (``hail`` does not support this combination.) |
254 | cdea7fa8 | Santi Raffa | |
255 | cdea7fa8 | Santi Raffa | To do so without breaking the File disk template, a new storage type must be |
256 | cdea7fa8 | Santi Raffa | added. Like RBD, it does not claim to support disk reporting. However, we can |
257 | cdea7fa8 | Santi Raffa | still make an effort of reporting stats to ``gnt-node list-storage``. |
258 | cdea7fa8 | Santi Raffa | |
259 | cdea7fa8 | Santi Raffa | The rationale is simple. For shared file and gluster storage, disk space is not |
260 | cdea7fa8 | Santi Raffa | a function of any one node. If storage types with disk space reporting are used, |
261 | cdea7fa8 | Santi Raffa | Hail expects them to give useful numbers for allocation purposes, but a shared |
262 | cdea7fa8 | Santi Raffa | storage system means disk balancing is not affected by node-instance allocation |
263 | cdea7fa8 | Santi Raffa | any longer. Moreover, it would be wasteful to mount a Gluster volume on each |
264 | cdea7fa8 | Santi Raffa | node just for running statvfs() if no machine was actually running gluster VMs. |
265 | cdea7fa8 | Santi Raffa | |
266 | cdea7fa8 | Santi Raffa | As a result, Gluster support for gnt-node list-storage is necessarily limited |
267 | cdea7fa8 | Santi Raffa | and nodes on which Gluster is available but not in use will report failures. |
268 | cdea7fa8 | Santi Raffa | Additionally, running ``gnt-node list`` will give an output like this:: |
269 | cdea7fa8 | Santi Raffa | |
270 | cdea7fa8 | Santi Raffa | Node DTotal DFree MTotal MNode MFree Pinst Sinst |
271 | cdea7fa8 | Santi Raffa | node1.example.com ? ? 744M 273M 477M 0 0 |
272 | cdea7fa8 | Santi Raffa | node2.example.com ? ? 744M 273M 477M 0 0 |
273 | cdea7fa8 | Santi Raffa | |
274 | cdea7fa8 | Santi Raffa | This is expected and consistent with behaviour in RBD. |
275 | cdea7fa8 | Santi Raffa | |
276 | cdea7fa8 | Santi Raffa | An alternative would have been to report DTotal and DFree as 0 in order to allow |
277 | cdea7fa8 | Santi Raffa | ``hail`` to ignore the disk information, but this incorrectly populates the |
278 | cdea7fa8 | Santi Raffa | ``gnt-node list`` DTotal and DFree fields with 0s as well. |
279 | cdea7fa8 | Santi Raffa | |
280 | cdea7fa8 | Santi Raffa | New configuration switches |
281 | cdea7fa8 | Santi Raffa | ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
282 | cdea7fa8 | Santi Raffa | |
283 | cdea7fa8 | Santi Raffa | Configurable at the cluster and node group level (``gnt-cluster modify``, |
284 | cdea7fa8 | Santi Raffa | ``gnt-group modify`` and other commands that support the `-D` switch to edit |
285 | cdea7fa8 | Santi Raffa | disk parameters): |
286 | cdea7fa8 | Santi Raffa | |
287 | cdea7fa8 | Santi Raffa | ``gluster:host`` |
288 | cdea7fa8 | Santi Raffa | The IP address or hostname of the Gluster server to connect to. In the default |
289 | cdea7fa8 | Santi Raffa | deployment of Gluster, that is any machine that is hosting a brick. |
290 | cdea7fa8 | Santi Raffa | |
291 | cdea7fa8 | Santi Raffa | Default: ``"127.0.0.1"`` |
292 | cdea7fa8 | Santi Raffa | |
293 | cdea7fa8 | Santi Raffa | ``gluster:port`` |
294 | cdea7fa8 | Santi Raffa | The port where the Gluster server is listening to. |
295 | cdea7fa8 | Santi Raffa | |
296 | cdea7fa8 | Santi Raffa | Default: ``24007`` |
297 | cdea7fa8 | Santi Raffa | |
298 | cdea7fa8 | Santi Raffa | ``gluster:volume`` |
299 | cdea7fa8 | Santi Raffa | The volume Ganeti should use. |
300 | cdea7fa8 | Santi Raffa | |
301 | cdea7fa8 | Santi Raffa | Default: ``"gv0"`` |
302 | cdea7fa8 | Santi Raffa | |
303 | cdea7fa8 | Santi Raffa | Configurable at the cluster level only (``gnt-cluster init``) and stored in |
304 | cdea7fa8 | Santi Raffa | ssconf for all nodes to read (just like shared file): |
305 | cdea7fa8 | Santi Raffa | |
306 | cdea7fa8 | Santi Raffa | ``--gluster-dir`` |
307 | cdea7fa8 | Santi Raffa | Where the Gluster volume should be mounted. |
308 | cdea7fa8 | Santi Raffa | |
309 | cdea7fa8 | Santi Raffa | Default: ``/var/run/ganeti/gluster`` |
310 | cdea7fa8 | Santi Raffa | |
311 | cdea7fa8 | Santi Raffa | The default values work if all of the Ganeti nodes also host Gluster bricks. |
312 | cdea7fa8 | Santi Raffa | This is possible, but `not` recommended as it can cause the host to hardlock due |
313 | cdea7fa8 | Santi Raffa | to deadlocks in the kernel memory (much in the same way RBD works). |
314 | cdea7fa8 | Santi Raffa | |
315 | cdea7fa8 | Santi Raffa | Future work |
316 | cdea7fa8 | Santi Raffa | =========== |
317 | cdea7fa8 | Santi Raffa | |
318 | cdea7fa8 | Santi Raffa | In no particular order: |
319 | cdea7fa8 | Santi Raffa | |
320 | cdea7fa8 | Santi Raffa | * Support the UDP transport. |
321 | cdea7fa8 | Santi Raffa | * Support mounting through NFS. |
322 | cdea7fa8 | Santi Raffa | * Filter ``gnt-node list`` so DTotal and DFree are not shown for RBD and shared |
323 | cdea7fa8 | Santi Raffa | file disk types, or otherwise report the disk storage values as "-" or some |
324 | cdea7fa8 | Santi Raffa | other special value to clearly distinguish it from the result of a |
325 | cdea7fa8 | Santi Raffa | communication failure between nodes. |
326 | cdea7fa8 | Santi Raffa | * Allow configuring the in-volume path Ganeti uses. |
327 | 50119242 | Weiwei Jia | |
328 | 50119242 | Weiwei Jia | .. vim: set textwidth=72 : |
329 | 50119242 | Weiwei Jia | .. Local Variables: |
330 | 50119242 | Weiwei Jia | .. mode: rst |
331 | 50119242 | Weiwei Jia | .. fill-column: 72 |
332 | 50119242 | Weiwei Jia | .. End: |