Revision ca8ac26a doc/design-2.0-cluster-parameters.rst
b/doc/design-2.0-cluster-parameters.rst | ||
---|---|---|
6 | 6 |
Objective |
7 | 7 |
--------- |
8 | 8 |
|
9 |
With the introduction of the HVM hypervisor in Ganeti 1.2 and the |
|
10 |
implementation of multiple hypervisor support in Ganeti 2.0, the way |
|
11 |
the cluster parameters are handled needs to be reorganized accordingly. |
|
9 |
We need to enhance the way attributes for instances and other clusters |
|
10 |
parameters are handled internally within Ganeti in order to have |
|
11 |
better flexibility in the following cases: |
|
12 |
|
|
13 |
- introducting new parameters |
|
14 |
- writing command line interfaces or APIs for these parameters |
|
15 |
- supporting new 2.0 features |
|
12 | 16 |
|
13 | 17 |
Background |
14 | 18 |
---------- |
... | ... | |
19 | 23 |
|
20 | 24 |
As a result of this, wether a particular parameter is valid for the |
21 | 25 |
actual hypervisor could either be guessed from the name but only |
22 |
really checked by following the code using it. |
|
23 |
|
|
24 |
This design doc aims to provide an approach to solve this.
|
|
26 |
really checked by following the code using it. Similar to this case,
|
|
27 |
other parameters are not valid in all cases, and were simply added to |
|
28 |
the top-level instance objects.
|
|
25 | 29 |
|
26 | 30 |
Overview |
27 | 31 |
-------- |
28 | 32 |
|
29 |
The instance level hypervisor parameters will be moved into a separate |
|
30 |
sub-tree of the instance name space. Also, a mechanismum to allow for |
|
31 |
automatic checking and possibly validation of hypervisor parameters |
|
32 |
will be outlined. |
|
33 |
Across all cluster configuration data, we have multiple classes of |
|
34 |
parameters: |
|
33 | 35 |
|
36 |
A. cluster-wide parameters (e.g. name of the cluster, the master); |
|
37 |
these are the ones that we have today, and are unchanged from the |
|
38 |
current model |
|
34 | 39 |
|
35 |
Detailed Design |
|
36 |
--------------- |
|
40 |
#. node parameters |
|
37 | 41 |
|
38 |
Hypervisor Parameter
|
|
39 |
~~~~~~~~~~~~~~~~~~~~
|
|
42 |
#. instance specific parameters, e.g. the name of disks (LV), that
|
|
43 |
cannot be shared with other instances
|
|
40 | 44 |
|
41 |
A hypervisor parameter (or hypervisor specific parameter) is defined |
|
42 |
as a parameter that is interpreted by the hypervisor support code in |
|
43 |
Ganeti and usually is specific to a particular hypervisor (like the |
|
44 |
kernel path for PVM which makes no sense for HVM). |
|
45 |
#. instance parameters, that are or can be the same for many |
|
46 |
instances, but are not hypervisor related; e.g. the number of VCPUs, |
|
47 |
or the size of memory |
|
45 | 48 |
|
46 |
Cluster parameters
|
|
47 |
~~~~~~~~~~~~~~~~~~
|
|
49 |
#. instance parameters that are hypervisor specific (e.g. kernel_path
|
|
50 |
or PAE mode)
|
|
48 | 51 |
|
49 |
The cluster parameter namespace will be extended with cluster level |
|
50 |
hypervisor specific parameters. The typical expected use case for this |
|
51 |
is to store default values for instance level hypervisor parameters. |
|
52 | 52 |
|
53 | 53 |
|
54 |
Instance parameters
|
|
55 |
~~~~~~~~~~~~~~~~~~~
|
|
54 |
Detailed Design
|
|
55 |
---------------
|
|
56 | 56 |
|
57 |
The only hypervisor parameter to remain at the top level of the |
|
58 |
instance namespace will be instance.hypervisor, specifying not |
|
59 |
only the hypervisor type to be used for that instance, but also |
|
60 |
implicitly the hypervisor type to use for parameter checks. |
|
57 |
The following definitions for instance parameters will be used below: |
|
61 | 58 |
|
62 |
All other instance level hypervisor parameters will be moved into the |
|
63 |
instance.hvparams namespace subtree. |
|
59 |
hypervisor parameter |
|
60 |
a hypervisor parameter (or hypervisor specific parameter) is defined |
|
61 |
as a parameter that is interpreted by the hypervisor support code in |
|
62 |
Ganeti and usually is specific to a particular hypervisor (like the |
|
63 |
kernel path for PVM which makes no sense for HVM). |
|
64 | 64 |
|
65 |
The names for hypervisor parameters in the instance.hvparams |
|
66 |
subtree should be choosen as generic as possible, especially if |
|
67 |
specific parameters could conceivably be useful for more than one |
|
68 |
hypervisor, e.g. |
|
69 |
instance.hvparams.vnc_console_port instead of using both |
|
70 |
instance.hvparams.hvm_vnc_console_port and |
|
71 |
instance.hvparams.kvm_vnc_console_port. |
|
65 |
backend parameter |
|
66 |
a backend parameter is defined as an instance parameter that can be |
|
67 |
shared among a list of instances, and is either generic enough not |
|
68 |
to be tied to a given hypervisor or cannot influence at all the |
|
69 |
hypervisor behaviour |
|
72 | 70 |
|
73 |
The instance.hvparams subtree will be implemented as a dict. |
|
71 |
proper parameter |
|
72 |
a parameter whose value is unique to the instance (e.g. the name of a LV, |
|
73 |
or the MAC of a NIC) |
|
74 | 74 |
|
75 |
Examples for instance level hypervisor parameters: |
|
75 |
As a general rule, for all kind of parameters, “None” (or in |
|
76 |
JSON-speak, “nil”) will no longer be a valid value for a parameter. As |
|
77 |
such, only non-default parameters will be saved as part of objects in |
|
78 |
the serialization step, reducing the size of the serialized format. |
|
76 | 79 |
|
77 |
:boot_order: boot device order |
|
78 |
:vnc_bind_address: bind address for VNC console |
|
79 |
:vnc_bind_port: bind port for VNC console |
|
80 |
:kernel_path: path to the kernel binary |
|
81 |
:disk_type: type of the virtual disk interface |
|
80 |
Cluster parameters |
|
81 |
~~~~~~~~~~~~~~~~~~ |
|
82 |
|
|
83 |
Cluster parameters remain as today, attributes at the top level of the |
|
84 |
Cluster object. In addition, two new attributes at this level will |
|
85 |
hold defaults for the instances: |
|
82 | 86 |
|
83 |
With this design, hypervisor specific parameters for disk and NIC |
|
84 |
devices are defined globally for all devices of that type in the instance. |
|
85 |
It may become necessary later to allow setting these parameters on a |
|
86 |
per device basis, which will require design changes to the cluster |
|
87 |
parameters. |
|
87 |
- hvparams, a dictionary indexed by hypervisor type, holding default |
|
88 |
values for hypervisor parameters that are not defined/overrided by |
|
89 |
the instances of this hypervisor type |
|
88 | 90 |
|
89 |
All other, non hypervisor related parameters of the instance remain at
|
|
90 |
the top level of the instance namespace. This includes:
|
|
91 |
- beparams, a dictionary holding (for 2.0) a single element 'default',
|
|
92 |
which holds the default value for backend parameters
|
|
91 | 93 |
|
92 |
- disks |
|
93 |
- nics |
|
94 |
- mem |
|
95 |
- vcpu |
|
96 |
- status |
|
97 |
- OS |
|
94 |
Node parameters |
|
95 |
~~~~~~~~~~~~~~~ |
|
98 | 96 |
|
97 |
Node-related parameters are very few, and we will continue using the |
|
98 |
same model for these as previously (attributes on the Node object). |
|
99 | 99 |
|
100 |
Instance parameters |
|
101 |
~~~~~~~~~~~~~~~~~~~ |
|
100 | 102 |
|
101 |
Hypervisor support requirements |
|
102 |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
103 |
As described before, the instance parameters are split in three: |
|
104 |
instance proper parameters, unique to each instance, instance |
|
105 |
hypervisor parameters and instance backend parameters. |
|
106 |
|
|
107 |
The “hvparams” and “beparams” are kept in two dictionaries at instance |
|
108 |
level. Only non-default parameters are stored (but once customized, a |
|
109 |
parameter will be kept, even with the same value as the default one, |
|
110 |
until reset). |
|
111 |
|
|
112 |
The names for hypervisor parameters in the instance.hvparams subtree |
|
113 |
should be choosen as generic as possible, especially if specific |
|
114 |
parameters could conceivably be useful for more than one hypervisor, |
|
115 |
e.g. instance.hvparams.vnc_console_port instead of using both |
|
116 |
instance.hvparams.hvm_vnc_console_port and |
|
117 |
instance.hvparams.kvm_vnc_console_port. |
|
118 |
|
|
119 |
There are some special cases related to disks and NICs (for example): |
|
120 |
a disk has both ganeti-related parameters (e.g. the name of the LV) |
|
121 |
and hypervisor-related parameters (how the disk is presented to/named |
|
122 |
in the instance). The former parameters remain as proper-instance |
|
123 |
parameters, while the latter value are migrated to the hvparams |
|
124 |
structure. In 2.0, we will have only globally-per-instance such |
|
125 |
hypervisor parameters, and not per-disk ones (e.g. all NICs will be |
|
126 |
exported as of the same type). |
|
127 |
|
|
128 |
Starting from the 1.2 list of instance parameters, here is how they |
|
129 |
will be mapped to the three classes of parameters: |
|
130 |
|
|
131 |
- name (P) |
|
132 |
- primary_node (P) |
|
133 |
- os (P) |
|
134 |
- hypervisor (P) |
|
135 |
- status (P) |
|
136 |
- memory (BE) |
|
137 |
- vcpus (BE) |
|
138 |
- nics (P) |
|
139 |
- disks (P) |
|
140 |
- disk_template (P) |
|
141 |
- network_port (P) |
|
142 |
- kernel_path (HV) |
|
143 |
- initrd_path (HV) |
|
144 |
- hvm_boot_order (HV) |
|
145 |
- hvm_acpi (HV) |
|
146 |
- hvm_pae (HV) |
|
147 |
- hvm_cdrom_image_path (HV) |
|
148 |
- hvm_nic_type (HV) |
|
149 |
- hvm_disk_type (HV) |
|
150 |
- vnc_bind_address (HV) |
|
151 |
- serial_no (P) |
|
152 |
|
|
153 |
|
|
154 |
Parameter validation |
|
155 |
~~~~~~~~~~~~~~~~~~~~ |
|
103 | 156 |
|
104 | 157 |
To support the new cluster parameter design, additional features will |
105 | 158 |
be required from the hypervisor support implementations in Ganeti. |
... | ... | |
107 | 160 |
The hypervisor support implementation API will be extended with the |
108 | 161 |
following features: |
109 | 162 |
|
110 |
:GetValidClusterParameters(): returns a list of valid cluster level |
|
111 |
parameters for this hypervisor, |
|
112 |
:GetValidInstanceParameters(): provide a list of valid instance level |
|
113 |
parameters for this hypervisor, |
|
114 |
:VerifyParameter(parameter_name, parameter_value): verifies the |
|
115 |
provided parameter against this hypervisor: |
|
116 |
|
|
117 |
- if the provided parameter name is valid for this hypervisor |
|
118 |
- if applicable, if the provided value is valid for this hypervisor |
|
119 |
|
|
120 |
The VerifyParameter(..) function will return True or False. |
|
163 |
:PARAMETERS: class-level attribute holding the list of valid parameters |
|
164 |
for this hypervisor |
|
165 |
:CheckParamSyntax(hvparams): checks that the given parameters are |
|
166 |
valid (as in the names are valid) for this hypervisor; usually just |
|
167 |
comparing hvparams.keys() and cls.PARAMETERS; this is a class method |
|
168 |
that can be called from within master code (i.e. cmdlib) and should |
|
169 |
be safe to do so |
|
170 |
:ValidateParameters(hvparams): verifies the values of the provided |
|
171 |
parameters against this hypervisor; this is a method that will be |
|
172 |
called on the target node, from backend.py code, and as such can |
|
173 |
make node-specific checks (e.g. kernel_path checking) |
|
174 |
|
|
175 |
Default value application |
|
176 |
~~~~~~~~~~~~~~~~~~~~~~~~~ |
|
177 |
|
|
178 |
The application of defaults to an instance is done in the Cluster |
|
179 |
object, via two new methods as follows: |
|
180 |
|
|
181 |
- ``Cluster.FillHV(instance)``, returns 'filled' hvparams dict, based on |
|
182 |
instance's hvparams and cluster's ``hvparams[instance.hypervisor]`` |
|
183 |
|
|
184 |
- ``Cluster.FillBE(instance, be_type="default")``, which returns the |
|
185 |
beparams dict, based on the instance and cluster beparams |
|
186 |
|
|
187 |
The FillHV/BE transformations will be used, for example, in the RpcRunner |
|
188 |
when sending an instance for activation/stop, and the sent instance |
|
189 |
hvparams/beparams will have the final value (noded code doesn't know |
|
190 |
about defaults). |
|
191 |
|
|
192 |
LU code will need to self-call the transformation, if needed. |
|
193 |
|
|
194 |
Opcode changes |
|
195 |
~~~~~~~~~~~~~~ |
|
196 |
|
|
197 |
The parameter changes will have impact on the OpCodes, especially on |
|
198 |
the following ones: |
|
199 |
|
|
200 |
- OpCreateInstance, where the new hv and be parameters will be sent as |
|
201 |
dictionaries; note that all hv and be parameters are now optional, as |
|
202 |
the values can be instead taken from the cluster |
|
203 |
- OpQueryInstances, where we have to be able to query these new |
|
204 |
parameters; the syntax for names will be ``hvparam/$NAME`` and |
|
205 |
``beparam/$NAME`` for querying an individual parameter out of one |
|
206 |
dictionary, and ``hvparams``, respectively ``beparams``, for the whole |
|
207 |
dictionaries |
|
208 |
- OpModifyInstance, where the the modified parameters are sent as |
|
209 |
dictionaries |
|
210 |
|
|
211 |
Additionally, we will need new OpCodes to modify the cluster-level |
|
212 |
defaults for the be/hv sets of parameters. |
|
213 |
|
|
214 |
Caveats |
|
215 |
------- |
|
216 |
|
|
217 |
One problem that might appear is that our classification is not |
|
218 |
complete or not good enough, and we'll need to change this model. As |
|
219 |
the last resort, we will need to rollback and keep 1.2 style. |
|
220 |
|
|
221 |
Another problem is that classification of one parameter is unclear |
|
222 |
(e.g. ``network_port``, is this BE or HV?); in this case we'll take |
|
223 |
the risk of having to move parameters later between classes. |
|
224 |
|
|
225 |
Security |
|
226 |
-------- |
|
121 | 227 |
|
228 |
The only security issue that we foresee is if some new parameters will |
|
229 |
have sensitive value. If so, we will need to have a way to export the |
|
230 |
config data while purging the sensitive value. |
|
122 | 231 |
|
123 |
The parameter validation will be used for every addition or change of |
|
124 |
hypervisor parameters. |
|
232 |
E.g. for the drbd shared secrets, we could export these with the |
|
233 |
values replaced by an empty string. |
Also available in: Unified diff