root / doc / design-query2.rst @ e5a246df
History | View | Annotate | Download (8.3 kB)
1 |
====================== |
---|---|
2 |
Query version 2 design |
3 |
====================== |
4 |
|
5 |
.. contents:: :depth: 4 |
6 |
.. highlight:: python |
7 |
|
8 |
Current state and shortcomings |
9 |
============================== |
10 |
|
11 |
Queries are used to retrieve information about the cluster, e.g. a list |
12 |
of instances or nodes. For historical reasons they use a simple data |
13 |
structure for their result. The client submits the fields it would like |
14 |
to receive and the query returns a list for each item (instance, node, |
15 |
etc.) available. Each item consists of another list representing the |
16 |
fields' values. |
17 |
|
18 |
This data structure has a few drawbacks. It can't associate a status |
19 |
(e.g. “node offline”) with fields as using special values can lead to |
20 |
ambiguities. Additionally it can't mark fields as “not found” as the |
21 |
list of returned columns must match the fields requested. |
22 |
|
23 |
Example:: |
24 |
|
25 |
>>> cli.GetClient().QueryNodes([], ["name", "pip", "mfree"], False) |
26 |
[ |
27 |
['node1.example.com', '192.0.2.18', 14800], |
28 |
['node2.example.com', '192.0.2.19', 31280] |
29 |
] |
30 |
|
31 |
There is no way for clients to determine the list of possible fields, |
32 |
meaning they have to be hardcoded. Selecting unknown fields raises |
33 |
an exception:: |
34 |
|
35 |
>>> cli.GetClient().QueryNodes([], ["name", "UnknownField"], False) |
36 |
ganeti.errors.OpPrereqError: (u'Unknown output fields selected: UnknownField', u'wrong_input') |
37 |
|
38 |
The client must also know each fields' kind, that is whether a field is |
39 |
numeric, boolean, describes a storage size, etc. Centralizing this |
40 |
information in one place, the master daemon, is desirable. |
41 |
|
42 |
|
43 |
Proposed changes |
44 |
---------------- |
45 |
|
46 |
The current query result format can not be changed as it's being used in |
47 |
various places. Changing the format from one Ganeti version to another |
48 |
would cause too much disruption. For this reason the ability to |
49 |
explicitly request a new result format must be added while the old |
50 |
format stays the default. |
51 |
|
52 |
The implementation of query filters is planned for the future. To avoid |
53 |
having to change the calls again, a (hopefully) future-compatible |
54 |
interface will be implemented now. |
55 |
|
56 |
In Python code, the objects described below will be implemented using |
57 |
subclasses of ``objects.ConfigObject``, providing existing facilities |
58 |
for de-/serializing. |
59 |
|
60 |
.. _query-request: |
61 |
|
62 |
Query request |
63 |
+++++++++++++ |
64 |
|
65 |
Each query operation will take a single parameter, a query request |
66 |
dictionary with the following properties: |
67 |
|
68 |
``kind`` |
69 |
Denotes request kind. One of the following strings: |
70 |
|
71 |
``query`` |
72 |
Execute a query on items, optionally filtered (see below). For a |
73 |
description of the result see :ref:`query-result`. |
74 |
``fields`` |
75 |
Return list of supported fields as :ref:`field definitions |
76 |
<field-def>`. For a complete description of the result see |
77 |
:ref:`fields-result`. |
78 |
|
79 |
``filter`` |
80 |
This will be used to filter queries. In this implementation only names |
81 |
can be filtered to replace the previous ``names`` parameter to |
82 |
queries. An empty filter (``None``) will return all items. To retrieve |
83 |
specific names, the filter must be specified as follows, with the |
84 |
inner part repeated for each name:: |
85 |
|
86 |
["|", ["=", "name", "node1"], ["=", "name", "node2"], …] |
87 |
|
88 |
Filters consist of S-expressions (``["operator", <operants…>]``) and |
89 |
extensions will be made in the future to allow for more operators and |
90 |
fields. Such extensions might include a Python-style "in" operator, |
91 |
but for simplicity only "=" is supported in this implementation. |
92 |
|
93 |
To reiterate: Filters for this implementation must consist of exactly |
94 |
one OR expression (``["|", …]``) and one or more name equality filters |
95 |
(``["=", "name", "…"]``). |
96 |
|
97 |
Support for synchronous queries, currently available in the interface |
98 |
but disabled in the master daemon, will be dropped. Direct calls to |
99 |
opcodes have to be used instead. |
100 |
|
101 |
.. _query-result: |
102 |
|
103 |
New query result format |
104 |
+++++++++++++++++++++++ |
105 |
|
106 |
The result is a dictionary with the following entries: |
107 |
|
108 |
``fields`` |
109 |
In-order list of a :ref:`field definition <field-def>` for each |
110 |
requested field, unknown fields are returned with the kind |
111 |
``unknown``. Length must be equal to number of requested fields. |
112 |
``data`` |
113 |
A list of lists, one list for each item found. Each item's list must |
114 |
have one entry for each field listed in ``fields``. Each field entry |
115 |
is a tuple of ``(status, value)``. ``status`` must be one of the |
116 |
following values: |
117 |
|
118 |
Normal (numeric 0) |
119 |
Value is available and matches the kind in the :ref:`field |
120 |
definition <field-def>`. |
121 |
Value unavailable (numeric 1) |
122 |
Exact meaning depends on query, e.g. node is unreachable or marked |
123 |
offline. Value must be ``None``. |
124 |
Unknown field (numeric 2) |
125 |
Field for this column is not known. Value must be ``None``. |
126 |
|
127 |
Example:: |
128 |
|
129 |
{ |
130 |
"fields": [ |
131 |
{ "name": "name", "title": "Name", "kind": "text", }, |
132 |
{ "name": "mfree", "title": "MemFree", "kind": "unit", }, |
133 |
# Unknown field |
134 |
{ "name": "xyz", "title": None, "kind": "unknown", }, |
135 |
{ "name": "mtotal", "title": "MemTotal", "kind": "unit", }, |
136 |
], |
137 |
|
138 |
"data": [ |
139 |
[(0, "node1"), (0, 128), (2, None), (0, 4096)], |
140 |
# Node not available |
141 |
[(0, "node2"), (1, None), (2, None), (1, None)], |
142 |
], |
143 |
} |
144 |
|
145 |
.. _fields-result: |
146 |
|
147 |
Field query result format |
148 |
+++++++++++++++++++++++++ |
149 |
|
150 |
The result is a dictionary with the following entries: |
151 |
|
152 |
``fields`` |
153 |
List of :ref:`field definitions <field-def>` for each available field. |
154 |
|
155 |
Example:: |
156 |
|
157 |
{ |
158 |
"fields": [ |
159 |
{ "name": "name", "title": "Name", "kind": "text", }, |
160 |
{ "name": "mfree", "title": "MemFree", "kind": "unit", }, |
161 |
{ "name": "mtotal", "title": "MemTotal", "kind": "unit", }, |
162 |
] |
163 |
} |
164 |
|
165 |
.. _field-def: |
166 |
|
167 |
Field definition |
168 |
++++++++++++++++ |
169 |
|
170 |
A field definition is a dictionary with the following entries: |
171 |
|
172 |
``name`` |
173 |
The field name as a regular expression. The latter is necessary to |
174 |
represent dynamic fields (e.g. NIC 3 of an instance). |
175 |
``title`` |
176 |
Human-readable title to use in output. Must not contain whitespace. |
177 |
``kind`` |
178 |
Field type, one of the following: |
179 |
|
180 |
.. TODO: Investigate whether there are fields with floating point |
181 |
.. numbers |
182 |
|
183 |
``unknown`` |
184 |
Unknown field (only used for :ref:`data queries <query-request>`) |
185 |
``text`` |
186 |
String |
187 |
``bool`` |
188 |
Boolean, true/false |
189 |
``number`` |
190 |
Numeric |
191 |
``unit`` |
192 |
Numeric, in megabytes |
193 |
``other`` |
194 |
Free-form type, depending on query |
195 |
|
196 |
More types can be added in the future, so clients should default to |
197 |
formatting any unknown types the same way as "other", which should be |
198 |
a string representation in most cases. |
199 |
|
200 |
Example 1 (item name):: |
201 |
|
202 |
{ |
203 |
"name": "name", |
204 |
"title": "Name", |
205 |
"kind": "text", |
206 |
} |
207 |
|
208 |
Example 2 (free memory):: |
209 |
|
210 |
{ |
211 |
"name": "mfree", |
212 |
"title": "MemFree", |
213 |
"kind": "unit", |
214 |
} |
215 |
|
216 |
Example 3 (list of primary instances):: |
217 |
|
218 |
{ |
219 |
"name": "pinst", |
220 |
"title": "PrimaryInstances", |
221 |
"kind": "other", |
222 |
} |
223 |
|
224 |
.. _old-result-format: |
225 |
|
226 |
Old result format |
227 |
+++++++++++++++++ |
228 |
|
229 |
To limit the amount of code necessary, the :ref:`new result format |
230 |
<query-result>` will be converted for older clients. Unavailable values |
231 |
are set to ``None``. If unknown fields were requested, the whole query |
232 |
fails as the client expects exactly the fields it requested. |
233 |
|
234 |
LUXI |
235 |
++++ |
236 |
|
237 |
Currently query calls take a number of parameters, e.g. names, fields |
238 |
and whether to use locking. These will continue to work and return the |
239 |
:ref:`old result format <old-result-format>`. To use the new query |
240 |
requests, the same calls must be invoked with a single parameter as the |
241 |
:ref:`query object <query-request>`. Only clients using the new call |
242 |
syntax will be able to make use of new features such as filters. |
243 |
|
244 |
Python |
245 |
++++++ |
246 |
|
247 |
The LUXI API is more or less mapped directly into Python. In addition to |
248 |
the existing stub functions new ones will be added for the new query |
249 |
requests. |
250 |
|
251 |
RAPI |
252 |
++++ |
253 |
|
254 |
The RAPI interface already returns dictionaries for each item, but to |
255 |
not break compatibility no changes should be made to the structure (e.g. |
256 |
to include field definitions). The proposal here is to add a new |
257 |
parameter to allow clients to execute the requests described in this |
258 |
proposal directly and to receive the unmodified result. The new formats |
259 |
are a lot more verbose, flexible and extensible. |
260 |
|
261 |
|
262 |
Other discussed solutions |
263 |
------------------------- |
264 |
|
265 |
Another solution discussed was to add an additional column for each |
266 |
non-static field containing the status. Clients interested in the status |
267 |
could explicitely query for it. |
268 |
|
269 |
.. vim: set textwidth=72 : |
270 |
.. Local Variables: |
271 |
.. mode: rst |
272 |
.. fill-column: 72 |
273 |
.. End: |