|
1 |
======================
|
|
2 |
Query version 2 design
|
|
3 |
======================
|
|
4 |
|
|
5 |
.. contents:: :depth: 4
|
|
6 |
.. highlight:: python
|
|
7 |
|
|
8 |
Current state and shortcomings
|
|
9 |
==============================
|
|
10 |
|
|
11 |
Queries are used to retrieve information about the cluster, e.g. a list
|
|
12 |
of instances or nodes. For historical reasons they use a simple data
|
|
13 |
structure for their result. The client submits the fields it would like
|
|
14 |
to receive and the query returns a list for each item (instance, node,
|
|
15 |
etc.) available. Each item consists of another list representing the
|
|
16 |
fields' values.
|
|
17 |
|
|
18 |
This data structure has a few drawbacks. It can't associate a status
|
|
19 |
(e.g. “node offline”) with fields as using special values can lead to
|
|
20 |
ambiguities. Additionally it can't mark fields as “not found” as the
|
|
21 |
list of returned columns must match the fields requested.
|
|
22 |
|
|
23 |
Example::
|
|
24 |
|
|
25 |
>>> cli.GetClient().QueryNodes([], ["name", "pip", "mfree"], False)
|
|
26 |
[
|
|
27 |
['node1.example.com', '192.0.2.18', 14800],
|
|
28 |
['node2.example.com', '192.0.2.19', 31280]
|
|
29 |
]
|
|
30 |
|
|
31 |
There is no way for clients to determine the list of possible fields,
|
|
32 |
meaning they have to be hardcoded. Selecting unknown fields raises
|
|
33 |
an exception::
|
|
34 |
|
|
35 |
>>> cli.GetClient().QueryNodes([], ["name", "UnknownField"], False)
|
|
36 |
ganeti.errors.OpPrereqError: (u'Unknown output fields selected: UnknownField', u'wrong_input')
|
|
37 |
|
|
38 |
The client must also know each fields' kind, that is whether a field is
|
|
39 |
numeric, boolean, describes a storage size, etc. Centralizing this
|
|
40 |
information in one place, the master daemon, is desirable.
|
|
41 |
|
|
42 |
|
|
43 |
Proposed changes
|
|
44 |
----------------
|
|
45 |
|
|
46 |
The current query result format can not be changed as it's being used in
|
|
47 |
various places. Changing the format from one Ganeti version to another
|
|
48 |
would cause too much disruption. For this reason the ability to
|
|
49 |
explicitly request a new result format must be added while the old
|
|
50 |
format stays the default.
|
|
51 |
|
|
52 |
The implementation of query filters is planned for the future. To avoid
|
|
53 |
having to change the calls again, a (hopefully) future-compatible
|
|
54 |
interface will be implemented now.
|
|
55 |
|
|
56 |
In Python code, the objects described below will be implemented using
|
|
57 |
subclasses of ``objects.ConfigObject``, providing existing facilities
|
|
58 |
for de-/serializing.
|
|
59 |
|
|
60 |
.. _query-request:
|
|
61 |
|
|
62 |
Query request
|
|
63 |
+++++++++++++
|
|
64 |
|
|
65 |
Each query operation will take a single parameter, a query request
|
|
66 |
dictionary with the following properties:
|
|
67 |
|
|
68 |
``kind``
|
|
69 |
Denotes request kind. One of the following strings:
|
|
70 |
|
|
71 |
``query``
|
|
72 |
Execute a query on items, optionally filtered (see below). For a
|
|
73 |
description of the result see :ref:`query-result`.
|
|
74 |
``fields``
|
|
75 |
Return list of supported fields as :ref:`field definitions
|
|
76 |
<field-def>`. For a complete description of the result see
|
|
77 |
:ref:`fields-result`.
|
|
78 |
|
|
79 |
``filter``
|
|
80 |
This will be used to filter queries. In this implementation only names
|
|
81 |
can be filtered to replace the previous ``names`` parameter to
|
|
82 |
queries. An empty filter (``None``) will return all items. To retrieve
|
|
83 |
specific names, the filter must be specified as follows, with the
|
|
84 |
inner part repeated for each name::
|
|
85 |
|
|
86 |
["|", ["=", "name", "node1"], ["=", "name", "node2"], …]
|
|
87 |
|
|
88 |
Filters consist of S-expressions (``["operator", <operants…>]``) and
|
|
89 |
extensions will be made in the future to allow for more operators and
|
|
90 |
fields. Such extensions might include a Python-style "in" operator,
|
|
91 |
but for simplicity only "=" is supported in this implementation.
|
|
92 |
|
|
93 |
To reiterate: Filters for this implementation must consist of exactly
|
|
94 |
one OR expression (``["|", …]``) and one or more name equality filters
|
|
95 |
(``["=", "name", "…"]``).
|
|
96 |
|
|
97 |
Support for synchronous queries, currently available in the interface
|
|
98 |
but disabled in the master daemon, will be dropped. Direct calls to
|
|
99 |
opcodes have to be used instead.
|
|
100 |
|
|
101 |
.. _query-result:
|
|
102 |
|
|
103 |
New query result format
|
|
104 |
+++++++++++++++++++++++
|
|
105 |
|
|
106 |
The result is a dictionary with the following entries:
|
|
107 |
|
|
108 |
``fields``
|
|
109 |
In-order list of a :ref:`field definition <field-def>` for each
|
|
110 |
requested field, unknown fields are returned with the kind
|
|
111 |
``unknown``. Length must be equal to number of requested fields.
|
|
112 |
``data``
|
|
113 |
A list of lists, one list for each item found. Each item's list must
|
|
114 |
have one entry for each field listed in ``fields``. Each field entry
|
|
115 |
is a tuple of ``(status, value)``. ``status`` must be one of the
|
|
116 |
following values:
|
|
117 |
|
|
118 |
Normal (numeric 0)
|
|
119 |
Value is available and matches the kind in the :ref:`field
|
|
120 |
definition <field-def>`.
|
|
121 |
Value unavailable (numeric 1)
|
|
122 |
Exact meaning depends on query, e.g. node is unreachable or marked
|
|
123 |
offline. Value must be ``None``.
|
|
124 |
Unknown field (numeric 2)
|
|
125 |
Field for this column is not known. Value must be ``None``.
|
|
126 |
|
|
127 |
Example::
|
|
128 |
|
|
129 |
{
|
|
130 |
"fields": [
|
|
131 |
{ "name": "name", "title": "Name", "kind": "text", },
|
|
132 |
{ "name": "mfree", "title": "MemFree", "kind": "unit", },
|
|
133 |
# Unknown field
|
|
134 |
{ "name": "xyz", "title": None, "kind": "unknown", },
|
|
135 |
{ "name": "mtotal", "title": "MemTotal", "kind": "unit", },
|
|
136 |
],
|
|
137 |
|
|
138 |
"data": [
|
|
139 |
[(0, "node1"), (0, 128), (2, None), (0, 4096)],
|
|
140 |
# Node not available
|
|
141 |
[(0, "node2"), (1, None), (2, None), (1, None)],
|
|
142 |
],
|
|
143 |
}
|
|
144 |
|
|
145 |
.. _fields-result:
|
|
146 |
|
|
147 |
Field query result format
|
|
148 |
+++++++++++++++++++++++++
|
|
149 |
|
|
150 |
The result is a dictionary with the following entries:
|
|
151 |
|
|
152 |
``fields``
|
|
153 |
List of :ref:`field definitions <field-def>` for each available field.
|
|
154 |
|
|
155 |
Example::
|
|
156 |
|
|
157 |
{
|
|
158 |
"fields": [
|
|
159 |
{ "name": "name", "title": "Name", "kind": "text", },
|
|
160 |
{ "name": "mfree", "title": "MemFree", "kind": "unit", },
|
|
161 |
{ "name": "mtotal", "title": "MemTotal", "kind": "unit", },
|
|
162 |
]
|
|
163 |
}
|
|
164 |
|
|
165 |
.. _field-def:
|
|
166 |
|
|
167 |
Field definition
|
|
168 |
++++++++++++++++
|
|
169 |
|
|
170 |
A field definition is a dictionary with the following entries:
|
|
171 |
|
|
172 |
``name``
|
|
173 |
The field name as a regular expression. The latter is necessary to
|
|
174 |
represent dynamic fields (e.g. NIC 3 of an instance).
|
|
175 |
``title``
|
|
176 |
Human-readable title to use in output. Must not contain whitespace.
|
|
177 |
``kind``
|
|
178 |
Field type, one of the following:
|
|
179 |
|
|
180 |
.. TODO: Investigate whether there are fields with floating point
|
|
181 |
.. numbers
|
|
182 |
|
|
183 |
``unknown``
|
|
184 |
Unknown field (only used for :ref:`data queries <query-request>`)
|
|
185 |
``text``
|
|
186 |
String
|
|
187 |
``bool``
|
|
188 |
Boolean, true/false
|
|
189 |
``number``
|
|
190 |
Numeric
|
|
191 |
``unit``
|
|
192 |
Numeric, in megabytes
|
|
193 |
``other``
|
|
194 |
Free-form type, depending on query
|
|
195 |
|
|
196 |
More types can be added in the future, so clients should default to
|
|
197 |
formatting any unknown types the same way as "other", which should be
|
|
198 |
a string representation in most cases.
|
|
199 |
|
|
200 |
Example 1 (item name)::
|
|
201 |
|
|
202 |
{
|
|
203 |
"name": "name",
|
|
204 |
"title": "Name",
|
|
205 |
"kind": "text",
|
|
206 |
}
|
|
207 |
|
|
208 |
Example 2 (free memory)::
|
|
209 |
|
|
210 |
{
|
|
211 |
"name": "mfree",
|
|
212 |
"title": "MemFree",
|
|
213 |
"kind": "unit",
|
|
214 |
}
|
|
215 |
|
|
216 |
Example 3 (list of primary instances)::
|
|
217 |
|
|
218 |
{
|
|
219 |
"name": "pinst",
|
|
220 |
"title": "PrimaryInstances",
|
|
221 |
"kind": "other",
|
|
222 |
}
|
|
223 |
|
|
224 |
.. _old-result-format:
|
|
225 |
|
|
226 |
Old result format
|
|
227 |
+++++++++++++++++
|
|
228 |
|
|
229 |
To limit the amount of code necessary, the :ref:`new result format
|
|
230 |
<query-result>` will be converted for older clients. Unavailable values
|
|
231 |
are set to ``None``. If unknown fields were requested, the whole query
|
|
232 |
fails as the client expects exactly the fields it requested.
|
|
233 |
|
|
234 |
LUXI
|
|
235 |
++++
|
|
236 |
|
|
237 |
Currently query calls take a number of parameters, e.g. names, fields
|
|
238 |
and whether to use locking. These will continue to work and return the
|
|
239 |
:ref:`old result format <old-result-format>`. To use the new query
|
|
240 |
requests, the same calls must be invoked with a single parameter as the
|
|
241 |
:ref:`query object <query-request>`. Only clients using the new call
|
|
242 |
syntax will be able to make use of new features such as filters.
|
|
243 |
|
|
244 |
Python
|
|
245 |
++++++
|
|
246 |
|
|
247 |
The LUXI API is more or less mapped directly into Python. In addition to
|
|
248 |
the existing stub functions new ones will be added for the new query
|
|
249 |
requests.
|
|
250 |
|
|
251 |
RAPI
|
|
252 |
++++
|
|
253 |
|
|
254 |
The RAPI interface already returns dictionaries for each item, but to
|
|
255 |
not break compatibility no changes should be made to the structure (e.g.
|
|
256 |
to include field definitions). The proposal here is to add a new
|
|
257 |
parameter to allow clients to execute the requests described in this
|
|
258 |
proposal directly and to receive the unmodified result. The new formats
|
|
259 |
are a lot more verbose, flexible and extensible.
|
|
260 |
|
|
261 |
|
|
262 |
Other discussed solutions
|
|
263 |
-------------------------
|
|
264 |
|
|
265 |
Another solution discussed was to add an additional column for each
|
|
266 |
non-static field containing the status. Clients interested in the status
|
|
267 |
could explicitely query for it.
|
|
268 |
|
|
269 |
.. vim: set textwidth=72 :
|
|
270 |
.. Local Variables:
|
|
271 |
.. mode: rst
|
|
272 |
.. fill-column: 72
|
|
273 |
.. End:
|