History | View | Annotate | Download (15.2 kB)
Support 'null' in Luxi QueryJobs names field
Python code sometimes sends this, so let's support it even though it'snon-standard.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Jobs.hs: move OpStatus and JobStatus ADTs to Types.hs
This leaves Ganeti/Jobs.hs and Test/Ganeti/Jobs.hs empty, but they're thetarget of a future move of some functions, so we leave them around, anddon't delete them, to avoid unnecessary delete/create diffs....
Switch luxi submit job calls to use MetaOpCode
This patch changes the luxi submit job calls to use wrapped opcodes,and therefore it changes Hbal to submit actual meta opcodes. Fornicety, hbal also submits a comment now, showing who generated thejob.
Signed-off-by: Iustin Pop <iustin@google.com>...
Rework custom fields handling
This patch changes a bit the handling of custom fields. Since ingeneral we use custom fields to aggregate multiple entries in the JSONobject into a safer data-type, we should also have a way to declarewhich extra entries this field covers (so that in the future we can...
Add a 'real' type for JobIds
Currently, the job ID is a simple type alias. This is suboptimal, asit means we can't use a custom JSON (or Arbitrary) instance for it.
The patch changes it into a newtype, and then a) simplifies somedeserialisation code and b) changes some more fields to this new type...
Remove read instances from our Haskell code
It turns out that optimising 'read' derived instances (via -O) forcomplex data types (like OpCode, or the various objects) can be slowto very slow. Disabling such instances results in (time make$all_our_haskell_binaries) large compile-time savings and also smaller...
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Convert tag objects to a safer type
Currently, we keep information about the "target" of a tag operationin a data type similar to (TagKind, Maybe String). This is unsafe, asnothing (at the type level) prevents us from accidentally having(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing)....
Switch Luxi TH code from simple to custom fields
This is needed so that we have more flexibility in generating Luxiserialisation code (deserialisation is still custom). Also, onlyexceptions are now using the 'simple' field types, so we might be ablelater to convert and remove that TH code as well....
htools: move TagObject from Luxi.hs to OpCodes.hs
In addition to ReqQueryTags in Luxi.hs, the TagObject ADT is alsorequired for the "kind" attribute of OpTagsSet and OpTagsDel, whichare coming to OpCodes.hs next. Hence, we move TagObject there, andadjust imports accordingly....
One more ghc 7.6 fix
This is only in master, so needed to be fixed separately.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add test for Luxi calls consistency hs/py
This tests that the same Luxi calls are defined in Python andHaskell. It doesn't test yet that their serialisation is correctthough.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Convert Luxi results to Ganeti errors
This a bit too complex patch converts the result of Luxi calls(submitJob, query*, etc.) from Result to ErrorResult. It thenimmediately revers this in the HTools/Backend/Luxi module, where wedon't need necessarily the full error type (just a nice error...
Fix a few issues found by newer hlint
Testing with a newer hlint found a few minor issues; but all are real,valid recommendations:
- don't use "if cond then f x else f y", but "f (if cond then x else y)" - "if a then b else True" is equivalent to the simpler "not a || b"...
Cleanup network timeouts and htools imports
This patch removes the last HTools module imports from non-htools code(the HTools.Types module), but it requires an associated cleanup:using luxi-specific constants for luxi timeouts (the only effect isthat one timeout decreases from 15 to 10, the default value in the...
Rename Ganeti/HTools/Utils.hs to Ganeti/Utils.hs
This is, I believe, the last non-htools specific file that still livedin the htools directory; it's already widely used in non-htools code,so let's move it before we add more functionality to this module....
Further hlint fixes
Commit 2cdaf22, “Re-enable standard hlint warnings”, got it almostright. The only problem is that (confusingly) the default set of hintsis not in HLint.Default, but in HLint.HLint (it includes Default andsome built-ins).
After changing the lint file to correctly include the defaults, we had...
Move JSON.hs and Compat.hs out from under HTools/
These two files are not htools-specific, so let's move them out of theHTools subdirectory/module hierarchy and directly under Ganeti.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rename the Qlang and Queryd modules
Per the new query module hierarchy, rename Qlang to Query/Language andQueryd to Query/Server. This way, all query-related functionality isnow "contained" in the Query/ directory.
Parameterize the Filter type
In preparation for introducing filtering functionality, we convert the'Filter' type from a '*' kind to a '* -> *' kind.
This allows us to define some general properties for the filter, andfor example introduce later an easy filter compilation, etc....
Add missing luxi query 'QueryFields'
This was missed; we add the definition and the de-serialisationsupport.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Reduce some more code duplication and split code
The Qlang module defines ResultStatus, but it was already defined inGaneti/Luxi.hs; let's remove the duplicate definition from there sincethe proper place is in the newer module.
Also, in order to ease testing, we split some confd functions into a...
Change the Luxi tags kind from String to a custom type
This will allow safer code when we implement the tags query.
Remove obsolete QrViaLuxi type
The actual query definitions are now in Qlang.hs, so let's use theItemType from there instead of luxi-defined type (which is alsoincomplete).
Remove obsolete conversion function for Luxi TH
Due to the (now removed) custom filter field, we needed a conversionfunction. Since now that field is gone, we can move to a simpler LuxiTH implementation.
Rename Query2.hs to Qlang.hs
While starting to use the new filter types, I realised that what iscurrently implemented is the equivalent of `lib/qlang.py', not`lib/query.py', since we only deal with data types for now and not theactual query runtime functionality (RPC, config, etc.)....
Switch Luxi Query operation to use a proper filter
Until now, since we didn't have a proper type for the encoded queryfilters, we were ignoring the filters and handled them as null values(JSNull and respectively ()). With the current Query2 implementation,...
Re-enable standard hlint warnings
Commit 5a1e31b4 (Add infrastructure for, and two extra hlint rules)was intended to add two extra hlint rules, but I didn't realise atthat time that "--hint" when first used overrides the built-inlints. As such, since then we were basically running with just those...
Enhance the Luxi interface implementation
This makes the implementation a bit nicer for both for server andclient side: we add a wrapper function with a better result type, anda few extra functions for building the response.
Add a server-side Luxi implementation
This is a trivial code change, but it allows us to finally test thesend-receive code on both client and server sides via a simplein-process server.
The unittest works, but it won't handle timeouts very nicely; it will...
Switch the Luxi interface from Strings to ByteStrings
I'm doing this change for future performance optimisations. Currentlywe use the Luxi interface just as a client, so not in the hot path,but when we'll use this as a server interface, we're interested to...
Switch job IDs to numeric
This has been a long-standing cleanup item, which we've alwaysrefrained from doing due to the high estimated effort needed.
In reality, it turned out that after some infrastructure improvements(the previous patches), the actual job queue-related changes are quite...
htools: abstract function for parsing job ids
Both the job id and submit job result parsing are abstracted intoseparate functions, so that later changes are more localised.
Also, this makes submitManyJobs itself easier to read.
Introduce a type for the ganeti job type
This will be used for easier change later.
Add decoding of Luxi calls and unittests for LuxiOp
This patch adds a hand-coded decoder for LuxiCall arguments, as thedata-structure is not uniform enough for automated generation (evenfor the serialisation, we had to add hints for some fields,de-serialisation is even harder)....
Auto-define a LuxiReq data type
We currently auto-generate a LuxiOp data type, which holds the entireoperation (including parameters). However, having a data type just forthe method call would be useful, so let's change THH to alsodefineSADT for the Luxi constructors....
Replace hardcoded constants with Constants.hs names
Now that luxi.py constants are exported, we can use them for moreconsistency.
Stop exporting JSON functionality from Utils.hs
This completes the Utils/JSON split started in commit f047f90f. Theimport graph should be cleaner now.
Cleanup hlint errors
First, we update the recommended hlint version to what I used to get aclean output (1.8.15). Most of the changes are:
- remove unneeded parentheses- some simplifications (intercalate " " → unwords, maybe … id → fromMaybe, etc.)- removal of some duplicate code (in previous patches)...
htools: reindent the rest of the files
Generalize the generation of ADT from raw types
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Luxi support for Query status in htools
Adjust htools code to new Luxi argument format
This partially undoes commit 92678b3, more specifically it removes theStore data type and the associated code, since all Luxi arguments arenow lists.
Furthermore, since the qfilter field on Query is complex (it's...
Rename filter and filter_ to qfilter
We currently use 'filter' as the OpCode, QueryRequest and RAPI fieldname for representing a query filter. However, since 'filter' is abuilt-in function, we actually have to use filter_ throughout the codein order to not override the built-in function....
Haskell support for generic Query in Luxi
Untill now htools did not have support for generic Query in Luxi. Thispatch introduces Query as a supported Luxi operation and replacesQueryNodes, QueryInstances and QueryGroups with Query.
Signed-off-by: Agata Murawska <agatamurawska@google.com>...
TH simplification for Luxi
This patch simplifies the generation of save constructors for LuxiOpby always using showJSON over an array of JSValues, instead of havingto pass showJSON in most cases, except the 5-tuple case.
Dots in docstings and hlint error fixes for htools
Signed-off-by: Agata Murawska <agatamurawska@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Use TemplateHaskell to create LUXI operations
htools: docstring fixes and improvements
No code changes (except one definition being moved around in QC.hs)are contained in this patch.
Merge remote branch 'htools/master'