Add more basic Ganeti types
These are needed both in opcodes and (potentially) outside, so we addthem in the main Types module.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Generalise the JSON "Container" type
Currently, we have some types that we kept as dictionaries in theobjects representation (due to inconsistent behaviour, or othertechnical reasons). This should be improved in the future, but in themeantime we can improve the Container type by allowing its keys to be...
Move the 'Hypervisor' type from Objects to Types
This is a very basic type and 'Objects' is a heavy-weight module. Bymoving it to 'types' we simplify (in the future) the import chains.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
Add more basic validation types
This mirrors the ht.py types PositiveInt, NonNegative, etc., exceptthat they work at a more generic level (any numeric type, respectivelyany non-empty list).
Add parser for DRBD /proc file
Signed-off-by: Michele Tartara <mtartara@google.com>...
Properly log errors when setting up daemon FDs
While writing the pipe-based reporting and trying various ways tobreak the startup, I fought for a while trying to understand why errorreporting was different when running the daemon as a user (with norights). It turns out that setupDaemonFDs wants to open the log file...
Switch opcode data type from normal to record constructors
Currently, the OpCode definitions are using normal constructors:
data OpCode = OpTestDelay Double Bool [String] | OpInstanceFailover String Bool (Maybe String) …
While this works for a few opcodes, it becomes unwieldy when dealing...
Fix docstrings for the Filter type
While looking at the opcode docs and clicking accidentally on thefilter type, I saw that the haddock formatting is broken due tonon-escaped use of special chars. Let's convert the ascii-likeformatting to haddock, and have nicer apidoc....
Split OpCode.hs and add module for opcode parameters
Due to TemplateHaskell stage restrictions, we can't define parametersin the same module as we're using them for TH, so we have to defineall module parameters in a separate module.
This patch therefore splits OpCodes.hs in two, adding that module and...
Create a new Ganeti/Types.hs module
There are already three cases where we copied type definitions betweenthe htools-specific types into the main ganeti code. Let's stop doingthis ☺ and create a common types module that holds these.
Note that there already exists BasicTypes.hs, but that refers to very...
Cleanup THH function use from built module namespace
Currently, THH.hs "injects" into the built code names of libraryfunctions like Text.JSON.makeObj, Ganeti.JSON.fromObj, etc. builtdirectly from strings, via (e.g.)
varE (mkName "makeObj")
This means that the "makeObj" name must exist in the target module,...
Expose changing job priority via LUXI
A new LUXI request is added, in both Python and Haskell.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix two hlint warnings
Sorry, I broke lint again :), by introducing two sub-standard changes.
Additionally, this silences an older existing warning that onlytriggers with some versions of hlint (e.g. 1.8.28 which is present inWheezy).
Signed-off-by: Iustin Pop <iustin@google.com>...
Merge branch 'devel-2.6' into master
Conflicts (both trivial): htools/Ganeti/Daemon.hs (_writePidFile rename)...
Merge branch 'stable-2.6' into devel-2.6
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix PID file writing in Haskell daemons
Currently, the code uses createFile, which has the effect of alwaystruncating the file. This is bad, as the content of the PID file iswiped even when we wouldn't be able to lock it!
We switch to openFd (createFile is just a wrapper over that), and we...
Remove unsafePerformIO usage
We need to change a few things, most importantly CLI options defaults,but otherwise we already used the path to files from functions whichwere already in the I/O monad, so we don't have to change much of thecode flow.
Additionally, Path.hs now has an explicit export list, to stop...
Change keyToFilename/readSSconfFile interaction
Currently, keyToFilename uses itself the default config path. In thelight of making that an function in the IO monad, let's remove the"default" path functionality from this function and make it alwaysrequire the config path; its caller, readSSConfFile, can then do the...
Change type of program options to 'IO [Options]'
Some options have defaults that depend on the environment, and wecould handle these in two ways:
- use a place-holder value (e.g. data X a = Default | Custom a) that is later read from the environment- move the options list to IO monad, where it can read the...
Annotate confd startup/luxi binding error
The luxi binding is now annotated for better readability.
Two small logging improvements
The "starting" message is changed to match the Python one, and in casethe preparation fails, we also log the error (beside printing it onstderr or writing it to the error reporting pipe), as at this timelogging is usually set up....
Split queryd run into prepare and exec
This will help with the general daemon split of prepare/run, and flagerrors earlier in the startup.
Move preparation steps of confd in prepMain
This does the address parsing earlier, before we fork, forbetter/faster error reporting.
Switch daemon startup to pipe-based error reporting
This changes the daemon reporting error to the same mechanism as inPython: use a pipe which is written to from the forked children.
Move and rename BasicTypes.annotateIOError
It turns out that annotateIOError already exists in the standardlibrary, with a different purpose (this made me waste 10 minutestrying to understand why the types were wrong…). Let's move this outof BasicTypes, since it's used only in Daemon.hs, and also rename it....
Switch writing of PID file to new reporting style
Currently, the writing of the PID file uses a "standard" errorreporting: catch exception, transform it into a 'Result' type, leavehandling of that to the caller. However, for daemon startup, weactually want exceptions to be propagated up until the handler which...
Convert tag objects to a safer type
Currently, we keep information about the "target" of a tag operationin a data type similar to (TagKind, Maybe String). This is unsafe, asnothing (at the type level) prevents us from accidentally having(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing)....
Switch Luxi TH code from simple to custom fields
This is needed so that we have more flexibility in generating Luxiserialisation code (deserialisation is still custom). Also, onlyexceptions are now using the 'simple' field types, so we might be ablelater to convert and remove that TH code as well....
Make THH:genSaveOpCode a bit more general
This can be improved, by taking all hardcoded names as parameters, toserve as a more-general "build save clause for a multi-constructordata type". I'm not renaming the function as well, since I don't knowexactly how much we can abstract later....
Rename Confd.hs to Confd/Types.hs
This should be the last module rename, promise!
We rename this to conform to the other hierarchies (e.g. Query), andto not have both Confd.hs and Confd/*.hs.
OpCodes.hs: add OpTagsSet and OpTagsDel
Also, adjust comment to $(genOpCode) block to avoid repetition of"only".
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
htools: move TagObject from Luxi.hs to OpCodes.hs
In addition to ReqQueryTags in Luxi.hs, the TagObject ADT is alsorequired for the "kind" attribute of OpTagsSet and OpTagsDel, whichare coming to OpCodes.hs next. Hence, we move TagObject there, andadjust imports accordingly....
Implement base checkFn/prepFn/execFn model for daemons
This mirrors the code in the Python code base, and is required forclean error reporting during startup.
This patch implements the basic infrastructure; the confd daemon isnot yet modified to take advantage of this, just the types are...
One more ghc 7.6 fix
This is only in master, so needed to be fixed separately.
Merge branch 'devel-2.6' into submit
Conflicts: Makefile.am (reordering, fixed) htools/Ganeti/Confd/Server.hs (hlint fixes on master) htools/Ganeti/Daemon.hs (hlint)...
Fix compatibility with newer Haskell libraries
This small patch fixes compatibility with a few newer Haskell libraries:
- base 4.6, included with ghc 7.6, removed the deprecated 'catch' function from Prelude, so our "import Prelude hiding (catch)" is now...
Add test for Luxi calls consistency hs/py
This tests that the same Luxi calls are defined in Python andHaskell. It doesn't test yet that their serialisation is correctthough.
Convert Luxi results to Ganeti errors
This a bit too complex patch converts the result of Luxi calls(submitJob, query*, etc.) from Result to ErrorResult. It thenimmediately revers this in the HTools/Backend/Luxi module, where wedon't need necessarily the full error type (just a nice error...
Move htools backends to a separate directory
Five modules under the HTools/ directories are backendimplementations, so let's move them to a separate directory, to moreclearly show the hierarchy. I wanted to do this for a while, butmerging between branches is always an issue, so let's do it know since...
Fix X509CertError definition in Haskell codebase
Thanks Dato for catching this.
Fix a few issues found by newer hlint
Testing with a newer hlint found a few minor issues; but all are real,valid recommendations:
- don't use "if cond then f x else f y", but "f (if cond then x else y)" - "if a then b else True" is equivalent to the simpler "not a || b"...
Add support for optional fields with null serialised
This follows a conversation we had for how to deal withoptional-but-required fields in JSON serialisations: fields which areoptional (can be either a given type or 'null'), but where the 'null'value is required. There are just a few of these in the Python code,...
Errors.hs: improve field names for ConfigVersionMismatch
Change {exp,act}Code to {exp,act}Ver, which gives a better idea thatthe integer fields represent version numbers.
Also:
- errors.py: update OpPrereqError's docstring to note that an error code is always expected as the second argument (it was previously...
THH.hs: delete isOptional, no longer used
The isOptional function is no longer used after a150585 (“Convertopcode TH code to the use of Field type”).
Convert query path from string errors to GanetiException
This patch converts all the call paths from 'Result' (which containsjust string errors) to 'ErrorResult', which holdsGanetiException-encoded errors. We can now return properOpPrereq/OpExec errors to the clients of the luxi/query socket....
Add exception utility functions
In Python, formatError also returns the exit code, but I find thatsplitting them leads to clearer code.
Add an Errors module mirroring the Python one
As described in the module doc string, while writing this it dawnedupon me that we're mixing all errors together into a single hierarchy(well, type on the Haskell side), which is not good. Some errors areused purely within noded, some in the CLI frontends, etc. so these...
Abstract Luxi template functionality
These are almost generic, so let's change the signatures a bit a makethem fully so.
Abstract a few types in THH.hs
This 'simple' way of defining objects will be used also for errors, solet's make it less Luxi-specific.
Fix setting of 'failN1' flag for corner case
This patch includes:
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
A few unittests improvements
Small simplifications of other unit tests using the (==?)operator when possible, and typo fixes.
BasicTypes.hs: fix docstring for eitherToResult
eitherToResult now converts from `Either a b` to `GenericResult`, notnecessarily from `Either String` only. Also, fix a typo.
Remove custom OpResult type/monad
Since we now have the GeneralResult as a multi-purpose monad, we canremove the custom OpResult monad, and just use 'GeneralResultFailMode' as our type. This allows removal of a few bits ofspecialised infrastructure, relying instead on the generic one....
Use exitErr instead of explicit error message and exitWith
Furthermore, a few messages have their capitalisation changed (fixed).
Signed-off-by: Helga Velroyen <helgav@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Generalise the Result type
Currently, our error monad—Result—has a plain string error type. Thisis not good, as we don't have structured errors, we can't pass backproper error information to Python code, etc.
To solve this, we generalise this type as 'GenericResult a', and make...
Group.hs: add 'allTags'; adjust loaders and test data for it
This commit adds a Group.allTags field to store the tags of node groups,and teaches each loader backend in HTools to populate it (additionally, theIAllocator class in lib/cmdlib.py now includes tags for groups too). Test...
Instance.hs: rename 'tags' to 'exclTags', provide 'allTags'
The mergeData function in Loader.hs included a step to filter an instance'stags to include only the exclusion tags (as specified via the commandline,or cluster-level tags). Later on, code in Node.hs assumed Instance.tags to...
Merge branch 'devel-2.6'
Conflicts: NEWS: Trivial lib/tools/ensure_dirs.py: constant moved to pathutils...
Add a very simple test rpc program
This only supports test delay for now, is not built by default (onlyon demand), and is also not installed anywhere.
confd: add the test_delay RPC call
Also add some more haddock structure to the module. The RPC callitself is rather trivial.
confd: fix RpcVersion call
By accident, we sent the node object as call data in this rpc call,instead the version request (i.e. nothing). This is due to the factthat the 'call' data comes second, not first in the function argument.
Cleanup HTools.Types/BasicTypes imports
Before we reorganised the source tree, the 'Result' type was exportedfrom HTools/Types.hs. This changed during the reorg, but at that timewe didn't change the exports; instead, we kept re-exporting it fromthe old module for compatibility....
Text.hs: update field lists in parseData comments
The comments in parseData had become out of date with the implementationsof load{Group,Node,Inst}. This commit updates the field list in comments tomatch the implementations.
Signed-off-by: Dato Simó <dato@google.com>...
Add a helper for query field checks
… and also use it to simplify 'needsLiveData'. Additionally, add anexplicit export list to Ganeti.Query.Types, since otherwise we'd(re)export all imported symbols.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Cleanup network timeouts and htools imports
This patch removes the last HTools module imports from non-htools code(the HTools.Types module), but it requires an associated cleanup:using luxi-specific constants for luxi timeouts (the only effect isthat one timeout decreases from 15 to 10, the default value in the...
Cleanup haddock documentation a bit
While grepping for htools imports in the non-htools subdirectory, Isaw that our haddock prologue and title are very very old and refer tothe old htools-only state. Let's cleanup a bit…
Add functionality for checking validity of names
This replicates in the Haskell Query2 implementation the behaviour ofthe Python code: if a "simple" filter is passed (one that containsonly Or aggregators and EQ binary ops on the name field), then anfailure is flagged if the given values are not known....
Use the new name filtering behaviour in query
We do this not quite generically, which means we have to addanother layer in the call chain, and rename the current queryfunction, plus add special-case code for each query type. Hopefully wewill be able to improve on this in the future....
Switch ordering of names on query to niceSort
This makes the "all" names queries consistent with the Pythonresults. The change requires updating the unittests, at which point aduplicate error message is simplified.
Add function for getting a group's merged disk params
… and use it in the Query implementation, removing the lastnon-correct query field for Groups.
Simplify a bit queryFields
We don't add a type class for fully-generic handling, but we doabstract the duplicate part.
Fix the node powered field
When initially implementing the node query, I thought the 'powered'field is a representation of the run-time powered status, which wouldmake its query complex.
In reality, it's a simple config query, which we can supporteasily. We also add a small helper, so that we don't hardcode the...
Add a makeSimpleFilter function
And associated unittests. This will be needed for classic-stylequeries.
Add support for classic queries
This patch adds support for classic-style queries (before query2) tothe query socket server. The patch is rather trivial, since as inPython we just piggy-back on the query2 implementation.
Fixup node disk free/total queries
The disk free/total values are optional ones, wrapped in a Maybe, sowe shouldn't directly serialise them. In order to simplify theembedded extraction, we add a small helper function.
Fixup hypervisor queries in node query
We need to only query the default (first enabled) hypervisor, not allhypervisors. For this, we need to add a manual check to ensure that wedon't have a corrupt config (there's no "NonEmptyList" type…).
Make regex-pcre an optional dependency
This patch makes regex-pcre optional, allowing its disable via apreprocessor define NO_REGEX_PCRE. This define will be added toconfig.ac/Makefile.am in the next patch.
The patch also changes multi-line strings into string concatenation,...
Fixup node live field names
The String parameter to 'nodeLiveFieldExtract' is the query2 fieldname, not the RPC-layer field name. Grrr for not having a real datatype for this.
Furthermore, we add some safety check that we don't return JSNull viarsNormal…...
Add an Utils.NiceSort() equivalent
This patch adds a NiceSort equivalent and the corresponding unittest(partially copied from Python unittest). The difference between thePython version and this one is that this implementation doesn't useregular expressions, and as such it doesn't have the 8-groups...
Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate optiontypes; this is not useful in build-bash-completion right now, butmight become so in the future.
Define the actual arguments that are supported by programs
This defines the arguments supported and then modifies the--help-completion output to include them too.
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisationformat for choices/suggests, to make the Python code simpler.
Add support for arguments in --help-completion
This extends the current option-only support with support forarguments too. They are serialised without a without the prefix (sojust with the type) and the min/max ranges (max="none" for unlimited).
A few new option types have been added as well....
Add missing ipolicy field
This was missing so far…
Rename Ganeti/HTools/Utils.hs to Ganeti/Utils.hs
This is, I believe, the last non-htools specific file that still livedin the htools directory; it's already widely used in non-htools code,so let's move it before we add more functionality to this module....
Add support for GANETI_ROOTDIR in Haskell code
unsafePerformIO is required to go from the IO monad to pure code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove explicit recursion from sanitizeDictResults
Since ERpcError is still not a monad, a simple foldr will have todo for now.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Pretty printing JSValue
Merge parseHttpResponse with rpcResultParse
parseHttpResponse is such a simple function that we can actuallydo the entire unpacking in rpcResultParse.
Simplify and generalize rpcResultFill
For now ERpcError is still not a monad, but we can still simplifyrpcResultFill implementation for different calls simply by abstractingit into two helpers.
Remove monadic results wherever possible
There is no need (at least right now) for some of the resultingtypes to be in additional monad, so let us just have more"normal" functions.
Realign instances of typeclasses
As per Iustin's suggestion, this patch aligns parts of typeclassinstance on "=".
Rename RpcError -> String function
As per Iustin's suggestion, show is not a good solution - so insteadthe name is now explainRpcError.
InstanceInfo RPC call
This patch implements single instance info call - somewhat similar toall_instances_info, except we give a specific instance name.
Current implementation of reading the InstanceInfo value is somewhatcounter-intuitive because when we query a node on thich there is...
Change RpcResult typeclass
For storage_list call, the result type depends on the call parameters.Therefore, we have to add call as an argument for rpcResultFill - andby extension, to the typeclass.
Signed-off-by: Agata Murawska <agatamurawska@google.com>...
Version RPC call
Implementation of node version query.
Node query now uses live data
Added support for NodeInfo RPC call as source of data for node query.
Node query now collects live fields
We make use of the parameter added in the previous patch and addoption to add live parameters to the query.