Change type of program options to 'IO [Options]'
Some options have defaults that depend on the environment, and wecould handle these in two ways:
- use a place-holder value (e.g. data X a = Default | Custom a) that is later read from the environment- move the options list to IO monad, where it can read the...
Annotate confd startup/luxi binding error
The luxi binding is now annotated for better readability.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Two small logging improvements
The "starting" message is changed to match the Python one, and in casethe preparation fails, we also log the error (beside printing it onstderr or writing it to the error reporting pipe), as at this timelogging is usually set up....
Split queryd run into prepare and exec
This will help with the general daemon split of prepare/run, and flagerrors earlier in the startup.
Move preparation steps of confd in prepMain
This does the address parsing earlier, before we fork, forbetter/faster error reporting.
Switch daemon startup to pipe-based error reporting
This changes the daemon reporting error to the same mechanism as inPython: use a pipe which is written to from the forked children.
Move and rename BasicTypes.annotateIOError
It turns out that annotateIOError already exists in the standardlibrary, with a different purpose (this made me waste 10 minutestrying to understand why the types were wrong…). Let's move this outof BasicTypes, since it's used only in Daemon.hs, and also rename it....
Switch writing of PID file to new reporting style
Currently, the writing of the PID file uses a "standard" errorreporting: catch exception, transform it into a 'Result' type, leavehandling of that to the caller. However, for daemon startup, weactually want exceptions to be propagated up until the handler which...
Convert tag objects to a safer type
Currently, we keep information about the "target" of a tag operationin a data type similar to (TagKind, Maybe String). This is unsafe, asnothing (at the type level) prevents us from accidentally having(TagCluster, Just "instance1.example.com"), or (TagInstance, Nothing)....
Switch Luxi TH code from simple to custom fields
This is needed so that we have more flexibility in generating Luxiserialisation code (deserialisation is still custom). Also, onlyexceptions are now using the 'simple' field types, so we might be ablelater to convert and remove that TH code as well....
Make THH:genSaveOpCode a bit more general
This can be improved, by taking all hardcoded names as parameters, toserve as a more-general "build save clause for a multi-constructordata type". I'm not renaming the function as well, since I don't knowexactly how much we can abstract later....
Rename Confd.hs to Confd/Types.hs
This should be the last module rename, promise!
We rename this to conform to the other hierarchies (e.g. Query), andto not have both Confd.hs and Confd/*.hs.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
OpCodes.hs: add OpTagsSet and OpTagsDel
Also, adjust comment to $(genOpCode) block to avoid repetition of"only".
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
htools: move TagObject from Luxi.hs to OpCodes.hs
In addition to ReqQueryTags in Luxi.hs, the TagObject ADT is alsorequired for the "kind" attribute of OpTagsSet and OpTagsDel, whichare coming to OpCodes.hs next. Hence, we move TagObject there, andadjust imports accordingly....
Implement base checkFn/prepFn/execFn model for daemons
This mirrors the code in the Python code base, and is required forclean error reporting during startup.
This patch implements the basic infrastructure; the confd daemon isnot yet modified to take advantage of this, just the types are...
One more ghc 7.6 fix
This is only in master, so needed to be fixed separately.
Merge branch 'devel-2.6' into submit
Conflicts: Makefile.am (reordering, fixed) htools/Ganeti/Confd/Server.hs (hlint fixes on master) htools/Ganeti/Daemon.hs (hlint)...
Merge branch 'stable-2.6' into devel-2.6
Fix compatibility with newer Haskell libraries
This small patch fixes compatibility with a few newer Haskell libraries:
- base 4.6, included with ghc 7.6, removed the deprecated 'catch' function from Prelude, so our "import Prelude hiding (catch)" is now...
Add test for Luxi calls consistency hs/py
This tests that the same Luxi calls are defined in Python andHaskell. It doesn't test yet that their serialisation is correctthough.
Convert Luxi results to Ganeti errors
This a bit too complex patch converts the result of Luxi calls(submitJob, query*, etc.) from Result to ErrorResult. It thenimmediately revers this in the HTools/Backend/Luxi module, where wedon't need necessarily the full error type (just a nice error...
Move htools backends to a separate directory
Five modules under the HTools/ directories are backendimplementations, so let's move them to a separate directory, to moreclearly show the hierarchy. I wanted to do this for a while, butmerging between branches is always an issue, so let's do it know since...
Fix X509CertError definition in Haskell codebase
Thanks Dato for catching this.
Fix a few issues found by newer hlint
Testing with a newer hlint found a few minor issues; but all are real,valid recommendations:
- don't use "if cond then f x else f y", but "f (if cond then x else y)" - "if a then b else True" is equivalent to the simpler "not a || b"...
Add support for optional fields with null serialised
This follows a conversation we had for how to deal withoptional-but-required fields in JSON serialisations: fields which areoptional (can be either a given type or 'null'), but where the 'null'value is required. There are just a few of these in the Python code,...
Errors.hs: improve field names for ConfigVersionMismatch
Change {exp,act}Code to {exp,act}Ver, which gives a better idea thatthe integer fields represent version numbers.
Also:
- errors.py: update OpPrereqError's docstring to note that an error code is always expected as the second argument (it was previously...
THH.hs: delete isOptional, no longer used
The isOptional function is no longer used after a150585 (“Convertopcode TH code to the use of Field type”).
Convert query path from string errors to GanetiException
This patch converts all the call paths from 'Result' (which containsjust string errors) to 'ErrorResult', which holdsGanetiException-encoded errors. We can now return properOpPrereq/OpExec errors to the clients of the luxi/query socket....
Add exception utility functions
In Python, formatError also returns the exit code, but I find thatsplitting them leads to clearer code.
Add an Errors module mirroring the Python one
As described in the module doc string, while writing this it dawnedupon me that we're mixing all errors together into a single hierarchy(well, type on the Haskell side), which is not good. Some errors areused purely within noded, some in the CLI frontends, etc. so these...
Abstract Luxi template functionality
These are almost generic, so let's change the signatures a bit a makethem fully so.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
Abstract a few types in THH.hs
This 'simple' way of defining objects will be used also for errors, solet's make it less Luxi-specific.
Fix setting of 'failN1' flag for corner case
This patch includes:
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
A few unittests improvements
Small simplifications of other unit tests using the (==?)operator when possible, and typo fixes.
BasicTypes.hs: fix docstring for eitherToResult
eitherToResult now converts from `Either a b` to `GenericResult`, notnecessarily from `Either String` only. Also, fix a typo.
Remove custom OpResult type/monad
Since we now have the GeneralResult as a multi-purpose monad, we canremove the custom OpResult monad, and just use 'GeneralResultFailMode' as our type. This allows removal of a few bits ofspecialised infrastructure, relying instead on the generic one....
Use exitErr instead of explicit error message and exitWith
Furthermore, a few messages have their capitalisation changed (fixed).
Signed-off-by: Helga Velroyen <helgav@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Generalise the Result type
Currently, our error monad—Result—has a plain string error type. Thisis not good, as we don't have structured errors, we can't pass backproper error information to Python code, etc.
To solve this, we generalise this type as 'GenericResult a', and make...
Merge branch 'devel-2.6' into master
Group.hs: add 'allTags'; adjust loaders and test data for it
This commit adds a Group.allTags field to store the tags of node groups,and teaches each loader backend in HTools to populate it (additionally, theIAllocator class in lib/cmdlib.py now includes tags for groups too). Test...
Instance.hs: rename 'tags' to 'exclTags', provide 'allTags'
The mergeData function in Loader.hs included a step to filter an instance'stags to include only the exclusion tags (as specified via the commandline,or cluster-level tags). Later on, code in Node.hs assumed Instance.tags to...
Merge branch 'devel-2.6'
Conflicts: NEWS: Trivial lib/tools/ensure_dirs.py: constant moved to pathutils...
Add a very simple test rpc program
This only supports test delay for now, is not built by default (onlyon demand), and is also not installed anywhere.
confd: add the test_delay RPC call
Also add some more haddock structure to the module. The RPC callitself is rather trivial.
confd: fix RpcVersion call
By accident, we sent the node object as call data in this rpc call,instead the version request (i.e. nothing). This is due to the factthat the 'call' data comes second, not first in the function argument.
Signed-off-by: Iustin Pop <iustin@google.com>...
Cleanup HTools.Types/BasicTypes imports
Before we reorganised the source tree, the 'Result' type was exportedfrom HTools/Types.hs. This changed during the reorg, but at that timewe didn't change the exports; instead, we kept re-exporting it fromthe old module for compatibility....
Text.hs: update field lists in parseData comments
The comments in parseData had become out of date with the implementationsof load{Group,Node,Inst}. This commit updates the field list in comments tomatch the implementations.
Signed-off-by: Dato Simó <dato@google.com>...
Add a helper for query field checks
… and also use it to simplify 'needsLiveData'. Additionally, add anexplicit export list to Ganeti.Query.Types, since otherwise we'd(re)export all imported symbols.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Cleanup network timeouts and htools imports
This patch removes the last HTools module imports from non-htools code(the HTools.Types module), but it requires an associated cleanup:using luxi-specific constants for luxi timeouts (the only effect isthat one timeout decreases from 15 to 10, the default value in the...
Cleanup haddock documentation a bit
While grepping for htools imports in the non-htools subdirectory, Isaw that our haddock prologue and title are very very old and refer tothe old htools-only state. Let's cleanup a bit…
Add functionality for checking validity of names
This replicates in the Haskell Query2 implementation the behaviour ofthe Python code: if a "simple" filter is passed (one that containsonly Or aggregators and EQ binary ops on the name field), then anfailure is flagged if the given values are not known....
Use the new name filtering behaviour in query
We do this not quite generically, which means we have to addanother layer in the call chain, and rename the current queryfunction, plus add special-case code for each query type. Hopefully wewill be able to improve on this in the future....
Switch ordering of names on query to niceSort
This makes the "all" names queries consistent with the Pythonresults. The change requires updating the unittests, at which point aduplicate error message is simplified.
Add function for getting a group's merged disk params
… and use it in the Query implementation, removing the lastnon-correct query field for Groups.
Simplify a bit queryFields
We don't add a type class for fully-generic handling, but we doabstract the duplicate part.
Fix the node powered field
When initially implementing the node query, I thought the 'powered'field is a representation of the run-time powered status, which wouldmake its query complex.
In reality, it's a simple config query, which we can supporteasily. We also add a small helper, so that we don't hardcode the...
Add a makeSimpleFilter function
And associated unittests. This will be needed for classic-stylequeries.
Add support for classic queries
This patch adds support for classic-style queries (before query2) tothe query socket server. The patch is rather trivial, since as inPython we just piggy-back on the query2 implementation.
Fixup node disk free/total queries
The disk free/total values are optional ones, wrapped in a Maybe, sowe shouldn't directly serialise them. In order to simplify theembedded extraction, we add a small helper function.
Fixup hypervisor queries in node query
We need to only query the default (first enabled) hypervisor, not allhypervisors. For this, we need to add a manual check to ensure that wedon't have a corrupt config (there's no "NonEmptyList" type…).
Make regex-pcre an optional dependency
This patch makes regex-pcre optional, allowing its disable via apreprocessor define NO_REGEX_PCRE. This define will be added toconfig.ac/Makefile.am in the next patch.
The patch also changes multi-line strings into string concatenation,...
Fixup node live field names
The String parameter to 'nodeLiveFieldExtract' is the query2 fieldname, not the RPC-layer field name. Grrr for not having a real datatype for this.
Furthermore, we add some safety check that we don't return JSNull viarsNormal…...
Add an Utils.NiceSort() equivalent
This patch adds a NiceSort equivalent and the corresponding unittest(partially copied from Python unittest). The difference between thePython version and this one is that this implementation doesn't useregular expressions, and as such it doesn't have the 8-groups...
Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate optiontypes; this is not useful in build-bash-completion right now, butmight become so in the future.
Define the actual arguments that are supported by programs
This defines the arguments supported and then modifies the--help-completion output to include them too.
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisationformat for choices/suggests, to make the Python code simpler.
Add support for arguments in --help-completion
This extends the current option-only support with support forarguments too. They are serialised without a without the prefix (sojust with the type) and the min/max ranges (max="none" for unlimited).
A few new option types have been added as well....
Add missing ipolicy field
This was missing so far…
Rename Ganeti/HTools/Utils.hs to Ganeti/Utils.hs
This is, I believe, the last non-htools specific file that still livedin the htools directory; it's already widely used in non-htools code,so let's move it before we add more functionality to this module....
Add support for GANETI_ROOTDIR in Haskell code
unsafePerformIO is required to go from the IO monad to pure code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove explicit recursion from sanitizeDictResults
Since ERpcError is still not a monad, a simple foldr will have todo for now.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Pretty printing JSValue
Merge parseHttpResponse with rpcResultParse
parseHttpResponse is such a simple function that we can actuallydo the entire unpacking in rpcResultParse.
Simplify and generalize rpcResultFill
For now ERpcError is still not a monad, but we can still simplifyrpcResultFill implementation for different calls simply by abstractingit into two helpers.
Remove monadic results wherever possible
There is no need (at least right now) for some of the resultingtypes to be in additional monad, so let us just have more"normal" functions.
Realign instances of typeclasses
As per Iustin's suggestion, this patch aligns parts of typeclassinstance on "=".
Rename RpcError -> String function
As per Iustin's suggestion, show is not a good solution - so insteadthe name is now explainRpcError.
InstanceInfo RPC call
This patch implements single instance info call - somewhat similar toall_instances_info, except we give a specific instance name.
Current implementation of reading the InstanceInfo value is somewhatcounter-intuitive because when we query a node on thich there is...
Change RpcResult typeclass
For storage_list call, the result type depends on the call parameters.Therefore, we have to add call as an argument for rpcResultFill - andby extension, to the typeclass.
Signed-off-by: Agata Murawska <agatamurawska@google.com>...
Version RPC call
Implementation of node version query.
Node query now uses live data
Added support for NodeInfo RPC call as source of data for node query.
Node query now collects live fields
We make use of the parameter added in the previous patch and addoption to add live parameters to the query.
Add live parameter to query
The tests we currently have assume, that all the data required forrunning the query is available - once we add live data, this will nolonger be the case.
This patch adds boolean parameter to query function, which tells itwhether to ignore live parameters gathering....
Conversion from RPC error to query status
Simple conversion for queries that use live fields gathered fromRPC calls.
Json helper for node query
This is a helper function that gets the JSON encoding of the firstelement of the list (if possible).
Create ResulT for monad transformations
This patch creates ResultT, a monad transformation used later inRPC <-> query integration.
NodeGroup query in Haskell
Implementation of nodegroup queries in Haskell. This is not yetcomplete as we are missing merged disk parameters and optionwant_diskparams is not implemented.
Config support for NodeGroup query
For some results in nodegroup queries we need agregation similar toone that is done for nodes.
Fix RPC call and result (de)serialization
Previous version of RPC calls implementation in Haskell did not takeinto account that the actual result type for queries is a list, nota dictionary.
This patch aims at fixing the problem "for now" - it is not a pretty...
Add alias for Either RpcError a
This was repeated a lot, so we should abstract this into a type.
Cleanup for RPC errors
Curl error messages are self contained and don't need "error" prefix;added RpcResult Error which will be used once proper deserializationof rpc response is in place.
Change cluster's hypervisors type
We now have Hypervisor type in Objects, might just as well use it.
Correct top comment in Query/Server
I assumed this was a copy-paste+forgetting to change the header comment.
Add nodedCertFile to Path
This patch adds missing Path entry and uses it in Rpc. It was not inplace before due to different build settings - i.e. if curl wasdisabled, there were no problems.
Remove generic options from individual programs
Currently, we test and require that each individual program (hbal,etc.) defines/supports the generic options (currently --help and--version). Even with the test, this is not optimal, since it requireschanges in many places whenever we modify the list of generic options,...
Add a data type for completion information
This can be used for either option or argument completion (althoughOptComplNone doesn't make sense for an argument :).
Annotate options with completion information
This is a quite boring patch, just adding annotation information toall existing options. Some of the annotations are not very good; butwe don't have support for more precise completion inbuild-bash-completion, so this is good enough....
Add option for displaying completion information
This patch adds support for a --help-completion option, which willdisplay the defined options and their completion information, in aformat designed to be parsed easily from Python, for integration intobuild-bash-completion....
Add printing of group score in hinfo
The global cluster score is less interesting than individual groupscores, for multi-group allocation purposes.
Replace a big tuple with a custom data type
This will make it easier to add new parameters.