Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate optiontypes; this is not useful in build-bash-completion right now, butmight become so in the future.
Signed-off-by: Iustin Pop <iustin@google.com>...
Define the actual arguments that are supported by programs
This defines the arguments supported and then modifies the--help-completion output to include them too.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisationformat for choices/suggests, to make the Python code simpler.
Add support for arguments in --help-completion
This extends the current option-only support with support forarguments too. They are serialised without a without the prefix (sojust with the type) and the min/max ranges (max="none" for unlimited).
A few new option types have been added as well....
Add missing ipolicy field
This was missing so far…
Rename Ganeti/HTools/Utils.hs to Ganeti/Utils.hs
This is, I believe, the last non-htools specific file that still livedin the htools directory; it's already widely used in non-htools code,so let's move it before we add more functionality to this module....
Add support for GANETI_ROOTDIR in Haskell code
unsafePerformIO is required to go from the IO monad to pure code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove explicit recursion from sanitizeDictResults
Since ERpcError is still not a monad, a simple foldr will have todo for now.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Pretty printing JSValue
Merge parseHttpResponse with rpcResultParse
parseHttpResponse is such a simple function that we can actuallydo the entire unpacking in rpcResultParse.
Simplify and generalize rpcResultFill
For now ERpcError is still not a monad, but we can still simplifyrpcResultFill implementation for different calls simply by abstractingit into two helpers.
Remove monadic results wherever possible
There is no need (at least right now) for some of the resultingtypes to be in additional monad, so let us just have more"normal" functions.
Realign instances of typeclasses
As per Iustin's suggestion, this patch aligns parts of typeclassinstance on "=".
Rename RpcError -> String function
As per Iustin's suggestion, show is not a good solution - so insteadthe name is now explainRpcError.
InstanceInfo RPC call
This patch implements single instance info call - somewhat similar toall_instances_info, except we give a specific instance name.
Current implementation of reading the InstanceInfo value is somewhatcounter-intuitive because when we query a node on thich there is...
Change RpcResult typeclass
For storage_list call, the result type depends on the call parameters.Therefore, we have to add call as an argument for rpcResultFill - andby extension, to the typeclass.
Signed-off-by: Agata Murawska <agatamurawska@google.com>...
Version RPC call
Implementation of node version query.
Node query now uses live data
Added support for NodeInfo RPC call as source of data for node query.
Node query now collects live fields
We make use of the parameter added in the previous patch and addoption to add live parameters to the query.
Add live parameter to query
The tests we currently have assume, that all the data required forrunning the query is available - once we add live data, this will nolonger be the case.
This patch adds boolean parameter to query function, which tells itwhether to ignore live parameters gathering....
Conversion from RPC error to query status
Simple conversion for queries that use live fields gathered fromRPC calls.
Json helper for node query
This is a helper function that gets the JSON encoding of the firstelement of the list (if possible).
Create ResulT for monad transformations
This patch creates ResultT, a monad transformation used later inRPC <-> query integration.
NodeGroup query in Haskell
Implementation of nodegroup queries in Haskell. This is not yetcomplete as we are missing merged disk parameters and optionwant_diskparams is not implemented.
Config support for NodeGroup query
For some results in nodegroup queries we need agregation similar toone that is done for nodes.
Fix RPC call and result (de)serialization
Previous version of RPC calls implementation in Haskell did not takeinto account that the actual result type for queries is a list, nota dictionary.
This patch aims at fixing the problem "for now" - it is not a pretty...
Add alias for Either RpcError a
This was repeated a lot, so we should abstract this into a type.
Cleanup for RPC errors
Curl error messages are self contained and don't need "error" prefix;added RpcResult Error which will be used once proper deserializationof rpc response is in place.
Change cluster's hypervisors type
We now have Hypervisor type in Objects, might just as well use it.
Correct top comment in Query/Server
I assumed this was a copy-paste+forgetting to change the header comment.
Add nodedCertFile to Path
This patch adds missing Path entry and uses it in Rpc. It was not inplace before due to different build settings - i.e. if curl wasdisabled, there were no problems.
Remove generic options from individual programs
Currently, we test and require that each individual program (hbal,etc.) defines/supports the generic options (currently --help and--version). Even with the test, this is not optimal, since it requireschanges in many places whenever we modify the list of generic options,...
Add a data type for completion information
This can be used for either option or argument completion (althoughOptComplNone doesn't make sense for an argument :).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Annotate options with completion information
This is a quite boring patch, just adding annotation information toall existing options. Some of the annotations are not very good; butwe don't have support for more precise completion inbuild-bash-completion, so this is good enough....
Add option for displaying completion information
This patch adds support for a --help-completion option, which willdisplay the defined options and their completion information, in aformat designed to be parsed easily from Python, for integration intobuild-bash-completion....
Add printing of group score in hinfo
The global cluster score is less interesting than individual groupscores, for multi-group allocation purposes.
Replace a big tuple with a custom data type
This will make it easier to add new parameters.
Use autoconf-based paths from Haskell instead of constants
Future changes will change Path.hs to use an environment variable.
constants: Move most paths to separate module
This is inpreparation for the implementation of virtual clusters. Manypaths will change based on an environment variable and are no longerconstant and should no longer be in “constants.py”. Since “constants.py”...
Merge branch 'devel-2.6' into submit
Fix bug in non-mirrored instance allocation
The function `allocateOnSingle' has a bug in the calculation of thecluster score used for deciding which of the many target nodes to usein placing the instance: it uses the original node list for the scorecalculation....
Fix deserialisation bug in ResultEntry
Found via the newly added unit-tests, which test most of theserialisation code in Query/Language (except for QueryResult, forwhich we already tests both sub-components separately).
Rework CLI modules and tests
While investigating how we could test the Daemon.hs module, I realisedthat we have a very, erm, sub-optimal situation:
- HTools/CLI.hs has a nice IO/pure separation testing in cmdline parsing, which allows some basic functionality to be tested, but...
Further hlint fixes
Commit 2cdaf22, “Re-enable standard hlint warnings”, got it almostright. The only problem is that (confusingly) the default set of hintsis not in HLint.Default, but in HLint.HLint (it includes Default andsome built-ins).
After changing the lint file to correctly include the defaults, we had...
Add entire ConfigData serialisation tests
Using the recently-added genArbitrary, we can now implement Arbitraryinstances for even "huge" objects like Cluster, so let's use that toimplement entire ConfigData serialisation tests.
Note that, as we don't have yet proper types for some of the Params...
Add some unittests for node queries
These new tests check that:
- no known fields return unknown- any unknown field returns unknown- the type of the fields is consistent between the getters and the field definition- the length of each result row corresponds with the number of fields...
Add unit test for serialisation of DiskLogicalId and Nodes
Since the DiskLogicalId type is manually serialised/deserialised (seeObjects.hs, `encodeDLid' and `decodeDLId'), let's add a test thatchecks that these are idempotent when combined.
Since we're at it, let's add the same test for Node serialisation,...
Move Version.hs up from under HTools/
This is another module that is generic, and not htools-specific.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Remove QC.hs and replace it with an auto-generated file
In order to have correct code coverage results, we must somehow importall production modules into the test runner. Until now, this was donemanually (when we didn't forget) in QC.hs.
To improve the situation, we remove QC.hs and replace it with an...
Split last two remaining tests from QC.hs
This makes QC.hs obsolete, but we will remove it in a later patch,once we automatically build a file with all the imports (to ensurecorrect coverage results).
Move JSON.hs and Compat.hs out from under HTools/
These two files are not htools-specific, so let's move them out of theHTools subdirectory/module hierarchy and directly under Ganeti.
Split most HTools test code into separate files
Except for Ganeti.HTools.JSON, which needs rename, we split all theother test suites into separate files.
We have to add another common test helper, due to import dependencies(sigh), but otherwise this split is pretty straightforward....
Split Luxi, Qlang, Ssconf and OpCodes tests
… from QC.hs into their own files, again mirroring the production codesource tree.
Split Rpc tests from QC
This required lots of other code moves, so I created it as astandalone patch.
Split one more module out of QC and add test helpers
This splits the confd/utils tests, and adds the TestCommon module forshared test code.
Split out Objects.hs from QC.hs
This is the first file split out from QC.hs - an easy one, since ithas just one test.
The patch changes the way we build hpc excludes, since now we'll havemany modules that need to be excluded, and hpc doesn't seem to be able...
Move the unittest helper to a new Test/Ganeti dir
This patch starts the move of the test haskell code from`htools/Ganeti/HTools/' to its more proper place of`htest/Test/Ganeti'.
Move haskell test code to htest/
This is the first commit of a series that will attempt to cleanup thetest code organisation, which evolved somewhat organically from theinitial pure htools functionality.
The proposed organisation of the tree will be as follows:...
Implement compilation of regexes at creation time
This means that the verification of the correctness of the regex isdone once, at the deserialisation/creation time, as in the Pythoncode. To do this, we have to change the FilterRegex type from an alias...
Rename the Qlang and Queryd modules
Per the new query module hierarchy, rename Qlang to Query/Language andQueryd to Query/Server. This way, all query-related functionality isnow "contained" in the Query/ directory.
Add filtering support in Query
This adds basic infrastructure for filtering (fully functional except,as usual, for runtime data), and then uses it for node queries.
Since the filtering exports regex matching as an externalfunctionality, we have to use a regex library. There are many flavours...
Add an Applicative instance for our Result type
This will allow us to run generic computations in an Applicativecontext, if a monadic one is not needed (or not applicable due toclass constraints).
Parameterize the Filter type
In preparation for introducing filtering functionality, we convert the'Filter' type from a '*' kind to a '* -> *' kind.
This allows us to define some general properties for the filter, andfor example introduce later an easy filter compilation, etc....
Implement QueryFields for Nodes
Since we have all the definitions already, we can easily enablethis. Manual testing shows no difference between the Python and theHaskell versions of node list-fields.
Putting the multiallocate pieces together
This is the final part:
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Adding allocList function
This function iterates over the tryMGAlloc function, updates the nodelist and instance list, and refeeds it recursively until no newinstances are left.
This allows us then to allocate multiple instances and see if they wouldfit together....
Add missing luxi query 'QueryFields'
This was missed; we add the definition and the de-serialisationsupport.
Add Query support for Nodes (no filtering, no RPC)
This is the initial support for Query2: basic infrastructure (exceptfiltering) and node query support (without RPC).
It implements all the fields (tests by comparison with list-fields onthe Python side), except that:...
One more SADT type - VType
This adds the VTYPE_* conversion to VType, used for QFT equivalencelater.
Expand THH with more functionality for parameters
This adds two related improvements to THH:
- for parameters, we declare a list with all their fields, so that Query2 can build the list of fields (e.g. for hvparams, or ndparams) automatically
- we declare a new type class for "DictObjects", i.e. objects which...
Add more node-related data types and functions
This is a simple type declaration for NodeRole, a NdParamObject typeclass, and a few related helper functions for nodes and node groups.
Add type classes for common objects
These mirror the TaggableObject in Python, in the sense that we willbe able to define generic functions for querying such fields.
Stub query2 call integration into QueryD
This patch corrects the definitions in Qlang.hs to match what Pythonexpects on the wire; this means replacing some manual data typedefinitions with 'buildObject' so that we get serialisation (and fieldnames) for free, adding (manually) JSON instances for types which are...
Fix error reporting for bad Luxi arguments in QueryD
Currently, the query daemon would simply close the connection to theclient without issuing a response, if parsing the arguments failed;the error was just logged.
Since this is very ugly from the client's point of view, we change it...
Improve error reporting in our JSON conversions
Reporting things such as the following in our error messages(indentation added by me, not originally present, so it's even worse):
JSArray [JSArray [JSRational False (1 % 1),JSString (JSONString {fromJSString = "a"})],...
Cleanup query socket before attempting to bind
This patch adds a simple function to ensure a socket doesn't existbefore binding to it. Additionally, we cleanup the exports ofQueryd.hs.
Complete and enable the haskell QueryClusterInfo call
Since we have now access to all cluster parameters, we can "fill" theparameters, and can finally enable it in CLI/RAPI.
Expand Objects.hs definitions
This patch adds the missing parameters in the cluster/group objects,for now as simple maps (dictionaries), without type safety. Therationale for adding them as such is:
- we need something to enable query functionality- since we don't modify the values, we don't risk introducing bugs...
Add a fillDict function
This is similar to the Python version, objects.py:FillDict.
Fix qualified import of Data.Map in QC.hs
As opposed to all other places in the code, we imported this asData.Map, leaving to longer names. Let's make it similar to the restof the code (import qualified Data.Map as Map).
Add two confd library tests
These test that we encode requests correctly and decode good messages,that a message with a too old/new timestamp is never accepted, andthat a message signed with a different key is never accepted.
Reduce some more code duplication and split code
The Qlang module defines ResultStatus, but it was already defined inGaneti/Luxi.hs; let's remove the duplicate definition from there sincethe proper place is in the newer module.
Also, in order to ease testing, we split some confd functions into a...
Add test for checking Haskell/Python opcode equivalence
This is a very big hack for testing the equivalence of Python andHaskell opcode definitions. See the docstring for details; I'm notvery happy with the solution but it does the job.
An alternate option would be to launch the Python code when...
Create a custom type for disk indices
While (again) trying to test Python/Haskell encoding interoperability,I found another bug: the disk index is declared in Python asht.TPositiveInt, but in Haskell just as Int, so it can take negativevalues too. Clearly we can do better, so let's add a wrapper type that...
Handle better 'null' values in optional fields
While testing Haskell⇔Python interoperability for opcodeserialisation, I found this bug: the Haskell code doesn't treatoptional fields with 'null' values as missing, which the Python codedoes, leading to differences....
Add test case for OpCode list equivalence
For now, we only test that we don't define extra opcodes in theHaskell code; once we have parity, we can enable the (for now)commented-out lines that check the reverse.
Add infrastructure for reading Python command output
This will be used in the future to make tests/assertions forPython/Haskell codebase equivalence (e.g. checking that the sameopcodes are defined, etc.).
(Side-note: this might not work in VPATH builds; I'll see how it goes)...
QCHelper: add support for defining HUnit test cases
This will allow us to use exactly the same method as for defining andusing QuickCheck properties. The differentiation is based on name,which is not very nice but is the same method used intest-framework-th, for example (so we will be able to switch...
Rename tests for nicer names in test output
With the change to test-framework, where individual tests are listed,QCHelper tries to remove a "prop_$group_" prefix from the functionnames, for shorter names. However, some test groups (LUXI) and testfunctions are not uniform, leading to long names, so we should fix...
Switch Haskell test harness to test-framework
This patch replaces our home-grown, and quite limited, test runnerinfrastructure with test-framework(http://batterseapower.github.com/test-framework/). The rationale fordoing so is as follows:
- we will need to add support for HUnit tests, so either we add more...
Simplify the Luxi client-server test
Since now we have the recvMsgExt, we can remove the handleEOFfunction.
Fix bug in the Luxi client-server test
We didn't close the server correctly, so for test sizes bigger thanthe max-open-files ulimit, the tests were failing in accept().
TH: one style fix and more docstrings
We were missing many docstrings in THH.hs, so let's add at least someof them, and fix some unquoted '/'. Additionally one style change hasbeen done.
OpCodes: build and export a list of all opcodes
This can be used for cross-checking with the Python code forconsistency on defined opcodes.
TH: Abstract function for computing constructor names
We'll need this in another place shortly, so let's abstract it and addproper verification of whether we were passed a type name correctly;the previous version would have failed with a pattern match failure,...
Enable tags query over the query socket
This patch adds the tags field to the objects which were missing it(all except Cluster), implements handling the LuxiCall QueryTags, andthen enables the use of the query socket in cli.ListTags, used by allcommands, and in the RAPI client....
Config.hs: Add a function to lookup group
Like in the python code, this is a bit more complex since groups areindexed by UUID, so we must fallback to lookup via the name (slow; butthe number of groups should be small).
Add a Functor instance for our Result type
This will allow us to use the simpler applicative form (<$>, <*>)instead of monadic liftM, liftM2, etc.
Change the Luxi tags kind from String to a custom type
This will allow safer code when we implement the tags query.
Remove obsolete QrViaLuxi type
The actual query definitions are now in Qlang.hs, so let's use theItemType from there instead of luxi-defined type (which is alsoincomplete).