Improve the confd man page
This was a very very thin man page; this patch expands it a bit, butit could use more help still.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate optiontypes; this is not useful in build-bash-completion right now, butmight become so in the future.
Signed-off-by: Iustin Pop <iustin@google.com>...
Enable bash completion for Haskell daemons too
This requires a few tiny changes to build-bash-completion and toMakefile rules, but is straightforward.
Define the actual arguments that are supported by programs
This defines the arguments supported and then modifies the--help-completion output to include them too.
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisationformat for choices/suggests, to make the Python code simpler.
Add support for arguments in --help-completion
This extends the current option-only support with support forarguments too. They are serialised without a without the prefix (sojust with the type) and the min/max ranges (max="none" for unlimited).
A few new option types have been added as well....
A few cleanups in Makefile.am
A lot of the lists in Makefile.am were not sorted properly (or atall); let's sort them for more sanity.
Additionally, check-local used to spew this big shell block, eventhough it does emit nice messages when failing, so we don't need to...
Add missing ipolicy field
This was missing so far…
Fixup TAGS generation with newer GHC
Newer GHC refuses to allow "-O" with interactive mode, so let's filterthat out. Furthermore, sometimes you don't have a clean tree exactlywhen you need to look up something/update the tags, so let's filterout the "-Werror" too....
Remove qualified import of Utils.hs into its test module
This is a leftover from the times when we had a single, huge testmodule; nowadays it's only an annoyance.
Rename Ganeti/HTools/Utils.hs to Ganeti/Utils.hs
This is, I believe, the last non-htools specific file that still livedin the htools directory; it's already widely used in non-htools code,so let's move it before we add more functionality to this module....
bdev: Remove unnecessary empty line
My local pylint didn't complain.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Check fingerprint of file with allowed file storage paths
This makes differences show up in “gnt-cluster verify”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Document the support for default iallocator
Updated the man pages for the commands that accept an iallocator: "." meansthe default iallocator.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Instance QA uses default iallocator
QA for recreate-disk and replace-disks now run also with the defaultiallocator.
Support for the default iallocator in replace-disks
"gnt-instance replace-disks" now behaves like the other commands, and usesthe default iallocator when "." is passed as the iallocator parameter.
Explicitly ask for the default iallocator in commands
Now "gnt-instance recreate-disks" uses the default iallocator when "." isspecified as the iallocator. For uniformity, the same behavior applies tothese commands: gnt-node evacuate gnt-instance migrate...
bdev: Add functions to verify file storage paths
- LoadAllowedFileStoragePaths: Loads a list of allowed file storage paths from a file- CheckFileStoragePath: Checks a path against the list of allowed paths
The unit test for “utils.IsBelowDir” is updated with cases which weren't...
jqueue: Look at archived jobs when watching
First: This enables the use of “gnt-job watch $id” for archived jobs.
Now, the reason for actually making this work is that duringsufficiently large group or node evacuations jobs are archived beforethe client gets to poll for their output. This led to situations where...
Design doc for partitioned Ganeti
Partitioned Ganeti allows to assign node resources in an exclusive manner,in order to limit interference between instances running on the same nodes.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
backend: Check for shared storage also
If normal file storage was disabled but shared storage enabled,“_TransformFileStorageDir” would still throw an exception.
in “opcodes._CheckStorageType” there's also a check, but I wasn't quitesure what the correct way of handling it was, so I added a TODO comment....
utils.FilterEmptyLinesAndComments: Return list
We don't use generators often and lists are easier to re-use.
Improve disk wipe unit test
- Don't hardcode node name in some places- Don't define functions inside functions- Simplify code for testing with and without offset, this is now in two separate tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Wipe added space when growing disks
This patch adds code to wipe newly added disk space when growing disksusing “gnt-instance grow-disk”. “New disk space” is defined as the deltabetween the old block device size (not necessarily equal to the amountrecorded in the configuration) and the new recorded size. Extra caution...
Add unit tests for cmdlib._WipeDisks
This is in preparation for adding disk wipe on growing disks.
Add unit test for FilterEmptyLinesAndComments
I somehow forgot this in the previous patch.
cmdlib._WipeDisks: Code formatting
- LogInfo takes *args, no need to replace values right away- Don't overwrite wipe_chunk_size right after it's been set
Factorize removing comments and empty lines from string
This will also be used for verifying the file storage directory.
ssconf: Fix mistake made in commit ee501db
Move a function from backend to ssconf
The “WriteSsconfFiles” function is used to write ssconf files. By movingit we can avoid importing backend into bootstrap. The latter is importedby CLI programs and backend doesn't have much to do with them.
cmdlib: Change wording of messages during disk wipe
Error messages don't need to say “please” and it's already obvious someinvestigation is needed. LogWarning already logs the message using“logging.error” internally.
Remove constant for disk wipe block size
It is dangerous to have this block size as a global constant as thatcould give the impression of it being easily changed. Doing so withoutfurther adjustments to how “dd” is called will lead to disks not beingwiped properly....
LUClusterVerify: Ignore /proc/drbd if DRBD is disabled
This fixes issue 190. The problem was that the check for DRBD wasenabled if LVM storage is used and didn't depend at all on whether DRBDis enabled.
cli: Exit with status 0 for --help
This patch is somewhat longer than was anticipated. Before, commands like“gnt-instance --help” would exit with the status code 1 (failure). Withthis patch, those commands exits with 0 (success) while unknown commandsstill print the usage screen and exit with status 1....
Remove the python confd server side code
In 2.7 we will only support the Haskell version, if enabled.
Since the original hconfd enabling was a bit hack-ish (copying overthe actual installed ganeti-confd, Python version), the Makefile.amchanges are a bit more involved than just the removal of the Python...
Remove option to select the Python version of confd
This will need some more configure.ac checks for required libraries,which will come later with the Haskell dependencies reorganisation.
Add first version of virtual cluster documentation
- Add document describing virtual clusters - Non-root clusters are not yet described, will be done in another patch- Change title of design document to avoid confusion
INSTALL: Typo, missing dependency
libghc-utf8-string-dev was missing.
Always distribute ganeti.utils.io_unittest-runasroot.py
Tests that depend on fakeroot were not distributed in the tarball createdby "make dist" when fakeroot was missing. Now all tests are distributed,and a warning is printed when a test requires fakeroot but fakeroot is...
vcluster-setup: Watcher, init script
- Generate a wrapper script for ganeti-watcher- Provide command line option to specify init script
vcluster-setup: Add more parameters to "gnt-cluster init"
This patch also adds two new parameters to the vcluster-setup script tonot configure /etc/hosts and networking. Build-time variables are nowused for paths.
Pylint cleanup
The previous commits introduced some pylint errors.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add support for GANETI_ROOTDIR in Haskell code
unsafePerformIO is required to go from the IO monad to pure code.
Modify batch-create to use MultiAlloc
This is the CLI part of the OpInstanceMultiAlloc change. There is someroom for improvement especially with user handling, but it works for nowand the code got simplified significantly.
QA tests still needs to be written....
Make validation error message more useful
Provide a custom Validate for OpInstanceMultiAlloc
This is needed, as we're not strictly separating validation frompopulating the defaults. We do this all in one function. If we nowreceive a non complete OpInstanceCreate opcode in instances, we wouldrun into AttributeErrors on those....
ganeti-cleaner: Separate queue cleaning code
This code does not need to run as root, therefore it's better to splitit out. It is now run with the same permissions as the master daemon.
Stop hardcoding root user
Some parts of the code still use a hardcoded user name: root. This patchreplaces all with a constant specified at build time. The end goal is tomake it possible to run a Ganeti cluster without any special privileges(of course this will prevent some functionality from working)....
Remove explicit recursion from sanitizeDictResults
Since ERpcError is still not a monad, a simple foldr will have todo for now.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Pretty printing JSValue
Merge parseHttpResponse with rpcResultParse
parseHttpResponse is such a simple function that we can actuallydo the entire unpacking in rpcResultParse.
Simplify and generalize rpcResultFill
For now ERpcError is still not a monad, but we can still simplifyrpcResultFill implementation for different calls simply by abstractingit into two helpers.
Remove monadic results wherever possible
There is no need (at least right now) for some of the resultingtypes to be in additional monad, so let us just have more"normal" functions.
Realign instances of typeclasses
As per Iustin's suggestion, this patch aligns parts of typeclassinstance on "=".
Rename RpcError -> String function
As per Iustin's suggestion, show is not a good solution - so insteadthe name is now explainRpcError.
Cleanup of build-time shell variable replacements
Instead of having a different set in (almost) every shell script, thisinserts the most commonly used variables at build time. This way thecode for injecting a root directory for virtual clusters also is just...
ensure-dirs: Don't hardcode ssconf file group
Otherwise chown(2) will fail when noded doesn't run as root.
Fix distcheck after commit 29386d3ee
Missing backslash.
InstanceRecreateDisks: new assertion for node locks
Make sure that all the nodes where a disk operation is peformed are locked.
Fix missing locks in InstanceRecreateDisks
In case of in-place recreate-disks operation, all node locks were releasedbefore starting the actual execution.
Add vcluster-setup utility
This utility can be used to configure an environment for a virtualcluster. It sets up entries in /etc/hosts, creates the necessarydirectory structure, and generates helper scripts.
Documentation for virtual clusters will come in a later patch....
ensure-dirs: Don't convert list to tuple
Tuples are data structures, not containers.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
daemon-util: Use function to determine if confd is enabled
… instead of comparing with two different values in two placse.
cfgshell: Remove pylint disable line
Commit 326830963 removed the use of a private member function. Alsoreplace '' with "" in one place.
Adding RAPI client for instance multi allocation
It was very easy to refactor CreateInstance to repurpose some part of itfor the instance multi allocation request. So we do this.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Populate the instance dict with opcode infos
This is needed so we can load it as an opcode. However, this informationshould not be available on the client, so we inject that info when wereceive the request.
Provided unittest to verify behaviour.
Signed-off-by: René Nussbaumer <rn@google.com>...
Revert unintentional change of daemon log file names
Commit 3329f4dea6 unintentionally changed the filenames of all daemonlog files. This patch reverts part of those changes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix bad wording in 2.3 design document
InstanceInfo RPC call
This patch implements single instance info call - somewhat similar toall_instances_info, except we give a specific instance name.
Current implementation of reading the InstanceInfo value is somewhatcounter-intuitive because when we query a node on thich there is...
instance_info now returns vcpus
instance_info and all_instances_info calls had different per-instancereturn types, this commit fixes the mismatch.
Change RpcResult typeclass
For storage_list call, the result type depends on the call parameters.Therefore, we have to add call as an argument for rpcResultFill - andby extension, to the typeclass.
Signed-off-by: Agata Murawska <agatamurawska@google.com>...
Version RPC call
Implementation of node version query.
Node query now uses live data
Added support for NodeInfo RPC call as source of data for node query.
Node query now collects live fields
We make use of the parameter added in the previous patch and addoption to add live parameters to the query.
Add live parameter to query
The tests we currently have assume, that all the data required forrunning the query is available - once we add live data, this will nolonger be the case.
This patch adds boolean parameter to query function, which tells itwhether to ignore live parameters gathering....
Conversion from RPC error to query status
Simple conversion for queries that use live fields gathered fromRPC calls.
Json helper for node query
This is a helper function that gets the JSON encoding of the firstelement of the list (if possible).
Create ResulT for monad transformations
This patch creates ResultT, a monad transformation used later inRPC <-> query integration.
Tests for NodeGroup query
This adds tests similar to those used for node query. For now theprop_queryGroup_noUnknown is disabled and commented out, as it isfasifiable with ndparams and ipolicy. It may be removed or fixedlater on.
Also, prop_queryGrooup_types has one less property checked - it is...
NodeGroup query in Haskell
Implementation of nodegroup queries in Haskell. This is not yetcomplete as we are missing merged disk parameters and optionwant_diskparams is not implemented.
Config support for NodeGroup query
For some results in nodegroup queries we need agregation similar toone that is done for nodes.
Fix RPC call and result (de)serialization
Previous version of RPC calls implementation in Haskell did not takeinto account that the actual result type for queries is a list, nota dictionary.
This patch aims at fixing the problem "for now" - it is not a pretty...
Add alias for Either RpcError a
This was repeated a lot, so we should abstract this into a type.
Cleanup for RPC errors
Curl error messages are self contained and don't need "error" prefix;added RpcResult Error which will be used once proper deserializationof rpc response is in place.
Change cluster's hypervisors type
We now have Hypervisor type in Objects, might just as well use it.
Correct top comment in Query/Server
I assumed this was a copy-paste+forgetting to change the header comment.
Add nodedCertFile to Path
This patch adds missing Path entry and uses it in Rpc. It was not inplace before due to different build settings - i.e. if curl wasdisabled, there were no problems.
Verify the node count just if the allocation was successful
Enable bash completion for htools scripts
This patch ties together the previous commits, by extendingbuild-bash-completion to call all htools scripts and get theircompletion information, then use that to build fake cli_optionsrepresenting them and finally generate the bash completion...
Add option for displaying completion information
This patch adds support for a --help-completion option, which willdisplay the defined options and their completion information, in aformat designed to be parsed easily from Python, for integration intobuild-bash-completion....
Annotate options with completion information
This is a quite boring patch, just adding annotation information toall existing options. Some of the annotations are not very good; butwe don't have support for more precise completion inbuild-bash-completion, so this is good enough....
Add a data type for completion information
This can be used for either option or argument completion (althoughOptComplNone doesn't make sense for an argument :).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Remove generic options from individual programs
Currently, we test and require that each individual program (hbal,etc.) defines/supports the generic options (currently --help and--version). Even with the test, this is not optimal, since it requireschanges in many places whenever we modify the list of generic options,...
Add printing of group score in hinfo
The global cluster score is less interesting than individual groupscores, for multi-group allocation purposes.
Replace a big tuple with a custom data type
This will make it easier to add new parameters.
Implement virtual cluster support in Python code
- pathutils: Prepend node-specific prefix path- RPC: Use virtual paths (see vcluster.py)- SSH: Pass environment variables, use destination's node directory when copying files using scp, use GANETI_HOSTNAME to determine hostname...
vcluster: Add more details to docstrings
As requested by René Nussbaumer. Hopefully things are more clear now.
Add tests for bash completion
Given the number of settings that need to be controlled a plain bashscript was a better choice over shelltestrunner. Just a few completionsare attempted, but among them should be the most critical ones (e.g.“gnt-instance add --node …”....
Fix shelltests for ganeti-*
They were never run.
bash_completion: Enable extglob while parsing file
In older versions of GNU Bash extended patterns, such as “@(…)”, are onlyavailable with the “extglob” shell option. As pointed out in [1] and [2],“extglob” must be enabled while parsing the code. Therefore the flag must be...