Add shelltests for the mon-collector
This commit adds shelltests for the mon-collector binary and for the DRBDdata collector.
Also, it fixes a small bug in the DRBD parser found thanks to the tests.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Enable equivalence checks of opcode list
This patch enhances the opcode list checks - instead of spawning aPython interpreter to display the opcode list, we export it staticallyin Constants.hs via a slight convert-constants change.
Furthermore, since we now have opcode parity, we enable full opcode...
Include networks in Makefile and bash completion
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Add custom code for CV_E* constants to convert-constants
Currently, the cluster verify errors are defined as follows:
CV_ECLUSTER_FOO = (TCLUSTER, "ECLUSTER_FOO", "description")
This means there's no standalone name for the string "ECLUSTER_FOO",which makes it hard to derive automatically a type for this union in...
Sort content of generated RPC code
This is similar to commit f5ce761. A stable sort order makes changesbetween versions easier to find.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Find coverage during configure, issue with Debian package
- Debian Squeeze and up have a package named “python-coverage”, but it doesn't use the same binary name as upstream (“coverage”).- Said package includes a patch to use symlinks instead of file copies...
Enable conversion of errors constants
These will be needed for the serialisation of errors.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
bash_completion: Fix mistake introduced in merge 6a74f58
This part of the code was duplicated by accident.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.6'
Improve Haskell configure options and detection
This patch cleans up the Haskell library detection and defaults.
First, it makes the base compiler/libraries required, per the email discussion.
It then adds two new small autoconf macros, on to check for a required...
Merge ganeti-master-cleaner back into ganeti-cleaner
As I wrote during/after the review on commit 2958c56, “ganeti-cleaner:Separate queue cleaning code”, while I appreciated the permissionseparation, I didn't like too much the file-based approach:
- it is a very simple script, and lots of the code is duplicated...
Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate optiontypes; this is not useful in build-bash-completion right now, butmight become so in the future.
Signed-off-by: Iustin Pop <iustin@google.com>...
Enable bash completion for Haskell daemons too
This requires a few tiny changes to build-bash-completion and toMakefile rules, but is straightforward.
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisationformat for choices/suggests, to make the Python code simpler.
Always distribute ganeti.utils.io_unittest-runasroot.py
Tests that depend on fakeroot were not distributed in the tarball createdby "make dist" when fakeroot was missing. Now all tests are distributed,and a warning is printed when a test requires fakeroot but fakeroot is...
bash_completion: Enable extglob while parsing file
In older versions of GNU Bash extended patterns, such as “@(…)”, are onlyavailable with the “extglob” shell option. As pointed out in [1] and [2],“extglob” must be enabled while parsing the code. Therefore the flag must be...
Enable bash completion for htools scripts
This patch ties together the previous commits, by extendingbuild-bash-completion to call all htools scripts and get theircompletion information, then use that to build fake cli_optionsrepresenting them and finally generate the bash completion...
Revert "bash_completion: Always enable extglob"
This reverts commit 94014b6328f813b674a1126ffd221705d7e315b3. Turns outI misunderstood the “extglob” shell option.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
bash_completion: Generate more compact version
First the numbers:$ stat --format '%s %n' doc/examples/bash_completion*77847 doc/examples/bash_completion86492 doc/examples/bash_completion-debug
The non-debug version doesn't use indentation and does not have some...
bash_completion: Always enable extglob
In older versions of GNU Bash extended patterns, such as “@(…)” are onlyavailable with the “extglob” shell option. This patch adds a wrapperfunction so that extglob is always enabled while doing any completion.Due to early returns inside the main completion function this seemed...
Use autoconf-based paths from Haskell instead of constants
Future changes will change Path.hs to use an environment variable.
Migrate build-bash-completion from constants to pathutils
File system paths moved from constants to pathutils.
Move generated htools test stubs to htest/
This moves the last (I think) htools-related bits out of test/ underhtest/.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Move haskell test code to htest/
This is the first commit of a series that will attempt to cleanup thetest code organisation, which evolved somewhat organically from theinitial pure htools functionality.
The proposed organisation of the tree will be as follows:...
Extend convert-constants support for dicts
This enhances convert-constants to not flatten dicts completely, butalso generate a so-called association list for them.
This allows either direct use of the 'lookup' function, or (forperformance) conversion to Data.Map and optimised lookup later....
Enhance convert-constants list generation
While looking at the hypervisor types in Constants.hs, I saw that the'hyperTypes' list is using strings instead of names. This is due tothe fact that we require the entire elements in the list to beidentified (homogeneous lists), but the string "xen-pvm" is declared...
convert-constants: also export qlang.py constants
Since these will be needed for Haskell's implementation of query2.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Ensure a stable content of the bash completion file
Currently, the order of commands in the bash completion file israndom, because the sub-commands are not sorted. This makes it harderto investigate the differences in packaged Ganeti or in installedGaneti, since chunks in this file will have a random order....
Improve the check-news script
Allow all errors to be displayed, instead of aborting at the firstone, and don't show stacktraces anymore.
Add checks from wrong hard-coded paths in manpages
This is trivial, but should prevent such things in the future.
Implement support for multi-module export
This add support for exporting constants from multiple modules (asopposed to hard-coding constants), and also makes the output morereadable by skipping things we know for sure we don't want to convert(as opposed to things we would like to but don't know how to...
Implement reverse mapping of values to names
This adds a bit of dumb mapping of values to names, while trying to besafe. This is the best we can do without resorting to parsing orinterpreting ASTs.
The difference in the output is:
-- | Converted from Python list or set ADMINST_ALL...
QA: Enable use of OR conditions in test checks
Until now “TestRunIf” and “TestEnabled” could only handle AND. With thispatch a new class named “Either” is added to “qa_config” and allows OR.The name “Either” was chosen instead of “Or” as the latter is very close...
Add example script for using RAPI test utilities
This script shows a few examples on how to use the RAPI input testclient. It is also run at “make check” time to ensure it's notcompletely broken.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rename htools-hpc to hpc-tools to fix laziness
This is a stupid/trivial patch. I didn't realise when addinghtools-hpc that this will "break" tab completion on htools; as in, Iwill have to add / manually always when completing files on thecommand line....
Build a coverage-enabled version of the tools
This allows the offline tests to also generate coverage data, and thehs-coverage target is changed to show both unit-test and offline-testscoverage.
The downside is that now we build yet-another-binary, which makes the...
Enable htools offline tests as Haskell tests
This enables the offline-test.sh run both in the special "hs-check" rule and during normal "make check".
Split check-man-warnings into two
This moves the dashes check to a new check-man-dashes script.
Rename check-man to check-man-warnings
Currently the check-man test is dependent on the MAN_HAS_WARNINGStest, which made sense as long as the script only did the warningscheck.
However, we should run the [em] tests always, and in general we couldrun other tests too, even if man doesn't support --warnings; so let's...
Add automated checking for non-escaped --
This checks to see if any man output has [em] in it, as that will notbe correctly show in ASCII. Unfortunately the check will depend onwhether the other man check is done, but I didn't want to introduceyet another short script for this (can be done, no issue)....
Add support for RE patterns to convert constants
This is a trivial conversion.
Add support for lists/frozensets in convert-constants
Unfortunately, we only support lists of simple types, and not evenlists of tuples. If we actually needed those, it would be possible toimplement them, with a bit more complexity in the converter.
Add support for tuples in convert-constants
More improvements to convert-constants
This prepares for tuple and other conversions.
Improve convert-constants to handle dictionaries
The two main drawbacks for convert-constants are the fact that itcan't handle sets/frozensets (mainly due to the fact that I don't knowhow useful this would be to the Haskell code) and that it cannotexport dictionaries....
gen-coverage: Change filter
It is not necessary to run a Python script to generate the filter, whichdidn't work very well anyway on systems with modules in/usr/share/pyshared.
check-news/NEWS: impose more standards
- Always two empty lines before a version.- Standardize how version names should be called.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
rpc: Add support for resolver options callback
This adds support for a callback returning name resolver options. Thisis required for powercycling offline nodes. While it would be possibleto implement this using a separate RPC client class, doing so would...
Fix failure in distcheck and coverage tests
autotools/testrunner was missing in distcheck. Now some changes from7004106119c5c56c93085ccfb1250b3561b5b6ad have been reverted, and testrunneris not built anymore. coverage now uses fakeroot when needed.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
unit tests: added fakeroot support
Tests ending in "-runasroot" or "-runasroot.py" are run through fakeroot,so activity that needs root privileges can be tested.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Extend RPC definition change with another parameter
This will be used in a later patch for parameter pre-processing. Fornow we just add the parameter as 'None' and add a consistency checkfor the parameter definitions.
build-rpc: Fail if call is defined more than once
check-python-code: Detect old-style pylint disable-msg lines
Commit b459a848d was supposed to replace all, but one was missed. Add acheck to autotools/check-python-code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
rpc: Fix issue with “test_delay”'s timeout
I passed the timeout calculation function in the wrong field of thedefinition. A small change is also needed in “build-rpc” to not abortwhen writing the docstring.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
RPC/test_delay: Use callable for timeout calculation
This avoids having to override the function in the RPC runner.
rpc: Move post-processor functions into definitions file
This way the generated code no longer contains arbitrary code.Post-processing functions are used by reference.
rpc: Use definitions directly instead of via generated code
Until now “autotools/build-rpc” would read the definition of all RPCsand write them to a new file, “lib/_generated_rpc.py” with somemodifications. With this patch the generated code loads the definitions...
Convert RPC definitions to dictionaries
This is in preparation to reducing the amount of generated code.
Fix pylint deprecated directive in check-header
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add tool to check Python file headers
Ensures licence and copyright headers are correct.
rpc: Improve argument definitions
Instead of in-place code now each argument has an (optional) kindassigned. The RPC client will then take care of encoding each argumentaccording to its definition. A new base (_RpcClientBase) is added totake care of encoding and dispatching calls....
Add script to generate RPC wrappers
A new script, “autotools/build-rpc”, will be used to generate code forRPC client wrappers. This is done because “lib/rpc.py” contains lots andlots of boilerplate code. Forthcoming patches will start convertingRPC wrappers....
Generalize docpp and sphinx_ext
Add error codes documentation
Merge branch 'devel-2.5'
Add check for standalone modules' imports
This check will abort if one of the standalone modules (currentlylib/rapi/client.py and tools/ganeti-listrunner) imports anything fromthe Ganeti source directory.
DeprecationWarning fixes for pylint
In version 0.21, pylint unified all the disable-* (and enable-*)directives to disable (resp. enable). This leads to a lot ofDeprecationWarning being emitted even if one uses the recommendedversion of pylint (0.21.1, as stated in devnotes.rst)....
check-python-code: Give location(s) of lines longer than 80 chars
Until now it would only say that there was a line longer than 80characters, but not where.
bash completion: Shorten function names
This saves another 5 kB from the generated script. An old debug commentwritten to the generated output is also removed.
check-news: Show per-file line number
… not the global line number.
bash completion: Group commands by arguments and options
This grouping, which was a TODO for a long time, reduces the script sizeby about 5kB.
Check option name format when building bash completion
This is just a style check. Option names should be consistent. Since allof them go through the “build-bash-completion” script, this seemed to bea good place.
Fix some issues in htools apidoc generation
First, auto-generated files were not processed if HsColour thus the'source' link in Haddock documentation was dangling.
Second, we document the original Python constant name in theGaneti.Constants source, so that it's easily retrievable (as opposed...
Implement conversion of Python constants to Haskell
With the merge of the repositories, we can now auto-generate the codefor Haskell constants from the Python code.
Currently this only handles the basic types (strings andintegers). Handling containers such as lists and dictionaries is only...
Enable htools unittests and coverage reports
Since we have two sets of coverage data, we make the namingconsistent:
- doc/py-coverage for Python coverage- doc/hs-coverage for Haskell coverage
We also need to exclude some more dirs from the list of checked...
Add script to generate query fields documentation
- All lines matching "QUERY_FIELDS_${resource}" in the input will be replaced with a definition list describing the fields for $resource- The core code is kept in the Sphinx extension, so that it could be...
QUERY_FIELDS_${resource}
Add script checking release dates in NEWS
This will detect human errors when setting a release date in NEWS.
Build epydoc's module list from Makefile
Querying node groups: add client/gnt_group.py
This implements `gnt-group list`.
Signed-off-by: Adeodato Simo <dato@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.3'
Conflicts: Makefile.am
Merge branch 'devel-2.2' into devel-2.3
Correct version check for release candidates
The tilde needs to be escaped and I forgot the space which should beused instead.
Check for editor-specific settings at build time
Fix version check
Don't ask … all I say is distcheck.
Add script to check version format
Only versions of the format “x.y.z” and “x.y.z~(rc|beta)N” (for N>0) areallowed.
Fix coverage reports
Currently, the coverage reports include the unittests themselves, andthis skewes unfairly the reports, as the coverage for the tests is veryhigh (since they all run).
To fix this, we export the ganeti temp dir from run-in-temp-dir, and we...
Remove last vestiges of docbook
This patch removes the last traces of docbook.
Conflicts: Makefile.am: Trivial
Makefile: Add new dist target for releases
A new script, autotools/check-tar, is used to check the resulting.tar.gz file for unwanted contents like wrong file owners orpermissions.
Add nodegroup bash autocomplation
We autocomplete both by nodegroup name and uuid.
cli: Use list of options shared between commands
The completion script for bash has to know about these options. Until nowthe list was in two places--once in cli.py and once inautotools/build-bash-completion. A shared list is used with this patch.
Move ShellWriter class to utils
Also add unittest.
Second attempt at fixing check-man
I was wrong, actually LANG-vs-LC_ALL only fixed one case, by mistake. Toget proper UTF-8 encoding, we need to enforce any UTF-8 locale. Wechoose the 'default' of en_US.UTF-8.