Remove the HTOOLS configuration variable
.. and update the code that uses it.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Tear down Py2Hs constant infrastructure
Tear down Python to Haskell constant conversion infrastructure, whichincludes eliminating the autotool 'convert-constants' and the Haskellmodule, namely 'Ganeti.PyConstants', which held the convertedcontants.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Hs2Py constants: additional module jstore
Add constants from additional modules ('ganeti.jstore') to the Haskellto Python constant generation.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Hs2Py constants: additional module errors
Add constants from module 'ganeti.errors' to the Haskell to Pythonconstant generation.
Hs2Py constants: additional module qlang
Add constants from additional modules ('ganeti.qlang') to the Haskellto Python constant generation.
Hs2Py constants: additional module luxi
Add constants from additional modules ('ganeti.luxi') to the Haskellto Python constant generation.
Remove module 'ganeti.constants' from 'convert-constants'
Given that there are no more constants in 'ganeti.constants' toconvert to Haskell, this module import and conversion logic can beremoved from the autotool 'convert-constants'.
Python autotool to print socket constants
Add Python script to autotools which prints socket related constants.These constants are a transitional solution for eliminating the Pythonto Haskell constant generation given that the final solution willrequire a change in the configuration and, therefore, postponed to...
Hs2Py constants: update Python references
Update references to constants that were previously in the generated'lib/_autoconf.py' and since the previous patch have been included inthe Hs2Py constant generation.
Import original constants instead of generated
Haskell to Python constant generation introduced duplicated constantsin Haskell. This patch eliminates the duplication and fixes Haskellto import the correct constants, instead of the generated ones.
Hook h2spy in Makefile.am
Eliminate Haskell constant for opcode names
Modify 'autotools/convert-constants' not to generate the Haskellconstant that holds all opcode names. By having Haskell generate thePython opcopdes, it becomes impossible to generate this constant dueto a cyclic dependency. However, this constant is used only in tests,...
Allow alpha versions not to have an entry in the NEWS file
Alpha versions are still under heavy development, and can therefore not have anentry in the NEWS file yet. The entry will have to be added before switching tothe first beta version.
Partially fixes Issue 448...
Introduce support for alpha versions
An alpha version can be used to name a release while a previous one is stillbeing developed.
E.g. 2.9.0~alpha1 can be out while 2.8.0~rc1 is being worked on.
When a version is marked as alpha, it is allowed to have unreleased versions...
Improve docstring
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Merge branch 'stable-2.7' into stable-2.8
This fixes the problem with the distcheck-release target failing needlessly.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Improve check for "unreleased" versions in NEWS
Currently this is checked only when distcheck-release is called.Check it in check-news instead, for all versions before the "current" one (as defined by configure.ac)
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Merge branch 'devel-2.7'
check-news: fix behavior on invalid date line
Before:$ ./autotools/check-news < ./NEWSTraceback (most recent call last): File "./autotools/check-news", line 150, in <module> main() File "./autotools/check-news", line 120, in main parsed_ts = time.mktime(time.strptime(m.group("date"), "%d %b %Y"))...
Add user and group for the monitoring agent
This commit adds the user and group variables for the monitoring agent in allthe configuration files where they are required.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix bug in man build rule
Currently the man page build rule is run without ‘-e’, so the actualchecks in there are just warnings (check-man-warnings,check-man-dashes).
The patch fixes this and also fixes the man pages which made me see theproblem.
Additionally, check-man-dashes is now verbose, otherwise it's hard to...
convert-constants: generate better Haddock markup
Currently the Python source constants are used as-is, which is notnice on the eyes (especially for unqualified names, it was hard toseparate them visually).
Signed-off-by: Iustin Pop <iustin@google.com>...
Fix convert-constants handling of booleans
It turns out that, in Python, booleans are also integers. So they fallunder the “isinstance(value, int)” case, resulting in all enable*constants being integers in Haskell, which is not nice, even thoughwe're not using them directly today....
check-news: Ensure release date is within three days
Commit b6ad806f contained a date a month in the future. With this patchall release dates in NEWS may at most be three days into the future(assuming the build machine's clock is correct).
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Revert "run-in-tempdir: Improve speed"
This reverts commit dd60179dfd8 as it broke coverage for Python.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
run-in-tempdir: Improve speed
By creating symlinks instead of actually creating files we still get thebenefits of using a temporary directory, but the preparation is aboutthree times as fast. Test used:
$ time bash -c 'for ((i=0; i < 100; ++i)); do autotools/run-in-tempdir true; done'...
run-in-tempdir: Copy doc/* only if needed
To preserve disk space on copying all files to a temporary directory,the “doc” directory is only copied when building documentation. Acomment describing the need is also added to Makefile.am, as well as asuperfluous direct invocation of Bash removed....
Move burnin to ganeti.tools.burnin
Similar to other scripts, “tools/burnin” just becomes a wrapper script.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Option to include man pages in documentation
Before this patch, HTML versions of man pages (man/*.rst) were alreadybuilt. However, since they are separate from the normal documentation,their content is not indexed for Sphinx' search functionality.Additionally it would simply be nice to have everything in one place....
Add script to check man page references
This script checks for some of the most obvious mistakes when formattingman page references (which should have the form “**ganeti**\(7)”). Whilethis works now, it is very hard to avoid ambiguities (e.g. references...
A bit more name cleanup
- gen-coverage → gen-py-coverage- coverage-htools.tix → coverage-hs.tix
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Move htest/ files under the test/ tree
htest/data becomes test/data/htools (basically reverting commit8feabc89), and htest/* becomes test/hs/*.
Most changes beside the rename are trivial s/…/…, with the exceptionof autotools/run-in-tempdir, which needed some more changes now that...
Move python test files to test/py
This is the first step of the test files reorganisation: moving test/*(except test/data) to new directory test/py/.
Rename htools/ to src/
Per offline discussions, this is the first patch of therenames. Tested with "make distcheck", seems to work fine.
The only change outside of the renaming is a bit of simplification inthe .gitignore rules; otherwise, simply s/htools/src/....
Optimise autotools/run-in-tempdir
I made a mistake when adding support for htools to run-in-tempdir, byneedlessly copying the htools binaries, when it's enough to linkthem. The copying is only really needed for python code, so we changethe Haskell binaries to linking and also a few other directories....
Add the gnt-storage client
Add a new client called 'gnt-storage'.The client interacts with the ExtStorage interface, similarly tothe way gnt-os interacts with the OS interface.
For now, only two commands are supported: 'info' and 'diagnose'.
'diagnose' calculates the node status of each provider on each node,...
Merge branch 'devel-2.6'
Add hroller htools personality
This is a new personality that for the moment doesn't do anything.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix job completion with big job queues
Accidentally stumbled upon this while testing unrelated code on amachine with ~3K active jobs - the bash completion unittest washanging.
Upon investigation, it turns out that bash's ${var//pattern/repl/} isprobably quadratic in the size of input (or worse, even):...
Enable bash completion for mon-collector
This just ties in all the pieces so far and enables bash completionfor it.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
bash-completion: add support for multi-cmd Haskell binaries
This patch adds support for parsing the command list out of a binary(very strict format), and then iterating over that and building thesub-commands options/arguments.
It also does a bit of general cleanup in the script....
Export jstore constants as well to Constants.hs
This single constant could be as well moved to constants.py. For now Ichoose to just export the module as well.
Generated RPC code: Don't write whitespace after paren
There is no obvious need for this space--the line would wrap at the nextspace, which comes soon after.
Add shelltests for the mon-collector
This commit adds shelltests for the mon-collector binary and for the DRBDdata collector.
Also, it fixes a small bug in the DRBD parser found thanks to the tests.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Enable equivalence checks of opcode list
This patch enhances the opcode list checks - instead of spawning aPython interpreter to display the opcode list, we export it staticallyin Constants.hs via a slight convert-constants change.
Furthermore, since we now have opcode parity, we enable full opcode...
Include networks in Makefile and bash completion
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Add custom code for CV_E* constants to convert-constants
Currently, the cluster verify errors are defined as follows:
CV_ECLUSTER_FOO = (TCLUSTER, "ECLUSTER_FOO", "description")
This means there's no standalone name for the string "ECLUSTER_FOO",which makes it hard to derive automatically a type for this union in...
Sort content of generated RPC code
This is similar to commit f5ce761. A stable sort order makes changesbetween versions easier to find.
Find coverage during configure, issue with Debian package
- Debian Squeeze and up have a package named “python-coverage”, but it doesn't use the same binary name as upstream (“coverage”).- Said package includes a patch to use symlinks instead of file copies...
Enable conversion of errors constants
These will be needed for the serialisation of errors.
bash_completion: Fix mistake introduced in merge 6a74f58
This part of the code was duplicated by accident.
Improve Haskell configure options and detection
This patch cleans up the Haskell library detection and defaults.
First, it makes the base compiler/libraries required, per the email discussion.
It then adds two new small autoconf macros, on to check for a required...
Merge ganeti-master-cleaner back into ganeti-cleaner
As I wrote during/after the review on commit 2958c56, “ganeti-cleaner:Separate queue cleaning code”, while I appreciated the permissionseparation, I didn't like too much the file-based approach:
- it is a very simple script, and lots of the code is duplicated...
Split the OptComplNumeric into integer/float ones
This allows the command completion to treat them as separate optiontypes; this is not useful in build-bash-completion right now, butmight become so in the future.
Enable bash completion for Haskell daemons too
This requires a few tiny changes to build-bash-completion and toMakefile rules, but is straightforward.
Support the new option types in build-bash-completion
Beside parsing the arguments as well, we also change the serialisationformat for choices/suggests, to make the Python code simpler.
Always distribute ganeti.utils.io_unittest-runasroot.py
Tests that depend on fakeroot were not distributed in the tarball createdby "make dist" when fakeroot was missing. Now all tests are distributed,and a warning is printed when a test requires fakeroot but fakeroot is...
bash_completion: Enable extglob while parsing file
In older versions of GNU Bash extended patterns, such as “@(…)”, are onlyavailable with the “extglob” shell option. As pointed out in [1] and [2],“extglob” must be enabled while parsing the code. Therefore the flag must be...
Enable bash completion for htools scripts
This patch ties together the previous commits, by extendingbuild-bash-completion to call all htools scripts and get theircompletion information, then use that to build fake cli_optionsrepresenting them and finally generate the bash completion...
Revert "bash_completion: Always enable extglob"
This reverts commit 94014b6328f813b674a1126ffd221705d7e315b3. Turns outI misunderstood the “extglob” shell option.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
bash_completion: Generate more compact version
First the numbers:$ stat --format '%s %n' doc/examples/bash_completion*77847 doc/examples/bash_completion86492 doc/examples/bash_completion-debug
The non-debug version doesn't use indentation and does not have some...
bash_completion: Always enable extglob
In older versions of GNU Bash extended patterns, such as “@(…)” are onlyavailable with the “extglob” shell option. This patch adds a wrapperfunction so that extglob is always enabled while doing any completion.Due to early returns inside the main completion function this seemed...
Use autoconf-based paths from Haskell instead of constants
Future changes will change Path.hs to use an environment variable.
Migrate build-bash-completion from constants to pathutils
File system paths moved from constants to pathutils.
Move generated htools test stubs to htest/
This moves the last (I think) htools-related bits out of test/ underhtest/.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Move haskell test code to htest/
This is the first commit of a series that will attempt to cleanup thetest code organisation, which evolved somewhat organically from theinitial pure htools functionality.
The proposed organisation of the tree will be as follows:...
Extend convert-constants support for dicts
This enhances convert-constants to not flatten dicts completely, butalso generate a so-called association list for them.
This allows either direct use of the 'lookup' function, or (forperformance) conversion to Data.Map and optimised lookup later....
Enhance convert-constants list generation
While looking at the hypervisor types in Constants.hs, I saw that the'hyperTypes' list is using strings instead of names. This is due tothe fact that we require the entire elements in the list to beidentified (homogeneous lists), but the string "xen-pvm" is declared...
convert-constants: also export qlang.py constants
Since these will be needed for Haskell's implementation of query2.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Ensure a stable content of the bash completion file
Currently, the order of commands in the bash completion file israndom, because the sub-commands are not sorted. This makes it harderto investigate the differences in packaged Ganeti or in installedGaneti, since chunks in this file will have a random order....
Improve the check-news script
Allow all errors to be displayed, instead of aborting at the firstone, and don't show stacktraces anymore.
Add checks from wrong hard-coded paths in manpages
This is trivial, but should prevent such things in the future.
Implement reverse mapping of values to names
This adds a bit of dumb mapping of values to names, while trying to besafe. This is the best we can do without resorting to parsing orinterpreting ASTs.
The difference in the output is:
-- | Converted from Python list or set ADMINST_ALL...
Implement support for multi-module export
This add support for exporting constants from multiple modules (asopposed to hard-coding constants), and also makes the output morereadable by skipping things we know for sure we don't want to convert(as opposed to things we would like to but don't know how to...
QA: Enable use of OR conditions in test checks
Until now “TestRunIf” and “TestEnabled” could only handle AND. With thispatch a new class named “Either” is added to “qa_config” and allows OR.The name “Either” was chosen instead of “Or” as the latter is very close...
Add example script for using RAPI test utilities
This script shows a few examples on how to use the RAPI input testclient. It is also run at “make check” time to ensure it's notcompletely broken.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rename htools-hpc to hpc-tools to fix laziness
This is a stupid/trivial patch. I didn't realise when addinghtools-hpc that this will "break" tab completion on htools; as in, Iwill have to add / manually always when completing files on thecommand line....
Build a coverage-enabled version of the tools
This allows the offline tests to also generate coverage data, and thehs-coverage target is changed to show both unit-test and offline-testscoverage.
The downside is that now we build yet-another-binary, which makes the...
Enable htools offline tests as Haskell tests
This enables the offline-test.sh run both in the special "hs-check" rule and during normal "make check".
Split check-man-warnings into two
This moves the dashes check to a new check-man-dashes script.
Rename check-man to check-man-warnings
Currently the check-man test is dependent on the MAN_HAS_WARNINGStest, which made sense as long as the script only did the warningscheck.
However, we should run the [em] tests always, and in general we couldrun other tests too, even if man doesn't support --warnings; so let's...
Add automated checking for non-escaped --
This checks to see if any man output has [em] in it, as that will notbe correctly show in ASCII. Unfortunately the check will depend onwhether the other man check is done, but I didn't want to introduceyet another short script for this (can be done, no issue)....
Add support for RE patterns to convert constants
This is a trivial conversion.
Add support for lists/frozensets in convert-constants
Unfortunately, we only support lists of simple types, and not evenlists of tuples. If we actually needed those, it would be possible toimplement them, with a bit more complexity in the converter.
Add support for tuples in convert-constants
More improvements to convert-constants
This prepares for tuple and other conversions.
Improve convert-constants to handle dictionaries
The two main drawbacks for convert-constants are the fact that itcan't handle sets/frozensets (mainly due to the fact that I don't knowhow useful this would be to the Haskell code) and that it cannotexport dictionaries....
gen-coverage: Change filter
It is not necessary to run a Python script to generate the filter, whichdidn't work very well anyway on systems with modules in/usr/share/pyshared.
check-news/NEWS: impose more standards
- Always two empty lines before a version.- Standardize how version names should be called.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
rpc: Add support for resolver options callback
This adds support for a callback returning name resolver options. Thisis required for powercycling offline nodes. While it would be possibleto implement this using a separate RPC client class, doing so would...
Fix failure in distcheck and coverage tests
autotools/testrunner was missing in distcheck. Now some changes from7004106119c5c56c93085ccfb1250b3561b5b6ad have been reverted, and testrunneris not built anymore. coverage now uses fakeroot when needed.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
unit tests: added fakeroot support
Tests ending in "-runasroot" or "-runasroot.py" are run through fakeroot,so activity that needs root privileges can be tested.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Extend RPC definition change with another parameter
This will be used in a later patch for parameter pre-processing. Fornow we just add the parameter as 'None' and add a consistency checkfor the parameter definitions.
build-rpc: Fail if call is defined more than once
check-python-code: Detect old-style pylint disable-msg lines
Commit b459a848d was supposed to replace all, but one was missed. Add acheck to autotools/check-python-code.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
rpc: Fix issue with “test_delay”'s timeout
I passed the timeout calculation function in the wrong field of thedefinition. A small change is also needed in “build-rpc” to not abortwhen writing the docstring.