History | View | Annotate | Download (43.8 kB)
Add list of design documents implemented in version 2.6
Each version should have its dedicated list.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Make stable-2.6 compatible with newer pep8
This is done so that all current branches can run with newer pep8;note that instead of fixing the problems (like I did on master), I'vejust silenced more. These should not be merged onto master!
Signed-off-by: Iustin Pop <iustin@google.com>...
Add new Makefile target to rebuild the whole dist
Due to the fact how the automake system works it doesn't rebuild alreadyprebuild files in distcheck. This lead to a bug, where a rebuild of thedocumentation was failing because we missed the fact that the files were...
Add missing rst files to Makefile.am
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add a Makefile rule to test unclean git status
This is trivial, but can be used easily from automated builds to checkthat the git tree is clean: files are not modified and untracked filesare not present.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Update time on Haskell binaries after build
This patch works around an issue in our build system. Since we don'tuse cabal or 'ghc -M', we don't track actual dependencies in ourMakefile; this in turn means that editing a file that only is used inthe main 'htools' binary will keep triggering 'ghc --make' for the...
Fix building of hcheck man page
The code used a space, not a tab, in Makefile.am, which means itactually broke the variable declaring the man pages.
Furthermore, we add some blank lines in the man page, otherwise theparagraphs will be run-in together....
Add a shell test for hbal and split instances
This is not perfect, as we only test that hbal completes successfullyand that it show a score improvement, but it's better than nothing.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Initial commit for introducting hcheck tool
Introduce infrastructure required to add Hcheck and build itsuccessfuly.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix make -n use
Due to the way $(MAKE) behaves (it forces execution of the commandsthat contain it), use of $(MAKE) in a command line together withside-effects commands will break stuff. It's better to keep it eitheron a separate line, or if not possible, move said commands to a...
Enable hlint in lint and reorganise this target
Currently, the lint target does a sequential: pep8, pylint, pylint onthe QA sources. hlint is run only when explicitly requested.
This patch reorganises the lint target by:
- splitting the current lint target into separate pylint (slow),...
Remove one obsolete hlint override
During the preparations for 2.6, we cleaned up the codebase to usenewer exceptions (e.g. commits 30d25dd8, 79ac58fa), so we can now dropthis override.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'stable-2.5' into stable-2.6
Conflicts: Makefile.am (expected, from the parallel build fixes) qa/qa_config.py (due to cherry-pick, ignored, simply checkout out with --ours)...
Fix parallel build failures
This is the 2.5 version of the "fix build failures":
- man/%.gen could be left over even in case of failure, due to automake bug- make man/%.gen runs RUN_IN_TEMPDIR, so let's depend on it, since that target has the proper dependencies (create needed dirs)...
Add XEN_CMD to _autoconf.py
This is needed so we can specify the xen command upon configure time
Add checks from wrong hard-coded paths in manpages
This is trivial, but should prevent such things in the future.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reconcile Makefile.am and test data files
Sorry, forgot this in previous commit.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>(cherry picked from commit 1a1e7ab3f7cfe156152fb69961115a2c85b2a82d)
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initiallywrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force- otherwise refuses to proceed, but it still returns exit code 0...
Fix Makefile dependencies for building Constants.hs
I was wondering why my changes to luxi.py did not trigger the rebuild…
QA: Enable use of OR conditions in test checks
Until now “TestRunIf” and “TestEnabled” could only handle AND. With thispatch a new class named “Either” is added to “qa_config” and allows OR.The name “Either” was chosen instead of “Or” as the latter is very close...
Add example script for using RAPI test utilities
This script shows a few examples on how to use the RAPI input testclient. It is also run at “make check” time to ensure it's notcompletely broken.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
QA: Add tests for “gnt-job list”
Update default instance kernel version
We switch from vmlinuz-2.6-… to vmlinuz-3-…. To do this nicely, wealso add a ./configure-time setting for the KVM instance kernel.
Add a special lexer for sphinx/pygments
This will be used throughout our docs for better formatting exampleshell sessions, with custom markup for comments, user fixed input anduser variable input.
Enable -Werror by default for htools
Since the code base is now "clean" across all supported GHC versions(6.12-7.4), we can enable -Werror again.
Add trivial tests for gnt-* cli
While testing some other stuff, I realised that the gnt-* commandscould be broken (as in, the script fails with syntax errors), but makecheck doesn't detect it. Since we have shelltest, we can now addtrivial tests for this case....
Enable selection between Python and Haskell confd
This patch changes configure.ac Makefile.am so that the user can pass:
- --disable-confd (or --enable-confd=no) to disable it completely- --enable-confd=yes or --enable-confd=python to select the traditional implementation (this is the default setting)...
Fix out-of-tree builds
The new shell tests do not succeed out-of-tree, due to static pathsand other issues. This trivial patch fixes these issue, make distchecknow passes.
Convert manual shell tests to shelltestrunner
This is more of a RFC… Basically most of the shell-based tests areconverted from exec+grep to shelltestrunner.
Things are not all fine and nice though:
- we have dependencies between tests, as some generate some data files...
Only build hconfd if --enable-confd was passed
A later, more complete patch, will allow selecting between either thePython version or the Haskell version. This is just a temporarysolution to help building without all the needed Haskell libraries.
Build test helpers that point to hpc-htools
Instead of using just shell constructs to run hpc-htools correctly(i.e. HTOOLS=role htools/hpc-htools …), let's add some shell fragmentsthat do this for us.
This will ease the running of tests directly.
Add skeleton ssconf module
This currently has only one export function in it, which will be usedfor future bind address functionality in daemons.
Rename htools-hpc to hpc-tools to fix laziness
This is a stupid/trivial patch. I didn't realise when addinghtools-hpc that this will "break" tab completion on htools; as in, Iwill have to add / manually always when completing files on thecommand line....
Add htools/rapi test directory to DIRS
Sorry!
Add tests for custom text files
Add RAPI shell-level unittest
Add new hail (shell) tests
Using some hand-crafted files (manually modified from a real clusterrunning master branch), we test that hail behaves as we expect:doesn't load obvious wrong data, can do relocations/allocations, etc.
Fix make distcheck after the new shell tests
With the new shell tests, we have for the first time the situationwhere tests require a binary built (via make), but the binary itselfis not the test. Hence, we need to declare and use check_SCRIPTS forit....
Build a coverage-enabled version of the tools
This allows the offline tests to also generate coverage data, and thehs-coverage target is changed to show both unit-test and offline-testscoverage.
The downside is that now we build yet-another-binary, which makes the...
Enable htools offline tests as Haskell tests
This enables the offline-test.sh run both in the special "hs-check" rule and during normal "make check".
htools: add confd daemon binary
This is currently not installed anywhere, just built.
htools: add confd server module
This contains a more-or-less complete implementation for theserver-side confd.
Note that most of the code is behaving identical to the Python code,with a notable exception: the asyncore/inotify code was changed (sinceHaskell doesn't have asyncore) to 3-thread system. While otherwise I...
htools: add basic daemon-related functionality
This is not complete for now, just the basic functionality has beenimplemented:
- daemonize- check we're running under the correct user- call setup logging
htools: add logging functionality
This adds a dependency on hslogger 1.1+, which is not available inDebian Squeeze, only in later versions.
htools: add runtime functionality
This duplicates the lib/runtime.py functionality, allowing us to checkfor runtime users and groups consistency.
htools: add a small module for crypto functions
This adds a dependency on the following new libraries:
- Data.HMAC module from the 'crypto' library- Data.Text from the 'text' library- Data.ByteString from the 'bytestring' library
In Debian, the bytestring library comes with the ghc compiler, whereas...
htools: add definitions for confd types
While we have some of these as plain types in Constants.hs, we addproper ADT definitions for them in a new file. Furthermore, we add theConfdRequest and ConfdReply types here (in Python they are inobjects.py).
htools: add a very basic lib/config.py equivalent
This only is able to load the configuration data for now.
htools: add partial implementation of lib/objects.py
This is partial since not all object types can be easily converted fornow (will need some changes on the Python side for this).
Most importantly, the *Params types do not have a good solution now:the Python code, due to its dynamic typing, hides the fact that we...
Small improvement for nicer unit-testing
In case you just run "make htools/test" to recompile the unittests,the old .tix file should be removed, otherwise you get the messageabout inconsistent tick boxes. Let's make the compilation processautomatically remove this, for easier testing....
Merge remote-tracking branch 'origin/devel-2.5'
Conflicts: Makefile.am (trivial, test data files added on both branches) lib/opcodes.py (trivial, master renamed filter→qfilter, 2.5 fixed the type of the parameter)...
Split check-man-warnings into two
This moves the dashes check to a new check-man-dashes script.
Rename check-man to check-man-warnings
Currently the check-man test is dependent on the MAN_HAS_WARNINGStest, which made sense as long as the script only did the warningscheck.
However, we should run the [em] tests always, and in general we couldrun other tests too, even if man doesn't support --warnings; so let's...
Makefile: improve the fix for .DELETE_ON_ERROR
I've investigated more and it seems that .DELETE_ON_ERROR not workingis a bug/regression in some versions of make; testing with make 3.82from Debian experimental makes this special target work asexpected. Also, make 3.81 works on trivial Makefiles, but not on our...
Explicitly remove the .x.in man pages on failure
hinfo: Adding man page
hinfo: Adding basic skeleton based on hbal
It prints the information which hbal prints in a nice formatted manner and ofcourse without doing any actions. An example output looks like this:
$ hinfo -m xen.example.comLoaded 4 nodes, 2 instancesCluster has 1 node group(s)...
Merge branch 'stable-2.5' into devel-2.5
Fix Makefile.am compatibility with automake 1.11.2
Automake 1.11.2 made the following change:
Unfortunately, this breaks our Makefile.am (issue 216) exactly because...
rapi.testutils: Add exported functions to verify opcode input/result
These can be used by third-party code to verify mock code. Further workon mocks is forthcoming, so this is only a start.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Makefile.am: fix permissions for Python scripts on install
Some Python scripts in /usr/lib/ganeti/ were getting the wrong permissions(their 'x' bit was cleared). This patch fixes that behavior.
This patch renames the variable 'dist_tools_PYTHON' to 'python_scripts'....
fmtjson: A tool to format JSON data
While debugging an issue recently I had the pleasure of looking at along, single-line string of JSON data. This simple utility makes itpossible to format such data into a readable format.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Split the list of program personalities into a module
… a mini-module, actually. This is only done for unit-testing, as wedon't want to duplicate the personalities list in the tests.
Add a Makefile target for profiling targets
This simplifies the building of profiling-enabled htools binaries;there are two targets, one used for the initial two-stage build, andone that can be used for incremental builds.
The patch adds a new internal variable for the htools/test binary,...
htools: split parts of HTools/Types.hs into BasicTypes.hs
The 'Result' type is common and:
- might be used outside of HTools-specific code too- is better split as we need these basic types for building the more complex ones in Types.hs
Merge branch 'devel-2.5'
Conflicts: NEWS: trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
KVM: support version reported by 1.0
This of course was working for all the rcs, but broke with 1.0 itself.
In addition: - split between running kvm --version and parsing its output - unittest parsing for various known --help outputs - updated NEWS file...
Fix failure in distcheck and coverage tests
autotools/testrunner was missing in distcheck. Now some changes from7004106119c5c56c93085ccfb1250b3561b5b6ad have been reverted, and testrunneris not built anymore. coverage now uses fakeroot when needed.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Makefile: Fix parallel distcheck
Running “make -jN distcheck” for any N > 1 would reliably break whenbuilding the RPC client wrappers. This was due to wrong dependencies inMakefile.am, fixed by this patch.
Add lock performance utility
I had an idea for improving locking performance. To see if it worked Iwrote this tool. Unfortunately the idea didn't quite work (brokeunittests left and right), but the tool is still handy for evaluatingfuture changes to the “SharedLock” class....
unit tests: Add tests for uid and gid handling in utils.WriteFile
These tests need fakeroot. If it's not present, they are not run (theydon't fail).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
unit tests: added fakeroot support
Tests ending in "-runasroot" or "-runasroot.py" are run through fakeroot,so activity that needs root privileges can be tested.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Add infrastructure for, and two extra hlint rules
This will allow us to easily use add new lint rules that westandardise on, if any.
Cleanup hlint errors
First, we update the recommended hlint version to what I used to get aclean output (1.8.15). Most of the changes are:
- remove unneeded parentheses- some simplifications (intercalate " " → unwords, maybe … id → fromMaybe, etc.)- removal of some duplicate code (in previous patches)...
Unify some file lists in Makefile.am
These were repeated needlessly; I hope I grouped them correctly.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
Add DRBD barriers disk parameters
Add the disk-barriers and meta-barriers parameters described in thedesign doc.
Add a small confd client
This can be used to test live servers; currently there's not directway to interact with a confd server, except for burnin's builtin tests(which were the source of this file).
Add toggle for enabling/disabling confd
Doesn't do anything yet.
Add the default master-ip-setup script
Add a shell script that will be the default replacement for the body ofbackend.ActivateMasterIp and backend.DeactivateMasterIp.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add tool to check Python file headers
Ensures licence and copyright headers are correct.
Rename lib/build/rpc_definitions.py to lib/rpc_defs.py
It will also be used at runtime, not just for building.
Add script to generate RPC wrappers
A new script, “autotools/build-rpc”, will be used to generate code forRPC client wrappers. This is done because “lib/rpc.py” contains lots andlots of boilerplate code. Forthcoming patches will start convertingRPC wrappers....
Add unittest for PycURL 7.19.0's reset issue
While we won't use this function in Ganeti (yet), users should be awareof the potential crashes caused by this functioni (discussed offline).If the installed version of PycURL is not affected, this test is a...
Check for qemu-img in configure and Makefile
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Revert "Rename utils.mlock to utils.cfunc"
The rename is not needed either, since we're not adding more code as ofnow.This reverts commit 57ca011e1cd2681948969724e2646edaac22da28.
Revert "utils.cfunc: Cleanup, more flexibility"
We discussed that this is not needed right now, and it breaks existingfunctionality and unittests.
This reverts commit 6915fe26da8dce41fc967d761f005390aa956161.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
utils.cfunc: Cleanup, more flexibility
- Split code using ctypes directly into a helper class- Don't load “libc.so.6”, but use handle for main program instead (see comment in code)- Clarify comment on errno with older ctypes versions- Rename unittest since it can't be used for other functions (modifies...
Rename utils.mlock to utils.cfunc
Renaming so that more code using ctypes could be added to the same file.
Add design doc for virtual(ised) clusters
I am currently able to run a 2-node virtual cluster on my machine,with a very ad-hoc setup. But the results show clearly that this isdoable, and that given the right tools, setting up such a cluster willbe quite easy....
Document some useful Haskell tips
This improves devnotes.rst with some tricks for Haskell development,and additionally it does two Makefile improvements:
- properly document lib/_vcsversion.py as a requirement for Constants.hs (but do not require rebuild when updated)...
Add design doc for the resource model changes
This is not complete, but is as close as I can get it for now. Iexpect people actually implementing the various changes to extend thedesign doc.
Fix Makefile rules for QCHelper.hs
Include QCHelper.hs in the distributed files, and also exclude it andthe THH.hs file from coverage reports.
Use TemplateHaskell instead of hand-coded instances
This patch replaces the current hard-coded JSON instances (all alike,just manual conversion to/from string) with auto-generated code basedon Template Haskell(http://www.haskell.org/haskellwiki/Template_Haskell)....
Split part of Utils.hs into JSON.hs
Utils is a bit big, let's split the JSON stuff (not all of it) into aseparate module that doesn't have any other dependencies.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Export: unittests
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>