History | View | Annotate | Download (52.7 kB)
Makefile: Fix list of directories
Commit dbee5c9 had a wrong path.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Highlight external links in HTML documentation
Based on MediaWiki's monobook skin.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Unit tests for bdev and utils.lvm
Also amended a comment of a tested method.The tests are for methods introduced in commit 63c73073 (LVM disk creationuses dedicated PVs).
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Support instance-minor pairing in the DRBD collector
This commits enables the DRBD data collector to use the Confd client togather information about the pairing between DRBD minors and instances.
For testing purposes, the DRBD data collector now requires either zero...
Add footer.rst as a dependency for all man pages
Commit 6aff051 removed an intermediate step, but forgot to add“footer.rst” as a direct dependency.
Remove intermediate footer.man/footer.html files
I don't remember why I introduced these when switching to pandoc, butthey are not absolutely needed. Instead of pre-generating them (whichmight save some trivial cpu time), we can simply pass footer.rst as an...
Enable a table of contents for the html version of man pages
This makes the (very dull) html version of the man pages a bit morereadable, until such a time that we can switch over to sphinx fully.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Stop manually building images out of dot files
The first dot files we added way back in the 2.0 release are managedmanually: we build the .pngs ourselves, track them in .gitingore,etc. Since then, we have switched to allow sphinx to build/manage themitself, so it makes sense to cleanup the build system and covert...
Inline the standalone dot files
This removes entirely the standalone dot files by inlining them in theRST documents - as they weren't referenced from more than one documenteither way.
Add small webserver for development
When working on documentation it can be helpful to use a browser. Insome environments it's not possible to access the files directly from agraphical browser. This trivial webserver exports all files in thecurrent directory and unlike alternatives such as thttpd, doesn't check...
Makefile.am: Add missing space
Commit acaf3ca missed this space before a backslash.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix the installation location of mon-collector
The mon-collector binary was erroneously installed twice: once in/usr/bin/ and once in /usr/lib/ganeti/.With this modification, it is now installed in /usr/lib/ganeti/ only,as intended.
In order to do this in a proper way, some parts of the Makefile.am where...
Move src/Ganeti/HTools/Program.hs to Program/Main.hs
This removes one more tab conflict; this is the last module in ourcode where we have both x.hs and x/.
Furthermore, we collapse all actual code into the new Main.hs module,leaving the htools.hs basically empty (will allow better testing in...
A bit more name cleanup
- gen-coverage → gen-py-coverage- coverage-htools.tix → coverage-hs.tix
Split the hs-check test rule
Currently, both the unit test and the shell tests are run (manually)from this rule. This has two downsides: you can't (compile and) runjust one of them easily, and make can't run them in parallel.
By splitting the rules, we can both of the above easily....
Rename test/hs/test to htest
The current base name of 'test' means that this binary will generateprofiling, coverage, etc. files with the name 'test.*', result intab-completion conflicts with the test directory. Let's slightlychange its base name for typing easiness....
Move htest/ files under the test/ tree
htest/data becomes test/data/htools (basically reverting commit8feabc89), and htest/* becomes test/hs/*.
Most changes beside the rename are trivial s/…/…, with the exceptionof autotools/run-in-tempdir, which needed some more changes now that...
Move python test files to test/py
This is the first step of the test files reorganisation: moving test/*(except test/data) to new directory test/py/.
Replace htools→hs in configure/makefile rules
This continues the cleanup of the 'htools' name, by replacing 'htools'with 'hs' in configure and makefile rules.
There is still 'WANT_HTOOLS' left as it is correct, but some of theconditionals in Makefile.am will need some cleanup (in a different...
Fix dependency on built python sources for hs-check
Since we call into Python for opcodes and other type compatibilitychecks, we need to ensure that the python sources arebuilt. Otherwise, full parallel make might or might not break,depending on timing....
Rename htools/ to src/
Per offline discussions, this is the first patch of therenames. Tested with "make distcheck", seems to work fine.
The only change outside of the renaming is a bit of simplification inthe .gitignore rules; otherwise, simply s/htools/src/....
Fix building of Constants.hs w.r.t .pyc files
Unless this is run under run-in-tempdir, it will create as usual the.pyc files.
Add the gnt-storage client
Add a new client called 'gnt-storage'.The client interacts with the ExtStorage interface, similarly tothe way gnt-os interacts with the OS interface.
For now, only two commands are supported: 'info' and 'diagnose'.
'diagnose' calculates the node status of each provider on each node,...
Add ExtStorage related man pages
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Extend the hcheck shell tests
This adds a few new hcheck tests, to extend the coverage.
Extend the hbal shell tests
After this patch, almost all of hbal except the actual Luxi executionis being covered by unit- or shelltests.
Fix build after addition of 'empty-cluster.data'
Of course I forgot to add this to TEST_FILES… and only distcheckcatches it.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Added class to contain information about a PV
This makes the code more readable and easier to upgrade.bdev.LogicalVolume.GetPVInfo and the code that depends on it have beenrefactored to use the new class.
utils.CheckVolumeGroupSize() has been moved to lib/utils/lvm.py, where more...
Add Confd client to the Haskell code base
The client queries all the master candidates in parallel, until the minimumnumber of replies, defined in the constant file, is received.A timeout prevents the waiting from being of indefinite length.
The reply to be returned to the function that made the query is decided...
Implement the External Storage Interface
With this commit we introduce the External Storage Interfaceto Ganeti, abbreviated: ExtStorage Interface.
The ExtStorage Interface provides Ganeti with the ability to interactwith externally connected shared storage pools, visible by all...
Add hroller htools personality
This is a new personality that for the moment doesn't do anything.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add support for job queries in hconfd
This adds support for job queries, including (basic) unit-tests.
I've tested this for memory and cpu usage as follows:
- 3600 jobs (live queue): - via masterd, default: ~1.1s (masterd: ~60MB ram) - via confd, default: ~1.1s (hconfd: ~25MB ram)...
Add a read-only job queue module
This patch adds implementation for a read-only job queue module,together with "full" test (as full as can be in a lazy language…).
One note about the behaviour of the job queue is the handling ofopcodes that fail validation: the 'input' opcode actually is a...
Enable bash completion for mon-collector
This just ties in all the pieces so far and enables bash completionfor it.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add more mon-collector shell tests
These test the newly-changed behaviour, and fix an inconsistency inthe hs-check target (versus hs-coverage).
Remove two hlint overrides
"Use comparing" was not needed, and "Use on" can be solved by actuallyusing on in the single case we have (the patch also reorders importsin that file).
Add a custom CLI module for DataCollectors
Currently, the mon-collector binary uses the HTools/CLI module, whichis OK but mean it links in lots of htools code. By copying that moduleto DataCollectors/CLI and removing the unneeded code, we reduce thenumber of modules it depends on fro 20 to 12, meaning both a shorter...
Fix profiling targets
As noted by Guido, there are problems when using the hs-prof andhs-prof-quick targets in the default configuration (compiling allprograms). The errors manifest in the form of wrong symbols duringcompilation.
I knew that the hs-prof targets, which compiled multiple objects with...
Split and extend Confd types tests
Some confd types were not tested for serialisation, so let's movethese (type-related) tests to their own module and extend them.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Export jstore constants as well to Constants.hs
This single constant could be as well moved to constants.py. For now Ichoose to just export the module as well.
Add version check for doc/design-draft.rst
This is similar to README or “doc/security.rst”. Having a comment thatneeds updating for a new version is more likely going to trigger a checkof the lists of design drafts.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Makefile/check-local: List all errors before failing
Use the “error” variable in multiple places and fail only once all testshave been completed.
Fix a bug in the hs-coverage rule of Makefile.am
The coverage information for mon-collector was not merged correctly.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add shelltests for the mon-collector
This commit adds shelltests for the mon-collector binary and for the DRBDdata collector.
Also, it fixes a small bug in the DRBD parser found thanks to the tests.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add the stand-alone data collector for DRBD
It is implemented as a single binary receiving, as its first parameter,the name of the actual data collector to be run.This way, its structure can be used for all the future data collectors.
Also, factored out of bdev.py into constants.py the location of the DRBD...
Network and address pool (again)
Implementation of the network objects and address pool. Functionalityas in the reverted commit b9a616e132af9bba718d2da8c94eeec7af886814, butnow using only the vector library and not the bit-vector library.Tested with vector library version 0.9 and 0.10., which are...
Make htools/Ganeti/Constants.hs depend on errors.py
It also uses values from “lib/errors.py”.
Revert "Network and address pool"
This reverts commit b9a616e132af9bba718d2da8c94eeec7af886814, whichdepends on the "bitvec" library. We need to investigate a bit further thatdependency, as it in turns depends on "vector" 0.9.1 or below, since0.10.* removed support for Data.Vector.Unboxed.Safe which it uses....
Network and address pool
Implementation of the network and address pool class inHaskell. Not complete yet. Includes unit tests that coverall functionality that is so far implemented.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add tool to clean up node
Sometimes a node is not removed properly from a cluster (especiallyduring development). This new tool stops all daemons and removes (aftermaking copies) the most critical files.
Add tool to configure node daemon
The design for this is in “doc/design-node-add.rst”. The tool receives aJSON data structure on stdin and configures the node's daemon afterverifying the received values.
Makefile: Reset environment for epydoc
epydoc allows overriding configuration values via environment variables.While this might be useful in certain use cases, but as no prefixwhatsoever is used, conflicts are easily created. Some people have theenvironment variable “NAME” set, effectively overriding the project name...
Add Ganeti.HTools.Graph
This module implements some algorithms on Data.Graph data structures.At the moment its main functionality is an LF-color implementation(greedy coloring in descending order of degree). There are also a fewextra functions to calculate the degree order, and convert the node to...
Small Makefile.am cleanups
Indentation within rules is fixed to always be 2 spaces (as opposed tomixed tabs/2 spaces/4 spaces). Additionally, the check-dirs rule,which was more complex than necessary, has been simplified: movedauto4mte.cache to DIRCHECK_EXCLUDE, and simplify the find call and the...
Fix long lines in Ganeti.HTools.Types and add Make check
Since hlint doesn't check style issues but rather code only issues,let's add a simple Makefile check for too-long lines, and fix thesingle bad case we have currently.
Signed-off-by: Iustin Pop <iustin@google.com>...
Two small haskell-related Makefile improvements
Just some cleanup:
- don't run hlint over hpc-htools, since it's just a symlink to htools.hs (so we'd get duplicate warnings)- build rpc-test in make really-all, via HS_ALL_PROGS variable
Improve the Haskell api doc directory creation system
Instead of having two different lists of directories, now there is only one.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Design for opportunistic locking
Add the design document for opportunistic locking.
Add unit tests for the JSON serialization of DRBD status
The serialization itself is done by Text.JSON, so the tests deal with checkingthat Text.JSON objects are created correctly from the DRBD parser datastructures.
Fix Haskell profiling targets
It turns out that in newer GHC versions, the suffix shouldn't beprefixed with a dot; otherwise name mangling is broken.
Fix Attoparsec test coverage check compatibility
The hpc tool, on squeeze, does not handle utf8 encoded files correctly.The Attoparsec test is just needed to verify whether the parser handlesuft8 correctly, so the solution is to exclude it from coverage checks....
ssconf: Verify file size when reading, add some tests
Until now ssconf would limit the amount read from files to 128 KiB andsilently ignored files larger than that. With this patch a check isadded by using fstat(2) on the file descriptor while it's being read....
Rename configure option for restricted commands
It is no longer “--enable-remote-commands”, but rather“--enable-restricted-commands”.
Include networks in Makefile and bash completion
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Iustin Pop <iustin@google.com>
Add DRBD parser unit tests
This adds tests that existing test files can be parsed by the Haskellparser as well, plus one new test file.
Add parser for DRBD /proc file
Split OpCode.hs and add module for opcode parameters
Due to TemplateHaskell stage restrictions, we can't define parametersin the same module as we're using them for TH, so we have to defineall module parameters in a separate module.
This patch therefore splits OpCodes.hs in two, adding that module and...
Create a new Ganeti/Types.hs module
There are already three cases where we copied type definitions betweenthe htools-specific types into the main ganeti code. Let's stop doingthis ☺ and create a common types module that holds these.
Note that there already exists BasicTypes.hs, but that refers to very...
configure: Shell cleanup
- Remote unnecessary quote characters- Change some uses of “test "$foo"” to “test n "$foo"” Merge “if/else/if” into “if/elif”- Wrap lines longer than 80 characters and add a test to “make check”
Prevent hlint crash on unicode chars
Some versions of hlint crash on the Attoparsec test file because theyare not able to correctly handle unicode characters if the proper encoding(--utf8) is not specified.
Rename design document for SSH setup
More design details will be added regarding setting up the node daemon,so this is more about adding a node than just setting up SSH.
backend: Implement remote commands
As per design document (doc/design-remote-commands.rst), a number ofrather strict tests is applied to any incoming request, a delay isinserted upon errors and returned error messages are very generic(unless it's the actual command that failed). There are unit tests for...
configure: Add option to enable remote commands
By default remote commands are disabled and need to be explicitelyenabled at build time.
Add ganeti-master-role.ocf example file
This allows controlling the cluster master role if the nodes are part ofa linux-HA cluster.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add ganeti-node-role ocf example file
This allows offlining nodes that don't respond if they are part of alinux-HA cluster.
Add new test for RAPI
Unlike existing tests, this actually tests RAPI at the interface withthe HTTP server. This way authentification can also be tested. A testfor “/2/query/…” is included as it's a bit special.
Automatically enable version-dependent GHC flags
Some GHC flags are very useful, but only appear in more recent GHCversions. To support the use of such flags while still supportingolder compilers, let's add conditional checks and enabling based onthe results....
Added attoparsec unit test for Unicode parsing
Attoparsec is known to have had issues with parsing non-ASCII strings.This test makes sure that parsing of Unicode characters works fine.
Signed-off-by: Michele Tartara <mtartara@google.com>[iustin: small doc string fixes]...
Improve make regen-vcs-version
"make regen-vcs-version" is run at every ./devel/upload time, in orderto have reasonably up-to-date information in the uploadedscripts/binaries. However, this means it currently triggersrecompilation of (at least) Ganeti/Version.hs, and sometimes more than...
Rename Confd.hs to Confd/Types.hs
This should be the last module rename, promise!
We rename this to conform to the other hierarchies (e.g. Query), andto not have both Confd.hs and Confd/*.hs.
Merge branch 'devel-2.6' into submit
Conflicts: Makefile.am (reordering, fixed) htools/Ganeti/Confd/Server.hs (hlint fixes on master) htools/Ganeti/Daemon.hs (hlint)...
Merge branch 'stable-2.6' into devel-2.6
Switch devel/upload to a static file
We had twice in the past days questions about devel/upload being"broken", since bash re-reads shell scripts during their run and thisfile can get regenerated due to Makefile changes.
Since we only need this to be dynamically built for 3 variables, let's...
Fix gnt-instance console with xl
- Rename xm-console-wrapper to xen-console-wrapper- Pass the xen command to use as a parameter
Remove setup-ssh
It has been superseeded by “prepare-node-join”.
Move htools backends to a separate directory
Five modules under the HTools/ directories are backendimplementations, so let's move them to a separate directory, to moreclearly show the hierarchy. I wanted to do this for a while, butmerging between branches is always an issue, so let's do it know since...
Add a simple unittest for THH code
This is very THH specific, and applies to all serialisations generatedby THH, so I'm adding it in its own module.
Probably we should add some more generic tests, but in general THHcode is tested by the various definitions; this new field type however...
Remove unused cache implementation
Note that this commit has no Makefile.am changes, as the files werenot actually used. So it's better to actually remove them.
Add a default `.ghci' file
This options file for ghci preseeds the correct include paths, so thatinteractive sessions don't need to always pass these args.
Add an Errors module mirroring the Python one
As described in the module doc string, while writing this it dawnedupon me that we're mixing all errors together into a single hierarchy(well, type on the Haskell side), which is not good. Some errors areused purely within noded, some in the CLI frontends, etc. so these...
Add small design for Linux HA integration
This documents the status (or wanted status) of some example ocf modulesI've written for Ganeti. They are far from perfect, but they should beshipped as a starting point for other people who want to run this to...
Replace LIBDIR in .in files
LIBDIR
Add initial implementation of prepare-node-join
This is a new tool as per the design document “design-ssh-setup”. Itreceives a JSON data structure on its standard input and configures theSSH daemon and root's SSH keys accordingly. Unit tests are included....
Find coverage during configure, issue with Debian package
- Debian Squeeze and up have a package named “python-coverage”, but it doesn't use the same binary name as upstream (“coverage”).- Said package includes a patch to use symlinks instead of file copies...
Add a new 'really-all' make target
This is used when one wants to build all binaries, including thosethat are used only for testing. A handy shortcut to make sure allbinaries can be built.
Add cluster monitoring agent design document
This design addresses the lack of a uniform way to query ganeti nodesfor real time information that can be used by monitoring.
Merge branch 'devel-2.6' into master
Add hbal-excl-tags.data to Makefile.am, missed in 0397694
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
gnt-job cancel: Confirmation and selection of jobs
New parameters, “--pending”, “--queued” and “--waiting”, are added toselect all jobs in the respective state. If one of those options is usedand “--force” is not given, the user is asked to confirm the operation....
Design for remote command execution via RPC
This is a first design for executing commands via RPC.