History | View | Annotate | Download (61.5 kB)
Add daemon split design doc
This describes the future planned structure of Ganeti daemons.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Make tools/users-setup explain its actions first
Before doing any changes, make tools/users-setup explain itsactions first, and ask the user for confirmation. If the firstargument is --yes-do-it, this step will be skipped.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Generate tools/users-setup from descriptions
Now that we have a machine readable description of the usersto be added to the system, use it to generate the tools/users-setupscript, so that we again have only one authoritative source ofthe user and group requirements. This also has the advantage of...
Provide ganeti user and group data in easy-to-parse form
Upon build time, also generate, in the directory doc/users, filescontaining the groups, the users with their primary group, and theadditional groupmemberships to be added. In this way, packages can use...
Add unit test for GetLinuxNodeInfo
The function was not unit tested previously.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add monitoring-query-format.rst to $(docinput)
Commit 431ff2c10 split the monitoring system documentation in two parts but didnot include the new monitoring-query-format.rst to $(docinput). As a result,current tarballs of 2.8 do not ship doc/monitoring-query-format.rst....
Fix documentation building rules
Implement the correct behavior to be kept when sphinx-build is not installed.It was already correctly checked, but instead of just disabling thedocumentation generation, an error message was printed, having "make" fail....
Merge branch 'stable-2.7' into stable-2.8
Conflicts: NEWS: trivial...
Add logrotate example
Add an example logrotate snippet, along with the relevant documentation.
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Enable unit tests again
A while ago, the execution of python unit tests wasaccidentally disabled for distcheck. This patchenables them again. This will lead to a couple oftests failing, but we decided to submit this firstand then fix the broken tests in separate patches....
Document ganeti-queryd
Add a man page for ganeti-queryd and update the documentation forganeti-confd in various places.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add queryd daemon (split from confd)
queryd is added as a new daemon which handles configuration queries overLUXI. This functionality was removed from confd, which now only queriesover the network.
The queryd user is added to the master group such that it can access...
Extract ConfigReader from Confd/Server.hs
Confd's functionality to watch the Ganeti configuration file isextracted to the ConfigReader module. No functional changes areintroduced.
This extraction makes will enable us to split queryd from confd, asqueryd will have to use the same functionality....
Fix issue with python coverage tests
The recently introduced check for python libraries required only for testing(commit 27df5b736ef72b3b12c07f32d64dbac95fe7a5ba) was not correct. Thispatch fixes the issue.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Disable python test if required libraries are missing
Some python libraries are only needed for the tests, but not for runningGaneti. If those libraries are missing, just disable the Python tests.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Add test for shrink heuristics over different resources
In this example, memory prevents adding more than one instance onthe first node. Yet, on the other nodes, 2 instances can be placedeach, if disks are shrunk appropriately. This, however, requiresto courageous decision to shrink disks next, even though more nodes...
Disable more version checks for alpha versions
Alpha versions should not check for version numbers in READMEsand documentation.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add tests for check-news
Unit tests verifying the behaviour of the check-news script are added.
This required adding a new make target, "autotools-check" directly called from"commit-check", because the usual tests are executed in the build directory,whereas these, being relative to the autotools, cannot be executed there because...
Fix bug in Makefile.am
There was a typo in the name of the script for launching offline tests.
Conflicts: (trival, take union of added files/tests) Makefile.am test/hs/shelltests/htools-hspace.test...
Add a test for hspace to respect instance policy
In this example, the instance policy for disks prevents allocationof more than one instance per node. The test verifies that tieredallocation still respects this and doesn't attempt to place policyviolating instances on the cluster....
Add tests for network-aware allocation
hail-alloc-invalid-network defines a cluster with two nodegroups and anallocation request which does not fit on any of the groups. Group 1 hasinvalid disk-templates while Group 2 is not connected to the rightnetworks....
Parse NIC data from allocation request in hail
Add a NIC type and extend the Instance type by a list of NIC's. Parsethe NIC's in allocation requests and store them for now. Later patcheswill make use of this field in order to ensure that the requestedinstance is only placed in node groups wich are connected to those...
Create overall design document for 2.8
Also, cleanup the list of draft designs.
Add Harep man page
Add the man page for the Ganeti auto-repair tool.
Sort cmdlib-related entries in Makefile.am
Files in the cmdlib directory are sorted alphabetically inMakefile.am.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
cmdlib: Extract instance query related functionality
Split instance.py further by extracting instance querying relatedlogical units and functions to instance_query.py.
cmdlib: Extract instance operation functionality
Split instance.py further by extracting instance operations(start/stop/reboot/etc.) related logical units and functions toinstance_operation.py.
The extracted operations have in common that they affect the operating...
cmdlib: Extract migration related functionality
Split instance.py further by extracting migration related logical unitsand functions to instance_migration.py.
cmdlib: Extract storage related functionality
Split instance.py further by extracting storage related logical unitsand functions to instance_storage.py.
Extract miscellaneous logical units from cmdlib
All remaining classes in init.py are extracted to misc.py.
Extract os related logical units from cmdlib
All LUOs* classes are extracted to operating_system.py.
Extract query related logical units from cmdlib
All LUQuery* classes are extracted to query.py.
Extract backup related logical units from cmdlib
All LUBackup* classes are extracted to backup.py.
Extract instance related logical units from cmdlib
All LUInstance* classes are extracted to instance.py. Common functionsare moved to common.py if used by non-instance logical units as well.Additionally, helper functions which are only used by LUBackup* and...
Extract node related logical units from cmdlib
All LUNode* classes are extracted to node.py. Common functions are movedto common.py if used by non-node logical units as well.
Extract group related logial units from cmdlib
All LUGroup* classes are moved to group.py. Common functions areextracted to common.py.
Extract cluster related logical units from cmdlib
All LUCluster* classes are extracted to cluster.py. Shared functions areextracted to common.py, helper functions only used by LUCluster* areextracted to cluster.py.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Extract test logical units from cmdlib
LUTest* are moved to test.py.
Extract network related logical units from cmdlib
LUNetwork* and associated helper functions are extracted to network.py.
Extract tags related logical units from cmdlib
LUTags* and their base class, TagsLU, are extracted to tags.py. Anadditional shared function, _ShareAll, is extracted to common.py forshared usage.
Extract base classes from cmdlib
Base classes holding common functionality is extracted into base.py.Utility functions used by both base classes and subclasses is moved tocommon.py.
Move cmdlib.py to cmdlib/__init__.py
cmdlib.py has grown really too much. Move it into its own package toallow splitting it further.
Signed-off-by: Guido Trotter <ultrotter@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Merge stable-2.7 into stable-2.8
Conflicts: INSTALL NEWS configure.ac src/Ganeti/Query/Group.hs test/hs/Test/Ganeti/Objects.hs test/hs/Test/Ganeti/Query/Query.hs
Disable compiling mon-collector if monitoring is disabled
Mon-collector kept being compiled even if monitoring was disabled duringconfigure.
Fixes Issue 441.
Add shelltests for hspace allocation
Both tiered and standard allocations are tested, with a single and a doublemin/max instance specification.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
This fixes the problem with the distcheck-release target failing needlessly.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Improve check for "unreleased" versions in NEWS
Currently this is checked only when distcheck-release is called.Check it in check-news instead, for all versions before the "current" one (as defined by configure.ac)
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Conflicts: NEWS doc/rapi.rst
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add unit tests for cfgupgrade with a real configuration
Test upgrade and downgrade from a realistic 2.7 configuration.
Add unit tests for hroller
Add a unit test for hroller in a situation where the orderis completely determined. There are 3 nodes where node 2 has,for each of the other nodes, an instanced shared with it. Sonode 2 has to be a reboot group of its own. There are no more...
Use correct name for design-storagespace.rst
This was renamed in b284f50 but the old name was still in use in themakefile and draft designs list.
Add design doc for device UUIDs and names
This commit adds the design document for adding UUID and name slots to NIC andDisk objects and the ability to refer to them by these slots.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Move HooksMaster out of the mcpu module
We need to do this, so that backend.py doesn't need to import mcpu, andthus indirectly cmdlib. This reduces the size of the node daemon byabout half, which is very important as it is pinned in memory.
This solves Issue 419....
Merge branch 'devel-2.7'
Fix burnin install path
Due to a bug in 8e55e20f burnin's installation path got changed.Moving tools/burnin to the right makefile target reverts this (whilepreserving the value of the original commit).
Add shelltests verifying hail applies disk ipolicy per disk
We test on two minimal examples, a positive and a negative one, wherethe possibility to allocate a node depends on whether the disk policyis checked on a per-disk level or on the total amount of disk space...
Add tool for creating users and groups
When executed in split user mode, Ganeti requires a specific set of usersand groups to be created in the appropriate way.
This commit introduces a tool that takes care of that automatically.
Add script for building chroot
Setting up a proper environment for building a Ganeti can be quite difficult.
This patch adds a script for building a compressed chroot with all therequired libraries and tools.
Add the core of the monitoring daemon
This commit adds the core infrastructure of the monitoring daemon,and integrates it in the build and test systems.
The actual functionality of the monitoring daemon is still completelymissing.
Add design document for the "reason trail"
This commit adds the design document for introducing "reason trails",tracing the reason why opcodes are executed, step by step.
Add user and group for the monitoring agent
This commit adds the user and group variables for the monitoring agent in allthe configuration files where they are required.
Signed-off-by: Iustin Pop <iustin@google.com>...
Fix bug in man build rule
Currently the man page build rule is run without ‘-e’, so the actualchecks in there are just warnings (check-man-warnings,check-man-dashes).
The patch fixes this and also fixes the man pages which made me see theproblem.
Additionally, check-man-dashes is now verbose, otherwise it's hard to...
Fix hscolour style sheet building
Currently we only symlink this into two hard-coded directories, whichbreaks the source code viewing for all the others.
The patch replaces this with automatic building; as playing withrelative symlinks is non-trivial, I just generate it many times (the...
Conflicts: Makefile.am (curl changes and new hs directories)...
Expand Haddock to run over test files as well
This patch does multiple enhancements to the way we build the Haddockdocs, motivated by the fact that if we don't run Haddock over testfiles as well, bad formatting can be submitted and will accumulateover time (without any checks)....
Implement Export queries in Haskell
This is a simple query as it has only two fields, however it's thefirst query that doesn't have a clear 'base' object and 1:1correspondence between such objects and the results (groups, nodes andnetworks do so).
We keep nodes as the 'base' object, since that's what we want to...
Switch the curl bindings from optional to required
Currently, we support curl being optional via some sporting exercises:ifdefs in the code, data types that represent 'Curl is disabled'state, etc. However, with the future work on RPC, we would have toeven make the dependencies list conditional on it, etc. This is too...
Add a Ganeti-specific implementation of Curl Multi
As we want to be able to run queries against multiple nodes inparallel, and furthermore in parallel with other work, we need toimplement the Curl Multi interface (see libcurl-multi(3)).
This patch adds a Ganeti-specific such implementation, to be used...
Makefile: Add dependencies
The RAPI documentation now depends on “connector.py” (handlerregistration) and “rlib2.py” (access permissions).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Unit tests for Query/Network.hs
This patch adds a couple of unit tests for Query/Network.hs.Note that they'll need to be adapted, once issue 362 is addressed.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add Haskell parser for "xm uptime"
In order to fetch precise information about the uptime of the VMsrunning in Xen, we need to analyze the output of the "xm uptime" command.
This commit adds the parser to do that, and its tests.
Add Haskell parser for "xm list --long"
In order to fetch precise information about the status of the VMs running inXen, we need to analyze the output of the "xm list --long" command.
Design correct reporting of storage free space
This is a long standing bug in Ganeti. Add a small design on how we planto fix this for Ganeti 2.7.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Rename lib/objectutils to outils.py
Back when this was introduced, I mentioned that it breaks heavily tabcompletion (ob<TAB> doesn't work anymore), but at that moment I didn'thave a suggestion what to name it. I think outils is good and shortenough, and doesn't conflict with anything else, so here it goes....
Add hroller design to Makefile and design-drafts
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Finish the remote→restricted commands rename
The documentation still points to /etc/ganeti/remote-commands,although the code is already using restricted-command. Update thedocumentation and a few docstrings accordingly.
Add unit tests for RADOSBLockDevice
Add unit tests for the RADOSBlockDevice JSON output parsing function,and modify the unit tests for the plain output parsing function towork with the new code.
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
Refactor storage of runtime exclusive storage flag in QA
This is a follow-up for “qa_config: Remove exclusive storage flag fromconfig”. Instead of storing the flag in a module-level variable it isnow stored within the new QA configuration class and unit tests are...
Workaround hlint behaviour with no warnings/errors
When hlint sees a perfectly nice code tree, it will skip generatingthe reports (both stdout and html). This means that usually the oldreport will remain in place, so it makes development harder, as the...
Add some more Haskell/Python equivalence tests
This would have caught the log file problem fixed in the previouspatch (9411474b), for example. Also we test user/group equivalence,name only.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Program/Harep.hs: add skeleton for the new auto-repair tool
harep(1) detects certain kind of problems with instances and applies theallowed set of solutions. See doc/design-autorepair.rst.
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
First part of Network Queries in Haskell
This is the beginning of the implementation of networkqueries. This includes establishing all infrastructureto run the network queries and implement querying ofsome simpler fields and the node group listing.
Signed-off-by: Helga Velroyen <helgav@google.com>...
hv_xen: Refactor getting node information, add tests
Refactor and add tests for getting node (Domain-0) information.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
hv_xen: Refactor running & parsing "xm list", add tests
This patch refactors “_RunXmList” and adds some tests.
Make Xen config path a build-time option
Stop hardcoding the path in “hv_xen.py”.
Makefile.am: Protect placeholders for sed
If the placeholder has exactly the same name as the variable used inautoconf it would be replaced as well. In the case of“ENABLE_SPLIT_QUERY” this led to the sed command “s#False#False#g”.Inserting additional quotes after the first @ sign prevents this....
ENABLE_SPLIT_QUERY
Makefile: Allow tarball creation when man-in-doc is active
Documentation with included man pages should not be distributed as itcontains fixed paths. Since the documentation with man pages is actuallywritten to a separate directory, tarball creation is still possible....
run-in-tempdir: Copy doc/* only if needed
To preserve disk space on copying all files to a temporary directory,the “doc” directory is only copied when building documentation. Acomment describing the need is also added to Makefile.am, as well as asuperfluous direct invocation of Bash removed....
Add reporting infrastructure for data collectors
This commit adds the part of the JSON report generation code that will becommon to all the data collectors, according to the format specified in thedesign document.
burnin: Add unittest for supported disk templates
When new a new disk template is added, burnin should be updated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Move burnin to ganeti.tools.burnin
Similar to other scripts, “tools/burnin” just becomes a wrapper script.
Link man pages in documentation
This patch depends on “Option to include man pages in documentation”. Inthe documentation build including man pages, all “:manpage:`…`”references are converted to links. For man pages not provided by Ganeti,Sphinx' standard formatting is used....
Option to include man pages in documentation
Before this patch, HTML versions of man pages (man/*.rst) were alreadybuilt. However, since they are separate from the normal documentation,their content is not indexed for Sphinx' search functionality.Additionally it would simply be nice to have everything in one place....
kvm: add two help dump files
One is the --help output for kvm 1.1.2 and the second one is the same as0.9.1 but with a fake option added between -drive and its boot=on|offoption: this tests that if boot=on|off appears in another option after-drive it is not accepted by our regexp....