History | View | Annotate | Download (75.9 kB)
Add a mock for the wconfd module
As our so-called unit tests for the logical units are morelike integration tests, calling the actual LU through an almostlive mcpu, we need a quite involved "mock" for the wconfd module,called indirectly through mcpu. The difficulties arise from the...
Add a mock version of livelocks
...as the will be used by mcpu for lock handling and that willrequire tests to be adapted accordingly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add a module for constructing Ssconf from the configuration
This migrates the functionality from PythonConfigWriter._UnlockedGetSsconfValues to Haskell.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add a module for handling compression
Since the Haskell 'zlib' library by default uses exceptions forsignalling errors, we need to use its internals to implement a proper,pure decompressing function.
Add a module for generated lenses for Ganeti.Objects
The module also adds type classes for lenses for common fields (serial,mTime, uuid and tags).
Splitting the lenses into a separate module avoids problems with unusedfunctions.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Add a module for custom generic Lens functions
The module currently re-exports Control.Lens and adds a function thatcreates lenses from data types by appending 'L' to its field names.
All Ganeti modules should import this module instead of Control.Lens...
Move helpers for specific fields into a separate module
These helpers aren't needed by other code in Ganeti.THH and since themodule is already large, it'll be better to separate them into theirown.
No functionality is changed by this patch, just functions moved....
Add utilities for liveliness lock files
To request resources from WConfD, requesters have to providethe name of a file they own an exclusive lock on. In this way,their death can be detected. Add utility functions to obtainsuch a file name.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add a function to detect death of a job
With jobs running in processes different from the lock management soon,we need to detect if a job holding locks dies, in order to release theresources.
Add a RunWithLocks QA utility function
This patch adds a QA utility function that acquires a set of locks, andattempts to run a given function with the locks in place. Should thegiven function block, this function does not detect this - laterpatches will address the issue....
Add serialization for LockRequest GanetiLocks
Add a module describing how abstract requests, like lock requests arepassed over the wire to WConfD.
Merge branch 'stable-2.11' into master
Fix the dependencies of Makefile.ghc
Include only the programs that were configured to be built, and let GHCfigure out the required Haskell sources by itself.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Include mond and confd only when configured
Previously they were included no matter of what the configuration was,failing to compile if some of the dependencies were missing.
Verify serialization/deserialization consistency for Locks
As we made Ganeti.Locking.Locks an instance of JSON, we better verifythat we can readJSON what we showJSON.
Provide fields for lock queries
For luxid to be feature-complete with respect to masterd, italso needs to answer requests about locks. This includes knowingthe fields available for locks.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>...
Add a module for functions specific to config. management
.. and include ConfigData in WConfd's ConfigState.
Currently it includes functions for asynchronous saving theconfiguration and reading and (over)writing it in the daemon state.
Load the configuration from the file at daemon startup and configure an...
Add a module for creating and using asynchronous workers
Each such worker has its own background thread and performs a giventask when triggered. Triggers arriving when the worker is processing areaccumulated and processed together in a subsequent worker action....
A separate module for functions for atomic file operations
Utils is getting too big, so better split this new set of functions intoa separate sub-module. This also allows us to use ResultG there.
Add logging functions lifted into MonadBase
Calling logging functions in a monad transformer stack requiresinstances for all the transformers.This patch adds a module with functions named just as in Ganeti.Logging,but lifted into MonadBase. This allows using them in any transformer...
A quick fix for broken 'make distcheck'
The offending target 'test/hs/offline-test.sh' requires the dependencieseven when the test isn't actually run. This is because currently ourMakefile doesn't distinguish between prerequisites for building testscripts and running them....
Fix tab/spaces indentation in Makefile.am
At some places a few items of lists use 8 spaces, while we use tabseverywhere (as makefiles require anyway). This patch unifies theindentation.
Show Haskell dependencies when compiling
Instead of showing the long GHC command line, show which file is beingcompiled and what its dependencies are. This gives more meaningfulinformation to developers.
Compile every Haskell object file separately
This allows parallel compilation of all targets, including the Haskellobject files. While re-starting GHC for every Haskell source almostdoubles the total CPU time, allowing it run in parallel at the end cuts...
Fix the generation of ListConstants with VPATH
When the build directory differs from the source directory, theautomatic variable '$<' points to the file found by prepending the path.This causes double '../' to be inserted when combined with'$(abs_top_srcdir)'. This patch fixes this....
Add a type class describing types that can serve as locks
Besides the lock order and Show, an additional datum is needed in orderto describe lock inclusion, used, e.g., for group locks. So add a typeclass describing this property.
Disable pylint warnings for ignored files
Some files (like RPC stubs) are marked to be ignored by pylint.However pylint emits warning messages to remind of such files.This disables the warnings.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add a module for the Ganeti lock structure
Already provide a module holding the structure of the locksavailable in Ganeti, so that the overall structure of WConfDcan be implemented. The actual locks still have to be added.
Verify Basic property for lock allocation
Verify the minimal consistency property for any formof lock handling: if a user holds an exclusive lock,then no other user can hold the same lock (neitherexclusively, nor shared).
Add data structure representing current lock status
To allow for jobs as processes, a central daemon (wconfd) willhandle allocation and release of locks. Add an appropriate datastructure to describe the current status of the locks.
Rename 'BUILT_PYTHON_SOURCES' to 'built_python_sources'
This patch fixes the warnings produced by ./autogen.sh
Makefile.am:310: variable `BUILT_PYTHON_SOURCES' is defined but no program or Makefile.am:310: library has `BUILT_PYTHON' as canonical name (possible typo)...
Add the Python client for WConfD
The client combines the abstract client class and the WConfDstub to provide a Python interface to WConfD.
Generate WConfD Python stubs from Haskell
The generation is included in hs2py.hs, together with otherHaskell to Python code generators.
Generate Python stubs by inspecting functions using TH
The names of the arguments of generated Python methods are derived fromHaskell types to be as descriptive as possible.
Update Makefile.am to properly install generated stubs
This involves adding a new sets of variables for the directory.
Add a Python directory for RPC generated stubs
Directory "lib/rpc/stubs" will contains RPC stubs generated fromHaskell.
Add the WConfD daemon itself
The daemon exposes the declared functions in Ganeti.WConfd.Core to RPCclients (currently just 'echo').
Add the WConfD daemon to build configuration files
Also list it in the Haskell datatype, constants, Python constants andtest configuration.
Add a module for WConfd core functions
There are no exported functions yet, except for `echo`, which justreturns its argument back to a client.
Add a monad for running all WConfD functions
This monad encapsulates working with the daemon and client state, aswell as failures and IO operations.
Add a generic RPC module
The main function in this module takes a list of names of functions anduses TH to create an RPC server, represented as a value of type Handler(from Ganeti.UDSServer). This can be readily used to create a daemonthat dispatches RPC requests to the given functions....
Add tests for Ganeti.THH.Types
In particular, test (de)serialization of "OneTuple".
Add common functions for working with TH's Type
In particular, functions for extracting types of arguments from afunction type, and for uncurrying functions with an arbitrary number ofarguments.
Add helper to handle CLIs that optionally spawn several jobs
This helper function detects whether an opcode returned a list of jobs(i.e., a result of the type ht.TJobIdListOnly) and in this case ituses 'ganeti.cli.JobExecutor' to wait for the jobs and determine the...
Make htools and luxid mandatory in the Makefile
Since Ganeti now uses luxid for core operations, it needs to be alwayspresent.
Add a config. conditional for running Haskell coverage
Similarly to PY_UNIT, add HS_UNIT to run Haskell coverage only if wehave the required modules.
Remove the HTOOLS configuration variable
.. and update the code that uses it.
Merge branch 'stable-2.10' into stable-2.11
Add 'kvm-ifup-os' to configure instance communication TAP
The script 'tools/kvm-ifup-os' configures TAP network interfaces forfor instances, routing, DHCP server, etc. Note that this script onlyconfigures TAP network interfaces that are used by the instance...
Add metadata daemon
... including, user, group, daemonizing code with command lineoptions, integration with the Snap HTTP server, and logic.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Let Haddock also generate Hoogle documentation
The Hoogle database is generated to doc/api/hs/ganeti.txt when Haddockis run. The database can be used by developers with a locally installedHoogle command line tool or for a web installation of Hoogle.
Merge branch 'stable-2.9' into stable-2.10
Merge branch 'stable-2.8' into stable-2.9
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix execution group of NodeD
The Node deamon was executed with the wrong gid (gnt-daemons) instead of the oneassigned to it by configure.ac.
Fixes Issue 707.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add code style document to documentation
The Ganeti code style has been stored on the project wiki at:
https://code.google.com/p/ganeti/wiki/StyleGuide https://code.google.com/p/ganeti/wiki/HaskellStyleGuide
This commit combines the two pages into an .rst file with minimal...
Export docs from wiki to repo
Export documents about 'cluster keys replacement' and 'design filebased storage' from the wiki to the repository.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Anchor OS reinstall design doc in Makefile and index
Add OS reinstall design doc to the list of design docs in theMakefile, otherwise it does not get compiled when modified, and add italso to the index page of the documentation, where all the otherdesign docs are anchored....
Add a module to support virtual clusters
Virtual clusters are an efficient way to test how Ganeti behaveson a large cluster without requiring a large number of machines.Now that more tasks like job replication are done by luxid, providethat functionality in Haskell as well....
Fix configure dirs
Fix overridable configure directories, namely, --bindir, --sbindir,and --mandir.
Variables 'bindir', 'sbindir', and 'mandir' are exported from'configure.ac' to 'Makefile.am'. At first, it would seem to bepossible to do the following:...
Add 'design-2.11.rst' which kvmd and instance shutdown
Add 'design-2.11.rst' which kvmd and instance shutdown to thetop-level documentation and Makefile.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Prepare master as new 2.12 branch
This adapts various file to make the current masterbranch the new stable-2.12 branch.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Unit tests for KVM daemon
Add unit tests for KVM daemon.
Manpage for KVM daemon
Add manpage for the KVM daemon.
Add KVM daemon daemonize
Add KVM daemon entry point, command-line options, backgrounding, etc
Add KVM daemon logic
Add KVM daemon logic, which contains monitors for Qmp sockets anddirectory/file watching.
KVM daemon datatype, user and group
Merge branch 'stable-2.10' into master
Add an empty post-upgrade hook
As 2.10 is the first version from which you can do automatic upgrades,there is nothing to do when going to any other version in the 2.10branch.
Design document for KVM daemon
Design document for KVM daemon which is needed by the instanceshutdown detection for KVM.
Add the first version of the RAPI workload script
The RAPI workload script supplies work for the RAPI compatibilitytests. The initial version does very little, but can be expandedas needed.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Utility functions to manipulate the candidate map
This patch adds a couple of utility functions to manipulatethe map of master candidate SSL certificate digests.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add an hsqueeze test for drbd instances
In this example, there are two drbd instances, rendering a total offour nodes ineligible for being offlined. Additionally, the mastermay not be offlined either, leaving a single candidate.
Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Move Transport from luxi.py to a separate module
Also create a new module for RPC errors.This allows it to be reused for other clients as well.
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.
Gluster: minimal implementation
Add Gluster to Ganeti by essentially cloning the shared file behavioureverywhere in the code base.
Signed-off-by: Santi Raffa <rsanti@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Use a data type when generating Python types of OpCodes
Currently they are generated only as Strings.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Rename PyValueInstances.hs to PyValue.hs
Now the file contains the type class declaration as well.
Split Luxi.hs into LUXI-specific functions and general ones
This will allow WConfD to use the general functions without importingLuxi.hs.
Fix 'htest' related targets dependencies
This fixes issue 634.
Add a scheduler to keep track of the job queue
In order to allow informed decissions on when to start a job,it is necessary for luxid to keep track of the (active partof the) job queue. Add a scheduler, similar to the config reader,that does this, but also schedules jobs to be executed. At the...
Remove --enable-split-query option
Switching from python to haskell queries, this patchremoves the option to dis/enable the haskell queriesat configure time.
Fix broken TAGS generation
This file is used for development to point to code definitions. Apparently thishas been broken since Februrary [cc40185]. The improved version fixes theproblem and uses Exuberant Ctags <http://ctags.sourceforge.net/>. It uses...
Merge 'hs2py-constants' into 'hs2py'
Merge code of 'hs2py-constants' into 'hs2py', which requires addingflag '--constants', and merge target 'src/hs2py-constants' into'src/hs2py' in 'Makefile.am', also updating dependencies andvariables.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Add flag '--opcodes' to 'hs2py'
This is a transitional step before merging 'hs2py-constants' and'hs2py'.
Sort design docs alphabetically
Sort design docs alphabetically in Makefile.am and fix whitespace.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Missing design docs from distribution
Fixes issue 609.
Move Haskell constants to proper module
Move Haskell constants from module 'Ganeti.HsConstants', which was atransitional module part of the Haskell to Python constant generationinfrastructure, to module 'Ganeti.Constants'.
Tear down Py2Hs constant infrastructure
Tear down Python to Haskell constant conversion infrastructure, whichincludes eliminating the autotool 'convert-constants' and the Haskellmodule, namely 'Ganeti.PyConstants', which held the convertedcontants.
Add 'AF_INET*' constants to Haskell's 'AutoConf'
Add 'AF_INET*' constants to the Haskell 'AutoConf' module, whichrequires extending the 'm4' invocation with additional substitutions,in the 'Makefile' target for 'src/AutoConf.hs'.