History | View | Annotate | Download (76.4 kB)
Move metad configuration related code to a separate module
... as an intermediate step before moving the responsbility ofcomputing the final OS parameter configuration from the configurationserver to the web server. This will allow the metadata daemon to...
KVM: move tap control functions to a submodule
Move all tap-related functionality to the hv_kvm.netdev submodule. Werename _OpenTap to OpenTap, since it will now be used as a publicfunction.
Also, change the hv_kvm tests to import the new code.
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>...
KVM: move monitor & QMP classes to a submodule
We move the HMP and QMP classes to the hv_kvm.monitor module.
Signed-off-by: Apollon Oikonomopoulos <apoikos@gmail.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Convert hv_kvm to a package
Weighing in at 3k LOC, it is probably time to split hv_kvm into a numberof modules before adding new code.
Add a WConfd RPC Client
The client is generated automatically by Ganeti.THH.HsRPC.
It also includes helper functions for creating a client object.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add a module for creating Haskell-to-Haskell RPC clients
It uses TH to create client functions with the corresponding signaturefrom a given list of server functions.
Add metadata daemon server entry point
... which starts the configuration and web servers.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add metadata daemon configuration server
The configuration server listens on a Unix socket for connections fromthe node daemon. The node daemon sends the instance parameters to theconfiguration server so they can be served through the metadata daemonweb server to the instances which have the communication mechanism...
Add metadata daemon types
The metadata daemon will be responsible for receiving the instanceparameters from the node daemon and keeping them in memory. The typeof the instance parameters in defined in a separate module because itwill be shared between the web server and the configuration server....
Move metadata daemon to its own directory
Currently, the metadata daemon is mainly a web server. However, a newconfiguration server will be introduced along with other metadatarelated modules. Therefore, this patch moves the current metadatadaemon web server to its own directory. This module is also renamed...
Merge branch 'stable-2.11' into master
Merge branch 'stable-2.10' into stable-2.11
Merge branch 'stable-2.9' into stable-2.10
Merge branch 'stable-2.8' into stable-2.9
Include qa/patch in Makefile
Fix the build by including the qa/patch directory in Makefile.am.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Move message formatting functions to separate file
The formatting functions in qa_utils.py cannot be used by modulesimported there, such as qa_config. This patch factors the functioncalls into a separate file. Also reorders imports in touched files.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Add a mock for the wconfd module
As our so-called unit tests for the logical units are morelike integration tests, calling the actual LU through an almostlive mcpu, we need a quite involved "mock" for the wconfd module,called indirectly through mcpu. The difficulties arise from the...
Add a mock version of livelocks
...as the will be used by mcpu for lock handling and that willrequire tests to be adapted accordingly.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add a module for constructing Ssconf from the configuration
This migrates the functionality from PythonConfigWriter._UnlockedGetSsconfValues to Haskell.
Add a module for handling compression
Since the Haskell 'zlib' library by default uses exceptions forsignalling errors, we need to use its internals to implement a proper,pure decompressing function.
Add a module for generated lenses for Ganeti.Objects
The module also adds type classes for lenses for common fields (serial,mTime, uuid and tags).
Splitting the lenses into a separate module avoids problems with unusedfunctions.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Add a module for custom generic Lens functions
The module currently re-exports Control.Lens and adds a function thatcreates lenses from data types by appending 'L' to its field names.
All Ganeti modules should import this module instead of Control.Lens...
Move helpers for specific fields into a separate module
These helpers aren't needed by other code in Ganeti.THH and since themodule is already large, it'll be better to separate them into theirown.
No functionality is changed by this patch, just functions moved....
Add utilities for liveliness lock files
To request resources from WConfD, requesters have to providethe name of a file they own an exclusive lock on. In this way,their death can be detected. Add utility functions to obtainsuch a file name.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add a function to detect death of a job
With jobs running in processes different from the lock management soon,we need to detect if a job holding locks dies, in order to release theresources.
Add a RunWithLocks QA utility function
This patch adds a QA utility function that acquires a set of locks, andattempts to run a given function with the locks in place. Should thegiven function block, this function does not detect this - laterpatches will address the issue....
Add serialization for LockRequest GanetiLocks
Add a module describing how abstract requests, like lock requests arepassed over the wire to WConfD.
Fix the dependencies of Makefile.ghc
Include only the programs that were configured to be built, and let GHCfigure out the required Haskell sources by itself.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Include mond and confd only when configured
Previously they were included no matter of what the configuration was,failing to compile if some of the dependencies were missing.
Verify serialization/deserialization consistency for Locks
As we made Ganeti.Locking.Locks an instance of JSON, we better verifythat we can readJSON what we showJSON.
Provide fields for lock queries
For luxid to be feature-complete with respect to masterd, italso needs to answer requests about locks. This includes knowingthe fields available for locks.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>...
Add a module for functions specific to config. management
.. and include ConfigData in WConfd's ConfigState.
Currently it includes functions for asynchronous saving theconfiguration and reading and (over)writing it in the daemon state.
Load the configuration from the file at daemon startup and configure an...
Add a module for creating and using asynchronous workers
Each such worker has its own background thread and performs a giventask when triggered. Triggers arriving when the worker is processing areaccumulated and processed together in a subsequent worker action....
A separate module for functions for atomic file operations
Utils is getting too big, so better split this new set of functions intoa separate sub-module. This also allows us to use ResultG there.
Add logging functions lifted into MonadBase
Calling logging functions in a monad transformer stack requiresinstances for all the transformers.This patch adds a module with functions named just as in Ganeti.Logging,but lifted into MonadBase. This allows using them in any transformer...
A quick fix for broken 'make distcheck'
The offending target 'test/hs/offline-test.sh' requires the dependencieseven when the test isn't actually run. This is because currently ourMakefile doesn't distinguish between prerequisites for building testscripts and running them....
Fix tab/spaces indentation in Makefile.am
At some places a few items of lists use 8 spaces, while we use tabseverywhere (as makefiles require anyway). This patch unifies theindentation.
Show Haskell dependencies when compiling
Instead of showing the long GHC command line, show which file is beingcompiled and what its dependencies are. This gives more meaningfulinformation to developers.
Compile every Haskell object file separately
This allows parallel compilation of all targets, including the Haskellobject files. While re-starting GHC for every Haskell source almostdoubles the total CPU time, allowing it run in parallel at the end cuts...
Fix the generation of ListConstants with VPATH
When the build directory differs from the source directory, theautomatic variable '$<' points to the file found by prepending the path.This causes double '../' to be inserted when combined with'$(abs_top_srcdir)'. This patch fixes this....
Add a type class describing types that can serve as locks
Besides the lock order and Show, an additional datum is needed in orderto describe lock inclusion, used, e.g., for group locks. So add a typeclass describing this property.
Disable pylint warnings for ignored files
Some files (like RPC stubs) are marked to be ignored by pylint.However pylint emits warning messages to remind of such files.This disables the warnings.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add a module for the Ganeti lock structure
Already provide a module holding the structure of the locksavailable in Ganeti, so that the overall structure of WConfDcan be implemented. The actual locks still have to be added.
Verify Basic property for lock allocation
Verify the minimal consistency property for any formof lock handling: if a user holds an exclusive lock,then no other user can hold the same lock (neitherexclusively, nor shared).
Add data structure representing current lock status
To allow for jobs as processes, a central daemon (wconfd) willhandle allocation and release of locks. Add an appropriate datastructure to describe the current status of the locks.
Rename 'BUILT_PYTHON_SOURCES' to 'built_python_sources'
This patch fixes the warnings produced by ./autogen.sh
Makefile.am:310: variable `BUILT_PYTHON_SOURCES' is defined but no program or Makefile.am:310: library has `BUILT_PYTHON' as canonical name (possible typo)...
Add the Python client for WConfD
The client combines the abstract client class and the WConfDstub to provide a Python interface to WConfD.
Generate WConfD Python stubs from Haskell
The generation is included in hs2py.hs, together with otherHaskell to Python code generators.
Generate Python stubs by inspecting functions using TH
The names of the arguments of generated Python methods are derived fromHaskell types to be as descriptive as possible.
Update Makefile.am to properly install generated stubs
This involves adding a new sets of variables for the directory.
Add a Python directory for RPC generated stubs
Directory "lib/rpc/stubs" will contains RPC stubs generated fromHaskell.
Add the WConfD daemon itself
The daemon exposes the declared functions in Ganeti.WConfd.Core to RPCclients (currently just 'echo').
Add the WConfD daemon to build configuration files
Also list it in the Haskell datatype, constants, Python constants andtest configuration.
Add a module for WConfd core functions
There are no exported functions yet, except for `echo`, which justreturns its argument back to a client.
Add a monad for running all WConfD functions
This monad encapsulates working with the daemon and client state, aswell as failures and IO operations.
Add a generic RPC module
The main function in this module takes a list of names of functions anduses TH to create an RPC server, represented as a value of type Handler(from Ganeti.UDSServer). This can be readily used to create a daemonthat dispatches RPC requests to the given functions....
Add tests for Ganeti.THH.Types
In particular, test (de)serialization of "OneTuple".
Add common functions for working with TH's Type
In particular, functions for extracting types of arguments from afunction type, and for uncurrying functions with an arbitrary number ofarguments.
Add helper to handle CLIs that optionally spawn several jobs
This helper function detects whether an opcode returned a list of jobs(i.e., a result of the type ht.TJobIdListOnly) and in this case ituses 'ganeti.cli.JobExecutor' to wait for the jobs and determine the...
Make htools and luxid mandatory in the Makefile
Since Ganeti now uses luxid for core operations, it needs to be alwayspresent.
Add a config. conditional for running Haskell coverage
Similarly to PY_UNIT, add HS_UNIT to run Haskell coverage only if wehave the required modules.
Remove the HTOOLS configuration variable
.. and update the code that uses it.
Add 'kvm-ifup-os' to configure instance communication TAP
The script 'tools/kvm-ifup-os' configures TAP network interfaces forfor instances, routing, DHCP server, etc. Note that this script onlyconfigures TAP network interfaces that are used by the instance...
Add metadata daemon
... including, user, group, daemonizing code with command lineoptions, integration with the Snap HTTP server, and logic.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Let Haddock also generate Hoogle documentation
The Hoogle database is generated to doc/api/hs/ganeti.txt when Haddockis run. The database can be used by developers with a locally installedHoogle command line tool or for a web installation of Hoogle.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix execution group of NodeD
The Node deamon was executed with the wrong gid (gnt-daemons) instead of the oneassigned to it by configure.ac.
Fixes Issue 707.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add code style document to documentation
The Ganeti code style has been stored on the project wiki at:
https://code.google.com/p/ganeti/wiki/StyleGuide https://code.google.com/p/ganeti/wiki/HaskellStyleGuide
This commit combines the two pages into an .rst file with minimal...
Export docs from wiki to repo
Export documents about 'cluster keys replacement' and 'design filebased storage' from the wiki to the repository.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Anchor OS reinstall design doc in Makefile and index
Add OS reinstall design doc to the list of design docs in theMakefile, otherwise it does not get compiled when modified, and add italso to the index page of the documentation, where all the otherdesign docs are anchored....
Add a module to support virtual clusters
Virtual clusters are an efficient way to test how Ganeti behaveson a large cluster without requiring a large number of machines.Now that more tasks like job replication are done by luxid, providethat functionality in Haskell as well....
Fix configure dirs
Fix overridable configure directories, namely, --bindir, --sbindir,and --mandir.
Variables 'bindir', 'sbindir', and 'mandir' are exported from'configure.ac' to 'Makefile.am'. At first, it would seem to bepossible to do the following:...
Add 'design-2.11.rst' which kvmd and instance shutdown
Add 'design-2.11.rst' which kvmd and instance shutdown to thetop-level documentation and Makefile.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Prepare master as new 2.12 branch
This adapts various file to make the current masterbranch the new stable-2.12 branch.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Unit tests for KVM daemon
Add unit tests for KVM daemon.
Manpage for KVM daemon
Add manpage for the KVM daemon.
Add KVM daemon daemonize
Add KVM daemon entry point, command-line options, backgrounding, etc
Add KVM daemon logic
Add KVM daemon logic, which contains monitors for Qmp sockets anddirectory/file watching.
KVM daemon datatype, user and group
Merge branch 'stable-2.10' into master
Add an empty post-upgrade hook
As 2.10 is the first version from which you can do automatic upgrades,there is nothing to do when going to any other version in the 2.10branch.
Design document for KVM daemon
Design document for KVM daemon which is needed by the instanceshutdown detection for KVM.
Add the first version of the RAPI workload script
The RAPI workload script supplies work for the RAPI compatibilitytests. The initial version does very little, but can be expandedas needed.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Utility functions to manipulate the candidate map
This patch adds a couple of utility functions to manipulatethe map of master candidate SSL certificate digests.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add an hsqueeze test for drbd instances
In this example, there are two drbd instances, rendering a total offour nodes ineligible for being offlined. Additionally, the mastermay not be offlined either, leaving a single candidate.
Split Luxi Client into a generic and a specific part
The generic part will be reused in WConfd.
Move Transport from luxi.py to a separate module
Also create a new module for RPC errors.This allows it to be reused for other clients as well.
Add a Python directory for RPC code to keep it at one place
Move rpc.py to rpc/node.py and modify imports in existing code.