Retry luxi/wconfd RPC calls if the connection is closed
Since the daemon can decide to close a client connection after atimeout, the client needs to be able to automatically reconnect.
This patch introduces this functionality into the RPC client:If an attempt to send data fails on 'Broken pipe', it's retried one more...
Fix generic uncurrying
in the case of n>1, the n-ary uncurry function is of the form\f (x_1, ..., x_n) -> ... rather than \f x_1 ... x_n -> ...
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add --mac-prefix to gnt-cluster modify manpage
Update the manpage for mac-prefix modification in 'gnt-cluster modify'command.
Signed-off-by: Dimitris Bliablias <bl.dimitris@gmail.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add unittest for gnt-cluster modify --mac-prefix
Add unittest for cluster mac-prefix modification support.
Allow cluster mac prefix modification
Extend LUClusterSetParams to allow the modification of the clustermac-prefix setting in 'gnt-cluster modify' command.
This fixes part of issue 239.
Show mac prefix setting in gnt-cluster info
Include mac-prefix setting in the output of 'gnt-cluster info'command.
Add documentation for "gnt-debug wconfd"
The missing part caused a doclint test to fail.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Add a command to gnt-debug to test various aspects of wconfd
For debugging purpose, support direct communication to WConfD fromthe commandline for some of its commands. For the time being, supportthe echo command.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Locks are owned by job id file path pairs
...according to our updated design.
Remove the client state from WConfD monad
As we decided to keep our protocols stateless, the clientstate is no longer needed.
design: keep WConfD's protocols stateless
To simplify the setup, keep all protocols of WConfD stateless. This impliesthat the job id has to be sent with every resource request. Also clarify thatthe owner of a resource is the pair of the job id and the file for job death...
Let config.py use WConfd for reading/writing the config
Currently it only relays the reads/writes to the file to WConfd,everything else yet remains in config.py.
Also if the 'ConfigWriter' is opened in "offline" mode (like inbootstrap.py), it doesn't use WConfd and resorts to the original...
Start WConfd temporarily during master failover
.. in order to update the configuration and distribute ssconf, beforestarting the daemons by the scripts.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Include WConfd in the list of daemons started on a master
.. and make it the first one so that masterd can use it when it starts.
Add missing constant for the name of WConfd
The constant is used when calling the daemon startup/shutdown scripts.
Add the --force-node option to WConfd
.. and fix the superfluous options list in ganeti-wconfd.hs
Add an option for disabling node checks at daemon startup
This is needed for WConfd, because during failover it needs to betemporarily started in order to update the configuration on the selectedmaster candidate.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Convert int to float when checking config. consistency
When reading the configuration file from RPC JSON, values without afloating point are parsed as 'int', not as 'float', and later theconsistency check fails.
This patch adds an automatic conversion from 'int' to 'float' during...
Prefer unless over when (not...)
...and by doing so, please lint.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
At startup, make WConfD read the lock allocation from disk
On startup, make wconfd read a the state of the lock allocation from disk,instead of starting from an empty allocation.
Note that, as opposed to the configuration, no finding a savedlog configuration is not an error; e.g., the node might be master...
Provide path to persist locks
As we aim to handle jobs as separate processes with lockshandled by WConfD, we need to persist the locks to ensurethey survive a restart of that deamon. Saving only on masterdaemon is enough, however.
Add a function to load lock allocation from disk
Lock allocation are stored on disk as their JSON representation.
Verify serialization/deserialization consistency for Locks
As we made Ganeti.Locking.Locks an instance of JSON, we better verifythat we can readJSON what we showJSON.
Make Locks a JSON instance
As requests to acquire or release locks come viainter-process communication, we need a serializationof the locks data type.
All locks are serialised as JSON objects, where thekey "type" describes the type of lock, and the remaining...
Add test for serialization of LockAllocation
Verify readJSON . showJSON = Ok for LockAllocations.
Make LockAllocation an instance of JSON
In this way, we can serialize a lock state to disk, so thatlocks can survive a restart of WConfD.
Merge branch 'stable-2.11' into master
Merge branch 'origin/stable-2.10' into stable-2.11
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Add query support for locks to luxid
While requests only get forwarded, it still helps toget luxid feature-complete with respect to master.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>Cherry-picked from commit a6e406ce376453e90e598c7be68809d6a7bd7d41...
Provide fields for lock queries
For luxid to be feature-complete with respect to masterd, italso needs to answer requests about locks. This includes knowingthe fields available for locks.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>...
Export reading and writing the configuration using RPC
Note that currently these functions lack any kind of locking andconcurrent calls to 'writeConfig' will result in lost changes in theconfiguration.
Add a module for functions specific to config. management
.. and include ConfigData in WConfd's ConfigState.
Currently it includes functions for asynchronous saving theconfiguration and reading and (over)writing it in the daemon state.
Load the configuration from the file at daemon startup and configure an...
Add more general conf. handling functions to WConfd's Monad
Add functions for reading and writing the configuration. Writingtriggers the asynchronous worker which saves the configuration into themaster file. The actual action that saves the configuration is passed as...
Add functions for saving the configuration to Ganeti.Config
.. so that it can be not only read, but also written to the masterconfiguration file.
Add a module for creating and using asynchronous workers
Each such worker has its own background thread and performs a giventask when triggered. Triggers arriving when the worker is processing areaccumulated and processed together in a subsequent worker action....
A separate module for functions for atomic file operations
Utils is getting too big, so better split this new set of functions intoa separate sub-module. This also allows us to use ResultG there.
Add a function for lifting ResultT from a MonadBase
Often we have `ResultT e IO a` as the return type of our primitivefunctions and we need to lift them into a monad stack that is also basedon IO. This function accomplishes this by lifting a ResultT in a base...
Add a type alias for "ResultT GanetiException IO"
This type occurs often in the upcoming code so it's worth having analias for it.
It's included in Errors instead of Utils, because it depends onGanetiException, and importing Errors from Utils causes a cyclic...
Add logging functions lifted into MonadBase
Calling logging functions in a monad transformer stack requiresinstances for all the transformers.This patch adds a module with functions named just as in Ganeti.Logging,but lifted into MonadBase. This allows using them in any transformer...
Add a MonadLog instance for strict StateT and IdentityT
This will be used for asynchronous workers, for which it's convenient touse StateT to keep their internal state.
Require lifted-base >= 0.1.2 for lifted IORef operations
This allows to use them in any IO-based monad.
Add default arguments for WConfd to scripts
In particular, this enables debug logging for WConfd on our testenvironment.
Merge branch 'origin/stable-2.9' into stable-2.10
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Make gnt-debug locks display fake job locks properly
When a job is dependent on other jobs, a fake lock is created whosepending entry contains a list of job ids waiting on the job. gnt-debuglocks did not expect the job ids to be ints, crashing when encountering...
Make NiceSort treat integers well
NiceSort is invoked on arrays that may contain strings, but in othersituations can contain ints as well. As this surprisingly makes sense,add a tiny modification to make NiceSort work in these conditions.
Merge branch 'stable-2.9' into stable-2.10
Conflicts: NEWS: take both additions configure.ac: ignore revision bump
Revision bump for 2.9.5
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Set release date for 2.9.5
A quick fix for broken 'make distcheck'
The offending target 'test/hs/offline-test.sh' requires the dependencieseven when the test isn't actually run. This is because currently ourMakefile doesn't distinguish between prerequisites for building testscripts and running them....
Fix tab/spaces indentation in Makefile.am
At some places a few items of lists use 8 spaces, while we use tabseverywhere (as makefiles require anyway). This patch unifies theindentation.
Merge branch 'stable-2.10' into stable-2.11
Add NEWS entry for 2.9.5
Describe the changes since 2.9.4 for the soon-to-be-releasedversion 2.9.5.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Merge branch 'stable-2.8' into stable-2.9
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix expression describing optional parameters
The NIC's network and vlan are also newly added, hence need to beconsidered optional to remain backwards compatible.
gitignore the files introduced by the compilation change
In particular, the test object and interface files and Makefile.ghc
Show Haskell dependencies when compiling
Instead of showing the long GHC command line, show which file is beingcompiled and what its dependencies are. This gives more meaningfulinformation to developers.
Compile every Haskell object file separately
This allows parallel compilation of all targets, including the Haskellobject files. While re-starting GHC for every Haskell source almostdoubles the total CPU time, allowing it run in parallel at the end cuts...
Add configuration options for Haskell compilation variants
Previously this was addressed by Makefile variables, which meant thatit was possible to build the same binary with different compilationoptions without recompiling.
Fix the generation of ListConstants with VPATH
When the build directory differs from the source directory, theautomatic variable '$<' points to the file found by prepending the path.This causes double '../' to be inserted when combined with'$(abs_top_srcdir)'. This patch fixes this....
Fix net-common script in case of routed mode
setup_route should run in case or routed mode and notopenvswitch.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>...
Improve locale generation instructions
Precise has a different way to handle locales w.r.t. debian distributions.This patch adds code to make the distinction.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>...
Document hspace's SPN parameters
With the introduction of spindles as resources, the machine-readableoutput of hspace(1) has been extended. Document the additional parameters.
Document spindles in TSPEC
As spindles are now considered a resource, they are also output inthe TSPEC parameter of hspace --machine-readable. Document thisinterface change.
Add missing package to build_chroot
A package was missing both from the Precise chroot and from the generic chroot.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Let the instance's tuple of nodes start with the primary
Before the tuple of nodes of an instance was created from a set, listingthe nodes in alphabetical order. This patch ensures that the primarynode is always the first one in the list.
Fix the test that checks for the order of instance's nodes
The test checks if the first node in the tuple is the primary node, butbecause it names the node so that it's the least one alphabetically, thetest always succeeds. This fixes the test.
man: Update the manual page of gnt-node FAILOVER
In particular explicitly note that a node failover causes all instancesto be stopped and later restarted (issue #724).
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Check the existence of system users and groups at bootstrap
Before, if any of these were missing, the creation of a cluster failedand the cluster remained in an inconsistent state, without thepossibility to destroy it or to re-create it (#603).
This patch calls 'GetEnts' during bootstrap, which tries to read all...
Configure luxid to run with the same user as masterd
Currently masterd and luxid run under separate user accounts, whichmakes it impossible for them to share access to the job queue.This patch adds the easiest fix - to configure luxid to run under thesame user as masterd....
Verify the result of opportunistic union
Verify that the result returned by the opportunistic union correctlyreflects the state change: locks not in the result are not changed,locks in the result are as requested.
Verify that opportunistic union only adds to the locks held
Add a test verifying that by opportunistic union the set of locksheld, and the level at which the locks are held, only increases.
Use opportunisticUnion in Arbitrary LockAllocation
Also allow the newly exported function opportunisticLockUnion in theconstruction of arbitrary lock allocations.
Support opportunistic lock allocation
Add a function to opportunistically allocate as many locks of a givenset as possible. This is equivalent to sequentially try the locks inincreasing lock order after restricting to those locks where therequired owner state is higher than the currently held one....
Generate arbitrary :: LockAllocation according to interface
While we know that freeLocks and intersectLocks are defined in termsof updateLocks, it is still cleaner to test against the interface: themodule Ganeti.Locking.Allocation exports several functions, and all...
Add a convenience function to restrict locks to a given set
As it is an operation the locking daemon is supposed to provide,add a function restricting the locks of a user to a given listand release all other locks.
Conflicts: lib/cmdlib/instance.py: manually apply 0973f9ed on...
Improve job status assert affected by race condition
In the sliver of time between choosing a waiting job to be executed andtrying to acquire locks for its execution, the status of the job can bechanged to canceling. An assert checking the job status neglected to...
Export and import Disk/NIC name
Name of Disk/NIC were not exported during backup until now.Use the exported info during gnt-backup import.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix backup import in case NIC is inside a network
Network UUID is written in .ini file during backup exportbut is not used by _ReadExportParams(). This patch fixes it.
Please note that in case a network is given, link and mode shouldnot be included in NIC options....
Override get() method of ConfigParser
During backup import/export SafeConfigParser() is used tosave/restore instance's configuration. There is a possibility if anexport is done with a different Ganeti version, a specific value notto be saved during export (e.g. the NIC/Disk name) but still...
Add precise to build_chroot
Modify the build_chroot script to support Ubuntu Precise as a targetenvironment.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Updating security doc wrt to SSL security
This patch updates the security document with respect tothe recent changes in RPC security. For details seedesign-node-security.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Update design docs for 2.11
Updating the status of the design docs in 2.11.
Smooth renewal of client certificates
This patch fixes another chicken-and-egg problem whichoccurred when the node certificates get renewed. Whenrenewing a node certificate, the previous certificatehas to be used to update the configuration. To address...
Enforce the order restrictions on group locks
Disallow requests for an exclusive lock, if the owner alreadyholds a shared (and only shared) lock on a group lock that lockbelongs to.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Document restrictions on lock implication
As owning a group lock has the same effect as owning agroup of locks, some restrictions have to be added toto avoid dead locks. Document them.
Make hbal deal with no-LVM storage space properly
Since 2.6, hbal crashes when used on a cluster where noLVM storage is enabled at all. The problem is that italways queries for fields that only sometimes makesense for certain types of storage. This patch will...
Revision bump for the 2.10.0 release
Remove the ~rc3 suffix from the version for the official 2.10.0 release.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Update NEWS file for 2.10.0
Mention the bug fix commited to the 2.8 branch and set the release dateto tomorrow.
Efficiently verify consistent lock requests
Most requests for lock updates are consistent and mentionevery lock only once. So verify this property efficientlyby comparing lengths. Only if the length do not coincidego through the actual (quadratic) comparison to find a...
Add a test verifying that lock-implication is honored
Locks can be included in one another. This inclusion can be violatedin two ways, viz.,- A holds a lock L and B holds an exclusive lock L belongs to, and- A holds an exclusive lock and B holds a lock L belongs to....
Support lock implication in allocation
Make the module abstractly handling lock allocationhonor the additional restrictions caused by lockimplications.
Verify that every owner mentioned as blocking actually is
If a request is blocked by multiple lock owners, verify that eachsingle one of them actually blocks the request. In other words,verify that, whenever all but one release their lock, the requeststill does not succeed....
Verify that the set of blocking owners is big enough
When a request is blocked, the list of blocking ownersshould exhaust all blocking reasons, i.e., if thoseowners release all their locks, the update must succeed.
Add a convenience function to free all locks of an owner
While freeing all locks of a single owner can easily be definedout of listLocks and updateLocks, it is worth having this functionin its own right. For example, it will be needed when the death of...
Make TestLock an instance of Lock
Equip the type of test locks with a simple group structureconsisting of two lock groups and a big lock.
Make GanetiLocks an instance of Lock