Add more general conf. handling functions to WConfd's Monad
Add functions for reading and writing the configuration. Writingtriggers the asynchronous worker which saves the configuration into themaster file. The actual action that saves the configuration is passed as...
Add functions for saving the configuration to Ganeti.Config
.. so that it can be not only read, but also written to the masterconfiguration file.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add a module for creating and using asynchronous workers
Each such worker has its own background thread and performs a giventask when triggered. Triggers arriving when the worker is processing areaccumulated and processed together in a subsequent worker action....
A separate module for functions for atomic file operations
Utils is getting too big, so better split this new set of functions intoa separate sub-module. This also allows us to use ResultG there.
Add a function for lifting ResultT from a MonadBase
Often we have `ResultT e IO a` as the return type of our primitivefunctions and we need to lift them into a monad stack that is also basedon IO. This function accomplishes this by lifting a ResultT in a base...
Add a type alias for "ResultT GanetiException IO"
This type occurs often in the upcoming code so it's worth having analias for it.
It's included in Errors instead of Utils, because it depends onGanetiException, and importing Errors from Utils causes a cyclic...
Add logging functions lifted into MonadBase
Calling logging functions in a monad transformer stack requiresinstances for all the transformers.This patch adds a module with functions named just as in Ganeti.Logging,but lifted into MonadBase. This allows using them in any transformer...
Add a MonadLog instance for strict StateT and IdentityT
This will be used for asynchronous workers, for which it's convenient touse StateT to keep their internal state.
Require lifted-base >= 0.1.2 for lifted IORef operations
This allows to use them in any IO-based monad.
Add default arguments for WConfd to scripts
In particular, this enables debug logging for WConfd on our testenvironment.
Fix tab/spaces indentation in Makefile.am
At some places a few items of lists use 8 spaces, while we use tabseverywhere (as makefiles require anyway). This patch unifies theindentation.
Merge branch 'stable-2.11' into master
Merge branch 'stable-2.10' into stable-2.11
Merge branch 'stable-2.9' into stable-2.10
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Merge branch 'stable-2.8' into stable-2.9
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix expression describing optional parameters
The NIC's network and vlan are also newly added, hence need to beconsidered optional to remain backwards compatible.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
gitignore the files introduced by the compilation change
In particular, the test object and interface files and Makefile.ghc
Show Haskell dependencies when compiling
Instead of showing the long GHC command line, show which file is beingcompiled and what its dependencies are. This gives more meaningfulinformation to developers.
Compile every Haskell object file separately
This allows parallel compilation of all targets, including the Haskellobject files. While re-starting GHC for every Haskell source almostdoubles the total CPU time, allowing it run in parallel at the end cuts...
Add configuration options for Haskell compilation variants
Previously this was addressed by Makefile variables, which meant thatit was possible to build the same binary with different compilationoptions without recompiling.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Fix the generation of ListConstants with VPATH
When the build directory differs from the source directory, theautomatic variable '$<' points to the file found by prepending the path.This causes double '../' to be inserted when combined with'$(abs_top_srcdir)'. This patch fixes this....
Fix net-common script in case of routed mode
setup_route should run in case or routed mode and notopenvswitch.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>...
Improve locale generation instructions
Precise has a different way to handle locales w.r.t. debian distributions.This patch adds code to make the distinction.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>...
Document hspace's SPN parameters
With the introduction of spindles as resources, the machine-readableoutput of hspace(1) has been extended. Document the additional parameters.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Document spindles in TSPEC
As spindles are now considered a resource, they are also output inthe TSPEC parameter of hspace --machine-readable. Document thisinterface change.
Add missing package to build_chroot
A package was missing both from the Precise chroot and from the generic chroot.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Let the instance's tuple of nodes start with the primary
Before the tuple of nodes of an instance was created from a set, listingthe nodes in alphabetical order. This patch ensures that the primarynode is always the first one in the list.
Fix the test that checks for the order of instance's nodes
The test checks if the first node in the tuple is the primary node, butbecause it names the node so that it's the least one alphabetically, thetest always succeeds. This fixes the test.
man: Update the manual page of gnt-node FAILOVER
In particular explicitly note that a node failover causes all instancesto be stopped and later restarted (issue #724).
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Check the existence of system users and groups at bootstrap
Before, if any of these were missing, the creation of a cluster failedand the cluster remained in an inconsistent state, without thepossibility to destroy it or to re-create it (#603).
This patch calls 'GetEnts' during bootstrap, which tries to read all...
Configure luxid to run with the same user as masterd
Currently masterd and luxid run under separate user accounts, whichmakes it impossible for them to share access to the job queue.This patch adds the easiest fix - to configure luxid to run under thesame user as masterd....
Verify the result of opportunistic union
Verify that the result returned by the opportunistic union correctlyreflects the state change: locks not in the result are not changed,locks in the result are as requested.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Verify that opportunistic union only adds to the locks held
Add a test verifying that by opportunistic union the set of locksheld, and the level at which the locks are held, only increases.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Use opportunisticUnion in Arbitrary LockAllocation
Also allow the newly exported function opportunisticLockUnion in theconstruction of arbitrary lock allocations.
Support opportunistic lock allocation
Add a function to opportunistically allocate as many locks of a givenset as possible. This is equivalent to sequentially try the locks inincreasing lock order after restricting to those locks where therequired owner state is higher than the currently held one....
Generate arbitrary :: LockAllocation according to interface
While we know that freeLocks and intersectLocks are defined in termsof updateLocks, it is still cleaner to test against the interface: themodule Ganeti.Locking.Allocation exports several functions, and all...
Add a convenience function to restrict locks to a given set
As it is an operation the locking daemon is supposed to provide,add a function restricting the locks of a user to a given listand release all other locks.
Conflicts: lib/cmdlib/instance.py: manually apply 0973f9ed on...
Improve job status assert affected by race condition
In the sliver of time between choosing a waiting job to be executed andtrying to acquire locks for its execution, the status of the job can bechanged to canceling. An assert checking the job status neglected to...
Export and import Disk/NIC name
Name of Disk/NIC were not exported during backup until now.Use the exported info during gnt-backup import.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix backup import in case NIC is inside a network
Network UUID is written in .ini file during backup exportbut is not used by _ReadExportParams(). This patch fixes it.
Please note that in case a network is given, link and mode shouldnot be included in NIC options....
Override get() method of ConfigParser
During backup import/export SafeConfigParser() is used tosave/restore instance's configuration. There is a possibility if anexport is done with a different Ganeti version, a specific value notto be saved during export (e.g. the NIC/Disk name) but still...
Add precise to build_chroot
Modify the build_chroot script to support Ubuntu Precise as a targetenvironment.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Updating security doc wrt to SSL security
This patch updates the security document with respect tothe recent changes in RPC security. For details seedesign-node-security.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Update design docs for 2.11
Updating the status of the design docs in 2.11.
Smooth renewal of client certificates
This patch fixes another chicken-and-egg problem whichoccurred when the node certificates get renewed. Whenrenewing a node certificate, the previous certificatehas to be used to update the configuration. To address...
Enforce the order restrictions on group locks
Disallow requests for an exclusive lock, if the owner alreadyholds a shared (and only shared) lock on a group lock that lockbelongs to.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Document restrictions on lock implication
As owning a group lock has the same effect as owning agroup of locks, some restrictions have to be added toto avoid dead locks. Document them.
Make hbal deal with no-LVM storage space properly
Since 2.6, hbal crashes when used on a cluster where noLVM storage is enabled at all. The problem is that italways queries for fields that only sometimes makesense for certain types of storage. This patch will...
Revision bump for the 2.10.0 release
Remove the ~rc3 suffix from the version for the official 2.10.0 release.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Update NEWS file for 2.10.0
Mention the bug fix commited to the 2.8 branch and set the release dateto tomorrow.
Efficiently verify consistent lock requests
Most requests for lock updates are consistent and mentionevery lock only once. So verify this property efficientlyby comparing lengths. Only if the length do not coincidego through the actual (quadratic) comparison to find a...
Add a test verifying that lock-implication is honored
Locks can be included in one another. This inclusion can be violatedin two ways, viz.,- A holds a lock L and B holds an exclusive lock L belongs to, and- A holds an exclusive lock and B holds a lock L belongs to....
Support lock implication in allocation
Make the module abstractly handling lock allocationhonor the additional restrictions caused by lockimplications.
Verify that every owner mentioned as blocking actually is
If a request is blocked by multiple lock owners, verify that eachsingle one of them actually blocks the request. In other words,verify that, whenever all but one release their lock, the requeststill does not succeed....
Verify that the set of blocking owners is big enough
When a request is blocked, the list of blocking ownersshould exhaust all blocking reasons, i.e., if thoseowners release all their locks, the update must succeed.
Add a convenience function to free all locks of an owner
While freeing all locks of a single owner can easily be definedout of listLocks and updateLocks, it is worth having this functionin its own right. For example, it will be needed when the death of...
Make TestLock an instance of Lock
Equip the type of test locks with a simple group structureconsisting of two lock groups and a big lock.
Make GanetiLocks an instance of Lock
Add a type class describing types that can serve as locks
Besides the lock order and Show, an additional datum is needed in orderto describe lock inclusion, used, e.g., for group locks. So add a typeclass describing this property.
Verify that releasing a lock always succeeds
There are no restrictions on releasing a lock; so this operationhas to succeed unconditionally. Verify this.
Add missing documentation to exported function listLocks
The module Ganeti.Locking.Allocation exports the function listLocks,hence a documentation string is required. Add it.
Increase the waiting time in gnt-job cancel test
The wait that is used for gnt-job cancel could cause flakiness if thewait is small enough for the job not to get canceled. This patchdoubles the waiting time - more than that is unfortunately probably abug....
Conflicts: NEWS: take both additions configure.ac: ignore suffix bump...
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix 'gnt-group add' output in QA
The command 'gnt-group add' might return a list of jobs, therefore, wecan print the job's output during the QA, in a way similar to that of'gnt-cluster verify', for example.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Constant for instance communication network mode
Create a new constant to hold the instance communication network modeas this constant will be necessary during the QA, and update thegeneral documentation about the constants related to the instancecommunication mechanism....
Add '-c | --instance-communication' flag to instance modify
Enable/disable instance comm via 'gnt-instance modify'
This patch adds the logic necessary to enable/disable the instancecommunication in a running instance via 'gnt-instance modify'. Withinstance communication enabled, the instance gets a new NIC that is...
Refactor instance comm NIC name creation
Refactor name creation for the NICs used in instance communication.These names are generated based on a prefix and the instance name.Also, these names must be unique within a single instance.
Extend 'AssertIn' with an optional error message
Extend 'AssertIn' with an optional error message, thus making iteasier to find the cause of an error when errors occur during the QA.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Add QA for instance communication with instance modify
Extend QA with tests related to feature that adds the instancecommunication mechanism to running instances, via 'gnt-instancemodify'.
Fix copy of NIC objects to be consistent with the other call
... which can be found just right below in the same module.
Fix param name to conform to convention of optional params
The opcode parameters can be optional and parameters that are optionalhave their names prefixed by 'pOpt'. This patch fixes with parameterto conform to this convention.
Fix whitespace and typos in comments
Disable pylint warnings for ignored files
Some files (like RPC stubs) are marked to be ignored by pylint.However pylint emits warning messages to remind of such files.This disables the warnings.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Disable deprecation warnings in PyRPC
Since on Debian Squeeze we have TH version 2.4, we need to use 'report'instead of 'reportWarning/reportError'. However, 'report' is deprecatedin later versions, therefore we need to disable the warning.
The warning doesn't manifest on Debian Squeeze or Wheezy, but causes...
Make the lock allocation part of WConfD's state
As WConfD is supposed to be the authoritative source forconfiguration and locks, it needs to have access to the lockstate as well.
Add a module for the Ganeti lock structure
Already provide a module holding the structure of the locksavailable in Ganeti, so that the overall structure of WConfDcan be implemented. The actual locks still have to be added.
Derive Ord JobId
Make the type JobId an instance of Ord. While the order itselfdoes not matter, it is a prerequisite to be an instance of Ordto be used as the type of lock owners.
Verify atomicity of lock updates
Add a test verifying the atomicity of an update, i.e., if an updatesucceeds, then all requests must be satisfied in the final state, andif it fails, the state must be unchanged.
Verify lock stability over updates
Add a test that verifies that updates by one user do notaffect the locks held by a different user.
Verify Basic property for lock allocation
Verify the minimal consistency property for any formof lock handling: if a user holds an exclusive lock,then no other user can hold the same lock (neitherexclusively, nor shared).
Add function describing lock updates
Add the pure part of the mechanism of updating locks. To allowfor efficient waiting on locks, return the set of owners of the lockson which the operation is blocked.
Add a function to leave the list monad
The list monad provides convenient syntax for non-deterministicalgorithms. Add a function leaving that monad with this intuitionin mind.
Add data structure representing current lock status
To allow for jobs as processes, a central daemon (wconfd) willhandle allocation and release of locks. Add an appropriate datastructure to describe the current status of the locks.
Rename 'BUILT_PYTHON_SOURCES' to 'built_python_sources'
This patch fixes the warnings produced by ./autogen.sh
Makefile.am:310: variable `BUILT_PYTHON_SOURCES' is defined but no program or Makefile.am:310: library has `BUILT_PYTHON' as canonical name (possible typo)...
Update design doc wrt to improved SSL design
This patch updates the design document of Ganeti's nodesecurity to make it consistent with the implementationand to extend it with a couple of suggestions to improvethe SSL security even more.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Test node certificate renewal in QA
This extends the QA by explicitely testing the renewalof SSL client certificates.
Use node UUID as client certificate serial number
It turns out, that some implementations of OpenSSL are morepedantic in checking the certficates than others. In thisparticular case, the SSL connection could not beestablished when the serial number of the certificates...
Revert "Temporarily remove SSL changes from NEWS file"
This reverts commit 89671df1c, which temporarily removedthe announcement of improved SSL security in the NEWS file.As this patch series fixes the SSL problem that causedthe disabling of the features, we are adding back the...
Revert "Disabling client certificate usage"
This reverts commit 45f75526b848, which was introduced totemporarily disable the implementation of SSL clientcertificates. As this patch series fixes the reason forthe disabling, we are rolling back the patch....
Add functions for atomic operations on files
Function 'atomicUpdateFile' extends 'atomicWriteFile' that allows anyaction to be run on a temporary file.
Function 'atomicUpdateLockedFile' additionally locks the original fileusing flock and checks if its state conforms to the last one....
Add mtime/ctime to ConfigData
It's used on the Python side, it's been missing on the Haskell side.
Re-export liftIO from BasicType
This makes usage of ReaderT's liftIO more convenient.
Add lib/rpc/stubs/ and src/ganeti-wconfd to .gitignore
All stubs are generated from Haskell during compilation.Also ignore the executable of the new daemon.
Fix an ambiguity in the documentation for GetNodesSshPorts
This ambiguity was introduced by adding the WConfd client.
Add the Python client for WConfD
The client combines the abstract client class and the WConfDstub to provide a Python interface to WConfD.