Add a function for lifting ResultT from a MonadBase
Often we have `ResultT e IO a` as the return type of our primitivefunctions and we need to lift them into a monad stack that is also basedon IO. This function accomplishes this by lifting a ResultT in a base...
Add a type alias for "ResultT GanetiException IO"
This type occurs often in the upcoming code so it's worth having analias for it.
It's included in Errors instead of Utils, because it depends onGanetiException, and importing Errors from Utils causes a cyclic...
Add logging functions lifted into MonadBase
Calling logging functions in a monad transformer stack requiresinstances for all the transformers.This patch adds a module with functions named just as in Ganeti.Logging,but lifted into MonadBase. This allows using them in any transformer...
Add a MonadLog instance for strict StateT and IdentityT
This will be used for asynchronous workers, for which it's convenient touse StateT to keep their internal state.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Merge branch 'stable-2.11' into master
Merge branch 'stable-2.10' into stable-2.11
Merge branch 'stable-2.9' into stable-2.10
Support opportunistic lock allocation
Add a function to opportunistically allocate as many locks of a givenset as possible. This is equivalent to sequentially try the locks inincreasing lock order after restricting to those locks where therequired owner state is higher than the currently held one....
Add a convenience function to restrict locks to a given set
As it is an operation the locking daemon is supposed to provide,add a function restricting the locks of a user to a given listand release all other locks.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Enforce the order restrictions on group locks
Disallow requests for an exclusive lock, if the owner alreadyholds a shared (and only shared) lock on a group lock that lockbelongs to.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Make hbal deal with no-LVM storage space properly
Since 2.6, hbal crashes when used on a cluster where noLVM storage is enabled at all. The problem is that italways queries for fields that only sometimes makesense for certain types of storage. This patch will...
Efficiently verify consistent lock requests
Most requests for lock updates are consistent and mentionevery lock only once. So verify this property efficientlyby comparing lengths. Only if the length do not coincidego through the actual (quadratic) comparison to find a...
Support lock implication in allocation
Make the module abstractly handling lock allocationhonor the additional restrictions caused by lockimplications.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add a convenience function to free all locks of an owner
While freeing all locks of a single owner can easily be definedout of listLocks and updateLocks, it is worth having this functionin its own right. For example, it will be needed when the death of...
Make GanetiLocks an instance of Lock
Add a type class describing types that can serve as locks
Besides the lock order and Show, an additional datum is needed in orderto describe lock inclusion, used, e.g., for group locks. So add a typeclass describing this property.
Add missing documentation to exported function listLocks
The module Ganeti.Locking.Allocation exports the function listLocks,hence a documentation string is required. Add it.
Conflicts: NEWS: take both additions configure.ac: ignore suffix bump...
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Constant for instance communication network mode
Create a new constant to hold the instance communication network modeas this constant will be necessary during the QA, and update thegeneral documentation about the constants related to the instancecommunication mechanism....
Add '-c | --instance-communication' flag to instance modify
Fix param name to conform to convention of optional params
The opcode parameters can be optional and parameters that are optionalhave their names prefixed by 'pOpt'. This patch fixes with parameterto conform to this convention.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Fix whitespace and typos in comments
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Disable deprecation warnings in PyRPC
Since on Debian Squeeze we have TH version 2.4, we need to use 'report'instead of 'reportWarning/reportError'. However, 'report' is deprecatedin later versions, therefore we need to disable the warning.
The warning doesn't manifest on Debian Squeeze or Wheezy, but causes...
Make the lock allocation part of WConfD's state
As WConfD is supposed to be the authoritative source forconfiguration and locks, it needs to have access to the lockstate as well.
Add a module for the Ganeti lock structure
Already provide a module holding the structure of the locksavailable in Ganeti, so that the overall structure of WConfDcan be implemented. The actual locks still have to be added.
Derive Ord JobId
Make the type JobId an instance of Ord. While the order itselfdoes not matter, it is a prerequisite to be an instance of Ordto be used as the type of lock owners.
Add function describing lock updates
Add the pure part of the mechanism of updating locks. To allowfor efficient waiting on locks, return the set of owners of the lockson which the operation is blocked.
Add a function to leave the list monad
The list monad provides convenient syntax for non-deterministicalgorithms. Add a function leaving that monad with this intuitionin mind.
Add data structure representing current lock status
To allow for jobs as processes, a central daemon (wconfd) willhandle allocation and release of locks. Add an appropriate datastructure to describe the current status of the locks.
Use node UUID as client certificate serial number
It turns out, that some implementations of OpenSSL are morepedantic in checking the certficates than others. In thisparticular case, the SSL connection could not beestablished when the serial number of the certificates...
Revert "Disabling client certificate usage"
This reverts commit 45f75526b848, which was introduced totemporarily disable the implementation of SSL clientcertificates. As this patch series fixes the reason forthe disabling, we are rolling back the patch....
Add functions for atomic operations on files
Function 'atomicUpdateFile' extends 'atomicWriteFile' that allows anyaction to be run on a temporary file.
Function 'atomicUpdateLockedFile' additionally locks the original fileusing flock and checks if its state conforms to the last one....
Add mtime/ctime to ConfigData
It's used on the Python side, it's been missing on the Haskell side.
Re-export liftIO from BasicType
This makes usage of ReaderT's liftIO more convenient.
Generate WConfD Python stubs from Haskell
The generation is included in hs2py.hs, together with otherHaskell to Python code generators.
Generate Python stubs by inspecting functions using TH
The names of the arguments of generated Python methods are derived fromHaskell types to be as descriptive as possible.
Add the WConfD daemon itself
The daemon exposes the declared functions in Ganeti.WConfd.Core to RPCclients (currently just 'echo').
Add the WConfD daemon to build configuration files
Also list it in the Haskell datatype, constants, Python constants andtest configuration.
Add a module for WConfd core functions
There are no exported functions yet, except for `echo`, which justreturns its argument back to a client.
Add a MonadLog instance for strict variant of RWS
This allows to log in monad stacks containing RWS.
Add a monad for running all WConfD functions
This monad encapsulates working with the daemon and client state, aswell as failures and IO operations.
Add a generic RPC module
The main function in this module takes a list of names of functions anduses TH to create an RPC server, represented as a value of type Handler(from Ganeti.UDSServer). This can be readily used to create a daemonthat dispatches RPC requests to the given functions....
Add common functions for working with TH's Type
In particular, functions for extracting types of arguments from afunction type, and for uncurrying functions with an arbitrary number ofarguments.
Another workaround for GHC linking error "unknown symbol"
This error appears after adding the dependency to lifted-base.See also #683 and ee7caf27.
I surmise that something like this is happening: Some code, that usesthe library, is only referenced through Template Haskell. A probable bug...
Add instances of MonadBase and MonadControl for ResultT
This allows to use lifted operations like 'fork' or 'bracket' insideResultT.
Allow clients of UDSServer to use different monads
.. as long as they're instances of "MonadBaseControl IO" and "MonadLog".This allows the UDSServer to call functions like "fork" within monadssuch as "ResultT e IO" or "ReaderT IO".
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Merge branch 'stable-2.8' into stable-2.9
Fix integer overflow problem in hbal
waitForJobs in src/Ganeti/Jobs.hs has an integer overflow that (at least onamd64) causes it to break after waiting for ~10 minutes. This results in hbalsleeping forever (when compiled with squeeze's ghc 6.12.1) or crashing (when...
Add missing space
Also, refactor the line to keep it under 80 chars.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Connect new groups to the instance communication network
When a new group is added, if the instance communication network isenabled, then this group must also be connected to this network.
'LUClusterSetParams' creates the instance communication net
Extend 'LUClusterSetParams' to create the user-supplied instancecommunication network in case this network does not exist. Note thatif the user-supplied network already exists, nothing needs to be done...
Add 'instance_communication_network' to 'OpClusterSetParams'
... and update the unit tests.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add 'instance_communication_parameter' to 'Cluster'
Fix indentation
Remove the HTOOLS configuration variable
.. and update the code that uses it.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Conflicts: lib/client/gnt_node.py: trivial src/Ganeti/Query/Query.hs: import ALL the functions
Consider job-IDs queried for twice only once
As reading jobs from disk is an expensive operation, when queryingfor jobs, we optimize by considering which values the job-id is askedfor in the filter. As any reasonable person would not add the sameclause twice in an Or-clause, the implicit assumption was that the...
Fix 'JobIdListOnly' type from 'List' to 'Map'
Allow classic queries to use either names or UUIDs
When UUIDs are used in CLI commands, such addressing of objects failsor succeeds inconsistently across object types. Worse yet, some callsdo not fail, but simply return no result. This is due to the way the...
luxid: give stern warnings about debug mode
Luxid as it is can leak private and secret parameters by loggingall requests as they arrive, before any preprocessing is done.
Warn the user stern warnings about this.
Signed-off-by: Santi Raffa <rsanti@google.com>...
OpCodes: modify InstanceReinstall for private, secret params
Modify InstanceReinstall to accept and process private and secretparameters.
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
OpCodes: modify InstanceCreate for private, secret params
Modify InstanceCreate to accept process private and secret parameters.
OpCodes: modify ClusterSetParams for private parameters
Modify ClusterSetParams to accept and process private parameters.
OpCodes: modify InstanceSetParams for private parameters
Modify InstanceSetParams to accept and process private parameters.
Add private OS parameters to cluster and instance conf
This updates objects, constructors and mocks for Instance and Clusterobjects in Python and Haskell.
Add Private types to Python, Haskell
This commit adds the private containers to Python and Haskell.
serializer: emit and encode Private values
For inbound data the simplest, safest thing to do is to traverse allJSON right after encoding and search for private parameters by key.
This ensures that all consumers of this data get Private valuestransparently and consistently; the serializing methods don't have to...
Remove wildcard luxi operation matching in luxid
In that way, we explicitly name the operations that are nothandled by luxid and explain the reason. In particular, wecan be sure that newly added luxid operations won't be forgottenin luxid.
Implement QueryExports in luxid
...by handling as a classical query, using that queries forexport are already implemented. Note that QueryExport is slightlydifferent from other Query* Luxi requests, in that the fields arenot passed with the request, but have a fixed value....
Add CLI and OpCode instance communication option
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Generate TAP names ('gnt.com.%d') for instance comm NICs
Add metadata daemon
... including, user, group, daemonizing code with command lineoptions, integration with the Snap HTTP server, and logic.
Change return type of internal rmJob
...to also provide the job itself. In this way, the function canalso be used for tasks that require temporarily removing a jobfrom the queue.
When enqueuing new jobs, respect job ID
When adding new jobs, don't add them at the end, but at aposition that fits with their job id. In this way, we canbuild operations that require fully dequeing a job an addingit later after some modifications.
Provide a function to change the priority of a queued job
There is a separation of responsibilities here. For jobs stillin the queue, it is the responsibility of the queue (scheduler),for started jobs, the job itself has to take care of it. To avoidthe job transitioning inbetween, it is temporarily dequeued during...
Implement ChangeJobPriority in luxid
For jobs still queued, we ask the queue to change the priority,and replicate the changed job. For jobs that have already beenstarted, we have to contact the job directly, which, at the moment,means forwarding the request to masterd....
Add a function changing the priority of an opcode
This pure function follows the semantic that an opcode, includingits priority, may only be changed if the opcode is not finalized.
Add a function to change the priority of a job
...by changing the priority of the non-finished opcodes.
Add functions for manipulating errors in Result(T)
There is often need to manipulate these errors, for example to convert a Stringfrom Result into an exception. These functions make this easier.
Function 'toErrorStr' lifts 'Result' to any 'MonadError'. This is useful...
Remove FromString in favor of Error from standard libraries
They have the very same functionality, and using our own FromString onlycauses unnecessary code duplication.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>...
Add Alternative instances for GenericResult and ResultT
This allows to use Alternative specific combinators, namely `optional`.
Cherry-pick of 78209a84b0f6be27fd381ac2...
Catch exceptions in 'readConfig' in Ganeti.Config
.. and convert them into a "Result".This is accomplished by using "liftIO" from ResultT.
Catch errors in 'lockFile' in Ganeti.Utils
Opening a file can fail as well, so catch errors there too.
Assert correct usage of AndRestArguments fields
The special field andRestArguments is intended to be only usedas the last field of an object, catching all remaining keys.Add a compile-time check to verify that it is used correctly.
Fix whitespace
Add andRestArguments to IDiskParams
In this way, we cann pass through the opaque parametersrequired for disk creation and modification in the case ofexternal storage.
Add function providing the canonical andRestArguments
The field catching the remaining fields will always be of the sameshape, so add a function for this to make usage simple.
Add additional constructor AndRestArguments to OptionalType
A field of this type will capture all the remaining fieldsof an object as JSValues. Obviously, the intended use isto have precisely one such field. This mechanism will allowto pass opaque values trough, as it is, e.g., required for...
User new error handling functions in SubmitJobToDrainedQueue
This somewhat shortens and simplifies the code.
Use new error functions when querying locks
This helps to handle errors coming from the Luxi client.
Use new error handling functions for querying jobs
Since we already touched getJobIDs, and this function is already basedon ResultT, use new error functions here as well.
Update getDirJobIDs to use ResultT
Also simplify code and remove unused functions.