Fix 'gnt-group add' output in QA
The command 'gnt-group add' might return a list of jobs, therefore, wecan print the job's output during the QA, in a way similar to that of'gnt-cluster verify', for example.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Constant for instance communication network mode
Create a new constant to hold the instance communication network modeas this constant will be necessary during the QA, and update thegeneral documentation about the constants related to the instancecommunication mechanism....
Add '-c | --instance-communication' flag to instance modify
Fix copy of NIC objects to be consistent with the other call
... which can be found just right below in the same module.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Fix param name to conform to convention of optional params
The opcode parameters can be optional and parameters that are optionalhave their names prefixed by 'pOpt'. This patch fixes with parameterto conform to this convention.
Fix whitespace and typos in comments
Disable pylint warnings for ignored files
Some files (like RPC stubs) are marked to be ignored by pylint.However pylint emits warning messages to remind of such files.This disables the warnings.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Disable deprecation warnings in PyRPC
Since on Debian Squeeze we have TH version 2.4, we need to use 'report'instead of 'reportWarning/reportError'. However, 'report' is deprecatedin later versions, therefore we need to disable the warning.
The warning doesn't manifest on Debian Squeeze or Wheezy, but causes...
Make the lock allocation part of WConfD's state
As WConfD is supposed to be the authoritative source forconfiguration and locks, it needs to have access to the lockstate as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add a module for the Ganeti lock structure
Already provide a module holding the structure of the locksavailable in Ganeti, so that the overall structure of WConfDcan be implemented. The actual locks still have to be added.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Derive Ord JobId
Make the type JobId an instance of Ord. While the order itselfdoes not matter, it is a prerequisite to be an instance of Ordto be used as the type of lock owners.
Verify atomicity of lock updates
Add a test verifying the atomicity of an update, i.e., if an updatesucceeds, then all requests must be satisfied in the final state, andif it fails, the state must be unchanged.
Verify lock stability over updates
Add a test that verifies that updates by one user do notaffect the locks held by a different user.
Verify Basic property for lock allocation
Verify the minimal consistency property for any formof lock handling: if a user holds an exclusive lock,then no other user can hold the same lock (neitherexclusively, nor shared).
Add function describing lock updates
Add the pure part of the mechanism of updating locks. To allowfor efficient waiting on locks, return the set of owners of the lockson which the operation is blocked.
Add a function to leave the list monad
The list monad provides convenient syntax for non-deterministicalgorithms. Add a function leaving that monad with this intuitionin mind.
Add data structure representing current lock status
To allow for jobs as processes, a central daemon (wconfd) willhandle allocation and release of locks. Add an appropriate datastructure to describe the current status of the locks.
Rename 'BUILT_PYTHON_SOURCES' to 'built_python_sources'
This patch fixes the warnings produced by ./autogen.sh
Makefile.am:310: variable `BUILT_PYTHON_SOURCES' is defined but no program or Makefile.am:310: library has `BUILT_PYTHON' as canonical name (possible typo)...
Merge branch 'stable-2.11' into master
Update design doc wrt to improved SSL design
This patch updates the design document of Ganeti's nodesecurity to make it consistent with the implementationand to extend it with a couple of suggestions to improvethe SSL security even more.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Test node certificate renewal in QA
This extends the QA by explicitely testing the renewalof SSL client certificates.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Use node UUID as client certificate serial number
It turns out, that some implementations of OpenSSL are morepedantic in checking the certficates than others. In thisparticular case, the SSL connection could not beestablished when the serial number of the certificates...
Revert "Temporarily remove SSL changes from NEWS file"
This reverts commit 89671df1c, which temporarily removedthe announcement of improved SSL security in the NEWS file.As this patch series fixes the SSL problem that causedthe disabling of the features, we are adding back the...
Revert "Disabling client certificate usage"
This reverts commit 45f75526b848, which was introduced totemporarily disable the implementation of SSL clientcertificates. As this patch series fixes the reason forthe disabling, we are rolling back the patch....
Add functions for atomic operations on files
Function 'atomicUpdateFile' extends 'atomicWriteFile' that allows anyaction to be run on a temporary file.
Function 'atomicUpdateLockedFile' additionally locks the original fileusing flock and checks if its state conforms to the last one....
Add mtime/ctime to ConfigData
It's used on the Python side, it's been missing on the Haskell side.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Re-export liftIO from BasicType
This makes usage of ReaderT's liftIO more convenient.
Add lib/rpc/stubs/ and src/ganeti-wconfd to .gitignore
All stubs are generated from Haskell during compilation.Also ignore the executable of the new daemon.
Fix an ambiguity in the documentation for GetNodesSshPorts
This ambiguity was introduced by adding the WConfd client.
Add the Python client for WConfD
The client combines the abstract client class and the WConfDstub to provide a Python interface to WConfD.
Generate WConfD Python stubs from Haskell
The generation is included in hs2py.hs, together with otherHaskell to Python code generators.
Add an RPC Python client for generated stub classes
The client provides _GenericInvoke(...) for a stub and uses its_GetSocketPath() for opening a Transport.
Generate Python stubs by inspecting functions using TH
The names of the arguments of generated Python methods are derived fromHaskell types to be as descriptive as possible.
Update Makefile.am to properly install generated stubs
This involves adding a new sets of variables for the directory.
Add a Python directory for RPC generated stubs
Directory "lib/rpc/stubs" will contains RPC stubs generated fromHaskell.
Let RPC clients handle their socket address
.. instead of AbstractClient itself. Also let every client call_InitTransport() as needed. This allows to determine socket addresseslater than during the initialization of a class.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Add the WConfD daemon itself
The daemon exposes the declared functions in Ganeti.WConfd.Core to RPCclients (currently just 'echo').
Add the WConfD daemon to build configuration files
Also list it in the Haskell datatype, constants, Python constants andtest configuration.
Add a module for WConfd core functions
There are no exported functions yet, except for `echo`, which justreturns its argument back to a client.
Add a MonadLog instance for strict variant of RWS
This allows to log in monad stacks containing RWS.
Add a monad for running all WConfD functions
This monad encapsulates working with the daemon and client state, aswell as failures and IO operations.
Add a generic RPC module
The main function in this module takes a list of names of functions anduses TH to create an RPC server, represented as a value of type Handler(from Ganeti.UDSServer). This can be readily used to create a daemonthat dispatches RPC requests to the given functions....
Add tests for Ganeti.THH.Types
In particular, test (de)serialization of "OneTuple".
Add a helper test method for checking deserialization
It tests if a deserialization method fails properly when an invalidinput is given.
Add common functions for working with TH's Type
In particular, functions for extracting types of arguments from afunction type, and for uncurrying functions with an arbitrary number ofarguments.
Another workaround for GHC linking error "unknown symbol"
This error appears after adding the dependency to lifted-base.See also #683 and ee7caf27.
I surmise that something like this is happening: Some code, that usesthe library, is only referenced through Template Haskell. A probable bug...
Add instances of MonadBase and MonadControl for ResultT
This allows to use lifted operations like 'fork' or 'bracket' insideResultT.
Allow clients of UDSServer to use different monads
.. as long as they're instances of "MonadBaseControl IO" and "MonadLog".This allows the UDSServer to call functions like "fork" within monadssuch as "ResultT e IO" or "ReaderT IO".
Add lifted-base to the list of requirements
This includes NEWS and INSTALL
Add dependencies for lifted-base to build_chroot
.. for both Debian distribution. Also check that the library is presentduring the configuration step.
Fix watcher tampering with instance userdown QA
The watcher was tampering with the instance user down QA because, whenthe watcher sees an instance marked an 'USER_down', it will cleanupthe instance, that is it will stop the instance, thus marking it as...
Rename some functions not to collide with opcode names
Rename some functions related to instance communication not to collidewith the naming convention used in the opcodes.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Refactor instance communication network add and connect
Factor out the opcodes 'OpNetworkAdd' and 'OpNetworkConnect' used in'LUClusterSetParams' and 'LUGroupAdd' in order to reduce codeduplication and keep the configuration of the instance communication...
Instance communication mechanism QA (for cluster only)
Extend QA with tests for the instance communication mechanism. Thesetests cover only the cluster creating the instance communicationnetwork and connecting it to the existing node group, and also if new...
Connect new groups to the instance communication network
When a new group is added, if the instance communication network isenabled, then this group must also be connected to this network.
'gnt-cluster modify' with '--instance-communication-network'
Extend CLI 'gnt-cluster modify' with'--instance-communication-network'. Given that the return type for'OpClusterSetParams' changed to optionally return a list of jobs, itis also necessary to handle the result of this opcode accordingly....
'LUClusterSetParams' creates the instance communication net
Extend 'LUClusterSetParams' to create the user-supplied instancecommunication network in case this network does not exist. Note thatif the user-supplied network already exists, nothing needs to be done...
Check prereq instance communication network in 'SetParams'
Later, the logical unit for 'OpClusterSetParams' will be responsiblefor creating the instance communication network in case it does notexist. For now, it is important to check whether the network the user...
Add 'instance_communication_network' to 'OpClusterSetParams'
... and update the unit tests.
Add helper to handle CLIs that optionally spawn several jobs
This helper function detects whether an opcode returned a list of jobs(i.e., a result of the type ht.TJobIdListOnly) and in this case ituses 'ganeti.cli.JobExecutor' to wait for the jobs and determine the...
Instance comm network from config instead of predefined
Add 'instance_communication_parameter' to 'Cluster'
Fix indentation
Correct test for existance of instances
Since python interprets an empty dictionary as 'False',testing for the existance of the 'instances' field inthe configuration has to be done by explicitely testingfor 'None'.
Correct number of empty lines
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Remove osparams_private when downgrading the configuration
The instance parameter "osparams_private" was newly introducedin 2.12. So remove it when downgrading the configuration to 2.11.
Remove osparams_private_cluster on config downgrade
The new cluster parameter "osparams_private_cluster" wasnewly introduced in 2.12. So remove it when downgradingto 2.11.
Make htools and luxid mandatory in the Makefile
Since Ganeti now uses luxid for core operations, it needs to be alwayspresent.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add a config. conditional for running Haskell coverage
Similarly to PY_UNIT, add HS_UNIT to run Haskell coverage only if wehave the required modules.
Remove the HTOOLS configuration variable
.. and update the code that uses it.
Fix start/stop scripts to run luxid unconditionally
and update the unit check the list of daemons.
Make luxid's Haskell dependencies mandatory
in the build configuration files.
Update the list dependencies in INSTALL
In particular, move luxid's dependencies to the section with mandatorydependencies.
Temporarily remove SSL changes from NEWS file
Since the current changes in the SSL handling havebeen partially disabled, remove their announcementin the NEWS file until there is something actuallyuser-facing happening.
Fix bracket style in rapi-workload
Pep8 complained about bracket indentation. This fixes it to satisfy Pep8as well as our style guide.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>(cherry-picked from ccded668c8a93fa25aaf21ddf5105270a04fe517)...
git-ignore automatically generated file tools/kvm-ifup-os
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Conflicts: lib/client/gnt_node.py: trivial src/Ganeti/Query/Query.hs: import ALL the functions
Gracefully handle queries for non-existing nodes
When adding a node, Ganeti checks whether the node is alreadypart of the cluster by querying for the node name. However,as queries are meant to return all nodes with the given name,it might well return the empty list when a new node is to be...
Consider job-IDs queried for twice only once
As reading jobs from disk is an expensive operation, when queryingfor jobs, we optimize by considering which values the job-id is askedfor in the filter. As any reasonable person would not add the sameclause twice in an Or-clause, the implicit assumption was that the...
Merge branch 'stable-2.10' into stable-2.11
Fix default for luxi clients in python
As masterd is going away, set default for all clients toluxid's socket.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.9' into stable-2.10
Fix 'devel/check_copyright' remote branch detection
... but do not try to check the copyrights for local branches which donot have a proper remote branch, which would fail at a later stepanyway.
Revision bump for 2.9.4
Set release date for 2.9.4
Remove query option from RAPI client
As all RAPI requests now go to luxid, and masterd is going away,remove option from RAPI client to chose a different socket.
Remove query option from GetClient
As all luxi clients talk to luxid now, and masterd willgo away, remove the option to use socket different fromluxid's.
Remove explicit reference to the query socket
Now that luxid's socket is the default socket anyway, do notpass the "query=True" parameter to GetClient. This will allowto get rid of this keyword argument, as masterd will go away.
Make watcher use luxid socket only
With luxid being feature-complete with respect to masterd,make the watcher use its socket exclusively. This is alsonecessary, as masterd will go away soon.
Fix 'JobIdListOnly' type from 'List' to 'Map'
Fix instance create and import parameters
Move OS parameter related constants to 'ganeti.cli' so they are usedboth by instance create and instance import from the CLI.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>...
Fix compatibility issues
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix 'UserDown' QA
Increase instance shutdown timeout and make call 'pkill' morereliable, which should fix the KVM QA.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Note UUID identification change in NEWS file
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Allow classic queries to use either names or UUIDs
When UUIDs are used in CLI commands, such addressing of objects failsor succeeds inconsistently across object types. Worse yet, some callsdo not fail, but simply return no result. This is due to the way the...
luxid: give stern warnings about debug mode
Luxid as it is can leak private and secret parameters by loggingall requests as they arrive, before any preprocessing is done.
Warn the user stern warnings about this.
Signed-off-by: Santi Raffa <rsanti@google.com>...
SimpleFillOS: add unit tests for OS parameter merging
Adds tests to ensure OS parameters are dealt with consistently.
NEWS: update with public and private parameters
Also warn about debug mode.