Let RPC clients handle their socket address
.. instead of AbstractClient itself. Also let every client call_InitTransport() as needed. This allows to determine socket addresseslater than during the initialization of a class.
Signed-off-by: Petr Pudlak <pudlak@google.com>...
Add the WConfD daemon itself
The daemon exposes the declared functions in Ganeti.WConfd.Core to RPCclients (currently just 'echo').
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add the WConfD daemon to build configuration files
Also list it in the Haskell datatype, constants, Python constants andtest configuration.
Add a module for WConfd core functions
There are no exported functions yet, except for `echo`, which justreturns its argument back to a client.
Add a MonadLog instance for strict variant of RWS
This allows to log in monad stacks containing RWS.
Add a monad for running all WConfD functions
This monad encapsulates working with the daemon and client state, aswell as failures and IO operations.
Add a generic RPC module
The main function in this module takes a list of names of functions anduses TH to create an RPC server, represented as a value of type Handler(from Ganeti.UDSServer). This can be readily used to create a daemonthat dispatches RPC requests to the given functions....
Add tests for Ganeti.THH.Types
In particular, test (de)serialization of "OneTuple".
Add a helper test method for checking deserialization
It tests if a deserialization method fails properly when an invalidinput is given.
Add common functions for working with TH's Type
In particular, functions for extracting types of arguments from afunction type, and for uncurrying functions with an arbitrary number ofarguments.
Another workaround for GHC linking error "unknown symbol"
This error appears after adding the dependency to lifted-base.See also #683 and ee7caf27.
I surmise that something like this is happening: Some code, that usesthe library, is only referenced through Template Haskell. A probable bug...
Add instances of MonadBase and MonadControl for ResultT
This allows to use lifted operations like 'fork' or 'bracket' insideResultT.
Allow clients of UDSServer to use different monads
.. as long as they're instances of "MonadBaseControl IO" and "MonadLog".This allows the UDSServer to call functions like "fork" within monadssuch as "ResultT e IO" or "ReaderT IO".
Add lifted-base to the list of requirements
This includes NEWS and INSTALL
Add dependencies for lifted-base to build_chroot
.. for both Debian distribution. Also check that the library is presentduring the configuration step.
Rename some functions not to collide with opcode names
Rename some functions related to instance communication not to collidewith the naming convention used in the opcodes.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Refactor instance communication network add and connect
Factor out the opcodes 'OpNetworkAdd' and 'OpNetworkConnect' used in'LUClusterSetParams' and 'LUGroupAdd' in order to reduce codeduplication and keep the configuration of the instance communication...
Instance communication mechanism QA (for cluster only)
Extend QA with tests for the instance communication mechanism. Thesetests cover only the cluster creating the instance communicationnetwork and connecting it to the existing node group, and also if new...
Connect new groups to the instance communication network
When a new group is added, if the instance communication network isenabled, then this group must also be connected to this network.
'gnt-cluster modify' with '--instance-communication-network'
Extend CLI 'gnt-cluster modify' with'--instance-communication-network'. Given that the return type for'OpClusterSetParams' changed to optionally return a list of jobs, itis also necessary to handle the result of this opcode accordingly....
'LUClusterSetParams' creates the instance communication net
Extend 'LUClusterSetParams' to create the user-supplied instancecommunication network in case this network does not exist. Note thatif the user-supplied network already exists, nothing needs to be done...
Check prereq instance communication network in 'SetParams'
Later, the logical unit for 'OpClusterSetParams' will be responsiblefor creating the instance communication network in case it does notexist. For now, it is important to check whether the network the user...
Add 'instance_communication_network' to 'OpClusterSetParams'
... and update the unit tests.
Add helper to handle CLIs that optionally spawn several jobs
This helper function detects whether an opcode returned a list of jobs(i.e., a result of the type ht.TJobIdListOnly) and in this case ituses 'ganeti.cli.JobExecutor' to wait for the jobs and determine the...
Instance comm network from config instead of predefined
Add 'instance_communication_parameter' to 'Cluster'
Fix indentation
Correct test for existance of instances
Since python interprets an empty dictionary as 'False',testing for the existance of the 'instances' field inthe configuration has to be done by explicitely testingfor 'None'.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Correct number of empty lines
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Remove osparams_private when downgrading the configuration
The instance parameter "osparams_private" was newly introducedin 2.12. So remove it when downgrading the configuration to 2.11.
Remove osparams_private_cluster on config downgrade
The new cluster parameter "osparams_private_cluster" wasnewly introduced in 2.12. So remove it when downgradingto 2.11.
Merge branch 'stable-2.11' into master
Make htools and luxid mandatory in the Makefile
Since Ganeti now uses luxid for core operations, it needs to be alwayspresent.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add a config. conditional for running Haskell coverage
Similarly to PY_UNIT, add HS_UNIT to run Haskell coverage only if wehave the required modules.
Remove the HTOOLS configuration variable
.. and update the code that uses it.
Fix start/stop scripts to run luxid unconditionally
and update the unit check the list of daemons.
Make luxid's Haskell dependencies mandatory
in the build configuration files.
Update the list dependencies in INSTALL
In particular, move luxid's dependencies to the section with mandatorydependencies.
Temporarily remove SSL changes from NEWS file
Since the current changes in the SSL handling havebeen partially disabled, remove their announcementin the NEWS file until there is something actuallyuser-facing happening.
Fix bracket style in rapi-workload
Pep8 complained about bracket indentation. This fixes it to satisfy Pep8as well as our style guide.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>(cherry-picked from ccded668c8a93fa25aaf21ddf5105270a04fe517)...
git-ignore automatically generated file tools/kvm-ifup-os
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Jose Lopes <jabolopes@google.com>
Conflicts: lib/client/gnt_node.py: trivial src/Ganeti/Query/Query.hs: import ALL the functions
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Gracefully handle queries for non-existing nodes
When adding a node, Ganeti checks whether the node is alreadypart of the cluster by querying for the node name. However,as queries are meant to return all nodes with the given name,it might well return the empty list when a new node is to be...
Consider job-IDs queried for twice only once
As reading jobs from disk is an expensive operation, when queryingfor jobs, we optimize by considering which values the job-id is askedfor in the filter. As any reasonable person would not add the sameclause twice in an Or-clause, the implicit assumption was that the...
Merge branch 'stable-2.10' into stable-2.11
Fix default for luxi clients in python
As masterd is going away, set default for all clients toluxid's socket.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'stable-2.9' into stable-2.10
Fix 'devel/check_copyright' remote branch detection
... but do not try to check the copyrights for local branches which donot have a proper remote branch, which would fail at a later stepanyway.
Revision bump for 2.9.4
Set release date for 2.9.4
Remove query option from RAPI client
As all RAPI requests now go to luxid, and masterd is going away,remove option from RAPI client to chose a different socket.
Remove query option from GetClient
As all luxi clients talk to luxid now, and masterd willgo away, remove the option to use socket different fromluxid's.
Remove explicit reference to the query socket
Now that luxid's socket is the default socket anyway, do notpass the "query=True" parameter to GetClient. This will allowto get rid of this keyword argument, as masterd will go away.
Make watcher use luxid socket only
With luxid being feature-complete with respect to masterd,make the watcher use its socket exclusively. This is alsonecessary, as masterd will go away soon.
Fix 'JobIdListOnly' type from 'List' to 'Map'
Fix instance create and import parameters
Move OS parameter related constants to 'ganeti.cli' so they are usedboth by instance create and instance import from the CLI.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>...
Fix compatibility issues
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
Fix 'UserDown' QA
Increase instance shutdown timeout and make call 'pkill' morereliable, which should fix the KVM QA.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Note UUID identification change in NEWS file
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Allow classic queries to use either names or UUIDs
When UUIDs are used in CLI commands, such addressing of objects failsor succeeds inconsistently across object types. Worse yet, some callsdo not fail, but simply return no result. This is due to the way the...
luxid: give stern warnings about debug mode
Luxid as it is can leak private and secret parameters by loggingall requests as they arrive, before any preprocessing is done.
Warn the user stern warnings about this.
Signed-off-by: Santi Raffa <rsanti@google.com>...
SimpleFillOS: add unit tests for OS parameter merging
Adds tests to ensure OS parameters are dealt with consistently.
NEWS: update with public and private parameters
Also warn about debug mode.
OpCodes: modify InstanceReinstall for private, secret params
Modify InstanceReinstall to accept and process private and secretparameters.
OpCodes: modify InstanceCreate for private, secret params
Modify InstanceCreate to accept process private and secret parameters.
OpCodes: modify ClusterSetParams for private parameters
Modify ClusterSetParams to accept and process private parameters.
CLI: add parameters for private and secret OS parameters
Define the CLI parameters for private and OS parameters.
OpCodes: modify InstanceSetParams for private parameters
Modify InstanceSetParams to accept and process private parameters.
Add private OS parameters to cluster and instance conf
This updates objects, constructors and mocks for Instance and Clusterobjects in Python and Haskell.
algo: add GetRepeatedKeys
We do not want public, private and secret parameters to haveoverlapping keys. This function implements this check.
Add Private types to Python, Haskell
This commit adds the private containers to Python and Haskell.
serializer: emit and encode Private values
For inbound data the simplest, safest thing to do is to traverse allJSON right after encoding and search for private parameters by key.
This ensures that all consumers of this data get Private valuestransparently and consistently; the serializing methods don't have to...
OpCodes test: fix argument order (expected/but got)
Fix expected/but got value in some unit tests.
RPCs: add docstrings for instance_os_add
This RPC's instance_osp input has a non-obvious, non documentedtype. This patch adds documentation to this RPC.
design: simplify how luxid determins job termination
With the job files, we already have designated files per job. Sothere is no need to additionally signal through a socket regularjob termination. This design also has the advantage that it isalready implemented and working in current luxid....
Switch to luxid for RAPI
...now that luxid knows of all RAPI requests.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Remove wildcard luxi operation matching in luxid
In that way, we explicitly name the operations that are nothandled by luxid and explain the reason. In particular, wecan be sure that newly added luxid operations won't be forgottenin luxid.
Implement QueryExports in luxid
...by handling as a classical query, using that queries forexport are already implemented. Note that QueryExport is slightlydifferent from other Query* Luxi requests, in that the fields arenot passed with the request, but have a fixed value....
Add instance communication CLI flags to the man page
Add 'kvm-ifup-os' to configure instance communication TAP
The script 'tools/kvm-ifup-os' configures TAP network interfaces forfor instances, routing, DHCP server, etc. Note that this script onlyconfigures TAP network interfaces that are used by the instance...
Add CLI and OpCode instance communication option
Add extra NIC to instance being created for communication
When an instance is being created, add an extra NIC (if necessary)which is meant to be used by the instance communication.
Generate TAP names ('gnt.com.%d') for instance comm NICs
Add TAP interface name as argument in '_OpenTap'
This is necessary in order to create TAP interfaces for thecommunication mechanism, which are named 'gnt.com.%d', where '%d' is aunique number within a given node.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Add metadata daemon
... including, user, group, daemonizing code with command lineoptions, integration with the Snap HTTP server, and logic.
Update the steps for implementing WConfD in the daemons doc
After Step 1 WConfD will handle the access to the configurationand will allow job-as-processes to access it.
Add job death detection to the daemons refactoring doc
Describe how to use file lock to implement it in a POSIX compliant way.
Let Haddock also generate Hoogle documentation
The Hoogle database is generated to doc/api/hs/ganeti.txt when Haddockis run. The database can be used by developers with a locally installedHoogle command line tool or for a web installation of Hoogle.
Remove NEWS entry for 2.9.4
Remove the NEWS entry for 2.9.4 temporarily until it actually getsreleased. Otherwise check-news would complain about the unreleasedversion below released ones.
The NEWS entry will be added back once 2.9.4 is released and the...
Document the change of noded's group in NEWS
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Change return type of internal rmJob
...to also provide the job itself. In this way, the function canalso be used for tasks that require temporarily removing a jobfrom the queue.
When enqueuing new jobs, respect job ID
When adding new jobs, don't add them at the end, but at aposition that fits with their job id. In this way, we canbuild operations that require fully dequeing a job an addingit later after some modifications.
Provide a function to change the priority of a queued job
There is a separation of responsibilities here. For jobs stillin the queue, it is the responsibility of the queue (scheduler),for started jobs, the job itself has to take care of it. To avoidthe job transitioning inbetween, it is temporarily dequeued during...
Implement ChangeJobPriority in luxid
For jobs still queued, we ask the queue to change the priority,and replicate the changed job. For jobs that have already beenstarted, we have to contact the job directly, which, at the moment,means forwarding the request to masterd....
Add a function changing the priority of an opcode
This pure function follows the semantic that an opcode, includingits priority, may only be changed if the opcode is not finalized.
Add a function to change the priority of a job
...by changing the priority of the non-finished opcodes.
Add functions for manipulating errors in Result(T)
There is often need to manipulate these errors, for example to convert a Stringfrom Result into an exception. These functions make this easier.
Function 'toErrorStr' lifts 'Result' to any 'MonadError'. This is useful...