ganeti-local
12 years agorpc: Call result processor once for each node result
Michael Hanselmann [Tue, 8 Nov 2011 15:39:32 +0000 (16:39 +0100)]
rpc: Call result processor once for each node result

… instead of calling it with the whole results dictionary. This
fixes an issue when replacing disks (and all other cases where
result processors are used).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeSetParams: Lock affected instances only
Michael Hanselmann [Wed, 2 Nov 2011 19:13:58 +0000 (20:13 +0100)]
LUNodeSetParams: Lock affected instances only

Until now LUNodeSetParams would lock all instances if a node's
secondary IP address was to be changed and would then release
all instances it didn't actually need. With this patch the LU
optimistically locks instances and, once it got the locks,
checks whether they're still correct. This is similar to how
node group locking is done.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoCheck BGL when adding/removing node
Michael Hanselmann [Thu, 3 Nov 2011 15:48:56 +0000 (16:48 +0100)]
Check BGL when adding/removing node

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRPC/test_delay: Use callable for timeout calculation
Michael Hanselmann [Thu, 3 Nov 2011 19:35:27 +0000 (20:35 +0100)]
RPC/test_delay: Use callable for timeout calculation

This avoids having to override the function in the RPC runner.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorpc: Move post-processor functions into definitions file
Michael Hanselmann [Thu, 3 Nov 2011 19:16:46 +0000 (20:16 +0100)]
rpc: Move post-processor functions into definitions file

This way the generated code no longer contains arbitrary code.
Post-processing functions are used by reference.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorpc: Use definitions directly instead of via generated code
Michael Hanselmann [Thu, 3 Nov 2011 19:05:42 +0000 (20:05 +0100)]
rpc: Use definitions directly instead of via generated code

Until now “autotools/build-rpc” would read the definition of all RPCs
and write them to a new file, “lib/_generated_rpc.py” with some
modifications. With this patch the generated code loads the definitions
and, at module load time, gets references to the actual definitions.
Further patches will even remove the calls to the post-processing
functions from the generated code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoConvert RPC definitions to dictionaries
Michael Hanselmann [Thu, 3 Nov 2011 18:46:43 +0000 (19:46 +0100)]
Convert RPC definitions to dictionaries

This is in preparation to reducing the amount of generated code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoquery: Use new SequenceToDict utility
Michael Hanselmann [Thu, 3 Nov 2011 18:12:32 +0000 (19:12 +0100)]
query: Use new SequenceToDict utility

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoImprove error handling in netmask change
Andrea Spadaccini [Fri, 4 Nov 2011 16:35:16 +0000 (16:35 +0000)]
Improve error handling in netmask change

- check if the master IP with the old netmask is up before attempting to
  change the netmask (to avoid a failed change netmask resulting in an
  undesired activation of the master IP);
- improve error messages of the backend function;
- in case of error, report the problem but otherwise change the cluster
  master_netmask parameter;
- remove duplicate error feedback.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd master_netmask to Cluster.UpgradeConfig
Andrea Spadaccini [Fri, 4 Nov 2011 15:27:14 +0000 (15:27 +0000)]
Add master_netmask to Cluster.UpgradeConfig

And also suppress pylint R0902 error about an object instance having
more than 20 attributes.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoFix cluster start failure due to missing netmask
Andrea Spadaccini [Fri, 4 Nov 2011 13:33:59 +0000 (13:33 +0000)]
Fix cluster start failure due to missing netmask

If the cluster version is upgraded from a version before commit
5a8648eb609f7e3a8d7ad7f82e93cfdd467a8fb5 to a version after that commit,
the master startup will fail because the ssconf file with the master
netmask will not be present, and it is read during the master voting
process.

This patch fixes this bug, making ssconf.GetMasterNetmask return a
default netmask value, and also corrects the docstring of this method.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoganeti(7): Adding documentation about env variables
René Nussbaumer [Fri, 4 Nov 2011 09:22:13 +0000 (10:22 +0100)]
ganeti(7): Adding documentation about env variables

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoutils.algo: Add utility to convert sequence to dictionary
Michael Hanselmann [Thu, 3 Nov 2011 18:03:48 +0000 (19:03 +0100)]
utils.algo: Add utility to convert sequence to dictionary

Useful for converting list of query fields to a dictionary and to
convert RPC definitions. Includes duplicate detection.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd QA test for “gnt-debug delay”
Michael Hanselmann [Thu, 3 Nov 2011 19:22:50 +0000 (20:22 +0100)]
Add QA test for “gnt-debug delay”

This would have caught a bug introduced with the recent RPC changes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Make “test_delay” RPC actually work
Michael Hanselmann [Thu, 3 Nov 2011 19:18:25 +0000 (20:18 +0100)]
rpc: Make “test_delay” RPC actually work

Until now it would just call itself, eventually failing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoQA: Add test for setting node's secondary IP address
Michael Hanselmann [Wed, 2 Nov 2011 19:17:29 +0000 (20:17 +0100)]
QA: Add test for setting node's secondary IP address

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agocmdlib: Allow specifying lock level when calculating node locks
Michael Hanselmann [Wed, 2 Nov 2011 15:54:51 +0000 (16:54 +0100)]
cmdlib: Allow specifying lock level when calculating node locks

This is needed to lock node resources.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoShow RPC calls from config in lock monitor
Michael Hanselmann [Wed, 26 Oct 2011 11:49:47 +0000 (13:49 +0200)]
Show RPC calls from config in lock monitor

With this patch all RPC calls at runtime of masterd will show up in the
lock monitor. There is a chicken-and-egg issue with initializing the
configuration with a context since the lock manager, containing the
monitor, requires the configuration. This is worked around by setting
the config's context only once the lock monitor is available.

Example:
rpc/node19.example.com/write_ssconf_files  Jq9/Job32/N_SET_PARAMjobqueue_update  Jq2/Job27/C_VERIFY_CONFIG

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoDerive IP hooks env variables from RPC parameter
Andrea Spadaccini [Wed, 2 Nov 2011 18:18:04 +0000 (18:18 +0000)]
Derive IP hooks env variables from RPC parameter

Let the environment variables of the master IP turnup/turndown be
derived from the parameter of the RPC itself (that is of type
objects.MasterNetworkParameters in both cases).

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoPass MasterNetworkParameters instances in RPCs
Andrea Spadaccini [Wed, 2 Nov 2011 17:51:58 +0000 (17:51 +0000)]
Pass MasterNetworkParameters instances in RPCs

Pass instances of objects.MasterNetworkParameters when calling RPCs for
activation and deactivation of master IP.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoUse MasterNetworkParameters attributes for RPC
Andrea Spadaccini [Wed, 2 Nov 2011 16:43:31 +0000 (16:43 +0000)]
Use MasterNetworkParameters attributes for RPC

Instead of manually unpacking the return values of
cfg.GetMasterNetworkParameters, let it return an instance of
objects.MasterNetworkParameters and pass its attributes.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoAdd object representing the master network params
Andrea Spadaccini [Wed, 2 Nov 2011 15:55:32 +0000 (15:55 +0000)]
Add object representing the master network params

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoUniform master IP activation and deactivation
Andrea Spadaccini [Wed, 2 Nov 2011 15:33:21 +0000 (15:33 +0000)]
Uniform master IP activation and deactivation

Add the master IP family parameter to the master IP deactivation RPCs,
so that the activation and deactivation interfaces are uniform.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoMake it possible to pass in flags using ENV variables
René Nussbaumer [Wed, 2 Nov 2011 15:07:15 +0000 (16:07 +0100)]
Make it possible to pass in flags using ENV variables

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoutils.algo: Add a function to insert a list into a list
René Nussbaumer [Wed, 2 Nov 2011 15:06:10 +0000 (16:06 +0100)]
utils.algo: Add a function to insert a list into a list

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix pylint deprecated directive in check-header
Andrea Spadaccini [Wed, 2 Nov 2011 13:34:17 +0000 (13:34 +0000)]
Fix pylint deprecated directive in check-header

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix overriding of ActivateMasterIp params
Andrea Spadaccini [Wed, 2 Nov 2011 13:31:52 +0000 (13:31 +0000)]
Fix overriding of ActivateMasterIp params

In the process of reworking the patches for master, I forgot to remove a
call to GetMasterInfo() that overrides the parameters passed via RPC.
This patch fixes this issue.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoAdd netmask and IP version to master IP hooks env
Andrea Spadaccini [Wed, 2 Nov 2011 11:47:28 +0000 (11:47 +0000)]
Add netmask and IP version to master IP hooks env

Add the master netmask and master IP version parameters to the hooks and
document them. In this way, the hook environment building function can
be reused for the planned master IP turnup changes.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoExplicitly pass params to change_master_netmask
Andrea Spadaccini [Thu, 27 Oct 2011 18:34:57 +0000 (19:34 +0100)]
Explicitly pass params to change_master_netmask

Make the master explicitly pass the parameters to the
change_master_netmask RPC, and change all the call flow to use the new
interface.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoExplicitly pass params to deactivate_master_ip
Andrea Spadaccini [Thu, 27 Oct 2011 18:18:00 +0000 (19:18 +0100)]
Explicitly pass params to deactivate_master_ip

Make the master explicitly pass the parameters to the
deactivate_master_ip RPC, and change all the call flow to use the new
interface.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoChange the semantics of stop_master RPC
Andrea Spadaccini [Fri, 28 Oct 2011 10:29:51 +0000 (11:29 +0100)]
Change the semantics of stop_master RPC

To avoid polluting the stop_master RPC interface with network
parameters, this patch removes the deactivation of the master IP from
this RPC.

The call to deactivate_master_ip is then added before every stop_master
call.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoExplicitly pass params to activate_master_ip
Andrea Spadaccini [Thu, 27 Oct 2011 16:08:34 +0000 (17:08 +0100)]
Explicitly pass params to activate_master_ip

To remove the usage of ssconf in the backend, the master needs to push
the parameters of activate_master_ip to the backend.

This patch changes the entire call path of activate_master_ip to use the
new interface.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agolocking: Make level names consistent, add check
Michael Hanselmann [Wed, 2 Nov 2011 12:19:01 +0000 (13:19 +0100)]
locking: Make level names consistent, add check

- Change all lock level names to their singular form
- Add assertion checking consistency between level names and lockset
  name

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd new lock level for node resource modifications
Michael Hanselmann [Wed, 2 Nov 2011 12:15:38 +0000 (13:15 +0100)]
Add new lock level for node resource modifications

This is in preparation for implementing a new resource model.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd tool to check Python file headers
Michael Hanselmann [Tue, 1 Nov 2011 15:42:36 +0000 (16:42 +0100)]
Add tool to check Python file headers

Ensures licence and copyright headers are correct.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoAdd documentation for the master IP hooks
Andrea Spadaccini [Tue, 11 Oct 2011 11:20:44 +0000 (12:20 +0100)]
Add documentation for the master IP hooks

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd master IP turnup and turndown hooks
Andrea Spadaccini [Tue, 25 Oct 2011 15:17:37 +0000 (16:17 +0100)]
Add master IP turnup and turndown hooks

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd RunLocalHooks decorator
Andrea Spadaccini [Tue, 25 Oct 2011 14:58:41 +0000 (15:58 +0100)]
Add RunLocalHooks decorator

Add the RunLocalHooks decorator, that allows the execution of hooks
locally. Also, add a RunLocalHooks method to HooksRunner, to adapt the
signature of HooksRunner.RunHooks to the one expected by HooksMaster,
and also to check that the hooks are being executed locally.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoGeneralize HooksMaster
Andrea Spadaccini [Tue, 25 Oct 2011 12:58:50 +0000 (13:58 +0100)]
Generalize HooksMaster

- remove any dependence on Logical Units from the HooksMaster;
- add a new function parameter to the constructor, a function that is
  expected to convert the results of the hooks execution in a format
  understood by the HooksMaster;
- add a factory method that builds a HooksMaster from a LU, keeping the
  interface of the old constructor of HooksMaster;
- remove usage of Processor.hmclass from external classes, introducing
  the Processor.BuildHooksMaster method;
- update unit tests.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoRAPI: Fix issue when replacing disks
Michael Hanselmann [Tue, 1 Nov 2011 15:02:16 +0000 (16:02 +0100)]
RAPI: Fix issue when replacing disks

This is another fallout from a suboptimal merge.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agorpc: Improve argument definitions
Michael Hanselmann [Mon, 31 Oct 2011 19:09:22 +0000 (20:09 +0100)]
rpc: Improve argument definitions

Instead of in-place code now each argument has an (optional) kind
assigned. The RPC client will then take care of encoding each argument
according to its definition. A new base (_RpcClientBase) is added to
take care of encoding and dispatching calls.

Future patches will also use these definitions in the node daemon.

A nice side-effect of this change is the ability to use stand-alone
functions, as opposed to class member methods, for most of the encoding.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorpc: Change {import,export}_start to take source/dest in single argument
Michael Hanselmann [Mon, 31 Oct 2011 17:47:15 +0000 (18:47 +0100)]
rpc: Change {import,export}_start to take source/dest in single argument

This simplifies the RPC argument encoding code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRename lib/build/rpc_definitions.py to lib/rpc_defs.py
Michael Hanselmann [Mon, 31 Oct 2011 17:24:04 +0000 (18:24 +0100)]
Rename lib/build/rpc_definitions.py to lib/rpc_defs.py

It will also be used at runtime, not just for building.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoClean up incorrect merge 7dcf333d
Michael Hanselmann [Mon, 31 Oct 2011 16:33:28 +0000 (17:33 +0100)]
Clean up incorrect merge 7dcf333d

The unittests weren't corrected for the master branch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoMerge branch 'devel-2.5'
Michael Hanselmann [Mon, 31 Oct 2011 15:16:42 +0000 (16:16 +0100)]
Merge branch 'devel-2.5'

* devel-2.5:
  Fix wrong headers and licences
  Update NEWS and increase to 2.4.5
  Fix parameters of RpcResult in hooks unit tests
  Fix a too long line.
  Move RenameFile to the new functions
  ensure_dirs: Move some useful functions into utils.
  Use JoinDisjointDicts in mcpu
  Add the JoinDisjointDicts function to utils.algo
  Fix queue archive creation with wrong permissions
  Ensure permission on the job queue version file
  OpGroupVerifyDisks: Fix wrong result type declaration
  RAPI: Make node evacuation actually work
  Bump version to 2.5.0~rc2
  Update NEWS for unreleased 2.4.5
  RAPI: Fix resource for replacing disks

Conflicts:
lib/rapi/rlib2.py: Merged bugfix from commit 539d65ba0

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoFix wrong headers and licences
Michael Hanselmann [Thu, 20 Oct 2011 15:37:17 +0000 (17:37 +0200)]
Fix wrong headers and licences

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Stephen Shirley <diamond@google.com>

12 years agoAdd missing get_migration_status RPC definition
Andrea Spadaccini [Fri, 28 Oct 2011 13:19:49 +0000 (14:19 +0100)]
Add missing get_migration_status RPC definition

Add the instance_get_migration_status RPC definition and the needed
helper.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agocmdlib: Fix issue when marking node as online
Michael Hanselmann [Thu, 27 Oct 2011 18:53:42 +0000 (20:53 +0200)]
cmdlib: Fix issue when marking node as online

When a node is marked as online (“gnt-node modify -O no …”), an RPC is
made to the node to check whether the node daemon is running. My recent
RPC changes led to offline nodes being ignored before the actual call is
made, thereby causing this check to fail. This is a workaround and needs
to be cleaned up later.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse GetClassFromIpFamily in ActivateMasterIp
Andrea Spadaccini [Mon, 24 Oct 2011 10:50:27 +0000 (11:50 +0100)]
Use GetClassFromIpFamily in ActivateMasterIp

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd shortcut for network-related cluster params
Andrea Spadaccini [Mon, 24 Oct 2011 10:26:26 +0000 (11:26 +0100)]
Add shortcut for network-related cluster params

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agorpc: Remove leftovers from timeout table
Michael Hanselmann [Wed, 26 Oct 2011 10:36:42 +0000 (12:36 +0200)]
rpc: Remove leftovers from timeout table

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoconfig: Add unlocked function to get all node objects
Michael Hanselmann [Wed, 26 Oct 2011 10:37:11 +0000 (12:37 +0200)]
config: Add unlocked function to get all node objects

This will be used for resolving node names.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoMerge branch 'devel-2.4' into devel-2.5
René Nussbaumer [Thu, 27 Oct 2011 13:28:02 +0000 (15:28 +0200)]
Merge branch 'devel-2.4' into devel-2.5

Conflicts:
configure.ac (trivial)

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoUpdate NEWS and increase to 2.4.5 v2.4.5
René Nussbaumer [Thu, 27 Oct 2011 12:57:10 +0000 (14:57 +0200)]
Update NEWS and increase to 2.4.5

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix parameters of RpcResult in hooks unit tests
Andrea Spadaccini [Wed, 26 Oct 2011 13:23:59 +0000 (14:23 +0100)]
Fix parameters of RpcResult in hooks unit tests

In FakeHooksRpcSuccess, the data parameter of the RpcResult constructor
was not enclosed in a tuple. While this does not make the test fail, it
must be fixed.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoFix RPC unittest
Michael Hanselmann [Wed, 26 Oct 2011 10:46:49 +0000 (12:46 +0200)]
Fix RPC unittest

Add missing “read_timeout” value.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoGeneralize the generation of ADT from raw types
Agata Murawska [Tue, 25 Oct 2011 13:11:29 +0000 (15:11 +0200)]
Generalize the generation of ADT from raw types

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLuxi support for Query status in htools
Agata Murawska [Mon, 24 Oct 2011 14:06:08 +0000 (16:06 +0200)]
Luxi support for Query status in htools

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFix a too long line.
René Nussbaumer [Wed, 26 Oct 2011 09:01:34 +0000 (11:01 +0200)]
Fix a too long line.

That's what you get for not running make lint :(

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Remove _Static* and other unused functions
Michael Hanselmann [Tue, 25 Oct 2011 19:07:09 +0000 (21:07 +0200)]
rpc: Remove _Static* and other unused functions

These are no longer needed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert last two calls to generated code
Michael Hanselmann [Tue, 25 Oct 2011 19:00:16 +0000 (21:00 +0200)]
rpc: Convert last two calls to generated code

These two calls, “upload_file” and “write_ssconf_files” are treated
separated as they're used by the configuration, where we can't use the
normal resolver.

There's still some duplicated code in rpc.py, but that will be taken
care of in future patches.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert call for HV parameter validation
Michael Hanselmann [Tue, 25 Oct 2011 17:17:10 +0000 (19:17 +0200)]
rpc: Convert call for HV parameter validation

Instead of filling the parameters in the RPC layer, that is now done
before the wrapper is called, thereby simplifying the wrapper.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert two more instance-specific calls
Michael Hanselmann [Tue, 25 Oct 2011 17:00:10 +0000 (19:00 +0200)]
rpc: Convert two more instance-specific calls

Interface changes were necessary as these took more parameters than were
actually passed over the wire. Those parameters were just passed to the
instance serialization function.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoMove bootstrap-related RPC to generated wrappers
Michael Hanselmann [Tue, 25 Oct 2011 16:29:06 +0000 (18:29 +0200)]
Move bootstrap-related RPC to generated wrappers

With this patch, only 6 RPC are left as old-style code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoConvert job queue's RPC to generated code
Michael Hanselmann [Tue, 25 Oct 2011 15:30:33 +0000 (17:30 +0200)]
Convert job queue's RPC to generated code

With these changes job queue RPC will finally show up on the lock
monitor. See below for an example. A job queue-specific class is used to
restrict the use of a static list for name resolution to the job queue.
Further improvements can be made to not re-create the whole RPC client
for every call (e.g. by using a more dynamic resolver), but for now this
works.

rpc/node8.example.com/jobqueue_update Jq8/Job9/TEST_DELAY

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert wrappers for starting import/export daemons
Michael Hanselmann [Mon, 24 Oct 2011 16:42:31 +0000 (18:42 +0200)]
rpc: Convert wrappers for starting import/export daemons

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert storage-related calls
Michael Hanselmann [Mon, 24 Oct 2011 16:02:33 +0000 (18:02 +0200)]
rpc: Convert storage-related calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert blockdev-related calls
Michael Hanselmann [Mon, 24 Oct 2011 15:57:41 +0000 (17:57 +0200)]
rpc: Convert blockdev-related calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert instance-related calls
Michael Hanselmann [Mon, 24 Oct 2011 15:56:26 +0000 (17:56 +0200)]
rpc: Convert instance-related calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert misc calls
Michael Hanselmann [Mon, 24 Oct 2011 15:54:04 +0000 (17:54 +0200)]
rpc: Convert misc calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert file-storage-related calls
Michael Hanselmann [Mon, 24 Oct 2011 15:45:06 +0000 (17:45 +0200)]
rpc: Convert file-storage-related calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert node-related calls
Michael Hanselmann [Mon, 24 Oct 2011 15:43:53 +0000 (17:43 +0200)]
rpc: Convert node-related calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert OS-related calls
Michael Hanselmann [Mon, 24 Oct 2011 15:43:27 +0000 (17:43 +0200)]
rpc: Convert OS-related calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert X509 calls
Michael Hanselmann [Mon, 24 Oct 2011 15:39:57 +0000 (17:39 +0200)]
rpc: Convert X509 calls

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Convert import/export functions
Michael Hanselmann [Mon, 24 Oct 2011 15:39:26 +0000 (17:39 +0200)]
rpc: Convert import/export functions

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Add helpers
Michael Hanselmann [Mon, 24 Oct 2011 16:04:12 +0000 (18:04 +0200)]
rpc: Add helpers

These will be used by the generated RPC wrappers.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Add support for generated RPC wrappers
Michael Hanselmann [Mon, 24 Oct 2011 15:37:02 +0000 (17:37 +0200)]
rpc: Add support for generated RPC wrappers

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd RPC definitions
Michael Hanselmann [Mon, 24 Oct 2011 15:34:42 +0000 (17:34 +0200)]
Add RPC definitions

These will be used for replacing the tons and tons of boilerplate code
in rpc.py.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc_definitions: Add helpers
Michael Hanselmann [Mon, 24 Oct 2011 14:32:59 +0000 (16:32 +0200)]
rpc_definitions: Add helpers

These helpers will be used to convert incoming parameters to
JSON-compatible types.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Disable timeout check
Michael Hanselmann [Mon, 24 Oct 2011 14:31:59 +0000 (16:31 +0200)]
rpc: Disable timeout check

This is in preparation for the definition-based RPC model, where
the timeout will be stored within the definition.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd script to generate RPC wrappers
Michael Hanselmann [Fri, 21 Oct 2011 13:28:05 +0000 (15:28 +0200)]
Add script to generate RPC wrappers

A new script, “autotools/build-rpc”, will be used to generate code for
RPC client wrappers. This is done because “lib/rpc.py” contains lots and
lots of boilerplate code. Forthcoming patches will start converting
RPC wrappers.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoMove RenameFile to the new functions
René Nussbaumer [Fri, 21 Oct 2011 13:33:32 +0000 (15:33 +0200)]
Move RenameFile to the new functions

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoensure_dirs: Move some useful functions into utils.
René Nussbaumer [Fri, 21 Oct 2011 11:26:44 +0000 (13:26 +0200)]
ensure_dirs: Move some useful functions into utils.

With this change we can easily reuse this functionality where it makes
sense on other parts of Ganeti.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoSmall resource model design update re console
Michael Hanselmann [Wed, 26 Oct 2011 07:09:27 +0000 (09:09 +0200)]
Small resource model design update re console

Starting with Ganeti 2.5 the console uses a query and no longer acquires
locks. See commit d6f46b6a94.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib: Use constant for DRBD meta device size
Michael Hanselmann [Wed, 26 Oct 2011 06:46:57 +0000 (08:46 +0200)]
cmdlib: Use constant for DRBD meta device size

… instead of a hardcoded value.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse JoinDisjointDicts in mcpu
Andrea Spadaccini [Mon, 24 Oct 2011 14:02:48 +0000 (15:02 +0100)]
Use JoinDisjointDicts in mcpu

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd the JoinDisjointDicts function to utils.algo
Andrea Spadaccini [Mon, 24 Oct 2011 14:01:20 +0000 (15:01 +0100)]
Add the JoinDisjointDicts function to utils.algo

Add a function that joins two dictionaries, enforcing the constraint
that the two key sets should be disjoint. Also, add unit tests for this
function.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix parameters to RPC "blockdev_rename"
Michael Hanselmann [Mon, 24 Oct 2011 14:38:00 +0000 (16:38 +0200)]
Fix parameters to RPC "blockdev_rename"

Before this patch it would take a variable number of arguments, which
isn't very good for a definition-based approach.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFix parameters to RPC "os_validate"
Michael Hanselmann [Mon, 24 Oct 2011 14:19:34 +0000 (16:19 +0200)]
Fix parameters to RPC "os_validate"

All other RPC wrappers take the node name(s) as the first parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUn-revert comments in utils.mlock
Michael Hanselmann [Fri, 21 Oct 2011 11:22:00 +0000 (13:22 +0200)]
Un-revert comments in utils.mlock

These updates and clarifications are still useful.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRemove “node_tcp_ping” RPC
Michael Hanselmann [Fri, 21 Oct 2011 12:23:48 +0000 (14:23 +0200)]
Remove “node_tcp_ping” RPC

It is not used anywhere.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoutils.ShellWriter: Don't indent empty lines
Michael Hanselmann [Fri, 21 Oct 2011 11:32:17 +0000 (13:32 +0200)]
utils.ShellWriter: Don't indent empty lines

Empty lines shouldn't get indented. Unittest included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoMerge branch 'stable-2.5' into devel-2.5
René Nussbaumer [Fri, 21 Oct 2011 13:00:57 +0000 (15:00 +0200)]
Merge branch 'stable-2.5' into devel-2.5

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoMerge branch 'devel-2.4' into stable-2.5
René Nussbaumer [Fri, 21 Oct 2011 12:59:50 +0000 (14:59 +0200)]
Merge branch 'devel-2.4' into stable-2.5

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohttp.client: Remove use of PycURL's “reset” function
Michael Hanselmann [Thu, 20 Oct 2011 13:59:05 +0000 (15:59 +0200)]
http.client: Remove use of PycURL's “reset” function

We don't re-use cURL objects anymore, so there's no need to reset them.
PycURL 7.19.0 has a reference counting bug leading to a crash after
a certain number of performed requests.

Since a unittest depended on this function for a test, it is replaced
with a per-request completion callback, which could also be used by
users of the HTTP client.

Unittests are updated and a test for the request object's “nice name”,
used by the lock monitor, is included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoAdd unittest for PycURL 7.19.0's reset issue
Michael Hanselmann [Thu, 20 Oct 2011 12:56:00 +0000 (14:56 +0200)]
Add unittest for PycURL 7.19.0's reset issue

While we won't use this function in Ganeti (yet), users should be aware
of the potential crashes caused by this functioni (discussed offline).
If the installed version of PycURL is not affected, this test is a
no-op.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoovfconverter: use qemu-img path from constants
Agata Murawska [Thu, 20 Oct 2011 15:20:11 +0000 (17:20 +0200)]
ovfconverter: use qemu-img path from constants

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoMerge branch 'devel-2.5'
Guido Trotter [Thu, 20 Oct 2011 13:30:09 +0000 (14:30 +0100)]
Merge branch 'devel-2.5'

* devel-2.5:
  Allow per-hypervisor optional files
  Add hypervisors ancillary files list
  xen: abstract a few hardcoded strings as constants

Conflicts:
lib/cmdlib.py
  - trivial

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix queue archive creation with wrong permissions
René Nussbaumer [Wed, 19 Oct 2011 14:51:27 +0000 (16:51 +0200)]
Fix queue archive creation with wrong permissions

On a master failover some of the archive dirs might have wrong
permissions in the non-root model. This is due to the nature of noded
still running as root and the job queue is synced that way. This patch
will fix this behaviour by setting the permissions accordingly.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix for buildbot pylint failure
Agata Murawska [Thu, 20 Oct 2011 11:58:52 +0000 (13:58 +0200)]
Fix for buildbot pylint failure

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>