ganeti-local
12 years agoWarn if we enable maintain-node-health without confd
Iustin Pop [Thu, 17 Nov 2011 11:31:10 +0000 (12:31 +0100)]
Warn if we enable maintain-node-health without confd

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdapt daemon-util to ENABLE_CONFD
Iustin Pop [Thu, 17 Nov 2011 11:19:22 +0000 (12:19 +0100)]
Adapt daemon-util to ENABLE_CONFD

We still allow explicit shutdown of confd, but we prevent manual
or automatic start-up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdapt watcher for ENABLE_CONFD
Iustin Pop [Thu, 17 Nov 2011 11:04:58 +0000 (12:04 +0100)]
Adapt watcher for ENABLE_CONFD

If confd is disabled, do not automatically restart it. Furthermore, we
can't run maintenance actions if it is disabled so log a warning.

Note that I haven't completely disabled the NodeMaintenance class with
ENABLE_CONFD = False because I think they are at two different levels
(e.g. we might have other maintenance actions done even with confd
disabled).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoPrevent runnning of confd tests in burnin
Iustin Pop [Thu, 17 Nov 2011 10:55:03 +0000 (11:55 +0100)]
Prevent runnning of confd tests in burnin

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd toggle for enabling/disabling confd
Iustin Pop [Thu, 17 Nov 2011 10:49:56 +0000 (11:49 +0100)]
Add toggle for enabling/disabling confd

Doesn't do anything yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix unittest bug related to offline instances
Iustin Pop [Thu, 17 Nov 2011 10:19:41 +0000 (11:19 +0100)]
Fix unittest bug related to offline instances

Currently, the code in Node.hs is overly strict: once a node's free
memory reaches 0, it will refuse to add any instances (offline or
not). I think this is a safe safeguard (I don't expect nodes to run
without at least 1MB of free memory), so rather than change this
behaviour we need to restrict the Node generation in the unittest to
skip such nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

12 years agohtools: reindent the rest of the files
Iustin Pop [Wed, 16 Nov 2011 18:14:48 +0000 (19:14 +0100)]
htools: reindent the rest of the files

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: re-indent IAlloc.hs
Iustin Pop [Wed, 16 Nov 2011 17:53:47 +0000 (18:53 +0100)]
htools: re-indent IAlloc.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: reindent hspace
Iustin Pop [Wed, 16 Nov 2011 17:19:42 +0000 (18:19 +0100)]
htools: reindent hspace

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: reindent hbal
Iustin Pop [Wed, 16 Nov 2011 17:18:16 +0000 (18:18 +0100)]
htools: reindent hbal

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: reindent CLI.hs
Iustin Pop [Wed, 16 Nov 2011 17:15:57 +0000 (18:15 +0100)]
htools: reindent CLI.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: re-indent QC.hs
Iustin Pop [Wed, 16 Nov 2011 17:11:41 +0000 (18:11 +0100)]
htools: re-indent QC.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: re-indent Node.hs
Iustin Pop [Wed, 16 Nov 2011 16:54:09 +0000 (17:54 +0100)]
htools: re-indent Node.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: finish re-indenting Cluster.hs
Iustin Pop [Wed, 16 Nov 2011 16:45:11 +0000 (17:45 +0100)]
htools: finish re-indenting Cluster.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agomasterd: Don't pass mainloop to server class
Michael Hanselmann [Wed, 16 Nov 2011 09:35:06 +0000 (10:35 +0100)]
masterd: Don't pass mainloop to server class

It is not used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoworkerpool: Allow processing of new tasks to be stopped
Michael Hanselmann [Wed, 16 Nov 2011 08:37:49 +0000 (09:37 +0100)]
workerpool: Allow processing of new tasks to be stopped

This is different from “Quiesce” in the sense that this function just
changes an internal flag and doesn't wait for the queue to be empty.
Tasks already being processed continue normally, but no new tasks will
be started. New tasks can still be added, but won't be processed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoworkerpool: Use loop to ignore spurious notifications
Michael Hanselmann [Wed, 16 Nov 2011 08:11:59 +0000 (09:11 +0100)]
workerpool: Use loop to ignore spurious notifications

This saves us from returning to the worker code when there is no
task to be processed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agojqueue: Factorize code checking for drained queue
Michael Hanselmann [Wed, 16 Nov 2011 08:06:23 +0000 (09:06 +0100)]
jqueue: Factorize code checking for drained queue

This is in preparation for a clean(er) shutdown of masterd.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib.TLReplaceDisks: Use itertools.count
Michael Hanselmann [Wed, 16 Nov 2011 07:08:34 +0000 (08:08 +0100)]
cmdlib.TLReplaceDisks: Use itertools.count

… instead of a variable which needs to be incremented for every step.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoStart converting the indentation to 2-chars
Iustin Pop [Mon, 14 Nov 2011 14:33:20 +0000 (15:33 +0100)]
Start converting the indentation to 2-chars

… also do some other small style cleanups.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agohtools: Nicer arbitrary instantiation
Agata Murawska [Tue, 15 Nov 2011 15:48:23 +0000 (16:48 +0100)]
htools: Nicer arbitrary instantiation

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: change behaviour for offline instances
Agata Murawska [Tue, 15 Nov 2011 10:28:40 +0000 (11:28 +0100)]
htools: change behaviour for offline instances

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: remove 'running' from instance options
Agata Murawska [Mon, 14 Nov 2011 13:54:32 +0000 (14:54 +0100)]
htools: remove 'running' from instance options

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: change runSt type to InstanceStatus
Agata Murawska [Thu, 10 Nov 2011 10:21:32 +0000 (11:21 +0100)]
htools: change runSt type to InstanceStatus

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: create InstanceStatus ADT
Agata Murawska [Thu, 10 Nov 2011 10:10:13 +0000 (11:10 +0100)]
htools: create InstanceStatus ADT

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoTransition into and out of offline instance state
Agata Murawska [Tue, 8 Nov 2011 10:05:43 +0000 (11:05 +0100)]
Transition into and out of offline instance state

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoIntroduce admin_state as 3-values data type
Agata Murawska [Tue, 8 Nov 2011 10:04:34 +0000 (11:04 +0100)]
Introduce admin_state as 3-values data type

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoRename admin_up to admin_state
Agata Murawska [Thu, 27 Oct 2011 11:29:41 +0000 (13:29 +0200)]
Rename admin_up to admin_state

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFixed typo in _VerifyResultRow
Agata Murawska [Fri, 28 Oct 2011 11:22:02 +0000 (13:22 +0200)]
Fixed typo in _VerifyResultRow

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoalgo: Make a dict from an flat list
René Nussbaumer [Tue, 15 Nov 2011 14:09:59 +0000 (15:09 +0100)]
algo: Make a dict from an flat list

This is in preparation to take deeper dict constructs from the command
line. You can feed the optionslist directly constructed of type
"identkeyval" to it and it returns a fully deflated dict.

This is mainly needed for the resource model changes where we have to
modify the disk_state which is a 3 level dict:

disk_type/name:disk_reserved=10g

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agolocking: Make some aliased methods public
Michael Hanselmann [Tue, 15 Nov 2011 13:46:21 +0000 (14:46 +0100)]
locking: Make some aliased methods public

Some methods, such as “_is_owned” and “list_owned”, have been aliased to
make them public for a while now. This patch makes the actual
implementation public.

SharedLock's “is_owned” needs to be aliased to “_is_owned” to remain
compatible with Python's built-in threaded.Condition class.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib._ReleaseLock: Do nothing if no locks are owned
Michael Hanselmann [Tue, 15 Nov 2011 11:23:43 +0000 (12:23 +0100)]
cmdlib._ReleaseLock: Do nothing if no locks are owned

The locking library doesn't like it when “release()” is called on
a lockset or lock which isn't held by the current thread. Instead
of modifying the library, which could have other side-effects,
this rather simple change avoids errors when a LU simply tries to
release all locks, even when it doesn't own any at a certain level.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse resource lock when setting node parameters
Michael Hanselmann [Tue, 15 Nov 2011 06:21:34 +0000 (07:21 +0100)]
Use resource lock when setting node parameters

Also acquire instance and resource locks in shared mode (see comment).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse node resource lock for replacing instance disks
Michael Hanselmann [Thu, 3 Nov 2011 16:12:19 +0000 (17:12 +0100)]
Use node resource lock for replacing instance disks

If early-release is not used, the resource lock is kept while waiting
for disks to sync.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoHold node resource locks while setting instance parameters
Michael Hanselmann [Thu, 3 Nov 2011 16:19:30 +0000 (17:19 +0100)]
Hold node resource locks while setting instance parameters

Important for when disks are converted. Release locks once they're not
needed anymore. Make liberal use of assertions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoHold node resource lock while moving instance
Michael Hanselmann [Thu, 3 Nov 2011 16:08:26 +0000 (17:08 +0100)]
Hold node resource lock while moving instance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAcquire node resource lock when removing instance
Michael Hanselmann [Thu, 3 Nov 2011 16:04:05 +0000 (17:04 +0100)]
Acquire node resource lock when removing instance

Removing an instance affects available disk space and memory.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse node resource lock when recreating instance disks
Michael Hanselmann [Thu, 3 Nov 2011 15:58:00 +0000 (16:58 +0100)]
Use node resource lock when recreating instance disks

Recreating disks conflicts with other disk operations, therefore the
node resource lock must be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUClusterRepairDiskSizes: Use node resource locks
Michael Hanselmann [Thu, 3 Nov 2011 15:40:06 +0000 (16:40 +0100)]
LUClusterRepairDiskSizes: Use node resource locks

Since this doesn't really touch the node, but it conflicts with e.g.
growing a disk, the resource lock must be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUInstanceGrowDisk: Use node resource lock
Michael Hanselmann [Wed, 2 Nov 2011 15:55:33 +0000 (16:55 +0100)]
LUInstanceGrowDisk: Use node resource lock

Also add one more feedback line. Downgrade instance lock to shared mode
while we're only waiting for disks to sync. The node lock is released
when not needed anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUInstanceCreate: Hold node resource lock
Michael Hanselmann [Wed, 2 Nov 2011 15:33:46 +0000 (16:33 +0100)]
LUInstanceCreate: Hold node resource lock

The node resource lock is released once the disks are in sync (that is,
after wiping).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeQueryvols: Acquire all locks in shared mode
Michael Hanselmann [Mon, 14 Nov 2011 13:12:08 +0000 (14:12 +0100)]
LUNodeQueryvols: Acquire all locks in shared mode

Nothing is being written to.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeQueryStorage: Acquire all locks in shared mode
Michael Hanselmann [Mon, 14 Nov 2011 13:11:48 +0000 (14:11 +0100)]
LUNodeQueryStorage: Acquire all locks in shared mode

Nothing is written to.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib: Share lock in LUInstanceConsole
Michael Hanselmann [Mon, 14 Nov 2011 09:36:50 +0000 (10:36 +0100)]
cmdlib: Share lock in LUInstanceConsole

No writes are being done.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeQuery: Call implementation's DeclareLocks function
Michael Hanselmann [Thu, 3 Nov 2011 16:22:34 +0000 (17:22 +0100)]
LUNodeQuery: Call implementation's DeclareLocks function

Just in case we ever add locks for querying nodes. Currently
_NodeQuery's DeclareLocks is a no-op function.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse master IP address setup script in backend
Andrea Spadaccini [Tue, 25 Oct 2011 09:54:10 +0000 (10:54 +0100)]
Use master IP address setup script in backend

Replace the code in backend.ActivateMasterIp and
backend.DeactivateMasterIp with the master IP address setup script,
either the default one or the one provided by the user.

- Convert to string the netmask parameter in _BuildMasterIpEnv
- Add the _RunMasterSetupScript function and let ActivateMasterIp and
  DeactivateMasterIp become wrappers for it;
- Change the failure model of DeactivateMasterIp, raising errors if the
  script fails instead of silently ignoring the failure

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoChange master IP address RPCs for external script
Andrea Spadaccini [Tue, 8 Nov 2011 17:09:56 +0000 (17:09 +0000)]
Change master IP address RPCs for external script

Change the master IP address RPC call chain to accept the
use_external_master_ip_script parameter. Introduces an unused parameter
in backend.ActivateMasterIp and backend.DeactivateMasterIp, that will be
used in the next commit.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUpdate cluster verify to check IP address scripts
Andrea Spadaccini [Tue, 8 Nov 2011 16:29:52 +0000 (16:29 +0000)]
Update cluster verify to check IP address scripts

Update cluster-verify to check the integrity of the default master IP
address setup script and the presence and executability of the external
one (if currently in use by the cluster).

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd --use-external-mip-script flag
Andrea Spadaccini [Tue, 8 Nov 2011 15:32:13 +0000 (15:32 +0000)]
Add --use-external-mip-script flag

- add a command line flag to gnt-cluster init and modify to change the
  value of the cluster parameter use_external_mip_script;
- add two constants representing the paths of the default script and of
  the external script;
- add documentation for the new option in the gnt-cluster man page;
- add use_external_mip_script to gnt-cluster info output.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd use_external_mip_script cluster parameter
Andrea Spadaccini [Tue, 8 Nov 2011 14:41:12 +0000 (14:41 +0000)]
Add use_external_mip_script cluster parameter

Add the use_external_mip_script cluster parameter, that represents
whether the master IP address turnup/turndown procedures must use a
script provided by the user (True) or the one provided by Ganeti
(False).

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd the default master-ip-setup script
Andrea Spadaccini [Mon, 24 Oct 2011 16:38:10 +0000 (17:38 +0100)]
Add the default master-ip-setup script

Add a shell script that will be the default replacement for the body of
backend.ActivateMasterIp and backend.DeactivateMasterIp.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agocheck-python-code: Detect old-style pylint disable-msg lines
Michael Hanselmann [Fri, 11 Nov 2011 12:47:24 +0000 (13:47 +0100)]
check-python-code: Detect old-style pylint disable-msg lines

Commit b459a848d was supposed to replace all, but one was missed. Add a
check to autotools/check-python-code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agoAdding basic elements for the new node params
René Nussbaumer [Thu, 3 Nov 2011 12:52:51 +0000 (13:52 +0100)]
Adding basic elements for the new node params

This patch adds the new fields to the objects.py as well as defines the
constants used in the dicts and their type.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Fix another result processor
Michael Hanselmann [Tue, 8 Nov 2011 17:17:37 +0000 (18:17 +0100)]
rpc: Fix another result processor

I forgot to change this in commit d9da5065c0.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Fix issue with “test_delay”'s timeout
Michael Hanselmann [Tue, 8 Nov 2011 15:40:18 +0000 (16:40 +0100)]
rpc: Fix issue with “test_delay”'s timeout

I passed the timeout calculation function in the wrong field of the
definition. A small change is also needed in “build-rpc” to not abort
when writing the docstring.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Call result processor once for each node result
Michael Hanselmann [Tue, 8 Nov 2011 15:39:32 +0000 (16:39 +0100)]
rpc: Call result processor once for each node result

… instead of calling it with the whole results dictionary. This
fixes an issue when replacing disks (and all other cases where
result processors are used).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeSetParams: Lock affected instances only
Michael Hanselmann [Wed, 2 Nov 2011 19:13:58 +0000 (20:13 +0100)]
LUNodeSetParams: Lock affected instances only

Until now LUNodeSetParams would lock all instances if a node's
secondary IP address was to be changed and would then release
all instances it didn't actually need. With this patch the LU
optimistically locks instances and, once it got the locks,
checks whether they're still correct. This is similar to how
node group locking is done.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoCheck BGL when adding/removing node
Michael Hanselmann [Thu, 3 Nov 2011 15:48:56 +0000 (16:48 +0100)]
Check BGL when adding/removing node

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRPC/test_delay: Use callable for timeout calculation
Michael Hanselmann [Thu, 3 Nov 2011 19:35:27 +0000 (20:35 +0100)]
RPC/test_delay: Use callable for timeout calculation

This avoids having to override the function in the RPC runner.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorpc: Move post-processor functions into definitions file
Michael Hanselmann [Thu, 3 Nov 2011 19:16:46 +0000 (20:16 +0100)]
rpc: Move post-processor functions into definitions file

This way the generated code no longer contains arbitrary code.
Post-processing functions are used by reference.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorpc: Use definitions directly instead of via generated code
Michael Hanselmann [Thu, 3 Nov 2011 19:05:42 +0000 (20:05 +0100)]
rpc: Use definitions directly instead of via generated code

Until now “autotools/build-rpc” would read the definition of all RPCs
and write them to a new file, “lib/_generated_rpc.py” with some
modifications. With this patch the generated code loads the definitions
and, at module load time, gets references to the actual definitions.
Further patches will even remove the calls to the post-processing
functions from the generated code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoConvert RPC definitions to dictionaries
Michael Hanselmann [Thu, 3 Nov 2011 18:46:43 +0000 (19:46 +0100)]
Convert RPC definitions to dictionaries

This is in preparation to reducing the amount of generated code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoquery: Use new SequenceToDict utility
Michael Hanselmann [Thu, 3 Nov 2011 18:12:32 +0000 (19:12 +0100)]
query: Use new SequenceToDict utility

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoImprove error handling in netmask change
Andrea Spadaccini [Fri, 4 Nov 2011 16:35:16 +0000 (16:35 +0000)]
Improve error handling in netmask change

- check if the master IP with the old netmask is up before attempting to
  change the netmask (to avoid a failed change netmask resulting in an
  undesired activation of the master IP);
- improve error messages of the backend function;
- in case of error, report the problem but otherwise change the cluster
  master_netmask parameter;
- remove duplicate error feedback.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd master_netmask to Cluster.UpgradeConfig
Andrea Spadaccini [Fri, 4 Nov 2011 15:27:14 +0000 (15:27 +0000)]
Add master_netmask to Cluster.UpgradeConfig

And also suppress pylint R0902 error about an object instance having
more than 20 attributes.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoFix cluster start failure due to missing netmask
Andrea Spadaccini [Fri, 4 Nov 2011 13:33:59 +0000 (13:33 +0000)]
Fix cluster start failure due to missing netmask

If the cluster version is upgraded from a version before commit
5a8648eb609f7e3a8d7ad7f82e93cfdd467a8fb5 to a version after that commit,
the master startup will fail because the ssconf file with the master
netmask will not be present, and it is read during the master voting
process.

This patch fixes this bug, making ssconf.GetMasterNetmask return a
default netmask value, and also corrects the docstring of this method.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoganeti(7): Adding documentation about env variables
René Nussbaumer [Fri, 4 Nov 2011 09:22:13 +0000 (10:22 +0100)]
ganeti(7): Adding documentation about env variables

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoutils.algo: Add utility to convert sequence to dictionary
Michael Hanselmann [Thu, 3 Nov 2011 18:03:48 +0000 (19:03 +0100)]
utils.algo: Add utility to convert sequence to dictionary

Useful for converting list of query fields to a dictionary and to
convert RPC definitions. Includes duplicate detection.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd QA test for “gnt-debug delay”
Michael Hanselmann [Thu, 3 Nov 2011 19:22:50 +0000 (20:22 +0100)]
Add QA test for “gnt-debug delay”

This would have caught a bug introduced with the recent RPC changes.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Make “test_delay” RPC actually work
Michael Hanselmann [Thu, 3 Nov 2011 19:18:25 +0000 (20:18 +0100)]
rpc: Make “test_delay” RPC actually work

Until now it would just call itself, eventually failing.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoQA: Add test for setting node's secondary IP address
Michael Hanselmann [Wed, 2 Nov 2011 19:17:29 +0000 (20:17 +0100)]
QA: Add test for setting node's secondary IP address

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agocmdlib: Allow specifying lock level when calculating node locks
Michael Hanselmann [Wed, 2 Nov 2011 15:54:51 +0000 (16:54 +0100)]
cmdlib: Allow specifying lock level when calculating node locks

This is needed to lock node resources.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoShow RPC calls from config in lock monitor
Michael Hanselmann [Wed, 26 Oct 2011 11:49:47 +0000 (13:49 +0200)]
Show RPC calls from config in lock monitor

With this patch all RPC calls at runtime of masterd will show up in the
lock monitor. There is a chicken-and-egg issue with initializing the
configuration with a context since the lock manager, containing the
monitor, requires the configuration. This is worked around by setting
the config's context only once the lock monitor is available.

Example:
rpc/node19.example.com/write_ssconf_files  Jq9/Job32/N_SET_PARAMjobqueue_update  Jq2/Job27/C_VERIFY_CONFIG

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoDerive IP hooks env variables from RPC parameter
Andrea Spadaccini [Wed, 2 Nov 2011 18:18:04 +0000 (18:18 +0000)]
Derive IP hooks env variables from RPC parameter

Let the environment variables of the master IP turnup/turndown be
derived from the parameter of the RPC itself (that is of type
objects.MasterNetworkParameters in both cases).

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoPass MasterNetworkParameters instances in RPCs
Andrea Spadaccini [Wed, 2 Nov 2011 17:51:58 +0000 (17:51 +0000)]
Pass MasterNetworkParameters instances in RPCs

Pass instances of objects.MasterNetworkParameters when calling RPCs for
activation and deactivation of master IP.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoUse MasterNetworkParameters attributes for RPC
Andrea Spadaccini [Wed, 2 Nov 2011 16:43:31 +0000 (16:43 +0000)]
Use MasterNetworkParameters attributes for RPC

Instead of manually unpacking the return values of
cfg.GetMasterNetworkParameters, let it return an instance of
objects.MasterNetworkParameters and pass its attributes.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoAdd object representing the master network params
Andrea Spadaccini [Wed, 2 Nov 2011 15:55:32 +0000 (15:55 +0000)]
Add object representing the master network params

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoUniform master IP activation and deactivation
Andrea Spadaccini [Wed, 2 Nov 2011 15:33:21 +0000 (15:33 +0000)]
Uniform master IP activation and deactivation

Add the master IP family parameter to the master IP deactivation RPCs,
so that the activation and deactivation interfaces are uniform.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoMake it possible to pass in flags using ENV variables
René Nussbaumer [Wed, 2 Nov 2011 15:07:15 +0000 (16:07 +0100)]
Make it possible to pass in flags using ENV variables

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoutils.algo: Add a function to insert a list into a list
René Nussbaumer [Wed, 2 Nov 2011 15:06:10 +0000 (16:06 +0100)]
utils.algo: Add a function to insert a list into a list

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix pylint deprecated directive in check-header
Andrea Spadaccini [Wed, 2 Nov 2011 13:34:17 +0000 (13:34 +0000)]
Fix pylint deprecated directive in check-header

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix overriding of ActivateMasterIp params
Andrea Spadaccini [Wed, 2 Nov 2011 13:31:52 +0000 (13:31 +0000)]
Fix overriding of ActivateMasterIp params

In the process of reworking the patches for master, I forgot to remove a
call to GetMasterInfo() that overrides the parameters passed via RPC.
This patch fixes this issue.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoAdd netmask and IP version to master IP hooks env
Andrea Spadaccini [Wed, 2 Nov 2011 11:47:28 +0000 (11:47 +0000)]
Add netmask and IP version to master IP hooks env

Add the master netmask and master IP version parameters to the hooks and
document them. In this way, the hook environment building function can
be reused for the planned master IP turnup changes.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoExplicitly pass params to change_master_netmask
Andrea Spadaccini [Thu, 27 Oct 2011 18:34:57 +0000 (19:34 +0100)]
Explicitly pass params to change_master_netmask

Make the master explicitly pass the parameters to the
change_master_netmask RPC, and change all the call flow to use the new
interface.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoExplicitly pass params to deactivate_master_ip
Andrea Spadaccini [Thu, 27 Oct 2011 18:18:00 +0000 (19:18 +0100)]
Explicitly pass params to deactivate_master_ip

Make the master explicitly pass the parameters to the
deactivate_master_ip RPC, and change all the call flow to use the new
interface.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoChange the semantics of stop_master RPC
Andrea Spadaccini [Fri, 28 Oct 2011 10:29:51 +0000 (11:29 +0100)]
Change the semantics of stop_master RPC

To avoid polluting the stop_master RPC interface with network
parameters, this patch removes the deactivation of the master IP from
this RPC.

The call to deactivate_master_ip is then added before every stop_master
call.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoExplicitly pass params to activate_master_ip
Andrea Spadaccini [Thu, 27 Oct 2011 16:08:34 +0000 (17:08 +0100)]
Explicitly pass params to activate_master_ip

To remove the usage of ssconf in the backend, the master needs to push
the parameters of activate_master_ip to the backend.

This patch changes the entire call path of activate_master_ip to use the
new interface.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agolocking: Make level names consistent, add check
Michael Hanselmann [Wed, 2 Nov 2011 12:19:01 +0000 (13:19 +0100)]
locking: Make level names consistent, add check

- Change all lock level names to their singular form
- Add assertion checking consistency between level names and lockset
  name

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd new lock level for node resource modifications
Michael Hanselmann [Wed, 2 Nov 2011 12:15:38 +0000 (13:15 +0100)]
Add new lock level for node resource modifications

This is in preparation for implementing a new resource model.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd tool to check Python file headers
Michael Hanselmann [Tue, 1 Nov 2011 15:42:36 +0000 (16:42 +0100)]
Add tool to check Python file headers

Ensures licence and copyright headers are correct.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoAdd documentation for the master IP hooks
Andrea Spadaccini [Tue, 11 Oct 2011 11:20:44 +0000 (12:20 +0100)]
Add documentation for the master IP hooks

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd master IP turnup and turndown hooks
Andrea Spadaccini [Tue, 25 Oct 2011 15:17:37 +0000 (16:17 +0100)]
Add master IP turnup and turndown hooks

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd RunLocalHooks decorator
Andrea Spadaccini [Tue, 25 Oct 2011 14:58:41 +0000 (15:58 +0100)]
Add RunLocalHooks decorator

Add the RunLocalHooks decorator, that allows the execution of hooks
locally. Also, add a RunLocalHooks method to HooksRunner, to adapt the
signature of HooksRunner.RunHooks to the one expected by HooksMaster,
and also to check that the hooks are being executed locally.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoGeneralize HooksMaster
Andrea Spadaccini [Tue, 25 Oct 2011 12:58:50 +0000 (13:58 +0100)]
Generalize HooksMaster

- remove any dependence on Logical Units from the HooksMaster;
- add a new function parameter to the constructor, a function that is
  expected to convert the results of the hooks execution in a format
  understood by the HooksMaster;
- add a factory method that builds a HooksMaster from a LU, keeping the
  interface of the old constructor of HooksMaster;
- remove usage of Processor.hmclass from external classes, introducing
  the Processor.BuildHooksMaster method;
- update unit tests.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoRAPI: Fix issue when replacing disks
Michael Hanselmann [Tue, 1 Nov 2011 15:02:16 +0000 (16:02 +0100)]
RAPI: Fix issue when replacing disks

This is another fallout from a suboptimal merge.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agorpc: Improve argument definitions
Michael Hanselmann [Mon, 31 Oct 2011 19:09:22 +0000 (20:09 +0100)]
rpc: Improve argument definitions

Instead of in-place code now each argument has an (optional) kind
assigned. The RPC client will then take care of encoding each argument
according to its definition. A new base (_RpcClientBase) is added to
take care of encoding and dispatching calls.

Future patches will also use these definitions in the node daemon.

A nice side-effect of this change is the ability to use stand-alone
functions, as opposed to class member methods, for most of the encoding.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorpc: Change {import,export}_start to take source/dest in single argument
Michael Hanselmann [Mon, 31 Oct 2011 17:47:15 +0000 (18:47 +0100)]
rpc: Change {import,export}_start to take source/dest in single argument

This simplifies the RPC argument encoding code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRename lib/build/rpc_definitions.py to lib/rpc_defs.py
Michael Hanselmann [Mon, 31 Oct 2011 17:24:04 +0000 (18:24 +0100)]
Rename lib/build/rpc_definitions.py to lib/rpc_defs.py

It will also be used at runtime, not just for building.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoClean up incorrect merge 7dcf333d
Michael Hanselmann [Mon, 31 Oct 2011 16:33:28 +0000 (17:33 +0100)]
Clean up incorrect merge 7dcf333d

The unittests weren't corrected for the master branch.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoMerge branch 'devel-2.5'
Michael Hanselmann [Mon, 31 Oct 2011 15:16:42 +0000 (16:16 +0100)]
Merge branch 'devel-2.5'

* devel-2.5:
  Fix wrong headers and licences
  Update NEWS and increase to 2.4.5
  Fix parameters of RpcResult in hooks unit tests
  Fix a too long line.
  Move RenameFile to the new functions
  ensure_dirs: Move some useful functions into utils.
  Use JoinDisjointDicts in mcpu
  Add the JoinDisjointDicts function to utils.algo
  Fix queue archive creation with wrong permissions
  Ensure permission on the job queue version file
  OpGroupVerifyDisks: Fix wrong result type declaration
  RAPI: Make node evacuation actually work
  Bump version to 2.5.0~rc2
  Update NEWS for unreleased 2.4.5
  RAPI: Fix resource for replacing disks

Conflicts:
lib/rapi/rlib2.py: Merged bugfix from commit 539d65ba0

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>