ganeti-local
12 years agoconstants: reindent a few dicts
Guido Trotter [Tue, 22 Nov 2011 10:07:45 +0000 (10:07 +0000)]
constants: reindent a few dicts

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoRemove BE_MEMORY from beparams but keep compatibility
Guido Trotter [Mon, 21 Nov 2011 11:18:22 +0000 (11:18 +0000)]
Remove BE_MEMORY from beparams but keep compatibility

Queries are already compatible (be/memory is an alias for be/maxmem) and
import/exports work. This patch patch fixes it for cluster init, modify
and instance add/start/modify.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoburnin: use mem_size as max and min
Guido Trotter [Mon, 21 Nov 2011 11:12:00 +0000 (11:12 +0000)]
burnin: use mem_size as max and min

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agounittests: use max/min memory
Guido Trotter [Mon, 21 Nov 2011 11:05:31 +0000 (11:05 +0000)]
unittests: use max/min memory

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib: use MAXMEM for all operations
Guido Trotter [Mon, 21 Nov 2011 10:56:50 +0000 (10:56 +0000)]
cmdlib: use MAXMEM for all operations

Since for now we can only start instances at their maximum memory, we
modify all checks to use that value. When we'll have better support for
using a value in between some of these checks have to move to minimum
memory.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoqa: use maximum and minimum memory
Guido Trotter [Mon, 21 Nov 2011 10:43:28 +0000 (10:43 +0000)]
qa: use maximum and minimum memory

test modification of either parameter, but also both at once.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohypervisors: use maximum memory for all operations
Guido Trotter [Mon, 21 Nov 2011 10:22:35 +0000 (10:22 +0000)]
hypervisors: use maximum memory for all operations

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoImportExport: use max and min memory params
Guido Trotter [Thu, 17 Nov 2011 14:09:32 +0000 (14:09 +0000)]
ImportExport: use max and min memory params

Import uses the old "memory" parameter to populate the two new ones, if
they're not overridden already.

FinalizeExport exports minmem and maxmem, but also memory, as maxmem, to
allow importing to older ganeti clusters.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoQuery: allow query on maximum and minimum memory
Guido Trotter [Thu, 17 Nov 2011 14:39:43 +0000 (14:39 +0000)]
Query: allow query on maximum and minimum memory

be/memory is kept as an alias.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoShowInstanceConfig: show max and min memory
Guido Trotter [Thu, 17 Nov 2011 14:12:44 +0000 (14:12 +0000)]
ShowInstanceConfig: show max and min memory

The old "memory" value is kept as maxmem, for now, for
retrocompatibility.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoinstance hooks: pass maximum and minimum memory
Guido Trotter [Thu, 17 Nov 2011 15:07:55 +0000 (15:07 +0000)]
instance hooks: pass maximum and minimum memory

Also pass the "memory" value for retrocompatibility, for now.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agobeparams: add min/max memory values
Guido Trotter [Thu, 10 Nov 2011 16:19:12 +0000 (16:19 +0000)]
beparams: add min/max memory values

For now the new "memory" parameter stays there, but it will be removed
later. The new values are just taken from the old one, in this patch.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agodesign-resource-model: update disk params section
Andrea Spadaccini [Mon, 21 Nov 2011 10:43:28 +0000 (10:43 +0000)]
design-resource-model: update disk params section

Simplify design by moving all the parameters to disk template level,
explaining why this is sub-optimal. Add notes about DRBD versions,
corner cases and parameters application time.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoSet DRBD sync speed in DRBD8.Assemble
Andrea Spadaccini [Wed, 16 Nov 2011 11:31:18 +0000 (11:31 +0000)]
Set DRBD sync speed in DRBD8.Assemble

Instead of relying on clients of the class for setting the device speed
(and, in general, the DRBD parameters), move this responsibility inside
the Assemble method.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agobuild-rpc: Fail if call is defined more than once
Michael Hanselmann [Mon, 21 Nov 2011 09:34:53 +0000 (10:34 +0100)]
build-rpc: Fail if call is defined more than once

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoReapply commit 2a6de57 after merge
Andrea Spadaccini [Fri, 18 Nov 2011 12:07:27 +0000 (12:07 +0000)]
Reapply commit 2a6de57 after merge

In the last merge I erroneously discarded the changes introduced by
commit 2a6de57 "Check the results of master IP RPCs". This commit
reintroduces them.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix QA breakage caused by merge 0e82dcf9
Michael Hanselmann [Mon, 21 Nov 2011 07:22:24 +0000 (08:22 +0100)]
Fix QA breakage caused by merge 0e82dcf9

Patch tested and confirmed to work by Andrea Spadaccini
<spadaccio@google.com>.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agomasterd: Initialize job queue only after RPC client
Michael Hanselmann [Thu, 17 Nov 2011 11:08:32 +0000 (12:08 +0100)]
masterd: Initialize job queue only after RPC client

Otherwise jobs started after an unclean master shutdown will fail as
they depend on the RPC client.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agomasterd: Shutdown only once running jobs have been processed
Michael Hanselmann [Thu, 17 Nov 2011 11:07:57 +0000 (12:07 +0100)]
masterd: Shutdown only once running jobs have been processed

Until now, if masterd received a fatal signal, it would start shutting
down immediately. In the meantime it would hang while jobs are still
processed. Clients couldn't connect anymore to retrieve a jobs' status.

This this patch masterd checks if any job is running before shutting
down. If there is it'll check again every five seconds. Once all jobs
are finished, it waits another five seconds to give clients a chance to
retrieve the jobs' status. After that masterd will shutdown in a clean
fashion.

If a second signal is received the old behaviour is preserved.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agodaemon: Support clean daemon shutdown
Michael Hanselmann [Thu, 17 Nov 2011 11:01:33 +0000 (12:01 +0100)]
daemon: Support clean daemon shutdown

Instead of aborting the main loop as soon as a fatal signal (SIGTERM or
SIGINT) is received, additional logic allows waiting for tasks to finish
while I/O is still being processed.

If no callback function is provided the old behaviour--shutting down
on the first signal--is preserved.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agodaemon: Allow custom maximum timeout for scheduler
Michael Hanselmann [Thu, 17 Nov 2011 10:56:34 +0000 (11:56 +0100)]
daemon: Allow custom maximum timeout for scheduler

This is needed in case the scheduler user (daemon.Mainloop in this case)
has other timeouts at the same time. Needed for clean master shutdown.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agojqueue: Add code to prepare for queue shutdown
Michael Hanselmann [Thu, 17 Nov 2011 10:55:18 +0000 (11:55 +0100)]
jqueue: Add code to prepare for queue shutdown

Doing so will prevent job submissions (similar to a drained queue),
but won't affect currently running jobs. No further jobs will be
executed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoworkerpool: Export function to check for running tasks
Michael Hanselmann [Wed, 16 Nov 2011 11:35:07 +0000 (12:35 +0100)]
workerpool: Export function to check for running tasks

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agodaemon: Use counter instead of boolean for mainloop abortion
Michael Hanselmann [Wed, 16 Nov 2011 11:34:50 +0000 (12:34 +0100)]
daemon: Use counter instead of boolean for mainloop abortion

Also log a message when a fatal signal was received and use dict.items.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: adjust imports for newer compilers
Iustin Pop [Thu, 17 Nov 2011 14:40:18 +0000 (15:40 +0100)]
htools: adjust imports for newer compilers

While testing with ghc 7.2, I saw that some imports we are using are
very old (from ghc 6.8 time), even though current libraries are using
different names.

We fix this and bump minimum documented version to ghc 6.12, as I
don't have 6.10 to test anymore (possibly still works with that
version, but better safe - both Ubuntu Lucid and Debian Squeeze ship
with 6.12 nowadays).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoMerge branch 'devel-2.5'
Andrea Spadaccini [Fri, 18 Nov 2011 11:27:16 +0000 (11:27 +0000)]
Merge branch 'devel-2.5'

* devel-2.5: (24 commits)
  LUInstanceCreate: Release unused node locks
  htools: rework message display construction
  hbal: handle empty node groups
  Document OpNodeMigrate's result for RAPI
  Ensure unused ports return to the free port pool
  Re-wrap a paragraph to eliminate a sphinx warning
  Fix newer pylint's E0611 error in compat.py
  Fail if node/group evacuation can't evacuate instances
  Update init script description
  LUInstanceRename: Compare name with name
  LUClusterRepairDiskSizes: Acquire instance locks in exclusive mode
  Update synopsis for “gnt-cluster repair-disk-sizes”
  Move hooks PATH environment variable to constants
  Check the results of master IP RPCs
  Add documentation for the master IP hooks
  Add master IP turnup and turndown hooks
  Add RunLocalHooks decorator
  Generalize HooksMaster
  Update NEWS for 2.5.0~rc4
  Bump version to 2.5.0~rc4
  ...

Conflicts:
NEWS
doc/hooks.rst
lib/backend.py
lib/cmdlib.py
lib/constants.py

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoMerge branch 'stable-2.5' into devel-2.5
Michael Hanselmann [Fri, 18 Nov 2011 10:09:33 +0000 (11:09 +0100)]
Merge branch 'stable-2.5' into devel-2.5

* stable-2.5:
  htools: rework message display construction
  hbal: handle empty node groups
  Document OpNodeMigrate's result for RAPI
  Fail if node/group evacuation can't evacuate instances
  LUInstanceRename: Compare name with name
  LUClusterRepairDiskSizes: Acquire instance locks in exclusive mode
  Update NEWS for 2.5.0~rc4
  Bump version to 2.5.0~rc4
  jqueue: Allow zero jobs to be submitted at once
  hail: don't select the primary as new secondary
  hail: add an extra safety check in relocate
  Bump version to 2.5.0~rc3

Conflicts:
configure.ac: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoMerge branch 'devel-2.4' into devel-2.5
Michael Hanselmann [Fri, 18 Nov 2011 07:27:30 +0000 (08:27 +0100)]
Merge branch 'devel-2.4' into devel-2.5

* devel-2.4:
  Ensure unused ports return to the free port pool
  Re-wrap a paragraph to eliminate a sphinx warning

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoadmin.rst update regarding offline state of the instance
Agata Murawska [Thu, 17 Nov 2011 14:59:39 +0000 (15:59 +0100)]
admin.rst update regarding offline state of the instance

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoNEWS update - offline instance state
Agata Murawska [Wed, 16 Nov 2011 15:59:37 +0000 (16:59 +0100)]
NEWS update - offline instance state

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoBackwards compatibity - added admin_up to query
Agata Murawska [Wed, 16 Nov 2011 16:08:49 +0000 (17:08 +0100)]
Backwards compatibity - added admin_up to query

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoMan page update: online/offline state of instance
Agata Murawska [Wed, 16 Nov 2011 15:41:15 +0000 (16:41 +0100)]
Man page update: online/offline state of instance

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd small node in admin.rst about confd disabling
Iustin Pop [Thu, 17 Nov 2011 11:33:28 +0000 (12:33 +0100)]
Add small node in admin.rst about confd disabling

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoWarn if we enable maintain-node-health without confd
Iustin Pop [Thu, 17 Nov 2011 11:31:10 +0000 (12:31 +0100)]
Warn if we enable maintain-node-health without confd

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdapt daemon-util to ENABLE_CONFD
Iustin Pop [Thu, 17 Nov 2011 11:19:22 +0000 (12:19 +0100)]
Adapt daemon-util to ENABLE_CONFD

We still allow explicit shutdown of confd, but we prevent manual
or automatic start-up.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdapt watcher for ENABLE_CONFD
Iustin Pop [Thu, 17 Nov 2011 11:04:58 +0000 (12:04 +0100)]
Adapt watcher for ENABLE_CONFD

If confd is disabled, do not automatically restart it. Furthermore, we
can't run maintenance actions if it is disabled so log a warning.

Note that I haven't completely disabled the NodeMaintenance class with
ENABLE_CONFD = False because I think they are at two different levels
(e.g. we might have other maintenance actions done even with confd
disabled).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoPrevent runnning of confd tests in burnin
Iustin Pop [Thu, 17 Nov 2011 10:55:03 +0000 (11:55 +0100)]
Prevent runnning of confd tests in burnin

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd toggle for enabling/disabling confd
Iustin Pop [Thu, 17 Nov 2011 10:49:56 +0000 (11:49 +0100)]
Add toggle for enabling/disabling confd

Doesn't do anything yet.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix unittest bug related to offline instances
Iustin Pop [Thu, 17 Nov 2011 10:19:41 +0000 (11:19 +0100)]
Fix unittest bug related to offline instances

Currently, the code in Node.hs is overly strict: once a node's free
memory reaches 0, it will refuse to add any instances (offline or
not). I think this is a safe safeguard (I don't expect nodes to run
without at least 1MB of free memory), so rather than change this
behaviour we need to restrict the Node generation in the unittest to
skip such nodes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Agata Murawska <agatamurawska@google.com>

12 years agohtools: reindent the rest of the files
Iustin Pop [Wed, 16 Nov 2011 18:14:48 +0000 (19:14 +0100)]
htools: reindent the rest of the files

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: re-indent IAlloc.hs
Iustin Pop [Wed, 16 Nov 2011 17:53:47 +0000 (18:53 +0100)]
htools: re-indent IAlloc.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: reindent hspace
Iustin Pop [Wed, 16 Nov 2011 17:19:42 +0000 (18:19 +0100)]
htools: reindent hspace

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: reindent hbal
Iustin Pop [Wed, 16 Nov 2011 17:18:16 +0000 (18:18 +0100)]
htools: reindent hbal

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: reindent CLI.hs
Iustin Pop [Wed, 16 Nov 2011 17:15:57 +0000 (18:15 +0100)]
htools: reindent CLI.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: re-indent QC.hs
Iustin Pop [Wed, 16 Nov 2011 17:11:41 +0000 (18:11 +0100)]
htools: re-indent QC.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: re-indent Node.hs
Iustin Pop [Wed, 16 Nov 2011 16:54:09 +0000 (17:54 +0100)]
htools: re-indent Node.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohtools: finish re-indenting Cluster.hs
Iustin Pop [Wed, 16 Nov 2011 16:45:11 +0000 (17:45 +0100)]
htools: finish re-indenting Cluster.hs

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agomasterd: Don't pass mainloop to server class
Michael Hanselmann [Wed, 16 Nov 2011 09:35:06 +0000 (10:35 +0100)]
masterd: Don't pass mainloop to server class

It is not used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoworkerpool: Allow processing of new tasks to be stopped
Michael Hanselmann [Wed, 16 Nov 2011 08:37:49 +0000 (09:37 +0100)]
workerpool: Allow processing of new tasks to be stopped

This is different from “Quiesce” in the sense that this function just
changes an internal flag and doesn't wait for the queue to be empty.
Tasks already being processed continue normally, but no new tasks will
be started. New tasks can still be added, but won't be processed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoworkerpool: Use loop to ignore spurious notifications
Michael Hanselmann [Wed, 16 Nov 2011 08:11:59 +0000 (09:11 +0100)]
workerpool: Use loop to ignore spurious notifications

This saves us from returning to the worker code when there is no
task to be processed.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agojqueue: Factorize code checking for drained queue
Michael Hanselmann [Wed, 16 Nov 2011 08:06:23 +0000 (09:06 +0100)]
jqueue: Factorize code checking for drained queue

This is in preparation for a clean(er) shutdown of masterd.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUInstanceCreate: Release unused node locks
Michael Hanselmann [Thu, 17 Nov 2011 11:36:18 +0000 (12:36 +0100)]
LUInstanceCreate: Release unused node locks

After iallocator ran we can release any unused node locks. Since they
must be in exclusive mode this should improve parallelization during
instance creation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib.TLReplaceDisks: Use itertools.count
Michael Hanselmann [Wed, 16 Nov 2011 07:08:34 +0000 (08:08 +0100)]
cmdlib.TLReplaceDisks: Use itertools.count

… instead of a variable which needs to be incremented for every step.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: rework message display construction
Iustin Pop [Tue, 15 Nov 2011 17:16:36 +0000 (18:16 +0100)]
htools: rework message display construction

While diagnosing some (unrelated) memory usage in htools, I've
stumbled upon some very bad behaviour in checkData: mapAccum is
non-strict, and the tuple we use also, so that results in the list of
list of messages being very bad space-wise (hundreds of MB of memory
for a simulated cluster with thousands of nodes, all with errors).

The new, explicit reuse of the old message list has a linear memory
behaviour. The only downside is that messages are listed in the
reverse order (which I'll fix on master).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohbal: handle empty node groups
Iustin Pop [Tue, 15 Nov 2011 10:15:34 +0000 (11:15 +0100)]
hbal: handle empty node groups

This patch changes an internal assert (which can only be triggered
when a node group is empty) into properly handling this case (and
returning empty node/instance lists).

While we could handle this in the backend (Cluster.splitNodeGroup)
this would actually mean than we change the behaviour for a cluster
with just two node groups, once of which is empty (where today we
don't require a node group argument).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoStart converting the indentation to 2-chars
Iustin Pop [Mon, 14 Nov 2011 14:33:20 +0000 (15:33 +0100)]
Start converting the indentation to 2-chars

… also do some other small style cleanups.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agohtools: Nicer arbitrary instantiation
Agata Murawska [Tue, 15 Nov 2011 15:48:23 +0000 (16:48 +0100)]
htools: Nicer arbitrary instantiation

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: change behaviour for offline instances
Agata Murawska [Tue, 15 Nov 2011 10:28:40 +0000 (11:28 +0100)]
htools: change behaviour for offline instances

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: remove 'running' from instance options
Agata Murawska [Mon, 14 Nov 2011 13:54:32 +0000 (14:54 +0100)]
htools: remove 'running' from instance options

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: change runSt type to InstanceStatus
Agata Murawska [Thu, 10 Nov 2011 10:21:32 +0000 (11:21 +0100)]
htools: change runSt type to InstanceStatus

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agohtools: create InstanceStatus ADT
Agata Murawska [Thu, 10 Nov 2011 10:10:13 +0000 (11:10 +0100)]
htools: create InstanceStatus ADT

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoTransition into and out of offline instance state
Agata Murawska [Tue, 8 Nov 2011 10:05:43 +0000 (11:05 +0100)]
Transition into and out of offline instance state

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoIntroduce admin_state as 3-values data type
Agata Murawska [Tue, 8 Nov 2011 10:04:34 +0000 (11:04 +0100)]
Introduce admin_state as 3-values data type

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoRename admin_up to admin_state
Agata Murawska [Thu, 27 Oct 2011 11:29:41 +0000 (13:29 +0200)]
Rename admin_up to admin_state

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFixed typo in _VerifyResultRow
Agata Murawska [Fri, 28 Oct 2011 11:22:02 +0000 (13:22 +0200)]
Fixed typo in _VerifyResultRow

Signed-off-by: Agata Murawska <agatamurawska@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoalgo: Make a dict from an flat list
René Nussbaumer [Tue, 15 Nov 2011 14:09:59 +0000 (15:09 +0100)]
algo: Make a dict from an flat list

This is in preparation to take deeper dict constructs from the command
line. You can feed the optionslist directly constructed of type
"identkeyval" to it and it returns a fully deflated dict.

This is mainly needed for the resource model changes where we have to
modify the disk_state which is a 3 level dict:

disk_type/name:disk_reserved=10g

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agolocking: Make some aliased methods public
Michael Hanselmann [Tue, 15 Nov 2011 13:46:21 +0000 (14:46 +0100)]
locking: Make some aliased methods public

Some methods, such as “_is_owned” and “list_owned”, have been aliased to
make them public for a while now. This patch makes the actual
implementation public.

SharedLock's “is_owned” needs to be aliased to “_is_owned” to remain
compatible with Python's built-in threaded.Condition class.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib._ReleaseLock: Do nothing if no locks are owned
Michael Hanselmann [Tue, 15 Nov 2011 11:23:43 +0000 (12:23 +0100)]
cmdlib._ReleaseLock: Do nothing if no locks are owned

The locking library doesn't like it when “release()” is called on
a lockset or lock which isn't held by the current thread. Instead
of modifying the library, which could have other side-effects,
this rather simple change avoids errors when a LU simply tries to
release all locks, even when it doesn't own any at a certain level.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse resource lock when setting node parameters
Michael Hanselmann [Tue, 15 Nov 2011 06:21:34 +0000 (07:21 +0100)]
Use resource lock when setting node parameters

Also acquire instance and resource locks in shared mode (see comment).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse node resource lock for replacing instance disks
Michael Hanselmann [Thu, 3 Nov 2011 16:12:19 +0000 (17:12 +0100)]
Use node resource lock for replacing instance disks

If early-release is not used, the resource lock is kept while waiting
for disks to sync.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoHold node resource locks while setting instance parameters
Michael Hanselmann [Thu, 3 Nov 2011 16:19:30 +0000 (17:19 +0100)]
Hold node resource locks while setting instance parameters

Important for when disks are converted. Release locks once they're not
needed anymore. Make liberal use of assertions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoHold node resource lock while moving instance
Michael Hanselmann [Thu, 3 Nov 2011 16:08:26 +0000 (17:08 +0100)]
Hold node resource lock while moving instance

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAcquire node resource lock when removing instance
Michael Hanselmann [Thu, 3 Nov 2011 16:04:05 +0000 (17:04 +0100)]
Acquire node resource lock when removing instance

Removing an instance affects available disk space and memory.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse node resource lock when recreating instance disks
Michael Hanselmann [Thu, 3 Nov 2011 15:58:00 +0000 (16:58 +0100)]
Use node resource lock when recreating instance disks

Recreating disks conflicts with other disk operations, therefore the
node resource lock must be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUClusterRepairDiskSizes: Use node resource locks
Michael Hanselmann [Thu, 3 Nov 2011 15:40:06 +0000 (16:40 +0100)]
LUClusterRepairDiskSizes: Use node resource locks

Since this doesn't really touch the node, but it conflicts with e.g.
growing a disk, the resource lock must be acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUInstanceGrowDisk: Use node resource lock
Michael Hanselmann [Wed, 2 Nov 2011 15:55:33 +0000 (16:55 +0100)]
LUInstanceGrowDisk: Use node resource lock

Also add one more feedback line. Downgrade instance lock to shared mode
while we're only waiting for disks to sync. The node lock is released
when not needed anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUInstanceCreate: Hold node resource lock
Michael Hanselmann [Wed, 2 Nov 2011 15:33:46 +0000 (16:33 +0100)]
LUInstanceCreate: Hold node resource lock

The node resource lock is released once the disks are in sync (that is,
after wiping).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeQueryvols: Acquire all locks in shared mode
Michael Hanselmann [Mon, 14 Nov 2011 13:12:08 +0000 (14:12 +0100)]
LUNodeQueryvols: Acquire all locks in shared mode

Nothing is being written to.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeQueryStorage: Acquire all locks in shared mode
Michael Hanselmann [Mon, 14 Nov 2011 13:11:48 +0000 (14:11 +0100)]
LUNodeQueryStorage: Acquire all locks in shared mode

Nothing is written to.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib: Share lock in LUInstanceConsole
Michael Hanselmann [Mon, 14 Nov 2011 09:36:50 +0000 (10:36 +0100)]
cmdlib: Share lock in LUInstanceConsole

No writes are being done.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoDocument OpNodeMigrate's result for RAPI
Michael Hanselmann [Fri, 11 Nov 2011 17:04:55 +0000 (18:04 +0100)]
Document OpNodeMigrate's result for RAPI

- Commit b7a1c8161 changed the LU to generate jobs
- Mention documented results in NEWS

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeQuery: Call implementation's DeclareLocks function
Michael Hanselmann [Thu, 3 Nov 2011 16:22:34 +0000 (17:22 +0100)]
LUNodeQuery: Call implementation's DeclareLocks function

Just in case we ever add locks for querying nodes. Currently
_NodeQuery's DeclareLocks is a no-op function.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse master IP address setup script in backend
Andrea Spadaccini [Tue, 25 Oct 2011 09:54:10 +0000 (10:54 +0100)]
Use master IP address setup script in backend

Replace the code in backend.ActivateMasterIp and
backend.DeactivateMasterIp with the master IP address setup script,
either the default one or the one provided by the user.

- Convert to string the netmask parameter in _BuildMasterIpEnv
- Add the _RunMasterSetupScript function and let ActivateMasterIp and
  DeactivateMasterIp become wrappers for it;
- Change the failure model of DeactivateMasterIp, raising errors if the
  script fails instead of silently ignoring the failure

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoChange master IP address RPCs for external script
Andrea Spadaccini [Tue, 8 Nov 2011 17:09:56 +0000 (17:09 +0000)]
Change master IP address RPCs for external script

Change the master IP address RPC call chain to accept the
use_external_master_ip_script parameter. Introduces an unused parameter
in backend.ActivateMasterIp and backend.DeactivateMasterIp, that will be
used in the next commit.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUpdate cluster verify to check IP address scripts
Andrea Spadaccini [Tue, 8 Nov 2011 16:29:52 +0000 (16:29 +0000)]
Update cluster verify to check IP address scripts

Update cluster-verify to check the integrity of the default master IP
address setup script and the presence and executability of the external
one (if currently in use by the cluster).

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd --use-external-mip-script flag
Andrea Spadaccini [Tue, 8 Nov 2011 15:32:13 +0000 (15:32 +0000)]
Add --use-external-mip-script flag

- add a command line flag to gnt-cluster init and modify to change the
  value of the cluster parameter use_external_mip_script;
- add two constants representing the paths of the default script and of
  the external script;
- add documentation for the new option in the gnt-cluster man page;
- add use_external_mip_script to gnt-cluster info output.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd use_external_mip_script cluster parameter
Andrea Spadaccini [Tue, 8 Nov 2011 14:41:12 +0000 (14:41 +0000)]
Add use_external_mip_script cluster parameter

Add the use_external_mip_script cluster parameter, that represents
whether the master IP address turnup/turndown procedures must use a
script provided by the user (True) or the one provided by Ganeti
(False).

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd the default master-ip-setup script
Andrea Spadaccini [Mon, 24 Oct 2011 16:38:10 +0000 (17:38 +0100)]
Add the default master-ip-setup script

Add a shell script that will be the default replacement for the body of
backend.ActivateMasterIp and backend.DeactivateMasterIp.

Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoEnsure unused ports return to the free port pool
Vangelis Koukis [Thu, 27 Oct 2011 17:04:20 +0000 (20:04 +0300)]
Ensure unused ports return to the free port pool

Ensure ports previously allocated by calling ConfigWriter's AllocatePort() are
returned to the pool of free ports when no longer needed:

 * Return the network_port of an instance when it is removed
 * Return the port used by a DRBD-based disk when it is removed

Signed-off-by: Vangelis Koukis <vkoukis@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoRe-wrap a paragraph to eliminate a sphinx warning
Iustin Pop [Mon, 14 Nov 2011 09:01:06 +0000 (10:01 +0100)]
Re-wrap a paragraph to eliminate a sphinx warning

This just makes sure that the paragraph doesn't contains lines that
start with :, which make Sphinx (1.0.7) complain.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agocheck-python-code: Detect old-style pylint disable-msg lines
Michael Hanselmann [Fri, 11 Nov 2011 12:47:24 +0000 (13:47 +0100)]
check-python-code: Detect old-style pylint disable-msg lines

Commit b459a848d was supposed to replace all, but one was missed. Add a
check to autotools/check-python-code.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agoFix newer pylint's E0611 error in compat.py
Guido Trotter [Thu, 10 Nov 2011 13:46:54 +0000 (13:46 +0000)]
Fix newer pylint's E0611 error in compat.py

These are triggered by our "stay-compatible" approach.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoAdding basic elements for the new node params
René Nussbaumer [Thu, 3 Nov 2011 12:52:51 +0000 (13:52 +0100)]
Adding basic elements for the new node params

This patch adds the new fields to the objects.py as well as defines the
constants used in the dicts and their type.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Fix another result processor
Michael Hanselmann [Tue, 8 Nov 2011 17:17:37 +0000 (18:17 +0100)]
rpc: Fix another result processor

I forgot to change this in commit d9da5065c0.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Fix issue with “test_delay”'s timeout
Michael Hanselmann [Tue, 8 Nov 2011 15:40:18 +0000 (16:40 +0100)]
rpc: Fix issue with “test_delay”'s timeout

I passed the timeout calculation function in the wrong field of the
definition. A small change is also needed in “build-rpc” to not abort
when writing the docstring.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorpc: Call result processor once for each node result
Michael Hanselmann [Tue, 8 Nov 2011 15:39:32 +0000 (16:39 +0100)]
rpc: Call result processor once for each node result

… instead of calling it with the whole results dictionary. This
fixes an issue when replacing disks (and all other cases where
result processors are used).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFail if node/group evacuation can't evacuate instances
Michael Hanselmann [Mon, 7 Nov 2011 19:42:31 +0000 (20:42 +0100)]
Fail if node/group evacuation can't evacuate instances

If an instance can't be evacuated, only a message would be printed. With
this change the operation always aborts. Newly added unittests check for
this behaviour.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUNodeSetParams: Lock affected instances only
Michael Hanselmann [Wed, 2 Nov 2011 19:13:58 +0000 (20:13 +0100)]
LUNodeSetParams: Lock affected instances only

Until now LUNodeSetParams would lock all instances if a node's
secondary IP address was to be changed and would then release
all instances it didn't actually need. With this patch the LU
optimistically locks instances and, once it got the locks,
checks whether they're still correct. This is similar to how
node group locking is done.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoCheck BGL when adding/removing node
Michael Hanselmann [Thu, 3 Nov 2011 15:48:56 +0000 (16:48 +0100)]
Check BGL when adding/removing node

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRPC/test_delay: Use callable for timeout calculation
Michael Hanselmann [Thu, 3 Nov 2011 19:35:27 +0000 (20:35 +0100)]
RPC/test_delay: Use callable for timeout calculation

This avoids having to override the function in the RPC runner.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>