ganeti-local
12 years agoAdd new spindle_count node parameter
Iustin Pop [Fri, 27 Jan 2012 13:29:31 +0000 (14:29 +0100)]
Add new spindle_count node parameter

Currently this is not handled by Ganeti, just recorded.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix upgrading of ndparams
Iustin Pop [Fri, 27 Jan 2012 13:26:35 +0000 (14:26 +0100)]
Fix upgrading of ndparams

Currently, we only upgrade the ndparams if they are missing
completely, which creates problems if we add any new parameters on an
already-upgraded cluster.

Fix this by adding an UpgradeNDParams function.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoSmall fixes to objects.UpgradeDiskParams
Iustin Pop [Fri, 27 Jan 2012 13:25:03 +0000 (14:25 +0100)]
Small fixes to objects.UpgradeDiskParams

Fix a typo, and cleanup the code a bit.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoopcodes: Add result checks for OpBackup*
Michael Hanselmann [Mon, 20 Feb 2012 16:21:51 +0000 (17:21 +0100)]
opcodes: Add result checks for OpBackup*

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoImprove opcode result check test to have a whitelist
Michael Hanselmann [Mon, 20 Feb 2012 12:29:38 +0000 (13:29 +0100)]
Improve opcode result check test to have a whitelist

Only whitelisted opcodes may not have a result check.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoconfig: Add check for disk's “iv_name”
Michael Hanselmann [Fri, 17 Feb 2012 16:49:35 +0000 (17:49 +0100)]
config: Add check for disk's “iv_name”

This check verifies “iv_name” of all instance disks. If one is wrong
(which shouldn't happen in the first place), cluster verification will
warn:

“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:
name of disk 1 should be 'disk/1', but is 'disk/4'”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoopcodes: Fix OP_RESULT for OobCommand
René Nussbaumer [Mon, 20 Feb 2012 10:35:11 +0000 (11:35 +0100)]
opcodes: Fix OP_RESULT for OobCommand

The result is a list of a list with elements with size of 2.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocli: Handle negative numbers when parsing key-value assignments
Michael Hanselmann [Fri, 17 Feb 2012 10:49:00 +0000 (11:49 +0100)]
cli: Handle negative numbers when parsing key-value assignments

To remove the last disk, or to add a NIC to the end, one can use the
index -1. This wouldn't work as intended as “-” is a special prefix.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoutils.text: Add function to truncate string
Michael Hanselmann [Fri, 17 Feb 2012 12:42:04 +0000 (13:42 +0100)]
utils.text: Add function to truncate string

The function adds an ellipse if the string was actually truncated. Also
start using it in mcpu for result checks (where the message is also
slightly changed to use a colon).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoFix breakage introduced by fa6dd6bb56
Michael Hanselmann [Fri, 17 Feb 2012 12:46:55 +0000 (13:46 +0100)]
Fix breakage introduced by fa6dd6bb56

Forgot “enumerate”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUInstanceSetParams: Update disk's “iv_name”
Michael Hanselmann [Wed, 15 Feb 2012 16:52:47 +0000 (17:52 +0100)]
LUInstanceSetParams: Update disk's “iv_name”

When modifications are made, disks may not have the same index anymore.
Updating all disks fixes this.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoApplyContainerMods: Fix issues with indices
Michael Hanselmann [Wed, 15 Feb 2012 17:50:08 +0000 (18:50 +0100)]
ApplyContainerMods: Fix issues with indices

When adding an item the index given to the callback function would be
incorrect under certain conditions. This patch also adds assertions and
more tests.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agocmdlib: Remove some users of “iv_name”
Michael Hanselmann [Wed, 15 Feb 2012 16:36:36 +0000 (17:36 +0100)]
cmdlib: Remove some users of “iv_name”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoConfigWriter: Stop using “iv_name”
Michael Hanselmann [Wed, 15 Feb 2012 16:09:20 +0000 (17:09 +0100)]
ConfigWriter: Stop using “iv_name”

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoConfigWriter.RenameInstance: Stop using iv_name, safer operation
Michael Hanselmann [Wed, 15 Feb 2012 16:06:29 +0000 (17:06 +0100)]
ConfigWriter.RenameInstance: Stop using iv_name, safer operation

Stop using the disk index encoded in “iv_name” when renaming an instance.

This patch also changes the code to operate on a copy of the instance
until the major changes have been applied. In the case of a failure we
won't loose the instance object anymore.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agognt-instance modify: Support new-style NIC/disk modifications
Michael Hanselmann [Tue, 14 Feb 2012 12:41:57 +0000 (13:41 +0100)]
gnt-instance modify: Support new-style NIC/disk modifications

This patch adds support for adding/removing NICs/disks at arbitrary
indices on the command line. To add a disk at a specified index, use
“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”.
Unittests are included.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoopcodes: Adding missing OP_RESULTs
René Nussbaumer [Fri, 10 Feb 2012 14:51:53 +0000 (15:51 +0100)]
opcodes: Adding missing OP_RESULTs

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agomcpu: Make the op result exception more verbose
René Nussbaumer [Mon, 13 Feb 2012 09:14:49 +0000 (10:14 +0100)]
mcpu: Make the op result exception more verbose

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib: De-duplicate code in _GenerateDiskTemplate
Michael Hanselmann [Wed, 15 Feb 2012 13:23:49 +0000 (14:23 +0100)]
cmdlib: De-duplicate code in _GenerateDiskTemplate

There has been a lot of duplicated code in _GenerateDiskTemplate,
and some cases of very similar, but not quite same duplicates. This
patch merges them.

Generating a disk's “logical_id” attribute is done via a
lambda/function. Maybe the ID's could be pre-computed and stored in a
list.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoAdd unittest for cmdlib._GenerateDiskTemplate
Michael Hanselmann [Tue, 14 Feb 2012 13:53:28 +0000 (14:53 +0100)]
Add unittest for cmdlib._GenerateDiskTemplate

This is in preparation to de-duplicating significant chunks of code in
cmdlib._GenerateDiskTemplate.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorapi.testutils: Add exported functions to verify opcode input/result
Michael Hanselmann [Tue, 14 Feb 2012 16:11:41 +0000 (17:11 +0100)]
rapi.testutils: Add exported functions to verify opcode input/result

These can be used by third-party code to verify mock code. Further work
on mocks is forthcoming, so this is only a start.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agocmdlib: Fix “unpack non-sequence” error
Michael Hanselmann [Tue, 14 Feb 2012 11:47:47 +0000 (12:47 +0100)]
cmdlib: Fix “unpack non-sequence” error

The callback is expected to return a two-valued tuple.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFix doc bug introduced in 12637df
Iustin Pop [Fri, 3 Feb 2012 16:22:05 +0000 (17:22 +0100)]
Fix doc bug introduced in 12637df

Commit 12637df changed/generalised how we build fields in the sphinx
extension, however it resulted in this uncaught-so-far result:

$ echo @QUERY_FIELDS_GROUP@ | ./autotools/docpp
<generator object BuildValuesDoc at 0x28fd370>

There's no point in yield-ing a single value (as opposed to a stream
of values), so let's just "return" it.

By the way, there's no test for this kind of error, but I don't think
it would happen too often, so I didn't add one (e.g. grep "<.* at
0x[0-9a-f]+>" man/*.[178]).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agohtools: allow rbd disk template
Guido Trotter [Mon, 13 Feb 2012 16:56:39 +0000 (16:56 +0000)]
htools: allow rbd disk template

Notes:
  - it'd be nice if there was a way to automatically generate the
    DiskTemplate list instead of manually specifying it. After all we
    have C.diskTemplates
  - Of course this actually does nothing, as for other
    "half-implemented" templates :(

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoApplyContainerMods: Return changes from callbacks
Michael Hanselmann [Mon, 13 Feb 2012 16:47:08 +0000 (17:47 +0100)]
ApplyContainerMods: Return changes from callbacks

… instead of passing the list of changes as a parameter.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUse the ipolicy constants for key names
Iustin Pop [Fri, 27 Jan 2012 15:59:04 +0000 (16:59 +0100)]
Use the ipolicy constants for key names

Sorry, I missed this when I wrote the code originally.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoLUInstanceSetParams: Convert to generic algorithm for NIC/disk changes
Michael Hanselmann [Fri, 10 Feb 2012 16:59:55 +0000 (17:59 +0100)]
LUInstanceSetParams: Convert to generic algorithm for NIC/disk changes

Unfortunately this got a bit messier than I intended, but then again it
cleans up a lot of messy code with heaps of local variables
(“this_nic_override”) and LU attributes (“nic_pnew”, “nic_pinst”). Most
of these variables were index by a number, or one of the
constants.DDM_* constants.

This patch moves the code for adding/modifying/removing a NIC/disk to
dedicated, small functions. The previously added generic algorithm for
applying changes to containers is then used to actually change the
instance's network interfaces or disks based on the requested
modifications. The LU now supports adding/removing disks/NICs in
arbitrary positions.

The compuation of all network interface changes has been moved to
CheckPrereq, so that its result can be used for hooks. For this to work
without side-effects, the NIC objects need to be copied (only done if
there are actual changes).

The command line utility still needs to be updated.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib: Add generic algorithm for modifying NICs/disks
Michael Hanselmann [Fri, 10 Feb 2012 16:57:23 +0000 (17:57 +0100)]
cmdlib: Add generic algorithm for modifying NICs/disks

While preparing this patch series I identified at least three different
implementations of the algorithm for adding/removing/changing
NICs/disks. These two functions and corresponding unittests provide a
generic implementation with added support for adding/removing arbitrary
disks or NICs.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFix unittest breakage after commit a2aadb34b
Michael Hanselmann [Mon, 13 Feb 2012 13:57:39 +0000 (14:57 +0100)]
Fix unittest breakage after commit a2aadb34b

Thou shalt run “make check” before sending patches.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUInstanceSetParams: Assertion on disk template/disk changes
Michael Hanselmann [Fri, 10 Feb 2012 16:59:40 +0000 (17:59 +0100)]
LUInstanceSetParams: Assertion on disk template/disk changes

Disk changes aren't allowed at the same time as a disk template change.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoOpInstanceSetParams: Make two type checks public
Michael Hanselmann [Fri, 10 Feb 2012 17:04:38 +0000 (18:04 +0100)]
OpInstanceSetParams: Make two type checks public

They'll be used for tests in cmdlib.py.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoOpInstanceSetParams: Accept more flexible NIC/disk modifications
Michael Hanselmann [Fri, 10 Feb 2012 16:55:39 +0000 (17:55 +0100)]
OpInstanceSetParams: Accept more flexible NIC/disk modifications

Start accepting a new form of NIC/disk modifications for adding/removing
arbitrary NICs/disks. Unlike before the index must always be given. An
index of “-1” stands for the last item.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoconstants: Add constant for modifying existing NIC/disk
Michael Hanselmann [Fri, 10 Feb 2012 16:56:26 +0000 (17:56 +0100)]
constants: Add constant for modifying existing NIC/disk

Until now it would not be possible to add/remove a NIC/disk in an
arbitrary position. This constant will be used in the data structures
necessary to add/remove arbitrary NICs/disks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoopcodes: Add comments to two parameter tests
Michael Hanselmann [Fri, 10 Feb 2012 16:54:51 +0000 (17:54 +0100)]
opcodes: Add comments to two parameter tests

These comments appear in the RAPI documentation.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agorbd disk template documentation and manpages
Stratos Psomadakis [Fri, 10 Feb 2012 10:29:13 +0000 (12:29 +0200)]
rbd disk template documentation and manpages

Add documentation and modify manpages for the RBD disk template.

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoRe-added constant mistakenly removed in 58f0ce16873
Michael Hanselmann [Wed, 1 Feb 2012 07:41:22 +0000 (08:41 +0100)]
Re-added constant mistakenly removed in 58f0ce16873

“INSTANCE_DOWN” is still being used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agorapi.client: Removing constants not needed anymore
René Nussbaumer [Tue, 31 Jan 2012 12:14:13 +0000 (13:14 +0100)]
rapi.client: Removing constants not needed anymore

They were not referenced through the whole code and were marked internal
only.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd stricter checks for OpInstanceSetParams.{nics,disks}
Michael Hanselmann [Tue, 31 Jan 2012 11:53:53 +0000 (12:53 +0100)]
Add stricter checks for OpInstanceSetParams.{nics,disks}

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoAdd ht.TMaybeListOf type check
Michael Hanselmann [Tue, 31 Jan 2012 11:53:32 +0000 (12:53 +0100)]
Add ht.TMaybeListOf type check

Replaces some uses of TOr(TNone, TListOf(…)).

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoStricter check for OS modifications passed to OpClusterSetParams
Michael Hanselmann [Tue, 31 Jan 2012 07:56:35 +0000 (08:56 +0100)]
Stricter check for OS modifications passed to OpClusterSetParams

Don't just check the first element of each item, but also make sure the
OS name is a string and not empty.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoLUInstanceSetParams: Allow no-op change of instance offline status
Michael Hanselmann [Tue, 31 Jan 2012 06:39:02 +0000 (07:39 +0100)]
LUInstanceSetParams: Allow no-op change of instance offline status

With this patch marking an instance already marked offline (or online)
as offline/online again becomes a no-op. Also removed the unused
INSTANCE_UP variable.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoOpInstanceSetParams: Merge {off,on}line_inst parameters
Michael Hanselmann [Mon, 30 Jan 2012 13:43:00 +0000 (14:43 +0100)]
OpInstanceSetParams: Merge {off,on}line_inst parameters

Instead of having two separate parameters, a single boolean parameter is
used. Unfortunately we need a third state to say “no change”, so the
value can be None, True or False (similar to other parameters). There
are no user interface changes.

New QA tests are added, too.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoipolicy: Make the keys of the dict consistent
René Nussbaumer [Fri, 27 Jan 2012 15:24:33 +0000 (16:24 +0100)]
ipolicy: Make the keys of the dict consistent

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agowalkthrough.rst: updates for maxmem/minmem
Guido Trotter [Fri, 27 Jan 2012 13:35:47 +0000 (13:35 +0000)]
walkthrough.rst: updates for maxmem/minmem

- Fix two example outputs
- Improve N+1 resolution section specifying how different memories
  affect N+1 calculation depending on the node

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoadmin.rst: add information about maxmem/minmem
Guido Trotter [Fri, 27 Jan 2012 13:33:31 +0000 (13:33 +0000)]
admin.rst: add information about maxmem/minmem

- Specify how maxmem and minmem are used at startup, failover and
  migrate
- Add a paragraph about how to manually change the runtime memory, and
  what Ganeti will and won't do automatically as of version 2.6

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoN+1: verify using minimum memory
Guido Trotter [Fri, 27 Jan 2012 13:33:06 +0000 (13:33 +0000)]
N+1: verify using minimum memory

Since instances can be started, failed over and migrated with less than
their maximum memory N+1 will use the minimum memory for verification.

Note that this accounts only for the instances being moved being
resized, and not for the ones already on the node, as ganeti will not
automatically resize other instances on the target node now when trying
to start/failover/migrate an instance.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFix integer overflow in Node unittest
Iustin Pop [Thu, 26 Jan 2012 18:41:13 +0000 (19:41 +0100)]
Fix integer overflow in Node unittest

When using a really random "extra cpus" variable, this can overflow
(especially on 32 bit) and thus the final value can be < 0, thus the
test will fail to fail.

It doesn't make sense to test with huge values, so let's just generate
a reasonable value.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoUse allow_runtime_chgs in cmdlib
Guido Trotter [Wed, 25 Jan 2012 15:37:07 +0000 (15:37 +0000)]
Use allow_runtime_chgs in cmdlib

- Report the attribute value in the hooks runtime variable.
- Use it to abort migration if not enough memory is available on the
  target node.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agocli/opcodes: add a --no-runtime-changes migrate opt
Guido Trotter [Wed, 25 Jan 2012 15:26:22 +0000 (15:26 +0000)]
cli/opcodes: add a --no-runtime-changes migrate opt

This will be used for now to avoid ballooning memory at live migration time.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoMigrate/Failover: allow less-than-max-mem op
Guido Trotter [Fri, 20 Jan 2012 14:33:29 +0000 (14:33 +0000)]
Migrate/Failover: allow less-than-max-mem op

Failover works magically after we relax the check: the start operation
will just use less memory, if less memory is available.

For migrate we need to balloon down the source memory before the
migration, if the target node doesn't have enough capacity.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFailover and Migrate: acquire node resource locks
Guido Trotter [Fri, 20 Jan 2012 14:28:13 +0000 (14:28 +0000)]
Failover and Migrate: acquire node resource locks

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agocmdlib._CheckNodeFreeMemory: return free memory
Guido Trotter [Mon, 12 Dec 2011 12:27:39 +0000 (12:27 +0000)]
cmdlib._CheckNodeFreeMemory: return free memory

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agodoc/rapi.rst: Document ipolicy parameter
René Nussbaumer [Thu, 26 Jan 2012 14:02:59 +0000 (15:02 +0100)]
doc/rapi.rst: Document ipolicy parameter

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoExpand the tiered alloc unittest to check allocation stats
Iustin Pop [Wed, 25 Jan 2012 14:00:48 +0000 (15:00 +0100)]
Expand the tiered alloc unittest to check allocation stats

This was missing before, so with the new metric (NCpus), it's a good
time to test this.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoSwitch over some hspace stats from RSpec to AllocInfo
Iustin Pop [Wed, 25 Jan 2012 10:33:59 +0000 (11:33 +0100)]
Switch over some hspace stats from RSpec to AllocInfo

This patch changes the allocation delta stats from RSpec to
AllocInfo. There's nothing interesting about it, but it should allow
more correct computations of Npus in hspace.

I also moved the AllocStats type alias from Cluster.hs to Types.hs
(just makes more sense).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoAdd a new type for holding allocation statistics
Iustin Pop [Wed, 25 Jan 2012 10:21:36 +0000 (11:21 +0100)]
Add a new type for holding allocation statistics

The current RSpec type doesn't cut it anymore (and it's used in other
places), so let's introduce a separate type for holding these
statistics.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoTrack 'normalised' used CPUs in Cluster.CStats
Iustin Pop [Tue, 24 Jan 2012 18:35:18 +0000 (19:35 +0100)]
Track 'normalised' used CPUs in Cluster.CStats

This will be used in hspace for proper tracking of used/pool/unavail
npus.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRe-indent code for readability
Iustin Pop [Tue, 24 Jan 2012 18:24:46 +0000 (19:24 +0100)]
Re-indent code for readability

Otherwise the docstrings for members take too many lines.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoRemove obsolete constant for vcpu_ratio
Iustin Pop [Tue, 24 Jan 2012 16:52:52 +0000 (17:52 +0100)]
Remove obsolete constant for vcpu_ratio

This is superseded now.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoChange meaning of the --max-cpu option
Iustin Pop [Tue, 24 Jan 2012 16:36:43 +0000 (17:36 +0100)]
Change meaning of the --max-cpu option

Instead of always overriding, we should update the nodes's maxcpu
setting only when the option is specified. Otherwise this will be read
from the cluster or from the Ganeti constant for non-live backends.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoTemporary fix for hspace's NPU calculation
Iustin Pop [Tue, 24 Jan 2012 16:19:31 +0000 (17:19 +0100)]
Temporary fix for hspace's NPU calculation

hspace computes the NPU (normalised CPU units) via the
(used|pool|avail) VCPU divided by the (currently cluster-wide) max
vcpu_ratio.

However, in the future we'll have variable vcpu_ratios, so we can't
the calculation as such. We temporarily change to computing the
average vcpu_ratio across the cluster, however long-term this needs to
be improved, as ∑ (Vi/Pi) ≠ ∑ Vi / ∑ Pi. In the meantime we implement
this patch to remove the use of a global mcpu variable.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoRemove Node.mCpu, replaced with the ipol vcpu_ratio
Iustin Pop [Tue, 24 Jan 2012 12:36:33 +0000 (13:36 +0100)]
Remove Node.mCpu, replaced with the ipol vcpu_ratio

Since the iPolicy contains the vcpu_ratio now, we can remove the
per-node mCpu value (which was the initial way to set this attribute).

Furthermore, we need to re-set the hiCpu value when changing the
policy.

And of course some associated unittest changes.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoAdd vcpu_ratio definition to the IPolicy type
Iustin Pop [Tue, 24 Jan 2012 11:12:24 +0000 (12:12 +0100)]
Add vcpu_ratio definition to the IPolicy type

This will be "naturally" read in the JSON-based backends (Luxi/Rapi),
and we add explicit code for handling it in the Text backend.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoExport the cluster ipolicy in iallocator
Iustin Pop [Mon, 23 Jan 2012 17:19:43 +0000 (18:19 +0100)]
Export the cluster ipolicy in iallocator

And remove a strange comment (the commit which introduced it,
d61df03e, doesn't explain it). Also document these new keys in
iallocator.rst.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix gnt-cluster modify check of passed options
Iustin Pop [Mon, 23 Jan 2012 17:08:50 +0000 (18:08 +0100)]
Fix gnt-cluster modify check of passed options

… and add the disk_templates/vcpu_ratio new entries. Since the default
ispecs_* are dicts, we can't compare them to none.

Additionally, add a QA test for this case (empty cluster modify).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd ipolicy_vcpu_ratio to command line
Iustin Pop [Mon, 23 Jan 2012 17:01:24 +0000 (18:01 +0100)]
Add ipolicy_vcpu_ratio to command line

This allows changing this value from the command line.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoAdd new ipolicy parameter vcpu_ratio
Iustin Pop [Mon, 23 Jan 2012 16:12:22 +0000 (17:12 +0100)]
Add new ipolicy parameter vcpu_ratio

This adds the new parameter and changes the validation routines to
handle IPOLICY_PARAMETER value as floats. Very very ugly code :(,
should be redone much cleaner.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoClarification on what objects.InstancePolicy is
Iustin Pop [Mon, 23 Jan 2012 14:43:21 +0000 (15:43 +0100)]
Clarification on what objects.InstancePolicy is

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoRename IPOLICY_PARAMETERS to IPOLICY_ISPECS
Iustin Pop [Mon, 23 Jan 2012 14:41:55 +0000 (15:41 +0100)]
Rename IPOLICY_PARAMETERS to IPOLICY_ISPECS

As opposed to the initial situation, the PARAMETERS will be more than
just the min/std/max ispecs. Let's rename it, before re-adding the
PARAMETERS constant with a different meaning.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoRemove two extraneous imports
Iustin Pop [Mon, 23 Jan 2012 14:37:45 +0000 (15:37 +0100)]
Remove two extraneous imports

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agognt-instance recreate-disks: Allow specifying new size
Michael Hanselmann [Fri, 20 Jan 2012 20:30:05 +0000 (21:30 +0100)]
gnt-instance recreate-disks: Allow specifying new size

With this change a new disk size can be specified when recreating disks.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoLUClusterSetParams: When ipolicy is updated warn for new violations
René Nussbaumer [Thu, 19 Jan 2012 13:48:07 +0000 (14:48 +0100)]
LUClusterSetParams: When ipolicy is updated warn for new violations

We need to loop over all node groups to apply their modification to make
sure that the new ipolicy actually affects their instance

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoLUGroupSetParams: Warn if the new ipolicy will violate some ipolicies
René Nussbaumer [Thu, 19 Jan 2012 12:29:45 +0000 (13:29 +0100)]
LUGroupSetParams: Warn if the new ipolicy will violate some ipolicies

If we modify the ipolicy we need to verify if any instance would now
violate the new ipolicy and warn about it. Any instance which already
violates the old one is ignored.

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agocmdlib: Add a helper function for a set of violating instances
René Nussbaumer [Thu, 19 Jan 2012 10:48:31 +0000 (11:48 +0100)]
cmdlib: Add a helper function for a set of violating instances

This helper will be used with the Op*SetParams and ipolicy verifications

Signed-off-by: René Nussbaumer <rn@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoRename ispecs-disk-templates to ipolicy-disk-templates
Iustin Pop [Mon, 23 Jan 2012 12:30:55 +0000 (13:30 +0100)]
Rename ispecs-disk-templates to ipolicy-disk-templates

The ispecs (min/std/max) are different from the other ipolicy
variables (disk templates for now, more in the future). As such, let's
rename the ispecs-disk-templates to ipolicy-disk-templates for
consistency with this logical difference.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agofmtjson: A tool to format JSON data
Michael Hanselmann [Mon, 23 Jan 2012 19:35:11 +0000 (20:35 +0100)]
fmtjson: A tool to format JSON data

While debugging an issue recently I had the pleasure of looking at a
long, single-line string of JSON data. This simple utility makes it
possible to format such data into a readable format.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoImplement rbd disk template
Constantinos Venetsanopoulos [Mon, 23 Jan 2012 12:27:39 +0000 (14:27 +0200)]
Implement rbd disk template

Introduce the rbd disk template, which handles provisioning and
management of instance disks as block devices mapped to rbd volumes
on a RADOS cluster.

The rbd template is of type DTS_EXT_MIRROR, meaning rbd-based
instances can failover or migrate to any node, without
primary-secondary limitations.

Use of this template requires the rbd kernel driver and rbd tool
on all VM-capable nodes, as well as an operational RADOS cluster,
accessible by them.

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoOpen VDI Port when spice_use_vdagent is enabled
Nikos Skalkotos [Mon, 16 Jan 2012 13:37:03 +0000 (15:37 +0200)]
Open VDI Port when spice_use_vdagent is enabled

To use the features spice agent provides, a dedicated virtio-serial
channel must be created in qemu-kvm. The communication between the agent
and the other spice components takes place over this channel.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Andrea Spadaccini <spadaccio@google.com>

12 years agoFix iallocator group objects
Iustin Pop [Mon, 23 Jan 2012 13:05:56 +0000 (14:05 +0100)]
Fix iallocator group objects

… this is yet another manual serialisation of objects :(

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoFix upgrading the cluster ipolicy
Iustin Pop [Mon, 23 Jan 2012 12:06:35 +0000 (13:06 +0100)]
Fix upgrading the cluster ipolicy

When running from git HEAD, just initialising the ipolicy when
non-existing is not good enough; we need to upgrade also missing keys,
etc.

This has downsides though, as it means we'll always 'rewrite' the
ipolicy, which might create silent data loss for unhandled changes
(e.g. key removal).

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: René Nussbaumer <rn@google.com>

12 years agoLUInstanceRecreateDisks: Actually acquire node resource locks
Michael Hanselmann [Fri, 20 Jan 2012 19:16:20 +0000 (20:16 +0100)]
LUInstanceRecreateDisks: Actually acquire node resource locks

Before this change an assertion would fail as the resource locks weren't
actually acquired.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agocmdlib._CheckInstanceState: Use instance name in error message
Michael Hanselmann [Fri, 20 Jan 2012 20:29:00 +0000 (21:29 +0100)]
cmdlib._CheckInstanceState: Use instance name in error message

The variable “instance” is an instance object, so to not get a
serialized version of the instance, the “name” attribute must be
used.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoUnbreak build after merge 2ea64524ce
Michael Hanselmann [Fri, 20 Jan 2012 18:50:11 +0000 (19:50 +0100)]
Unbreak build after merge 2ea64524ce

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agognt-cluster command: Add option to prepend node name to output
Michael Hanselmann [Fri, 20 Jan 2012 18:30:52 +0000 (19:30 +0100)]
gnt-cluster command: Add option to prepend node name to output

With this patch a new option (“-M”, like dsh) is added to “gnt-cluster
command” to prepend the node name to all output lines.

$ gnt-cluster command -M uname -a | grep Linux
node18.example.com: Linux node18.example.com [...]
node19.example.com: Linux node19.example.com [...]

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agocmdlib: Drop use of “len(…) != 0”
Michael Hanselmann [Fri, 20 Jan 2012 17:44:54 +0000 (18:44 +0100)]
cmdlib: Drop use of “len(…) != 0”

Found by Constantinos Venetsanopoulos.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoMerge branch 'devel-2.5'
Guido Trotter [Fri, 20 Jan 2012 17:36:51 +0000 (17:36 +0000)]
Merge branch 'devel-2.5'

* devel-2.5:
  Migrate: don't check for free memory on cleanup
  Revert "cli: Disable abbreviation matching for options"

Conflicts:
lib/cli.py
          - trivial
lib/cmdlib.py
          - merge new condition

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoMerge branch 'stable-2.5' into devel-2.5
Guido Trotter [Fri, 20 Jan 2012 16:25:00 +0000 (16:25 +0000)]
Merge branch 'stable-2.5' into devel-2.5

* stable-2.5:
  Migrate: don't check for free memory on cleanup

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agocli: Log fully quoted command line
Michael Hanselmann [Fri, 20 Jan 2012 16:58:12 +0000 (17:58 +0100)]
cli: Log fully quoted command line

Instead of logging the concatenated arguments, with this patch every
command will properly quote its full command line. This makes it easier
to just copy-and-paste commands from “commands.log” to re-run them. Also
remove a case of “x = a or b” for non-boolean values.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoMigrate: don't check for free memory on cleanup
Guido Trotter [Fri, 20 Jan 2012 14:30:56 +0000 (14:30 +0000)]
Migrate: don't check for free memory on cleanup

Cleanup just updates the config with the correct location of the
instance, or informs of its down status, but never starts it. As such
there's no point in checking for enough free memory. Actually this check
could prevent a perfectly safe cleanup operation if a node is busy.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoLUInstanceStartup: check for minimum memory
Guido Trotter [Mon, 9 Jan 2012 15:45:27 +0000 (15:45 +0000)]
LUInstanceStartup: check for minimum memory

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agohypervisors: start instances with max available mem
Guido Trotter [Mon, 9 Jan 2012 17:20:20 +0000 (17:20 +0000)]
hypervisors: start instances with max available mem

If we have available more than MINMEM but less than MAXMEM, start a new
instance with that value, rather than MAXMEM.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoInstance startup: lock primary node
Guido Trotter [Mon, 12 Dec 2011 12:36:50 +0000 (12:36 +0000)]
Instance startup: lock primary node

This will be used to balloon other instances on the node if needed.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agokvm: always start in paused state
Guido Trotter [Mon, 9 Jan 2012 16:57:41 +0000 (16:57 +0000)]
kvm: always start in paused state

Currently kvm starts in a frozen state if cpu pinning is set.
This is useful for other purposes as well (eg. ballooning before the
instance uses the memory), so we move the functionality out of the cpu
pinning code.

Note that before the "continue" command was executed in a finally after
trying to set the cpu pinning. This didn't help if anything else in the
function failed before that, though. As such we just move it out, as
anyway a failure above will cause an exception, so the user will know
their start command hasn't succeeded anyway.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

12 years agoRevert "cli: Disable abbreviation matching for options"
Michael Hanselmann [Thu, 19 Jan 2012 16:18:25 +0000 (17:18 +0100)]
Revert "cli: Disable abbreviation matching for options"

This reverts commit 232aab3f4f602a19f1226e85c3a3ecb245d60af4. We
shouldn't change the parsing of command line options in the middle of
the 2.5.x series.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoMerge branch 'devel-2.5'
Michael Hanselmann [Thu, 19 Jan 2012 12:55:06 +0000 (13:55 +0100)]
Merge branch 'devel-2.5'

* devel-2.5:
  Fix wrong option names in QA and cluster-merge
  Bump version to 2.5.0~rc5, update NEWS
  Add UnescapeAndSplit unittest for multi-escapes
  Fix a bug in command line option parsing code
  cli: Disable abbreviation matching for options
  Bump version to 2.5.0~rc5, update NEWS
  Add UnescapeAndSplit unittest for multi-escapes
  Fix a bug in command line option parsing code

Conflicts:
lib/cli.py: Trivial

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoMerge branch 'stable-2.5' into devel-2.5
Michael Hanselmann [Thu, 19 Jan 2012 12:45:57 +0000 (13:45 +0100)]
Merge branch 'stable-2.5' into devel-2.5

* stable-2.5:
  Bump version to 2.5.0~rc5, update NEWS
  Add UnescapeAndSplit unittest for multi-escapes
  Fix a bug in command line option parsing code

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoFix wrong option names in QA and cluster-merge
Michael Hanselmann [Wed, 18 Jan 2012 19:20:23 +0000 (20:20 +0100)]
Fix wrong option names in QA and cluster-merge

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoBump version to 2.5.0~rc5, update NEWS
Michael Hanselmann [Mon, 9 Jan 2012 16:27:10 +0000 (17:27 +0100)]
Bump version to 2.5.0~rc5, update NEWS

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

12 years agoAdd UnescapeAndSplit unittest for multi-escapes
Iustin Pop [Wed, 30 Nov 2011 09:33:52 +0000 (10:33 +0100)]
Add UnescapeAndSplit unittest for multi-escapes

This would have caught the bug in the first place. Argh,
hand-generated test cases!

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

12 years agoFix a bug in command line option parsing code
Nikos Skalkotos [Tue, 29 Nov 2011 12:30:46 +0000 (14:30 +0200)]
Fix a bug in command line option parsing code

Fix bug affecting command line options of "keyval" type. Although
escaping commands with \ is supported, it is is not applied to the
input recursively.

Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>