opcodes: Annotate the OP_RESULT of query operations
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Replace single- with double-quotes
In at least two cases "%s" is replaced with str(), too.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Small fixup to rapi docstrings
The fast that most classes don't override the @cvars means that directreferences to FillOpcode can fail (they do on my workstation, but noton buildbot?). Anyway, for safety, it's best to qualify the name.
Additionally a small typo is fixed....
Small update to LogicalUnit.ExpandNames docstring
… to make even more obvious what's the difference between a declaredlock level with an empty list of locks and no lock level.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Export ndparams in RAPI node query
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add ndp/* fields to group/node query
Also change the meaning of group ndparams to mean the actual (notcustom) nd params, and add custom_ndparams for the current meaning ofndparams.
Export ndparams in iallocator
Strangely, these were not exported at all before.
Add ipolicy parameter spindle_ratio
This will represent the instance (count) per spindle_count of theirnode(s). Debatable whether we need to add a per-instancespindle_weight.
Add new spindle_count node parameter
Currently this is not handled by Ganeti, just recorded.
Fix upgrading of ndparams
Currently, we only upgrade the ndparams if they are missingcompletely, which creates problems if we add any new parameters on analready-upgraded cluster.
Fix this by adding an UpgradeNDParams function.
Signed-off-by: Iustin Pop <iustin@google.com>...
Small fixes to objects.UpgradeDiskParams
Fix a typo, and cleanup the code a bit.
opcodes: Add result checks for OpBackup*
config: Add check for disk's “iv_name”
This check verifies “iv_name” of all instance disks. If one is wrong(which shouldn't happen in the first place), cluster verification willwarn:
“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:...
opcodes: Fix OP_RESULT for OobCommand
The result is a list of a list with elements with size of 2.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
cli: Handle negative numbers when parsing key-value assignments
To remove the last disk, or to add a NIC to the end, one can use theindex 1. This wouldn't work as intended as “” is a special prefix.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
utils.text: Add function to truncate string
The function adds an ellipse if the string was actually truncated. Alsostart using it in mcpu for result checks (where the message is alsoslightly changed to use a colon).
Fix breakage introduced by fa6dd6bb56
Forgot “enumerate”.
LUInstanceSetParams: Update disk's “iv_name”
When modifications are made, disks may not have the same index anymore.Updating all disks fixes this.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
ApplyContainerMods: Fix issues with indices
When adding an item the index given to the callback function would beincorrect under certain conditions. This patch also adds assertions andmore tests.
cmdlib: Remove some users of “iv_name”
ConfigWriter: Stop using “iv_name”
ConfigWriter.RenameInstance: Stop using iv_name, safer operation
Stop using the disk index encoded in “iv_name” when renaming an instance.
This patch also changes the code to operate on a copy of the instanceuntil the major changes have been applied. In the case of a failure we...
gnt-instance modify: Support new-style NIC/disk modifications
This patch adds support for adding/removing NICs/disks at arbitraryindices on the command line. To add a disk at a specified index, use“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”....
opcodes: Adding missing OP_RESULTs
mcpu: Make the op result exception more verbose
cmdlib: De-duplicate code in _GenerateDiskTemplate
There has been a lot of duplicated code in _GenerateDiskTemplate,and some cases of very similar, but not quite same duplicates. Thispatch merges them.
Generating a disk's “logical_id” attribute is done via a...
Add unittest for cmdlib._GenerateDiskTemplate
This is in preparation to de-duplicating significant chunks of code incmdlib._GenerateDiskTemplate.
rapi.testutils: Add exported functions to verify opcode input/result
These can be used by third-party code to verify mock code. Further workon mocks is forthcoming, so this is only a start.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
cmdlib: Fix “unpack non-sequence” error
The callback is expected to return a two-valued tuple.
Fix doc bug introduced in 12637df
Commit 12637df changed/generalised how we build fields in the sphinxextension, however it resulted in this uncaught-so-far result:
$ echo QUERY_FIELDS_GROUP | ./autotools/docpp<generator object BuildValuesDoc at 0x28fd370>...
QUERY_FIELDS_GROUP
ApplyContainerMods: Return changes from callbacks
… instead of passing the list of changes as a parameter.
LUInstanceSetParams: Convert to generic algorithm for NIC/disk changes
Unfortunately this got a bit messier than I intended, but then again itcleans up a lot of messy code with heaps of local variables(“this_nic_override”) and LU attributes (“nic_pnew”, “nic_pinst”). Most...
cmdlib: Add generic algorithm for modifying NICs/disks
While preparing this patch series I identified at least three differentimplementations of the algorithm for adding/removing/changingNICs/disks. These two functions and corresponding unittests provide a...
LUInstanceSetParams: Assertion on disk template/disk changes
Disk changes aren't allowed at the same time as a disk template change.
OpInstanceSetParams: Make two type checks public
They'll be used for tests in cmdlib.py.
OpInstanceSetParams: Accept more flexible NIC/disk modifications
Start accepting a new form of NIC/disk modifications for adding/removingarbitrary NICs/disks. Unlike before the index must always be given. Anindex of “-1” stands for the last item.
constants: Add constant for modifying existing NIC/disk
Until now it would not be possible to add/remove a NIC/disk in anarbitrary position. This constant will be used in the data structuresnecessary to add/remove arbitrary NICs/disks.
opcodes: Add comments to two parameter tests
These comments appear in the RAPI documentation.
Re-added constant mistakenly removed in 58f0ce16873
“INSTANCE_DOWN” is still being used.
rapi.client: Removing constants not needed anymore
They were not referenced through the whole code and were marked internalonly.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add stricter checks for OpInstanceSetParams.{nics,disks}
Add ht.TMaybeListOf type check
Replaces some uses of TOr(TNone, TListOf(…)).
Stricter check for OS modifications passed to OpClusterSetParams
Don't just check the first element of each item, but also make sure theOS name is a string and not empty.
LUInstanceSetParams: Allow no-op change of instance offline status
With this patch marking an instance already marked offline (or online)as offline/online again becomes a no-op. Also removed the unusedINSTANCE_UP variable.
OpInstanceSetParams: Merge {off,on}line_inst parameters
Instead of having two separate parameters, a single boolean parameter isused. Unfortunately we need a third state to say “no change”, so thevalue can be None, True or False (similar to other parameters). There...
ipolicy: Make the keys of the dict consistent
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
N+1: verify using minimum memory
Since instances can be started, failed over and migrated with less thantheir maximum memory N+1 will use the minimum memory for verification.
Note that this accounts only for the instances being moved beingresized, and not for the ones already on the node, as ganeti will not...
Use allow_runtime_chgs in cmdlib
- Report the attribute value in the hooks runtime variable.- Use it to abort migration if not enough memory is available on the target node.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cli/opcodes: add a --no-runtime-changes migrate opt
This will be used for now to avoid ballooning memory at live migration time.
Migrate/Failover: allow less-than-max-mem op
Failover works magically after we relax the check: the start operationwill just use less memory, if less memory is available.
For migrate we need to balloon down the source memory before themigration, if the target node doesn't have enough capacity....
Failover and Migrate: acquire node resource locks
cmdlib._CheckNodeFreeMemory: return free memory
doc/rapi.rst: Document ipolicy parameter
Fix gnt-cluster modify check of passed options
… and add the disk_templates/vcpu_ratio new entries. Since the defaultispecs_* are dicts, we can't compare them to none.
Additionally, add a QA test for this case (empty cluster modify).
Export the cluster ipolicy in iallocator
And remove a strange comment (the commit which introduced it,d61df03e, doesn't explain it). Also document these new keys iniallocator.rst.
Add ipolicy_vcpu_ratio to command line
This allows changing this value from the command line.
Add new ipolicy parameter vcpu_ratio
This adds the new parameter and changes the validation routines tohandle IPOLICY_PARAMETER value as floats. Very very ugly code :(,should be redone much cleaner.
Clarification on what objects.InstancePolicy is
Rename IPOLICY_PARAMETERS to IPOLICY_ISPECS
As opposed to the initial situation, the PARAMETERS will be more thanjust the min/std/max ispecs. Let's rename it, before re-adding thePARAMETERS constant with a different meaning.
gnt-instance recreate-disks: Allow specifying new size
With this change a new disk size can be specified when recreating disks.
LUClusterSetParams: When ipolicy is updated warn for new violations
We need to loop over all node groups to apply their modification to makesure that the new ipolicy actually affects their instance
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
LUGroupSetParams: Warn if the new ipolicy will violate some ipolicies
If we modify the ipolicy we need to verify if any instance would nowviolate the new ipolicy and warn about it. Any instance which alreadyviolates the old one is ignored.
Signed-off-by: René Nussbaumer <rn@google.com>...
cmdlib: Add a helper function for a set of violating instances
This helper will be used with the Op*SetParams and ipolicy verifications
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Rename ispecs-disk-templates to ipolicy-disk-templates
The ispecs (min/std/max) are different from the other ipolicyvariables (disk templates for now, more in the future). As such, let'srename the ispecs-disk-templates to ipolicy-disk-templates forconsistency with this logical difference....
Implement rbd disk template
Introduce the rbd disk template, which handles provisioning andmanagement of instance disks as block devices mapped to rbd volumeson a RADOS cluster.
The rbd template is of type DTS_EXT_MIRROR, meaning rbd-basedinstances can failover or migrate to any node, without...
Open VDI Port when spice_use_vdagent is enabled
To use the features spice agent provides, a dedicated virtio-serialchannel must be created in qemu-kvm. The communication between the agentand the other spice components takes place over this channel.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix iallocator group objects
… this is yet another manual serialisation of objects :(
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix upgrading the cluster ipolicy
When running from git HEAD, just initialising the ipolicy whennon-existing is not good enough; we need to upgrade also missing keys,etc.
This has downsides though, as it means we'll always 'rewrite' theipolicy, which might create silent data loss for unhandled changes...
LUInstanceRecreateDisks: Actually acquire node resource locks
Before this change an assertion would fail as the resource locks weren'tactually acquired.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
cmdlib._CheckInstanceState: Use instance name in error message
The variable “instance” is an instance object, so to not get aserialized version of the instance, the “name” attribute must beused.
Unbreak build after merge 2ea64524ce
gnt-cluster command: Add option to prepend node name to output
With this patch a new option (“-M”, like dsh) is added to “gnt-clustercommand” to prepend the node name to all output lines.
$ gnt-cluster command -M uname -a | grep Linuxnode18.example.com: Linux node18.example.com [...]...
cmdlib: Drop use of “len(…) != 0”
Found by Constantinos Venetsanopoulos.
Merge branch 'devel-2.5'
Conflicts: lib/cli.py - trivial lib/cmdlib.py - merge new condition
Merge branch 'stable-2.5' into devel-2.5
cli: Log fully quoted command line
Instead of logging the concatenated arguments, with this patch everycommand will properly quote its full command line. This makes it easierto just copy-and-paste commands from “commands.log” to re-run them. Alsoremove a case of “x = a or b” for non-boolean values....
Migrate: don't check for free memory on cleanup
Cleanup just updates the config with the correct location of theinstance, or informs of its down status, but never starts it. As suchthere's no point in checking for enough free memory. Actually this check...
LUInstanceStartup: check for minimum memory
hypervisors: start instances with max available mem
If we have available more than MINMEM but less than MAXMEM, start a newinstance with that value, rather than MAXMEM.
Instance startup: lock primary node
This will be used to balloon other instances on the node if needed.
kvm: always start in paused state
Currently kvm starts in a frozen state if cpu pinning is set.This is useful for other purposes as well (eg. ballooning before theinstance uses the memory), so we move the functionality out of the cpupinning code.
Note that before the "continue" command was executed in a finally after...
Revert "cli: Disable abbreviation matching for options"
This reverts commit 232aab3f4f602a19f1226e85c3a3ecb245d60af4. Weshouldn't change the parsing of command line options in the middle ofthe 2.5.x series.
Fix a bug in command line option parsing code
Fix bug affecting command line options of "keyval" type. Althoughescaping commands with \ is supported, it is is not applied to theinput recursively.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>...
cli: Disable abbreviation matching for options
Python's “optparse” module does option name prefix matching by default.Since this can lead to confusing behaviour, e.g. by specifying “--force”for a command which only has a “--force-multi” option, this patch...
gnt-instance modify -m|--runtime-memory
Implement memory ballooning in xen
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Implement memory ballooning in kvm
xen: add the "maxmem" parameter in instances config
This is set to the same amount as the memory for now, but will allowstarting instances with less memory than their maximum.
Implement memory ballooning in the fake hv
No-op memory ballooning in chroot and lxc
Currently they have no memory limits set.
Add instance_balloon_memory rpc
Hypervisor: allow ballooning of instance memory
This new hypervisor call sets the memory of an instance to a new value,through ballooning.
Accept both PUT and POST in noded
Currently, noded requires PUT, even though the semantics of the RPCcalls do not match a PUT. We change the code accept both PUT and POST,with the intention to remove the PUT support in a later version.
On the client side, the RPC code is changed to send POST requests....
gnt-instance: Adapt replace-disks to the new opcode
TLReplaceDisk: Add ipolicy checks
Further fixes to instance policy validation
As a followup from "Remove extraneous check in policy creation", thereare more places where we build an ipolicy, and then manually check forits validity. This is very bad style, as it duplicates theverification code across many places....
Add new disk_templates parameter to instance policy
This is a bit more complex patch, as it requires changing theassumption that all keys in the policy dict points to values that arethemselves dicts. Right now we introduce an assumption that anynon-dicts are lists, we'll see in the future if this holds or whether...
Remove extraneous check in policy creation
The values are already checked in CreateIPolicy, no need to manuallycheck them again.
Move the instance specs options to cli.py
Currently these are defined twice, instead of a single time incli.py. Also adds the new disk_templates option to the common block,even though it's not yet used.