htools: add support for listing spindles for nodes
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
htools: add spindle-related attribute to nodes
This adds the spindleCount, hiSpindles and instSpindles attributes.
The spindleCount is equivalent to spindle_count on the ganeti side (anode parameter). hiSpindles is the maximum instance-used spindles, and...
Add spindle ratio to htool's IPolicy
Just the usual data/type declarations, read/save in the text backend,etc. Also does a bit of unit-test cleanup (the way we buildipolicies).
hinfo: Adding man page
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hinfo: Reorganize the code to separate functions
Also rename some functions as they don't reflect their actions anymore
Two tiny fixed related to runtime functionality
Adds an assert and fixes a typo.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
hinfo: Adding basic skeleton based on hbal
It prints the information which hbal prints in a nice formatted manner and ofcourse without doing any actions. An example output looks like this:
$ hinfo -m xen.example.comLoaded 4 nodes, 2 instancesCluster has 1 node group(s)...
Add automated checking for non-escaped --
This checks to see if any man output has [em] in it, as that will notbe correctly show in ASCII. Unfortunately the check will depend onwhether the other man check is done, but I didn't want to introduceyet another short script for this (can be done, no issue)....
Fix man pages to not use unescaped --
I've seen that man pages, as generated by the version of pandoc weuse, show single dashes in option names instead of double ones (versus -). After bringing it up with upstream(http://groups.google.com/group/pandoc-discuss/browse_thread/thread/9c4589a4001d42f9/95ee8dae8932dc93),...
Add job result descriptions to RAPI documentation
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove a superfluous warning in LUNodeRemove
Since we run the post-hooks explicitly in the Exec() function (via_RunPostHook) after we removed the target node from the config, wewill get a:
WARNING Node 'node2', which is about to be removed, was not found in...
opcodes: Remove dependency on query module
There's no need to verify the field definitions on every query. They arestatic for all intents and purposes anyway and verified at module loadtime.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
http.server: Factorize request handling even more
This splits even more parts of the request handling code into a separateclass. Doing so allows us to reuse this part of the code for tests (e.g.mocks). Unlike before now the error handling can also be reused....
http.server: Move error message formatting to handler class
Like before this patch moves more functionality from the actual serverclass into a separate handler class. At the same time the function ischanged to return both content-type and body instead of relying on a...
noded: Shorter docstring for mlockall(2) executor
Split handling HTTP requests into separate class
Until now HTTP requests were handled in the same class as incomingconnections (http.server.HttpServer). With this change the requesthandling is delegated to a separate class which can be re-used in tests...
http.server: Factorize request handling
Remove deprecated “QueryLocks” LUXI request
This has been deprecated since Ganeti 2.4 and hasn't been used byGaneti's code since. I'm not aware of any external users.
Add result checks for OpTags*
opcodes: Annotate the OP_RESULT of query operations
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Replace single- with double-quotes
In at least two cases "%s" is replaced with str(), too.
Fix asyncnotifier unit tests
On my workstation, if I run the following two tests in this exactorder:
- TestSingleFileEventHandler.testError- TestSingleFileEventHandler.testReplace
the second test will fail, because there is no "unregister" ofprevious tests (and the poll() done by asyncore will fire on the...
Small fixup to rapi docstrings
The fast that most classes don't override the @cvars means that directreferences to FillOpcode can fail (they do on my workstation, but noton buildbot?). Anyway, for safety, it's best to qualify the name.
Additionally a small typo is fixed....
Small update to LogicalUnit.ExpandNames docstring
… to make even more obvious what's the difference between a declaredlock level with an empty list of locks and no lock level.
Export ndparams in RAPI node query
Add ndp/* fields to group/node query
Also change the meaning of group ndparams to mean the actual (notcustom) nd params, and add custom_ndparams for the current meaning ofndparams.
Export ndparams in iallocator
Strangely, these were not exported at all before.
Add ipolicy parameter spindle_ratio
This will represent the instance (count) per spindle_count of theirnode(s). Debatable whether we need to add a per-instancespindle_weight.
Add new spindle_count node parameter
Currently this is not handled by Ganeti, just recorded.
Fix upgrading of ndparams
Currently, we only upgrade the ndparams if they are missingcompletely, which creates problems if we add any new parameters on analready-upgraded cluster.
Fix this by adding an UpgradeNDParams function.
Signed-off-by: Iustin Pop <iustin@google.com>...
Small fixes to objects.UpgradeDiskParams
Fix a typo, and cleanup the code a bit.
opcodes: Add result checks for OpBackup*
Improve opcode result check test to have a whitelist
Only whitelisted opcodes may not have a result check.
config: Add check for disk's “iv_name”
This check verifies “iv_name” of all instance disks. If one is wrong(which shouldn't happen in the first place), cluster verification willwarn:
“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:...
opcodes: Fix OP_RESULT for OobCommand
The result is a list of a list with elements with size of 2.
cli: Handle negative numbers when parsing key-value assignments
To remove the last disk, or to add a NIC to the end, one can use theindex 1. This wouldn't work as intended as “” is a special prefix.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
utils.text: Add function to truncate string
The function adds an ellipse if the string was actually truncated. Alsostart using it in mcpu for result checks (where the message is alsoslightly changed to use a colon).
Fix breakage introduced by fa6dd6bb56
Forgot “enumerate”.
LUInstanceSetParams: Update disk's “iv_name”
When modifications are made, disks may not have the same index anymore.Updating all disks fixes this.
ApplyContainerMods: Fix issues with indices
When adding an item the index given to the callback function would beincorrect under certain conditions. This patch also adds assertions andmore tests.
cmdlib: Remove some users of “iv_name”
ConfigWriter: Stop using “iv_name”
ConfigWriter.RenameInstance: Stop using iv_name, safer operation
Stop using the disk index encoded in “iv_name” when renaming an instance.
This patch also changes the code to operate on a copy of the instanceuntil the major changes have been applied. In the case of a failure we...
gnt-instance modify: Support new-style NIC/disk modifications
This patch adds support for adding/removing NICs/disks at arbitraryindices on the command line. To add a disk at a specified index, use“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”....
opcodes: Adding missing OP_RESULTs
mcpu: Make the op result exception more verbose
cmdlib: De-duplicate code in _GenerateDiskTemplate
There has been a lot of duplicated code in _GenerateDiskTemplate,and some cases of very similar, but not quite same duplicates. Thispatch merges them.
Generating a disk's “logical_id” attribute is done via a...
Add unittest for cmdlib._GenerateDiskTemplate
This is in preparation to de-duplicating significant chunks of code incmdlib._GenerateDiskTemplate.
rapi.testutils: Add exported functions to verify opcode input/result
These can be used by third-party code to verify mock code. Further workon mocks is forthcoming, so this is only a start.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
cmdlib: Fix “unpack non-sequence” error
The callback is expected to return a two-valued tuple.
Fix doc bug introduced in 12637df
Commit 12637df changed/generalised how we build fields in the sphinxextension, however it resulted in this uncaught-so-far result:
$ echo QUERY_FIELDS_GROUP | ./autotools/docpp<generator object BuildValuesDoc at 0x28fd370>...
QUERY_FIELDS_GROUP
htools: allow rbd disk template
Notes: - it'd be nice if there was a way to automatically generate the DiskTemplate list instead of manually specifying it. After all we have C.diskTemplates - Of course this actually does nothing, as for other...
ApplyContainerMods: Return changes from callbacks
… instead of passing the list of changes as a parameter.
Use the ipolicy constants for key names
Sorry, I missed this when I wrote the code originally.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
LUInstanceSetParams: Convert to generic algorithm for NIC/disk changes
Unfortunately this got a bit messier than I intended, but then again itcleans up a lot of messy code with heaps of local variables(“this_nic_override”) and LU attributes (“nic_pnew”, “nic_pinst”). Most...
cmdlib: Add generic algorithm for modifying NICs/disks
While preparing this patch series I identified at least three differentimplementations of the algorithm for adding/removing/changingNICs/disks. These two functions and corresponding unittests provide a...
Fix unittest breakage after commit a2aadb34b
Thou shalt run “make check” before sending patches.
LUInstanceSetParams: Assertion on disk template/disk changes
Disk changes aren't allowed at the same time as a disk template change.
OpInstanceSetParams: Make two type checks public
They'll be used for tests in cmdlib.py.
OpInstanceSetParams: Accept more flexible NIC/disk modifications
Start accepting a new form of NIC/disk modifications for adding/removingarbitrary NICs/disks. Unlike before the index must always be given. Anindex of “-1” stands for the last item.
constants: Add constant for modifying existing NIC/disk
Until now it would not be possible to add/remove a NIC/disk in anarbitrary position. This constant will be used in the data structuresnecessary to add/remove arbitrary NICs/disks.
opcodes: Add comments to two parameter tests
These comments appear in the RAPI documentation.
rbd disk template documentation and manpages
Add documentation and modify manpages for the RBD disk template.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>...
Re-added constant mistakenly removed in 58f0ce16873
“INSTANCE_DOWN” is still being used.
rapi.client: Removing constants not needed anymore
They were not referenced through the whole code and were marked internalonly.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add stricter checks for OpInstanceSetParams.{nics,disks}
Add ht.TMaybeListOf type check
Replaces some uses of TOr(TNone, TListOf(…)).
Stricter check for OS modifications passed to OpClusterSetParams
Don't just check the first element of each item, but also make sure theOS name is a string and not empty.
LUInstanceSetParams: Allow no-op change of instance offline status
With this patch marking an instance already marked offline (or online)as offline/online again becomes a no-op. Also removed the unusedINSTANCE_UP variable.
OpInstanceSetParams: Merge {off,on}line_inst parameters
Instead of having two separate parameters, a single boolean parameter isused. Unfortunately we need a third state to say “no change”, so thevalue can be None, True or False (similar to other parameters). There...
ipolicy: Make the keys of the dict consistent
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
N+1: verify using minimum memory
Since instances can be started, failed over and migrated with less thantheir maximum memory N+1 will use the minimum memory for verification.
Note that this accounts only for the instances being moved beingresized, and not for the ones already on the node, as ganeti will not...
admin.rst: add information about maxmem/minmem
- Specify how maxmem and minmem are used at startup, failover and migrate- Add a paragraph about how to manually change the runtime memory, and what Ganeti will and won't do automatically as of version 2.6...
walkthrough.rst: updates for maxmem/minmem
- Fix two example outputs- Improve N+1 resolution section specifying how different memories affect N+1 calculation depending on the node
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix integer overflow in Node unittest
When using a really random "extra cpus" variable, this can overflow(especially on 32 bit) and thus the final value can be < 0, thus thetest will fail to fail.
It doesn't make sense to test with huge values, so let's just generate...
Use allow_runtime_chgs in cmdlib
- Report the attribute value in the hooks runtime variable.- Use it to abort migration if not enough memory is available on the target node.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cli/opcodes: add a --no-runtime-changes migrate opt
This will be used for now to avoid ballooning memory at live migration time.
Migrate/Failover: allow less-than-max-mem op
Failover works magically after we relax the check: the start operationwill just use less memory, if less memory is available.
For migrate we need to balloon down the source memory before themigration, if the target node doesn't have enough capacity....
Failover and Migrate: acquire node resource locks
cmdlib._CheckNodeFreeMemory: return free memory
doc/rapi.rst: Document ipolicy parameter
Add vcpu_ratio definition to the IPolicy type
This will be "naturally" read in the JSON-based backends (Luxi/Rapi),and we add explicit code for handling it in the Text backend.
Remove Node.mCpu, replaced with the ipol vcpu_ratio
Since the iPolicy contains the vcpu_ratio now, we can remove theper-node mCpu value (which was the initial way to set this attribute).
Furthermore, we need to re-set the hiCpu value when changing thepolicy....
Temporary fix for hspace's NPU calculation
hspace computes the NPU (normalised CPU units) via the(used|pool|avail) VCPU divided by the (currently cluster-wide) maxvcpu_ratio.
However, in the future we'll have variable vcpu_ratios, so we can'tthe calculation as such. We temporarily change to computing the...
Change meaning of the --max-cpu option
Instead of always overriding, we should update the nodes's maxcpusetting only when the option is specified. Otherwise this will be readfrom the cluster or from the Ganeti constant for non-live backends.
Remove obsolete constant for vcpu_ratio
This is superseded now.
Re-indent code for readability
Otherwise the docstrings for members take too many lines.
Track 'normalised' used CPUs in Cluster.CStats
This will be used in hspace for proper tracking of used/pool/unavailnpus.
Add a new type for holding allocation statistics
The current RSpec type doesn't cut it anymore (and it's used in otherplaces), so let's introduce a separate type for holding thesestatistics.
Switch over some hspace stats from RSpec to AllocInfo
This patch changes the allocation delta stats from RSpec toAllocInfo. There's nothing interesting about it, but it should allowmore correct computations of Npus in hspace.
I also moved the AllocStats type alias from Cluster.hs to Types.hs...
Expand the tiered alloc unittest to check allocation stats
This was missing before, so with the new metric (NCpus), it's a goodtime to test this.
Fix gnt-cluster modify check of passed options
… and add the disk_templates/vcpu_ratio new entries. Since the defaultispecs_* are dicts, we can't compare them to none.
Additionally, add a QA test for this case (empty cluster modify).
Export the cluster ipolicy in iallocator
And remove a strange comment (the commit which introduced it,d61df03e, doesn't explain it). Also document these new keys iniallocator.rst.
Add ipolicy_vcpu_ratio to command line
This allows changing this value from the command line.
Add new ipolicy parameter vcpu_ratio
This adds the new parameter and changes the validation routines tohandle IPOLICY_PARAMETER value as floats. Very very ugly code :(,should be redone much cleaner.
Clarification on what objects.InstancePolicy is
Rename IPOLICY_PARAMETERS to IPOLICY_ISPECS
As opposed to the initial situation, the PARAMETERS will be more thanjust the min/std/max ispecs. Let's rename it, before re-adding thePARAMETERS constant with a different meaning.
Remove two extraneous imports
gnt-instance recreate-disks: Allow specifying new size
With this change a new disk size can be specified when recreating disks.
LUClusterSetParams: When ipolicy is updated warn for new violations
We need to loop over all node groups to apply their modification to makesure that the new ipolicy actually affects their instance
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...