hinfo: Adjust the verbosity
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
hinfo: Gather and print group statistics
Explicitly remove the .x.in man pages on failure
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
More -- fixes
Because .DELETE_ON_ERROR doesn't work consistently (works when thetarget we're talking about is a final target, but not forintermediates), I missed a lot of man checks.
htools: Adding printTable helper
This helper abstract away all the unlines and unwords call you've to doafter you got a formatTable back.
Fixup a Node unittest
This fixes the prop_Node_rMem test, which cares about memory only;otherwise we could fail due to other problems (e.g. not enough disk).
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Allow overriding the target test count
This is not perfect, as the override applies identical target testvalues to both the 'fast' and 'slow' tests (making the slow tests takea long time), but it allows a quick override for manual runs.
Signed-off-by: Iustin Pop <iustin@google.com>...
Add two more unittests for Node add/remove
These just test that add/remove are idempotent. This is not perfect,as we use unsorted lists for some values (instead of sets), so whenusing non-empty nodes this would break (but for empty nodes, a1-element list is sorted, so it's fine)....
Use the spindles metric in cluster scores
This makes balancing use the new metric.
Track instance-used spindles in node set/add/remove
This simply tracks the instance-used spindles (using hard-coded '1'per instance), and additionally prevents additions in soft-mode whenwe go over the limit.
Note: there's an assymetry between addSec and removeSec (basically...
Shorten some function names
Since we use the functions always module-qualified, let's drop the`instance` prefix and use a shorter one.
Simplify logic in Node operations
This patch adds two helper functions and uses them for a somewhatsimplified logic in the add/remove pri/sec node functions.
htools: support spindles in simu backend
htools: support spindles in text load/save
If missing, we'll default to 1 spindle.
This also updates the text file format with the ipolicy, which wasforgotten.
htools: add support for listing spindles for nodes
htools: add spindle-related attribute to nodes
This adds the spindleCount, hiSpindles and instSpindles attributes.
The spindleCount is equivalent to spindle_count on the ganeti side (anode parameter). hiSpindles is the maximum instance-used spindles, and...
Add spindle ratio to htool's IPolicy
Just the usual data/type declarations, read/save in the text backend,etc. Also does a bit of unit-test cleanup (the way we buildipolicies).
hinfo: Adding man page
hinfo: Reorganize the code to separate functions
Also rename some functions as they don't reflect their actions anymore
Two tiny fixed related to runtime functionality
Adds an assert and fixes a typo.
hinfo: Adding basic skeleton based on hbal
It prints the information which hbal prints in a nice formatted manner and ofcourse without doing any actions. An example output looks like this:
$ hinfo -m xen.example.comLoaded 4 nodes, 2 instancesCluster has 1 node group(s)...
Add automated checking for non-escaped --
This checks to see if any man output has [em] in it, as that will notbe correctly show in ASCII. Unfortunately the check will depend onwhether the other man check is done, but I didn't want to introduceyet another short script for this (can be done, no issue)....
Fix man pages to not use unescaped --
I've seen that man pages, as generated by the version of pandoc weuse, show single dashes in option names instead of double ones (versus -). After bringing it up with upstream(http://groups.google.com/group/pandoc-discuss/browse_thread/thread/9c4589a4001d42f9/95ee8dae8932dc93),...
Add job result descriptions to RAPI documentation
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove a superfluous warning in LUNodeRemove
Since we run the post-hooks explicitly in the Exec() function (via_RunPostHook) after we removed the target node from the config, wewill get a:
WARNING Node 'node2', which is about to be removed, was not found in...
opcodes: Remove dependency on query module
There's no need to verify the field definitions on every query. They arestatic for all intents and purposes anyway and verified at module loadtime.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
http.server: Factorize request handling even more
This splits even more parts of the request handling code into a separateclass. Doing so allows us to reuse this part of the code for tests (e.g.mocks). Unlike before now the error handling can also be reused....
http.server: Move error message formatting to handler class
Like before this patch moves more functionality from the actual serverclass into a separate handler class. At the same time the function ischanged to return both content-type and body instead of relying on a...
noded: Shorter docstring for mlockall(2) executor
Split handling HTTP requests into separate class
Until now HTTP requests were handled in the same class as incomingconnections (http.server.HttpServer). With this change the requesthandling is delegated to a separate class which can be re-used in tests...
http.server: Factorize request handling
Remove deprecated “QueryLocks” LUXI request
This has been deprecated since Ganeti 2.4 and hasn't been used byGaneti's code since. I'm not aware of any external users.
Add result checks for OpTags*
opcodes: Annotate the OP_RESULT of query operations
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Replace single- with double-quotes
In at least two cases "%s" is replaced with str(), too.
Fix asyncnotifier unit tests
On my workstation, if I run the following two tests in this exactorder:
- TestSingleFileEventHandler.testError- TestSingleFileEventHandler.testReplace
the second test will fail, because there is no "unregister" ofprevious tests (and the poll() done by asyncore will fire on the...
Small fixup to rapi docstrings
The fast that most classes don't override the @cvars means that directreferences to FillOpcode can fail (they do on my workstation, but noton buildbot?). Anyway, for safety, it's best to qualify the name.
Additionally a small typo is fixed....
Small update to LogicalUnit.ExpandNames docstring
… to make even more obvious what's the difference between a declaredlock level with an empty list of locks and no lock level.
Export ndparams in RAPI node query
Add ndp/* fields to group/node query
Also change the meaning of group ndparams to mean the actual (notcustom) nd params, and add custom_ndparams for the current meaning ofndparams.
Export ndparams in iallocator
Strangely, these were not exported at all before.
Add ipolicy parameter spindle_ratio
This will represent the instance (count) per spindle_count of theirnode(s). Debatable whether we need to add a per-instancespindle_weight.
Add new spindle_count node parameter
Currently this is not handled by Ganeti, just recorded.
Fix upgrading of ndparams
Currently, we only upgrade the ndparams if they are missingcompletely, which creates problems if we add any new parameters on analready-upgraded cluster.
Fix this by adding an UpgradeNDParams function.
Small fixes to objects.UpgradeDiskParams
Fix a typo, and cleanup the code a bit.
opcodes: Add result checks for OpBackup*
Improve opcode result check test to have a whitelist
Only whitelisted opcodes may not have a result check.
config: Add check for disk's “iv_name”
This check verifies “iv_name” of all instance disks. If one is wrong(which shouldn't happen in the first place), cluster verification willwarn:
“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:...
opcodes: Fix OP_RESULT for OobCommand
The result is a list of a list with elements with size of 2.
cli: Handle negative numbers when parsing key-value assignments
To remove the last disk, or to add a NIC to the end, one can use theindex 1. This wouldn't work as intended as “” is a special prefix.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
utils.text: Add function to truncate string
The function adds an ellipse if the string was actually truncated. Alsostart using it in mcpu for result checks (where the message is alsoslightly changed to use a colon).
Fix breakage introduced by fa6dd6bb56
Forgot “enumerate”.
LUInstanceSetParams: Update disk's “iv_name”
When modifications are made, disks may not have the same index anymore.Updating all disks fixes this.
ApplyContainerMods: Fix issues with indices
When adding an item the index given to the callback function would beincorrect under certain conditions. This patch also adds assertions andmore tests.
cmdlib: Remove some users of “iv_name”
ConfigWriter: Stop using “iv_name”
ConfigWriter.RenameInstance: Stop using iv_name, safer operation
Stop using the disk index encoded in “iv_name” when renaming an instance.
This patch also changes the code to operate on a copy of the instanceuntil the major changes have been applied. In the case of a failure we...
gnt-instance modify: Support new-style NIC/disk modifications
This patch adds support for adding/removing NICs/disks at arbitraryindices on the command line. To add a disk at a specified index, use“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”....
opcodes: Adding missing OP_RESULTs
mcpu: Make the op result exception more verbose
cmdlib: De-duplicate code in _GenerateDiskTemplate
There has been a lot of duplicated code in _GenerateDiskTemplate,and some cases of very similar, but not quite same duplicates. Thispatch merges them.
Generating a disk's “logical_id” attribute is done via a...
Add unittest for cmdlib._GenerateDiskTemplate
This is in preparation to de-duplicating significant chunks of code incmdlib._GenerateDiskTemplate.
rapi.testutils: Add exported functions to verify opcode input/result
These can be used by third-party code to verify mock code. Further workon mocks is forthcoming, so this is only a start.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
cmdlib: Fix “unpack non-sequence” error
The callback is expected to return a two-valued tuple.
Fix doc bug introduced in 12637df
Commit 12637df changed/generalised how we build fields in the sphinxextension, however it resulted in this uncaught-so-far result:
$ echo QUERY_FIELDS_GROUP | ./autotools/docpp<generator object BuildValuesDoc at 0x28fd370>...
QUERY_FIELDS_GROUP
htools: allow rbd disk template
Notes: - it'd be nice if there was a way to automatically generate the DiskTemplate list instead of manually specifying it. After all we have C.diskTemplates - Of course this actually does nothing, as for other...
ApplyContainerMods: Return changes from callbacks
… instead of passing the list of changes as a parameter.
Use the ipolicy constants for key names
Sorry, I missed this when I wrote the code originally.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
LUInstanceSetParams: Convert to generic algorithm for NIC/disk changes
Unfortunately this got a bit messier than I intended, but then again itcleans up a lot of messy code with heaps of local variables(“this_nic_override”) and LU attributes (“nic_pnew”, “nic_pinst”). Most...
cmdlib: Add generic algorithm for modifying NICs/disks
While preparing this patch series I identified at least three differentimplementations of the algorithm for adding/removing/changingNICs/disks. These two functions and corresponding unittests provide a...
Fix unittest breakage after commit a2aadb34b
Thou shalt run “make check” before sending patches.
LUInstanceSetParams: Assertion on disk template/disk changes
Disk changes aren't allowed at the same time as a disk template change.
OpInstanceSetParams: Make two type checks public
They'll be used for tests in cmdlib.py.
OpInstanceSetParams: Accept more flexible NIC/disk modifications
Start accepting a new form of NIC/disk modifications for adding/removingarbitrary NICs/disks. Unlike before the index must always be given. Anindex of “-1” stands for the last item.
constants: Add constant for modifying existing NIC/disk
Until now it would not be possible to add/remove a NIC/disk in anarbitrary position. This constant will be used in the data structuresnecessary to add/remove arbitrary NICs/disks.
opcodes: Add comments to two parameter tests
These comments appear in the RAPI documentation.
rbd disk template documentation and manpages
Add documentation and modify manpages for the RBD disk template.
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>...
Re-added constant mistakenly removed in 58f0ce16873
“INSTANCE_DOWN” is still being used.
rapi.client: Removing constants not needed anymore
They were not referenced through the whole code and were marked internalonly.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add stricter checks for OpInstanceSetParams.{nics,disks}
Add ht.TMaybeListOf type check
Replaces some uses of TOr(TNone, TListOf(…)).
Stricter check for OS modifications passed to OpClusterSetParams
Don't just check the first element of each item, but also make sure theOS name is a string and not empty.
LUInstanceSetParams: Allow no-op change of instance offline status
With this patch marking an instance already marked offline (or online)as offline/online again becomes a no-op. Also removed the unusedINSTANCE_UP variable.
OpInstanceSetParams: Merge {off,on}line_inst parameters
Instead of having two separate parameters, a single boolean parameter isused. Unfortunately we need a third state to say “no change”, so thevalue can be None, True or False (similar to other parameters). There...
ipolicy: Make the keys of the dict consistent
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
N+1: verify using minimum memory
Since instances can be started, failed over and migrated with less thantheir maximum memory N+1 will use the minimum memory for verification.
Note that this accounts only for the instances being moved beingresized, and not for the ones already on the node, as ganeti will not...
admin.rst: add information about maxmem/minmem
- Specify how maxmem and minmem are used at startup, failover and migrate- Add a paragraph about how to manually change the runtime memory, and what Ganeti will and won't do automatically as of version 2.6...
walkthrough.rst: updates for maxmem/minmem
- Fix two example outputs- Improve N+1 resolution section specifying how different memories affect N+1 calculation depending on the node
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix integer overflow in Node unittest
When using a really random "extra cpus" variable, this can overflow(especially on 32 bit) and thus the final value can be < 0, thus thetest will fail to fail.
It doesn't make sense to test with huge values, so let's just generate...
Use allow_runtime_chgs in cmdlib
- Report the attribute value in the hooks runtime variable.- Use it to abort migration if not enough memory is available on the target node.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cli/opcodes: add a --no-runtime-changes migrate opt
This will be used for now to avoid ballooning memory at live migration time.
Migrate/Failover: allow less-than-max-mem op
Failover works magically after we relax the check: the start operationwill just use less memory, if less memory is available.
For migrate we need to balloon down the source memory before themigration, if the target node doesn't have enough capacity....
Failover and Migrate: acquire node resource locks
cmdlib._CheckNodeFreeMemory: return free memory
doc/rapi.rst: Document ipolicy parameter
Re-indent code for readability
Otherwise the docstrings for members take too many lines.
Track 'normalised' used CPUs in Cluster.CStats
This will be used in hspace for proper tracking of used/pool/unavailnpus.
Add a new type for holding allocation statistics
The current RSpec type doesn't cut it anymore (and it's used in otherplaces), so let's introduce a separate type for holding thesestatistics.
Switch over some hspace stats from RSpec to AllocInfo
This patch changes the allocation delta stats from RSpec toAllocInfo. There's nothing interesting about it, but it should allowmore correct computations of Npus in hspace.
I also moved the AllocStats type alias from Cluster.hs to Types.hs...
Expand the tiered alloc unittest to check allocation stats
This was missing before, so with the new metric (NCpus), it's a goodtime to test this.