Add job support to query2 via LUXI
This enables the use of filters through query2 when listing jobs.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
jqueue: Cache prepared field list in _JobChangesChecker
… instead of re-calculating it on every file change.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
NEWS: Deprecate LUXI calls replaced with query2
Adding the “luxi” namespace is necessary in “sphinx_ext”.
jqueue: Convert GetInfo to query2
This rather inefficient implementation (fields are evaluated on everycall to GetInfo) is not good for WaitForJobChanges and doesn't supportfilters, but that will be rectified in later patches.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
query: Add definitions for job fields
qlang.MakeFilter: Enable use of different name field
Jobs don't have a “name” field, so we must be able to controlthe field used for simple filters.
Merge cli.FormatTimestamp and utils.FormatTime
… to some degree at least. Unittests are included.
constants: Don't hardcode priorities for LOCK_ATTEMPTS_TIMEOUT
Also include unittest for LOCK_ATTEMPTS_TIMEOUT.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
jqueue._QueuedOpCode: Change a docstring
There was a typo and it's not necessary to repeat the class name.
locking: Remove unused OldStyleQueryLocks
No longer used after commit 090377807.
LUNodeAdd: Make the version call only dependend on DNS
Also move the version check into prereq to abort before alter clusterstate if the version mismatch.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RPC: Add a new client type for DNS only
This patch moves the “call_version” to a new RPC client definition andthen adds a new runner using the DNS resolver for getting the hostaddress.
The standard “BootstrapRunner”, where the call was before, tries to...
Update default instance kernel version
We switch from vmlinuz-2.6-… to vmlinuz-3-…. To do this nicely, wealso add a ./configure-time setting for the KVM instance kernel.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix escaping of percent signs in the shell lexer
Of course, we do have cases where we want to escape the percent signs,and our regexes were not fully correct for this case.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add a special lexer for sphinx/pygments
This will be used throughout our docs for better formatting exampleshell sessions, with custom markup for comments, user fixed input anduser variable input.
Fix hardcoded Xen kernel path
We already have a ./configure-time variable for this, but it seems tobe actually unused.
Typo fix: s/aditional/additional/
Trivial fix for a typo in message output of LUInstanceSetParams
Signed-off-by: Alexander Schreiber <als@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix exported constants
I "forgot" to run the unittests before commit :(
Export the AF_INET/AF_INET6 constants
These are needed to ensure that htools uses the same numeric values asPython.
By the way, I'm not sure what's the best option: importing socket intoconstants (this patch), or adding custom exports toautotools/convert-constants....
Update gnt-node evacuate man page/help text
Fix gnt-group --help display
Copy-paste mismatch :)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Rename spindle_usage to spindle_use on Ganeti side
Fix TLMigrateInstance use of IAllocator
hail now expects correctly that relocate_from is of equal length withthe number of required nodes (fixme: there's a lot of not welldocumented behaviour here… not nice for any other potentialIAllocators). As such, we need to pass just the instance's primary...
ipolicy: Keep track of spindle usage
iallocator: Add the spindle_usage to iallocator
This adapts the Ganeti side to export the spindle_usage
constants: Add new spindle_usage parameter to BE
Fix lint error in commit 035b33e2
Commit 035b33e2 forgot one blank space, and current pylint in DebianSid doesn't run on our code… so I didn't realise this until aftercommit, sorry.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix tempfile reset code & test on newer Python
Python 2.7.3 (rc status) and 3.2.3/3.3 (rc, respectively alpha status)have fixed http://bugs.python.org/issue12856 which we worked aroundourselves.
This means two things:
- we don't need to manually reset the module...
Merge remote-tracking branch 'origin/devel-2.5'
Conflicts: Makefile.am (trivial, test data files added on both branches) lib/opcodes.py (trivial, master renamed filter→qfilter, 2.5 fixed the type of the parameter)...
Two tiny fixed related to runtime functionality
Adds an assert and fixes a typo.
Remove a superfluous warning in LUNodeRemove
Since we run the post-hooks explicitly in the Exec() function (via_RunPostHook) after we removed the target node from the config, wewill get a:
WARNING Node 'node2', which is about to be removed, was not found in...
opcodes: Remove dependency on query module
There's no need to verify the field definitions on every query. They arestatic for all intents and purposes anyway and verified at module loadtime.
http.server: Factorize request handling even more
This splits even more parts of the request handling code into a separateclass. Doing so allows us to reuse this part of the code for tests (e.g.mocks). Unlike before now the error handling can also be reused....
http.server: Move error message formatting to handler class
Like before this patch moves more functionality from the actual serverclass into a separate handler class. At the same time the function ischanged to return both content-type and body instead of relying on a...
noded: Shorter docstring for mlockall(2) executor
Split handling HTTP requests into separate class
Until now HTTP requests were handled in the same class as incomingconnections (http.server.HttpServer). With this change the requesthandling is delegated to a separate class which can be re-used in tests...
http.server: Factorize request handling
Remove deprecated “QueryLocks” LUXI request
This has been deprecated since Ganeti 2.4 and hasn't been used byGaneti's code since. I'm not aware of any external users.
Add result checks for OpTags*
opcodes: Annotate the OP_RESULT of query operations
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Replace single- with double-quotes
In at least two cases "%s" is replaced with str(), too.
Small fixup to rapi docstrings
The fast that most classes don't override the @cvars means that directreferences to FillOpcode can fail (they do on my workstation, but noton buildbot?). Anyway, for safety, it's best to qualify the name.
Additionally a small typo is fixed....
Small update to LogicalUnit.ExpandNames docstring
… to make even more obvious what's the difference between a declaredlock level with an empty list of locks and no lock level.
Export ndparams in RAPI node query
Add ndp/* fields to group/node query
Also change the meaning of group ndparams to mean the actual (notcustom) nd params, and add custom_ndparams for the current meaning ofndparams.
Export ndparams in iallocator
Strangely, these were not exported at all before.
Add ipolicy parameter spindle_ratio
This will represent the instance (count) per spindle_count of theirnode(s). Debatable whether we need to add a per-instancespindle_weight.
Add new spindle_count node parameter
Currently this is not handled by Ganeti, just recorded.
Fix upgrading of ndparams
Currently, we only upgrade the ndparams if they are missingcompletely, which creates problems if we add any new parameters on analready-upgraded cluster.
Fix this by adding an UpgradeNDParams function.
Signed-off-by: Iustin Pop <iustin@google.com>...
Small fixes to objects.UpgradeDiskParams
Fix a typo, and cleanup the code a bit.
opcodes: Add result checks for OpBackup*
config: Add check for disk's “iv_name”
This check verifies “iv_name” of all instance disks. If one is wrong(which shouldn't happen in the first place), cluster verification willwarn:
“ERROR: cluster: Instance 'inst.example.com' has wrongly named disks:...
opcodes: Fix OP_RESULT for OobCommand
The result is a list of a list with elements with size of 2.
cli: Handle negative numbers when parsing key-value assignments
To remove the last disk, or to add a NIC to the end, one can use theindex 1. This wouldn't work as intended as “” is a special prefix.
utils.text: Add function to truncate string
The function adds an ellipse if the string was actually truncated. Alsostart using it in mcpu for result checks (where the message is alsoslightly changed to use a colon).
Fix breakage introduced by fa6dd6bb56
Forgot “enumerate”.
LUInstanceSetParams: Update disk's “iv_name”
When modifications are made, disks may not have the same index anymore.Updating all disks fixes this.
ApplyContainerMods: Fix issues with indices
When adding an item the index given to the callback function would beincorrect under certain conditions. This patch also adds assertions andmore tests.
cmdlib: Remove some users of “iv_name”
ConfigWriter: Stop using “iv_name”
ConfigWriter.RenameInstance: Stop using iv_name, safer operation
Stop using the disk index encoded in “iv_name” when renaming an instance.
This patch also changes the code to operate on a copy of the instanceuntil the major changes have been applied. In the case of a failure we...
gnt-instance modify: Support new-style NIC/disk modifications
This patch adds support for adding/removing NICs/disks at arbitraryindices on the command line. To add a disk at a specified index, use“--disk 3:size=16G”. To remove the second disk, use “--disk 2:remove”....
opcodes: Adding missing OP_RESULTs
mcpu: Make the op result exception more verbose
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initiallywrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force- otherwise refuses to proceed, but it still returns exit code 0...
cmdlib: De-duplicate code in _GenerateDiskTemplate
There has been a lot of duplicated code in _GenerateDiskTemplate,and some cases of very similar, but not quite same duplicates. Thispatch merges them.
Generating a disk's “logical_id” attribute is done via a...
Add unittest for cmdlib._GenerateDiskTemplate
This is in preparation to de-duplicating significant chunks of code incmdlib._GenerateDiskTemplate.
rapi.testutils: Add exported functions to verify opcode input/result
These can be used by third-party code to verify mock code. Further workon mocks is forthcoming, so this is only a start.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
cmdlib: Fix “unpack non-sequence” error
The callback is expected to return a two-valued tuple.
Fix doc bug introduced in 12637df
Commit 12637df changed/generalised how we build fields in the sphinxextension, however it resulted in this uncaught-so-far result:
$ echo QUERY_FIELDS_GROUP | ./autotools/docpp<generator object BuildValuesDoc at 0x28fd370>...
QUERY_FIELDS_GROUP
ApplyContainerMods: Return changes from callbacks
… instead of passing the list of changes as a parameter.
LUInstanceSetParams: Convert to generic algorithm for NIC/disk changes
Unfortunately this got a bit messier than I intended, but then again itcleans up a lot of messy code with heaps of local variables(“this_nic_override”) and LU attributes (“nic_pnew”, “nic_pinst”). Most...
cmdlib: Add generic algorithm for modifying NICs/disks
While preparing this patch series I identified at least three differentimplementations of the algorithm for adding/removing/changingNICs/disks. These two functions and corresponding unittests provide a...
LUInstanceSetParams: Assertion on disk template/disk changes
Disk changes aren't allowed at the same time as a disk template change.
OpInstanceSetParams: Make two type checks public
They'll be used for tests in cmdlib.py.
OpInstanceSetParams: Accept more flexible NIC/disk modifications
Start accepting a new form of NIC/disk modifications for adding/removingarbitrary NICs/disks. Unlike before the index must always be given. Anindex of “-1” stands for the last item.
constants: Add constant for modifying existing NIC/disk
Until now it would not be possible to add/remove a NIC/disk in anarbitrary position. This constant will be used in the data structuresnecessary to add/remove arbitrary NICs/disks.
opcodes: Add comments to two parameter tests
These comments appear in the RAPI documentation.
Accept both PUT and POST in noded
This is a partial cherry-pick from7530364ddbe949bc34fc26f25ba3f5d921beb021 on master:
Currently, noded requires PUT, even though the semantics of the RPCcalls do not match a PUT. We change the code accept both PUT and POST,...
Merge branch 'stable-2.5' into devel-2.5
Re-added constant mistakenly removed in 58f0ce16873
“INSTANCE_DOWN” is still being used.
rapi.client: Removing constants not needed anymore
They were not referenced through the whole code and were marked internalonly.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add stricter checks for OpInstanceSetParams.{nics,disks}
Add ht.TMaybeListOf type check
Replaces some uses of TOr(TNone, TListOf(…)).
Stricter check for OS modifications passed to OpClusterSetParams
Don't just check the first element of each item, but also make sure theOS name is a string and not empty.
LUInstanceSetParams: Allow no-op change of instance offline status
With this patch marking an instance already marked offline (or online)as offline/online again becomes a no-op. Also removed the unusedINSTANCE_UP variable.
Fix type check for OpQuery.filter
Just using ht.TListOf as a type check doesn't work correctly. Thefunction must be called with the expected item type. In this specificcase TListOf was always called with the filter as a value, and theresult of that call evaluated to truth. Since filters can be quite...
OpInstanceSetParams: Merge {off,on}line_inst parameters
Instead of having two separate parameters, a single boolean parameter isused. Unfortunately we need a third state to say “no change”, so thevalue can be None, True or False (similar to other parameters). There...
ipolicy: Make the keys of the dict consistent
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
N+1: verify using minimum memory
Since instances can be started, failed over and migrated with less thantheir maximum memory N+1 will use the minimum memory for verification.
Note that this accounts only for the instances being moved beingresized, and not for the ones already on the node, as ganeti will not...
Fix explanation of gnt-node evacuate --primaries-only
Furthermore, correct the --help display on evacuate.
Use allow_runtime_chgs in cmdlib
- Report the attribute value in the hooks runtime variable.- Use it to abort migration if not enough memory is available on the target node.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cli/opcodes: add a --no-runtime-changes migrate opt
This will be used for now to avoid ballooning memory at live migration time.
Migrate/Failover: allow less-than-max-mem op
Failover works magically after we relax the check: the start operationwill just use less memory, if less memory is available.
For migrate we need to balloon down the source memory before themigration, if the target node doesn't have enough capacity....
Failover and Migrate: acquire node resource locks
cmdlib._CheckNodeFreeMemory: return free memory
doc/rapi.rst: Document ipolicy parameter
Fix gnt-cluster modify check of passed options
… and add the disk_templates/vcpu_ratio new entries. Since the defaultispecs_* are dicts, we can't compare them to none.
Additionally, add a QA test for this case (empty cluster modify).
Export the cluster ipolicy in iallocator
And remove a strange comment (the commit which introduced it,d61df03e, doesn't explain it). Also document these new keys iniallocator.rst.
Add ipolicy_vcpu_ratio to command line
This allows changing this value from the command line.