Annotate disk params on instance_os_add
We call _OpenRealBD during the process and this needs disk parameters towork. This was reported by Constantinos.
The fix is very ugly though.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Annotate disks upon blockdev_shutdown
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Annotate disks on blockdev_remove
This annotates the disks for the blockdev_remove where it isappropriate. It leaves out 2 cases were we can't reliably annotate diskparameters due to lack of knowledge what we should annotate. Those casesaffects only lvs used for drbd, so it doesn't affect the bug reported by...
Annotate disk params on blockdev_getmirrorstatus_multi
This is also related to the bug reported by Constantinos,as we've only one getmirrorstatus_multi call in whole cmdlib, we justannotate them while we are building the disk list.
Signed-off-by: René Nussbaumer <rn@google.com>...
Annotate disk parameters on blockdev_getmirrorstatus
Not annotating them works for DRBD but not for RBD as reported byConstantinos.
Fix bug in instance net changes
_PrepareNicModification returns the invalid type, which triggers anassert resulting in a mysterious error:
Failure: command execution error:
Without any explanation. We fix this by removing the return value from_PrepareNicModification, and instead returning the expected type...
bdev: Fix DRBD error reporting for syncer rate
The function is supposed to return a list, not a string. Without thispatch the error message ends up as “C, a, n, ', t, , c, h, a, n, g, e,…”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Fix inconsistent punctuation in rapi docs and opcodes
While reviewing the RAPI docs (for commit 003306f9), I realised thatthe punctuation used in RAPI is inconsistent, especially with regardsto definition lists.
This patch changes all such lists to not have punctuation, except for...
Export more information in RAPI group queries
Sadly we don't have a way to test that, for example, all(G|N|I)Q_CONFIG fields are exported in RAPI, so for now we don't testthis; but we should, in the future.
Signed-off-by: Iustin Pop <iustin@google.com>...
Verify the options on diskparameters
This prevents from setting for example drbd options on the plain disktemplate.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Verify user supplied dicts against defaults
This verifies the user (especially in nested dicts) does notprovide a key which is not seen in the defaults dict for that dict.
query2: Add <, >, <=, >= comparison operators
These can be used, for example, to get jobs submitted after a certaintimestamp.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix creation of plain instances with --no-wait-for-sync
As reported on the devel mailing list by Christos Stavrakakis,creation of plain instances is broken when the --no-wait-for-sync flagis passed, because in that case WaitForSync is not called, hence...
Fixed typos, formatting, etc and added a comment
This patch only touches comments and man pages.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix epydoc raises L{...} usage
For epydoc raise statement, it already expects an exception which willbe hyperlinked; if passing an L{…} construct, it will instead breakthe formatting (the @raise will not be recognised, but merged into theprevious paragraph)....
Merge branch 'stable-2.5' into stable-2.6
Move KVM keymap from GenerateRuntime to ExecuteRuntime
Per issue 243, "side-effects" are GenerateRuntime are bad as theyexecute only on the initial node of the instance. By moving thewrite-out of the keymap file to ExecuteRuntime, it will be done bothat start and at migrate time....
Add the keymap directory to the list of runtime KVM dirs
Commit 4f580fef added the keymap support, but missed that thisdirectory needs to be ensured/created at hypervisor init time.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve error message for auto-promote/node modify
This has been reported internally 3-4 times already, and the currentversion (from 8b437a6e) is still not good enough, it seems.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move the ssconf file prefix constant to constants.py
This way, it can be reused in the Haskell code too.
Fix a type issue and bad logic in cluster verification
Commit 2e04d454 introduced the new offline state for the instancestate, but being a big monolithic patch it sneaked in something thatdoesn't make sense.
The checks for extra instances (either wrongly up or just unknown) are...
Support xl as a xen interface command
xm and xl are mostly compatible, there is just a change in migrate,which for now we simple bridge by comparing the command string.
Also XEN_CMD is now set during configure time. This will then be fixedin Ganeti 2.7 when we make it an hvparams....
Fix GenericCurlConfig when pycurl is linked against nss
Redhat links its pycurl against NSS/ so this check fails and thereforemakes ganeti-watcher unusable.
Even though this seems to fix the problem and works, further testing onthis might be needed.
Document that the disk/hypervisor state is not used
This didn't get completed in time for 2.6, so we document that it'snot working as is. Well, it's not broken, just not used by anything.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Re-wrap some lines in watcher code
These were using exactly 80 chars, and I like them smaller.
_MaybeParseUnit: The elements are dicts not arrays
Make it possible to reset vcpu/spindle ratio to default
Add man page documentation for cpu_mask hv parameter
This is adapted from the design doc.
Also fixes a typo in cmdlib.py.
gnt-group: Adding vcpu/spindle ratio to ipolicy flags
gnt-cluster modify: Add ipolicy for spindle ratio opt
cli: Add ipolicy_spindle_ratio to CreateIPolicyFromOpts
gnt-instance info: Rework beparams listings
This patch reworks the beparams listing to the new style. However, dueto this process the naming of the fields with mixed casing changed tolowercase and is therefore not 100% compatible with the previous versions....
constants: Add FIXME regarding CPU pinning bitfield
Allow units in ipolicy disk/mem CLI changes
This makes "gnt-cluster modify --specs-mem-size max=16g" work. Thedownside (due to how we iterate) is that error messages are not veryclear:
$ gnt-cluster modify --specs-mem-size max=16a Failure: prerequisites not met for this operation:...
Move some ipolicy functions from objects to cli
These are only used in CLI programs to parse command line options andsuch they do not belong in object.py (I will change them more and Idon't want to add more code in objects.py).
Beautify a couple of error messages
Fix _ComputeNewInstanceViolations logic
This function did the opposite: was computing which old instanceviolated the specs but no longer do it now. new - old is the expectedbehaviour.
Beautify disk ipolicy violations in cluster-verify
Currently, we only get:
instance3: ['disk-size value 512 is not in range [1024, 1048576]'
which doesn't explain which disk we are talking about. This patchextends the verification functions to take an additional parameter...
LUInstanceCreate: Run rename script on instance import
If an instance is imported with a different name, network settings may have tobe changed. Since import scripts may not already to the right thing, we decidedto run the rename script. The same technique is already used for inter-cluster...
gnt-group add: Fix diskparam fill
This was a pretty non-obvious bug. A cluster looks sane aftergnt-cluster init, however on a daemon restart the diskparameters had thedefault filled in. The same applies to gnt-group add. This is due to thenature that UpgradeConfig() from NodeGroups did just populate them with...
gnt-group modify: Fix an update issue with diskparams
gnt-group: Add info command
cli: Make FormatParameterDict recursive
This is useful if you have nested parameter dicts like in diskparams.
query: Expose diskparamters through query
gnt-cluster info: Print and format disk parameters
Merge branch 'devel-2.5'
Merge commit 'v2.5.1' into devel-2.5
Remove some duplicate code in the Xen config writing
Should be straightforward; the only difference from before is that wewill remove the 'auto' file even during migration finalise.
Fix gnt-group --help display
Copy-paste mismatch :)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>(cherry picked from commit 36c70d4ddc508dd1ffdcc806d617d5100f4cb265)
Fix hardcoded Xen kernel path
We already have a ./configure-time variable for this, but it seems tobe actually unused.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>(cherry picked from commit 3c4afa2e93d9499ce39c1aed575dacb549d35083)...
Fix grow-disk handling of invalid units
The reason why grow-disk was doing:
$ gnt-instance grow-disk instance3 0 -64Unhandled Ganeti error: Invalid format
Is because it does it's own ParseUnit call, and doesn't transform thatinto a nicer message.
Accept both PUT and POST in noded
This is a partial cherry-pick from7530364ddbe949bc34fc26f25ba3f5d921beb021 on master:
Currently, noded requires PUT, even though the semantics of the RPCcalls do not match a PUT. We change the code accept both PUT and POST,...
Update synopsis for “gnt-cluster repair-disk-sizes”
Mention that instances can be passed on the CLI when “--help” is used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>(cherry picked from commit eb5ac108d146644200df98b9f90dae003dcea426)...
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initiallywrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force- otherwise refuses to proceed, but it still returns exit code 0...
apidoc: Fix some typos and errors introduced by my previous patches
bdev: Add a guard to verify disk parameters are set
LUGroup*: Fix inheritance of disk parameters
bdev: Take the params directly from disk without refill
Special case blockdev_find
Similiar to blockdev_create we sometimes do find on children. This fixesthose cases. However, this is not very nice.
Special case blockdev_create
This is due to the nature of bdev. We spread some logic into cmdlib anddeal for example with it's children recursively. This makes it hard toannotate the disk parameters in a generic way as we don't always dealwith the top most disk. But the disk parameters are depending on the top...
cmdlib: Adding annotation helper for special cases
cmdlib: Remove all diskparams calculations not required anymore
cmdlib: Adapt the rpc calls
The following (blockdev) RPC calls are not converted yet (as they arenot straight forward or need more research):
rpc: Adapt the callbacks to the new encoder type
We change the cmdlib side on the next patch.
rpc: Adding new encoders for annotating disk parameters
rpc: Adding helper to annotate disk params
config: Adding convenience method for disk parameters
objects: Add helper function to fill disk parameters
Add a default PATH variable to OS scripts env
In commit 896a03f6 I cleaned up the environment for OS scripts,however I think that was a bit too extreme - it breaks our owninstance-debootstrap hooks, because for example dpkg (called from thegrub script) requires PATH to be set....
Move hooks PATH environment variable to constants
Move the contents of the PATH environment variable for hooks toconstants, and use its value in the code and in the hooks documentation.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Fix exception re-raising in Python Luxi clients
Commit e687ec01 (present in 2.5 since the 2.5 beta 3) did consistencyfixes across the code-base. Unfortunately this was done without enoughchecks on the actual meaning of one of the fixes, which means error...
Allow clock skews in certificate verification
Currently we allow for up to NODE_MAX_CLOCK_SKEW time differencebetween nodes in some operations, but not everywhere: SSL certificateverification (import/export, both intra and inter-cluster) has a zerolimit (downwards), and a week upwards. This can cause even...
Correct capitalisation of two Luxi calls
Two Luxi calls have inconsistent an name/value mapping (in the Pythoncode):
- REQ_AUTOARCHIVE_JOBS versus AutoArchiveJobs (versus AutoarchiveJobs)- REQ_QUEUE_SET_DRAIN_FLAG versus SetDrainFlag (no Queue)
While these are only a consistency issue, let's fix them so that the...
Fix LVM volume listing with newer LVM
Per commit 0304f0e, newer LVM has extended the lv_attr field. However,that commit was incomplete as we examine this attribute in anotherplace in the code.
Thanks to user alperhome, the _LVSLINE_REGEX in lib/backend.py also...
Make setup-ssh behave more friendly
So while testing my previous patch I run setup-ssh manually withoutany arguments, and it did nothing (no complains, no messages,anything). That was very surprising, so let's make it behave better ifno nodes are passed....
Add more RAPI test utilities
This patch adds a mock RAPI client to test input values to methods. Allmethods either raise an exception if there was a problem or return None.Third-party code can use this to test their input values to the RAPIclient.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
rapi.testutils.FakeCurl: Add header support
With this patch headers are constructed from the PycURL optionsand passed to the mock implementation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Move _FakeCurl from tests/ganeti.rapi.client to ganeti.rapi.testutils
This is preparation for the mock system, where we need the same cURLmock.
Signed-off-by: René Nussbaumer <rn@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
errors: Add exception for RAPI testing utilities
This exception is raised to abort before actually sending a LUXI call(there is no LUXI server involved in the test). The testing utilitiescatch the exception to report a success (i.e. the code didn't throw...
luxi: Add list of all requests
This will be used in a unittest for RAPI testing utilities.
server.rapi: Factorize RAPI user loading
By moving the code loading RAPI users from a file into a standalonefunction it becomes easier to use the RAPI request handler intests.
RemoteApiHandler: Add support for mocked LUXI client
This will be used for providing a type-checking RAPI client for tests.
Add absolute grow-disk mode at CLI level
Add 'absolute' grow-disk mode at OpCode/LU level
This also improves the log messages for the (default) relative mode("by %s to %s").
Ensure that the grow disk amount is positive
The CLI currently doesn't permit this, but a direct job submissionwould allow it.
Log all external commands execution
This logs all external commands in normal (non-debug) mode. This willleak the DRBD secrets, but in any case we do log failed commands, soit's not a significant change.
Change CertificateError to subclass GanetiApiError
Signed-off-by: Chris Schrier <schrierc@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add opcode parameter descriptions
Fixes QA after commit cfdf561dade7.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Chris Schrier <schrierc@google.com>
Stop acquiring BGL for LUXI queries
Short description: This fixes an issue whereby masterd would becomeunresponsive on the LUXI socket, leading to client timeouts. While madeworse in 2.5, the underlying issue was already present in 2.4.
Longer description: Until now all LUXI queries would acquire the BGL...
Convert LUClusterConfigQuery to query2
The main intention of this patch is to make it possible to retrievecluster tags via query2. While at it I decided to convertLUClusterConfigQuery right away. Some of the values returned byLUClusterQuery are also included, but the conversion of LUClusterQuery...
Stop using locks in LUXI "QueryTags"
Also mark it as deprecated in NEWS as normal queries can be usedinstead.
Remove unused constants
These are not used anywhere in Python or Haskell.
Convert listing exports to query2
This solves one case where locks are acquired during LUXI queries.Pretty late into the transition I noticed that OpBackupQuery had a“use_locking” parameter for a long time, but didn't use it. Sincemost of the other changes were already and this allows exports to...
utils.algo: Use str.isdigit instead of regular expression
str.isdigit is about 4x faster than using a regular expression ("\d+").This is in the inner sorting code so speed matters.
$ python -m timeit -s 'import re; s = re.compile("^\d+$")' \'s.match(""); s.match("Hello World"); s.match("1234")'...
Include PycURL error code in GanetiApiError.
Signed-off-by: Chris Schrier <schrierc@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Drop objects.QueryRequest
It was only used in one place and wasn't really necessary.
gnt-os modify: Add "--submit" option
gnt-node: Add "--submit" and "--priority" to commands
gnt-instance: Add "--submit" and "--priority" to commands