Annotate disk params on instance_os_add
We call _OpenRealBD during the process and this needs disk parameters towork. This was reported by Constantinos.
The fix is very ugly though.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix a few style issue in hcheck
This fixes a couple of issue I've seen while fixing the List import:
- removes over-use of printf with putStr/show- wraps some lines- removes superfluous parentheses in constructs like 'IO (a)'- makes sure that in type signatures, the '->' are placed at the...
Fix old-style import
Commit 1213f9d6 (re)added an old-style import, which fails with recentcompilers.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Style fix in hcheck
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Style fixes in shelltests
Simplify stats printing
PrintGroupStats and printClusterStats had very similar codestructure. This patch abstracts those two into printStats, making ita little more generic. Also, printStats has its arguments documentedas requested in previous patch series....
Cleaner simulation of rebalance
This replaces the 'if .. then .. else ..' with a large block of code in'else' branch by separating this code into a separate function.
Small reordering and renaming in hcheck
Reordered functions and shortened one function name.
Move 'need rebalance' check from cluster check
As suggested by Rene, we should keep the check separate from printing.
Always print all variables in machine readable version
As discussed in cover letter of patch series introducing hcheck, inmachine readable version we should always print values of all variables.When hbal simulation was not run, FINAL values are equal to INIT values....
Print idx <-> uuid mappings in machine readable mode
As per Iustin Pop's suggestion, we are printing group mappingsfrom group idx (known only to htools) to group uuid, which identifiesthe group uniquely. This is required because group uuid and namemay contain characters such as '-' which are not valid parts of...
Human readable group name improved
Instead of printing group id in human readable version of hcheck,we now print group name.
Print info about ability to rebalance
As suggested during offline discussion, this patch adds a new bashvariable and description field in hcheck, which informs if therebalance was possible. This is to determine if there are splitinstances on the cluster, in which case rebalance is impossible....
Improve readability for hcheck-related changes
As suggested by Rene, I added comments to types for printKeysand printFinal. Also, realigned some lines in Hspace.
Annotate disks upon blockdev_shutdown
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Annotate disks on blockdev_remove
This annotates the disks for the blockdev_remove where it isappropriate. It leaves out 2 cases were we can't reliably annotate diskparameters due to lack of knowledge what we should annotate. Those casesaffects only lvs used for drbd, so it doesn't affect the bug reported by...
Annotate disk params on blockdev_getmirrorstatus_multi
This is also related to the bug reported by Constantinos,as we've only one getmirrorstatus_multi call in whole cmdlib, we justannotate them while we are building the disk list.
Signed-off-by: René Nussbaumer <rn@google.com>...
Annotate disk parameters on blockdev_getmirrorstatus
Not annotating them works for DRBD but not for RBD as reported byConstantinos.
Fix for man entry for htools - added hinfo
Fixes in man page for hinfo
Update to the install file - hinotify dependency
Hinotify was not marked as a dependency for hconfd.
Release Ganeti 2.6.0rc1
Shelltestrunner tests for hcheck
Simple tests for hcheck using shelltestrunner. Among other, we checkthat we can run hcheck on multi-group cluster.
Machine readable hcheck functionality
Machine readable output for hcheck.
Human readable hcheck functionality
Full hcheck functionality is implemented in this patch. Hcheck runsper-group checks, provides summary for the entire cluster and simulatesrebalance if required, the printing the modified metrics.In this patch, only the human-readable output is available....
Allowing rebalance to run silently
Part of hcheck's simulation required us to silently run hbal. For thatpurpose we expose iterateDepth and allow it to run silently.
Generalize functions used by both hspace and hcheck
As hspace and hcheck both use machine readable options, they requiresimilar functions - which are now generalized and moved to CLI andUtils.
Initial commit for introducting hcheck tool
Introduce infrastructure required to add Hcheck and build itsuccessfuly.
NEWS: Improve formatting, wording
Clarify some things, use ```` where appropriate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Prepare the update of the NEWS file
This updates the NEWS file for the upcoming rc1 release.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Extend burnin to do NIC changes
This would have caught our previous issue, since QA otherwise doesn'tdo add, remove or changes of NICs.
Signed-off-by: Iustin Pop <iustin@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix bug in instance net changes
_PrepareNicModification returns the invalid type, which triggers anassert resulting in a mysterious error:
Failure: command execution error:
Without any explanation. We fix this by removing the return value from_PrepareNicModification, and instead returning the expected type...
Update the hooks documentation
Also provide some extended unittests to catch those cases.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
QA: Fix modification of HV parameters via RAPI
This is basically another case of commit d488adb.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
bdev: Fix DRBD error reporting for syncer rate
The function is supposed to return a list, not a string. Without thispatch the error message ends up as “C, a, n, ', t, , c, h, a, n, g, e,…”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Fix corner case in quick-check generated tests
Thanks to Agata Murawska, a random seed that generates bad test caseswas found ("seed was 1768143307 2118231514, test size 74"). This isdue to the fact that some nodes were generated with tMem=0, fMem=0,which means that pMem was NaN and thus the node was not comparing...
Change default for DRBD barriers
Commit 8a69b3a8 changed the DRBD barriers disabled by default if“--disable-drbd-barriers” is used to “bfd”, where “d” stands fordisk-drain. DRBD's upstream updated their documentation1 tosay disk drains should always be used....
Fix inconsistent punctuation in rapi docs and opcodes
While reviewing the RAPI docs (for commit 003306f9), I realised thatthe punctuation used in RAPI is inconsistent, especially with regardsto definition lists.
This patch changes all such lists to not have punctuation, except for...
Export more information in RAPI group queries
Sadly we don't have a way to test that, for example, all(G|N|I)Q_CONFIG fields are exported in RAPI, so for now we don't testthis; but we should, in the future.
Signed-off-by: Iustin Pop <iustin@google.com>...
QA: Fix breakage on cluster initialization
The hypervisors are a list and need to be comma-separated.
Verify the options on diskparameters
This prevents from setting for example drbd options on the plain disktemplate.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Verify user supplied dicts against defaults
This verifies the user (especially in nested dicts) does notprovide a key which is not seen in the defaults dict for that dict.
QA: Test some instance parameters depending on hypervisor
For example, the “initrd_path” parameter is only available with xen-pvm,whereas “boot_order” is only available with xen-hvm.
QA: Verify hypervisors, make available through functions
Some parts of QA should only be run with certain hypervisors (e.g.testing instance HV parameters). To make that happen we always definethe cluster's hypervisors and provide a function to get the default...
Fix cfgupgrade unittests
Sorry, I broke the cfgupgrade unittests via 904910c4, since thatcommit added the requirement for the "instances" dict in theconfiguration.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Upgrade wrong iv_names at cfgupgrade time
Since Ganeti 2.6 does warn about disks with wrong iv_names, it wouldbe better to fix them when upgrading to 2.6.
Not sure if this is too brittle and whether we want to add moresafeguards on the contents of the config/instance/disks before...
query2: Add <, >, <=, >= comparison operators
These can be used, for example, to get jobs submitted after a certaintimestamp.
Fix creation of plain instances with --no-wait-for-sync
As reported on the devel mailing list by Christos Stavrakakis,creation of plain instances is broken when the --no-wait-for-sync flagis passed, because in that case WaitForSync is not called, hence...
Fixed typos, formatting, etc and added a comment
This patch only touches comments and man pages.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix daemon-util with non-root user models
Commit 4b42c3d6 broke non-root user mode since, while trying to do acleanup a move all local variable definitions to the start of thefunction; however, the plain_name var is only defined later, so thisactually doesn't work....
Bump version for beta 2 release
I've added the changes under beta1, not beta2, since it's easier tokeep it separate.
Fix epydoc raises L{...} usage
For epydoc raise statement, it already expects an exception which willbe hyperlinked; if passing an L{…} construct, it will instead breakthe formatting (the @raise will not be recognised, but merged into theprevious paragraph)....
Merge branch 'stable-2.5' into stable-2.6
Fix race condition in test for *FileID functions
In this test the “file ID” of a temporary file is compared against thefile ID gathered via an open file descriptor to the same file. Forreasons unknown to me utime(2) is called in-between to update theinode's a- and mtime. Depending on the file system's timestamp...
Move KVM keymap from GenerateRuntime to ExecuteRuntime
Per issue 243, "side-effects" are GenerateRuntime are bad as theyexecute only on the initial node of the instance. By moving thewrite-out of the keymap file to ExecuteRuntime, it will be done bothat start and at migrate time....
Add the keymap directory to the list of runtime KVM dirs
Commit 4f580fef added the keymap support, but missed that thisdirectory needs to be ensured/created at hypervisor init time.
Fix make -n use
Due to the way $(MAKE) behaves (it forces execution of the commandsthat contain it), use of $(MAKE) in a command line together withside-effects commands will break stuff. It's better to keep it eitheron a separate line, or if not possible, move said commands to a...
Enable hlint in lint and reorganise this target
Currently, the lint target does a sequential: pep8, pylint, pylint onthe QA sources. hlint is run only when explicitly requested.
This patch reorganises the lint target by:
- splitting the current lint target into separate pylint (slow),...
Improve error message for auto-promote/node modify
This has been reported internally 3-4 times already, and the currentversion (from 8b437a6e) is still not good enough, it seems.
Fix QA import of existing instances
Commit f346a7d9 broke the TestInstanceImport test in QA, since itdropped the "always generate" parameter for backup import. We add anoverride mac parameter so that, no matter what is defined in theconfig file, imported instances (can) have a different MAC....
Remove one obsolete hlint override
During the preparations for 2.6, we cleaned up the codebase to usenewer exceptions (e.g. commits 30d25dd8, 79ac58fa), so we can now dropthis override.
Fix prefix bug in Haskell ssconf implementation
This patch fixes a (brown-bag) bug in the ssconf implementation wherethe looked-for filenames were not prefixed with the ssconf fileprefix.
Additionally, a test for this is added (which fails without the fix)....
Move the ssconf file prefix constant to constants.py
This way, it can be reused in the Haskell code too.
QA: Add configuration options for static MAC address
In some QA environments static MAC addresses must be used.
QA: Ignore tags using regular expression
In some QA environments external scripts may add tags. With this patch aregular expression can be used to ignore them during QA runs.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Conflicts: Makefile.am (expected, from the parallel build fixes) qa/qa_config.py (due to cherry-pick, ignored, simply checkout out with --ours)...
Fix parallel build failures
This is the 2.5 version of the "fix build failures":
- man/%.gen could be left over even in case of failure, due to automake bug- make man/%.gen runs RUN_IN_TEMPDIR, so let's depend on it, since that target has the proper dependencies (create needed dirs)...
QA: Add default setting for tests
Commit 1010ec70b1 enabled all tests by default. In some scenarios wherethe configuration is static one may not want newly added tests to berun.
QA: add toggle for fewer common instance tests
Currently, the "common" instance tests (shutdown, rapi stoppedconsole, stopped modify, startup, rapi shutdown, rapi startup, list,info, modify, rapi modify , console, rapi console, reinstall, rename,rapi rename, rename and back, rapi rename and back, grow disk, reboot,...
QA: stop logging SSH arguments for each invocation
Currently, this is how the QA log looks like:
Command: ssh -oEscapeChar=none -oBatchMode=yes -l root -t -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes -oControlPath=/tmp/ganeti-qa-multiplexer.DMzkuH -oControlMaster=no node14 exit...
Switch QA to tty-less mode
This makes ssh generate much less noise when QA is run itself withouta tty, as otherwise we get thousands of lines of:
Pseudo-terminal will not be allocated because stdin is not a terminal.
in the QA logs (which are already big enough)....
Fix a type issue and bad logic in cluster verification
Commit 2e04d454 introduced the new offline state for the instancestate, but being a big monolithic patch it sneaked in something thatdoesn't make sense.
The checks for extra instances (either wrongly up or just unknown) are...
Make the xen-cmd test more readable and portable
doc/hooks: Clarify hook directory structure
I was not able to find where the “.d” suffix is documented, so thisclarification should make it more obvious.
Support xl as a xen interface command
xm and xl are mostly compatible, there is just a change in migrate,which for now we simple bridge by comparing the command string.
Also XEN_CMD is now set during configure time. This will then be fixedin Ganeti 2.7 when we make it an hvparams....
Add XEN_CMD to _autoconf.py
This is needed so we can specify the xen command upon configure time
Fix GenericCurlConfig when pycurl is linked against nss
Redhat links its pycurl against NSS/ so this check fails and thereforemakes ganeti-watcher unusable.
Even though this seems to fix the problem and works, further testing onthis might be needed.
Fix a tiny typo in a design doc
Reported by user ewoud, thanks!
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Document that the disk/hypervisor state is not used
This didn't get completed in time for 2.6, so we document that it'snot working as is. Well, it's not broken, just not used by anything.
Correct the watcher state files in the man page
The move to per-group state files forgot to update the man page ofganeti-watcher; let's fix this and add some more details on whatremoval means and when it's needed.
Re-wrap some lines in watcher code
These were using exactly 80 chars, and I like them smaller.
Add support for SIGHUP handling in Haskell daemons
This patch adds support for handling SIGHUP and reopening thestdout/stderr logfile in daemon mode (but not in foreground mode).
Rework logging setup for Haskell daemons
This fixes a missing functionality: closing of stdout/stderr whenforking daemons. Without this, starting the daemons from the commandline seems to work, but doing it from utils.RunCmd breaks, sincewithout closing the standard descriptors, RunCmd never finishes....
Adapt example init.d script to work on Fedora
Update the INSTALL doc for Fedora users
kvm-ifup: Use /bin/bash
As reported by Ben Beuchler, kvm-ifup uses bash-specific constructs anddoesn't work with Ubuntu's default shell. We use bash all over the placeanyway, so changing the shebang line is the easiest solution.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Merge branch 'devel-2.6' into stable-2.6
It seems Michael submitted the daemon-util patch on devel-2.6, and Ididn't realise it. It's way too early in the release cycle of 2.6 tohave diverging branches, so I'm merging devel-2.6 into stable-2.6 andthen will ff the devel branch to the merge point....
Version bump for 2.6.0~beta1
Finally!
Adjust cfgupgrade for new minor version
Also does some abstracting of the versions.
Add support to daemon-util for distributions without start-stop-daemon
This adds support to daemon-util for Red Hat based distributions thatdo not have a start-stop-daemon. If /sbin/start-stop-daemon is notavailable, daemon-util will source /etc/rc.d/init.d/functions....
Improve the check-news script
Allow all errors to be displayed, instead of aborting at the firstone, and don't show stacktraces anymore.
Fix NEWS file
Too many empty lines…
_MaybeParseUnit: The elements are dicts not arrays
Make it possible to reset vcpu/spindle ratio to default
Add man page documentation for cpu_mask hv parameter
This is adapted from the design doc.
Also fixes a typo in cmdlib.py.
Add checks from wrong hard-coded paths in manpages
This is trivial, but should prevent such things in the future.
Fix hardcoded paths in gnt-cluster.rst man page
Also reworks a bit the sentence and fixes a typo.
First round of NEWS file updates for 2.6
More will come, this is just what I took from the existing NEWSentries.
gnt-group: Adding vcpu/spindle ratio to ipolicy flags