Prepare NEWS file for Ganeti 2.6.0 rc3
Update NEWS regarding rc3 fixes and document changes in behaviour.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA updated to test instance removing when sec. is offline
Added test checking that it is possible to remove an instance with drbddisk when non-primary node is offline.
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Ignore offline node errors when removing disks
When we delete DRBD disks from some instance, we do not want to geterrors due to nodes other than that instance's primary being offline.
Allow instance disc activation with offline secondaries
Currently, this is not allowed, so one can't run a replace-disks; thisbreaks any non-invasive method of recovering the redundancy of theinstance if its disks are already stopped (but it still works if the...
RAPI regression beparams/memory fix
For compatibility with the old Ganeti version, we want to keep thebeparams/memory field around for another release. This patch fixes thisregression.
Fix redistribution of files w.r.t. offline nodes
Currently, _RedistributeAncillaryFiles computes two lists: the list ofonline nodes (for all files redistribution), and the list ofvm_capable nodes, for hypervisor-specific files. However, thevm_capable list includes offline nodes too, leading to warning...
Fix cluster verify error on master-ip-setup script
This error does not show up until we exceed the pool of mastercandidates and have nodes which are not master candidates.
The background is that we check for master-ip-setup script on mastercandidates and expect them not to be on the other nodes. However, we...
Add a Makefile rule to test unclean git status
This is trivial, but can be used easily from automated builds to checkthat the git tree is clean: files are not modified and untracked filesare not present.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Add test helpers to .gitignore
Commit 3096524e enabled building of test helpers, but forgot to addthem to .gitignore, resulting in "unclean" git status output.
Fix a typo in NEWS
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Release Ganeti 2.6.0rc2
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Prepare NEWS file for 2.6.0 rc2
We are about to release another version of Ganeti 2.6, so we prepare theNEWS file with the changes from rc1 to rc2.
Update time on Haskell binaries after build
This patch works around an issue in our build system. Since we don'tuse cabal or 'ghc -M', we don't track actual dependencies in ourMakefile; this in turn means that editing a file that only is used inthe main 'htools' binary will keep triggering 'ghc --make' for the...
hcheck: add two simple type aliases for readability
The same types are reused a couple of times, so let's add a couple oftype aliases for easier change later and readability.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
hcheck: rework output mode
Looking at the output of hcheck, in human readable mode, it looks likeit always starts with a blank line. This is not nice, so I wanted toredo this to start cleanly.
However, looking at the code, I realised that we need some internal...
hcheck: reword and fix typo in instance/offline msg
Add hlint warning for wrong use of 'length'
While removing split instances check from hcheck, I saw a wrong use of'length': this should only be used when one needs the actual length ofthe list, and not when one wants to check whether the list is empty or...
Remove can_rebalance functionality from hcheck
Since hbal can now run rebalances even in the presence of splitinstances, we can remove this check and always run the rebalance.
Fix building of hcheck man page
The code used a space, not a tab, in Makefile.am, which means itactually broke the variable declaring the man pages.
Furthermore, we add some blank lines in the man page, otherwise theparagraphs will be run-in together....
Fix tiny typo in htools.rst
Backtracking through git blame, it turns out than commit 49148d15changed "\fB20,102400,16384,4\fR" into "**B20,102400,16384,4**". Butof course that's a mistake, since the markup was \fB, not just \f.
Signed-off-by: Iustin Pop <iustin@google.com>...
Allow param `modify' during gnt-instance modify
With the new gnt-instance modify syntax for addition and removal ofdisks/NICs on arbitrary indexes, we hit an assertion if the userpasses `modify' as one of the disk's parameters. E.g::
gnt-instance modify --disk 2:modify,size=3G instance1...
Correct a wrong --help option for --simulate
This was not update when the alloc policy was added to the parameter.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add a shell test for hbal and split instances
This is not perfect, as we only test that hbal completes successfullyand that it show a score improvement, but it's better than nothing.
hbal: relax restrictions on split instances
The recent patch series on hcheck made me realise how much pain wehave due to hbal not being able to work (at all) with split instances.
This is sub-optimal; ideally, hbal would degrade its behaviour, butnot completely refuse to work. This patch does this, by marking any...
Add newline at the end of shelltest files
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add forgotten unittest changes for instance_os_add
The previous patch which fixed disk parameters didn't adapt theunittests so it lead to failing QA.
Annotate disk params on instance_os_add
We call _OpenRealBD during the process and this needs disk parameters towork. This was reported by Constantinos.
The fix is very ugly though.
Fix a few style issue in hcheck
This fixes a couple of issue I've seen while fixing the List import:
- removes over-use of printf with putStr/show- wraps some lines- removes superfluous parentheses in constructs like 'IO (a)'- makes sure that in type signatures, the '->' are placed at the...
Fix old-style import
Commit 1213f9d6 (re)added an old-style import, which fails with recentcompilers.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Style fix in hcheck
Style fixes in shelltests
Simplify stats printing
PrintGroupStats and printClusterStats had very similar codestructure. This patch abstracts those two into printStats, making ita little more generic. Also, printStats has its arguments documentedas requested in previous patch series....
Cleaner simulation of rebalance
This replaces the 'if .. then .. else ..' with a large block of code in'else' branch by separating this code into a separate function.
Small reordering and renaming in hcheck
Reordered functions and shortened one function name.
Move 'need rebalance' check from cluster check
As suggested by Rene, we should keep the check separate from printing.
Always print all variables in machine readable version
As discussed in cover letter of patch series introducing hcheck, inmachine readable version we should always print values of all variables.When hbal simulation was not run, FINAL values are equal to INIT values....
Print idx <-> uuid mappings in machine readable mode
As per Iustin Pop's suggestion, we are printing group mappingsfrom group idx (known only to htools) to group uuid, which identifiesthe group uniquely. This is required because group uuid and namemay contain characters such as '-' which are not valid parts of...
Human readable group name improved
Instead of printing group id in human readable version of hcheck,we now print group name.
Print info about ability to rebalance
As suggested during offline discussion, this patch adds a new bashvariable and description field in hcheck, which informs if therebalance was possible. This is to determine if there are splitinstances on the cluster, in which case rebalance is impossible....
Improve readability for hcheck-related changes
As suggested by Rene, I added comments to types for printKeysand printFinal. Also, realigned some lines in Hspace.
Annotate disks upon blockdev_shutdown
Annotate disks on blockdev_remove
This annotates the disks for the blockdev_remove where it isappropriate. It leaves out 2 cases were we can't reliably annotate diskparameters due to lack of knowledge what we should annotate. Those casesaffects only lvs used for drbd, so it doesn't affect the bug reported by...
Annotate disk params on blockdev_getmirrorstatus_multi
This is also related to the bug reported by Constantinos,as we've only one getmirrorstatus_multi call in whole cmdlib, we justannotate them while we are building the disk list.
Signed-off-by: René Nussbaumer <rn@google.com>...
Annotate disk parameters on blockdev_getmirrorstatus
Not annotating them works for DRBD but not for RBD as reported byConstantinos.
Fix for man entry for htools - added hinfo
Fixes in man page for hinfo
Update to the install file - hinotify dependency
Hinotify was not marked as a dependency for hconfd.
Release Ganeti 2.6.0rc1
Shelltestrunner tests for hcheck
Simple tests for hcheck using shelltestrunner. Among other, we checkthat we can run hcheck on multi-group cluster.
Machine readable hcheck functionality
Machine readable output for hcheck.
Human readable hcheck functionality
Full hcheck functionality is implemented in this patch. Hcheck runsper-group checks, provides summary for the entire cluster and simulatesrebalance if required, the printing the modified metrics.In this patch, only the human-readable output is available....
Allowing rebalance to run silently
Part of hcheck's simulation required us to silently run hbal. For thatpurpose we expose iterateDepth and allow it to run silently.
Generalize functions used by both hspace and hcheck
As hspace and hcheck both use machine readable options, they requiresimilar functions - which are now generalized and moved to CLI andUtils.
Initial commit for introducting hcheck tool
Introduce infrastructure required to add Hcheck and build itsuccessfuly.
NEWS: Improve formatting, wording
Clarify some things, use ```` where appropriate.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Prepare the update of the NEWS file
This updates the NEWS file for the upcoming rc1 release.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Extend burnin to do NIC changes
This would have caught our previous issue, since QA otherwise doesn'tdo add, remove or changes of NICs.
Signed-off-by: Iustin Pop <iustin@google.com>Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix bug in instance net changes
_PrepareNicModification returns the invalid type, which triggers anassert resulting in a mysterious error:
Failure: command execution error:
Without any explanation. We fix this by removing the return value from_PrepareNicModification, and instead returning the expected type...
Update the hooks documentation
Also provide some extended unittests to catch those cases.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
QA: Fix modification of HV parameters via RAPI
This is basically another case of commit d488adb.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
bdev: Fix DRBD error reporting for syncer rate
The function is supposed to return a list, not a string. Without thispatch the error message ends up as “C, a, n, ', t, , c, h, a, n, g, e,…”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Fix corner case in quick-check generated tests
Thanks to Agata Murawska, a random seed that generates bad test caseswas found ("seed was 1768143307 2118231514, test size 74"). This isdue to the fact that some nodes were generated with tMem=0, fMem=0,which means that pMem was NaN and thus the node was not comparing...
Change default for DRBD barriers
Commit 8a69b3a8 changed the DRBD barriers disabled by default if“--disable-drbd-barriers” is used to “bfd”, where “d” stands fordisk-drain. DRBD's upstream updated their documentation1 tosay disk drains should always be used....
Fix inconsistent punctuation in rapi docs and opcodes
While reviewing the RAPI docs (for commit 003306f9), I realised thatthe punctuation used in RAPI is inconsistent, especially with regardsto definition lists.
This patch changes all such lists to not have punctuation, except for...
Export more information in RAPI group queries
Sadly we don't have a way to test that, for example, all(G|N|I)Q_CONFIG fields are exported in RAPI, so for now we don't testthis; but we should, in the future.
QA: Fix breakage on cluster initialization
The hypervisors are a list and need to be comma-separated.
Verify the options on diskparameters
This prevents from setting for example drbd options on the plain disktemplate.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Verify user supplied dicts against defaults
This verifies the user (especially in nested dicts) does notprovide a key which is not seen in the defaults dict for that dict.
QA: Test some instance parameters depending on hypervisor
For example, the “initrd_path” parameter is only available with xen-pvm,whereas “boot_order” is only available with xen-hvm.
QA: Verify hypervisors, make available through functions
Some parts of QA should only be run with certain hypervisors (e.g.testing instance HV parameters). To make that happen we always definethe cluster's hypervisors and provide a function to get the default...
Fix cfgupgrade unittests
Sorry, I broke the cfgupgrade unittests via 904910c4, since thatcommit added the requirement for the "instances" dict in theconfiguration.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Upgrade wrong iv_names at cfgupgrade time
Since Ganeti 2.6 does warn about disks with wrong iv_names, it wouldbe better to fix them when upgrading to 2.6.
Not sure if this is too brittle and whether we want to add moresafeguards on the contents of the config/instance/disks before...
query2: Add <, >, <=, >= comparison operators
These can be used, for example, to get jobs submitted after a certaintimestamp.
Fix creation of plain instances with --no-wait-for-sync
As reported on the devel mailing list by Christos Stavrakakis,creation of plain instances is broken when the --no-wait-for-sync flagis passed, because in that case WaitForSync is not called, hence...
Fixed typos, formatting, etc and added a comment
This patch only touches comments and man pages.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix daemon-util with non-root user models
Commit 4b42c3d6 broke non-root user mode since, while trying to do acleanup a move all local variable definitions to the start of thefunction; however, the plain_name var is only defined later, so thisactually doesn't work....
Bump version for beta 2 release
I've added the changes under beta1, not beta2, since it's easier tokeep it separate.
Fix epydoc raises L{...} usage
For epydoc raise statement, it already expects an exception which willbe hyperlinked; if passing an L{…} construct, it will instead breakthe formatting (the @raise will not be recognised, but merged into theprevious paragraph)....
Merge branch 'stable-2.5' into stable-2.6
Fix race condition in test for *FileID functions
In this test the “file ID” of a temporary file is compared against thefile ID gathered via an open file descriptor to the same file. Forreasons unknown to me utime(2) is called in-between to update theinode's a- and mtime. Depending on the file system's timestamp...
Move KVM keymap from GenerateRuntime to ExecuteRuntime
Per issue 243, "side-effects" are GenerateRuntime are bad as theyexecute only on the initial node of the instance. By moving thewrite-out of the keymap file to ExecuteRuntime, it will be done bothat start and at migrate time....
Add the keymap directory to the list of runtime KVM dirs
Commit 4f580fef added the keymap support, but missed that thisdirectory needs to be ensured/created at hypervisor init time.
Fix make -n use
Due to the way $(MAKE) behaves (it forces execution of the commandsthat contain it), use of $(MAKE) in a command line together withside-effects commands will break stuff. It's better to keep it eitheron a separate line, or if not possible, move said commands to a...
Enable hlint in lint and reorganise this target
Currently, the lint target does a sequential: pep8, pylint, pylint onthe QA sources. hlint is run only when explicitly requested.
This patch reorganises the lint target by:
- splitting the current lint target into separate pylint (slow),...
Improve error message for auto-promote/node modify
This has been reported internally 3-4 times already, and the currentversion (from 8b437a6e) is still not good enough, it seems.
Fix QA import of existing instances
Commit f346a7d9 broke the TestInstanceImport test in QA, since itdropped the "always generate" parameter for backup import. We add anoverride mac parameter so that, no matter what is defined in theconfig file, imported instances (can) have a different MAC....
Remove one obsolete hlint override
During the preparations for 2.6, we cleaned up the codebase to usenewer exceptions (e.g. commits 30d25dd8, 79ac58fa), so we can now dropthis override.
Fix prefix bug in Haskell ssconf implementation
This patch fixes a (brown-bag) bug in the ssconf implementation wherethe looked-for filenames were not prefixed with the ssconf fileprefix.
Additionally, a test for this is added (which fails without the fix)....
Move the ssconf file prefix constant to constants.py
This way, it can be reused in the Haskell code too.
QA: Add configuration options for static MAC address
In some QA environments static MAC addresses must be used.
QA: Ignore tags using regular expression
In some QA environments external scripts may add tags. With this patch aregular expression can be used to ignore them during QA runs.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Conflicts: Makefile.am (expected, from the parallel build fixes) qa/qa_config.py (due to cherry-pick, ignored, simply checkout out with --ours)...
Fix parallel build failures
This is the 2.5 version of the "fix build failures":
- man/%.gen could be left over even in case of failure, due to automake bug- make man/%.gen runs RUN_IN_TEMPDIR, so let's depend on it, since that target has the proper dependencies (create needed dirs)...
QA: Add default setting for tests
Commit 1010ec70b1 enabled all tests by default. In some scenarios wherethe configuration is static one may not want newly added tests to berun.
QA: add toggle for fewer common instance tests
Currently, the "common" instance tests (shutdown, rapi stoppedconsole, stopped modify, startup, rapi shutdown, rapi startup, list,info, modify, rapi modify , console, rapi console, reinstall, rename,rapi rename, rename and back, rapi rename and back, grow disk, reboot,...
QA: stop logging SSH arguments for each invocation
Currently, this is how the QA log looks like:
Command: ssh -oEscapeChar=none -oBatchMode=yes -l root -t -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes -oControlPath=/tmp/ganeti-qa-multiplexer.DMzkuH -oControlMaster=no node14 exit...
Switch QA to tty-less mode
This makes ssh generate much less noise when QA is run itself withouta tty, as otherwise we get thousands of lines of:
Pseudo-terminal will not be allocated because stdin is not a terminal.
in the QA logs (which are already big enough)....
Fix a type issue and bad logic in cluster verification
Commit 2e04d454 introduced the new offline state for the instancestate, but being a big monolithic patch it sneaked in something thatdoesn't make sense.
The checks for extra instances (either wrongly up or just unknown) are...
Make the xen-cmd test more readable and portable
doc/hooks: Clarify hook directory structure
I was not able to find where the “.d” suffix is documented, so thisclarification should make it more obvious.