QA: Ignore tags using regular expression
In some QA environments external scripts may add tags. With this patch aregular expression can be used to ignore them during QA runs.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Merge branch 'stable-2.5' into stable-2.6
Conflicts: Makefile.am (expected, from the parallel build fixes) qa/qa_config.py (due to cherry-pick, ignored, simply checkout out with --ours)...
Fix parallel build failures
This is the 2.5 version of the "fix build failures":
- man/%.gen could be left over even in case of failure, due to automake bug- make man/%.gen runs RUN_IN_TEMPDIR, so let's depend on it, since that target has the proper dependencies (create needed dirs)...
QA: Add default setting for tests
Commit 1010ec70b1 enabled all tests by default. In some scenarios wherethe configuration is static one may not want newly added tests to berun.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
QA: add toggle for fewer common instance tests
Currently, the "common" instance tests (shutdown, rapi stoppedconsole, stopped modify, startup, rapi shutdown, rapi startup, list,info, modify, rapi modify , console, rapi console, reinstall, rename,rapi rename, rename and back, rapi rename and back, grow disk, reboot,...
QA: stop logging SSH arguments for each invocation
Currently, this is how the QA log looks like:
Command: ssh -oEscapeChar=none -oBatchMode=yes -l root -t -oStrictHostKeyChecking=yes -oClearAllForwardings=yes -oForwardAgent=yes -oControlPath=/tmp/ganeti-qa-multiplexer.DMzkuH -oControlMaster=no node14 exit...
Switch QA to tty-less mode
This makes ssh generate much less noise when QA is run itself withouta tty, as otherwise we get thousands of lines of:
Pseudo-terminal will not be allocated because stdin is not a terminal.
in the QA logs (which are already big enough)....
Fix a type issue and bad logic in cluster verification
Commit 2e04d454 introduced the new offline state for the instancestate, but being a big monolithic patch it sneaked in something thatdoesn't make sense.
The checks for extra instances (either wrongly up or just unknown) are...
Make the xen-cmd test more readable and portable
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
doc/hooks: Clarify hook directory structure
I was not able to find where the “.d” suffix is documented, so thisclarification should make it more obvious.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Support xl as a xen interface command
xm and xl are mostly compatible, there is just a change in migrate,which for now we simple bridge by comparing the command string.
Also XEN_CMD is now set during configure time. This will then be fixedin Ganeti 2.7 when we make it an hvparams....
Add XEN_CMD to _autoconf.py
This is needed so we can specify the xen command upon configure time
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix GenericCurlConfig when pycurl is linked against nss
Redhat links its pycurl against NSS/ so this check fails and thereforemakes ganeti-watcher unusable.
Even though this seems to fix the problem and works, further testing onthis might be needed.
Signed-off-by: René Nussbaumer <rn@google.com>...
Fix a tiny typo in a design doc
Reported by user ewoud, thanks!
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Document that the disk/hypervisor state is not used
This didn't get completed in time for 2.6, so we document that it'snot working as is. Well, it's not broken, just not used by anything.
Correct the watcher state files in the man page
The move to per-group state files forgot to update the man page ofganeti-watcher; let's fix this and add some more details on whatremoval means and when it's needed.
Signed-off-by: Iustin Pop <iustin@google.com>...
Re-wrap some lines in watcher code
These were using exactly 80 chars, and I like them smaller.
Add support for SIGHUP handling in Haskell daemons
This patch adds support for handling SIGHUP and reopening thestdout/stderr logfile in daemon mode (but not in foreground mode).
Rework logging setup for Haskell daemons
This fixes a missing functionality: closing of stdout/stderr whenforking daemons. Without this, starting the daemons from the commandline seems to work, but doing it from utils.RunCmd breaks, sincewithout closing the standard descriptors, RunCmd never finishes....
Adapt example init.d script to work on Fedora
Update the INSTALL doc for Fedora users
kvm-ifup: Use /bin/bash
As reported by Ben Beuchler, kvm-ifup uses bash-specific constructs anddoesn't work with Ubuntu's default shell. We use bash all over the placeanyway, so changing the shebang line is the easiest solution.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Merge branch 'devel-2.6' into stable-2.6
It seems Michael submitted the daemon-util patch on devel-2.6, and Ididn't realise it. It's way too early in the release cycle of 2.6 tohave diverging branches, so I'm merging devel-2.6 into stable-2.6 andthen will ff the devel branch to the merge point....
Version bump for 2.6.0~beta1
Finally!
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adjust cfgupgrade for new minor version
Also does some abstracting of the versions.
Add support to daemon-util for distributions without start-stop-daemon
This adds support to daemon-util for Red Hat based distributions thatdo not have a start-stop-daemon. If /sbin/start-stop-daemon is notavailable, daemon-util will source /etc/rc.d/init.d/functions....
Improve the check-news script
Allow all errors to be displayed, instead of aborting at the firstone, and don't show stacktraces anymore.
Fix NEWS file
Too many empty lines…
_MaybeParseUnit: The elements are dicts not arrays
Make it possible to reset vcpu/spindle ratio to default
Add man page documentation for cpu_mask hv parameter
This is adapted from the design doc.
Also fixes a typo in cmdlib.py.
Add checks from wrong hard-coded paths in manpages
This is trivial, but should prevent such things in the future.
Fix hardcoded paths in gnt-cluster.rst man page
Also reworks a bit the sentence and fixes a typo.
First round of NEWS file updates for 2.6
More will come, this is just what I took from the existing NEWSentries.
gnt-group: Adding vcpu/spindle ratio to ipolicy flags
gnt-cluster modify: Add ipolicy for spindle ratio opt
cli: Add ipolicy_spindle_ratio to CreateIPolicyFromOpts
gnt-instance info: Rework beparams listings
This patch reworks the beparams listing to the new style. However, dueto this process the naming of the fields with mixed casing changed tolowercase and is therefore not 100% compatible with the previous versions....
Fix a tiny typo
Was showing in the man page as --diskN, without a space.
constants: Add FIXME regarding CPU pinning bitfield
Allow units in ipolicy disk/mem CLI changes
This makes "gnt-cluster modify --specs-mem-size max=16g" work. Thedownside (due to how we iterate) is that error messages are not veryclear:
$ gnt-cluster modify --specs-mem-size max=16a Failure: prerequisites not met for this operation:...
Move some ipolicy functions from objects to cli
These are only used in CLI programs to parse command line options andsuch they do not belong in object.py (I will change them more and Idon't want to add more code in objects.py).
Beautify a couple of error messages
Fix _ComputeNewInstanceViolations logic
This function did the opposite: was computing which old instanceviolated the specs but no longer do it now. new - old is the expectedbehaviour.
Beautify disk ipolicy violations in cluster-verify
Currently, we only get:
instance3: ['disk-size value 512 is not in range [1024, 1048576]'
which doesn't explain which disk we are talking about. This patchextends the verification functions to take an additional parameter...
qa_utils: Factorize code for getting entity name
This reduces code duplication and provides better error messages ifsomething is incorrect while getting a node or instance name.
NEWS: Style fixes
- Quote where appropriate- Small wording changes
qa_utils: Accept string as instance for check
Some tests don't have access to an instance dictionary.
QA: Cleanup after commit 5fa0375e4a
Some tests had wrong assumptions on the instance status and someneeded explicit start or stop commands.
LUInstanceCreate: Run rename script on instance import
If an instance is imported with a different name, network settings may have tobe changed. Since import scripts may not already to the right thing, we decidedto run the rename script. The same technique is already used for inter-cluster...
ganeti.query_unittest: Adding testcase for diskparams
gnt-group add: Fix diskparam fill
This was a pretty non-obvious bug. A cluster looks sane aftergnt-cluster init, however on a daemon restart the diskparameters had thedefault filled in. The same applies to gnt-group add. This is due to thenature that UpgradeConfig() from NodeGroups did just populate them with...
gnt-group modify: Fix an update issue with diskparams
Make x509 unittest testClockSkew a bit less flaky
Since the tested function actually uses time.time(), it cannot be makefully stable, but 1 second is very dangerous; let's just test SKEW * 2and higher since that should be good (if the delta between _GenCert...
gnt-group: Add info command
cli: Make FormatParameterDict recursive
This is useful if you have nested parameter dicts like in diskparams.
query: Expose diskparamters through query
gnt-cluster info: Print and format disk parameters
Merge branch 'devel-2.5'
Merge commit 'v2.5.1' into devel-2.5
Remove some duplicate code in the Xen config writing
Should be straightforward; the only difference from before is that wewill remove the 'auto' file even during migration finalise.
Update NEWS and bump version for 2.5.1 release
Fix gnt-group --help display
Copy-paste mismatch :)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>(cherry picked from commit 36c70d4ddc508dd1ffdcc806d617d5100f4cb265)
Fix hardcoded Xen kernel path
We already have a ./configure-time variable for this, but it seems tobe actually unused.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>(cherry picked from commit 3c4afa2e93d9499ce39c1aed575dacb549d35083)...
Fix grow-disk handling of invalid units
The reason why grow-disk was doing:
$ gnt-instance grow-disk instance3 0 -64Unhandled Ganeti error: Invalid format
Is because it does it's own ParseUnit call, and doesn't transform thatinto a nicer message.
Accept both PUT and POST in noded
This is a partial cherry-pick from7530364ddbe949bc34fc26f25ba3f5d921beb021 on master:
Currently, noded requires PUT, even though the semantics of the RPCcalls do not match a PUT. We change the code accept both PUT and POST,...
Preserve bridge MTU in KVM ifup script
Closes: #201 - KVM_IFUP does not set bridge-MTU on tap devicesSigned-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>(cherry picked from commit a1ec8695a6b453acdc2fa746a27be73c614b2e87)...
Update synopsis for “gnt-cluster repair-disk-sizes”
Mention that instances can be passed on the CLI when “--help” is used.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>(cherry picked from commit eb5ac108d146644200df98b9f90dae003dcea426)...
Reconcile Makefile.am and test data files
Sorry, forgot this in previous commit.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>(cherry picked from commit 1a1e7ab3f7cfe156152fb69961115a2c85b2a82d)
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initiallywrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force- otherwise refuses to proceed, but it still returns exit code 0...
Enable lvmstrap to run under Linux 3.x
Extend the kernel version check to also accept Linux 3.x as valid.
Signed-off-by: Alexander Schreiber <als@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>(cherry picked from commit 1bf72492d381aacb5c488f1a87ac7665b9ddc6c7)...
apidoc: Fix some typos and errors introduced by my previous patches
bdev: Add a guard to verify disk parameters are set
LUGroup*: Fix inheritance of disk parameters
bdev: Take the params directly from disk without refill
Special case blockdev_find
Similiar to blockdev_create we sometimes do find on children. This fixesthose cases. However, this is not very nice.
Special case blockdev_create
This is due to the nature of bdev. We spread some logic into cmdlib anddeal for example with it's children recursively. This makes it hard toannotate the disk parameters in a generic way as we don't always dealwith the top most disk. But the disk parameters are depending on the top...
cmdlib: Adding annotation helper for special cases
cmdlib: Remove all diskparams calculations not required anymore
cmdlib: Adapt the rpc calls
The following (blockdev) RPC calls are not converted yet (as they arenot straight forward or need more research):
rpc: Adapt the callbacks to the new encoder type
We change the cmdlib side on the next patch.
rpc: Adding new encoders for annotating disk parameters
rpc: Adding helper to annotate disk params
config: Adding convenience method for disk parameters
objects: Add helper function to fill disk parameters
QA: Decorate tests with instance checks
Start using “InstanceCheck” decorator where appropriate. Use direct callfor check in places where the instance status changes during the test.
The parameters of “TestInstanceImport” had to be re-ordered to have the...
QA: Infrastructure for hook script to check instance status
This script can be used to check if an instance is running or stopped atvarious points during a QA run. Environment variables are used to passthe most essential information.
QA: Support passing keyword arguments to test functions
Until now this wasn't possible.
Add a default PATH variable to OS scripts env
In commit 896a03f6 I cleaned up the environment for OS scripts,however I think that was a bit too extreme - it breaks our owninstance-debootstrap hooks, because for example dpkg (called from thegrub script) requires PATH to be set....
Move hooks PATH environment variable to constants
Move the contents of the PATH environment variable for hooks toconstants, and use its value in the code and in the hooks documentation.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>...
Add note to the install doc about bridge MAC issues
Thanks to Faidon Liambotis for explaining this on the external IRCchannel.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Faidon Liambotis <paravoid@gmail.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix exception re-raising in Python Luxi clients
Commit e687ec01 (present in 2.5 since the 2.5 beta 3) did consistencyfixes across the code-base. Unfortunately this was done without enoughchecks on the actual meaning of one of the fixes, which means error...
Allow clock skews in certificate verification
Currently we allow for up to NODE_MAX_CLOCK_SKEW time differencebetween nodes in some operations, but not everywhere: SSL certificateverification (import/export, both intra and inter-cluster) has a zerolimit (downwards), and a week upwards. This can cause even...
Improve formatting in the hooks documentation
The pyeval role generates literal (e.g. ``OP_CLUSTER…``) formatting,which differs from the other headers. This is confusing, so let's adda note that these are virtual operations, which makes the output more...
Add decoding of Luxi calls and unittests for LuxiOp
This patch adds a hand-coded decoder for LuxiCall arguments, as thedata-structure is not uniform enough for automated generation (evenfor the serialisation, we had to add hints for some fields,de-serialisation is even harder)....
Some unit tests improvements
In preparation for unit-testing the Luxi definitions, this improvesthe auto-generation of OpCodes: currently this uses unbounded plainStrings for names and fields, and this creates too big test sizes(e.g. when automatically generating job sets)....
Auto-define a LuxiReq data type
We currently auto-generate a LuxiOp data type, which holds the entireoperation (including parameters). However, having a data type just forthe method call would be useful, so let's change THH to alsodefineSADT for the Luxi constructors....
Replace hardcoded constants with Constants.hs names
Now that luxi.py constants are exported, we can use them for moreconsistency.
Fix Makefile dependencies for building Constants.hs
I was wondering why my changes to luxi.py did not trigger the rebuild…