utils.algo: Use str.isdigit instead of regular expression
str.isdigit is about 4x faster than using a regular expression ("\d+").This is in the inner sorting code so speed matters.
$ python -m timeit -s 'import re; s = re.compile("^\d+$")' \'s.match(""); s.match("Hello World"); s.match("1234")'...
Include PycURL error code in GanetiApiError.
Signed-off-by: Chris Schrier <schrierc@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Drop objects.QueryRequest
It was only used in one place and wasn't really necessary.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
gnt-os modify: Add "--submit" option
gnt-node: Add "--submit" and "--priority" to commands
gnt-instance: Add "--submit" and "--priority" to commands
gnt-group: Add "--submit" and "--priority" to commands
gnt-cluster modify: Add "--submit" option
gnt-backup: Add "--submit" to two commands
Document "--submit" in ganeti.7
Like “--priority” and “--dry-run”, the “--submit” option is availablefor many commands and can be documented in a central place. This patchalso fixes a small number of style issues.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Fix further QA failures due to query changes
Hopefully these will be the last ones…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix error in opcode result processing
LUXI queries are processed without callbacks (seeserver.masterd.ClientOps._Query). With commit 07923a3c the logic forchecking an opcode's result for jobs to submit was changed andsubsequently raised an exception (“'NoneType' object has no attribute...
Add "show" as alias for "info" to gnt commands
This patch adds support for "show" as an alias for "info" tognt-(cluster|instance|node|os). It already exists in gnt-job.
Signed-off-by: Alexander Schreiber <als@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Copy debug level, priority and set comment for LU-generated opcodes
Before this patch, a node evacuation submitted with high priority wouldonly compute the solution at that priority, but the actual evacuationran at normal priority.
Fix QA failures with "gnt-job list"
Jobs have no “name” field.
gnt-job list: Add options for commonly used filters
While “gnt-job list” would also accept filters on the command line (e.g.“'status == "error"'”, having shortcuts in the form of options comes inhandy.
Merge branch 'devel-2.5'
gnt-* {add,list,remove}-tags: Unify options
- Listing tags is a query, so neither “--priority” nor “--submit” make sense- Support both options for adding/removing tags- Also remove “--submit” from “gnt-node health”; it doesn't work and doesn't make sense for listing node health...
Merge branch 'stable-2.5' into devel-2.5
Bump version for 2.5.0 final release
Also update NEWS file.
Merge branch 'devel-2.4' into stable-2.5
configure.ac: Fix “too many arguments” error
If GHC_PKG_QUICKCHECK contains multiple values, the test would failwith “too many arguments”.
Fix extra whitespace
Sorry, didn't catch this before…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>(cherry picked from commit 54b010cad1ea0a536ed037bf315a04dd1c079964)...
Further fixes concerning drbd port release
Commit 3b3b1bc does not entirely fix the bug introduced in commitf396ad8. It fixes consistency of config data in permanent storage, butdoes not ensure consistency in data held in runtime memory of masterd.
The bug of duplicate ports is still triggered when LUInstanceRemove()...
Fix a bug concerning TCP port release
Commit f396ad8 returns the TCP port used by DRBD disk back to theTCP/UDP port pool using AddTcpUdpPort().
However, AddTcpUdpPort() writes the config on every invocation,using _WriteConfig(). This causes two problems:...
Fix query unittests after converting jobs to query2
I missed these among some shelltest-related failures.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
QA: Add tests for “gnt-job list”
gnt-job list: Switch to query2
This brings “gnt-job list” up to the same level as “gnt-instance list”with filters. Further updates will add more parameters for the mostcommon filters (e.g. only running jobs).
Also update the man page.
gnt-job info: Convert to query2
Otherwise detecting unavailable jobs is hard (“status” is None, is thisan error or just an unavailable job?).
Add job support to query2 via LUXI
This enables the use of filters through query2 when listing jobs.
jqueue: Cache prepared field list in _JobChangesChecker
… instead of re-calculating it on every file change.
NEWS: Deprecate LUXI calls replaced with query2
Adding the “luxi” namespace is necessary in “sphinx_ext”.
jqueue: Convert GetInfo to query2
This rather inefficient implementation (fields are evaluated on everycall to GetInfo) is not good for WaitForJobChanges and doesn't supportfilters, but that will be rectified in later patches.
query: Add definitions for job fields
qlang.MakeFilter: Enable use of different name field
Jobs don't have a “name” field, so we must be able to controlthe field used for simple filters.
Merge cli.FormatTimestamp and utils.FormatTime
… to some degree at least. Unittests are included.
constants: Don't hardcode priorities for LOCK_ATTEMPTS_TIMEOUT
Also include unittest for LOCK_ATTEMPTS_TIMEOUT.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
jqueue._QueuedOpCode: Change a docstring
There was a typo and it's not necessary to repeat the class name.
locking: Remove unused OldStyleQueryLocks
No longer used after commit 090377807.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
ganeti.initd: Add “status” action
Eric Rostetter sent a patch adding a “status” action, but unfortunatelyhis code was apparently specific to Red Hat. I hope this implementationis more distribution-agnostic; after all “status_of_proc” is part ofLSB. Example output:...
Add whitelist for opcodes using BGL
Update IP pool management design doc
Update IP pool managenent design doc to be consistentwith the implementation.
LUOobCommand: acquire BGL in shared mode
Fixed a typo so that now LUOobCommand acquires the BLG in shared mode, asintended.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
LUNodeAdd: Make the version call only dependend on DNS
Also move the version check into prereq to abort before alter clusterstate if the version mismatch.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RPC: Add a new client type for DNS only
This patch moves the “call_version” to a new RPC client definition andthen adds a new runner using the DNS resolver for getting the hostaddress.
The standard “BootstrapRunner”, where the call was before, tries to...
Update install document
Update admin doc
Update walkthrough document
Update default instance kernel version
We switch from vmlinuz-2.6-… to vmlinuz-3-…. To do this nicely, wealso add a ./configure-time setting for the KVM instance kernel.
Update INSTALL and devnotes documents
Added the new Haskell library requirements, for both normal anddeveloper usage.
Furthermore, all commands are now converted to the shell-examplelexer.
Fix escaping of percent signs in the shell lexer
Of course, we do have cases where we want to escape the percent signs,and our regexes were not fully correct for this case.
Add a special lexer for sphinx/pygments
This will be used throughout our docs for better formatting exampleshell sessions, with custom markup for comments, user fixed input anduser variable input.
ganeti.7: Add more filter examples
Enable -Werror by default for htools
Since the code base is now "clean" across all supported GHC versions(6.12-7.4), we can enable -Werror again.
Switch to new-style exception handling
Currently, we're using Prelude.catch to handle I/O errors inhtools. This style of error handling has been deprecated for a while,but it still used to work without warnings.
However, the GHC release 7.4 has started to emit deprecation warnings...
Change a type computation for compatibility with 6.12
This is the last warning related to TemplateHaskell that was 6.12specific; for some reason, it doesn't "see" that traw/tname were used.
The patch just replaces the quoting syntax with an explicit type...
Fix compatibility with TemplateHaskell from GHC 7.4
GHC 7.4 has updated the TemplateHaskell library, and it turns out thatthe way we built the JSON instance implementation for showJSON was notgood (probably this is why GHC 6.12 was generating some warnings)....
Add trivial tests for gnt-* cli
While testing some other stuff, I realised that the gnt-* commandscould be broken (as in, the script fails with syntax errors), but makecheck doesn't detect it. Since we have shelltest, we can now addtrivial tests for this case....
Fix hardcoded Xen kernel path
We already have a ./configure-time variable for this, but it seems tobe actually unused.
Fix docstring bug
Fix a typo introduced in commit c85b15c1, which breaks epydoc.
Enable selection between Python and Haskell confd
This patch changes configure.ac Makefile.am so that the user can pass:
- --disable-confd (or --enable-confd=no) to disable it completely- --enable-confd=yes or --enable-confd=python to select the traditional implementation (this is the default setting)...
Fix qemu-img configure.ac check
By accident, commit a002ed7 introduced the qemu-img checks in thehtools block. I found this also by mistake while investigatinganother issue :)
LUNodeAdd: Verify version in Prereq
There are other ways to leave the cluster in a broken state than justthe version check. However they are not very trivial to fix in 2.5. Soleave it up to 2.6 for a nicer fix.
Signed-off-by: René Nussbaumer <rn@google.com>...
Fix LV status parsing to accept newer LVM
LVM version 2.02.93 (or at least, sometimes after .88) has extend thelv_attr field with two more flag; we only care about the first digit,so let's change the "!= 6" check to "< 6".
Thanks to Robin H Johnson <robbat2@gentoo.org> for finding this issue....
gnt-instance info: Show node group information
This requires acquiring the node group locks in shared mode.
cmdlib: Factorize checking acquired node group locks
The “cur_group_uuid” parameter is optional to prepare for using thefactorized code from LUInstanceQueryData.
Rework exit model
While updating the confd code, I realised that we have lots ofduplication in the exit model for the various programs.
So this patch attempts to abstract all the exits via a couple of newfunctions; sorry for the somewhat big patch, but I hope the payoff is...
Bump version for 2.5.0~rc6 release
Fix out-of-tree builds
The new shell tests do not succeed out-of-tree, due to static pathsand other issues. This trivial patch fixes these issue, make distchecknow passes.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
cmdlib: Stop forking in LUClusterQuery
While debugging another issue we realized that LUClusterQuery forks.This turned out to be the “platform.architecture” function from thePython library. It uses the “file” command to determine the architectureof the Python binary....
Convert manual shell tests to shelltestrunner
This is more of a RFC… Basically most of the shell-based tests areconverted from exec+grep to shelltestrunner.
Things are not all fine and nice though:
- we have dependencies between tests, as some generate some data files...
Add command line option for controlling syslog use
… and enable it in hconfd.
Add support for syslog logging to Ganeti.Logging
Currently this is initialised to no from Daemon.hs, but will in thefuture allow command-line options for controlling it.
locking: Notify only once on release
Don't notify for every released lock in shared mode. The last one isenough.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
locking: Handle spurious notifications on lock acquire
This was already a TODO since the implementation of lock priorities inSeptember 2010. Under certain conditions a waiting acquire can benotified at a time when it can't actually get the lock. In this case it...
locking: Fix lock deletion with timeout
While working on another SharedLock fix I realized timeouts on lockdeletion don't work very well if the timeout actually expires. Thispatch fixes the issue and adds a new unittest.
Move _TimeoutExpired to utils
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit f8326fcaac87958241d78526e5868d23d78ac286)
Revert "Stop acquiring BGL for LUXI queries"
This reverts commit 0fa753bad2cf5a0cf88953347e5da3aebbf21956.
Turns out there are more queries acquiring locks than we'd like. Thispatch goes to version 2.6 and a separate patch fixes the immediateissues in LUClusterVerifyConfig....
LUClusterVerifyConfig: Share BGL, acquire all locks in shared mode
Instead of acquiring the BGL in exclusive mode (which blocks all otheroperations), we acquire all locks for groups, nodes and instances inshared mode before verifying the configuration....
KVM: don't add -nographic using spice
This fixes issue 222.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Only build hconfd if --enable-confd was passed
A later, more complete patch, will allow selecting between either thePython version or the Haskell version. This is just a temporarysolution to help building without all the needed Haskell libraries.
Signed-off-by: Iustin Pop <iustin@google.com>...
Build test helpers that point to hpc-htools
Instead of using just shell constructs to run hpc-htools correctly(i.e. HTOOLS=role htools/hpc-htools …), let's add some shell fragmentsthat do this for us.
This will ease the running of tests directly.
Allow hail to read data from stdin
This patch makes hail treat '-' as denoting stdin, per the usual Unixconvention. This will help with testing.
Update hconfd bind address handling
Instead of hardcoded IPv4 INADDR_ANY, this patch changes hconfd to useeither the any network for the configured cluster address family(ipv4/ipv6), or whatever the user passes in via the --bind option.
Add skeleton ssconf module
This currently has only one export function in it, which will be usedfor future bind address functionality in daemons.
Add two utility functions
These both are work with/on the Result type, so we add them toBasicTypes. The functions will be used as more generic versions ofsome more specialised functions that are right now spread across themodules.
Add the bind-address option
This implements the same logic as the Python code: if the option isnot used, use the default appropriate for the cluster, otherwise tryto parse and use whatever was passed in.
Stop acquiring BGL for LUXI queries
Short description: This fixes an issue whereby masterd would becomeunresponsive on the LUXI socket, leading to client timeouts. While madeworse in 2.5, the underlying issue was already present in 2.4.
Longer description: Until now all LUXI queries would acquire the BGL...
Typo fix: s/aditional/additional/
Trivial fix for a typo in message output of LUInstanceSetParams
Signed-off-by: Alexander Schreiber <als@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fix exported constants
I "forgot" to run the unittests before commit :(
Export the AF_INET/AF_INET6 constants
These are needed to ensure that htools uses the same numeric values asPython.
By the way, I'm not sure what's the best option: importing socket intoconstants (this patch), or adding custom exports toautotools/convert-constants....
Rename htools-hpc to hpc-tools to fix laziness
This is a stupid/trivial patch. I didn't realise when addinghtools-hpc that this will "break" tab completion on htools; as in, Iwill have to add / manually always when completing files on thecommand line....
EPO: Pass the no_remember parameter to preserve state
htools: fix long version of --port for daemons
The dashes do not need to be in, if they are then the resulting optionis:
-p PORT ----port=PORT Network port (default: 1814)
Add htools/rapi test directory to DIRS
Sorry!