Import: unittests
Signed-off-by: Agata Murawska <agatamurawska@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Revert "Fail if dictionary uses invalid keys" and "Support newer “json” module"
This reverts commit fd0351aef246f5d36e641209429e2ec093d325f8 and9869e771704ada62bab001e729c52a36525ef081. The built-in module is a lotslower in Python 2.6.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
serializer: Fail if dictionary uses invalid keys
JSON only supports a very restricted set of types for dictionary keys,among them strings, booleans and “null”. Integers and floats areconverted to strings. Since this can cause a lot of confusion in Python,...
RAPI: Add resource to powercycle node
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RAPI: Add resource to recreate instance's disks
This was still missing from RAPI.
Merge branch 'devel-2.5'
docs unittest: Add verification of opcodes covered by RAPI
All opcodes which are not yet covered or can't with the current design(e.g. cluster initialization) are excluded. This test is added to thedocs unittest since here the code already has access to a list of all...
docs unittest: Split into two separate test cases
The RAPI and hooks documentation aren't related at all.
docs unittest: Make _ReadDocFile a global function
rlib2: Declare all opcodes and equivalents
By declaring all used opcodes or opcodes equivalent to the operationsexecuted in a resource we will be able to ensure all opcodes are coveredby RAPI (with some exceptions).
Add SPICE support to gnt-instance console
Also update related unit tests.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix unittest breakage from commit 132cdb87159e
rapi: Re-add “/2” resource
Like “/”, it'll just return a JSON null.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Wrap long line in ganeti.rapi.rlib2_unittest.py
rlib2: Improve test for _ParseInstanceReinstallRequest
rlib2: Add test for /2/info
rlib2: Add tests for /, /version and /2/features
rlib2: Convert /2/instances to OpcodeResource
rlib2: Convert /2/*/tags to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage/repair to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage/modify to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/storage to OpcodeResource
rlib2: Convert /2/groups/[group_name] to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/role to OpcodeResource
Also fix a typo in a constant name.
rlib2: Convert /2/instances/[inst]/disk/[idx]/grow to OpcodeResource
rlib2: Convert /2/instances/[inst]/modify to OpcodeResource
rlib2: Convert /2/instances/[inst]/rename to OpcodeResource
rlib2: Convert /2/instances/[inst]/failover to OpcodeResource
rlib2: Convert /2/instances/[inst]/migrate to OpcodeResource
rlib2: Convert /2/instances/[inst]/export to OpcodeResource
rlib2: Convert /2/instances/[inst]/prepare-export to OpcodeResource
rlib2: Convert /2/instances/[inst]/deactivate-disks to OpcodeResource
rlib2: Convert /2/instances/[inst]/activate-disks to OpcodeResource
rlib2: Convert /2/instances/[inst]/replace-disks to OpcodeResource
rlib2: Convert /2/instances/[inst]/shutdown to OpcodeResource
rlib2: Convert /2/instances/[inst]/startup to OpcodeResource
rlib2: Convert /2/instances/[inst]/reboot to OpcodeResource
rlib2: Convert /2/instances/[inst]/info to OpcodeResource
rlib2: Convert /2/instances/[inst] to OpcodeResource
rlib2: Convert /2/groups/[group_name]/assign-nodes to OpcodeResource
rlib2: Convert /2/groups/[group_name]/rename to OpcodeResource
rlib2: Convert /2/groups/[group_name]/modify to OpcodeResource
rlib2: Convert /2/groups to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/evacuate to OpcodeResource
rlib2: Convert /2/nodes/[node_name]/migrate to OpcodeResource
rlib2: Convert /2/redistribute-config to OpcodeResource
rlib2: Convert /2/modify to OpcodeResource
Also add unittests.
rlib2: Add unittests for two error conditions
Adds tests for errors on connecting to the server and for a full queue.
baserlib: Add more generic base class for opcode resources
This base class, which employs a meta class for the actual work, allowseasier definitions of RAPI resources using opcodes. Follow-up patcheswill change some of the existing RAPI resources.
The long-term goal with these changes is to make it easier to verify the...
rapi: Remove “/2” resource, deprecate “/”
These were never really useful. Neither gave a complete list ofavailable resources—the documentation in doc/rapi.rst is much better atthat.
Since some monitoring code might use the “/” resource it's kept around...
utils: Introduce IsBelowDir
This is mainly a wrapper to overcome the limitation of commonprefixwhich makes a string by string comparisation and reports the commonprefix in both strings. However this is bad for directory handling.
IsBelowDir works around this limitation and should be used in favour of...
Draft implementation of QMP connection
Basic implementation of the QMP connection and related tests.
Merge branch 'devel-2.4' into devel-2.5
Conflicts: NEWS (trivial) configure.ac (trivial) daemons/ensure-dirs.in (deleted)
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
utils: Fix UnescapeAndSplit parsing bug
If a value passed to UnescapeAndSplit ended with a backslash anexception would be raised:
$ gnt-instance modify -H mem=x\\ inst1.example.com[…] e2 = slist.pop(0)IndexError: pop from empty list
Added function for parsing multiple CPU pinning mask
Signed-off-by: Tsachy Shacham <tsachy@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adding missing test data for commit 7a380ddfc
Fix a parsing issue with DRBD 8.3.11 in the Linux Kernel
In the Linux kernel commit 4b0715f096 introduced a display bug into/proc/drbd which broke our regex.
The bug was first introduced into Linux 2.6.39-rc1. This bug is stillunfixed as of today.
This patch adapt the regular expression to workaround this bug for the...
ensure-dirs: Check mode and owner before changing
This avoids many calls to chmod(2) and chown(2), and thereby ctimeupdates.
Since I had to update the unittests anyway I untangled the code a bit,split it into more separate functions and added some more tests....
ensure-dirs: Refine error handling on stat(2)
The “_stat_fn” function is renamed to “_lstat_fn” to reflect itsfunction. The try/except block just wraps calling lstat(2) and nothingelse.
Clarify job ID-related type checks, add unittests
Instead of a rather complicated expression only “JobId” is output. JobID lists (like generated by “SubmitManyJobs”) are limited to two-itemlists. Unittests are added.
Added helper functions in netutils and related constants
Added the following functions to netutils:- IsValidInterface- GetInterfaceIpAddresses- _GetIpAddressesFromIpOutput
Added the following static methods to netutils.IPAddress:- GetAddressFamilyFromVersion...
rlib: Expose node group tags
Commit 1ffd26739d3 added support for tagging node groups. Also add acheck for exposed fields.
Detect globbing patterns as query arguments
Short: this patch enables the use of “gnt-instance list '*.site'”.
Detailed description: This patch changes the command line interface codeto try to deduce the kind of filter from the arguments to a “list”command. If it's a list of plain names an old-style name filter is used....
Implement globbing operator for filters
The operators “=*” and “!*” do globbing in filters, e.g.:
$ gnt-instance list --no-headers -o name 'name =* "*.site"'inst1.site.example.com
Bump version to 2.5.0~beta1
cleaner: Remove watcher's instance status file after 21 days
utils.ReadFile: Add pre-read callback
This will be used by the watcher to store the file's fstat(2). It mustbe done from the filehandle.
Fix unittest failure after list_owned changes
We just need an object that has a list_owned method.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
ganeti-cleaner: Remove old watcher state files
Watcher state files can stay around if node groups are removed. Withthis patch they're removed after 21 days.
Add primary/second nodes' group as query fields
These will be very useful for ganeti-watcher as it needs to retrieveinstances by group.
Add ht-based result checks to opcodes
This adds the infrastructure necessary to check opcode results usinght-based functions. Checks are added for two opcodes.
Merge branch 'devel-2.4'
Reopen log file only once after SIGHUP
Commit b6fa9a44 added a re-openable log handler. The log file isreopened when a daemon is sent a HUP signal. Due to a bug in the code,fixed by this patch, the log file would be reopened for every single logmessage thereafter....
Implement instance failover via RAPI
No idea why this was missed before.
Export job dependencies through lock monitor
This makes them visible to the user. Example:
$ gnt-debug locks -o name,pendingName Pendingjob/890 job:891,892job/892 job:894
Make lock monitor more versatile
With this change it'll be possible to register other lock informationproviders. One usecase for this are job dependencies, which can be shownin the output of “gnt-debug locks”, too.
The lock monitor is changed to accept more than one return value from...
Rename *_STATUS_WAITLOCK to …_WAITING
This patch renames the {JOB,OP}_STATUS_WAITLOCK constants to {JOB,OP}_STATUS_WAITING, as per design document for chained jobs.
Fix locking issue with job dependencies
When jobs waiting for a dependency are notified, they're re-added to thequeue. This would require owning the queue lock in exclusive mode, butsince the function doing so is called from within the job/opcodeprocessor, it only holds the lock in shared mode....
jqueue: Implement submitting multiple jobs with dependencies
With this change users of the “SubmitManyJobs” interface can userelative job dependencies. Relative job IDs in dependencies are resolvedbefore handing the job off to the workerpool.
jqueue: Add “writable” flag to memory objects
Basically only one instance of the job, the one being processed,should be serialized to disk and replicated to other nodes. Withthis flag assertions can be added in various places.
Implement chained jobs
An overview is available in the design document for this change,doc/design-chained-jobs.rst.
When a job enters the job processor, the current opcode's dependenciesare evaluated. If a referenced job has not yet reached the desired...
Adding a wrapper around connecting to kvm console
The wrapper will connect to the console, and check in the background ifthe instance is paused, unpausing it as necessary.
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cli.GetOnlineNodes: Support node group filter, use query2
This patc changes cli.GetOnlineNodes to use query2, which does thefiltering in the master daemon, and adds a new parameter to filter bynode group.
Unittests were added for the old implementation and then adopted to...
ht: Add new check for numbers
Places which receive floats can usually also deal with integers, e.g.OpTestDelay. Tests are added and the new check function is used for theaforementioned opcode and verifying query results.
Change RAPI for new node evacuation opcode
The change is not backwards compatible, see the updated NEWS file.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Adding basic abstraction layer for caching
This includes an own simple cache implementation and aninterface to a memcache instance.
Fix _checkRsaPrivateKey for newer key generation
Keys generated under debian sid just read "BEGIN PRIVATE KEY" ratherthan "BEGIN RSA PRIVATE KEY".
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix incomplete merge
Commit 66bd7445 changed the semantics of _JobProcessor on finishedjobs, and updated the related unittests in the 2.4 branch. It was thenmerged to master, however on master there was an additional test forthis case, which was not updated....
jqueue: Fix potential race condition when cancelling queued jobs
When a job was cancelled, its status would be changed and the filewritten again. Since this was a final status, the job file could bemoved anytime for archival. If the job was still in the queue, however,...
gnt-node migrate: Use LU-generated jobs
Until now LUNodeMigrate used multiple tasklets to evacuate all primaryinstances on a node. In some cases it would acquire all node locks,which isn't good on big clusters. With upcoming improvements to the LUsfor instance failover and migration, switching to separate jobs looks...
ht: Add checks for anything, regexp, job ID, container items
The check for container items is useful for tuples and/or lists withnon-uniform values. The “anything” check can be used when any valueshould be accepted for an item.
The job ID check, which uses the regexp check, will be used for...
GetEntResolver: Make it possible to resolve uid/gid to name
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils.algo: Add InvertDict to invert a dict
Improve hooks documentation unittest
Also check for the opcode ID.
Split LUClusterVerify into LUClusterVerify{Config,Group}
With this change, LUClusterVerifyConfig becomes a "light" LU that onlyverifies the global config and other, master-only settings, and the bulk ofnode/instance verification is done by LUClusterVerifyGroup, which only acts...
ht: Add strict check for dictionaries
This allows checking specific dictionary items, unlike TDictor TDictOf.