opcodes: Add function for compact summary
Depending on the opcode and its parameters, the existing “Summary”function can give a rater long summary. For displaying the summary inlogs and in the lock monitor, it should be shorter. Hence this newfunction is added to just use the opcode ID with common prefixes...
locking: Make parameter to condition's wait() positional
It is always used in the locking code. Unittests are updated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'devel-2.4'
SetEtcHostsEntry: maintain existing ordering
Currently RemoveEtcHostsEntry keeps the ordering, but SetEtcHostsEntrynot, as it will always write the new entry at the end of file. Ipersonally dislike this as it "uglifies" my custom host files, so thispatch makes it update the record instead in-place so to say instead of...
Fix WriteFile with unicode data
Unicode is fun, indeed:
len(buffer("abc"))
3
len(buffer(u"abc"))
12
So we can't pass unicode data to buffer(), as the result will be towrite the in-memory (usually UTF-32) representation to disk.
Signed-off-by: Iustin Pop <iustin@google.com>...
RAPI: Add support for tagging node groups
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
qlang: Add function to distinguish filters from names
qlang: Add parser for query filter language
With this parser, command line utilities will be able to provide filtersthrough query2 in a simplistic language. Example filters:
name "node3.example.com" master or (name "node4.example.com") be/memory == 128 and name =~ /^web/i...
Add instance query field for OS parameters
These were not available as a query field before. Update unittestsand description text for the other “..params” fields.
utils.WriteFile: Close file before renaming
Issue 154 (http://code.google.com/p/ganeti/issues/detail?id=154)reported an “Operation not supported” error when writing instanceexports to a mounted CIFS filesystem. Experimentation showed the errorto only occur when using rename(2) on an opened file. Various references...
Merge branch 'stable-2.4' into devel-2.4
Increase the lock timeouts before we block-acquire
This has been observed to cause problems on real clusters via thefollowing mechanism:
- a long job (e.g. a replace-disks) is keeping an exclusive lock on an instance- the watcher starts and submits its query instances opcode which...
utils: Add function generating regex for DNS name globbing
The intent of this function is to be able to provide a globbing operatoror query filters. One should be able to say, for example, something tothe effect of “gnt-instance shutdown '*.site'”.
Also rename a variable in MatchNameComponent....
query: Add implementation of regex match operator
So far this operator was not implemented. This patch adds an additionalvalue preparation function to the function table for binary operators,used to compile the regular expression. Unittests are included....
Add a simple wrapper over utils.Retry
The new wrapper makes moving legacy code to utils.Retry or addingretries in existing code simpler.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
test.ganeti.process_unittest: Fix race condition
There was a race condition on heavily loaded testsystem causing randomlyto fail the timeout unittests as the signal handler is not yet setup butthe timeout has already hit.
Therefore we introduce a workaround to wait until a program reached a...
RAPI client: Remove support for version 0 instance creation requests
Implement submitting jobs from logical units
The design details can be seen in the design document(doc/design-lu-generated-jobs.rst).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rewrite of ensure-dirs in python
I provided unittest to test the important pieces of the infrastructure.The one remaining function (ResuriveEnsure) is not easy to unittestbut also not critical if it fails to operate correctly.
Signed-off-by: René Nussbaumer <rn@google.com>...
RAPI client: Tidy and test WaitForJobCompletion
- Use constants- Don't sleep if no delay is given- Mark function as deprecated: it uses polling instead of waiting for changes (but the latter needs authentication); it can still be used- Add unittests...
RAPI client: Add job status constants
Split BuildHooksEnv of LUs
Commit dd7f677623 added another call to BuildHooksEnv to providepost-phase status variables. Since BuildHooksEnv also built the nodelists, that meant they have to be built twice. First a rather strictcheck was used, but it turned out to be more tricky. Commit b423c51336...
Remove restrictive hook node list check
Commit dd7f67762 added a restrictive check for the node lists returnedby BuildHooksEnv, leading to errors with some LUs, one of which wasfixed in commit 0dfa2c227. As it turns out, other LUs have similarissues, some not easy to fix. This patch disables the restrictive check...
hooks: Provide variables with post-opcode values
When a hook is called, it is provided with a number of variablesdescribing the status of the instance/node/etc. before the operation.Some opcodes provide extra variables to see modified values from hooks,...
locking: Fix race condition in lock monitor
In some rare cases it can happen that a lock is re-created very soonafter deletion, while the old instance hasn't been destructed yet. Insuch a case the code would detect a duplicate name and raise anexception....
RAPI: Add support for querying resources
- Access is only permitted for authenticated clients (queries can return sensitive data)- Filters can be specified when sending a PUT request- Updates RAPI client, documentation and tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add support for query resources in RAPI URIs
constants: Rename QR_OP_*, add QR_VIA_RAPI
Commit 28b71a76 added a list of resources which can be queried usingLUXI. Unfortunately the variable was named “QR_OP_LUXI”, which can beconfusing. This patch renames “QR_OP_QUERY” to “QR_VIA_OP”, “QR_OP_LUXI”...
qlang: Remove unused ReadSimpleFilter
utils: Export NiceSortKey function
The ability to split a string into a list of strings and integers can behandy elsewhere and is necessary for sorting query results by names.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Convert OsDiagnose to query
cfgupgrade: Fix critical bug overwriting RAPI users file
The cfgupgrade tool was designed to be idempotent, that means it couldbe run several times and still give produce the correct result. Ganeti2.4 moved the file containing the RAPI users to a separate directory...
query: Add operator for truth
The “?” operator is the equivalent of “if var” in Python.
Fix test rapi request
This was wrong even before, as it contained both a bridge and a link,which are incompatible when passed together. Fix by passing only thelink field.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
query: Fix bug when names are specified
If the client/caller would specify names through the use of a filter,the result would be sorted. This is a regression over earlier Ganetiversions and verified in QA. This patch adds an optional parameter tocontrol the sorting and provides unittests....
query: Add support for filters
This patch adds a “compiler” for query filters, converting them to acallable function used while preparing the query result. In addition, ahints call allows some analysis to be done on the query (e.g. referencednames), making data collection more efficient....
Add query field flags
Some fields (e.g. “name”) should be treated specially when comparing forequality. Hypervisor names should use normal rules, but for node names,“node2” should be equivalent with “node2.example.com”.
To make these differences, a new field for flags is added to the query...
Query: Add additional test to node.powered for OOB handling
This patch covers the testing of _GetNodePower of query.py not only forunavailable but also for OOB support and returning the rightnode.powered.
Fix test output splattering
Instead of hardcode to ToStdout in the helper class, we should usea user provided feedback function. In unittests this is a noop.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add query field descriptions
- Make “doc” parameter to MakeField non-optional- Add descriptions for all fields
Add function to format ordinals
See [1] for the rules.
[1] http://en.wikipedia.org/wiki/Names_of_numbers_in_English#Ordinal_numbers
Introducing gnt-cluster epo
This is a convenience command to do an automated EPO in the possible limits ofGaneti.
Add constants for node roles
Add constants for instance status
They've been hardcoded for too long.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Merge remote branch 'devel-2.4'
query_unittest: Fix argument to set()
Commit e431074f introduced an uncatched bug. This patch fixes this. Theset is expecting a list or iteratable to work on, so it splitted theprovided instance name into a set of characters. This caused theexp_status never been set and therefore not catched in one assert rule...
query: Use aliases for legacy instance NIC fields
Add unittest for cli.FormatResultError
Also make the parameter for verbosity mandatory.
Change the list formatting to a 'special' chars
And also enable verbose display via the, well, verbose option. Manpage and tests are updated, and the formatting is moved from 4 ifstatements to a data structure.
NodeQuery: mark live fields as UNAVAIL for non-vm_capable nodes
Since we don't have the data per design, UNAVAIL is appropriate here,while NODATA is not.
The patch also adds a comment: if we extend the live fields list tocontain other data in the future, we need to reevaluate this solution....
RAPI: Use FillOpCode for adding node group
Make OpGroupRename consistent with OpInstanceRename
OpInstanceRename uses “instance_name” (like almost all other OpInstance*opcodes), not “old_name”, to specify the original name. OpGroupRename ismade consistent by renaming “old_name” to “group_name”....
RAPI: Clean up instance creation, use generated docs
- Use FillOpCode and unify parameter names between RAPI and opcode- Generate parameter documentation- Improve opcode parameter documentation
RAPI: Use FillOpCode for modifying node group
RAPI: Use FillOpCode for renaming instances
baserlib.FillOpCode: Allow parameter rename
Some RAPI parameters don't match the name of the underlyingopcode. With this patch they can be renamed while fillingthe opcode.
RAPI: Use FillOpCode for replacing disks
Add opcode parameter descriptions
These will be used in automatically generated parts of the RAPIdocumentation. Not all opcodes are documented so far, but each addedopcode's parameters need to be fully documented (verified by anassertion).
opcodes: Prepare addition of parameter descriptions
For now they're all set to None, but a following patch will set descriptions.
RAPI: Use FillOpCode for modifying instance
RAPI: Use FillOpCode for migrating instance
RAPI: Use FillOpCode for exporting instance
utils.SetupLogging: Return function to reopen log file
This function can be used from a SIGHUP handler to reopen log files.Initial, simple unittests are included.
Introduce re-openable log record handler
This patch adds a new log handler class based on the standard library'sBaseRotatingHandler. This new class allows the log file to be re-opened,e.g. upon receiving a SIGHUP signal. The latter will be implemented in...
Add RAPI resource for instance console
Export console information as query field
This makes it possible to get the console information via a LUXI query.
Fix instance list for instances running multiple times
If for some reason (e.g. failed migration) one instance is runningon multiple nodes the output can become inconsistent. To get that errorand make it consistent between runs we make the call on the secondary...
Fix unittest breakage on Python 2.4/2.5
Commit 70b0d2a29 broke unittests on Python 2.4 and 2.5. Turns out thatPython 2.6 and above allow classes to be passed as custom test runners,whereas earlier versions don't.
Ensure all resources are used by RAPI client
Check for duplicate RAPI URIs and handlers
RAPI client: Wrap /2/redistribute-config resource
RAPI client: De-/activating instance disks
Add unittest for RAPI client's ModifyInstance
Rename QRFS_* to RS_*
This patch renames QRFS_* to RS_* fields so they can be used in otherplaces (i.e. LUs) without confusion, as this was initially meant forquery operations.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
query: Add alias support in _PrepareFieldList
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
query: Change internal result computation
While looking at the query library, I realized that while we have fivefield statuses, making this a 5-dimensional space, four of them areshrunk to a single possible value (None). Hence it should be possible toconvert this into a single value space plus extra 4 special constants....
KVM: Perform network configuration in Ganeti
This patch introduces network configuration for KVM in Ganeti.
There are three problems with having KVM perform network configuration via ifupscripts: a) Ganeti never gets to know the tap interface that is associated with an...
Rename OpGetTags and LUGetTags
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Rename OpTestJobqueue and LUTestJobqueue
Check consistency of the class names and OP_ID
As the class names should be now consistent with the OP_IDs, we add acheck for wrongly-defined OP_IDs.
However, the future removal of the hand-coded OP_IDs will render thisobsolete, so this check is introduced just to make sure that the...
Rename OpStartupInstance and LUStartupInstance
Rename OpMigrateInstance and LUMigrateInstance
Rename OpReinstallInstance and LUReinstallInstance
Rename OpRenameInstance and LURenameInstance
Rename OpSetInstanceParams and LUSetInstanceParams
Rename OpShutdownInstance and LUShutdownInstance
Rename OpAssignGroupNodes and LUAssignGroupNodes
Rename OpRenameGroup and LURenameGroup
Rename OpCreateInstance and LUCreateInstance
Rename OpVerifyCluster and LUVerifyCluster
Rename OpExportInstance and LUExportInstance
Bump version for Ganeti 2.4.0~beta1
Update the version in all necessary places. Update NEWS with releasedate.
List node parameters (if any) in gnt-node info
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix typos in RAPI docstrings, add unittest
This patch fixes a number of typos and standardizes RAPI resourcedocstrings. A unittest is added.
Add ability to retain specified fds open in RunCmd
Passing tap devices to KVM as file descriptors requires that the respectivefile decriptors remain open during utils.RunCmd execution. To this direction,we add a “noclose_fds” keyword argument to utils.RunCmd, accepting a list of...
Add tests for objects.Instance