Implement submitting jobs from logical units
The design details can be seen in the design document(doc/design-lu-generated-jobs.rst).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rewrite of ensure-dirs in python
I provided unittest to test the important pieces of the infrastructure.The one remaining function (ResuriveEnsure) is not easy to unittestbut also not critical if it fails to operate correctly.
Signed-off-by: René Nussbaumer <rn@google.com>...
RAPI client: Tidy and test WaitForJobCompletion
- Use constants- Don't sleep if no delay is given- Mark function as deprecated: it uses polling instead of waiting for changes (but the latter needs authentication); it can still be used- Add unittests...
RAPI client: Add job status constants
Split BuildHooksEnv of LUs
Commit dd7f677623 added another call to BuildHooksEnv to providepost-phase status variables. Since BuildHooksEnv also built the nodelists, that meant they have to be built twice. First a rather strictcheck was used, but it turned out to be more tricky. Commit b423c51336...
Remove restrictive hook node list check
Commit dd7f67762 added a restrictive check for the node lists returnedby BuildHooksEnv, leading to errors with some LUs, one of which wasfixed in commit 0dfa2c227. As it turns out, other LUs have similarissues, some not easy to fix. This patch disables the restrictive check...
hooks: Provide variables with post-opcode values
When a hook is called, it is provided with a number of variablesdescribing the status of the instance/node/etc. before the operation.Some opcodes provide extra variables to see modified values from hooks,...
RAPI: Add support for querying resources
- Access is only permitted for authenticated clients (queries can return sensitive data)- Filters can be specified when sending a PUT request- Updates RAPI client, documentation and tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add support for query resources in RAPI URIs
constants: Rename QR_OP_*, add QR_VIA_RAPI
Commit 28b71a76 added a list of resources which can be queried usingLUXI. Unfortunately the variable was named “QR_OP_LUXI”, which can beconfusing. This patch renames “QR_OP_QUERY” to “QR_VIA_OP”, “QR_OP_LUXI”...
qlang: Remove unused ReadSimpleFilter
Convert OsDiagnose to query
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
query: Add operator for truth
The “?” operator is the equivalent of “if var” in Python.
Fix test rapi request
This was wrong even before, as it contained both a bridge and a link,which are incompatible when passed together. Fix by passing only thelink field.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
query: Fix bug when names are specified
If the client/caller would specify names through the use of a filter,the result would be sorted. This is a regression over earlier Ganetiversions and verified in QA. This patch adds an optional parameter tocontrol the sorting and provides unittests....
query: Add support for filters
This patch adds a “compiler” for query filters, converting them to acallable function used while preparing the query result. In addition, ahints call allows some analysis to be done on the query (e.g. referencednames), making data collection more efficient....
Add query field flags
Some fields (e.g. “name”) should be treated specially when comparing forequality. Hypervisor names should use normal rules, but for node names,“node2” should be equivalent with “node2.example.com”.
To make these differences, a new field for flags is added to the query...
Query: Add additional test to node.powered for OOB handling
This patch covers the testing of _GetNodePower of query.py not only forunavailable but also for OOB support and returning the rightnode.powered.
utils: Export NiceSortKey function
The ability to split a string into a list of strings and integers can behandy elsewhere and is necessary for sorting query results by names.
Fix test output splattering
Instead of hardcode to ToStdout in the helper class, we should usea user provided feedback function. In unittests this is a noop.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add query field descriptions
- Make “doc” parameter to MakeField non-optional- Add descriptions for all fields
Add function to format ordinals
See [1] for the rules.
[1] http://en.wikipedia.org/wiki/Names_of_numbers_in_English#Ordinal_numbers
Introducing gnt-cluster epo
This is a convenience command to do an automated EPO in the possible limits ofGaneti.
Add constants for node roles
Add constants for instance status
They've been hardcoded for too long.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Merge remote branch 'devel-2.4'
query_unittest: Fix argument to set()
Commit e431074f introduced an uncatched bug. This patch fixes this. Theset is expecting a list or iteratable to work on, so it splitted theprovided instance name into a set of characters. This caused theexp_status never been set and therefore not catched in one assert rule...
query: Use aliases for legacy instance NIC fields
Add unittest for cli.FormatResultError
Also make the parameter for verbosity mandatory.
Merge branch 'devel-2.4'
Change the list formatting to a 'special' chars
And also enable verbose display via the, well, verbose option. Manpage and tests are updated, and the formatting is moved from 4 ifstatements to a data structure.
Signed-off-by: Iustin Pop <iustin@google.com>...
NodeQuery: mark live fields as UNAVAIL for non-vm_capable nodes
Since we don't have the data per design, UNAVAIL is appropriate here,while NODATA is not.
The patch also adds a comment: if we extend the live fields list tocontain other data in the future, we need to reevaluate this solution....
RAPI: Use FillOpCode for adding node group
Make OpGroupRename consistent with OpInstanceRename
OpInstanceRename uses “instance_name” (like almost all other OpInstance*opcodes), not “old_name”, to specify the original name. OpGroupRename ismade consistent by renaming “old_name” to “group_name”....
RAPI: Clean up instance creation, use generated docs
- Use FillOpCode and unify parameter names between RAPI and opcode- Generate parameter documentation- Improve opcode parameter documentation
RAPI: Use FillOpCode for modifying node group
RAPI: Use FillOpCode for renaming instances
baserlib.FillOpCode: Allow parameter rename
Some RAPI parameters don't match the name of the underlyingopcode. With this patch they can be renamed while fillingthe opcode.
RAPI: Use FillOpCode for replacing disks
Add opcode parameter descriptions
These will be used in automatically generated parts of the RAPIdocumentation. Not all opcodes are documented so far, but each addedopcode's parameters need to be fully documented (verified by anassertion).
opcodes: Prepare addition of parameter descriptions
For now they're all set to None, but a following patch will set descriptions.
RAPI: Use FillOpCode for modifying instance
RAPI: Use FillOpCode for migrating instance
RAPI: Use FillOpCode for exporting instance
utils.SetupLogging: Return function to reopen log file
This function can be used from a SIGHUP handler to reopen log files.Initial, simple unittests are included.
Introduce re-openable log record handler
This patch adds a new log handler class based on the standard library'sBaseRotatingHandler. This new class allows the log file to be re-opened,e.g. upon receiving a SIGHUP signal. The latter will be implemented in...
Add RAPI resource for instance console
Export console information as query field
This makes it possible to get the console information via a LUXI query.
Fix instance list for instances running multiple times
If for some reason (e.g. failed migration) one instance is runningon multiple nodes the output can become inconsistent. To get that errorand make it consistent between runs we make the call on the secondary...
Fix unittest breakage on Python 2.4/2.5
Commit 70b0d2a29 broke unittests on Python 2.4 and 2.5. Turns out thatPython 2.6 and above allow classes to be passed as custom test runners,whereas earlier versions don't.
Ensure all resources are used by RAPI client
Check for duplicate RAPI URIs and handlers
RAPI client: Wrap /2/redistribute-config resource
RAPI client: De-/activating instance disks
Add unittest for RAPI client's ModifyInstance
Rename QRFS_* to RS_*
This patch renames QRFS_* to RS_* fields so they can be used in otherplaces (i.e. LUs) without confusion, as this was initially meant forquery operations.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
query: Add alias support in _PrepareFieldList
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
query: Change internal result computation
While looking at the query library, I realized that while we have fivefield statuses, making this a 5-dimensional space, four of them areshrunk to a single possible value (None). Hence it should be possible toconvert this into a single value space plus extra 4 special constants....
KVM: Perform network configuration in Ganeti
This patch introduces network configuration for KVM in Ganeti.
There are three problems with having KVM perform network configuration via ifupscripts: a) Ganeti never gets to know the tap interface that is associated with an...
Rename OpGetTags and LUGetTags
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Rename OpTestJobqueue and LUTestJobqueue
Check consistency of the class names and OP_ID
As the class names should be now consistent with the OP_IDs, we add acheck for wrongly-defined OP_IDs.
However, the future removal of the hand-coded OP_IDs will render thisobsolete, so this check is introduced just to make sure that the...
Rename OpStartupInstance and LUStartupInstance
Rename OpMigrateInstance and LUMigrateInstance
Rename OpReinstallInstance and LUReinstallInstance
Rename OpRenameInstance and LURenameInstance
Rename OpSetInstanceParams and LUSetInstanceParams
Rename OpShutdownInstance and LUShutdownInstance
Rename OpAssignGroupNodes and LUAssignGroupNodes
Rename OpRenameGroup and LURenameGroup
Rename OpCreateInstance and LUCreateInstance
Rename OpVerifyCluster and LUVerifyCluster
Rename OpExportInstance and LUExportInstance
Bump version for Ganeti 2.4.0~beta1
Update the version in all necessary places. Update NEWS with releasedate.
List node parameters (if any) in gnt-node info
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix typos in RAPI docstrings, add unittest
This patch fixes a number of typos and standardizes RAPI resourcedocstrings. A unittest is added.
Add ability to retain specified fds open in RunCmd
Passing tap devices to KVM as file descriptors requires that the respectivefile decriptors remain open during utils.RunCmd execution. To this direction,we add a “noclose_fds” keyword argument to utils.RunCmd, accepting a list of...
Add tests for objects.Instance
utils: Drop RemovePidFile
It's equivalent to a simple RemoveFile now.
utils: Move NewUUID to utils.io
utils: Move more text-related code
utils: Move process-related code into separate file
utils: Move RunningTimeout to utils.algo
utils: Move code manipulating /etc/hosts to separate file
Fix utils.io unittest
“import signal” was missing, but of course it didn't show during mytesting.
utils: Move X509-related code into separate file
utils: Move I/O-related code into separate file
utils: Move code related to file locking into separate file
utils: Move wrappers into separate file
utils: Use function to disable fork
Use a function instead of a variable written by another module.
utils: Change RemovePidFile to take path, not name
This avoids having to monkey-patch the utils module for unittests.Monkey patching is evil and caused a bug while moving code around.
Add unit tests for CheckAssignmentForSplitInstances
Signed-off-by: Adeodato Simo <dato@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Expose OpAssignGroupNodes over RAPI and RAPI client
Validate RAPI resource paths
To stay consistent, RAPI resources should only use a restricted set ofcharacters. No uppercase, no “_”, etc. This is enforced by this patch.
utils: Move hashing-related code into separate file
utils: Split algorithms into separate file
utils: Split Retry & co. into separate file
utils: Move text-related functions into separate file
utils: Move Mlockall into separate file
The unittest file is renamed to match the other utils.* tests.