Expose whether htools was enabled to Python code
This exports whether htools was enabled at configure-time, and adds aconstant for our reference iallocator.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
test.ganeti.process_unittest: Fix race condition
There was a race condition on heavily loaded testsystem causing randomlyto fail the timeout unittests as the signal handler is not yet setup butthe timeout has already hit.
Therefore we introduce a workaround to wait until a program reached a...
RAPI client: Remove support for version 0 instance creation requests
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RAPI server: Drop support for instance creation format 0
Ganeti 2.1.3, released in June 2010, added support for a new, extensibleinstance creation request format, called version 1. This patch removessupport for the old and undocumented version 0 format....
Improved GanetiRapiClient docstrings
- Added @rtype and/or @return where missing- Fixed @param for Query() filter_ parameter (colon was missing)
Signed-off-by: Simeon Miteff <simeon.miteff@gmail.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Relax instance ERROR on admin_down on offline node
This fixes a issue, where an stopped instances is reported as ERRORin cluster verify if it lives on a offline node. As the instances isdown this shouldn't happen.
Signed-off-by: René Nussbaumer <rn@google.com>...
Implement submitting jobs from logical units
The design details can be seen in the design document(doc/design-lu-generated-jobs.rst).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
watcher: improve logging a bit
Add some debug logging to detail why we don't run some steps.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Rewrite of ensure-dirs in python
I provided unittest to test the important pieces of the infrastructure.The one remaining function (ResuriveEnsure) is not easy to unittestbut also not critical if it fails to operate correctly.
Add opcode summary to SubmitManyJobs errors
Requested-by: Iustin Pop <iustin@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RAPI client: Tidy and test WaitForJobCompletion
- Use constants- Don't sleep if no delay is given- Mark function as deprecated: it uses polling instead of waiting for changes (but the latter needs authentication); it can still be used- Add unittests...
RAPI client: Add job status constants
RAPI client: Job IDs are strings
Split BuildHooksEnv of LUs
Commit dd7f677623 added another call to BuildHooksEnv to providepost-phase status variables. Since BuildHooksEnv also built the nodelists, that meant they have to be built twice. First a rather strictcheck was used, but it turned out to be more tricky. Commit b423c51336...
RAPI client: fix epydoc formatting
Add a helper function to the RAPI client
This adds a new method WaitForJobCompletion that can be used forclient who are not interested in the entire job log, just in itscompletion status.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>...
Remove restrictive hook node list check
Commit dd7f67762 added a restrictive check for the node lists returnedby BuildHooksEnv, leading to errors with some LUs, one of which wasfixed in commit 0dfa2c227. As it turns out, other LUs have similarissues, some not easy to fix. This patch disables the restrictive check...
Fix hook node list when adding node
This broke QA (and everyone trying to add a node) by complaining aboutdifferent node lists.
hooks: Provide variables with post-opcode values
When a hook is called, it is provided with a number of variablesdescribing the status of the instance/node/etc. before the operation.Some opcodes provide extra variables to see modified values from hooks,...
HooksMaster: Add more assertions for variable names
Also replace explicit loop with dict.update.
mcpu: Tidy HooksMaster a bit
- Dictionary indentation- Add empty lines for readability- Simplify conditional code
cmdlib: Factorize running post-pase hook
qlang: Remove unused import
RAPI: Add support for querying resources
- Access is only permitted for authenticated clients (queries can return sensitive data)- Filters can be specified when sending a PUT request- Updates RAPI client, documentation and tests
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add support for query resources in RAPI URIs
masterd: Simplify code for field queries
Instead of going via cmdlib and using special cases for differentresources, the list of fields is used directly.
constants: Rename QR_OP_*, add QR_VIA_RAPI
Commit 28b71a76 added a list of resources which can be queried usingLUXI. Unfortunately the variable was named “QR_OP_LUXI”, which can beconfusing. This patch renames “QR_OP_QUERY” to “QR_VIA_OP”, “QR_OP_LUXI”...
qlang: Remove unused ReadSimpleFilter
TLReplaceDisks: Add check if disks are activated
Previously we failed later with a rather useless error message. Thispatch fixes this and tells the user to activate-disks if replace-disksis in the need of activated disks rather than abort with a cryptic error...
LUOsDiagnose: Move legacy behaviour into filter
The behaviour of LUOsDiagnose needs special treatment. Commit d22dfef7changed it to not return hidden, blacklisted or invalid OSes if therespective field is not requested. This behaviour needs to be preserved...
Convert OsDiagnose to query
qlang: Add some more documentation for filters
It's not perfect, but at least some more.
query: Add conversion wrapper
Allows converting the value of a column before returning it. Useful forsorting while still using one of the other generic functions forretrieving the value.
Fix epydoc warning about unknown reference
config: Wrap MatchNameComponent, reduce lock duration
- Remove duplication by merging two MatchNameComponent into a wrapper- Reduce lock duration by getting list of names under lock and then matching names without the lock- Also, ExpandNodeName's docstring is fixed....
opcodes: Document OpQueryFields' parameters
Instance failover: fix bug for INT_MIRROR cases
Patches db366d9a and aac4511a added support for EXT_MIRROR instances,but inadvertently introduced a bug: for INT_MIRROR cases, we don'tneed (actually we can't support) neither an iallocator nor a targetnode....
gnt-cluster epo: Adding --power-delay flag
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
gnt-node power: Adding --power-delay flag
cli.py: Adding POWER_DELAY_OPT
The command line option --power-delay sets the time waited between powerons.
OpOobCommand: Adding power on delay
This delays the invocation of the power on of the next node. So if youpower on a bunch of nodes it will not blow the fuse.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
OpOobCommand: Document all fields
gnt-cluster epo: Adding --shutdown-timeout
This adds the --shutdown-timeout flag to gnt-cluster epo to specify theshutdown timeout for instance shutdown.
Rename DTS_NET_MIRROR to DTS_INT_MIRROR
DTS_INT_MIRROR better contrasts DTS_EXT_MIRROR.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>[iustin@google.com: updated patch for changed context]Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
KVM: use cache=none for shared disk templates
Disable host cache for externally mirrored disks to avoid cache incoherency.Without this, migrations between the same two nodes may end up in diskcorruption.
This is a runtime override of cluster defaults, mostly a workaround....
Shared storage instance failover
Modify LUFailoverInstance to enable shared storage instances to failover.Shared storage instance failover requires either a target node or aniallocator to determine the target node. If none is given, the cluster default...
Shared storage node migration
Modify LUNodeMigrate to provide node migration for nodes with instances usingshared storage. gnt-node migrate has to be passed an iallocator for migrationof shared storage instances to be performed. When using a shared storage...
Shared storage instance migration
Modify LUMigrateInstance and TLMigrateInstance to allow instance migrations forinstances with DTS_EXT_MIRROR disk templates.
Migrations of shared storage instances require either a target node, or aniallocator to determine the target node. If none is given, the cluster default...
CLI changes to facilitate shared storage migration/failover
Add DST_NODE_OPT to cli.py to use for directly specifying the target nodeduring migration/failover.
gnt-instance failover/migrate also get passed an iallocator option.
gnt-node failover/migrate get only a target_node option....
Migration and failover: add iallocator and target_node slots
Add iallocator and target_node slots to OpMigrateInstance andOpFailoverInstance to facilitate shared-storage-backed instance mobility. Addiallocator slot to OpMigrateNode (no explicit target_node in this case)....
IAllocator changes to work with shared storage
Make cmdlib.IAllocator shared-storage-aware. IAllocator requires secondarynodes only on DTS_NET_MIRROR disk templates and requires no secondaries forDTS_EXT_MIRROR templates.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
Shared block storage support
This patch introduces basic shared block storage support.
It introduces a new storage backend, bdev.PersistentBlockDevice, touse as a backend for shared block storage. The new bdev requires a newBLOCKDEV_DRIVER_MANUAL constant with the value "manual" and uses it as...
Add bdev_sizes RPC call
The bdev_sizes multi-node RPC call returns the sizes of the requestedblock devices on the desired nodes. Its intended use is to verify theexistence of a block device on a given node for shared block storagesupport.
Block device paths are expected to lie under constants.BLOCKDEV_DIR...
QA: Improve tests for gnt-os
- Test OS lists via command line and RAPI- Test “gnt-os diagnose” and “gnt-os info”
Log log-file reopening
This makes the log files get an record notifying of the reopen, so asto force creation of the log files soon after rotation.
Merge branch 'stable-2.4'
query: Add operator for truth
The “?” operator is the equivalent of “if var” in Python.
Merge branch 'devel-2.3' into devel-2.4
Merge branch 'devel-2.2' into devel-2.3
Fix LUClusterRepairDiskSizes and rpc result usage
This LU was introduced before the RPC result conversion from .data to.payload, and it has managed to keep the old-style usage (how? it'sthe only LU that does so). Fix by changing to payload, and add some...
Fix RPC mismatch in blockdev_getsize[s]
Commit 92fd2250 added consistency checks in the RPC layer, which brokethe call_blockdev_getsizes RPC call (declared with 's' at the end inrpc.py, without 's' in the node daemon).
The immediate fix is to correct the rpc function name, the long term...
RAPI: fix evacuate node resource
PollJob returns the whole op_results, hence a list of opcode results.
LUInstanceRename: Fail if renamed hostname mismatch
There's a problem if you run gnt-instance rename with a non FQDN and therenamed LU tries to resolve the hostname to make it FQDN. It could bethat this resolved hostname was just a CNAME to another name which leads...
Merge remote branch 'stable-2.4' into devel-2.4
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Remove bridge field from rapi daemon
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove bridge field from rapi client
Remove deprecated 'bridge' nic parameter
This has been a synonym for "link" since a few major versions.Add a NEWS entry so we won't forget to mention it at release time.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Unhardcode constants from instance batch create
Add comments about INIC, IDISK and NIC constants
query: Fix bug when names are specified
If the client/caller would specify names through the use of a filter,the result would be sorted. This is a regression over earlier Ganetiversions and verified in QA. This patch adds an optional parameter tocontrol the sorting and provides unittests....
Add DTS_MIRRORED frozenset
Use DTS_MIRRORED to indicate mirrored disk templates that allowmigrations/failover.
DTS_MIRRORED is the union of DTS_EXT_MIRROR and DTS_NET_MIRROR.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
Shared file storage initialization code
Add shared file storage handling during cluster initialization.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Core shared file storage support
This patch introduces core file storage support, consisting of the following:
A configure-time switch for enabling/disabling shared file storagesupport and controlling the shared file storage location:--with-shared-file-storage-dir=. Shared file storage configuration is then...
cmdlib: Allow use of more complex filters
This patch finally enables the use of complex filters through opcodesand LUXI.
query: Add support for filters
This patch adds a “compiler” for query filters, converting them to acallable function used while preparing the query result. In addition, ahints call allows some analysis to be done on the query (e.g. referencednames), making data collection more efficient....
Add query field flags
Some fields (e.g. “name”) should be treated specially when comparing forequality. Hypervisor names should use normal rules, but for node names,“node2” should be equivalent with “node2.example.com”.
To make these differences, a new field for flags is added to the query...
qlang: Add new operators
These are only used programmatically, not exposed to the user.
Fix LU processor's GetECId
The exception was never actually raised.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Adeodato Simo <dato@google.com>
gnt-cluster master-failover: Undrain queue
- Move functions for drain status (tracked via file) from jqueue to jstore- Undrain queue on master failover if necessary- Add QA test
Merge branch 'devel-2.4' into stable-2.4
Fix potential data-loss bug in disk wipe routines
For the 2.4 release, we only add the missing RPC calls. However, thisneeds to be fixed properly, by preventing usage of mis-configureddisks.
Also add a bit more logging so that it's directly clear on which node...
1-char comment typo fix
Signed-off-by: Stephen Shirley <diamond@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
gnt-* list-fields: Show field description
gnt-instance reboot start instance if not yet started
This patch starts the instance when gnt-instance reboot is invoked on ainstance already stopped.
utils: Export NiceSortKey function
The ability to split a string into a list of strings and integers can behandy elsewhere and is necessary for sorting query results by names.
Expand some acronyms, add to glossary
Fix test output splattering
Instead of hardcode to ToStdout in the helper class, we should usea user provided feedback function. In unittests this is a noop.
Add script to generate query fields documentation
- All lines matching "QUERY_FIELDS_${resource}" in the input will be replaced with a definition list describing the fields for $resource- The core code is kept in the Sphinx extension, so that it could be...
QUERY_FIELDS_${resource}
Add query field descriptions
- Make “doc” parameter to MakeField non-optional- Add descriptions for all fields
query: Add support for field description
Add function to format ordinals
See [1] for the rules.
[1] http://en.wikipedia.org/wiki/Names_of_numbers_in_English#Ordinal_numbers
Introducing gnt-cluster epo
This is a convenience command to do an automated EPO in the possible limits ofGaneti.
Move OOB_TIMEOUT_OPT to cli.py
OOB commands where we need to be able to specify the timeout areenhancing over the border of just gnt-node so we move it into cli.pyfor easy reuse in other cli parts.
Add constants for node roles
Add constants for instance status
They've been hardcoded for too long.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
query: Use aliases for legacy instance NIC fields
Merge branch 'devel-2.4'
Add unittest for cli.FormatResultError
Also make the parameter for verbosity mandatory.
Fix title of query field containing instance name
cmdlib: Fix pylint error