History | View | Annotate | Download (353.1 kB)
Rename some constants and relax pylint rules
For the type system, we want a slightly relaxed rule for constantnaming, so we update the pylint rule. But the old _TPInt and _TNEStringwere not clear enough, so we expand them.
Signed-off-by: Iustin Pop <iustin@google.com>...
Introduce a micro type system for opcodes
Currently, we have one structual validation for opcode attributes: the_OP_REQP, which checks that a given attribute is not 'None', and therest of the checks are done at runtime. This means our type system hastwo types: None versus Not-None....
LU.CheckPrereq: do not require implementation
Currently, the base class LogicalUnit's CheckPrereq will raiseNotImplementedError, which means that the child LUs have to implementit. However, many LUs don't actually have a need for this function(hence the many "pass" statements as the only body)....
Some more CheckPrereq/CheckArguments cleanup
For a few LUs, a few tests in, or even the whole CheckPrereq, can bemoved to CheckArguments, as they don't touch state and only do a 'type'validation.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add OS verification support to cluster verify
For this, we needed to extend the NodeImage class with a few extravariables, and we do a trick in the node verification where we pick thefirst node that returned valid OS data as the reference node, and then...
Move opcode attribute defaults to data structures
LUExportInstance had two opcode fields set to default via both_CheckBooleanOpField and getattr(…, False).
Cleanup LU.ExpandNames versus CheckArguments
When LogicalUnit.CheckArguments was introduced, not all code dealingwith static argument checking was moved to it; many of these checks wereleft in ExpandNames. With time, most of them migrated, and this patch...
Abstract export mode validity check
The export mode is checked in two places with the exact same code…
Remove the obsolete EvacuateNode OpCode/LU
All code has been switched to the new-style LU… time for cleanup.
Show OS parameters in cluster/instance info
Add support for OS parameters during instance add
This is not yet complete, as it lacks proper support for instanceimport.
_GetUpdatedParams: enhance value removal options
This patch adds controls for whether we recognizeconstants.VALUE_DEFAULT or not as a default value, and also addsdash-prefixes as another way for parameter removal.
Add support for modifying cluster OS parameters
We use _GetUpdatedParams in order to support removal too, and thenvalidate the OS parameters if the OS exists.
Add support for modifying instance OS parameters
We move the instance OS rename checks earlier, as we need to run thevalidation against the new OS, if it has changed.
Add support for OS parameters during import/export
Nothing special here, just copy/adjust the beparams code.
LUDiagnoseOS: add more fields, cleanup
This patch exports all the way from backend a new field ‘api_version’which holds the list of support API versions, and exposes the (alreadycomputed) ‘parameters’ field.
The patch also reworks (again) the field calculation in its Exec()...
Silence a pylint warning
The OS parameters code will bump the number of lines over 10K, and thuswe need to silence this (no, we don't want any other module to becomethis big…, so we use a targeted silence only).
Add reading of OS parameters from disk
The patch also modifies the internal methods in LUDiagnoseOS and gnt-osto deal with the format change of call_os_diagnose.
Introduce an RPC call for OS parameters validation
While we only support the 'parameters' check today, the RPC call isgeneric enough that will be able to support other checks in the future.The backend function will both validate the parameters list (so as to...
Add repetition count to the TestDelay opcode
If the repetition count is not passed or is passed as 0 we sleep exactlyone time, otherwise we sleep "repeat" times and log in between.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.1'
Fix a small bug introduced in cf26a87a
Commit cf26a87a added a tiny typo, which would break non-FQDN argumentsto modify node storage.
Fix the type of 'valid' attribute in LUDiagnoseOS
The update of the valid status in LUDiagnoseOS says:
valid = valid and osl and osl[0][1]
However, in Python, “True and []” (which '[]' we get for an invalid OS)will result in “[]”, and thus the valid field for an OS will be either...
Rework LUSetInstanceParams._GetUpdatedParams
Currently, this function does three things:- special handling of constants.VALUE_DEFAULT- type enforcing of the resulting dict- filling the dictionary with defaults
However, except for the first one, the second two do not belong in this...
Add cluster.SimpleFill*() functions
Currently, the existing cluster.Fill* functions take as argument aninstance. This means that in any case where we don't have an actualinstance object, we have to resort to calling the low-levelobjects.FillDict function....
Merge branch 'devel-2.1' into master
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Fix a bug in instance startup with custom hvparams
Since the introduction of OS-specific hvparams, we shouldn't ever useobjects.FillDict directly for instances, but instead go via the clusterobject. Otherwise the os_hvp will be ignored.
Fix unsafe variant initializer in _TryOSFromDisk
In case an OS has inconsistent declarations, we might get into a casewhere one node reports a valid variants list (with OS API >=15), andanother node has OS API < 15, in which case its supported_variants gets...
Start instance after creating snapshots for export
This restores functionality lost in commit 387794f8. Found duringtests using QA scripts. An instance should be started after ithas been temporarily shutdown for an export.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Use import/export magic for backup/import and inter-cluster moves
This should prevent bugs in our code from accidentally overwritingdisks.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
import/export: Validate remote host/port
The hostname and port received from the remote cluster shouldbe validated, just in case.
Enforce consistency in disks and nics input dicts
With this change unknown disk and nic parameters will be refused, ratherthan silently ignored, so that one can't pass them in by mistake and notrealize what went wrong.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
TLMigrateInstance: pass lu to _Check*
The various _Check* helper functions expect an lu to be passed in, butthe TL is passed instead. This works... sometimes! :)
Show formatted ETA for disk sync and import/export
LUGrowDisk: fix operation on down instances
Currently it's impossible to grow a disk if an instance is shutdown,because the disk could not be assembled. Now we take care of assemblingit, and shutting it down after.
Allow disk operation to act on a subset of disks
If the disks= parameter is passed, we can assemble/wait forsync/shutdown only some disks belonging to an instance, rather than all.
This is useful to only activate/sync/shutdown the affected disk whengrowing it....
Distribute cluster domain secret
The cluster domain secret file was not distributed to other nodes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Put common import/export daemon options into object
The X509 key name and CA are passed from cmdlib all the way tothe backend import/export daemon. With the addition of an optionto choose the compression method, another parameter would haveto be passed all the way. By moving these options to a separate...
Implement opcode changes for remote-import
Implement opcode changes for remote-export
Add opcode to prepare export
To prepare a remote export, the X509 key and certificate need to be generated.A handshake value is also returned for an easier check whether both clustersshare the same cluster domain secret.
Add function to read cluster domain secret
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
LUExportInstance: Remove instance only if export was successful
Until now, the instance was always removed (if asked for by theuser). In case of export errors however, it shouldn't.
Return disk_template from LUQueryInstanceData
Inter-cluster instance moves need the disk template. As they runLUQueryInstanceData to get an instance's details, the disk templatemust be returned.
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittestthe new function for two things: correct reporting on real case (forlocalhost), and correct reporting with a mocked-out TcpPing that returns...
Fix tiny typo in cluster verify
LUExportInstance: Move exporting code to helper class
This will simplify the implementation of intra-cluster instanceexports and reduces the number of local variables inLUExportInstance.Exec.
verify cluster: check /etc/hosts consistency
If we are told to modify /etc/hosts, then verify its consistency duringcluster verify.
Cleanup /etc/hosts during node removal
It seems that commit d8470559 dropped the cleanup of /etc/hosts when anode is removed from the cluster. I don't know for sure, but it seemsaccidental. As long as we add it to /etc/hosts, we should clean it uptoo....
Move cmdlib._VerifyCertificate to utils
This function will also be useful for inter-cluster instancemoves for verifying certificates.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
cmdlib: Convert instance import to new style
Use instance data transfer utility to import instances.
cmdlib: Convert instance export to new style
With this change, exports of several disks will occur in parallel. Errorhandling has also been improved.
Factorize LUExportInstance
Apart from moving parts of the code to separate functions, error handlingis also improved by making sure snapshots are always removed.
LUSetClusterParams: initialize needed parameters
… since the opcode doesn't auto-initialize to None.
Conflicts: doc/security.rst trivial lib/cli.py trivial
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix changing the list of enabled hypervisors
When enabling a new hypervisor, we must ensure that we have itsparameters in the cluster (global) hvparams dictionary.
Furthermore, we add a verify check for this case (this should be betterreplaced with an auto-fix?)....
Readd node: allow changed primary IP
The primary IP is not used for DRBD disks, only for the job queue andinternal commands. As such, it's trivial (with a quiet job queue) tore-ip the node.
The patch just relaxes the checks in LUAddNode, and manual testing shows...
Add --add-uids/--remove-uids to gnt-cluster modify
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add --uid-pool option to gnt-cluster modify
Fix cluster behaviour with disabled file storage
There are a few issues with disabled file storage:- cluster initialization is broken by default, as it uses the 'no' setting which is not a valid path- some other parts of the code require the file storage dir to be a...
Merge remote branch 'devel-2.1'
Add a identify-defaults options for import
When importing an instance, all the saved valued will be used asexplicitly specified values, overriding the cluster defaults. This meansexport+import will change the status (from default to explicitlyspecified) of parameters....
Fix create/import verification of hvparams
Currently the instance creation checks the cluster hv defaults + the newparameters for validity, ignoring the os-specific hvparams (this was anoversight during the implementation of the os hvp). This patch uses the...
Reuse NIC information from export
If the user doesn't pass any nics in import, do not use a defaultone-nic, but instead read the nics from the export file as is.
Fortunately the export and the way nics are read from the command lineare compatible…
Reuse backend parameters from export
Similar to the previous patches, if we're missing some parameters andthe export has them (either in the new style or old-style), we reusethem.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Reuse disk information from export
If the user doesn't pass the disk information on import, automaticallyreuse the number and size of disks. This loses the iv_name attribute,but that is only cosmetic and cannot be changed by the user.
Reuse hypervisor parameters in import
If available, we reuse the parameters from the export info.
Read disk template from export info
This patch changes the instance import to read the disk templateautomatically from the export info, if the opcode doesn't alreadyspecify a disk template.
To do this, we have a couple of additional changes:
- change from required parameter to optional one for disk_template...
CreateInstance: separate the reading of the export
We move the reading of the export to a separate function, to simplifyCheckPrepreq and also read it earlier. This will allow building themissing opcode parameters from the export information, instead of...
Move code from ExpandNames to CheckPrereq
This is needed since only in CheckPrereq we have the nodes locked, andfuture import enhancements will need to have access to the export infoduring the parameter build.
CreateInstance: Move some code to CheckArguments
ExpandNames holds too much non-locking code (first LU to be converted toExpandNames, and we didn't have CheckArguments at that poin), and thispatch moves the checks that are lock-independent to CheckArguments....
Handle errors better for wrong nic_count in export
This fixes an old 'FIXME' entry.
Add a new cluster parameter maintain_node_health
This will be used to conditionally enable the watcher node maintenancefeature.
Allow file storage to be grown
Fix burnin error when trying to grow a file volume
Abstract the growable disk types in a ganeti constants, and only rundisk grow, from burnin, on them.
A rewrite of LUClusterVerify
Per issue 90, current cluster verify is very very brittle. It's one ofthe oldest pieces of code, with only additions without cleanups over thelast years.
Among its problems:
- data initialization interspersed with verification of RPC results,...
Some epydoc fixes
Merge remote branch 'origin/devel-2.1'
Instance creation: implement --no-install mode
This is a simple patch that adds the no-install mode for instancecreation, allowing import from foreign source of the actual OS (insteadof requiring the preparation of data in a form expected by the import...
Allow OS changes without reinstallation
This patch modifies LUSetInstanceParms to allow OS name changes, withoutreinstallation, in case an OS gets renamed on-disk.
cmdlib: Abstract OS checks
This patch moves the node-has-os checks to a separate function.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix behaviour of gnt-node modify -C no
The current check on whether we require auto_promote or not is wrong, aswe check whether we will have exactly the correct number of mastercandidates left. But it is fine if we have more (e.g. when CPS=10 andmc_remaning=19) than the current number, and in that case we shouldn't...
Conflicts: lib/bootstrap.py: Trivial lib/constants.py: Trivial
Rightname confd's HMAC key
Currently, the ganeti-confd's HMAC key is called “cluster HMAC key” orsimply “HMAC key” everywhere. With the implementation of inter-clusterinstance moves, another HMAC key will be introduced for signing criticaldata. They can not be the same, so this patch clarifies the purpose of the...
Implement conversion from drbd to plain
This is much simpler than the opposite, with fewer possibilities offailures.
Implement conversion from plain to drbd
This patch adds a new mode to instance modify, the changing of the disktemplate. For now only plain to drbd conversion is supported, and thenew secondary node must be specified manually (no iallocator support).
The procedure for conversion works as follows:...
Abstract check that an instance is down
Multiple LUs require that an instance is not running while they operateon the instance (reinstall, rename, modify, recreate disks, deactivatedisks). The code to do this check is duplicate many times, and not very...
Abstract node free disk space check
Both create instance and grow disk check the free disk space on nodesusing the same, duplicate code. Since we'll need this in other places inthe future, we abstract the check into a new function.
The patch adjusts the error message to be more in-line with the one for...