Fix burnin error when trying to grow a file volume
Abstract the growable disk types in a ganeti constants, and only rundisk grow, from burnin, on them.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix backend.VerifyNode behaviour for VG problems
In case LVM is broken, backend.GetVolumeList will raise an RPC exception(as expected since it's a function exposed over RPC). Therefore we mustbe prepared to catch any such exceptions, so that we don't fail the...
Introduce a bool CLI option type
This option type enforces its value to either True or False, relievingthe scripts from manually parsing the values in each function.
We also update the bash completion code to use the option type ifpossible.
Signed-off-by: Iustin Pop <iustin@google.com>...
A rewrite of LUClusterVerify
Per issue 90, current cluster verify is very very brittle. It's one ofthe oldest pieces of code, with only additions without cleanups over thelast years.
Among its problems:
- data initialization interspersed with verification of RPC results,...
Some epydoc fixes
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Adding missing documentation to make the docs better
Also fixed a typo I noticed.
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Remove race condition in FileStorage.Create
Rather than checking that the file doesn't exist, and then creating it,we create it with O_CREAT | O_EXCL, making sure the checking/creation isatomic.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
KVM: Check instances for actual liveness
Currently if we find a live process with the pid we saved we assume kvmis alive. What could happen, though, is that the pidfile has beenreused.
In order to avoid that we change the check to make sure, everywhere,...
KVM: improve GetInstanceInfo docstring
Revert "Only override any and all if not defined"
This reverts commit bd5617020a50bcd08269330638d64078c1b30b71.
Turns out our and python's any/all are not compatible.
Adding RAPI call to deactivate-disks for an instance
Adding RAPI call for activate-disks on an instance
SimpleConfigReader: add docstrings
All non-oneliner functions, after this patch, have their docstring.
Replace @keyword with @param in confd client
@keyword was used inappropriately.
ConfdClient: add synchronous features
By sending requests with async=False, and receiving replies withReceiveReply we can more easily use confd from a synchronous client.
ConfdCountingCallback
This new confd callback counts received replies for the registeredqueries.
move http.WaitForSocketCondition to utils
WaitForSocketCondition: rename, handle EINTR
- Rename WaitForSocketCondition to SingleWaitForFdCondition - Avoid potentially infinite loop, if we continue to get interrupted - Handle eintr correctly - Avoid the poller try/finally, as the poller object gets destroyed...
AsyncUDPSocket.process_next_packet
This function allows receiving socket data synchronously.
AsyncUDPSocket: abstract do_read function
This basically implements read handling, without catching allexceptions. When using the socket in synchronous mode, it's useful toavoid losing exception data (which, in an async daemon, can only belogged)
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Only override any and all if not defined
If any or all are already defined (because we're using a new version ofpython) just link them inside "utils" rather than redefining them.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
backend: Two small style fixes
- Pass keyword parameter as such- Replace “not x == y” with “x != y”
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Allow cluster copy file over the replication net
This patch introduces the option “--use-replication-network” for thecluster copyfile functionality, which is useful if the primary andsecondary network are significantly different (see issue 32).
Enhance cli.GetOnlineNodes query/filtering
This patch allows GetOnlineNodes to return the secondary IPs instead ofthe node names, and to provide filtering of the master node (required tobe done in this function in case we return the secondary IPs).
Instance creation: implement --no-install mode
This is a simple patch that adds the no-install mode for instancecreation, allowing import from foreign source of the actual OS (insteadof requiring the preparation of data in a form expected by the import...
Allow OS changes without reinstallation
This patch modifies LUSetInstanceParms to allow OS name changes, withoutreinstallation, in case an OS gets renamed on-disk.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cmdlib: Abstract OS checks
This patch moves the node-has-os checks to a separate function.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve “gnt-cluster renew-crypto”
- Report exception text immediately instead of just logging it- Remove leftover assertion from when it still used “gnt-cluster modify”
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix behaviour of gnt-node modify -C no
The current check on whether we require auto_promote or not is wrong, aswe check whether we will have exactly the correct number of mastercandidates left. But it is fine if we have more (e.g. when CPS=10 andmc_remaning=19) than the current number, and in that case we shouldn't...
Rename SSL_CERT_FILE to NODED_CERT_FILE
To be consistent with RAPI_CERT_FILE, the rather generic named“SSL_CERT_FILE” constant is renamed to “NODED_CERT_FILE”. The actual filename is not changed.
Rightname confd's HMAC key
Currently, the ganeti-confd's HMAC key is called “cluster HMAC key” orsimply “HMAC key” everywhere. With the implementation of inter-clusterinstance moves, another HMAC key will be introduced for signing criticaldata. They can not be the same, so this patch clarifies the purpose of the...
Clarify the error message for ':' in PV names
As described in issue 93, just saying ':' is not a valid char can beconfusing.
Implement conversion from drbd to plain
This is much simpler than the opposite, with fewer possibilities offailures.
Implement conversion from plain to drbd
This patch adds a new mode to instance modify, the changing of the disktemplate. For now only plain to drbd conversion is supported, and thenew secondary node must be specified manually (no iallocator support).
The procedure for conversion works as follows:...
Abstract check that an instance is down
Multiple LUs require that an instance is not running while they operateon the instance (reinstall, rename, modify, recreate disks, deactivatedisks). The code to do this check is duplicate many times, and not very...
Abstract node free disk space check
Both create instance and grow disk check the free disk space on nodesusing the same, duplicate code. Since we'll need this in other places inthe future, we abstract the check into a new function.
The patch adjusts the error message to be more in-line with the one for...
Abstract disk template verification
This is a simple check, but we'll need it in multiple places.
Implement disk adoption mode in gnt-instance
This patch modifies the parsing of the “--disk” argument to instancecreate to accept “adopt” as a valid key, which builds the correct diskstructure for OpCreateInstance.
LUCreateInstance: implement disk adoption mode
This new mode, valid only for the plain template disk, allows creationof an instance based on existing logical volumes (preserving data),rather than creation of new volumes and OS creation.
The new mode works as follows:...
ConfigWriter: add an LV reservation manager
This patch adds an LV reservation manager to be used for LV names. Sincewe now have four such managers, we create a list for easier release.
LUCreateInstance: Move parameter init earlier
This way, the parameters are available in CheckArguments too.
Implement replacing cluster certs and keys via “gnt-cluster renew-crypto”
Recent changes to “gnt-cluster verify” made it complain on expiring SSLcertificates. While it was possible to replace the SSL certificates andother cluster secrets manually before, doing so was cumbersome. Cluster...
cli: Add helper function to stop and start whole cluster
Replacing cluster certificates and keys requires all cluster daemons to beshut down. This might also be handy for the cluster merger tool, thoughthe function might need a few more extensions.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
bootstrap: Add new function to create cluster certs and keys
The code to generate cluster certificates, keys and secrets is currentlyspread over several places. It makes sense to move it to a separatefunction as we want to provide the user with the ability to automatically...
utils.CreateBackup: Use human-readable instead of seconds since Epoch
Seconds since the Epoch are not easily readable by a human. Using aformatted timestamp makes it easier (e.g.“….backup-2010-03-12_14_02_43.…”). This patch also makes OS logfiles usethis formatted timestamp....
Verify cluster certificates in LUVerifyCluster
When using pyOpenSSL 0.7 or above, LUClusterVerify will start to show awarning 30 days before a certificate expires. 7 days before thecertificate expires, the warning becomes an error. Once expired,LUVerifyCluster will always report an error. The latter is also supported...
utils: Add function to extract X509 cert validity
X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and“notAfter” as strings. Parsing them turned out to be a challenge since...
Add constant with cluster X509 certificates
Merge branch 'stable-2.1' into devel-2.1
Improve cluster verify with hypervisor errors
In case the hypervisor has issues on one node, currentlybackend.VerifyNode will exit via an exception (two exit paths possible,one via HypervisorError from hypervisor.Verify(), and one via RPCFailfrom GetInstanceList). This is bad as it invalidates all other checks of...
Fix wrong indentation
Sorry…
Extend JobExecutor to allow custom feedback_fn
cli.JobExecutor: poll jobs in execution order
… rather than submission order. The results are still returned in thesubmission order, and for this we needed to track internally the indexof the submission.
Add a partition function to split a list in two
This is similar to the Haskell function, except that the signature isreverse to match the 'any' and 'all' Python functions.
Fix cluster verify with simulate-errors
In simulate errors mode, the test "ntime_diff is not None" will beignored, and thus a None value will try to be formatted as %.01f. Weworkaround this by formatting it before, and then only using %s, whichcan format a 'None' value....
KVM: remove unused variable
We don't need the pwentry when checking if a username exists, just to besure the KeyError is not returned. Remove the variable, and thus shut uplint.
Validate the os-specific hypervisor parameters
This adds a validation similar to the one for cluster-wide hypervisorparamters.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
KVM: add security model and domain parameters
Initially we only support the "user" model (in which the user runningthe virtual machine can be specified as an additional parameter).
We use usernames rather than uids in this mode, because the kvm -runasflag doesn't support uids anyway, and we check the passed username for...
KVM security: add global constants
These constants add two new kvm hypervisor parameters, specifying thesecurity model (user/pool) and the security domain, within that model.
http.auth: Disable pylint warnings
Implement verify checks for node/instance names
Since we index the nodes and instances by their name, we should havechecks that the dict key to object.name mapping is correct.
Fix a python 2.6.5 compatibility
The upcoming python 2.6.5 release has a change that makes delattr(obj,attr) fail for slots-enabled objects if the attr is not already set.
To prevent against this, we only run the delattr if the attribute isalready set....
KVM: Remove boot restriction for paravirtual nics
Newer virtio can boot from the network perfectly well, so there's nopoint in keeping this restriction in place. This will still fail onolder kernels.
KVM: pass the instance name as the first kvm flag
This makes it the first argument show, for example under "ps".
http.server: Improve request logging in debug mode
Provide unittests for http.auth
To simplify writing unittests, one data structure class in http.server isalso changed. According to the coverage utility, this provides 95%coverage.
http.auth: Fix bug with checking hashed passwords
When username and password were sent for a resource not requiringauthentication, it wouldn't be accepted if the user in question had ahashed password. The reason was that the function GetAuthRealm used to...
Add the auto_promote option to cli and gnt-node
This allows one to cleanly set a node offline and promote as neededother nodes.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Rework the node modify for mc-demotion
The current code in LUSetNodeParms regarding the demotion from mastercandidate role is complicated and duplicates the code in ConfigWriter,where such decisions should be made. Furthermore, we still cannot demotenodes (not even with force), if other regular nodes exist....
Fix node volumes list for stripped volumes
Currently backend.NodeVolumes() drops everything except the first PV,thus we get a truncated result. The patch is not the nicest, as Pythondoesn't have a simple `concat' function, so I had to change the listcomprehension to an explicit loop....
Fix typo that makes cluster verify to ignore hooks
The return from LUVerifyCluster should be True (or equivalent) for pass,and False (or equivalent) for fail. The HooksCallBack function uses '1'(= True) when a hook fails, which is exactly the opposite of what we...
Fix redistribute config and offline nodes
We need to manually filter out offline nodes before usingrpc.call_upload_file and rpc.call_write_ssconf_files, since these methodare static (they work without a ConfigWriter instance) and thus do notknow which nodes are offline and which are not)....
Add support for per-os-hypervisor parameters
This patch implements all modifications to support per-os-hypervisorparameters in the framework.
Signed-off-by: René Nussbaumer <rn@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
cli: Add ArgOs for later use in gnt-os
KVM: Fix unintended qemu-level bridging of nics
Each nic should be connected to its own qemu vlan, to avoid them allbridging together.
Signed-off-by: Timothy Kuhlman <timkuhlman@gmail.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>...
Support passing in file object in utils.FileLock
This way we can re-use file objects opened in other places. Also add moreunittests.
Support arguments in utils.RunInSeparateProcess
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Convert utils.FileLock to utils.Retry
Validate the hostnames at creation time
This patch adds validation of new names used, i.e. at cluster init time,node add time, and instance creation.
For instances, especially when using «--no-name-check» (which skips DNSchecks), we should validate the give name, and also normalize it...
Add a function to validate and normalize hostnames
This differs slightly from the specification, by allowing names to startwith digits, not checking the length of individual components, andallowing underscores.
ListVisibleFiles: require normalized path names
This patch changes ListVisibleFiles to raise ProgrammerError if it'spassed a non-absolute/non-normalized path name, and adds unittests forthis behaviour.
Switch more code to PathJoin
This should remove most of the remaining constructs which can bereplaced by PathJoin.
Add caller-validation on Disk.StaticDevPath
Since in objects we don't have access to utils.py, we add a warning thatthe result value from objects.Disk.StaticDevPath might not be a validpath, and change its only caller to validate the path.
hv_kvm: remove hard-coded path constructs
This switches hv_kvm to PathJoin. There are still a few cases of directpath construction, but those should be safe.
hv_fake: remove hard-coded path constructs
This changes to hv_fake to PathJoin.
hv_chroot: remove hard-coded path constructs
This patch abstract the computation of an instance's root directory intoa separate function (that uses PathJoin instead of "%s/%s").
Add strict name validation for the LVM backend
Currently we don't enforce name validation for the LVM backend, on theidea that LVM itself will reject invalid names and we catch thoseerrors.
However, recent LVM documents the accepted VG/LV name space, so it's...
Implement disabling of file-based storage
Rationale: the file-based storage backend can add/remove files under acertain directory. However, the master node is also controlling thesetting of the file-based root directory, so basically it means we can'tprevent arbitrary modifications by the master of the node's filesystem....
Replace os.path.sep.join(seq) with utils.PathJoin
This is a no-op change, but at least we concentrate the calls to pathjoins into a single function.
A use in utils.FindFile is left as-is (don't want to raise exceptionsthere, at least for now).
Abstract OS log names computation
The various OS operations create log files in a specific directory(constants.LOG_OS_DIR). The construction of the log names is howeverspread and duplicated across multiple functions.
This patch abstracts this into a separate function that also validates...
Remove superfluous warnings in HooksRunner
For non-existing hooks (the majority of cases probably), logging awarning every time is not helpful. So we first check if we have a validdirectory.
Switch from os.path.join to utils.PathJoin
This passes a full burnin with lots of instances, and should be safe aswe mostly to join a known root (various constants) to a run-timevariable.
utils: Add a PathJoin function
This will replace os.path.join since it is not safe for directorytraversal issues.
Add an extra safety layer to _CleanDirectory
In order to protect from accidental use of _CleanDirectory on a randomdirectory, we add a list of allowed clean directories, somewhat similarto _ALLOWED_UPLOAD_FILES (but statically computed).
Avoid absolute path for privileged commands
Using absolute path for a privileged command is a bad idea as this path may vary.For example /usr/sbin/brctl in Debian and /sbin/brctl in ALTLinux. Using $PATH isa better idea.
Signed-off-by: Vitaly Kuznetsov <vitty@altlinux.ru>...
Make SSH_CONFIG_DIR customizable
This patch adds ability to customize ssh config directory with --with-ssh-config-dir(instead of hardcoded /etc/ssh value). This is useful in Linux distributions withcustom ssh config directories (/etc/openssh in ALTLinux, for example)....
Add NLD constants to Ganeti
This avoids the need for them to be injected in the nbma repository.
Merge remote branch 'origin/devel-2.0' into devel-2.1
Conflicts: NEWS: Trivial configure.ac: Trivial...
Fix two potentially endless loops in http library
The first can be problematic if poll(2) returns POLLHUP|POLLERR on asocket. Before it would be only be respected for SOCKOP_RECV, but sincethey can also occur on other socket operations, esp. in combination with...