Check prereq instance communication network in 'SetParams'
Later, the logical unit for 'OpClusterSetParams' will be responsiblefor creating the instance communication network in case it does notexist. For now, it is important to check whether the network the user...
Add helper to handle CLIs that optionally spawn several jobs
This helper function detects whether an opcode returned a list of jobs(i.e., a result of the type ht.TJobIdListOnly) and in this case ituses 'ganeti.cli.JobExecutor' to wait for the jobs and determine the...
Instance comm network from config instead of predefined
Add 'instance_communication_parameter' to 'Cluster'
Merge branch 'stable-2.11' into master
Remove the HTOOLS configuration variable
.. and update the code that uses it.
Signed-off-by: Petr Pudlak <pudlak@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Conflicts: lib/client/gnt_node.py: trivial src/Ganeti/Query/Query.hs: import ALL the functions
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Gracefully handle queries for non-existing nodes
When adding a node, Ganeti checks whether the node is alreadypart of the cluster by querying for the node name. However,as queries are meant to return all nodes with the given name,it might well return the empty list when a new node is to be...
Merge branch 'stable-2.10' into stable-2.11
Fix default for luxi clients in python
As masterd is going away, set default for all clients toluxid's socket.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Remove query option from RAPI client
As all RAPI requests now go to luxid, and masterd is going away,remove option from RAPI client to chose a different socket.
Remove query option from GetClient
As all luxi clients talk to luxid now, and masterd willgo away, remove the option to use socket different fromluxid's.
Remove explicit reference to the query socket
Now that luxid's socket is the default socket anyway, do notpass the "query=True" parameter to GetClient. This will allowto get rid of this keyword argument, as masterd will go away.
Make watcher use luxid socket only
With luxid being feature-complete with respect to masterd,make the watcher use its socket exclusively. This is alsonecessary, as masterd will go away soon.
Fix instance create and import parameters
Move OS parameter related constants to 'ganeti.cli' so they are usedboth by instance create and instance import from the CLI.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>...
Fix compatibility issues
Signed-off-by: Santi Raffa <rsanti@google.com>Reviewed-by: Jose A. Lopes <jabolopes@google.com>
luxid: give stern warnings about debug mode
Luxid as it is can leak private and secret parameters by loggingall requests as they arrive, before any preprocessing is done.
Warn the user stern warnings about this.
Signed-off-by: Santi Raffa <rsanti@google.com>...
OpCodes: modify InstanceReinstall for private, secret params
Modify InstanceReinstall to accept and process private and secretparameters.
OpCodes: modify InstanceCreate for private, secret params
Modify InstanceCreate to accept process private and secret parameters.
OpCodes: modify ClusterSetParams for private parameters
Modify ClusterSetParams to accept and process private parameters.
CLI: add parameters for private and secret OS parameters
Define the CLI parameters for private and OS parameters.
OpCodes: modify InstanceSetParams for private parameters
Modify InstanceSetParams to accept and process private parameters.
Add private OS parameters to cluster and instance conf
This updates objects, constructors and mocks for Instance and Clusterobjects in Python and Haskell.
algo: add GetRepeatedKeys
We do not want public, private and secret parameters to haveoverlapping keys. This function implements this check.
Add Private types to Python, Haskell
This commit adds the private containers to Python and Haskell.
serializer: emit and encode Private values
For inbound data the simplest, safest thing to do is to traverse allJSON right after encoding and search for private parameters by key.
This ensures that all consumers of this data get Private valuestransparently and consistently; the serializing methods don't have to...
RPCs: add docstrings for instance_os_add
This RPC's instance_osp input has a non-obvious, non documentedtype. This patch adds documentation to this RPC.
Switch to luxid for RAPI
...now that luxid knows of all RAPI requests.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Petr Pudlak <pudlak@google.com>
Add CLI and OpCode instance communication option
Signed-off-by: Jose A. Lopes <jabolopes@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add extra NIC to instance being created for communication
When an instance is being created, add an extra NIC (if necessary)which is meant to be used by the instance communication.
Generate TAP names ('gnt.com.%d') for instance comm NICs
Add TAP interface name as argument in '_OpenTap'
This is necessary in order to create TAP interfaces for thecommunication mechanism, which are named 'gnt.com.%d', where '%d' is aunique number within a given node.
Signed-off-by: Jose A. Lopes <jabolopes@google.com>...
Merge branch 'stable-2.9' into stable-2.10
Workaround for monitor bug related to greeting msg
QMP may return multiple greeting messages upon connection.This is reported on qemu-devel. The fix is one-liner butuntil it get's released this is a quick and dirty workaroundthat flushes the client's buffer after getting the first...
hotplug: Verify if a command succeeded or not
Just after issuing _CallHoplugCommands() we invoke_VerifyHotplugCommand() which parses `info pci` resultand searches for given PCI slot and device id.
If we previously had removed a device but it is still there...
hotplug: Call each qemu commmand with an own socat
Previously we issued one socat command with two "\n" separatedactions (e.g. netdev_add ...\ndevice_add...)
After having observed a strange monitor behavior [1] splittingthose commands and introducing a sleep time in between, may reduce...
Fix missing '@raise' in docstring
Fix missing 'L{...}' in docstring
Rename 'CdromOption' to '_CdromOption'
... because it is a protected definition.
Fix specification of TIDiskParams
Commit 580b1fdd incorrectly assumes that disk parameters arejust the standard ones, whereas the man page explicitly statesthat additional parameters can be passed as well, if they makesense for the chosen storage type. Fix this....
Make BlockDev subclasses adhere the interface for Create
In commit 702c3270 two new parameters were added to theCreate function of BlockDev. Make subclasses also adherethis specification.
Make the LUInstanceCreate return node names, not UUIDs
The LUInstanceCreate returned names instead of UUIDs in 2.6. Along theway, the names were internally replaced with UUIDs, and the abstractionleaked. This patch fixes the issue.
Signed-off-by: Hrvoje Ribicic <riba@google.com>...
Make BlockDev subclasses adhere to new interface
In commit 702c3270 two new parameters were added to theconstructor of BlockDev. Make the subclassess accept theseadditional parameters as well.
Make disk.name and disk.uuid available in bdev
Until now Disk name and uuid was not available on bdev level.In case of ExtStorage, this info is useful, and may be for othertemplates in the future too.
This patch treats the name and uuid object slots just like the size...
upgrade: start daemons after ensure-dirs
On upgrading a cluster, we only can rely on daemons startingup cleanly, if all needed directories are generated first. Soensure-dirs needs to be run first.
Gracefully handle degraded instances in verification
The current code assumes that every instance either is of typediskless or has at least one disk. However, with the option toremove individual disk degraded 0-disk non-diskless instancescan occur. While such instances usually are not useful, Ganeti...
Be aware of the degraded case when cleaning up an instance
In the case of a degraded file-based instance, the file storage directoryfor that instance cannot be obtained by looking at the first disk. Usethe standard location, computed from first principles, in this case....
Preserve disk basename on instance rename
For file-based instances, upon rename, the directory containingthe instance disks is moved. Therefore, the basename needs tobe preserved in this case. Fix this. Note that so far, thisworked by accident as before 94e252a3 file names used to be...
Add QA tests for RAPI multi-instance allocation
The instance multi-allocation had no tests to detect its breakage, andthis patch fixes that.
Signed-off-by: Hrvoje Ribicic <riba@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix multi-allocation RAPI method
The OpInstanceMultiAlloc that the instances-multi-alloc RAPI methoduses accepts a list of OpInstanceCreate opcodes rather than a list ofdictionaries as provided by the method. This patch correctly constructsthe opcodes, allowing the RAPI call to work as expected....
Assign unique filenames to filebased disks
With the new format for cmdline arguments, the user is able to add adisk to an instance at a specific index. But filebased disks' filenameshave the form "{0}/disk{1}" where '{0}' is the file_storage_dir and'{1}' is the index of the disk. So if an instance has 3 disks and we...
Add missing import
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Disabling client certificate usage
This patch temporarily disables the usage of the clientSSL certificates. The handling of RPC connections had aconceptional flaw, because the certificates lack a propersignature. For this, Ganeti needs to implement a CA,...
Fix 'hvparams' of '_InstanceStartupMemory' on hypervisors
Most hypervisors were calling '_InstanceStartupMemory' but not passingthe 'hvparams' keyword argument. Actually, it is not necessary topass this argument given that it is an attribute in the instance...
Merge branch 'stable-2.8' into stable-2.9
Run drbdsetup syncer only on network attach
As late as DRBD 8.3.11, the drbdsetup syncer command has a bug causingnodes to hang from time to time, requiring manual intervention to fix.The use of the command cannot be avoided, but the incidence of use can...
Add correct locking of master node to gnt-debug delay
The gnt-debug delay command required locks for all nodes except themaster - this patch fixes the issue by adding master to the lockswhenever needed.
Add job id type assert to jqueue.py
While the changes introduced in previous patches should stop any jobid parameters reaching the queue as strings, add an assertion here tocatch any strings making it through.
Add job id transformation/check to Luxi Python client
This patch adds checks to the Luxi client, making sure that job idsare converted from strings to ints before being passed on, or that anerror is reported.
query: fix detection of master in _GetNodeRole()
Commit 1c3231aa changed the invocation of _GetNodeRole() to pass themaster node by UUID and not by name, but didn't change theimplementation to compare the nodes by name. As a result, the masternode (which is also a master candidate) would always fall through to the...
Move vcluster-related constants to Constants.hs
...as, in that way, they will also be available in Haskell,where job replication happens as well.
Fix KVM cdrom image URL in the second cdrom drive
kvm: check that the ISO image is there if it's a URL
Perform a simple urllib2 check on ISO images specified as URL beforeinstance start, so as to work around qemu bug #597575 [1].
[1] https://bugs.launchpad.net/qemu/+bug/597575
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>...
Allow KVM to boot from HTTP
New versions of KVM support booting from HTTP-hosted ISO images, vialibcurl. This patch adds a proper check to allow defining either a sane,absolute path or an HTTP URL as an iso image path.
Remove "format=raw" from the cdrom device options when iso_image starts...
Include target node in hooks nodes for migration
In case of DRBD, hooks run on both primary (source) and secondary(target) nodes. To get the same behavior for DTS_EXT_MIRROR, where wedo not have secondary node, we should explicitly add target node tohooks nodes during instance migration/failover....
Reset version-specific downgrades
As stable-2.11 has been branched off and revisions bumped, downgradesare now supposed to go to 2.11. Currently they are a no-op and, inparticular, ssl-certificates are still expected in 2.11.
Make max_running_jobs queryable
As we have introduced a new cluster parameter, it shouldbe also visible when querying about the cluster configuration.
Add a command-line parameter for max_running_jobs
...so that this opcode parameter can become available for 'gnt-cluster modify'.
Add opcode parameter for the maximal number of running jobs
This parameter of OpClusterSetParams will allow to set themaximal number of jobs to be run simultaneously.
Add parameter max_running_jobs to the cluster configuration
This cluster-wide parameter will determine how many non-finalized jobs maximallyshould be in a not queued state at the same time.
Simplify 'GetMasterInfo' RPC
RPC 'GetMasterInfo' returns several fields, namely, 'master_netdev','master_ip', 'master_netmask', 'master_node', and 'primary_ip_family',of which only the 'master_node' is actually used.
Add certificate of auto-promoted master candidates to map
When a normal node is auto-promoted to be a mastercandidate, its SSL client certificate digest needsto be added to the map of candidate certificatesas well.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Hook KVM hypervisor with KVM daemon shutdown files
User shutdown hypervisor parameter
Add user shutdown parameter for KVM. Based on this parameter, decidewhat information to report for a KVM instance, for example,distinguish between 'ADMIN_down' and 'USER_down'.
Add helper function to tell if a daemon is alive
Add helper function 'utils.IsDaemonAlive' to tell if a daemon is aliveby name. This function will be necessary for the KVM hypervisor todetermine if the KVM daemon is running and otherwise start it.
Fix docstring for 'AsyncStreamServer'
Remove deprecated _ERROR_DATA_KEY in QMP
Commit de253f14 of QEMU repo "BREAKS QMP's compatibility forthe error response" as it removes "data" key from qmp errorresponse messages. To this end we only log "class" and "desc" values of the message.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Add utility to compare versions
This will be needed, e.g., for post-upgrade task, as theyhave to decide whether a feature was not yet present atthe version started from.
Merge branch 'stable-2.10' into master
Run postupgrade hook after upgrade
To allow for necessary last-moment adaptions, of the new cluster,we run the post-upgrade hook of the target version, providingthe version we originally started from.
Provide path to post-upgrade
Also add the current version to the intent-to-upgrade file
Our design states, that the intent-to-upgrade file contains "the currentversion of ganeti, the version to change to, and the process ID". Make theimplementation fit with that design.
admin.rst: update and reword disk template section
The disk template section was not updated for Gluster. This commitalso refactors the section slightly by unifying the different remarksabout /etc/ganeti/file-storage-paths.
sphinx_ext is also changed in order to not hardcode too much...
Remove certification on 2.11 to 2.10 downgrade
While version 2.10 ignores any leftover client certificates, theirpresence will prevent a the cluster working after an upgrade backto version 2.11 again. So we have to remove them right at thedowngrade.
Add support for version-specific downgrade tasks
Upgrading can have no specific knowledge about additionaltasks besides upgrading the configuration, as upgrades needto be able to go to any future version (within the same majorversion). Downgrading, however, is version specific and always...
Improve backwards compatibility of Issue 649 fix
Commit e6e4ff4cf8d0100f331f94f7a27aa1e03a5d0e7d fixed Issue 649 by switching theseparator for usb_devices from comma to space. That solved the problem withthe command line, but RAPI was able to work with commas too, so, for backwards...
Correct exception when ssconf file does not exist
After an upgrade to 2.11, the ssconf file for the mastercertificates might not exist. Based on the non-existance,noded falls back to a compatibility mode regarding dealingwith SSL certificates. The check for the ssconf file...
Create client certificate for normal nodes
The vcluster QA revealed a bug in the SSL certificatehandling code, where certificates were only createdwhen the node is a master-candidate. However, every nodeshould have a certificate, but only the digests of the...
Change usb_devices separator to whitespace
The usb_devices parameter was using comma as a list separator, but this cannotwork because comma is already used as the hypervisor parameter separator.
Change it to use whitespace as a separator, in accordance to what already done...
Verify client certificates
This patch adds a step to 'gnt-cluster verify' to verifythe existence and validity of the nodes' clientcertificates. Since this is a crucial point of thesecurity concept, the verification is very detailed withexpressive error messages and well tested by unit tests....
Verify incoming RPCs against candidate map
From this patch on, incoming RPC calls are checked againstthe map of valid master candidate certificates. If no mapis present, the cluster is assumed to be inbootstrap/upgrade mode and compares the incoming call...
Handle promoting/demoting nodes wrt to client certificates
This patch makes Ganeti correctly handle the clientcertificates when nodes get promoted to master candidatesor demoted to normal nodes.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Hrvoje Ribicic <riba@google.com>
Extend RPC call to create SSL certificates
So far the RPC call 'node_crypto_tokens' did only retrievethe certificate digest of an existing certificate. Thiscall is now enhanced to also create a new certificate andreturn the respective digest. This will be used in various...
Create client SSL certificates on cluster init
This patch makes Ganeti create a client SSL certificate forthe master node on cluster initialization. Note that some ofthe code in this patch is later moved into an LU to serverequirements for crypto renewal and updates, but for this...