Hotplug: KVM support
Hotplug will not be supported for: - KVM < 1.0 - existing devices in the cluster - python-fdsend module is not installed (NIC hotplug) - chroot (Disk hotplug) - security mode other than None (Disk hotplug)If no hotplug takes place modifications will take place after reboot....
Hotplug: objects and constants additions
Add pci slot in NIC/Disk objects. This slot will be used onlyby hypervisor code. Currently only KVM will use it and store ittemporarily in runtime files.Add HOTPLUG_* constants to define device types an hotplug actions....
Extend gnt-instance show to show vlan parameter
Extensions for "gnt-instance show" to output the vlan parameter whichwas introduced by my recent patch series.This is done by extending the nic object that is used to output theinfo. Every other occurrence of this object is adopted as well....
Warn instead of crashing on preexisting wrong data type
Using None in a VTYPE_MAYBE_STRING is wrong, because it cannot reallybe serialized to a Haskel Nothing, but given that this can (and does)influence external components already using None, it is turned to a...
Fix bug in NIC default configuration
The NIC configuration used the python "None" instead of a proper Nothing.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Support proper encoding of Nothing in constants file
The Nothing value of a Haskell Maybe needs to be properly encoded.If a Python "None" is used, it is not going to be handled properly when theresulting JSON is decoded by Haskell.
This patch adds support for proper "Nothing" encoding....
Fix KeyErrors with network mode and vlan
This patch fixes some KeyErrors that were introduced by the vlan implementationand showed up when certain parameters were missing:gnt-cluster init without network modegnt-isntance add without vlan
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>...
Merge branch 'stable-2.9' into master
Set default OVS in cluster init
This commit adds functionality to gnt_cluster to set the defaultmaster-netdev for openvswitch, when openvswitch mode is configured,instead of DEFAULT_BRIDGE.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Support VLAN parameter during instance creation
If VLAN parameter is present, check its format:either .<id>[:id], :id[:id...], or just id.
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add VLAN-capabilities for Xen Hypervisor
This implementation depends on vif-openvswitch provided by Xen 4.3and provides Xen with switch name and vlan as "link".
Extending nicparams to support vlan parameter
This parameter is used to store the vlans each nic is connected to.It is only used if the network is in openvswitch mode.
Merge branch 'stable-2.8' into stable-2.9
Rename queryd to luxid
As queryd will, in the future, handle all LUXI request, queue jobs andmost likely perform various other tasks, it is renamed to luxid already.This will safe some headache when upgrading Ganeti installations, as wedon't have to deal with a daemon rename....
Fix deprecation warning about exception
This fixes a deprecation warning regarding the retrievalof the error message from an exception.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Move bdev unit tests to filestorage unit tests
Some code was moved from bdev to file storage inprevious patches, but the related unit tests wereaccidentally left in bdev. Moving and adjustingthem now.
Signed-off-by: Helga Velroyen <helgav@google.com>...
GenerateDiskTemplate: Proper check for valid disk templates
This patch makes an FileStorageError to be risen with aproper error message instead of just an assertion in casethe given disk template is not a valid disk template atall.
Add queryd daemon (split from confd)
queryd is added as a new daemon which handles configuration queries overLUXI. This functionality was removed from confd, which now only queriesover the network.
The queryd user is added to the master group such that it can access...
Fix parsing of drbdsetup show output for DRBD 8.4
DRBD 8.4 contains two `disk` entries under the `volume` section. In somecircumstances, the parsing code could wrongly use the empty one whichcauses DRBD8Dev._SlowAssemble() to fail. This fixes issue 516....
Improve debugging output of DRBD8 status
Signed-off-by: Brian Candler <b.candler@pobox.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix conflicts check in LUNetworkDisconnect
The function _NetworkConflictCheck expects a list of instances aslast argument, not a list of pairs of instance names and instances.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Merge branch 'stable-2.8' into master
Merge branch 'stable-2.7' into stable-2.8
Conflicts:...
Change method dispatch in ClientOps to enforce luxi.REQ_ALL
ClientOps' handle_request dispatches on the luxi request received. Changethis to first verify if the request is luxi.REQ_ALL. In this way, we catchprogramming errors introducing "secret" luxi requests earlier. This is relevant,...
Verify file storage path
This patch adds two verification steps to 'gnt-clusterverify':- The configured file storage directory is checked against the allowed file storage directories file.- We check whether the configured file storage directory is existing and writable on each node....
Prepare verification code for new file path verification
This patch prepares the verification code for addinga new verification step for the file storage paths:- It moves a couple of file storage helper functions from bdev to filestorage (since they make more sense there...
Remove file storage path from autoconf
This patch removes the option to configure the filestorage path at configure time, because it is nowreplaced by a runtime option for gnt-cluster init.
storage utils: read file storage dir from cluster config
This patch make the storage utils read the file storagedirectory from the cluster config instead of the constants.
Remove ENABLE_FILE_STORAGE completely
This patch removes the ENABLE_FILE_STORAGE completelyfrom the remaining files, such as Makefile, configure,constants, pathutils and objects.
Opcodes: move storage type checks to cmdlib
So far, there was a static check whether or not a givenstorage type of an opcode was enabled. This check isno longer possibe at this layer, because the informationwhether or not file/shared file storage is enabled must...
cmdlib: remove usage of ENABLE_FILE_STORAGE
This patch removes all usages of the constant'ENABLE_FILE_STORAGE' from the cmdlib code andreplaces them by checking the cluster's configuration.
backend: remove ENABLE_FILE_STORAGE
This patch removes the usage of the ENABLE_FILE_STORAGEconstant in the backend code. To avoid having to passit through various RPC calls, we instead move the checkto cmdlib.
cluster init: check for enabled disk templates
The purpose of this patch is to remove the usage of the'ENABLE_FILE_STORAGE' constant. To get there, we do somerefactoring, add unit tests and add a FIXME for a forgottentest regarding the file storage location....
cluster verify: adjust path verification
This patch adjusts the verification of file storage pathsby checking the list of enabled disk templates insteadof the 'ENABLE_FILE_STORAGE' constant.
Utility functions to check if a disk template is enabed
This patch adds some helper functions to the config andstorage utils which check whether a disk template isenabled or not. The functions themselves are quite smallbut they will be used quite often and therefore should...
Add default file storage path to path utils
This patch adds the default storage path to the pathutilsfor now. At the end of this patch series, it will no longerbe determined at configure time and therefore be removedfrom the configure.ac file.
Allow modify_etc_hosts to be changed
The modify_etc_hosts options, enabling the cluster to modify the /etc/hostsfiles of nodes, and to keep them in sync, could only be set at cluster inittime.
With this commit it can now be changed through modify_etc_hosts as well....
Add --modify-etc-hosts option for CLI tools
The option will be used by gnt-cluster modify in the following commit.
Add luxiReqQueryNetworks to LuxiOp
When the QueryNetwork was introduced as a method, apparentlyit was forgotten in the Haskell world. Add it here as well.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Expose bulk parameter for GetJobs in RAPI client
This patch exposes the bulk argument of the jobs resource onthe RAPI python wrapper, making it possible to retrieve statusinformation about all jobs with a single call.
Signed-off-by: Leon Handreke <lhandreke@google.com>...
Fix RAPI to include missing network fields
Fix RAPI interface to include missing network fields. Specifically, fixinstance queries to include the network name for instance NICs. Also fixnetwork queries to include missing common fields, like uuid and serial....
Add support for querying network timestamps
Add creation and modified timestamps when creating a new network, andextend the available query fields for networks with these fields,namely 'ctime' and 'mtime'.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>...
Factor out check for storage params
This patch factors out the sanity checks for storageparameters in its own function. Unit tests included.
Tighten sanity checks for '_GetLvmVgSpaceInfo'
This patch tightens the sanity checks for the input of'_GetLvmVgSpaceInfo' and provides unit tests for it.
More sanity checks for spindle space reporting
This patch adds a wrapper around the space reportingfor spindles which includes more sanity checks forthe input. Unit tests provided.
Unit tests for _GetVgSpindlesInfo
This patch provides unit tests for _GetVgSpindlesInfo.In order to mock the used function for the 'bdev' moduleI made the low-level storage function an optional parameterof _GetVgSpindlesInfo. I also found out that mocked...
Unit tests for _GetVgInfo
This patch provides unit tests for the backend's_GetVgInfo function. In order to mock the underlyingbackend function, it was necessary to make it anoptional parameter of the function.
Unittesting for instance_storage._CheckNodesFreeDiskOnVG
This patch refactors the '_CheckNodesFreeDiskOnVG' ofcmdlib.instance_storage to make it at least a littlemore testible and provides the unit tests for it.
Add function for checking file access permissions
The CanRead function checks whether a user of the local machine (specifiedby name) can access a given file.
IsUserInGroup is a helper function for CanRead, but might also be usedindependently, so its name does not begin with an underscore....
Cluster verify checks server.pem permissions
Currently, ConfD must be able to access server.pem (though this is likely tochange in the future). If this is not true, all sorts of weird things happen,such as "gnt-node list" printing lots of question marks instead of actual...
Fix type error in the call of _NetworkConflictCheck
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Conflicts: NEWS: trivial configure.ac: trivial
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Fix typo in iallocator
Fix propagation of storage parameters to/from backend
This patch fixes two problems with the storage reportingthat showed up in the QA for exclusive storage:
- The processing of storage space information for instance operations wrongly assumed that the volume group's...
Fix gnt-network info
QueryNetworks() returns the uuids of the instances that areinside a network. Since QueryInstances() uses names and not uuidsfor search, we create a uuid2name dict to get each instance nameand then retrieve its nics and networks.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Improve handling of storage info
This patch improves the processing of storage informationas result of the 'node_info' call in the following way:- It removes the override for LVM storage. This was needed in MakeLegacyNode info to be compatible to the...
Let instance creation fall back to default disk template
So far, the '-t' option of 'gnt-instance add' is mandatory.This patch makes it optional. In case, no disk template isspecified, the first of the list of enabled_disk_templatesis used.
LUInstanceCreate: factor out disk checks
The 'CheckArguments' method of 'LUInstanceCreate' is waytoo long and should be broken up in smaller pieces. Thispatch starts with that and factors out the disk checks.No functional changes whatsoever.
Make DRBD replication protocol configurable
Add a new disk parameter which allows to choose the DRBD replicationprotocol. The protocol is fixed to "C" during dual primary mode though.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make names more descriptive
LUGroupVerifyDisks had some not very descriptive names which made ithard to read and understand the code. This patch renames some methodsand variables to make it more readable.
Also, the return type of MapInstanceLvsToNodes was changed to contain...
Check DRBD status on verify-disks
Enhance `gnt-cluster verify-disks` such that it checks the DRBD statusof all disks. If the status of at least one disk of an instance iseither StandAlone or Diskless, the instance disks are activatedautomatically.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
noded.py: remove fallback of 'node_info'
In order to change backend and frontend code of the RPCcall 'node_info' in separate patches, we included afallback to the old implementation. Since all RPC callersare adjusted, this can be removed now.
Extend RPC call 'node_info' by storage parameters
This patch extends the actual RPC call to accept storageparameters with each storage units. It adjusts all codewhich performs this code by integrating the 'exclusivestorage' flag into the lvm storage unit....
Add general storage parameters to node info call
As described in the design doc about storage types,we plan to generalize the RPC call "node info" wrt tostorage types. This patch extends the call by acceptinga list of storage units, that is not only identified...
Export CPUs used by the node OS
They are exported through the LUXI, RAPI, and IAllocator interfaces.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix docstrings for hypervisor functions returning node info
The documentation was out-of-sync with the code.
Hypervisors report CPUs used by the node OS
The XenHypervisor's "dom0_cpus" parameter (introduced in 1d60fec6 but neverused) is renamed to "cpu_dom0" for consistency with other parameternames. The analogous information is returned for the other hypervisors....
Don't exceed the standard PV size when growing a disk
With exclusive storage, PVs are supposed to be used only up to thestandard PV size.
Enable disk growth with exclusive storage
Disk growth is now supported also with exclusive storage, so we let theoperation continue.
Enable disk growth test in qa also with exclusive storage.
Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions fordisk growth. LVM volumes handles the flag by reusing the same PVs, as disksare constrained by the number of their spindles....
Add exclusive_storage to blockdev_grow RPC
The flag is passed to the RPC, but it's still ignored by noded.
Make NIC setup more flexible in case of Xen
For every NIC write down a file that contains NIC'sinfo (MAC, mode, link, network details), that canbe sourced by any vif script. The file location is:/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>....
iallocator: storage processing accepts space_info structure
This is a small refactoring to make the processing ofthe storage space information not accept a node inforesult, but the more specific space info structure instead.
iallocator: get rid of MakeLegacyNodeInfo
So far, the iallocator makes a node info call, convertsthe result into a legacy node info structure and thenretrieves various values from it. This patch removesthe call to MakeLegacyNodeInfo completely, by letting the...
iallocator: factor out instance memory summation
The iallocator adjusts the free memory by a calculationof how much memory is actually currently used by instanceson the node. This calculation is rather complicated anddeserves its own function to increase testability. No...
iallocator: extract storage info directly from node info
So far, the iallocator made a node info RPC call, convertedthe result into a legacy node info format and then extractedthe storage information from that. This patch removes themiddle step of this procedure, skipping the legacy node info...
iallocator: computation of storage data
This patch extracts the processing of the storage-relateddata from the RPC call 'node_info' in a separate functionto increase testability. In addition, this patch containsunit tests for this function.
rpc.py: override default storage type in node info
This patch changes the semantics of dealing with the resultof the RPC call "node_info" when converting them into aLegacyNodeInfo. A legacy node info, can only hold storageinformation about one storage type, which so far was always...
iallocator: factor out 'get_attr' method
This patch extracts an inline 'get_attr' method to extractdata from the result of the RPC call 'node_info'. It was sofar an inline function in the middle of a bigger method, which madethe code hard to read.
iallocator: prepare RPC call 'node_info'
This patch does a small refactoring in the allocator. It extractsthe preparation and actual call of the 'node_info' call into aseparate function. This is done for better testability. No functionalchanges otherwise....
Disable pylint E1101 on hypothetical import
In some versions of python the module IN does not providethe constant SO_PEERCRED (issue 191). So in commit 069a4ba work around was added. However, this work around stillleaves lint warnings when working with those versions of...
Provide the right fix for the kvm_extra parameter
With the previous adjustment the arguments were boxed into one extraarray. This actually makes the parameter work (with the same caveats).
The manpage is also updated with the limitations.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
kvm: don't pass whole extra as a single argument
If extra is longer than a single word passing it the way it was passedwon't work. Of course this solution is also only partial, as won't takeinto account any form of quoting. :/
gnt-cluster info (py): add enabled disk templates
This fixes issue 485. In the python implementation ofthe cluster config info, the enabled_disk_templateswere missing.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
grow disk: call SetDiskID before blockdev_getsize
Missing this call caused blockdev_getsize to be called on the wrongphysical device, and fail.
This fixes Issue 497.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Catch DeviceCreationError unhandled exceptions
_CreateBlockDevInner is called twice in TLReplaceDisks. While thisshould be fixed, right now this leaves the DeviceCreationError exceptionunhandled, which causes a problem due to the fact that this exception...
Fix permissions of the queue archive dir
The permissions were set to 0740, but of course being a directory thecorrect permissions are 0750.
Index instances by their UUID
No longer index instances by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the...
Node-UUID related cleanup
Some variables which hold node UUID's are renamed to reflect theirindent better.Use cfg.GetNodeName(uuid) instead of cfg.GetNodeInfo(uuid).name whereappropriate.
Fix wrong assert
An assert wrongly verified that the node name instead of the node UUIDis in the set of locked nodes. This patch fixes this.
Fix instance add for missing node specs
Don't pass None to cfg.GetNodeNames. Instead, only resolve the nodenames if it's sure that the UUID list is valid.
This fixes issue 495.
Add support to migration_mode in fake HV
The fake hypervisor pretends to support migration, but did not supportthe migration_mode parameter yet. As this parameter is required duringinstance migrations, support for it is added in this patch.
Check DRBD status in cluster verify
If the status of DRBD disks is degraded or in a faulty state, issue anerror in cluster verify.
Fix reference to NodeInfo.name
This fixes a bug introduced during the move from node names to nodeUUIDs as indexes.
Also, a typo is corrected.
Merge branch 'stable-2.8' into 'master'
Refactor NodeInfo RPC regarding storage reporting
The NodeInfo RPC call is refactored to handle now more thanjust storage reporting for volume groups.
Since NodeInfo now returns storage space information notnecessarily for volume groups, but also for other storage...