Improve handling of storage info
This patch improves the processing of storage informationas result of the 'node_info' call in the following way:- It removes the override for LVM storage. This was needed in MakeLegacyNode info to be compatible to the...
Let instance creation fall back to default disk template
So far, the '-t' option of 'gnt-instance add' is mandatory.This patch makes it optional. In case, no disk template isspecified, the first of the list of enabled_disk_templatesis used.
Signed-off-by: Helga Velroyen <helgav@google.com>...
LUInstanceCreate: factor out disk checks
The 'CheckArguments' method of 'LUInstanceCreate' is waytoo long and should be broken up in smaller pieces. Thispatch starts with that and factors out the disk checks.No functional changes whatsoever.
Make DRBD replication protocol configurable
Add a new disk parameter which allows to choose the DRBD replicationprotocol. The protocol is fixed to "C" during dual primary mode though.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make names more descriptive
LUGroupVerifyDisks had some not very descriptive names which made ithard to read and understand the code. This patch renames some methodsand variables to make it more readable.
Also, the return type of MapInstanceLvsToNodes was changed to contain...
Check DRBD status on verify-disks
Enhance `gnt-cluster verify-disks` such that it checks the DRBD statusof all disks. If the status of at least one disk of an instance iseither StandAlone or Diskless, the instance disks are activatedautomatically.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
noded.py: remove fallback of 'node_info'
In order to change backend and frontend code of the RPCcall 'node_info' in separate patches, we included afallback to the old implementation. Since all RPC callersare adjusted, this can be removed now.
Extend RPC call 'node_info' by storage parameters
This patch extends the actual RPC call to accept storageparameters with each storage units. It adjusts all codewhich performs this code by integrating the 'exclusivestorage' flag into the lvm storage unit....
Add general storage parameters to node info call
As described in the design doc about storage types,we plan to generalize the RPC call "node info" wrt tostorage types. This patch extends the call by acceptinga list of storage units, that is not only identified...
Export CPUs used by the node OS
They are exported through the LUXI, RAPI, and IAllocator interfaces.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix docstrings for hypervisor functions returning node info
The documentation was out-of-sync with the code.
Hypervisors report CPUs used by the node OS
The XenHypervisor's "dom0_cpus" parameter (introduced in 1d60fec6 but neverused) is renamed to "cpu_dom0" for consistency with other parameternames. The analogous information is returned for the other hypervisors....
Don't exceed the standard PV size when growing a disk
With exclusive storage, PVs are supposed to be used only up to thestandard PV size.
Enable disk growth with exclusive storage
Disk growth is now supported also with exclusive storage, so we let theoperation continue.
Enable disk growth test in qa also with exclusive storage.
Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions fordisk growth. LVM volumes handles the flag by reusing the same PVs, as disksare constrained by the number of their spindles....
Add exclusive_storage to blockdev_grow RPC
The flag is passed to the RPC, but it's still ignored by noded.
Make NIC setup more flexible in case of Xen
For every NIC write down a file that contains NIC'sinfo (MAC, mode, link, network details), that canbe sourced by any vif script. The file location is:/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>....
iallocator: storage processing accepts space_info structure
This is a small refactoring to make the processing ofthe storage space information not accept a node inforesult, but the more specific space info structure instead.
iallocator: get rid of MakeLegacyNodeInfo
So far, the iallocator makes a node info call, convertsthe result into a legacy node info structure and thenretrieves various values from it. This patch removesthe call to MakeLegacyNodeInfo completely, by letting the...
iallocator: factor out instance memory summation
The iallocator adjusts the free memory by a calculationof how much memory is actually currently used by instanceson the node. This calculation is rather complicated anddeserves its own function to increase testability. No...
iallocator: extract storage info directly from node info
So far, the iallocator made a node info RPC call, convertedthe result into a legacy node info format and then extractedthe storage information from that. This patch removes themiddle step of this procedure, skipping the legacy node info...
iallocator: computation of storage data
This patch extracts the processing of the storage-relateddata from the RPC call 'node_info' in a separate functionto increase testability. In addition, this patch containsunit tests for this function.
rpc.py: override default storage type in node info
This patch changes the semantics of dealing with the resultof the RPC call "node_info" when converting them into aLegacyNodeInfo. A legacy node info, can only hold storageinformation about one storage type, which so far was always...
iallocator: factor out 'get_attr' method
This patch extracts an inline 'get_attr' method to extractdata from the result of the RPC call 'node_info'. It was sofar an inline function in the middle of a bigger method, which madethe code hard to read.
iallocator: prepare RPC call 'node_info'
This patch does a small refactoring in the allocator. It extractsthe preparation and actual call of the 'node_info' call into aseparate function. This is done for better testability. No functionalchanges otherwise....
Merge branch 'stable-2.8' into master
Merge branch 'stable-2.7' into stable-2.8
Provide the right fix for the kvm_extra parameter
With the previous adjustment the arguments were boxed into one extraarray. This actually makes the parameter work (with the same caveats).
The manpage is also updated with the limitations.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
kvm: don't pass whole extra as a single argument
If extra is longer than a single word passing it the way it was passedwon't work. Of course this solution is also only partial, as won't takeinto account any form of quoting. :/
gnt-cluster info (py): add enabled disk templates
This fixes issue 485. In the python implementation ofthe cluster config info, the enabled_disk_templateswere missing.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
grow disk: call SetDiskID before blockdev_getsize
Missing this call caused blockdev_getsize to be called on the wrongphysical device, and fail.
This fixes Issue 497.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Catch DeviceCreationError unhandled exceptions
_CreateBlockDevInner is called twice in TLReplaceDisks. While thisshould be fixed, right now this leaves the DeviceCreationError exceptionunhandled, which causes a problem due to the fact that this exception...
Fix permissions of the queue archive dir
The permissions were set to 0740, but of course being a directory thecorrect permissions are 0750.
Index instances by their UUID
No longer index instances by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the...
Node-UUID related cleanup
Some variables which hold node UUID's are renamed to reflect theirindent better.Use cfg.GetNodeName(uuid) instead of cfg.GetNodeInfo(uuid).name whereappropriate.
Fix wrong assert
An assert wrongly verified that the node name instead of the node UUIDis in the set of locked nodes. This patch fixes this.
Fix instance add for missing node specs
Don't pass None to cfg.GetNodeNames. Instead, only resolve the nodenames if it's sure that the UUID list is valid.
This fixes issue 495.
Add support to migration_mode in fake HV
The fake hypervisor pretends to support migration, but did not supportthe migration_mode parameter yet. As this parameter is required duringinstance migrations, support for it is added in this patch.
Check DRBD status in cluster verify
If the status of DRBD disks is degraded or in a faulty state, issue anerror in cluster verify.
Fix reference to NodeInfo.name
This fixes a bug introduced during the move from node names to nodeUUIDs as indexes.
Also, a typo is corrected.
Merge branch 'stable-2.8' into 'master'
Refactor NodeInfo RPC regarding storage reporting
The NodeInfo RPC call is refactored to handle now more thanjust storage reporting for volume groups.
Since NodeInfo now returns storage space information notnecessarily for volume groups, but also for other storage...
Utility functions for storage types
Handling various storage types for the free space reportingrequires some utility functions. They will be invoked inlater patches. Unit tests are included.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Integrate free space reporting for file storage
This patch integrates the free space reporting for filestorage. So far, it is only called, when the defaultdisk template is a file-based disk template ("file" or"sharedfile").
Renamining space reporting for file storage
Before being integrated, the function for space reportingfor file storage is renamed to a more expressive name.
Add storage type to NodeInfo result
So far, the storage information returned from RPC callNodeInfo contained only information about volume groups.In order to extend the storage reporting to other storagetype, we include another field "type" in the result of...
Use cluster's hvparams in LUInstSetParams
Since the instance does not have hvparams at that point,we have to use the cluster's hvparams here. This fixesa crash when using 'gnt-instance modify'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Inline simple "alias" variables
Remove variables in the form "var = self.var" throughout all LU's. Those"alias" variables make it harder to read the code (it's no longerimmediately clear what's a field and what's a local variable), and canintroduce subtle bug (especially when not used consistently and when...
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the ConfigWriter, including all methods....
Add cluster name to instance migration RPC
This fixes a 'FIXME' in the xen hypervisor. The instancemigration requires the cluster's name. This should comefrom the RPC call, but so far was hackishly taken fromssconf. Ssconf is intended to be used by scripts only and...
Support VNC binding to an interface
Currently, the VNC bind address can only be specified as an IPaddress. Given that this is a cluster-wide parameter, therearen't many sensible choices besides 127.0.0.1 and 0.0.0.0. Thispatch allows to also specify it as the address of a specifed...
Remove XEN_CMD from constants, adjust to PowercycleNode
This patch extend the RPC to powercycle a node by ahvparams parameter. This parameter is used by thexen hypervisor to determine which xen command to use.The respective LU is adjusted as well.
Since this was the last operation depending on the...
hv_xen.py: make hvparams mandatory, remove fallbacks
This patch removes the fallback to the xen command from theauto config and adds an exception instead. Also, thehvparams parameter of the functions GetCommand, _RunXen and_GetInstanceList are made mandatory....
Use hvparams in GetAllInstancesInfo
This patch extends the GetAllInstancesInfo method of thehypervisors by a hvparams parameter. This change propagatesup to the RPC call all_instances_info, which also neededto be extended. All callers of this call were adjusted...
Use hvparams in GetInstanceInfo
This patch extends the GetInstanceInfo function of thehypervisors by an hvparams parameters. This leads toan extension of the instance_info RPC call and theadjustmend of all classes using this call.
backend.py: remove fallback implementation of GetNodeInfo
This removes the fallback implementation of the backend'sGetNodeInfo function to the old implementation. By now,all calling code is adjusted to the new hvparams parameterand thus, the old implementation should not be necessary...
Adjust all callers of RPC 'call_node_info'
This patch contains all adjustments necessary to code thatuses the 'call_node_info' call. The adjustment is necessarybecause the call's signature changed from receiving a listof hypervisor names to a list of tuples...
Add hvparams to RPC call 'node_info'
This patch adds the hvparams parameter to the RPC call'node_info'. It also adjusts the related code in noded.pyand Query/Node.hs
backend.py: use hvparams in GetNodeInfo
This patch extends the GetNodeInfo function of the backendmodule by a hvparams parameter. For now, it contains afallback solution for when the parameter is None. Thisfallback will be removed in a later patch of this series....
hypervisors: add hvparams to GetNodeInfo
This patch extends the GetNodeInfo function of thehypervisors by a hvparams parameter. The parameteris currently only used by the xen hypervisor to determinewhich xen toolstack ('xm' or 'xl') to use.
hv_xen.py: renaming residual 'xm' occurrences
Some methods still refered to 'xm', which is inaccurate nowthat xen can also be run with 'xl'. No functional changesotherwise.
Watcher: Use hvparams when listing instances
This patch makes the watcher use the hvparams from ssconfto retrieve the list of instances from the hypervisors.
Include hvparams in ssconf files
The watcher needs access to the cluster's hypervisorparameters. This patch extends the ssconf infrastructureby adding a hvparams file for each hypervisor. The formatof the hvparams file is a simple key=value format.
Use hvparams in instance migration
This patch makes use of hvparams on instance migration todetermine the list of instance before the migration. This way, inxen, the xl/xm choice is respected.
Verify: node info and instance list
This patch enhances node verification by:- Factoring out the verification of the node info and the instance list to increase testability.- Make both of them use the hvparams parameter.
Verify: xen toolstack, hypervisor and hvparams
This patch extends the node verification by:- Adding a check for the xen toolstack when the hypervisor is verified.- Factoring out the hypervisor verification in a subfunction to increase testability.- Factoring out the hvparams verification in a...
rpc / noded: add hvparams to instance_list call
The 'instance_list' call needs to be extended to accepthvparams. This is one of the few RPCs where it isnecessary, because there is no instance given whosehvparams could be reused.
backend.py: use instance's hvparams whenever possible
The backend module's function 'ListInstances' is calledwithin the module several times. In many cases, it ispossible to reuse the instance's hvparams for the'ListInstances' call and thus avoiding having to extend...
backend.py: GetInstanceListForHypervisor
In same cases, the call 'GetInstanceList' is only done forone hypervisor. In those cases, it is more convenient tonot provide the full collection of hypervisor parametersfor all hypervisors, but only the ones for this hypervisor....
backend.py: hvparams in 'GetInstanceList'
Propagating the use of hvparams further up the callinghierarchy, this patch makes the backend module use thehvparams in 'GetInstanceList'. Unit tests provided.
hv_xen.py: _RunXen and GetInstanceList use hvparams
Propagating the use of hvparams further up in the callinghierarchy, the functions '_RunXen' and 'GetInstanceList'use the given hvparams. Unit tests provided. Calling function'StopInstance' adjusted in this patch as well....
hv_xen.py: rename RunXmList to RunInstanceList
Since one can retrieve xen's instance list with both, 'xm'or 'xl', the function name 'RunXmList' is no longerappropriate. Renaming it to 'RunInstanceList'. Nofunctional changes otherwise.
hv_xen.py: rename GetXmList to GetInstanceList
Since one can retrieve xen's instance list via both 'xm'or 'xl', it is no longer appropriate to call the function'GetXmList'. This patch renames it to 'GetInstanceList'.Other than that, there are no functional changes....
hv_xen._GetCommand: retrieve xen command from hvparams
This patch adds a (so far optional) hvparams parameterto the '_GetCommand' function. This dictionary is usedto retrieve the xen command (xm or xl). It is optionalfor now to make the refactoring possible without breaking...
Add xen command (xm, xl) to hv parameters
This patch adds the xen command (xm, xl) as hypervisorparameter to the constants and adds validation tothe xen hypervisors.
Make SubmitOpCode expect options without print_jobid
As opposed to SubmitOrSend, the SubmitOpCode function iscalled also by commands not supporting submit options. Henceonly inspect these options, if they are actually present.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Add an option to print job id of submitted jobs on stdout
When submitting a job, the job id, prefixed by 'JobID: ', is output tostderr. While this is easy to understand for humans, for scripting, itis more convenient to have (additionally) the id on stdout (and only...
Add new command gnt_job wait
This command tacitly waits for a job to finish. In thatway, job dependency can also be handled in shell scripts.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Export node spindles
Node spindles (queried live) are exported through the LUXI, RAPI, andiallocator interfaces.
node_info RPC reports info about free spindles
node_info RPC can be used to get information about free/total spindles.
Use RpcResult.Warn where appropriate the cmdlib
To avoid repetitive code, make use of the functionality factoredout to the Warn method of RpcResult.
Add a Warn function to RpcResult
At various places, LU code just wants to warn abouta failed rpc, but continue nevertheless. So factor outthat common checking, message formatting, and notifying.
Add --force option to gnt-cluster modify
Make the force option of the ClusterSetParams Opcode availableon the command line.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add a force option to the ClusterSetParams Opcode
If set, the op code will, in particular, try to set the master IPon the new netdev, even if shutting down the master IP on the oldnetdev failed.
Export device UUIDs to hooks and OS scripts
Export UUIDs and names of instance NICs and disks to the environmentof OS scripts and instance related hooks.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
gnt-cluster info: only parameters for enabled hypervisors
In the output of gnt-cluster info, restrict the hypervisor parametersto those hypervisors that are enabled. This fixes issue 443.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Add constant with the path of the diskstats
This will be used by the diskstats storage collector.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Check real spindles in ipolicies
When exclusive storage is enabled, the spindles in instance disks are usedto check the instance policies (as outlined in design-partitioned.rst).
Export instance disk spindles
Instance disk spindles can be queried via LUXI and RAPI, and are exportedthrough the allocator interface. This is a prerequisite for htools tohandle spindles.
The length of a RAPI query including all the instance fields now has become...
Conflicts: doc/iallocator.rst...
kvm: Use -uuid to set system UUID
Use kvm '-uuid' option to set guest's system UUID to instance's UUID.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Ensure the queue socket has the right permissions
RAPI needs to be able to perform queries as well.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Export connected networks to IAllocator
IAllocators should not put instances on nodes in nodegroups which arenot connected to the network the instance should be connected to.Therefore, export the networks a node group is connected to to theIAllocator, so it can account for this restriction....
Add generic daemon options support binding to interfaces
In some situations, it is desirable to bind one of our servers only toa particular interface, like eth0, instead of an IP address. Therefore,add an option -i to specify an interface; its address will then be bound...
Re-activate previously active disks in watcher
The watcher process previously (incorrectly) examined the admin_statefield to figure out if the disks of an instance should be activatedafter a node reboot. This is changed to use the disks_active field of...
Make the disks_active flag queryable
gnt-instance list now also supports the disks_active field.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Consider disks_active in UpgradeConfig
The disks_active flag is updated when masterd reads the configuration.Also, cfgupgrade now removes the disks_active flag during downgrades.
Add disks_active to configuration
This flag tracks if the disks of an instace are supposed to be active.That's the case when an instance is running or when its disks gotactivated explicitly (and in a couple of other cases).It will be used by watcher to re-activate disks after a node reboot....