Factor out lv info gathering function
The buildJsonReport function will soon have to perform the coupling ofinstance data with LVInfo data. In preparation for that, in order to makeit more readable, the instructions for obtaining LVInfos are factored out...
Add "instance" field to LVInfo
Extend the LVInfo data structure with the field for storing the name ofthe instance it is paired with.
Update the tests accordingly.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Better specify LVInfo "instance" field
Specify that it is serialized as "null" if empty.
Factor out the getInstances function
The getInstances function can be useful in general, but is defined insidethe InstStatus data collector. This commit takes it out and adds it toa proper (newly created) library.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Disable failing QA test
See issue 516 for an explanation of the problem, and why it was requiredto disable this test.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Disable enabled-template-test for exclusive storage
This patch disables the tests regarding enabled disktemplates for exclusive storage, because the semanticsthere are different and not covered in the test.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Fix propagation of storage parameters to/from backend
This patch fixes two problems with the storage reportingthat showed up in the QA for exclusive storage:
- The processing of storage space information for instance operations wrongly assumed that the volume group's...
Fix typo in iallocator
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix gnt-network info
QueryNetworks() returns the uuids of the instances that areinside a network. Since QueryInstances() uses names and not uuidsfor search, we create a uuid2name dict to get each instance nameand then retrieve its nics and networks.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Extraction of storage info by type
There was a bug in the node queries. It was assumed thatthe returned storage space information was in a particularorder. With the changes in the storage reporting, thisorder is not that reliable anymore, in particular, the...
Improve handling of storage info
This patch improves the processing of storage informationas result of the 'node_info' call in the following way:- It removes the override for LVM storage. This was needed in MakeLegacyNode info to be compatible to the...
Mention default disk template in NEWS file
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add default disk template selection to man pages
This documents the newly implemented behavior of pickingthe disk template from the list of enabled disk templateson instance creation.
Mention alternative template selection in design doc
This is a small adjustment of the design doc'design-storagetypes.rst' with respect to the selectionof the default disk template.
Let instance creation fall back to default disk template
So far, the '-t' option of 'gnt-instance add' is mandatory.This patch makes it optional. In case, no disk template isspecified, the first of the list of enabled_disk_templatesis used.
LUInstanceCreate: factor out disk checks
The 'CheckArguments' method of 'LUInstanceCreate' is waytoo long and should be broken up in smaller pieces. Thispatch starts with that and factors out the disk checks.No functional changes whatsoever.
Document the 'protocol' disk parameter
Document the new disk parameter 'protocol' which is used to set the DRBDreplication protocol.
Make DRBD replication protocol configurable
Add a new disk parameter which allows to choose the DRBD replicationprotocol. The protocol is fixed to "C" during dual primary mode though.
Revert "QA: Make Query tests test for '?' values"
This reverts commitbfbef99b9f056638411aa894e300aa095132e931. The problem is thatin some cases it makes sense for '?' to be displayed. Thisis the case for storage reporting on clusters that have onlythe 'diskless' disk template enabled. It would be nicer,...
Add QA test for verify-disks with broken DRBD
gnt-cluster verify-disks supports automatic activation of broken DRBDdisks. This new QA test verifies that it actually finds instances withbroken DRBD disks and activates the disks of those instances.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Check DRBD status on verify-disks
Enhance `gnt-cluster verify-disks` such that it checks the DRBD statusof all disks. If the status of at least one disk of an instance iseither StandAlone or Diskless, the instance disks are activatedautomatically.
Make names more descriptive
LUGroupVerifyDisks had some not very descriptive names which made ithard to read and understand the code. This patch renames some methodsand variables to make it more readable.
Also, the return type of MapInstanceLvsToNodes was changed to contain...
QA: Make Query tests test for '?' values
When something goes wrong in the Haskell implementation ofqueries, the 'gnt-* list' command does not actually fail,but it just displays '?' as a sign of failure. We so fardo not test for that, which makes errors in the haskell...
Turn 'exclusive_storage' into storage parameter (hs)
This is the haskell implementation of my patch "Extend RPC call'node_info' by storage parameters". It turns the 'exclusivestorage' flag into a storage parameter of the LVM storage types.Besides that, this patch moves some types into the Types.hs....
noded.py: remove fallback of 'node_info'
In order to change backend and frontend code of the RPCcall 'node_info' in separate patches, we included afallback to the old implementation. Since all RPC callersare adjusted, this can be removed now.
Extend RPC call 'node_info' by storage parameters
This patch extends the actual RPC call to accept storageparameters with each storage units. It adjusts all codewhich performs this code by integrating the 'exclusivestorage' flag into the lvm storage unit....
Add general storage parameters to node info call
As described in the design doc about storage types,we plan to generalize the RPC call "node info" wrt tostorage types. This patch extends the call by acceptinga list of storage units, that is not only identified...
backend unit tests: remove unused imports and variables
This patch removes some unused imports and an unusedvariable of the backend unit test. No functional changesotherwise.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Prevent LV parser compile error
The LV parser is not compiled correctly by more recent versions of GHCbecause of more strict checks.
lvCommand is a surely non-empty list, but the compiler still refuses itasking for explicitly management of the empty list case....
Add htools shell test for VCPU handling
This tests that VCPUs used by both instances and node OS are correctlyaccounted in allocation.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Load CPUs used by the node OS in htools
A new field is added to the Node type, and it's used to initialize the usedCPUs field.
The signature of Node.create has been split among lines to match theparameter list.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Export CPUs used by the node OS
They are exported through the LUXI, RAPI, and IAllocator interfaces.
Hypervisors report CPUs used by the node OS
The XenHypervisor's "dom0_cpus" parameter (introduced in 1d60fec6 but neverused) is renamed to "cpu_dom0" for consistency with other parameternames. The analogous information is returned for the other hypervisors....
Fix docstrings for hypervisor functions returning node info
The documentation was out-of-sync with the code.
Add LV collector to the monitoring daemon
Allow the monitoring daemon to use the LV data collector.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add tests for the LV data collector
This commit adds a few shelltests for the logical volume data collector.
Also, it performs some cleanup of the comments already present in the file:given that the number of the tests should be changed manually, and this is...
Add LV data collector
This commit adds the LV data collector.
Also, the lvCommand function was not providing the correct value as expected bythe readProcess function, so it was fixed.
Add tests for LVs parser
Add unit tests and Quickcheck tests for the LV parser.
Also, fix the alphabetic order of some imports in htest.hs.
Add function testing parsers with QuickCheck
Parsing a string automatically generated by QuickCheck can be useful in manycases. A function for doing this is added to the common test functions.
Add function for generating arbitrary UUID
Add a function for generating arbitrary UUIDs to be used for QuickCheck tests.
Add LV parser
Add the parser for getting the information about the logical volumes in thesystem.
Add design for LV data collector
Add a section describing the logical volume data collector to the designdocument for the monitoring agent.
Minor fixes to monitoring design doc
Add a missing full stop, and fix a sentence about the storage collectors.
Fix documentation formatting issue
One line was longer than 80 characters.
Add test for the --full-evacuation option
In this example cluster, there are four pairs of nodes, each hostingtwo instances. Each node has disk capacity for an additional instance.Therefore, for normal online rolling reboots, there are two node groups,...
hroller: option --full-evacuation
Add an option to hroller, to plan for full evacuation of thenodes to be rebooted, i.e., also plan for replacement secondarynodes for all instances on the node after migrating out instanceswith this node as primary.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Extract a partition functional
Separate the partitionNonRedundant function in hroller into ageneral functional that partitions a list of nodes accordingto some clearing strategy and the specialization of movingnon-redundant instances out. In this way, we don't have to...
Extract functional for greedily clearing nodes
The method clearNodes in hroller greedily clears nodes ofnon-redundant instances by moving them to a different node. This patchseparates the greedy clearing algorithm from the specialization tonon-redundant instances; in this way, we don't have to duplicate code...
Make hroller not consider offline nodes for evacuation
When planing on where to evacuate the non-redundant instancesof the nodes to be rebooted, it doesn't make sense to consideroffline nodes. So add this restriction to hroller.
Update comments in hroller code
hroller schedules moves of instances to have rebooted nodesfree of instances with this node as primary. Update the commentsto reflect that this move planning is for non-redundant instancesonly.
Clarify meaning of --print-moves in hroller man page
When given the option --print-moves, hroller prints a list ofinstance relocations after each reboot group. This sequentialpresentation can lead to the wrong impression, that these areall the moves to be carried out. Therefore, explicitly mention...
Remove obsolete TODO
Originally, hroller started as a tool for offline maintenance only.There it made sense to warn about instances still running. By now,default planning is to migrate instance off the nodes to be rebooted,with options for other behavior (like pretending that all instances...
Allow customization of the chroot
As different users have different needs about the chroot, allowcustomization in its build script. This is achieved as follows.- The environment variable EXTRA_DEBIAN_PACKAGES is honored, and the packages named there are additionally installed in...
build_chroot: fix duplicate install
In the last change to the chroot build script, theapt-get command 'install' was moved to the APT_INSTALLmacro; however, it was forgotten to remove it outsidethe macro, causing an error, as there is no packagewith name 'install'....
build_chroot: use APT_INSTALL macro for packet installation
In this way, it is easier to change all the apt calls in the chroot.In particular, this patch fixes the missing -y option for the installationof the tools needed to build debian packages.
Don't exceed the standard PV size when growing a disk
With exclusive storage, PVs are supposed to be used only up to thestandard PV size.
Enable disk growth with exclusive storage
Disk growth is now supported also with exclusive storage, so we let theoperation continue.
Enable disk growth test in qa also with exclusive storage.
Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions fordisk growth. LVM volumes handles the flag by reusing the same PVs, as disksare constrained by the number of their spindles....
Add exclusive_storage to blockdev_grow RPC
The flag is passed to the RPC, but it's still ignored by noded.
Design doc for openvswitch support
This patch includes the design document for openvswitch support
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make NIC setup more flexible in case of Xen
For every NIC write down a file that contains NIC'sinfo (MAC, mode, link, network details), that canbe sourced by any vif script. The file location is:/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>....
Refactor NIC configuration scripts
Add new script vif-ganeti that could be used instead ofdefault vif-bridge in order NIC customization can be moreflexible.
Extract common code from kvm-ifup and insert it in a newfile net-common that is being sourced by kvm-ifup and...
Document vif-ganeti usage
In order to be able to use it as vif-script one has tocreate a symbolic link inside /etc/xen/scripts pointingto /usr/lib/ganeti/vif-ganeti and then add is a hypervisorparameter.
Update design document for hroller
The requirement to be able to plan full node evacuation came up afterthe initial design. So add a section describing the algorithm, iffull evacuation is requested.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
iallocator: storage processing accepts space_info structure
This is a small refactoring to make the processing ofthe storage space information not accept a node inforesult, but the more specific space info structure instead.
iallocator: get rid of MakeLegacyNodeInfo
So far, the iallocator makes a node info call, convertsthe result into a legacy node info structure and thenretrieves various values from it. This patch removesthe call to MakeLegacyNodeInfo completely, by letting the...
iallocator: factor out instance memory summation
The iallocator adjusts the free memory by a calculationof how much memory is actually currently used by instanceson the node. This calculation is rather complicated anddeserves its own function to increase testability. No...
iallocator: extract storage info directly from node info
So far, the iallocator made a node info RPC call, convertedthe result into a legacy node info format and then extractedthe storage information from that. This patch removes themiddle step of this procedure, skipping the legacy node info...
Make NodeInfo (hs) accept arbitrary storage types
So far, the Haskell implementation of NodeInfo justrequests storage information about volume groups.With this patch, storage info for abitrary storagetypes can be requested.
Storage utility functions for Haskell
In order to extend the Haskell version of the NodeInfoquery, we need some utility functions to deal withdisk templates and storage types.
iallocator: computation of storage data
This patch extracts the processing of the storage-relateddata from the RPC call 'node_info' in a separate functionto increase testability. In addition, this patch containsunit tests for this function.
rpc.py: override default storage type in node info
This patch changes the semantics of dealing with the resultof the RPC call "node_info" when converting them into aLegacyNodeInfo. A legacy node info, can only hold storageinformation about one storage type, which so far was always...
iallocator: factor out 'get_attr' method
This patch extracts an inline 'get_attr' method to extractdata from the result of the RPC call 'node_info'. It was sofar an inline function in the middle of a bigger method, which madethe code hard to read.
iallocator: prepare RPC call 'node_info'
This patch does a small refactoring in the allocator. It extractsthe preparation and actual call of the 'node_info' call into aseparate function. This is done for better testability. No functionalchanges otherwise....
Typo in iallocator design doc
Fix lint error
Unset LV name if no LVM disk templates are enabled
If there are no LVM based disk templates enabled in QA, reset the LVname after the corresponding gnt-cluster modify tests. Otherwisegnt-cluster verify would complain about the missing volume group.
Also make sure that no disk templates requiring LVM are ever enabled...
Skip drbd-usermode-helper test if DRBD is disabled
Changing the drbd usermode helper in gnt-cluster modify fails if DRBD isnot installed on the nodes. Thus skip this test if DRBD is not in theset of enabled disk templates.
Add route to VCluster network in vcluster-setup
This route is required on the Fedora test machine, as otherwise fpingwould not work and hang forever.
Replace tempfile by mktemp in QA code
The tempfile command only exists in Debian. In order to run the VClusterQA also on Fedora, replace its use by mktemp.
Merge branch 'stable-2.8' into master
Merge branch 'stable-2.7' into stable-2.8
Provide the right fix for the kvm_extra parameter
With the previous adjustment the arguments were boxed into one extraarray. This actually makes the parameter work (with the same caveats).
The manpage is also updated with the limitations.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
kvm: don't pass whole extra as a single argument
If extra is longer than a single word passing it the way it was passedwon't work. Of course this solution is also only partial, as won't takeinto account any form of quoting. :/
NEWS and version updates for 2.7.0~rc3
Time for a new release.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
For node queries allow short forms of host names
For node queries use the host-name filter instead of the simpleequality-based one.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Provide a special filter for host names
For host names, usually short forms are used, e.g., node1 or node1.subinstead of the full qualified node1.sub.example.com. Therefore comparingnode names only by equality is too restrictive. This patch provides an...
gnt-cluster info (py): add enabled disk templates
This fixes issue 485. In the python implementation ofthe cluster config info, the enabled_disk_templateswere missing.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix node modify QA tests for large cluster
If the node count in a cluster exceeds the master candidate pool size(which is 10 by default), it's no longer possible to remove the mastercandidate role from a node without passing --auto-promote (even with--force). This behavior is perfectly fine, but the TestModifyNode QA...
Version bump to 2.8.0~beta1
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Change version numbers in documentation
Some of the documents have the version number of Ganeti. This commit updatesit in preparation for the 2.8 version bump.
Doc for GlusterFS Ganeti Support
This patch adds a design document for introducing GlusterFS Support in Ganeti.
Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix issue with python coverage tests
The recently introduced check for python libraries required only for testing(commit 27df5b736ef72b3b12c07f32d64dbac95fe7a5ba) was not correct. Thispatch fixes the issue.
grow disk: call SetDiskID before blockdev_getsize
Missing this call caused blockdev_getsize to be called on the wrongphysical device, and fail.
This fixes Issue 497.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Catch DeviceCreationError unhandled exceptions
_CreateBlockDevInner is called twice in TLReplaceDisks. While thisshould be fixed, right now this leaves the DeviceCreationError exceptionunhandled, which causes a problem due to the fact that this exception...
Fix daemon QA tests
TestInstanceConsecutiveFailures now leaves the test instance in the samestate it got it.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Disable python test if required libraries are missing
Some python libraries are only needed for the tests, but not for runningGaneti. If those libraries are missing, just disable the Python tests.
Better specify what packages to install
Improve install guide
Add command for updating the list of cabal packages, before startinginstalling them.