Fix docstrings for hypervisor functions returning node info
The documentation was out-of-sync with the code.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Add LV collector to the monitoring daemon
Allow the monitoring daemon to use the LV data collector.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add tests for the LV data collector
This commit adds a few shelltests for the logical volume data collector.
Also, it performs some cleanup of the comments already present in the file:given that the number of the tests should be changed manually, and this is...
Add LV data collector
This commit adds the LV data collector.
Also, the lvCommand function was not providing the correct value as expected bythe readProcess function, so it was fixed.
Add tests for LVs parser
Add unit tests and Quickcheck tests for the LV parser.
Also, fix the alphabetic order of some imports in htest.hs.
Add function testing parsers with QuickCheck
Parsing a string automatically generated by QuickCheck can be useful in manycases. A function for doing this is added to the common test functions.
Add function for generating arbitrary UUID
Add a function for generating arbitrary UUIDs to be used for QuickCheck tests.
Add LV parser
Add the parser for getting the information about the logical volumes in thesystem.
Add design for LV data collector
Add a section describing the logical volume data collector to the designdocument for the monitoring agent.
Minor fixes to monitoring design doc
Add a missing full stop, and fix a sentence about the storage collectors.
Fix documentation formatting issue
One line was longer than 80 characters.
Add test for the --full-evacuation option
In this example cluster, there are four pairs of nodes, each hostingtwo instances. Each node has disk capacity for an additional instance.Therefore, for normal online rolling reboots, there are two node groups,...
hroller: option --full-evacuation
Add an option to hroller, to plan for full evacuation of thenodes to be rebooted, i.e., also plan for replacement secondarynodes for all instances on the node after migrating out instanceswith this node as primary.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Extract a partition functional
Separate the partitionNonRedundant function in hroller into ageneral functional that partitions a list of nodes accordingto some clearing strategy and the specialization of movingnon-redundant instances out. In this way, we don't have to...
Extract functional for greedily clearing nodes
The method clearNodes in hroller greedily clears nodes ofnon-redundant instances by moving them to a different node. This patchseparates the greedy clearing algorithm from the specialization tonon-redundant instances; in this way, we don't have to duplicate code...
Make hroller not consider offline nodes for evacuation
When planing on where to evacuate the non-redundant instancesof the nodes to be rebooted, it doesn't make sense to consideroffline nodes. So add this restriction to hroller.
Update comments in hroller code
hroller schedules moves of instances to have rebooted nodesfree of instances with this node as primary. Update the commentsto reflect that this move planning is for non-redundant instancesonly.
Clarify meaning of --print-moves in hroller man page
When given the option --print-moves, hroller prints a list ofinstance relocations after each reboot group. This sequentialpresentation can lead to the wrong impression, that these areall the moves to be carried out. Therefore, explicitly mention...
Remove obsolete TODO
Originally, hroller started as a tool for offline maintenance only.There it made sense to warn about instances still running. By now,default planning is to migrate instance off the nodes to be rebooted,with options for other behavior (like pretending that all instances...
Allow customization of the chroot
As different users have different needs about the chroot, allowcustomization in its build script. This is achieved as follows.- The environment variable EXTRA_DEBIAN_PACKAGES is honored, and the packages named there are additionally installed in...
build_chroot: fix duplicate install
In the last change to the chroot build script, theapt-get command 'install' was moved to the APT_INSTALLmacro; however, it was forgotten to remove it outsidethe macro, causing an error, as there is no packagewith name 'install'....
build_chroot: use APT_INSTALL macro for packet installation
In this way, it is easier to change all the apt calls in the chroot.In particular, this patch fixes the missing -y option for the installationof the tools needed to build debian packages.
Don't exceed the standard PV size when growing a disk
With exclusive storage, PVs are supposed to be used only up to thestandard PV size.
Enable disk growth with exclusive storage
Disk growth is now supported also with exclusive storage, so we let theoperation continue.
Enable disk growth test in qa also with exclusive storage.
Limit the PVs used for disk growth with exclusive storage
The exclusive storage flag is passed along in the backend functions fordisk growth. LVM volumes handles the flag by reusing the same PVs, as disksare constrained by the number of their spindles....
Add exclusive_storage to blockdev_grow RPC
The flag is passed to the RPC, but it's still ignored by noded.
Design doc for openvswitch support
This patch includes the design document for openvswitch support
Signed-off-by: Sebastian Gebhard <sege@fs.ei.tum.de>Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Make NIC setup more flexible in case of Xen
For every NIC write down a file that contains NIC'sinfo (MAC, mode, link, network details), that canbe sourced by any vif script. The file location is:/var/run/ganeti/xen-hypervisor/nic/<domname>/<nicidx>....
Refactor NIC configuration scripts
Add new script vif-ganeti that could be used instead ofdefault vif-bridge in order NIC customization can be moreflexible.
Extract common code from kvm-ifup and insert it in a newfile net-common that is being sourced by kvm-ifup and...
Document vif-ganeti usage
In order to be able to use it as vif-script one has tocreate a symbolic link inside /etc/xen/scripts pointingto /usr/lib/ganeti/vif-ganeti and then add is a hypervisorparameter.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Update design document for hroller
The requirement to be able to plan full node evacuation came up afterthe initial design. So add a section describing the algorithm, iffull evacuation is requested.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
iallocator: storage processing accepts space_info structure
This is a small refactoring to make the processing ofthe storage space information not accept a node inforesult, but the more specific space info structure instead.
Signed-off-by: Helga Velroyen <helgav@google.com>...
iallocator: get rid of MakeLegacyNodeInfo
So far, the iallocator makes a node info call, convertsthe result into a legacy node info structure and thenretrieves various values from it. This patch removesthe call to MakeLegacyNodeInfo completely, by letting the...
iallocator: factor out instance memory summation
The iallocator adjusts the free memory by a calculationof how much memory is actually currently used by instanceson the node. This calculation is rather complicated anddeserves its own function to increase testability. No...
iallocator: extract storage info directly from node info
So far, the iallocator made a node info RPC call, convertedthe result into a legacy node info format and then extractedthe storage information from that. This patch removes themiddle step of this procedure, skipping the legacy node info...
Make NodeInfo (hs) accept arbitrary storage types
So far, the Haskell implementation of NodeInfo justrequests storage information about volume groups.With this patch, storage info for abitrary storagetypes can be requested.
Storage utility functions for Haskell
In order to extend the Haskell version of the NodeInfoquery, we need some utility functions to deal withdisk templates and storage types.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
iallocator: computation of storage data
This patch extracts the processing of the storage-relateddata from the RPC call 'node_info' in a separate functionto increase testability. In addition, this patch containsunit tests for this function.
rpc.py: override default storage type in node info
This patch changes the semantics of dealing with the resultof the RPC call "node_info" when converting them into aLegacyNodeInfo. A legacy node info, can only hold storageinformation about one storage type, which so far was always...
iallocator: factor out 'get_attr' method
This patch extracts an inline 'get_attr' method to extractdata from the result of the RPC call 'node_info'. It was sofar an inline function in the middle of a bigger method, which madethe code hard to read.
iallocator: prepare RPC call 'node_info'
This patch does a small refactoring in the allocator. It extractsthe preparation and actual call of the 'node_info' call into aseparate function. This is done for better testability. No functionalchanges otherwise....
Typo in iallocator design doc
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Fix lint error
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Unset LV name if no LVM disk templates are enabled
If there are no LVM based disk templates enabled in QA, reset the LVname after the corresponding gnt-cluster modify tests. Otherwisegnt-cluster verify would complain about the missing volume group.
Also make sure that no disk templates requiring LVM are ever enabled...
Skip drbd-usermode-helper test if DRBD is disabled
Changing the drbd usermode helper in gnt-cluster modify fails if DRBD isnot installed on the nodes. Thus skip this test if DRBD is not in theset of enabled disk templates.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Add route to VCluster network in vcluster-setup
This route is required on the Fedora test machine, as otherwise fpingwould not work and hang forever.
Replace tempfile by mktemp in QA code
The tempfile command only exists in Debian. In order to run the VClusterQA also on Fedora, replace its use by mktemp.
Merge branch 'stable-2.8' into master
Merge branch 'stable-2.7' into stable-2.8
Provide the right fix for the kvm_extra parameter
With the previous adjustment the arguments were boxed into one extraarray. This actually makes the parameter work (with the same caveats).
The manpage is also updated with the limitations.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
kvm: don't pass whole extra as a single argument
If extra is longer than a single word passing it the way it was passedwon't work. Of course this solution is also only partial, as won't takeinto account any form of quoting. :/
NEWS and version updates for 2.7.0~rc3
Time for a new release.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
For node queries allow short forms of host names
For node queries use the host-name filter instead of the simpleequality-based one.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Provide a special filter for host names
For host names, usually short forms are used, e.g., node1 or node1.subinstead of the full qualified node1.sub.example.com. Therefore comparingnode names only by equality is too restrictive. This patch provides an...
gnt-cluster info (py): add enabled disk templates
This fixes issue 485. In the python implementation ofthe cluster config info, the enabled_disk_templateswere missing.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix node modify QA tests for large cluster
If the node count in a cluster exceeds the master candidate pool size(which is 10 by default), it's no longer possible to remove the mastercandidate role from a node without passing --auto-promote (even with--force). This behavior is perfectly fine, but the TestModifyNode QA...
Version bump to 2.8.0~beta1
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Change version numbers in documentation
Some of the documents have the version number of Ganeti. This commit updatesit in preparation for the 2.8 version bump.
Doc for GlusterFS Ganeti Support
This patch adds a design document for introducing GlusterFS Support in Ganeti.
Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix issue with python coverage tests
The recently introduced check for python libraries required only for testing(commit 27df5b736ef72b3b12c07f32d64dbac95fe7a5ba) was not correct. Thispatch fixes the issue.
Signed-off-by: Michele Tartara <mtartara@google.com>...
grow disk: call SetDiskID before blockdev_getsize
Missing this call caused blockdev_getsize to be called on the wrongphysical device, and fail.
This fixes Issue 497.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Catch DeviceCreationError unhandled exceptions
_CreateBlockDevInner is called twice in TLReplaceDisks. While thisshould be fixed, right now this leaves the DeviceCreationError exceptionunhandled, which causes a problem due to the fact that this exception...
Fix daemon QA tests
TestInstanceConsecutiveFailures now leaves the test instance in the samestate it got it.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Disable python test if required libraries are missing
Some python libraries are only needed for the tests, but not for runningGaneti. If those libraries are missing, just disable the Python tests.
Better specify what packages to install
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Improve install guide
Add command for updating the list of cabal packages, before startinginstalling them.
Fix typo in the documentation index
Fix typos in the documentation index
This patch fixes some typos in the documentation index.
Update security document wrt confd access to SSL cert
The fact that confd has been given access to the server.pem certificateneeds to be described in the security document.
Fix permissions of the queue archive dir
The permissions were set to 0740, but of course being a directory thecorrect permissions are 0750.
Add a warning about the restriction of gnt-node evacuate
Even though gnt-node evacuate should be able to evacuate all instancesat once, this currently is not the case. Add a warning about this deficitfor the time being.
Add gnt-node migrate to QA
Make it possible to optionally run gnt-node migrate during QA.
Convert instance indices to UUIDs in configuration
cfgupgrade replaces the occurences of instance names (where they areused as index) with the instance UUID.
It also supports the inverse replacement, so downgrades aresupported too.
Index instances by their UUID
No longer index instances by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the...
Node-UUID related cleanup
Some variables which hold node UUID's are renamed to reflect theirindent better.Use cfg.GetNodeName(uuid) instead of cfg.GetNodeInfo(uuid).name whereappropriate.
Add tools for building deb packages to build_chroot
The chroot only had tools for building and testing Ganeti, but not forbuilding its Debian packages. They are added by this commit.
Fix wrong assert
An assert wrongly verified that the node name instead of the node UUIDis in the set of locked nodes. This patch fixes this.
Improve the final message of build_chroot
Give more precise instructions on the next steps to actually run thechroot.
Make build_chroot self-contained
The configuration file are included in build_chroot, and areautomatically created by the script if it is not able to find themalready in the directory pointed by $DATA_DIR.
Note that the configuration file directory is cleaned if it's temporary,...
Fix testGetHvDefaults testcase
The introduction of the migration_mode parameter to the fake hypervisorbroke the testGetHvDefaults. This patch fixes this by taking the defaultparameters of the fake hypervisor into account.
Fix instance add for missing node specs
Don't pass None to cfg.GetNodeNames. Instead, only resolve the nodenames if it's sure that the UUID list is valid.
This fixes issue 495.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Skip Xen monitoring QA for non-Xen clusters
Instead of asserting that the Xen instance status collector fails fornon-Xen clusters just skip the test completely.Otherwise QA would fail for non-Xen hypervisors if Xen happens to beinstalled on the cluster as well....
Add support to migration_mode in fake HV
The fake hypervisor pretends to support migration, but did not supportthe migration_mode parameter yet. As this parameter is required duringinstance migrations, support for it is added in this patch.
Adapt VCluster script to current situation
vcluster-setup is adapted to the current Ganeti command line arguments.Also, a bug for diskless instances in qa_instance is fixed.
Check DRBD status in cluster verify
If the status of DRBD disks is degraded or in a faulty state, issue anerror in cluster verify.
Add confd user to masterd group
The server.pem file is 0440 (materd_uid, masterd_gid) and cannot beaccessed by confd.
This patch contains a workaround for 2.8.0~beta1 (adding conf to themasterd group in the user setup tool, and remembering the users to...
In tiered allocation, cut non-promising shrinking tries
The heuristics for tiered allocation has been improved in that itchooses to shrink such a resource next where by shrinking only thisresource a valid allocation can be made, if such a resource exists....
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Version bump to 2.9.0~alpha1
Bump the version number to 2.9.0~alpha1.
This also requires updating version numbers in cfgupgrade tool.
Furthermore, a data file for test representing a 2.8 configuration had theversion number set to 2.7 in order to prevent tests failing while the current...
Fix reference to NodeInfo.name
This fixes a bug introduced during the move from node names to nodeUUIDs as indexes.
Also, a typo is corrected.
Merge branch 'stable-2.8' into 'master'
More agressively line break man pages
Newer versions of pandoc (as, e.g., shipped with Ubuntu 13.04)more agressively quote, in particular minus signs. That sometimesleads to unintended line breaks, which are a problem, if the newline starts with a dot....
Version bump to 2.8.0~alpha1
Now that alpha versions are supported, we can bump the version numberfor branch stable-2.8 to 2.8.
This also requires updating the cfgupgrade tool.
Thanks to the previous patches, all the other documents can be upgradedduring the alpha lifetime, before switching to beta (that will enable...
Revert "Storage utility functions for Haskell"
This reverts commit88d27b8aa8adc2e5ced773909f1d40812c5a6ea7.
Revert "Make NodeInfo (hs) accept arbitrary storage types"
This reverts commite89525a859b2e841c08fce506c0b68b97c7efe61.
Make cfgupgrade idempotent
One of the unit tests checks whether cfgupgrade's downgrade option isidempotent. It is, but a version number check made it impossible toactually downgrade multiple times.
With this change, both the current-version version number and the...
Update config version number when downgrading
The downgrade option of the cfgupgrade tool was not changing the versionnumber in the configuration file of the cluster while performing thedowngrade.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>