Fix reference to NodeInfo.name
This fixes a bug introduced during the move from node names to nodeUUIDs as indexes.
Also, a typo is corrected.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Merge branch 'stable-2.8' into 'master'
Version bump to 2.8.0~alpha1
Now that alpha versions are supported, we can bump the version numberfor branch stable-2.8 to 2.8.
This also requires updating the cfgupgrade tool.
Thanks to the previous patches, all the other documents can be upgradedduring the alpha lifetime, before switching to beta (that will enable...
Revert "Storage utility functions for Haskell"
This reverts commit88d27b8aa8adc2e5ced773909f1d40812c5a6ea7.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Revert "Make NodeInfo (hs) accept arbitrary storage types"
This reverts commite89525a859b2e841c08fce506c0b68b97c7efe61.
Make cfgupgrade idempotent
One of the unit tests checks whether cfgupgrade's downgrade option isidempotent. It is, but a version number check made it impossible toactually downgrade multiple times.
With this change, both the current-version version number and the...
Update config version number when downgrading
The downgrade option of the cfgupgrade tool was not changing the versionnumber in the configuration file of the cluster while performing thedowngrade.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Make NodeInfo (hs) accept arbitrary storage types
So far, the Haskell implementation of NodeInfo justrequests storage information about volume groups.With this patch, storage info for abitrary storagetypes can be requested.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Storage utility functions for Haskell
In order to extend the Haskell version of the NodeInfoquery, we need some utility functions to deal withdisk templates and storage types.
Rename directory 'Block' to 'Storage'
This patch renames the 'Block' directory to 'Storage' inthe Haskell code base. The same rename was done in thepython code base earlier this quarter. We generalize thename, because we needed a place for general storage...
Add test for shrink heuristics over different resources
In this example, memory prevents adding more than one instance onthe first node. Yet, on the other nodes, 2 instances can be placedeach, if disks are shrunk appropriately. This, however, requiresto courageous decision to shrink disks next, even though more nodes...
Improve hspace shrinking strategy
In tired allocation, hspace shrinks that resource of the instancenext, that causes failure on most nodes. While, this is not a badstrategy in general, it can lead hspace into a dead end if for a largenumber of nodes a particular resource blocks any further allocation of...
Convenience function for iterating while the result is Ok
For a function f :: a -> GenericResult a, iterate it (in the sense of themonad), until the result is Bad; return the list of values occurred.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Provide witness for the sum-type structure of GenericResult
GenericResult, while rightfully a type of its own, is isomorphicto Either. So, also provide the case analysis function (i.e., theuniversal arrow out of the sum).
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Disable more version checks for alpha versions
Alpha versions should not check for version numbers in READMEsand documentation.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Allow alpha versions not to have an entry in the NEWS file
Alpha versions are still under heavy development, and can therefore not have anentry in the NEWS file yet. The entry will have to be added before switching tothe first beta version.
Partially fixes Issue 448...
Add tests for check-news
Unit tests verifying the behaviour of the check-news script are added.
This required adding a new make target, "autotools-check" directly called from"commit-check", because the usual tests are executed in the build directory,whereas these, being relative to the autotools, cannot be executed there because...
Introduce support for alpha versions
An alpha version can be used to name a release while a previous one is stillbeing developed.
E.g. 2.9.0~alpha1 can be out while 2.8.0~rc1 is being worked on.
When a version is marked as alpha, it is allowed to have unreleased versions...
Improve docstring
Fix bug in Makefile.am
There was a typo in the name of the script for launching offline tests.
Refactor NodeInfo RPC regarding storage reporting
The NodeInfo RPC call is refactored to handle now more thanjust storage reporting for volume groups.
Since NodeInfo now returns storage space information notnecessarily for volume groups, but also for other storage...
Utility functions for storage types
Handling various storage types for the free space reportingrequires some utility functions. They will be invoked inlater patches. Unit tests are included.
Integrate free space reporting for file storage
This patch integrates the free space reporting for filestorage. So far, it is only called, when the defaultdisk template is a file-based disk template ("file" or"sharedfile").
Unit tests for ApplyStorageSpaceFunction
So far, the function 'ApplyStorageSpaceFunction' was notcovered by unit tests. Before the integration of spacereporting for file storage, we add those missing tests.
Renamining space reporting for file storage
Before being integrated, the function for space reportingfor file storage is renamed to a more expressive name.
Add storage type to NodeInfo result
So far, the storage information returned from RPC callNodeInfo contained only information about volume groups.In order to extend the storage reporting to other storagetype, we include another field "type" in the result of...
Use cluster's hvparams in LUInstSetParams
Since the instance does not have hvparams at that point,we have to use the cluster's hvparams here. This fixesa crash when using 'gnt-instance modify'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Merge branch 'stable-2.7' into stable-2.8
Conflicts: (trival, take union of added files/tests) Makefile.am test/hs/shelltests/htools-hspace.test...
Add a test for hspace to respect instance policy
In this example, the instance policy for disks prevents allocationof more than one instance per node. The test verifies that tieredallocation still respects this and doesn't attempt to place policyviolating instances on the cluster....
Make shrinkByType aware of individual disks
When shrinking an instances, you can't just get smaller disk footprintwhile leaving the individual disks as they are. Make the shrinkheuristic aware of that fact, and decrease all individual disks aswell. Fixes issue 484....
Fix lookup of xen toolstack in Haskell
There was a bug in the haskell implementation of nodequery which made the lookup of the xen toolstack xm/xlfail.
Inline simple "alias" variables
Remove variables in the form "var = self.var" throughout all LU's. Those"alias" variables make it harder to read the code (it's no longerimmediately clear what's a field and what's a local variable), and canintroduce subtle bug (especially when not used consistently and when...
Convert node indices to UUIDs in configuration
cfgupgrade replaces the occurences of node names (where they are used asindex) with the node UUID.It also supports the inverse replacement, so downgrades are supportedtoo.
Tests for cfgupgrade are adapted as well....
Index nodes by their UUID
No longer index nodes by their name but by their UUID in the clusterconfig. This change changes large parts of the code, as the followingadjustments were necessary: * Change the index key to UUID in the configuration and the ConfigWriter, including all methods....
Add cluster name to instance migration RPC
This fixes a 'FIXME' in the xen hypervisor. The instancemigration requires the cluster's name. This should comefrom the RPC call, but so far was hackishly taken fromssconf. Ssconf is intended to be used by scripts only and...
Restructure documentation index page
The index page of the documentation used to be just a list of links.Now it is a proper introduction to all the other pages.
Fixes Issue 424.
Add missing parenthesis to description of --machine-readable
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Support VNC binding to an interface
Currently, the VNC bind address can only be specified as an IPaddress. Given that this is a cluster-wide parameter, therearen't many sensible choices besides 127.0.0.1 and 0.0.0.0. Thispatch allows to also specify it as the address of a specifed...
Update NEWS file for 2.8
Add to the NEWS file the list of modifications performed during the 2.8development cycle.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Remove XEN_CMD from configure.ac
Removes the option --with-xen-cmd from the configurescript.
Remove XEN_CMD from constants, adjust to PowercycleNode
This patch extend the RPC to powercycle a node by ahvparams parameter. This parameter is used by thexen hypervisor to determine which xen command to use.The respective LU is adjusted as well.
Since this was the last operation depending on the...
hv_xen.py: make hvparams mandatory, remove fallbacks
This patch removes the fallback to the xen command from theauto config and adds an exception instead. Also, thehvparams parameter of the functions GetCommand, _RunXen and_GetInstanceList are made mandatory....
Use hvparams in GetAllInstancesInfo
This patch extends the GetAllInstancesInfo method of thehypervisors by a hvparams parameter. This change propagatesup to the RPC call all_instances_info, which also neededto be extended. All callers of this call were adjusted...
Use hvparams in GetInstanceInfo
This patch extends the GetInstanceInfo function of thehypervisors by an hvparams parameters. This leads toan extension of the instance_info RPC call and theadjustmend of all classes using this call.
backend.py: remove fallback implementation of GetNodeInfo
This removes the fallback implementation of the backend'sGetNodeInfo function to the old implementation. By now,all calling code is adjusted to the new hvparams parameterand thus, the old implementation should not be necessary...
Adjust all callers of RPC 'call_node_info'
This patch contains all adjustments necessary to code thatuses the 'call_node_info' call. The adjustment is necessarybecause the call's signature changed from receiving a listof hypervisor names to a list of tuples...
Add hvparams to RPC call 'node_info'
This patch adds the hvparams parameter to the RPC call'node_info'. It also adjusts the related code in noded.pyand Query/Node.hs
backend.py: use hvparams in GetNodeInfo
This patch extends the GetNodeInfo function of the backendmodule by a hvparams parameter. For now, it contains afallback solution for when the parameter is None. Thisfallback will be removed in a later patch of this series....
hypervisors: add hvparams to GetNodeInfo
This patch extends the GetNodeInfo function of thehypervisors by a hvparams parameter. The parameteris currently only used by the xen hypervisor to determinewhich xen toolstack ('xm' or 'xl') to use.
hv_xen.py: renaming residual 'xm' occurrences
Some methods still refered to 'xm', which is inaccurate nowthat xen can also be run with 'xl'. No functional changesotherwise.
Watcher: Use hvparams when listing instances
This patch makes the watcher use the hvparams from ssconfto retrieve the list of instances from the hypervisors.
Include hvparams in ssconf files
The watcher needs access to the cluster's hypervisorparameters. This patch extends the ssconf infrastructureby adding a hvparams file for each hypervisor. The formatof the hvparams file is a simple key=value format.
Use hvparams in instance migration
This patch makes use of hvparams on instance migration todetermine the list of instance before the migration. This way, inxen, the xl/xm choice is respected.
Verify: node info and instance list
This patch enhances node verification by:- Factoring out the verification of the node info and the instance list to increase testability.- Make both of them use the hvparams parameter.
Verify: xen toolstack, hypervisor and hvparams
This patch extends the node verification by:- Adding a check for the xen toolstack when the hypervisor is verified.- Factoring out the hypervisor verification in a subfunction to increase testability.- Factoring out the hvparams verification in a...
rpc / noded: add hvparams to instance_list call
The 'instance_list' call needs to be extended to accepthvparams. This is one of the few RPCs where it isnecessary, because there is no instance given whosehvparams could be reused.
backend.py: use instance's hvparams whenever possible
The backend module's function 'ListInstances' is calledwithin the module several times. In many cases, it ispossible to reuse the instance's hvparams for the'ListInstances' call and thus avoiding having to extend...
backend.py: GetInstanceListForHypervisor
In same cases, the call 'GetInstanceList' is only done forone hypervisor. In those cases, it is more convenient tonot provide the full collection of hypervisor parametersfor all hypervisors, but only the ones for this hypervisor....
backend.py: hvparams in 'GetInstanceList'
Propagating the use of hvparams further up the callinghierarchy, this patch makes the backend module use thehvparams in 'GetInstanceList'. Unit tests provided.
hv_xen.py: _RunXen and GetInstanceList use hvparams
Propagating the use of hvparams further up in the callinghierarchy, the functions '_RunXen' and 'GetInstanceList'use the given hvparams. Unit tests provided. Calling function'StopInstance' adjusted in this patch as well....
hv_xen.py: rename RunXmList to RunInstanceList
Since one can retrieve xen's instance list with both, 'xm'or 'xl', the function name 'RunXmList' is no longerappropriate. Renaming it to 'RunInstanceList'. Nofunctional changes otherwise.
hv_xen.py: rename GetXmList to GetInstanceList
Since one can retrieve xen's instance list via both 'xm'or 'xl', it is no longer appropriate to call the function'GetXmList'. This patch renames it to 'GetInstanceList'.Other than that, there are no functional changes....
hv_xen._GetCommand: retrieve xen command from hvparams
This patch adds a (so far optional) hvparams parameterto the '_GetCommand' function. This dictionary is usedto retrieve the xen command (xm or xl). It is optionalfor now to make the refactoring possible without breaking...
Unit tests for hv_xen._GetCommand
This patch adds unit tests for the current state of the'GetCommand' method. The tests are rather trivial, buthaving tests before starting to refactor seems to bea good idea.
Add xen command (xm, xl) to hv parameters
This patch adds the xen command (xm, xl) as hypervisorparameter to the constants and adds validation tothe xen hypervisors.
Fixing docstring of hv_xen_unittest.py
The introductory comment of hv_xen_unittest.py referred tothe wrong hypervisor.
Make SubmitOpCode expect options without print_jobid
As opposed to SubmitOrSend, the SubmitOpCode function iscalled also by commands not supporting submit options. Henceonly inspect these options, if they are actually present.
Add an option to print job id of submitted jobs on stdout
When submitting a job, the job id, prefixed by 'JobID: ', is output tostderr. While this is easy to understand for humans, for scripting, itis more convenient to have (additionally) the id on stdout (and only...
Add new command gnt_job wait
This command tacitly waits for a job to finish. In thatway, job dependency can also be handled in shell scripts.
Add type annotation to avoid monomorphism restriction
Even though we need the let-bound variable showMoves onlyat type [(String, String)] -> IO (), it's most general typewould be (PrintfArg a, PrintfArg b) => [(a, b)] -> IO ().This causes the monomorphism restriction apply to that binding,...
add option --print-moves to hroller
If non-redundant instances are present in the cluster, hroller willplan for them to move to other nodes while the group is rebooted.This adds an option to also show this plan.
New tests for hspace and exclusive storage
hspace is run on clusters with exclusive storage enabled.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
hspace prints info about spindles
Statistics about spindles are tracked. In human-readable output, spindlesare printed only when used (i.e., exclusive storage is enabled). Formachine-oriented output, they are always there.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Add support for shrinking an instance spindles-wise
This makes tiered allocation in hspace work also with respect to spindles.
Spindles become part of htools resource spec
Spindles are now part of resource spec. Instances get created with spindlesspecified (which are just ignored when exclusive storage is disabled).
htools cluster score takes spindles into account
When exclusive storage is enabled, spindles are used instead of disk spaceto compute the cluster score.
Comments and variable names in computePDsk has been changed to match theactual code.
Update spindles when moving instances in htools
Spindles get updated, and errors raised when not enough free spindlesexist. No new error is raised when exclusive storage is disabled.
Unit tests included.
Unit tests for htools and exclusive storage
The existing tests are run also on nodes with exclusive storage enabled. Thevalues for spindles and exclusive storage are set in a consistent way, forboth nodes and instances.
Load complete instance disk information through LUXI
Information about size and spindles of all the disks of an instance is loadedby the LUXI backend, instead of faking one equivalent big disk. In this wayinstance policy checks are more accurate.
Load node spindles data in htools
The data structure for nodes gets a new field for free spindles, and theexisting field for total spindles gets renamed to avoid identifying it withthe node parameter that had the same name. These fields get filled with...
Refactor reading live data in htools
This simplifies different handling of individual items.
Export node spindles
Node spindles (queried live) are exported through the LUXI, RAPI, andiallocator interfaces.
node_info RPC reports info about free spindles
node_info RPC can be used to get information about free/total spindles.
Update htools text-backend documentation
Both instances and nodes have new fields.
Use RpcResult.Warn where appropriate the cmdlib
To avoid repetitive code, make use of the functionality factoredout to the Warn method of RpcResult.
Add a Warn function to RpcResult
At various places, LU code just wants to warn abouta failed rpc, but continue nevertheless. So factor outthat common checking, message formatting, and notifying.
Add --force option to gnt-cluster modify
Make the force option of the ClusterSetParams Opcode availableon the command line.
Add a force option to the ClusterSetParams Opcode
If set, the op code will, in particular, try to set the master IPon the new netdev, even if shutting down the master IP on the oldnetdev failed.
Document the monitoring daemon in the admin guide
The administrator's guide must include a section describing the monitoringdaemon and its API, in order for the users to be able to use it.
Factor out the API of the monitoring daemon
It will be included by both the design document and the user documentation.
Add monitoring agent info to the security document
Add information regarding the monitoring agent and its security tothe security document.
Fix shadowing of library function
The "reads" field shadows a library function from Prelude. This commitfixes the problem.
Export device UUIDs to hooks and OS scripts
Export UUIDs and names of instance NICs and disks to the environmentof OS scripts and instance related hooks.
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Signed-off-by: Klaus Aehlig <aehlig@google.com>...
gnt-cluster info: only parameters for enabled hypervisors
In the output of gnt-cluster info, restrict the hypervisor parametersto those hypervisors that are enabled. This fixes issue 443.
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Add shelltests for diskstats
Check that the behavior of running the diskstats data collector is asexpected.
Let the monitoring daemon provide the diskstats collector
Add the collector to the list of those provided my the monitoring daemon.
Add diskstats data collector
Add a new data collector responsible for gathering disk performancestatistics.
Add constant with the path of the diskstats
This will be used by the diskstats storage collector.
Add a CLI parameter for input files
For many data collectors it is useful (especially for testing) to have topossibility to specify an input file.
This commit adds a generic option for doing that.
Add unit tests for the Diskstats parser
Add one test parsing an actual /proc/diskstats file and one QuickChecktest.