Add spindles field to disk object
The field is filled with the value provided on the command line.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Check that spindles are not given when they are not allowed
Masterd checks that specifications for new disks don't include spindleswhen exclusive storage is disabled.
Add "spindles" option to instance disk creation commands
The option is parsed but ignored, for the moment.
Fix style inconsistencies in gnt-instance man page
Escaping and initial capitals were not uniform.
cfgupgrade: Downgrade is a NO-OP
The configuration is still the same as in 2.8 (the reference stable versionfor this branch), so downgrade shouldn't do anything.
Unit tests are also updated, with a new 2.8 configuration file. Theconfiguration file used for the upgrade+downgrade test was tailored to the...
Design doc for internal shutdown detection
Ganeti is currently not able to detect a legit shutdown request performed by auser from inside a Xen domain.
This patch provides a design document to implement a mechanism able to cope withsuch events.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Document recent hroller changes in the NEWS file
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Document hroller options recently added
hroller now also supports the options --skip-non-redundant and--ignore-non-redundant, and this should be documented in theman page as well.
While there, also use the same order in the options sectionas in the synopsis, and in the synopsis group the algorithms...
Extend hroller tests by options for non-redundant instances
The cluster now consists of 3 nodes, with drbd instances betweennodes 1 and 2, and 2 and 3. Additionally, nodes 1 and 3 each containa non-redundant instance, but node 2 cannot hold two additional...
Test for hroller taking non-redundant instances into account
The example cluster consists of 6 nodes, each hosting 2 instances andhaving capacity for 3. So, while the drbd-induced graph consists ofonly insulated nodes, no more than two nodes can be rebooted at the...
hroller: option to ignore non-redundant instances
Add an option to hroller restoring the old behavior on not takingany non-redundant instances into account when forming rebootgroups.
Make hroller also plan for non-redundant instances
Non-redundant instances need to be moved to a different nodebefore maintenance of the node. Even though they can be moved toany node, there must be enough capacity to host the instances of thereboot group to be evacuated....
hroller: option to skip nodes with non-redundant instances
So far, hroller ignores the fact, that non-redundant instances exist.One option to deal is non-redundant instances is to not schedule thosenodes for reboot. This is supported by adding the option --skip-non-redundant....
Remove trailing whitespace
Improve installation documentation
Based on user feedback the installation documentation is clarified andextended.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
RPC 'node_info': <storage_type,key> instead of vg_names
This replaces the field 'vg_names' in the RPC call of 'node info' by'storage_units'. A storage unit is a tuple <storage_type,key>and a generalization of a vg_name. The list of vg names is replaced by...
Merge branch 'stable-2.8' into master
Merge branch 'stable-2.7' into stable-2.8
Update NEWS with disk creation fixes
Also document a couple more fixes.
Sort cmdlib-related entries in Makefile.am
Files in the cmdlib directory are sorted alphabetically inMakefile.am.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
cmdlib: Cleanup public/private functions
All functions/classes which are used outside of their defining module(with tests as an exception) no longer have a leading underscore.
cmdlib: Extract instance query related functionality
Split instance.py further by extracting instance querying relatedlogical units and functions to instance_query.py.
cmdlib: Extract instance operation functionality
Split instance.py further by extracting instance operations(start/stop/reboot/etc.) related logical units and functions toinstance_operation.py.
The extracted operations have in common that they affect the operating...
cmdlib: Extract migration related functionality
Split instance.py further by extracting migration related logical unitsand functions to instance_migration.py.
cmdlib: Extract storage related functionality
Split instance.py further by extracting storage related logical unitsand functions to instance_storage.py.
Reformat and define exports in cmdlib/__init__.py
cmdlib/__init__.py now simply defines the interface of the cmdlib moduleby importing all classes which should be visible to clients.
Also don't ignore C0302 (Too many lines in module) any more.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Extract miscellaneous logical units from cmdlib
All remaining classes in init.py are extracted to misc.py.
Extract os related logical units from cmdlib
All LUOs* classes are extracted to operating_system.py.
Extract query related logical units from cmdlib
All LUQuery* classes are extracted to query.py.
Extract backup related logical units from cmdlib
All LUBackup* classes are extracted to backup.py.
Extract instance related logical units from cmdlib
All LUInstance* classes are extracted to instance.py. Common functionsare moved to common.py if used by non-instance logical units as well.Additionally, helper functions which are only used by LUBackup* and...
Extract node related logical units from cmdlib
All LUNode* classes are extracted to node.py. Common functions are movedto common.py if used by non-node logical units as well.
Extract group related logial units from cmdlib
All LUGroup* classes are moved to group.py. Common functions areextracted to common.py.
Extract cluster related logical units from cmdlib
All LUCluster* classes are extracted to cluster.py. Shared functions areextracted to common.py, helper functions only used by LUCluster* areextracted to cluster.py.
Extract test logical units from cmdlib
LUTest* are moved to test.py.
Extract network related logical units from cmdlib
LUNetwork* and associated helper functions are extracted to network.py.
Extract tags related logical units from cmdlib
LUTags* and their base class, TagsLU, are extracted to tags.py. Anadditional shared function, _ShareAll, is extracted to common.py forshared usage.
Extract base classes from cmdlib
Base classes holding common functionality is extracted into base.py.Utility functions used by both base classes and subclasses is moved tocommon.py.
Don't fail to deactivate master IP if already down
The master IP setup script now checks if the master IP is actuallyconfigured on the machine before trying to remove the IP.
This fixes issue 460.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Compatibility test for instances
This patch introduces a test to check the compatibilityof the Haskell and the Python representation of instances.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Instance generators
This patch introduces and enhances generators forinstances:- 'genInstWithNets' is split into the generation of an arbitrary instance and enhancing an instance with nets- 'genInst' calls 'genInstWithNets' with an empty set of initial networks to provide a reasonable default...
Annotate every arbitrary instance field
The Arbitrary instance of the 'Instance' object is writtenusing the <*> syntax. Since it often uses the 'arbitrary'generator for the instance's fields it is hard to figureout which 'arbitrary' fills which instance field. This...
Generators for disks
This patch adds generators for Disk instances to the Haskelltest code. It uses somewhat more reasonable generators tofill the fields instead of just arbitrary values.'genDiskWithChildren' is a generator that generates a diskwith a specified number of disk children. To avoid shooting...
Use os.statvfs to determine free disk space
This simplifies my previous commit (820bade90) by using os.statvfsinstead of parsing the output of 'df'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Backend function for file storage space reporting
This adds functionality to retrieve disk space informationfor file storage. It calls the 'df' tool and parses itsoutput to extract the total and free amount of disk spaceon the disk where the given path is located....
Reorder unit tests in Makefile.am
In a previous commit, I accidentially changed the order ofunit tests in Makefile.am to not be alphabetically anymore.This fixes it.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Remove extra newline
Also, properly set the date of the last modification.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Make HS ConfD client IPv6 compatible
The Haskell ConfD client was assuming internet addresses to be IPv4. Thispatch modifies the client so that it is able to automatically detect theprotocol it should use by analyzing the address it is told to connect to....
Factor out resolveAddr function
This function can be useful to many parts of the code to convert the stringrepresentation of an IP (v4 or v6) address into the proper data type.
Add MonD to the watcher
The monitoring daemon should always be alive, therefore it's added to thewatcher.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Start the monitoring and node daemons together
Add the monitoring daemon to the command starting the node daemon, given thatthey both have to be started on all nodes.
Note that daemon-util only supports starting one daemon at the time, so theactual command has to be composed as a sequence of two different daemon-util...
Add a constant stating whether monitoring is enabled
Add QA for recreating single instance disks
So far QA only recreated the whole set of disks at once.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Add QA for gnt-instance modify --disk
Just a very basic test that adds and then removes a disk.
Clean up when "gnt-instance modify" fails to create a disk
cmdlib.LUInstanceSetParams now uses helper functions to create and wipedisks, so that when the creation of a disk fails, any leftover device iscleaned up. As a bonus, exceptions raised by _CreateBlockDev() are caught...
recreate-disks honors the prealloc_wipe_disks flag
Now even recreate-disks wipes the newly-created disks, if the flag is set.
Introduce wrapper for cmdlib._WipeDisks()
The wrapper handles errors by logging them and cleaning up freshly-createddisks.
Also, the correct disk is used in the error message when an error happensin cmdlib._CreateDisks() and the resulting disk clean-up fails....
Don't catch an exception that cannot be raised
Since 9b221ea4, _CreateBlockDev() doesn't raise OpExecError any more. Yetsome code was left in place to catch it. By removing that code we have twoadvantages:1. Dead code is removed.2. If for whatever reason _CreateBlockDev() raises OpExecError, the...
Wipe disks added through "gnt-instance modify"
In issue 353 Sascha Lucas reported that disks are not wiped when addedthrough “gnt-instance modify”. This patch adds this functionality andupdates the docstring for “_WipeDisks”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Move 'container.py' to storage directory
Moving 'container.py' to the storage directory.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Rename dir 'block' to 'storage'
Renaming the 'block' directory to 'storage', because I plan toplace code there that is related to file storage and leavingit named 'block' would be misleading.
Rename storage.py to container.py
Renaming 'storage.py' to 'container.py'. It will be moved into the new'storage' directory, which will come in later patches to avoid clashes ofnotation.
Monitoring QA: Remove superfluous import
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
Non-Xen support for monitoring QA
The QA tests the Xen instance status collector, but that is expected to failwhen run on machines that do not use Xen.
Add QA for instance status collector
This commit introduces the QA for the instance status collector.
Begin the first QA for a monitoring-related component, the files and somefunctions are named after monitoring because they are meant to containfuture monitoring QAs as well....
Use dcName in mon-collector
Instead of manually specify the name of the data collectors in mon-collector,just use the dcName field each of them exports.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Monitoring design doc: better specify field names
The name of the list of instances was not specified.
Also, fix a line that was longer than 80 characters.
Factor out the mergeStatuses function
It will be used by multiple data collectors, not only the DRBD collector.
Add global status field to the instance status collector
The global status is computed from the statuses of the single instances.
The output json format is adapted to include this piece of information, asprescribed by the design document.
Add instance status collector to mon-collector man page
Add a section related to the new collector.
Also, fix some formatting issue (white spaces, line longer than 80 chars)in the DRBD collector section.
Export the Instance Status collector report
It will need to be accessed by the monitoring daemon.
Run the monitoring daemon as root
The monitoring daemon needs to be able to run some commands that require rootaccess (such as "xm") in order to fulfill its duties.
Add inst-status-xen to the monitoring daemon
Enable the monitoring daemon to invoke the Xen instance status data collector.Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
QA: factor out some instance management functions
Some functions for managing instances will have to be used by new upcomingunit tests, so they are taken out of the instances QA file and put in a newutilities file accessible by other QA files as well....
Add dependency on the process library
The tests are already using this library, so it's not really a new builddependency, but it was not specified esplicitly.
Furthermore, it's going to be used by the instance status collector, so it'sadded to the requirements for the monitoring subsystem....
Add HS functions for getting the instance reason path
The getInstReasonFilename is built to resemble the python correspondingfunction.
Add module containing function for getting info from Xen
The Xen instance status data collector will require to get some informationfrom the hypervisor. This commit introduces a module providing such functions.
Add the core of the instance status collector
Add the Xen instance status data collector with only its core features.The next commits will add more reporting functionalities.
The access to the collector is made possible through the mon-collectortool....
Export the actual instance state
Compute the actual state of the instance and export it.
Determine status of one instance
Added function for determining whether the status of an instance is ok, and torepresent this information in the corresponding field in the report.
Include the reason trail in the instance collector output
Fetch the reason trail from file, failing gracefully if it is not found, andinclude it in the output of the instance status data collector.
Export Instance Status collector information
Name, version, format version, category and kind of the Instance Status datacollector are now exported.
Factor out function for building report
Instead of building the report as part of the "Main" function, have itbuilt by its own dedicated function, so that it will be able to export itdirectly to the monitoring daemon when needed.
Add example for online rolling reboots using tags
While this use case was described in the design document, andmentioned several times as motivation for changes in commit messages,it has never been added to a user-facing documentation. This commitadds at least an example to the man page....
Move cmdlib.py to cmdlib/__init__.py
cmdlib.py has grown really too much. Move it into its own package toallow splitting it further.
Signed-off-by: Guido Trotter <ultrotter@google.com>Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Allow build_chroot to work from any directory
build_chroot used to work only if launched from ./devel/, whereas nowit can be launched from anywhere, and it will store the resulting filesin the current directory.
Fixes Issue 459.
build_chroot: check whether the data dir exists
If the data directory is not in the expected place, the script complainswith an error message and stops, instead of giving obscure messages.
Partially fixes Issue 459.
Extend hroller test to also verify tag-based node selection
While the multiple-tags test was added to verify that coloring is doneonly after node selection (otherwise it wouldn't be possible to get inboth cases a single reboot group), it can easily be extended to also...
Add a test for online rolling reboot scheduling
In the example configuration, the graph constructed by just connectingprimary and secondary instances is two-colorable. However, when takingconflicting locations of secondary nodes into account, three reboot...
Support online-maintenance in hroller
Make hroller take into account the nodes (redundant) instanceswill be migrated to. This be behavior can be overridden by the--offline-maintenance option which will make hroller plan underthe assumption that all instances will be shutdown before starting...
Support construction of the graph of all reboot constraints
For online rolling reboots, there are two kind of restrictions. First,we cannot reboot the primary and secondary nodes of an instancetogether. Secondly, two nodes cannot be rebooted simultaneously, if...
Add option --one-step-only to hroller
Add a new option to hroller to only output information about the firstreboot group. Together with the option --node-tags this allows for thefollowing work flow. First tag all nodes; then repeatedly compute thefirst node group, handle these nodes and remove the tags. In between...
Sort reboot groups by size
Make hroller output the node groups not containing the master nodesorted by size, largest group first. The master node still remainsthe last node of the last reboot group. In this way, most progressis made when switching back to normal cluster operations after the...
Fix expectation in hroller test
Regular expressions are not shell globs. So "any symbol" is expressedby a dot, not a question mark. In this case, the confusion lead to atoo liberal expectation, hence the test passed. Fix it nevertheless.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Refactor check for exclusive_storage in LUInstanceCreate
The order of evaluation of the conditions is changed, so it's easier to addmore (foreseen) checks for exclusive_storage.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Refactor disk checks in LUInstanceSetParams
Prereq checks relative disks are grouped together and moved in a separatemethod. This reduces the clutter in CheckPrereq().
Fix a misspelled word in design-storagetypes
Signed-off-by: Weiwei Jia <harryxiyou@gmail.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix lint errors (redundant bracket)
Add a test demonstrating the --node-tags option of hroller
The example is a cluster of 6 nodes, paired into 3 group by threeinstances. So the whole cluster would need two reboot groups. The twotags select, in two different ways, one node of each group. So, when...
Add option to hroller to select nodes based on tags
Add option --node-tags to tell hroller to consider only nodeswith these tags. A use case would be a tag tracking on whichnodes the maintenance has not yet been carried out, e.g., ifrolling reboots are interleaved with other cluster operations....