Use dcName in mon-collector
Instead of manually specify the name of the data collectors in mon-collector,just use the dcName field each of them exports.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Monitoring design doc: better specify field names
The name of the list of instances was not specified.
Also, fix a line that was longer than 80 characters.
Factor out the mergeStatuses function
It will be used by multiple data collectors, not only the DRBD collector.
Add dependency on the process library
The tests are already using this library, so it's not really a new builddependency, but it was not specified esplicitly.
Furthermore, it's going to be used by the instance status collector, so it'sadded to the requirements for the monitoring subsystem....
Add HS functions for getting the instance reason path
The getInstReasonFilename is built to resemble the python correspondingfunction.
Add module containing function for getting info from Xen
The Xen instance status data collector will require to get some informationfrom the hypervisor. This commit introduces a module providing such functions.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add the core of the instance status collector
Add the Xen instance status data collector with only its core features.The next commits will add more reporting functionalities.
The access to the collector is made possible through the mon-collectortool....
Export the actual instance state
Compute the actual state of the instance and export it.
Determine status of one instance
Added function for determining whether the status of an instance is ok, and torepresent this information in the corresponding field in the report.
Include the reason trail in the instance collector output
Fetch the reason trail from file, failing gracefully if it is not found, andinclude it in the output of the instance status data collector.
Export Instance Status collector information
Name, version, format version, category and kind of the Instance Status datacollector are now exported.
Factor out function for building report
Instead of building the report as part of the "Main" function, have itbuilt by its own dedicated function, so that it will be able to export itdirectly to the monitoring daemon when needed.
Add example for online rolling reboots using tags
While this use case was described in the design document, andmentioned several times as motivation for changes in commit messages,it has never been added to a user-facing documentation. This commitadds at least an example to the man page....
Extend hroller test to also verify tag-based node selection
While the multiple-tags test was added to verify that coloring is doneonly after node selection (otherwise it wouldn't be possible to get inboth cases a single reboot group), it can easily be extended to also...
Add a test for online rolling reboot scheduling
In the example configuration, the graph constructed by just connectingprimary and secondary instances is two-colorable. However, when takingconflicting locations of secondary nodes into account, three reboot...
Support online-maintenance in hroller
Make hroller take into account the nodes (redundant) instanceswill be migrated to. This be behavior can be overridden by the--offline-maintenance option which will make hroller plan underthe assumption that all instances will be shutdown before starting...
Support construction of the graph of all reboot constraints
For online rolling reboots, there are two kind of restrictions. First,we cannot reboot the primary and secondary nodes of an instancetogether. Secondly, two nodes cannot be rebooted simultaneously, if...
Add option --one-step-only to hroller
Add a new option to hroller to only output information about the firstreboot group. Together with the option --node-tags this allows for thefollowing work flow. First tag all nodes; then repeatedly compute thefirst node group, handle these nodes and remove the tags. In between...
Sort reboot groups by size
Make hroller output the node groups not containing the master nodesorted by size, largest group first. The master node still remainsthe last node of the last reboot group. In this way, most progressis made when switching back to normal cluster operations after the...
Fix expectation in hroller test
Regular expressions are not shell globs. So "any symbol" is expressedby a dot, not a question mark. In this case, the confusion lead to atoo liberal expectation, hence the test passed. Fix it nevertheless.
Signed-off-by: Klaus Aehlig <aehlig@google.com>...
Refactor check for exclusive_storage in LUInstanceCreate
The order of evaluation of the conditions is changed, so it's easier to addmore (foreseen) checks for exclusive_storage.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
Refactor disk checks in LUInstanceSetParams
Prereq checks relative disks are grouped together and moved in a separatemethod. This reduces the clutter in CheckPrereq().
Fix lint errors (redundant bracket)
Signed-off-by: Klaus Aehlig <aehlig@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add a test demonstrating the --node-tags option of hroller
The example is a cluster of 6 nodes, paired into 3 group by threeinstances. So the whole cluster would need two reboot groups. The twotags select, in two different ways, one node of each group. So, when...
Add option to hroller to select nodes based on tags
Add option --node-tags to tell hroller to consider only nodeswith these tags. A use case would be a tag tracking on whichnodes the maintenance has not yet been carried out, e.g., ifrolling reboots are interleaved with other cluster operations....
Make Rapi backed set node tags correctly
Since the htools representation of a node now allows addingthe node tags, populate this field correctly in the Rapibackend.
Make LUXI backed set node tags correctly
Since the htools representation of a node now allows addingthe node tags, populate this field correctly in the LUXIbackend.
Extend the text format to contain node tags
In order to allow htools to make use of node tags, add them to thetext format. This is done by adding a new column at the end of thenode lines. If this column is missing, the default value (whichis the empty list) is left unchanged, thus yielding the current...
Extend the Node in the htools to allow adding node tags
Since hroller (and probably other tools in the future) will supportnode selection based on node tags, extend the node data structure toallow adding this information.
Make hroller filter the nodes before coloring the graph
Hroller used to first compute a coloring of the node graph and thenfilter out the nodes that it had to work on. While the only filteringwas according to node groups this did not make a difference, as there...
Make mkNodeGraph ignore edges to non-present nodes
Change the behavior of mkNodeGraph to tacitly ignore all instanceswhere one of the nodes is not in the list of nodes. In this way, wecan construct sub-graphs by filtering the nodes and ignoring anypossibly added isolated nodes for the missing indexes....
Add tests for the -O option of hroller
In hroller, the option -O can be used to mark certain nodes as offline.These nodes should then not be part of any reboot group. Add teststo verify this behavior.
Update hroller man page
In commit 7dbe4c725 the new option --force was introduced tohroller. Change the man page to reflect this change.
Mention DRBD 8.4 support in NEWS
Mention the main features of DRBD 8.4 support in the NEWS file.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix `drbdsetup show` parsing for DRBD 8.4
In DRBD 8.4, Ganeti does no longer use indexed external meta data butflexible external metadata. Therefore, there is no longer a meta dataindex in the output of `drbdsetup show`.
Signed-off-by: Thomas Thrainer <thomasth@google.com>...
Correction of design doc regarding RPC extension
This corrects the storage types design doc regarding the specificationof the RPC call.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Bernardo dal Seno <bdalseno@google.com>
Change design of algorithm for computing rolling reboots
Instead of computing a coloring for one condition first and then refiningit for the other condition, we can construct a graph with edges for allconditions that prevent simultaneous reboots. This will not only result...
Support /var/run being a symlink in upload
Starting from Debian Wheezy, /var/run is a symlink to /run. If rsyncdoes not treat receiving symlinks to directories as directories, itwould remove the original symlink and create a new, empty directorywhere only the files to sync are put in. In the case of /var/run, this...
Split DRBD8Dev into DRBD8 and DRBD8Dev
All functionality specific to a single DRBD8 devide is now in DRBD8Dev,whereas functionality which is valid for the whole DRBD "installation" on a device is collected in DRBD8.
This makes it possible to remove a couple of FIXME's and clarifies the...
Check for uniform DRBD version in node group
This check is performed during gnt-cluster verify and outputs warningsif at least two different DRBD versions are found within a node group.In such a case, all nodes with their installed DRBD version aredisplayed....
Rename DRBD8 to DRBD8Dev
Right now the DRBD8 class has multiple responsibilities: a) itreprensents a device which can be set up, grown, etc. and b) itrepresents the whole DRBD system on a node which has a usermode helper,which knows how to shut down all devices, etc. Therefore, the DRBD8Dev...
Make DRBD version queryable from noded
gnt-cluster verify should issue a warning if there are multiple DRBDversions present in a node group. In order to do so, the DRBD versionhas to be queryable from noded.
Merge remote-tracking branch 'origin/stable-2.8'
Conflicts: lib/bdev.py lib/cmdlib.py
Disk templates are "allowed" and not "enabled" in ipolicies
The way disk templates were displayed in instance policies became confusingsince the introduction of the enabled_disk_templates cluster parameter.
Fix issue 440.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Unit test for FillIPolicy keeping unknown keys
This makes sure that issue 401 doesn't come back.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Fix white spaces in partitioned design doc
Bullet lists start after a blank line.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Reuse current spindles in design for partitioned
Renaming the current parameters dealing with spindles is nice, as theirmeaning would be clearer, but that would mean changing a lot of code,change external interfaces, and renaming internal variables for little...
Merge stable-2.7 into stable-2.8
Conflicts: INSTALL NEWS configure.ac src/Ganeti/Query/Group.hs test/hs/Test/Ganeti/Objects.hs test/hs/Test/Ganeti/Query/Query.hs
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Cleanup of the INSTALL file
Curl is not optional anymore. Remove reference to it in the optional librariessection.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Support DRBD 8.4.3 command syntax
DRBD 8.4.3 does not support specifying volume ids directly whilecreating the meta device. It requires a slightly different syntax (usingflex-external as volume id) which is fortunately also supported by (atleast) DRBD 8.4.2....
gnt-cluster modify: check vg name only on vm-capable nodes
This fixes issue 432: when setting the volume group name, the presenceof the volume group was checked on all nodes. This patch now restrictsthe check only to the vm-capable ones.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Disable inter-cluster instance move for file instances
Inter-cluster instance moves depends on exporting an instance. For file-basedinstance, exporting is not supported. Therefore, this patch disablesinter-cluster instance moves for file-based instances and informs the user...
Improve error message for file instance creation
Currently, when creating an instance with file storage fails,the error message is rather general. This patch adds a hinton what to do as a possible fix.
Update NEWS for 2.7.0 rc1
Move the "main" 2.7 section up to rc1, and just leave the delta inbeta3. Then add a new beta3 to rc1 delta.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
admin guide updates for 2.7
- Mention the file-storage-paths config file- Add information about sharedfile and ext storage types- Change one manpage reference to a proper link
Mention file-storage-paths in UPGRADE
It was in NEWS, but not mentioned at all in the upgrade notes.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Klaus Aehlig <aehlig@google.com>
kvm: use -machine type,accel= for newer qemu
-enable-kvm and -disable-kvm are deprecated in favor of accel=kvm passedas a machine specification. This fixes compatibility with newer qemuversions.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
kvm: Move parameter validation before use
We had parameter validation when the runtime is generated, but in themiddle of the function. Move up, before any parameter is accessed.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Thomas Thrainer <thomasth@google.com>
kvm: Use -display none rather than -nographic
Newer versions of kvm (from 1.4) don't support -nographic with-daemonize. When -display exists, use -display none instead.
This fixes Issue 389.
INSTALL updates for wheezy, and fixes
- Remove mention of python 2.4 specific modules, as that version is not supported anymore- Add bitarray and ipaddr- Remove a double space- Add mandatory C library packages too- Mention the option of using drbd modules directly from your kernel...
Disable compiling mon-collector if monitoring is disabled
Mon-collector kept being compiled even if monitoring was disabled duringconfigure.
Fixes Issue 441.
Add confd as a dependency of monitoring
Some data collectors require confd data information. Therefore, monitoring isdependent on confd, but this was not checked.
This commit introduces such a check and partially fixes Issue 441.
Add reason trail to haskell opcode format
The haskell type definition of opcodes should remain aligned with the pythonone.
Add the reason trail type definition in Haskell
It will be added to the haskell definition of opcodes, to keep it aligned tothe python one, and it will be used for fetching the reason trail by theinstance status data collector.
Update "reason" field in instance status design
Now the reason field is implemented according to the reason trail designdocument.
Remove outdated TODO comment
Add proper error management to monitoring daemon
Produce a personalized 404 error when the requested resource is not available.
Produce a report for all the data collectors
Implement the API function of the monitoring daemon that provides the reportof all the data collectors.
Generate report for a single collector
Allow to ask the monitoring daemon for the report of one specific datacollector.
Export the full report from DRBD data collector
Export the full report instead of just the data from the DRBD data collector.
Add monitoring daemon to the masterd group
The hmac.key file is owned by group masterd and is required to communicate withthe confd.
The monitoring daemon need to communicate with the confd so it has to be part ofthe masterd group.
Update JSON serialization for "category"
Change the JSON serialization for the "category" field of data collectors,in accordance to the latest version of the design document.
Improve monitoring agent design document
Specify more precisely the format of the category names.
Implement the "list collectors" API in monitoring
Implement the handler for outputting the list of collectors(name, category, kind) in JSON format.
Fix bug in group queries related to node/instance fields
Since we use the primitive string type for group UUIDs, the groupfields have a bug where we pass the group name as filter for nodetests, whereas the nodes themselves use the group UUID. This results...
gnt-instance: new hypervisor parameter 'vif_type'
This patch fixes issue 247. It empowers the user to specifyexplicitly the vif type of a nic configuration of a Xen HVMinstance. This includes the option of setting it to '' on thecommandline, causing Ganeti to leave out the 'type' parameter...
man gnt-instance: new hypervisor parameter 'vif_type'
This patch extends the man page of the 'gnt-instance' commandby describing the newly implemented hypervisor parameter'vif_type'.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'stable-2.8' into master
Cleanup DRBD 8.4 devices properly during QA
QA cleans up DRBD 8.4 devices via drbdsetup, which currently only workedfor DRBD 8.3. This (rather ugly) command makes sure that DRBD 8.4devices are cleaned up as well.
Add command generator for DRBD 8.4
The DRBD84CmdGenerator class, which generates commands suited for DRBD8.4, is added. A common baseclass for DRBD83CmdGenerator andDRBD84CmdGenerator is introduced as well.
Add test for version dispatching code
Based on the version of DRBD (as reported through /proc/drbd), the DRBD8class chooses helper objects which adapt for the differences betweenDRBD <= 8.3 and DRBD 8.4. This new test verifies that the proper objectsare created....
Add dependency on python-mock
Python Mock (http://www.voidspace.org.uk/python/mock/) is a mockinglibrary for Python. It is included in the standard Python distributionas of Python 3.3, but has been around for quite a while. Debian unstablecontains a version we require....
Extract DRBD related tests
As the DRBD code was extracted from bdev.py, the corresponding testsshould as well be extracted into their own file. That's what this patchperforms.
Signed-off-by: Thomas Thrainer <thomasth@google.com>Signed-off-by: Michele Tartara <mtartara@google.com>...
Extract command generation logic from DRBD8
In order to easily support the DRBD 8.4 command syntax, this patchextracts all command generation logic (i.e. where calls to `drbdsetup`are assembled) into a new class DRBD83CmdGenerator.
Use DRBD84ShowInfo based on DRBD version
Now the DRBD8 class actually checks which version of DRBD is running onthe system and chooses the right class to parse the `drbdsetup show`output.
Add `drbdsetup show` parser for DRBD 8.4
Common functionality between the DRBD 8.3 and DRBD 8.4 parser has beenextracted into BaseShowInfo. A test which verifies the behaviour isincluded, but the DRBD84ShowInfo class is not yet used within the DRBD8class....
Extract DRBD info parsing into drbd_info.py
Parsing the status (/proc/drbd) and `drbdsetup show` output takes quitea bit of code, which has to be augmented by even more with the DRBD 8.4support. So extract all the related classes into their own file.
Extract DRBD8ShowInfo class
This class parses the `drbdsetup show` output and represents it in aneasily accessible format. It got extracted so that 1) the DRBD8 classcan focus more on DRBD logic rather than parsing and 2) it's easier toadapt the parser to new formats....
Add test for DRBD 8.4 /proc/drbd parsing
The current parser already supports the output of DRBD 8.4 /proc/drbd,so add tests which actually verify this behaviour.Note that the empty lines in proc_drbd84.txt are actually present in theoutput of DRBD 8.4, they always appear when minors are not contiguous....
Add test for empty version in /proc/drbd
The previous patch removed a (rather implicit) test for /proc/drbd notcontaining a version line. This adds an explicit test for this case.
Rename _JoinProcDataPerMinor to _JoinLinesPerMinor
In the new context of the DRBD8Info class, the name_JoinProcDataPerMinor doesn't fit any more. _JoinLinesPerMinor reflectsmore precisely what's done, and doesn't imply that the data comes from/proc/drbd....
Extract /proc/drbd parsing code into DRBD8Info
As the DRBD8 class got bigger due to the previous merge of BaseDRBD, nowparts of it are ripped out into DRBD8Info. This new class parses/proc/drbd and exposes the information in an easily accessible way. This...
Remove BaseDRBD
BaseDRBD was probably useful when DRBD 0.7 and DRBD 8 were supported.However, there is only one subclass of BaseDRBD remaining (DRBD8), andthe separation of responsibilities between those two classes was ratherrandomly chosen.
The unification into one class also eases the introduction of...
Rename _MassageProcData to _JoinProcDataPerMinor
That's what the method actually does, so there is no reason why weshould use a funny but unreadable name.
Add shelltests for hspace allocation
Both tiered and standard allocations are tested, with a single and a doublemin/max instance specification.
hspace: Handle multiple ipolicy specs
With tiered allocation, hspace uses all the max specs in turn as theinitial instance spec.
QA: Test multiple instance specs
The test of the interaction between policy and instances is extended.
QA: Handle multiple instance specs
Existing tests are updated to cope with the new instance specs format.
Unit test for cli.FormatPolicyInfo()
We try to parse the string obtained by formatting a policy, and see if weget the same policy.
Add command-line support for multiple specs in ipolicy
Command line options accept multiple min/max specifications in instancepolicies. The output of show-ispecs-cmd is updated accordingly.