HRoller design updates
- Specify that there will be options for selecting nodes by at least nodegroups and tags, rather than just individually.- Specify a better handling for non-redundant instances (eg. plain or file) which today are simply ignored...
baserlib: Make "_OPCODE_ATTRS" public
"_OPCODE_ATTRS" will also be used in the Sphinx extension.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Makefile: Add dependencies
The RAPI documentation now depends on “connector.py” (handlerregistration) and “rlib2.py” (access permissions).
RAPI documentation: Add link targets
A newly added table of RAPI access permissions will link to allresources and their supported methods. For this reason link targets needto be added.
The GET method on “/2/nodes/[node_name]” was not separately mentioned....
Helper to retrieve access permissions for RAPI resource
Merge similar code into a helper function.
Sphinx extension: Module-level constant for tab width
Use a module-level constant for the tab width.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
RAPI documentation: Assertion for console fields
Assert that the documented fields are equal to those in the actualobject.
Status change reason support for Reboot
Add support to the Reboot command for specifying the reason for the laststatus change.
Some features are implemented as functions, even if used only once, becausethey will be used by the future patches introducing reason support for all...
Infrastructure for specifying instance status change reason
This patch introduces some infrastructural modifications that will be used bythe following commits to implement the support for specifying the reason forthe last status change of an instance....
Unit tests for Query/Network.hs
This patch adds a couple of unit tests for Query/Network.hs.Note that they'll need to be adapted, once issue 362 is addressed.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Make Confd client usable for testing
Allow the Confd client to be able to connect to an arbitrary serverinstead of just the real one running on a cluster.
This is meant to be used for testing.
If a data collector using the Confd client is tested with shelltest, it will...
Add Haskell parser for "xm uptime"
In order to fetch precise information about the uptime of the VMsrunning in Xen, we need to analyze the output of the "xm uptime" command.
This commit adds the parser to do that, and its tests.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add Haskell parser for "xm list --long"
In order to fetch precise information about the status of the VMs running inXen, we need to analyze the output of the "xm list --long" command.
Add request type to Confd server for getting instance list
Add to Confd server a new request type (and its implementation) to ask forthe list of instances in a node.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Update monitoring agent report format
This commit expands the monitoring agent design document improving thedescription of the general report file format.
There can be two kinds of data collectors (performance and status collectors).
Every collector can now be part of a category of collectors (such as...
Design correct reporting of storage free space
This is a long standing bug in Ganeti. Add a small design on how we planto fix this for Ganeti 2.7.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Merge branch 'devel-2.7'
Revert "Disable live-RPC queries under split query"
This reverts commit fb251c2c4c582ec0d6c00a6f6c5e134ed5196e03. On themaster branch we want to continue to have them enabled.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
Rename lib/objectutils to outils.py
Back when this was introduced, I mentioned that it breaks heavily tabcompletion (ob<TAB> doesn't work anymore), but at that moment I didn'thave a suggestion what to name it. I think outils is good and shortenough, and doesn't conflict with anything else, so here it goes....
Add hroller design to Makefile and design-drafts
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Hroller design doc
Hroller in 2.7 was implemented (as an experiment) but never explicitlydesigned. This design mentions the current functionality, and explainsthe changes that will happen for the next release to support livemaintenance scheduling (rolling reboot)....
Fix typo in gnt-group manpage
The --ipol-disk-templates parameter should actually be--ipolicy-disk-templates.
Signed-off-by: Ben Lipton <ben.lipton@gmail.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix wrong type in a docstring of the RAPI subsystem
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA: Set disk template directly via instance object
Use a method on the instance object instead of going through“qa_config.GetInstanceTemplate”. This provides for better encapsulation.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Remove qa_config.GetInstanceNicMac
Call the wrapped instance method directly.
QA: Use constants for disk templates
Disk template names shouldn't be hardcoded.
Fix a merge issue in 3c296f56
Lines removed in devel-2.7 (250a9404) weren't removed in the merge withmaster.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add a function to change an OpCode's priority
This simply updates the metaopcode submit priority.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add functions to parse CLI-level format of priorities
The current serialisation format for submit priorities isinteger-based, same as the opcode json serialisation. But for CLIlevel, we need to support a string-based format, so we add functionsto parse and format this representation....
Add CLI-level option to override the priority
This just defined the new priority, with the same name as the Python one.
Enable use of the priority option in hbal
This patch adds the option to hbal, and uses it to tweak the submittedjobs. There are also two small shelltests for testing the parsing.
Finish the remote→restricted commands rename
The documentation still points to /etc/ganeti/remote-commands,although the code is already using restricted-command. Update thedocumentation and a few docstrings accordingly.
Signed-off-by: Iustin Pop <iustin@google.com>...
Make hbal opcode annotation more generic
Currently, hbal code always uses annotateOpCode function, which meanswe would have to pass the options data to all function in the callchain if we wanted to make this more flexible.
By abstracting the type of the annotator and passing it as an argument...
Force conflicts check in LUNetworkDisconnect
Until now if one disconnects a network with --no-conflicts-checkand then remove it, there is a possibility to leave instances with NICsreferencing non-existing networks. This causes network queries,instance removal and modification to fail....
If _UnlockedLookupNetwork() fails raise error
Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returningNone) in case it does not find the requested network. Remove useless andduplicate code such as:
if net_uuid is None: raise...
This is a cherry-pick of commit 1cce2c4....
Removes check for conflicts from NetworkDisconnect
This removes the check for conflicts from the Haskellversion of the OpCode NetworkDisconnect. This alignesthe Haskell code with the patch"Force conflicts check in LUNetworkDisconnect" (whichis currently under review). I will submit these patches...
Change default xen root path to /dev/xvda1
All recent-enough versions of linux see the xen paravirtual device as/dev/xvd*.
This doesn't break old installations, as the default is only used on newclusters.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix rbd showmapped output parsing
'rbd showmapped' output formatting differs between older and newer versions ofthe ceph tools. Try to use json output formatting if available (currentlyavailable only in the ceph master branch). For bobtail, argonaut and older...
Add unit tests for RADOSBLockDevice
Add unit tests for the RADOSBlockDevice JSON output parsing function,and modify the unit tests for the plain output parsing function towork with the new code.
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
QA: Support additional arguments for initialization
In some scenarios it can be useful to add more arguments to “gnt-clusterinit”, such as “--no-drbd-storage”, if DRBD is not installed.
qa_utils: Fix order of arguments passed to _AssertRetCode
Improve reporting on errors.AddressPoolError exceptions
This patch improves the error messages given when a“errors.AddressPoolError” exception is caught. Includes some small stylefixes.
Add note about lv-tags rename
This is not included on the 2.6 branch, so update the NEWS for 2.7.0 beta1.
Remove network_type slot (Issue 363)
This slot was not used by Ganeti so the same info can beprovided via tags. In order not to break configuration datawe add a FromDict() method in Network config object thatremoves the deprecated network_type (if found) and then invoke...
Remove family and size from network objects
This info is not used by Ganeti and therefore is removed.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
Make use of HooksDict() for networks
This can be used in hypervisor code as well. For consistencyexport *NETWORK_NAME and not *NETWORK throughout the code.
QA: Remove dictionary-like interface from config objects
These are no longer used.
QA: Drop dictionary support from qa_utils._GetName
This is no longer needed now that instance/node objects in QA haveattributes. Use a helper function instead to retrieve an entity'sidentifying attribute.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
QA: Convert instances from items to attributes
The new objects support attributes. Item access (like a dictionary) isgoing away in a couple of patches.
QA: Convert nodes from items to attributes
The new objects support attributes, which are cleaner thandictionary-style access. Item access (like a dictionary) is going awayin a couple of patches. Also, pylint is better at checking attributesthan dictionary entries....
QA: Release nodes through method
Instead of having a module-level function in “qa_config” to release anode, a new method on node objects is used.
Moved uniformity check for exclusive_storage flag
Cluster-verify used to check that the value of exclusive_storage is uniformwithin node groups. Now, it's impossible to change the flag for a singlenode, so that check has been removed and an equivalent one has been added...
"exclusive_storage" cannot be changed on single nodes
There's never been support for a configuration where nodes in the same nodegroup have different values of the exclusive_storage flag. This patchdisables the possibility to change the flag for individual nodes....
Upgrades made on loading the configuration are always saved
Before, only some upgrades were written back to the configuration file. Alittle refactoring of _UpgradeConfig() has been done to write unit tests.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
QA: Convert nodes to objects
Up until now nodes were stored as a dictionary. The keys were hardcodedin a lot of places and entries modified directly.
This patch introduces a new class for nodes in QA named “_QaNode”. Itstill supports accessing details via dictionary syntax, but that will be...
Show correct daemon name on Luxi connect errors
Since now confd also serves a Luxi endpoint, the current message incli.FormatError is misleading when actually failing to connect toit. The patch adds a somewhat hackish way to show the right daemonname....
Update the security document for Ganeti 2.7
This should address 359, but I'm not sure if it covers all of 2.7…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA: Release instances through method
Instead of having a module-level function in “qa_config” to release aninstance, a new method on instance objects is used.
QA: Convert instances to objects
Up until now instances were stored as a dictionary. The keys werehardcoded in a lot of places and entries modified directly.
This patch introduces a new class for instances in QA named“_QaInstance”. It still supports accessing details via dictionary...
QA: Release instance in ganeti-qa
Release instances directly from “ganeti-qa” instead of doing so in thethree tests removing instances. This is in line with how nodes arereleased and makes it easier to find and debug double-release issues.
QA: Stop using "in" operator for node
The new configuration object for nodes will not support it anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
QA: Support entity objects in qa_utils._GetName
This is a temporary solution until nodes and instances are converted toobjects with attributes.
qa_config: Simplify IsTemplateSupported
The same code can be written in a single expression. The earlier patchnamed “Refactor storage of runtime exclusive storage flag in QA”provided the unit test.
Refactor storage of runtime exclusive storage flag in QA
This is a follow-up for “qa_config: Remove exclusive storage flag fromconfig”. Instead of storing the flag in a module-level variable it isnow stored within the new QA configuration class and unit tests are...
Refactor QA configuration code
Ever since its introduction (sometime before commit cec9845 in September2007), the QA configuration was stored in a dictionary at module-levelin “qa/qa_config.py”. The configuration was loaded, verified andevaluated using module-level functions. Since then the configuration has...
qa_config: Remove exclusive storage flag from config
Commit 6a0f22e added the ability to track the status of the cluster-wideflag for exclusive storage. It did so in a way which modifies thein-memory configuration dictionary. This patch changes the code to use a...
Make ParticalNic's network field of type String
This was applied to "master" along with extra changes affecting themaster branch only. Cherry-picking just the Objects.hs change.
ConfigData: run UpgradeConfig on network objects
Although this does nothing for now, running it is safe, and consistentwith how other objects behave.
OS environment: add network information
1) Move the hooks environment dict generator inside the object. Thisalso adds missing values such as network family and uuid.2) Use the same generator both for the os environment and for theinstance hooks.3) Update manpage and hooks documentation....
Fix OCF files installation in devel/upload
LIBDIR was not defined, so the files ended up directly under /, notgood. Now they're properly installed under /usr/lib/ocf, as perhttp://linux-ha.org/wiki/OCF_Resource_Agents.
Also add "set -u" to prevent future occurrences of this kind of bug....
Make gnt-os list work with no OSes
When absolutely no OSes are present on the cluster, the result ofOpOsDiagnose is an empty list. This is currently handled in gnt-os asan error condition, probably due to how OpOsDiagnose used to returnerrors in the past....
baserlib: Fix two mistakes in docstring
The method names were wrong due to copy & paste.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
vcluster: Make _MakeNodeRoot public
The QA code will also have to generate virtual paths and this functioncomes in handy.
Workaround hlint behaviour with no warnings/errors
When hlint sees a perfectly nice code tree, it will skip generatingthe reports (both stdout and html). This means that usually the oldreport will remain in place, so it makes development harder, as the...
Remove use of 'head' and add hlint warning for it
Since 'head' is unsafe to use in most cases, this patch removes itsuse from most of the code, adds a lint warning for it (and for tail aswell), and adds override annotations in the few cases where it'sactually OK to use it (mainly when using head over the result of...
Harep.hs: fix a couple typos in comments and docstrings
Signed-off-by: Dato Simó <dato@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
hv_kvm: Error messages, punctuation, other style fixes
- Capitalize acronyms such as “QMP” or “SPICE”- Remove punctuation from error messages- Improve error reporting when sending a monitor command failed- Put “or” at end of line- Fix indentation for a concatenated string...
hv_kvm: Add TODO regarding monitor commands
Monitor calls should be replaced with QMP once KVM >= 0.14 is required.
Add release date for 2.7.0 beta1
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Fix improperly formatted docstring
Change the docstring of chompPrefix to prevent the error"doc comment parse failed" that was raised by some version ofhaddock while generating the documentation for this function.
Version bump for 2.7.0~beta1
Also update the documentation target versions and the cfgupgrade tool torecognize 2.6 and upgrade to 2.7
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Update hooks.rst for Ganeti 2.7
- The migrate instance hook now also executes on the primary node in the "pre" phase.- Instance hooks now get full network information on each nic, if a nic's network is set.
Redirect output for gnt-* list/list-fields commands
This enabled the redirection; QA output after this patch should becomemuch much smaller (on a 'quick' QA, we are left with ~0.5MB output onstdout and 64MB redirected output).
Add function to execute QA commands with redirected output
Some QA commands generate non-trivial output: for a full QA, the gnt-*list commands generate more than 300MB of data. To make the QA logsmore parseable by humans, we need support for redirecting output of...
Implement option to skip logging of QA commands
This patch adds an optional kwarg that allows skipping the "Command:…" line in QA output. This is helpful if we only need to run somemaintenance actions, that are QA-related and not Ganeti-related.
Making ParticalNic's network field of type String
This is yet another fix for type confusion between pythonand haskell. ;) The network field of PartialNic should bea string and not of type Network. This makes it necessaryto add a helper function to look up a network by name...
Fix typo in a comment
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add some more Haskell/Python equivalence tests
This would have caught the log file problem fixed in the previouspatch (9411474b), for example. Also we test user/group equivalence,name only.
Fix Haskell log file naming after virtual cluster changes
Commit 3329f4de changed the Haskell log file from constants tofunctions, but introduced a bug: it uses now the daemon name insteadof the correct log file, which means "ganeti-confd.log" instead of...
QA: Clean up after "instance-remove-drbd-offline" test
This test used to leave a node with orphan logical volumes and unused DRBDminors. Now, the expected leftovers are cleaned up after the test, andcluster-verify is run.
qa_config: Remove useless "options" variable
It is only used in one place and not necessary.
Extract container converters
“objects.ConfigObject” contains two useful functions for working withcontainers of serialized objects, “_ContainerToDicts” and“_ContainerFromDicts”. This patch separates those functions and movesthem into “objectutils” as they'll be useful for converting parts of the...
hv_xen: Fix epydoc error
“utils.RunCmd” is re-exported from “utils.process.RunCmd”. Epydocdoesn't fully understand this, so we have to refer to the original.
objects: Improve handling of TCP/UDP port pool
- Handle de-serialization correctly when pool is not defined- Serialize to empty list when the attribute is None (this should never happen in reality as the attribute is always set when de-serializing)- Add tests...
Switch KVM to multi-error verify results
This uses the new _FormatVerifyResults helper function to returnmultiple errors.
Make LXC fail verification if cgroup is not mounted
Since LXC now relies on cgroup memory limits to enforce memory, let'smake hypervisor verification (and thus cluster-verify) return errorswhen the cgroup filesystem is not mounted.
Add a helper function for hypervisor verification
This will allow easier multi-error results from hypervisors; rightnow, we only report the first error, which is not nice.
hv_lxc: fix whitespace errors
The latest lxc patches included a few whitespace style errors, that makelint fail. This patch fixes those.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>