Implement network locking in Instance queries
This is needed in case more info than each nic's network uuid is tobe returned. We need to lock networks to get valid data. For nowonly the name is returned as an extra field. All other can be addedwith trivial effort....
Changes in query to support nic.network as uuid
Queries now return the network uuid as well as it's name. Here weonly use info provided be LUInstanceQueryData context.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Modify query LUs to supoprt nic.network as uuid
Make _InstanceQuery gather all network info related to instance'sNICs and in case of NETQ_INST in _NetworkQuery get all networkuuids directly from nic.network
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Add GetInstanceNetworks() config method
This will be needed for Instance Queries. It walks through theinstance's NICs and returns a list network uuids that the NICsare attached to.
cmdlib changes to support nic.network as uuid
Refactor Instance related LUs to support nic.network asa uuid. This removes all the unnecessary invocations toLookupNetwork().
Make network config methods take uuid as argument
This will be needed in the following patches where nic.networkwill refer to network's uuid and not name.
Document possible hypervisor bug in serial_console
This is described in Issue 83.
Update both the install guide and the gnt-instance manpage.In the install guide move another KVM specific example inside a new KVMadmonition for the hypervisor parameters section....
Update man pages wrt ExtStorage
Also link to its design doc in the design-2.7 doc
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>...
Rename lib/objectutils to outils.py
Back when this was introduced, I mentioned that it breaks heavily tabcompletion (ob<TAB> doesn't work anymore), but at that moment I didn'thave a suggestion what to name it. I think outils is good and shortenough, and doesn't conflict with anything else, so here it goes....
Fix typo in gnt-group manpage
The --ipol-disk-templates parameter should actually be--ipolicy-disk-templates.
Signed-off-by: Ben Lipton <ben.lipton@gmail.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix wrong type in a docstring of the RAPI subsystem
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add a function to change an OpCode's priority
This simply updates the metaopcode submit priority.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add functions to parse CLI-level format of priorities
The current serialisation format for submit priorities isinteger-based, same as the opcode json serialisation. But for CLIlevel, we need to support a string-based format, so we add functionsto parse and format this representation....
Add CLI-level option to override the priority
This just defined the new priority, with the same name as the Python one.
Enable use of the priority option in hbal
This patch adds the option to hbal, and uses it to tweak the submittedjobs. There are also two small shelltests for testing the parsing.
Finish the remote→restricted commands rename
The documentation still points to /etc/ganeti/remote-commands,although the code is already using restricted-command. Update thedocumentation and a few docstrings accordingly.
Signed-off-by: Iustin Pop <iustin@google.com>...
Make hbal opcode annotation more generic
Currently, hbal code always uses annotateOpCode function, which meanswe would have to pass the options data to all function in the callchain if we wanted to make this more flexible.
By abstracting the type of the annotator and passing it as an argument...
Force conflicts check in LUNetworkDisconnect
Until now if one disconnects a network with --no-conflicts-checkand then remove it, there is a possibility to leave instances with NICsreferencing non-existing networks. This causes network queries,instance removal and modification to fail....
If _UnlockedLookupNetwork() fails raise error
Make _UnlockedLookupNetwork() raise OpPrereqError (instead of returningNone) in case it does not find the requested network. Remove useless andduplicate code such as:
if net_uuid is None: raise...
This is a cherry-pick of commit 1cce2c4....
Removes check for conflicts from NetworkDisconnect
This removes the check for conflicts from the Haskellversion of the OpCode NetworkDisconnect. This alignesthe Haskell code with the patch"Force conflicts check in LUNetworkDisconnect" (whichis currently under review). I will submit these patches...
Change default xen root path to /dev/xvda1
All recent-enough versions of linux see the xen paravirtual device as/dev/xvd*.
This doesn't break old installations, as the default is only used on newclusters.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Fix rbd showmapped output parsing
'rbd showmapped' output formatting differs between older and newer versions ofthe ceph tools. Try to use json output formatting if available (currentlyavailable only in the ceph master branch). For bobtail, argonaut and older...
Add unit tests for RADOSBLockDevice
Add unit tests for the RADOSBlockDevice JSON output parsing function,and modify the unit tests for the plain output parsing function towork with the new code.
Signed-off-by: Stratos Psomadakis <psomas@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
QA: Support additional arguments for initialization
In some scenarios it can be useful to add more arguments to “gnt-clusterinit”, such as “--no-drbd-storage”, if DRBD is not installed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
qa_utils: Fix order of arguments passed to _AssertRetCode
Improve reporting on errors.AddressPoolError exceptions
This patch improves the error messages given when a“errors.AddressPoolError” exception is caught. Includes some small stylefixes.
Add note about lv-tags rename
This is not included on the 2.6 branch, so update the NEWS for 2.7.0 beta1.
Remove network_type slot (Issue 363)
This slot was not used by Ganeti so the same info can beprovided via tags. In order not to break configuration datawe add a FromDict() method in Network config object thatremoves the deprecated network_type (if found) and then invoke...
Remove family and size from network objects
This info is not used by Ganeti and therefore is removed.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Guido Trotter <ultrotter@google.com>
Make use of HooksDict() for networks
This can be used in hypervisor code as well. For consistencyexport *NETWORK_NAME and not *NETWORK throughout the code.
Moved uniformity check for exclusive_storage flag
Cluster-verify used to check that the value of exclusive_storage is uniformwithin node groups. Now, it's impossible to change the flag for a singlenode, so that check has been removed and an equivalent one has been added...
"exclusive_storage" cannot be changed on single nodes
There's never been support for a configuration where nodes in the same nodegroup have different values of the exclusive_storage flag. This patchdisables the possibility to change the flag for individual nodes....
Upgrades made on loading the configuration are always saved
Before, only some upgrades were written back to the configuration file. Alittle refactoring of _UpgradeConfig() has been done to write unit tests.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Show correct daemon name on Luxi connect errors
Since now confd also serves a Luxi endpoint, the current message incli.FormatError is misleading when actually failing to connect toit. The patch adds a somewhat hackish way to show the right daemonname....
Update the security document for Ganeti 2.7
This should address 359, but I'm not sure if it covers all of 2.7…
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Make ParticalNic's network field of type String
This was applied to "master" along with extra changes affecting themaster branch only. Cherry-picking just the Objects.hs change.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
ConfigData: run UpgradeConfig on network objects
Although this does nothing for now, running it is safe, and consistentwith how other objects behave.
OS environment: add network information
1) Move the hooks environment dict generator inside the object. Thisalso adds missing values such as network family and uuid.2) Use the same generator both for the os environment and for theinstance hooks.3) Update manpage and hooks documentation....
Fix OCF files installation in devel/upload
LIBDIR was not defined, so the files ended up directly under /, notgood. Now they're properly installed under /usr/lib/ocf, as perhttp://linux-ha.org/wiki/OCF_Resource_Agents.
Also add "set -u" to prevent future occurrences of this kind of bug....
Make gnt-os list work with no OSes
When absolutely no OSes are present on the cluster, the result ofOpOsDiagnose is an empty list. This is currently handled in gnt-os asan error condition, probably due to how OpOsDiagnose used to returnerrors in the past....
baserlib: Fix two mistakes in docstring
The method names were wrong due to copy & paste.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Workaround hlint behaviour with no warnings/errors
When hlint sees a perfectly nice code tree, it will skip generatingthe reports (both stdout and html). This means that usually the oldreport will remain in place, so it makes development harder, as the...
Remove use of 'head' and add hlint warning for it
Since 'head' is unsafe to use in most cases, this patch removes itsuse from most of the code, adds a lint warning for it (and for tail aswell), and adds override annotations in the few cases where it'sactually OK to use it (mainly when using head over the result of...
Add release date for 2.7.0 beta1
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Version bump for 2.7.0~beta1
Also update the documentation target versions and the cfgupgrade tool torecognize 2.6 and upgrade to 2.7
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Update hooks.rst for Ganeti 2.7
- The migrate instance hook now also executes on the primary node in the "pre" phase.- Instance hooks now get full network information on each nic, if a nic's network is set.
Redirect output for gnt-* list/list-fields commands
This enabled the redirection; QA output after this patch should becomemuch much smaller (on a 'quick' QA, we are left with ~0.5MB output onstdout and 64MB redirected output).
Add function to execute QA commands with redirected output
Some QA commands generate non-trivial output: for a full QA, the gnt-*list commands generate more than 300MB of data. To make the QA logsmore parseable by humans, we need support for redirecting output of...
Implement option to skip logging of QA commands
This patch adds an optional kwarg that allows skipping the "Command:…" line in QA output. This is helpful if we only need to run somemaintenance actions, that are QA-related and not Ganeti-related.
Fix typo in a comment
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Add some more Haskell/Python equivalence tests
This would have caught the log file problem fixed in the previouspatch (9411474b), for example. Also we test user/group equivalence,name only.
Fix Haskell log file naming after virtual cluster changes
Commit 3329f4de changed the Haskell log file from constants tofunctions, but introduced a bug: it uses now the daemon name insteadof the correct log file, which means "ganeti-confd.log" instead of...
QA: Clean up after "instance-remove-drbd-offline" test
This test used to leave a node with orphan logical volumes and unused DRBDminors. Now, the expected leftovers are cleaned up after the test, andcluster-verify is run.
Switch KVM to multi-error verify results
This uses the new _FormatVerifyResults helper function to returnmultiple errors.
Make LXC fail verification if cgroup is not mounted
Since LXC now relies on cgroup memory limits to enforce memory, let'smake hypervisor verification (and thus cluster-verify) return errorswhen the cgroup filesystem is not mounted.
Add a helper function for hypervisor verification
This will allow easier multi-error results from hypervisors; rightnow, we only report the first error, which is not nice.
hv_lxc: fix whitespace errors
The latest lxc patches included a few whitespace style errors, that makelint fail. This patch fixes those.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
s/nic/NIC/ in manpages
For consistency, let's make sure NICs are capitalized always in the sameway.
LXC: add support for the memory controller
Add support for the memory resource controller, useful to actually placememory limits on instances.
Support is still optional, in case the kernel doesn't have it compiledin, or in the case of Debian has it dependent on a kernel command-line...
LXC: adapt hv for newer lxc userspace tools
Currently hv_lxc depends on the behavior and output of older LXC tools,which have since changed, making it unable to function in currentdistributions (e.g. Debian wheezy).
Adapt the tools and expectation for the output and make it into a...
Fix typo in ganeti-noded man page
Add a missing "in".
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA: Run instance tests with different cluster configurations
Instance tests are run with or without the exclusive-storage flag set. Moreconfigurations can be added easily.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
QA: Instance tests check the configuration before running
Instance test functions check the current configuration, and they will runthe actual tests only if the configuration supports them. This will beused for refactoring in following patches.
QA: Use lists of nodes as argument to instance tests
Some instance test functions took two node arguments, some took one, andsome took two but the second argument could be None. This patch makes suchfunctions uniform by using a list of nodes as an argument. This simplifies...
QA: Refactored launching of the various instance tests
The code launching instance tests for "plain" and "drbd" templates has beenunified. In this way it's easier to add more templates, and to re-use the sametests with different configurations.
QA: Run cluster-verify after instance tests
We want to make sure that instance operations don't leave the cluster in abad state. The "instance-remove-drbd-offline" test leaves some debrisbehind, so it's been moved to the last position.
QA: Added functions to allocate/free N nodes at once
These make possible to simplify the logic of tests requiring morenodes. Used in the following patches.
QA: Added exclusive_storage to qa_config
QA configuration now tracks the state of the exclusive_storage flag atcluster level. This will be used to selectively enable tests according tothe configuration.
Also, it's now possible to specify the initial value of the...
QA fix: TestInstanceExportWithRemove() updates configuration
When the instance is removed from the cluster by this test, it's also freedin the QA configuration.
QA cleanup: Removed instance-disk-failure test
The test was broken, out of sync with the rest of the code, and prone tocrashes. Until someone does a better job, it's better to remove thetest. This simplifies further refactoring.
QA: Added disk template to config
The disk templates used by the instances are tracked in the QAconfiguration. This will be used to selectively enable tests according tothe configuration.
QA: Better initialization of a global dictionary
The global 'cfg' variable is initialized to an empty dictionary, so thereis no more need to disable some pylint checks.
QA: Cluster-verify reports shared PVs with exclusive storage
Exclusive storage forbids sharing PVs between unrelated LVs. This is a testthat cluster-verify correctly report such cases.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>...
QA: Added constants for LVM volumes
The LVM volume group becomes a configurable parameter, and a prefix forlogical volume names used in tests is introduced as a constant.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
QA: Templates incompatible with exclusive storage reported
Cluster-verify should report instances whose disk template is notcompatible with the exlusive storage setting.
QA: Uniformity check for exclusive_storage in cluster-verify
Cluster-verify should report nodes in the same nodegroup with differentvalues of the exclusive_storage flag.
QA: Test for basic features of exclusive storage
Check that creation of instances doesn't trigger any error.
QA: Added function to read cluster configuration fields
This function will be used in following patches.
QA: Added function to check cluster-verify result
Now it's possible to check that cluster-verify reports all the expectederrors, instead of simply checking that it fails.
Document better gnt-network
Specify clearly that the nic parameters depend on thenetwork-to-nodegroup connection.
Clarify in gnt-network how instance nics are connected to a network.
Fix the gnt-instance section in the htools manpage
Update 2.7 design doc
Move two designs that have been implemented in 2.7 to design-2.7 and outof drafts.
Other designs were partly implemented: explain what part was, and leavethem as drafts pending their completion.
update 2.7 NEWS file
Include a handful of important changes that weren't mentioned.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Disable live-RPC queries under split query
Currently, the node listing RPC is very slow due to missingparallelisation. For the 2.7 release, we reset these back to masterd,hoping to revert them by the time 2.8 is ready.
There are a number of queries that I've left pointing to confd, as...
hv_kvm: Original error message, keyword parameter
- Include original error message when creating TAP interface failed- Pass keyword parameter as such
CLI.hs: fix double spaces in option help strings
Some help strings with continuation backslashes ('\') were providing aspace both before and after the backslash, resulting in double spaces inhelp output. Provide it only after the backslash, which fixes the issue and...
Add new NIC parameter to man page of 'gnt-instance'
According to doc/design-network.rst, there is a new NICparameter 'network', which can be used to link an instanceto a network. This was so far undocumented in the manpage of 'gnt-instance'.
Signed-off-by: Helga Velroyen <helgav@google.com>...
Fix spelling issue in hroller.rst
'explicitly' was spelled the wrong way
Signed-off-by: Benjamin Kerensa <bkerensa@ubuntu.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
kvm: fix bug while fetching -device list
_GetKVMOutput expects the command to succeed, but unfortunately on someversions of kvm "-device ?" will output a correct list of devices, whileexiting with an error code.
To fix this we accept failure in that case (note that this doesn't...
hv_xen: Remove config after shutdown was successful
If stopping an instance failed, the configuration would already be goneand other operations depending on it (e.g. migration) would no longerwork. With this patch the configuration file is only removed once the...
hv_*: Always return from Verify, style fixes
Change all “Verify” methods in hypervisor abstractions to explicitelyreturn None if no problem was detected. Remove punctuation from errormessages. Update docstrings with “@return” and some small mistakes.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
_VerifyErrors()._Error() and _ErrorIf() are now consistent
_Error() didn't contain the logic for demoting errors to warnings and formarking an operation as failed. Now _ErrorIf() is just a minimal wrapperfor _Error().
Unit tests included.
Merge branch 'master' into devel-2.7
Handle the result of QueryGroups() correctly
If no group is given for the “gnt-network connect“/“… disconnect”commands, the client uses the result of “QueryGroups()” which is a listof lists. Use “itertools.chain()” to handle the return value correctly....
hv_xen: Compose file name outside error handling
In _ReadConfigFile, the filename should be prepared outside thetry/except block. Fixes bad code formatting, too.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
testutils: Module-level method for reading test data
This patch moves the functions “_ReadTestData” and “_TestDataFilename”to module level. They do not depend on a class instance at all.
Some cases of “utils.ReadFile(self._TestDataFilename(…))” are also...
hv_base: Remove empty constructor
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Add test for backend._GetBlockDevSymlinkPath
Add a unit test for the trivial “_GetBlockDevSymlinkPath” function inbackend (small changes in the function were required).
Fix format string of KVM output
This fixes a missing 's' in the format string andthe wrong quotes. Those bugs were introduced incommit 6e043e60.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>