Rename/make uniform the other query entities
Following the new naming style introduced in Exports.hs, this patchrenames the other resources to export non-qualified names (fieldMap asopposed to nodeFieldMap), and to use qualified module imports.
Also fixes a haddock issue in a docstring....
QA: Support diskless instances
This is an initial implementation of diskless instances.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
QA: Compare ipolicy directly with None
If a value was set to numeric 0, the parameter wouldn't be passed to thecommand.
QA: List all ipolicy options in example configuration
This makes it easier to know what options are available.
QA: Update assertion for diskless instances
QA: Disable reinstall test for diskless instances
QA: Fix dictionary formatting
Make it match the style guide.
QA: Disable drbd-offline test if no DRBD instance is used
If “instance-add-drbd-disk” is not enabled it usually also means to notuse DRBD at all.
Test AsyncStreamServer with abstract unix sockets
This was meant to be since "the beginning" but was never submitted assomehow it failed with python 2.4. Now that the minimum python versionhas been increased it can be added.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
QA: Disable RAPI tests for virtual clusters
The QA script connects to the RAPI daemon. In virtual cluster setups thedaemon is usually listening on a local interface not reachable from theoutside.
QA: Add virtual cluster support to queue drain test
The path to the queue drain file must use the virtual node path.
QA: Disable copyfile test for virtual clusters
This test will need some more work for virtual clusters. Disable it inthe meantime.
QA: Disable OOB tests for virtual clusters
They do not work properly at the moment.
QA: Update virtual node's RAPI users file
The file is located below the virtual node directory.
QA: Read virtual node's RAPI certificate
The virtual node prefix must be added to the path.
qa_os: Disable checks for virtual clusters
The OS definitions are not virtualized. Tests configuring different OSdefinitions on different nodes don't work properly.
qa_instance: Virtual cluster support for instance list
Use path of virtual node when retrieving ssconf's instance list.
qa_daemon: Virtual cluster support
Use virtual node path when removing the watcher status file.
qa_utils: Support virtual cluster for backup files
“qa_utils.BackupFile” is always called with the path for a normal node.In virtual clusters the node prefix path must be added.
qa_utils: Virtual cluster support for SSH commands
In virtual clusters, all “gnt-*” and “ganeti-*” commands requireenvironment variables to be set for the virtual cluster root directoryand node name. The “cmd” script generated by “vcluster-setup” is awrapper setting those variables....
qa_config: Support callable test checks
Until now all tests depended on simple AND/OR conditions. With theintroduction of virtual cluster support, some tests can only run ifvirtual clusters are not in use (e.g. not yet supported or notpossible). Instead of introducing more logic for enabling/disabling...
QA: Add config entries for virtual cluster
If these new options are set, a virtual cluster is created and used.Virtual clusters consist of 1..N virtual nodes on a single physical (orvirtual) node. The design of virtual clusters is described in“doc/design-virtual-clusters.rst”....
Fix typo in docstring
Signed-off-by: Christos Stavrakakis <cstavr@grnet.gr>Reviewed-by: Michele Tartara <mtartara@google.com>
rapi.rst: Replace manual method lists with directive
Many resources, but not all, list their supported methods. With a newreST directive, this list is now automatically generated and also liststhe required permissions.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
HRoller: allow filtering by node group
Accept the -G option, and if it's passed require that it matches anodegroup, then only output nodes belonging to that group.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
HRoller: print only online nodes
To make the graphs work even when instances live on offline nodes (eg.because we're offlining them just to exclude them, or because they haveinstance still on them) we just filter them out at the end, when we'regoing to print out the result....
Officially require Python 2.6+
We have no way to test Python 2.4 compatibility anymore, and thatversion is long deprecated. From version 2.8+ require Python 2.6 orabove (but still not 3.0).
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
bootstrap: Wait for SSH daemon to become reachable
In some situations after restarting the SSH daemon by“prepare-node-join” it's not immediately reachable. Like with the masterand node daemons, waiting for it to become reachable instead of failinghard can help in such situations....
TcpPing: Better error handling, logging
Log every attempt to reach a port on another node. Don't silently returnFalse if a target not recognized as an IP address is given (e.g. ahostname).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Wipe disks added through "gnt-instance modify"
In issue 353 Sascha Lucas reported that disks are not wiped when addedthrough “gnt-instance modify”. This patch adds this functionality andupdates the docstring for “_WipeDisks”.
vcluster-setup: Create $vnode/etc/ganeti directory
This directory is required by “cfgupgrade” and some other tools.
constants: Use SSH constant for daemon ports
Moving the SSH/SCP constants around was necessary for this.
Fix style violations in commit 4a90bd4, small changes
- Commit 4a90bd4 contained a rather large number of style violations: - Indentation/sequence formatting - Wrapping of long lines - Docstrings whose first line was wrapped - A stray backslash in a docstring...
sphinx_ext: New directive for supported methods
Until now many resources, but not all and not consistently list theirsupported methods (e.g. “Supports the following commands: ``GET`` …”).Not only is it easy for this list to get out of date, but it would also...
sphinx_ext: Factorize getting RAPI handlers
The list of resources and handlers will also be used for generatinga per-resource table.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
sphinx_ext: Factorize handler methods/access
The factorized parts will be used to show a small table with methods andrequired permissions for each resource.
Use "none" instead of "everyone" in RAPI documentation
The list of required access permissions uses the word “everyone” to mean“no special permissions are needed”. The word “none” fits better.
.gitignore: Add test/hs/harep
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Dato Simó <dato@google.com>
Rewrap a comment in constants.py
As it was 80 chars, and you know that bothers me :)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michele Tartara <mtartara@google.com>
Enable split queries for export in CLI
Rapi doesn't seem to export backup lists, so we enable the split queryonly in gnt-backup.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Improve the rpc-test program
This is an ugly patch, sorry. It adds the following features torpc-test, to help with (stress) testing the Haskell RPC client:
- customisable repeat count for the RPCs- customisable parallelisation factor- options to show timing stats and other information...
Implement Export queries in Haskell
This is a simple query as it has only two fields, however it's thefirst query that doesn't have a clear 'base' object and 1:1correspondence between such objects and the results (groups, nodes andnetworks do so).
We keep nodes as the 'base' object, since that's what we want to...
Fix exports query with failed RPCs
When the RPC to a node has failed (for whatever reason), we currentlyreturn _FS_UNAVAIL, which means "this node doesn't supportexports". However, that is misleading: all nodes support exports, butwe failed to list them at the moment, so _FS_NODATA (temporary...
Add export_list RPC call definitions
This is straightforward, as the call has no parameters and a very simple return type.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Improve TemplateHaskell code to support empty objects
Currently, an empty objects will generate warnings as the arguments ofvarious functions are unused. By adding conditional code for this, wecan support generation of empty objects, e.g. like needed in Rpc code....
Fix Curl/Internal.hs not being in gitignore
This is an autogenerated file, so it has to go into gitignore, thanksMichael!
Add option to redirect luxi socket for debugging
Currently, the choice between masterd and conf/queryd is madestatically, based on whether a query is converted or not and whethersplit queries are enabled. This makes it hard to test and developthese new queries, so this patch adds an override for the socket....
Improve RAPI documentation on users and options
- Document fields in “rapi/users” file in the order they appear (username, password, options)- Mention new “--require-authentication” option- Clarify use of “write” on “GET” method
sphinx_ext: Drop text map for RAPI permissions
Other parts of the documentation directly refer to the constants through“pyeval”. There's no need for this map as it's just additional work tomaintain.
RAPI: Add flag to require authentication
Most RAPI resources do not require authentication for the “GET” method.In some setups it can be desirable to always require authentication.This patch adds a command line parameter to always require it.
Some unrelated minor typos in the “ganeti-rapi” man page are also fixed....
Describe format of the monitoring agent HTTP queries
The previous patch was incomplete when erroneously pushed.
This patch, originally intended as part of the previous one, fixes it.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Update the monitoring agent design document adding the format of the HTTPqueries that will have to be used to query the monitoring agent.
Signed-off-by: Michele Tartara <mtartara@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Remove test of obsolete getNetworkUuid method
This will remove the tests for the getNetworkUuid method.The method will be obsolete after the changes regardingidentification of networks by UUID and not name are mergedinto master. Will send another patch for removing the...
Switch the RPC module over to the multi interface
This replaces the very-basic parMap of IO actions (fully serialised,as parMap won't work here), to the multi interface.
This makes a simple "time gnt-node list" on a 6-node cluster go from3.2s to ~0.9s, and allows even better parallelisation - before,...
Add a Ganeti-specific implementation of Curl Multi
As we want to be able to run queries against multiple nodes inparallel, and furthermore in parallel with other work, we need toimplement the Curl Multi interface (see libcurl-multi(3)).
This patch adds a Ganeti-specific such implementation, to be used...
Add two utility functions for handling Either lists
These two functions permit operating in bulk on only the Left or Rightvalues in the original list, then reassembling the list back in theoriginal order.
Simplify RPC error cases
This patch removes the node from the RPC error constructursCurlLayerError and OfflineNodeError. The rationale is that we anywayreturn tuples (node, result), and removing this duplication allowssimplified signatures/calls in the execution of RPC calls....
Reduce duplication of curl options computation
Some curl option are request-specific, but not node specific: e.g. rpctimeout, etc. The patch changes the HttpClientRequest type so that wecan pre-seed such options, instead of rebuilding the list in eachindividual request execution....
Switch the curl bindings from optional to required
Currently, we support curl being optional via some sporting exercises:ifdefs in the code, data types that represent 'Curl is disabled'state, etc. However, with the future work on RPC, we would have toeven make the dependencies list conditional on it, etc. This is too...
qa_config.AcquireNode: Use sort key function
Use a sort key function instead of using a comparing function (“cmp=…”).The latter is not supported in Python 3 and using a sort key function iseasier.
qa_config: Add repr for instance/node classes
Having a readable representation makes debugging easier.
Test "GetNodeInfo" for Xen hypervisor
This patch adds unit tests for success and failure of the “GetNodeInfo”method for the Xen hypervisor abstractions.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>
QA: Add validation for cluster name
QA: Acquire instance via method
Also make “used” a property.
Merge branch 'devel-2.7'
NEWS: Fix the fix in commit 82b6f9a
February 2, 2013 was a Wednesday, not a Thursday.
NEWS: Fix release date for 2.7.0 beta1
Commit b6ad806f used a wrong date.
Build table with access permissions for RAPI resources
Sometimes it can be difficult to determine the access permissions neededfor a certain RAPI resource without looking at code. This table, addedat the end of “rapi.rst”, shows all resources and the permissions needed...
Allow generating different RAPI resource lookup tables
Until now the RAPI resource table returned by“rapi.connector.GetHandlers” always uses fixed strings or compiledregular expressions as dictionary keys. Now that a table of all RAPIresources (and their access rights) should be written for the...
HRoller design updates
- Specify that there will be options for selecting nodes by at least nodegroups and tags, rather than just individually.- Specify a better handling for non-redundant instances (eg. plain or file) which today are simply ignored...
Minor fixes regarding nic.network change
Make LookupNetwork() return None in case target is None. This fixesIssue 380. Rapi passes network=None and the lookup should not fail.
Make network client aware of new nic.network.gnt-network info showsthe IPs of each instance inside the network. It parses nic.networks...
baserlib: Make "_OPCODE_ATTRS" public
"_OPCODE_ATTRS" will also be used in the Sphinx extension.
Makefile: Add dependencies
The RAPI documentation now depends on “connector.py” (handlerregistration) and “rlib2.py” (access permissions).
RAPI documentation: Add link targets
A newly added table of RAPI access permissions will link to allresources and their supported methods. For this reason link targets needto be added.
The GET method on “/2/nodes/[node_name]” was not separately mentioned....
Fix issue 378
In case a NIC is not inside a network then netinfo None. Thusnetinfo["name"] fails.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>Reviewed-by: Helga Velroyen <helgav@google.com>
Helper to retrieve access permissions for RAPI resource
Merge similar code into a helper function.
Sphinx extension: Module-level constant for tab width
Use a module-level constant for the tab width.
RAPI documentation: Assertion for console fields
Assert that the documented fields are equal to those in the actualobject.
Status change reason support for Reboot
Add support to the Reboot command for specifying the reason for the laststatus change.
Some features are implemented as functions, even if used only once, becausethey will be used by the future patches introducing reason support for all...
Infrastructure for specifying instance status change reason
This patch introduces some infrastructural modifications that will be used bythe following commits to implement the support for specifying the reason forthe last status change of an instance....
NEWS: Mention multi-alloc and fix typo
As far as I could see, the ability to allocate multiple instances viaRAPI was not mentioned at all. A superfluous word is also removed.
Unit tests for Query/Network.hs
This patch adds a couple of unit tests for Query/Network.hs.Note that they'll need to be adapted, once issue 362 is addressed.
Signed-off-by: Helga Velroyen <helgav@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Make Confd client usable for testing
Allow the Confd client to be able to connect to an arbitrary serverinstead of just the real one running on a cluster.
This is meant to be used for testing.
If a data collector using the Confd client is tested with shelltest, it will...
Add Haskell parser for "xm uptime"
In order to fetch precise information about the uptime of the VMsrunning in Xen, we need to analyze the output of the "xm uptime" command.
This commit adds the parser to do that, and its tests.
Signed-off-by: Michele Tartara <mtartara@google.com>...
Add Haskell parser for "xm list --long"
In order to fetch precise information about the status of the VMs running inXen, we need to analyze the output of the "xm list --long" command.
Add request type to Confd server for getting instance list
Add to Confd server a new request type (and its implementation) to ask forthe list of instances in a node.
Update monitoring agent report format
This commit expands the monitoring agent design document improving thedescription of the general report file format.
There can be two kinds of data collectors (performance and status collectors).
Every collector can now be part of a category of collectors (such as...
Add cfgupgrade for changing nic.network to uuid
This gets all network info from config_data and parsesall instances and their NICs and makes the substitution.
Remove useless code in backend for network hooks
In backend NICs arrive with netinfo filled. If nic.network is not Nonenic.netinfo is not too. Thus all the info is derived from HooksDict()and nic.network must not be checked.
Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>...
Show network name and not uuid in instance info
This was before the case too. Now is bit trickier because nic.networkis uuid. Info must derive from nic.netinfo.
Add IQ_NETWORKS in query tests
Implement network locking in Instance queries
This is needed in case more info than each nic's network uuid is tobe returned. We need to lock networks to get valid data. For nowonly the name is returned as an extra field. All other can be addedwith trivial effort....
Changes in query to support nic.network as uuid
Queries now return the network uuid as well as it's name. Here weonly use info provided be LUInstanceQueryData context.
Modify query LUs to supoprt nic.network as uuid
Make _InstanceQuery gather all network info related to instance'sNICs and in case of NETQ_INST in _NetworkQuery get all networkuuids directly from nic.network
Add GetInstanceNetworks() config method
This will be needed for Instance Queries. It walks through theinstance's NICs and returns a list network uuids that the NICsare attached to.
cmdlib changes to support nic.network as uuid
Refactor Instance related LUs to support nic.network asa uuid. This removes all the unnecessary invocations toLookupNetwork().
Make network config methods take uuid as argument
This will be needed in the following patches where nic.networkwill refer to network's uuid and not name.
Document possible hypervisor bug in serial_console
This is described in Issue 83.
Update both the install guide and the gnt-instance manpage.In the install guide move another KVM specific example inside a new KVMadmonition for the hypervisor parameters section....
Update man pages wrt ExtStorage
Also link to its design doc in the design-2.7 doc
Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>...
Design correct reporting of storage free space
This is a long standing bug in Ganeti. Add a small design on how we planto fix this for Ganeti 2.7.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Helga Velroyen <helgav@google.com>