Synnefo » snf-ganeti » ganeti-local

Move htest/ files under the test/ tree

htest/data becomes test/data/htools (basically reverting commit
8feabc89), and htest/* becomes test/hs/*.

Most changes beside the rename are trivial s/…/…, with the exception
of autotools/run-in-tempdir, which needed some more changes now that...

Return master ip script output in case of failure

Currently only the exit code is returned, which is somewhat pointless:
for failures, we already know it's ≠ 0, so we don't have enough
information (reported on the mailing list). By logging the output as
well (if any), the user will have more information available for...

Fix LUInstanceRecreateDisks with iallocator

Recent iallocator changes (commit fb60bc6a, “iallocator: Add node
whitelist”) broke the parameter validation for LUInstanceRecreateDisks
(the other LUs were manually fixed already):

$ gnt-instance recreate-disks -I hail --dry-run instance3...

Rename htools/ to src/

Per offline discussions, this is the first patch of the
renames. Tested with "make distcheck", seems to work fine.

The only change outside of the renaming is a bit of simplification in
the .gitignore rules; otherwise, simply s/htools/src/....

Fix building of Constants.hs w.r.t .pyc files

Unless this is run under run-in-tempdir, it will create as usual the
.pyc files.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Add ExtStorage related man pages

• ganeti-extstorage-interface man page
• gnt-storage man page

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Extend the hbal shell tests

After this patch, almost all of hbal except the actual Luxi execution
is being covered by unit- or shelltests.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix build breakage after exclusive storage commits

This fixes:

htest/Test/Ganeti/Rpc.hs:50:14:
Couldn't match expected type `Rpc.RpcCallNodeInfo'
against inferred type `Data.Map.Map String Bool
-> Rpc.RpcCallNodeInfo'...

Refactor checks for a new template in LUInstanceSetParams

All the checks have been moved to a private method, so as to make
CheckPrereq() smaller and more readable (pylint was complaining about it).

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...

Free disk space reporting follows exclusive storage rules

When exclusive_storage is set, nodes report only the space that could be
used to create new disk according to the rules set in the
design-partitioned doc (i.e., non-empty disks are ignored and the space...

Add exclusive_storage parameter to node_info RPC

The flag is passed to the low-level functions that need it, but it's not
yet used. Also, RPCs get just a default value for now.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

QA: Do not pass "--bridge" to "gnt-cluster init"

Commit ec0652a (June 2009) removed the option.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix build after addition of 'empty-cluster.data'

Of course I forgot to add this to TEST_FILES… and only distcheck
catches it.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

gnt-network(8): Re-add editor settings and re-wrap

Commit a04cc7a added “gnt-network.rst”, but didn't copy the editor
settings from another reST file.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Call blockdev_create RPCs with the exclusive_storage flag

The flag is read from the configuration and passed to the RPC.

Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Added class to contain information about a PV

This makes the code more readable and easier to upgrade.
bdev.LogicalVolume.GetPVInfo and the code that depends on it have been
refactored to use the new class.

utils.CheckVolumeGroupSize() has been moved to lib/utils/lvm.py, where more...

Add htest/hroller to .gitignore

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Silence erroneous hlint warning

This patch adds an ignore for an hlint warning that wants camelcase to be used
in the name of a "case" test.
But if camelcase is used, it is not a valid test name anymore.

Signed-off-by: Michele Tartara <mtartara@google.com>...

gnt-network: Fix CLI issues with node groups

The original implementation reserved the name “all” to specify all node
groups. That would prevent the use of a node group actually named “all”.

This patch changes the order and definition of the command line...

gnt-network CLI improvements

- Use unpacking instead of addressing indivdual list items
- Add “--submit” and “--priority” support where possible
- Re-use LUXI client where possible
- Fixed some typos

There is still an issue that a node group named “all” can not be...

Change network error text

This error is thrown if a network name already exists, not if the
network itself (e.g. “1.2.3.0/24”) is defined.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Fix iallocator for nodegroups with no tags

_ComputeNodeGroupData() raises error if a group has no tags
when list(gdata.tags) is invoked. Use “gdata.GetTags()” instead.

Signed-off-by: Dimitris Aragiorgis <dimara@grnet.gr>
Signed-off-by: Michael Hanselmann <hansmi@google.com>...

Add Confd client to the Haskell code base

The client queries all the master candidates in parallel, until the minimum
number of replies, defined in the constant file, is received.
A timeout prevents the waiting from being of indefinite length.

The reply to be returned to the function that made the query is decided...

Add the parseReply function to the Confd utils

This function will be used by the Haskell Confd client to parse the replies
it receives from the server.

Unit test is included as well.

Signed-off-by: Michele Tartara <mtartara@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Make the parseRequest function more general

The parseRequest function of the Confd utils can be used to parse both request
(in the server) and reply (in the client, soon to be implemented) signed
messages.
This patch changes the signature of the function to allow this, and its name...

Move Confd getCurrentTime to Utils

The Confd server implementation has a getCurrentTime function.
This commit moves it to the Utils.hs file of Confd, so that it will be
available to be used by the future Confd client as well.

Signed-off-by: Michele Tartara <mtartara@google.com>...

ExtStorage support in htools

This just adds new constructors for the disk template and related
types, enabling the Ext storage.

Signed-off-by: Constantinos Venetsanopoulos <cven@grnet.gr>
Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

confd: strip newlines before logging the request

The Python confd client sends newlines in messages (due to use of
DumpSignedJson/DumpJson), so that results in ugly debug output.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

gnt-cluster(8): Quoting, small fixes

Quote fixed strings where appropriate and replace “nic” (an acronym)
with “network interface”.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Guido Trotter <ultrotter@google.com>

Multiple ExtStorage Providers and ext-params

Add support for passing parameters to the ext template (ext-params).
Take advantage of disk-params, that don't seem to make much sense in
this template (ExtStorage Providers are not predefined and we don't
know their needs) and use them to pass the ext-params dynamically to...

Implement the External Storage Interface

With this commit we introduce the External Storage Interface
to Ganeti, abbreviated: ExtStorage Interface.

The ExtStorage Interface provides Ganeti with the ability to interact
with externally connected shared storage pools, visible by all...

Make node groups' networks field a dict of nicparams

The networks field of the node groups class is a dictionary
of nic params and not a list of networks. This should fix
the broken QA.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Final update of NEWS file for 2.6.2 and version bump

I'm already setting this to a release date of tomorrow, since QA on
the 2.6 branch has been clean.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Michael Hanselmann <hansmi@google.com>

Display command description in htools usage message

By abstracting the command/personality formatting, we can easily
switch htools to the same output as the rest of the commands.

Signed-off-by: Iustin Pop <iustin@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

NEWS: Opportunistic locking, clarifications

Some rewording, some clarifications, and a new item on opportunistic
locking.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Helga Velroyen <helgav@google.com>

Detect wrong Python module structure

If, for one reason or another, a module named "ganeti.ganeti" can be
succesfully imported (e.g. through a symlink in the wrong place), error
which are diffcult to debug can occur. This piece of code tests for the
unlikely case of that happening....

Use hroller to print nodes to reboot by groups

This is highly incomplete, as per the manpage BUGS section.

Signed-off-by: Guido Trotter <ultrotter@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

One more offline+shutdown fix

Currently gnt-instance shutdown --force of an offline instance
resets the offline status. Which of course is bad and completely
unintended. Fix. (caught by the new qa tests)

Signed-off-by: Guido Trotter <ultrotter@google.com>...

_NetworkQuery: Simplify statistics computation

The code can loop directly over “self.wanted”, saving a lookup. An
explicit loop is also replaced with a generator.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

gnt-network: Fix import for textwrap

The style guide says to use “import foo”-style imports, not “from foo
import bar” unless it's a Ganeti module. There are some places with
exceptions, but this one certainly isn't warranted. Also fix the import
order.
...

cmdlib: Replace period with colon in error message

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

gnt-network(8): s/ipv6/IPv6/

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

OpNetworkConnect: Check for network mode

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Improve descriptions of network query fields

They should be in the same style as other descriptions.

Signed-off-by: Michael Hanselmann <hansmi@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Renaming fam -> net_family on test network generation

Small renaming to make the test more readable.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Michele Tartara <mtartara@google.com>

Add a network fields to NIC params and node group type

This adds a network field to the NIC params and a list of networks
to the node group type. It adjusts the nodegroup's unit tests accordingly.

Signed-off-by: Helga Velroyen <helgav@google.com>
Reviewed-by: Iustin Pop <iustin@google.com>

Fix job completion with big job queues

Accidentally stumbled upon this while testing unrelated code on a
machine with ~3K active jobs - the bash completion unittest was
hanging.

Upon investigation, it turns out that bash's ${var//pattern/repl/} is
probably quadratic in the size of input (or worse, even):...

Remove some unused Python code

This patch removes code which is no longer used due to refactoring:

- http.InitSsl, last usage removed in commit 33231500 (“Convert RPC
client to PycURL”)
- rapi.baserlib.MakeParamsDict, last usage remove in commit 4e5a68f8...

Add support for job queries in hconfd

This adds support for job queries, including (basic) unit-tests.

I've tested this for memory and cpu usage as follows:

- 3600 jobs (live queue):
- via masterd, default: ~1.1s (masterd: ~60MB ram)
- via confd, default: ~1.1s (hconfd: ~25MB ram)...

Add a read-only job queue module

This patch adds implementation for a read-only job queue module,
together with "full" test (as full as can be in a lazy language…).

One note about the behaviour of the job queue is the handling of
opcodes that fail validation: the 'input' opcode actually is a...