Add new module for virtual clusters
This module will take care of managing paths for virtual clusters.Unittests are included (100% coverage).
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
bootstrap.SetupNodeDaemon: Stop hardcoding bind address
Commit b43dcc5a11 added support for IPv6. To have the node daemon bindto an IPv6 address on startup it changed the code to pass a static bindaddress (0.0.0.0 or ::0). If a user had configured another bind address...
hv_kvm: Use pu.CONF_DIR variable
… instead of using SYSCONFDIR directly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
utils.filelock: Remove executable bit from lock files
There's no need for lock files to be executable.
LUInstanceCreate: Include group name in error message
“pnode.group” contains the UUID, which isn't very user-friendly.
ShellWriter: Add parameter to disable indentation
This will be used to write a more compact bash completion script.
constants: Stop using wildcard import for pathutils
It still needs to be imported for hypervisor parameter defaults, butafter this change paths are no longer exported from constants.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Migrate lib/uidpool.py from constants to pathutils
File system paths moved from constants to pathutils.
Migrate lib/utils/*.py from constants to pathutils
Migrate lib/watcher/__init__.py from constants to pathutils
Migrate lib/tools/ensure_dirs.py to pathutils
Migrate lib/ssconf.py from constants to pathutils
Migrate lib/ssh.py from constants to pathutils
Migrate lib/mcpu.py from constants to pathutils
Migrate lib/ovf.py from constants to pathutils
Migrate lib/rapi/baserlib.py from constants to pathutils
Migrate lib/rpc.py from constants to pathutils
Migrate lib/server/*.py from constants to pathutils
Migrate lib/hypervisor/*.py from constants to pathutils
Migrate lib/{jqueue,jstore}.py from constants to pathutils
Migrate lib/luxi.py from constants to pathutils
Migrate lib/masterd/instance.py from constants to pathutils
Migrate lib/client/*.py from constants to pathutils
Migrate lib/cmdlib.py from constants to pathutils
Migrate lib/confd/*.py from constants to pathutils
Migrate lib/config.py from constants to pathutils
Migrate lib/backend.py from constants to pathutils
Migrate lib/bootstrap.py from constants to pathutils
Migrate lib/cli.py from constants to pathutils
constants: Move most paths to separate module
This is inpreparation for the implementation of virtual clusters. Manypaths will change based on an environment variable and are no longerconstant and should no longer be in “constants.py”. Since “constants.py”...
constants: Add missing word in comment
It was missing in commit 243cdbcc on May 7, 2008.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Always_failover doesn't require --allow-failover anymore
If an administrator sets always_failover, it means that there is no needfor another explicit approval to failover instead of migrating.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Remove useless lock in recreate-disks
The old primary node was being locked to do RPC, but we don't lock nodesfor RPC anymore, so the lock is useless.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Small style adjustment
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Make iallocator request types more strict
This adapts the type of some request input fields to more stricter ones.
Rework the required_nodes field
This was used to verify the result of instance allocation. However, thisis not logic which belongs to the cmdlib rather than to the iallocatorrequest itself. So abstract it by its own function. And update theValidateResult accordingly....
objects, rpc: Code cleanup
objects.Disk.ComputeLDParams: Simplify structure, don't use andoverwrite local variables.
rpc.AnnotateDiskParams: Replace explicit loop.
RAPI client: Fix typo in docstring
Also update the copyright header.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Disable pylint warnings for unknown members
For some reason pylint can't determine, that the members are calculatedupon runtime and complains. This is weird as the same mechanism works onopcode/objects. I haven't found the issue for it yet, so let's just...
Adapt gnt-debug iallocator
Add the new multi-alloc request to the test allocation framework. Fornow we just create --count identical instances. This might need furtherimprovements later.
Adding new multi-allocation request
Now that we've the refactoring done, we can easily add the new mode andeverything works.
Switch all iallocator calls to the new one
After the refactor, the old calls are not working anymore, so we switchto the new ones.
Refactor IAllocator code
The IAllocator class was handling all the requests on its own, passingin parameters on top level which works, but is hard to maintain and notflexible.
With the upcoming change to the IAllocator for MultiAllocate we can'tuse the toplevel parameters anymore. Therefore, we refactor the code...
Make the slots functionality more modular
As we will introduce another set of containers using the slots trickwe abstract away as much as possible to separate bases classes. Thechild classes then adapt them for their needs. This leads to less code...
Move _MakeLegacyNodeInfo to rpc
This function will also be needed for the iallocator refactor. Also thisfunction does not belong to cmdlib but rather to rpc. Lets move itthere.
Move _CalculateGroupIPolicy and _ComputeDiskSize
This makes it possible to better untangle the IAllocator code, whichwould otherwise lead to a recursive import structure as we need thosehelpers there as well.
Signed-off-by: René Nussbaumer <rn@google.com>...
Merge branch 'devel-2.6' into submit
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>
Merge branch 'stable-2.6' into devel-2.6
Fix gnt-debug iallocator
There was an issue with the recent ipolicy introduction which lead to abug in gnt-debug iallocator. It was not providing the spindle_use fieldand therefore it wont let you create a valid iallocator request.
Merge branch 'devel-2.6'
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
Adding new IAllocator mode to constants
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix warnings/errors with newer pylint
To help developing Ganeti on newer distributions, let's try to fixpylint warnings/errors. I'm using pylint from current Debian wheezy:pylint 0.25.1, astng 0.23.1, common 0.58.0, and we have 3 things thatneeds fixing....
Fix decorator uses which crash newer pylint
Pylint version:
pylint 0.25.1, astng 0.23.1, common 0.58.0
crashes when passing the fully-qualified decorator name with:
File "/usr/lib/pymodules/python2.7/pylint/checkers/base.py", line 161, in visit_function...
Move parameter title definitions to constants
This fixes an old TODO for moving these; now they can be reused in theHaskell Query2 implementation.
Fix typo in node role description
I'd rather not introduce this typo in the Haskell code too, and alsonot having them identical is not good.
Complete and enable the haskell QueryClusterInfo call
Since we have now access to all cluster parameters, we can "fill" theparameters, and can finally enable it in CLI/RAPI.
Enable tags query over the query socket
This patch adds the tags field to the objects which were missing it(all except Cluster), implements handling the LuxiCall QueryTags, andthen enables the use of the query socket in cli.ListTags, used by allcommands, and in the RAPI client....
Switch RAPI to use same tags listing as CLI
Currently, RAPI uses queries (as in QueryNode with field tags) forgetting the tags, whereas the CLI uses the QueryTags Luxicommand. Since this discrepancy is not good, let's switch RAPI to usethe same method as the CLI....
Implement support for query only clients in Rapi
This implements the same query=True|False functionality as inGetClient for cli.py, however since the RAPI code is much moreunit-tested (and the unit-test clients are mocked, for the most part,without support for addresses) we have to do many adaptations in the...
Add configure-time switch for split queries
And expand cli.GetClient() to allow opening the query socket, insteadof the main master socket.
Finally, enable the query socket use in gnt-cluster version, sincethat is already implemented fully in Queryd.hs/hconfd....
Add new definition for query socket
This is not used yet, but is helpful when developing the Luxi endpointfor hconfd.
Move necessary constants from rpc & rpc_defs to constants
As timeout constants will now be used also on Haskell side, they havebeen moved from rpc_defs and rpc to constants. The same is true forconnection timeout for connecting to nodes.
Signed-off-by: Agata Murawska <agatamurawska@google.com>...
Bump pep8 version to 1.2
Debian Wheezy will ship with this version, and it has many improved checks compared to 0.6, so let's:
- bump version in the docs- silence some new checks that are wrong due to our indent=2 instead of 4- fix lots of errors in the code where the indentation was wrong by 1...
Change node parameter oob_program to VTYPE_STRING
Since this is an inheritable parameter, having it as a MABYE_STRINGcreates only problems (per our derivation rules). We change it toSTRING, with the default "", meaning no program. Note that most of the...
Fix computation of disk sizes in _ComputeDiskSize
Currently, hail fails with FailDisk when trying to add an instanceof type: 'file', 'sharedfile' and 'rbd'.
This is due to a "0" or None value in the corresponding dict inside_ComputeDiskSize, which results in a "O" or non Int value of the...
Add verification of RPC results in _WipeDisks
Due to an oversight, the pause/resume sync RPC calls in _WipeDiskslack the verification of the overall RPC status, and directly iterateover the payload. The code actually doing the wipe does verifycorrectly the results. This can result in jobs failing with a hard to...
Fix double use of PRIORITY_OPT in gnt-node migrate
This breaks the command, as optparse considers that an error.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
rapi client: accept arbitrary shutdown arguments
The "ignore_offline_nodes" parameter is unsupported. Rather thanexplicitely adding it, just pass all keyword arguments in the body ofthe query, and rapi on the other side will do the right thing.
Supports for old arguments that were passed via the query is unchanged....
Handle offline nodes for "instance down" checks
When offlining an instance because its primary node is down, we must beable to cope with the situation.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Switch job IDs to numeric
This has been a long-standing cleanup item, which we've alwaysrefrained from doing due to the high estimated effort needed.
In reality, it turned out that after some infrastructure improvements(the previous patches), the actual job queue-related changes are quite...
Add infrastructure for building numeric namefield filters
Currently, all the CLI helpers in qlang.py and cli.py assume that allnamefields are string, which leads to various breakage in case the'reactually not.
To improve the flexibility of the helpers, we add a bit of...
Expand debug messages in gnt-debug test-jobqueue
It wasn't clear for which tests we were ignoring the errors.
Fix uses of OpPrereqError without code info
A while back, we did cleanup the code and ensured (manually) that useof OpPrereqError includes an errors.ECODE_* field as secondargument. Since we cannot automate the check for this, it turns outthat more and more such usage has crept over the years, including in...
Fix 'explicitely' common typo
It seems that 'explicitely' is wrong, and that the right form is'explicitly'. This is just fixing the typo plus adjusting affectedparagraphs.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix issue in LUClusterVerifyGroup with multi-group clusters
In case LUClusterVerifyGroup is run on a group which doesn't containthe master node, the following could happen:
- master node is selected due to the explicit check- if the order of nodes in the 'absent_nodes' list is such that the...
Fix node group modification of node parameters
Commit 904b3bfe tried to fix the deletion of custom ndparams fromgroup, but instead broke both modification and deletion: because werun ForceDictType on self.op.ndparams instead of the updatednew_ndparams, we can neither delete nor set properly spindle_count...
Front-end and doc to use allocator in recreate-disks
The recreate-disks command no longer requires an explicit list of nodes.
Signed-off-by: Adam Ingrassia <api@google.com>Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Allocator support in recreate-disks LU
This affects only the backend.
Added method to call an allocator for recreate-disks
Ideally this should have used relocation mode, but to due lacking ofsupport in current allocators, allocation mode has been used instead.
Signed-off-by: Adam Ingrassia <api@google.com>Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Support for checking the group of only the primary node
_CheckInstanceNodeGroups behaves in the old way with the default valueof the new parameter.
Signed-off-by: Adam Ingrassia <api@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge commit 'v2.5.2' into stable-2.6
Conflicts: NEWS (trivial, merged the entries)...
Fix boot=on flag for CDROMs
This generalises commit 4304964a to cdroms too, since they havesomewhat the same logic. We just abstract the needs_boot_flag into aseparate variable, and then reuse it in the cdrom section.
Note that the logic of what 'if=' type to pass to KVM was very...
KVM: only pass boot flag once
This addresses issue 230: passing two methods of booting to KVM can,depending on the KVM version, confuse it.
Note that commit 4304964a introduced a partial fix for this (but onlyfor disks, and keyed on KVM versions). However, it didn't fix cdrom...
Make error classes more consistent
This removes "pass" from a few definitions, making all errorsconsistent.
Add support for -cpu option in KVM
This adds supports for the (non-validated) cpu parameter to KVM.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'stable-2.6'
Fix setting ipolicy on node groups
On node groups we don't have the std field. However, the InstancePolicyobject always verifies that the std value is within a given range. As wefill it up with defaults if not set (as it happens to be on node groups)and the min value is higher than the default std value (taken from...
Fix --no-headers for the new list-drbd command
Sorry, I forgot that with GenerateTable this needs to be handledmanually. Fixed now and tested in both ways.
(But to be honest, this should be abstracted in GenerateTable, insteadof the 'if' test in all its callers.)...
Add a new gnt-node command list-drbd
This uses confd to query the DRBD minors, which is very special; noother command currently does so.
Since the backend is only implemented in the Haskell version of confd,we have checks that 1) confd is enable, and 2) hs confd is also...
Add a new unused confd query
This is not implemented currently.
Fix a docstring in bdev's DRBD8 class
It seems this was not updated since the move to static minors…
Ensure that disk.params is always defined (and a dict)
Commit cce46164 fixed upgrading from other 2.6 versions, butaccidentally broke upgrading from 2.5 (disk.params was left as None,which breaks FillDict). The simplest way to handle params is to always...
Another small consistency fix with if branches
While looking at the testability of this piece of code, I found anotherconsistency problem. We have two if branches instead of one, withelif's.
Fix inconsistency in the LUXI protocol w.r.t. args
This inconsistency was found during rebalancing. Hbal failed because,Ganeti couldn't load the opcode. After digging through the cause, aninconsistency with the "args" field in the LUXI protocol was triggered...
Fix UpgradeConfig of Disk object regards disk params
This bug was found during disk parameter debugging. While looking at theconfig some values seem present on the disk parameters, but that's notexpected behaviour. This patch fixes this, and also fixes the "broken"...
Allow reinstall even when secondaries are offline
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Agata Murawska <agatamurawska@google.com>