cmdlib: Factorize checking acquired node group locks
The “cur_group_uuid” parameter is optional to prepare for using thefactorized code from LUInstanceQueryData.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
gnt-instance info: Show node group information
This requires acquiring the node group locks in shared mode.
cmdlib: Stop forking in LUClusterQuery
While debugging another issue we realized that LUClusterQuery forks.This turned out to be the “platform.architecture” function from thePython library. It uses the “file” command to determine the architectureof the Python binary....
locking: Notify only once on release
Don't notify for every released lock in shared mode. The last one isenough.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
locking: Handle spurious notifications on lock acquire
This was already a TODO since the implementation of lock priorities inSeptember 2010. Under certain conditions a waiting acquire can benotified at a time when it can't actually get the lock. In this case it...
locking: Fix lock deletion with timeout
While working on another SharedLock fix I realized timeouts on lockdeletion don't work very well if the timeout actually expires. Thispatch fixes the issue and adds a new unittest.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Move _TimeoutExpired to utils
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Bernardo Dal Seno <bdalseno@google.com>
(cherry picked from commit f8326fcaac87958241d78526e5868d23d78ac286)
EPO: Pass the no_remember parameter to preserve state
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'stable-2.5' into devel-2.5
Fix Makefile.am compatibility with automake 1.11.2
Automake 1.11.2 made the following change:
Unfortunately, this breaks our Makefile.am (issue 216) exactly because...
Reconcile Makefile.am and test data files
Sorry, forgot this in previous commit.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Workaround changed LVM behaviour
The vgreduce command has changed behaviour from when we initiallywrote the code (2.02.02 versus 2.02.66, 4 years delta):
- if there are LVs which will be impacted, it requires --force- otherwise refuses to proceed, but it still returns exit code 0...
Accept both PUT and POST in noded
This is a partial cherry-pick from7530364ddbe949bc34fc26f25ba3f5d921beb021 on master:
Currently, noded requires PUT, even though the semantics of the RPCcalls do not match a PUT. We change the code accept both PUT and POST,...
Fix type check for OpQuery.filter
Just using ht.TListOf as a type check doesn't work correctly. Thefunction must be called with the expected item type. In this specificcase TListOf was always called with the filter as a value, and theresult of that call evaluated to truth. Since filters can be quite...
Fix explanation of gnt-node evacuate --primaries-only
Furthermore, correct the --help display on evacuate.
Makefile.am: fix permissions for Python scripts on install
Some Python scripts in /usr/lib/ganeti/ were getting the wrong permissions(their 'x' bit was cleared). This patch fixes that behavior.
This patch renames the variable 'dist_tools_PYTHON' to 'python_scripts'....
devel/upload: Fix permissions for installed directories
Permissions for the directories created during install depended on theumask of the user running the script. Now umask is reset inside the scriptto remove such dependency.
Signed-off-by: Bernardo Dal Seno <bdalseno@google.com>...
Fix cluster verification issues on multi-group clusters
This patch attempts to fix a number of issues with “gnt-cluster verify”in presence of multiple node groups and DRBD8 instances split over nodesin more than one group.
- Look up instances in a group only by their primary node (otherwise...
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Migrate: don't check for free memory on cleanup
Cleanup just updates the config with the correct location of theinstance, or informs of its down status, but never starts it. As suchthere's no point in checking for enough free memory. Actually this check...
Revert "cli: Disable abbreviation matching for options"
This reverts commit 232aab3f4f602a19f1226e85c3a3ecb245d60af4. Weshouldn't change the parsing of command line options in the middle ofthe 2.5.x series.
Fix wrong option names in QA and cluster-merge
Fix a bug in command line option parsing code
Fix bug affecting command line options of "keyval" type. Althoughescaping commands with \ is supported, it is is not applied to theinput recursively.
Signed-off-by: Nikos Skalkotos <skalkoto@grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>...
Add UnescapeAndSplit unittest for multi-escapes
This would have caught the bug in the first place. Argh,hand-generated test cases!
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Bump version to 2.5.0~rc5, update NEWS
cli: Disable abbreviation matching for options
Python's “optparse” module does option name prefix matching by default.Since this can lead to confusing behaviour, e.g. by specifying “--force”for a command which only has a “--force-multi” option, this patch...
Merge branch 'devel-2.4' into stable-2.5
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
KVM: support version reported by 1.0
This of course was working for all the rcs, but broke with 1.0 itself.
In addition: - split between running kvm --version and parsing its output - unittest parsing for various known --help outputs - updated NEWS file...
doc/admin: Clarify archived jobs
Also mention that archived jobs can be viewed using “gnt-job info”.
jqueue: Factorize checking job processor's result
This allows for more unittesting.
jqueue unittest: Rename simple fake-job class
Also add a parameter for priority, to be used in an upcomingpatch.
jqueue: Fix epylint errors introduced in 37d76f1e4
jqueue: Fix deadlock between job queue and dependency manager
When an opcode is about to be processed its dependencies areevaluated using “_JobDependencyManager.CheckAndRegister”. Dueto its nature that function requires a lock on the manager'sinternal structures. All of this happens while the job queue...
locking: Add “__repr__” to SharedLock and PipeCondition
These help when debugging.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>
daemon.GenericMain: Don't generate backtrace on conflicting daemons
Instead, print a nicer error message. This should fix issue 200.
utils.io.WritePidFile: Improve error reporting
If the PID file is already locked by another process, try to readthe content and report it as part of the error message.
utils.ListVisibleFiles: Hide “lost+found” directories
If a “lost+found” directory is found at a filesystem's root path it isignored. In all other cases directory entries named “lost+found” aretreated normally. Unittests are included. Fixes issue 153.
Fix race condition in test for *FileID functions
In this test the “file ID” of a temporary file is compared against thefile ID gathered via an open file descriptor to the same file. Forreasons unknown to me utime(2) is called in-between to update theinode's a- and mtime. Depending on the file system's timestamp...
Merge branch 'devel-2.4' into devel-2.5
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
ConfigWriter: Fix epydoc error
The parameter is called “mods”, not “modes”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Andrea Spadaccini <spadaccio@google.com>(cherry picked from commit 1730d4a1ab56ef36d082b614d3d0ab13f3e14a85)
LUGroupAssignNodes: Fix node membership corruption
Note: This bug only manifests itself in Ganeti 2.5, but since theproblematic code also exists in 2.4, I decided to fix it there.
If a node was assigned to a new group using “gnt-group assign-nodes” the...
Fix pylint warning on unreachable code
Commit c50452c3186 added an exception when all instances should beevacuated off a node, but did so in a way which made pylint complainabout unreachable code.
LUNodeEvacuate: Disallow migrating all instances at once
There is a design issue in the iallocator interface which prevents usfrom doing this.
Separate OpNodeEvacuate.mode from iallocator
Until now the iallocator constants for node evacuation(IALLOCATOR_NEVAC_*) were also used for the opcode. However, it turnedout this was due to a misunderstanding and is incorrect. This patch addsnew constants (with the same values) and changes the affected places....
LUNodeEvacuate: Locking fixes
When evacuating a node, only an assertion without informative text wasused to check if the necessary node locks had been acquired. This was ontop of evaluating the list of nodes without having a node group lock, sothis was changed as well....
Fix error when removing node
ConfigWriter.GetAllInstancesInfo returns a dictionary, not a list.Removing a node would fail with “too many values to unpack”.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
LUInstanceCreate: Release unused node locks
After iallocator ran we can release any unused node locks. Since theymust be in exclusive mode this should improve parallelization duringinstance creation.
htools: rework message display construction
While diagnosing some (unrelated) memory usage in htools, I'vestumbled upon some very bad behaviour in checkData: mapAccum isnon-strict, and the tuple we use also, so that results in the list oflist of messages being very bad space-wise (hundreds of MB of memory...
hbal: handle empty node groups
This patch changes an internal assert (which can only be triggeredwhen a node group is empty) into properly handling this case (andreturning empty node/instance lists).
While we could handle this in the backend (Cluster.splitNodeGroup)...
Document OpNodeMigrate's result for RAPI
- Commit b7a1c8161 changed the LU to generate jobs- Mention documented results in NEWS
Ensure unused ports return to the free port pool
Ensure ports previously allocated by calling ConfigWriter's AllocatePort() arereturned to the pool of free ports when no longer needed:
Re-wrap a paragraph to eliminate a sphinx warning
This just makes sure that the paragraph doesn't contains lines thatstart with :, which make Sphinx (1.0.7) complain.
Fix newer pylint's E0611 error in compat.py
These are triggered by our "stay-compatible" approach.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Fail if node/group evacuation can't evacuate instances
If an instance can't be evacuated, only a message would be printed. Withthis change the operation always aborts. Newly added unittests check forthis behaviour.
Update init script description
LUInstanceRename: Compare name with name
… instead of object with name.
LUClusterRepairDiskSizes: Acquire instance locks in exclusive mode
Instances are modified if their disk size doesn't match.
Update synopsis for “gnt-cluster repair-disk-sizes”
Mention that instances can be passed on the CLI when “--help” is used.
Move hooks PATH environment variable to constants
Move the contents of the PATH environment variable for hooks toconstants, and use its value in the code and in the hooks documentation.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Check the results of master IP RPCs
A failed gnt-cluster (de)activate-master-ip would not produce any outputto the user. This patch adds code that checks for the results of theRPCs and raise an exception if appropriate.
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>...
Add documentation for the master IP hooks
Reviewed-by: Michael Hanselmann <hansmi@google.com>Signed-off-by: Andrea Spadaccini <spadaccio@google.com>
Add master IP turnup and turndown hooks
Add RunLocalHooks decorator
Add the RunLocalHooks decorator, that allows the execution of hookslocally. Also, add a RunLocalHooks method to HooksRunner, to adapt thesignature of HooksRunner.RunHooks to the one expected by HooksMaster,and also to check that the hooks are being executed locally....
Generalize HooksMaster
- remove any dependence on Logical Units from the HooksMaster;- add a new function parameter to the constructor, a function that is expected to convert the results of the hooks execution in a format understood by the HooksMaster;...
Fix wrong headers and licences
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Stephen Shirley <diamond@google.com>
Update NEWS for 2.5.0~rc4
I forgot this in the previous patch.
Bump version to 2.5.0~rc4
Merge branch 'stable-2.4' into stable-2.5
Conflicts: configure.ac: Trivial
Conflicts: configure.ac (trivial)
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
jqueue: Allow zero jobs to be submitted at once
If cmdlib.LUNodeMigrate was called for a node without primary instancesit would try to submit an empty list of jobs. This was never visible viaCLI as there we check the list of primary instances first.
Update NEWS and increase to 2.4.5
Fix parameters of RpcResult in hooks unit tests
In FakeHooksRpcSuccess, the data parameter of the RpcResult constructorwas not enclosed in a tuple. While this does not make the test fail, itmust be fixed.
hail: don't select the primary as new secondary
This just adds the primary node of the instance as 'non-allocable'during the choosing of the new secondary.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>(cherry picked from commit 7073b3a86856bcd8d8a62c0b72f82deaabb8d8f1)...
hail: add an extra safety check in relocate
If we select the primary as new secondary, better to fail than returnwrong data to Ganeti.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>(cherry picked from commit f25508bef4e85032f0468e5a6f0f8930ff154e66)...
Fix a too long line.
That's what you get for not running make lint :(
Move RenameFile to the new functions
ensure_dirs: Move some useful functions into utils.
With this change we can easily reuse this functionality where it makessense on other parts of Ganeti.
Bump version to 2.5.0~rc3
Use JoinDisjointDicts in mcpu
Signed-off-by: Andrea Spadaccini <spadaccio@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add the JoinDisjointDicts function to utils.algo
Add a function that joins two dictionaries, enforcing the constraintthat the two key sets should be disjoint. Also, add unit tests for thisfunction.
Fix queue archive creation with wrong permissions
On a master failover some of the archive dirs might have wrongpermissions in the non-root model. This is due to the nature of nodedstill running as root and the job queue is synced that way. This patchwill fix this behaviour by setting the permissions accordingly....
Allow per-hypervisor optional files
Rather than just allowing files for all nodes to be optional, we allowoptional files to be per-category. The way this works is that they mustbe included in both lists (the new code checks for this).
The code also removes a duplicate assert (present both in verify and...
Add hypervisors ancillary files list
These lists will be used to declare some of the files not necessarilyneeded on all nodes. The files selected are files without which thevarious hypervisors can still work, but that when they are presentshould be synchronized across the cluster (or node group)....
xen: abstract a few hardcoded strings as constants
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>