gnt-debug: Extend job queue tests
Test multiple opcodes, also with failure.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
jqueue: Mark opcodes following failed ones as failed, too
When an opcode fails, the job queue would leave following opcodes as “queued”,which can be quite confusing. With this patch, they're all marked as failed andassertions are added to check this.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
jqueue: Work around race condition between job processing and archival
This is a simplified version of a patch I sent earlier to make sure the jobfile is only written once with a finalized status.
jqueue: More checks for cancelling queued job
We can also check when the lock status is updated. This willimprove job cancelling.
errors: Function to check whether value is encoded error
jqueue: Add more debug output
gnt-backup: Pass error code to OpPrereqError
Merge branch 'devel-2.1'
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Re-add the 'live' parameter to migration opcodes
This patch reintroduces the live parameter, for backwards compatibilityat the Luxi level. This way, clients can work transparently with both2.1 and 2.2, even though sub-optimally.
Signed-off-by: Iustin Pop <iustin@google.com>...
Fix path in ganeti-rapi man page
This takes care of issue 116.
workerpool: Don't keep reference to task arguments
The workerpool should not keep any reference to task arguments afterthey were processed by RunTask. Doing so led to jobs being cachedby the job queue's WeakValueDictionary for longer than they should've...
cli.SubmitOpCode: Pass keyword parameter as keyword
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
gnt-backup: Don't show confusing message w/o target node
“gnt-backup export” requires the target node. Until now, the masterdaemon would complain that the “parameter'OP_BACKUP_EXPORT.target_node' fails validation”. With this patch,an additional check is done in the client program....
masterd.instance: Add missing argument
_DiskTransferPrivate takes three parameters, not two.
Adjust message in case ~/.ssh is no directory
Use actual path, not something hardcoded.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI client: Fix docstring for migrating instance
QA: Test renaming instance via RAPI
RAPI client: Support renaming instances
Allow renaming instances via RAPI
RAPI client: Don't re-use PycURL object
With this patch, a new PycURL object will be created for each request.This should make the RAPI client safe for simultaneous calls frommultiple threads. Unittests are adjusted accordingly.
An unnecessary variable assignment is also removed from the unittest...
Add --no-name-check to 'gnt-instance rename' man page
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix bug in bdev when drbd version format is x.x.x.x
This patch fixes a bug reported in [0]. Newer drbd versions can haveanother digit beyond the regular major, minor and point release digits.We modify the regex used to match that with an optional parts which is...
Bump version to 2.2.0~rc0
move-instance: Use constants for parameters
Allow instance NIC's IP address to be None
Also add some assertions.
Test instance NIC and disk parameter names
Add new parameter type “maybe string”
Before strict checking was implemented, NIC IP addresses could be setto “None”. Commit bd061c35 added more strict checking, includingenforcing the IP address to be a string. With this new type, itcan again be set to None....
cmdlib: Change expected type for source CA on remote import
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
move-instance: Pass OS parameters to new instance
Update NEWS file for the first release candidate
Fix a few job archival issues
This patch fixes two issues with job archival. First, theLoadJobFromDisk can return 'None' for no-such-job, and we shouldn't addNone to the job list; we can't anyway, as this raises an exception:
node1# gnt-job archive foo...
burning: fix handling of empty job sets
If we call burning with only existing instance, then it will fail tocreate any of them, and thus in the removal phase it won't have anythingto remove. Since calling luxi.SUBMIT_MULTIPLE_JOBS with an empty job set...
Change semantics of --force-multi for reinstall
Currently, we require both --force and --force-multiple for skipping theconfirmation on instance reinstalls. After offline conversations, thishas been deemed to be excessive, and this patch changes the meaning of...
Change handling of non-Ganeti errors in jqueue
Currently, if a job execution raises a Ganeti-specific error (i.e.subclass of GenericError), then we encode it as (error class, [errorargs]). This matches the RAPI documentation.
However, if we get a non-Ganeti error, then we encode it as simply...
Implement gnt-cluster master-ping
This can be used from shell-scripts to quickly check the status of themaster node, before launching a series of jobs (and handling the failureof the jobs due to masterd other issues).
Instance migration: remove error on missing link
Since we don't support upgrades from 1.2.4 without restarting theinstance, the 'not restarted since 1.2.5' check/error iswrong/misleading.
Since the live migration works anyway without the links (it recreates...
Add check for RAPI paths to start with /2
During a discussion in July 2010 it was decided that we'll stabilize on /2. Seemessage ID <20100716180012.GA9423@google.com> for reference.
Ensure assertions are evaluated in tests
A lot of assertions are used in Ganeti's code. Some unittests even checkwhether AssertionError is raised in some cases. Explicitely ensuringassertions are evaluated makes sure those tests don't fail andassertions are checked....
RAPI client: The os argument for instance reinstalls is optional
Signed-off-by: David Knowles <dknowles@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RAPI client: Support migrating instances
QA: Test instance migration via CLI and RAPI
workerpool: Add two additional assertions
workerpool: Change signature of AddTask function to not use *args
By changing it to a normal parameter, which must be a sequence, we canstart using keyword parameters.
Before this patch all arguments to “AddTask(self, *args)” were passed asarguments to the worker's “RunTask” method. Priorities, which should be...
RAPI: Support migrating instances
workerpool: Move waiting for new tasks for a worker to the pool
This way fewer private variables of the pool are accesssed by the worker.
workerpool: Remove unused worker method
HasRunningTask is never used except for an assertion, where wedon't really need the lock.
workerpool: Additional check in BaseWorker.ShouldTerminate
Document that it should only be called from within RunTask andadd an assertion for this. This means we can no longer use amethod on the pool and hence remove WorkerPool.ShouldWorkerTerminate.
workerpool: Use common function to add tasks
Fix install document regarding DRBD usage
This is related to issue 105.
Update RAPI documentation for the OS changes
Rename masterfailover to master-failover
Most (all?) of our commands use dash-separator: replace-disks,verify-disks, add-tags, etc. “gnt-cluster masterfailover” is an oldexception to this rule.
The patch replaces it with master-failover, add a compatiblity alias,...
RAPI: Add os params to instance creation v1
Since the RAPI QA suite doesn't seem to offer easy testing of failedcreations, I didn't add this to the QA. Pointers on how to do it arewelcome.
The patch also changes the 'os' argument to be required, since that is...
makefile: fix TAGS building
“find .” requires that “-path” arguments start with a dot, otherwisethey are not matches. Additionally, we also include the QA files in thetags, for easier search while modifying the QA suite.
Improve handling of lost jobs
Currently, if the cli.JobExecutor class is being used, and one of thejobs is being archived before it can check its result, it will raise astracktrace as _ChooseJob is not prepared to handle this case.
This case makes JobExecutor work better with lost jobs (it still reports...
luxi: convert permission errors into exception
This patch adds handling of permission errors so that we don't showtracebacks when a non-root user runs a gnt-* command. Since in thefuture we'll have different permissions, we need to handle this in RAPItoo....
cmdlib: Return new name from rename operations
The new name is then displayed by the clients.
gnt-instance rename: Fix bug and rename params
This patch fixes a bug when gnt-instance rename was invoked with--no-name-check. It renames the internal variables to be consistent withthe ones in equivalent instance add code. Furthermore it checks whether...
QA: add tests for the reserved lvs feature
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Add modification of the reserved logical volumes
This doesn't allow addition/removal of individual volumes, onlywholesale replace of the entire list. It can be improved later, if weever get generic container parameters.
The man page changes replaces some tabs with spaces (hence the...
Add printing of reserved_lvs in cluster info
Introuce a new cluster parameter - reserved_lvs
This parameter, which is a list of regular expression patterns, willmake cluster verify ignore any such LVs. It will not prevent creation orremoval of such volumes by the backend code.
Change the meaning of call_node_start_master
Currently, backend.StartMaster (the function behind this RPC call) willactivate the master IP and then, if the start_daemons parameter is true,it will also activate the master role.
While this works, it has two issues:...
masterd: move the IP activation from Exec to Check
Currently, the master IP activation is done in the Exec function. Sincethe original masterd process returns after forking, and Exec is run inthe (grand)child process, this means that after 'ganeti-masterd' has...
Move the UsesRPC decorator from cli to rpc
This is needed because not just the cli scripts need this decorator, butthe master daemon too (and it already duplicated the code once).
In cli.py we just leave a stub, so that we don't have to modify all thescripts to import rpc.py....
watcher: smarter handling of instance records
This patch implements a few changes to the instance handling. First, oldinstances which no longer exist on the cluster are removed from thestate file, to keep things clean.
Second, the instance restart counters are reset every 8 hours, since...
Update the RAPI node migrate for the 'live' change
This patch adds handling of the new 'mode' parameter to the RAPI server,while keeping compatibility with the old mode. Note that in the old mode(when 'live' is being passed), the auto-mode doesn't work....
Update the RAPI client for the migration mode
See the discussion on the previous patch about this. Basically unless wewant to a add a new 'feature' marking for the live migration parameter,there is no simple way to handle this nicely in the client.
Given that the client was/is marked as experimental, this patch simply...
Fix burnin and live migration
This is breakage from the original 'live' parameter changes.
Rename the OpMigrate* parameter 'live' to 'mode'
This is needed as now the parameter is no longer boolean, but tri-state.
Rename migration type to migration mode
This is in preparation for the rename of the opcode 'live' parameter to'mode'.
utils: Fix incorrect docstring
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.1' into master
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix issue when changing the disk template to drbd
If we pass the current primary node, the conversion will fail horriblywith LVM creation errors. Instead, we catch and check for thiscondition in CheckPrereq.
Remove a couple of empty design sections
The 2.1 and 2.2 designs contain sections with no actual content, as theyare detailed for each single change. Removing the global empty ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
Disable 'invalid name' pylint warning for tools/setup-ssh
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
Always set commonName in X509 certificates
Due to the current switch of the RPC client to PycURL, a bug with newerversions of libcurl surfaced. When the 'Subject' or 'Issuer' of'server.pem' were empty, SSL handshake failed.
This patch changes the certificate generation functions such that they...
Adding tool to setup SSH on a remote host
This prepares the remote node to be joined into a cluster
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Adding new (optional) dependency to configure.ac
Adding constants for setup-ssh
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Change AddAuthorizedKey to also allow filehandles
This is required to use this function over paramikosftp file handles.
Update .gitignore for vcs-version
RAPI client: Encode empty body to JSON
If the body consists of an empty dict, it should also be encoded.
Introduce git reference/tag tracking for debugging
This patch adds a new vcs-version file that is generated via git (andcan be adapted if VCS is changed) and then embebbded as VCS_VERSION inthe constants module.
This means two things:- local modifications without committing to git (or when using a tar.gz...
Fix epydoc warning "Lists must be indented."
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Convert RPC client to PycURL
Instead of using our custom HTTP client, using PycURL's multiinterface allows us to get rid of the HTTP client threadpool.The majority of the code is still in the ganeti.http.clientmodule.
A simple per-thread HTTP client pool gives cURL a chance to...
Implement lock names for debugging purposes
This patch adds lock names to SharedLocks and LockSets, that can be usedlater for displaying the actual locks being held/used in places where weonly have the lock, and not the entire context of the locking operation....
Conflicts: NEWS - merge configure.ac - keep 2.2 version
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Bump up version to release 2.1.6
Update NEWS file for 2.1.6
Fix pylint complaints introduced in commit e58f87a958c
Due to a small mistake I missed three non-critical pylint complaints forcommit e58f87a958c. They're fixed with this patch.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Signed-off-by: Iustin Pop <iustin@google.com>...
LXC: Add cpu_mask hypervisor parameter
Also implement syntax checking.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add ParseCpuMask() utility function
Also adds a generic ParseError exception.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add a migration type global hypervisor parameter
Since migration live/non-live is more stable (e.g.) for Xen-PVM versusXen-HVM, we introduce a new parameter for what mode we should use bydefault (if not overridden by the user, in the opcode).
The meaning of the opcode 'live' field changes from boolean to either...
Add test for some aspects of job queue
This new opcode and gnt-debug sub-command test some aspects of thejob queue, including the status of a job. The bug fixed in commit2034c70d507 was identified using this test. A future patch willrun this test automatically from the QA scripts....
LUVerifyCluster: update _ValidateNode description
Change _ValidateNode description to reflect what the function actually does.
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
KVM hypervisor: Use utils.ShellWriter for network script
This patch converts hv_kvm to use utils.ShellWriter for writingthe network script. It also adds a few unittests (the firstfor any hypervisor modules).
Move ShellWriter class to utils
Also add unittest.
Rename test for utils.IgnoreProcessNotFound
Usually our tests are named “Test…”.
jqueue: Factorize code waiting for job changes
By splitting the _WaitForJobChangesHelper class into multiple smallerclasses, we gain in several places:
- Simpler code, less interaction between functions and variables- Easy to unittest (close to 100% coverage)...
Merge remote branch 'origin/devel-2.1'
Conflicts: test/ganeti.rapi.client_unittest.py: Trivial test/ganeti.rapi.rlib2_unittest.py: Trivial...