Statistics
| Branch: | Tag: | Revision:

root / lib / jqueue.py @ 653bc0f1

History | View | Annotate | Download (75.7 kB)

# Date Author Comment
653bc0f1 02/27/2014 03:17 pm Michele Tartara

Make the AddReason method public

It will need to be accessed from outside the class too in one of the next
commits.

Signed-off-by: Michele Tartara <>
Reviewed-by: Klaus Aehlig <>

ead368bb 02/24/2014 04:42 pm Klaus Aehlig

Merge branch 'stable-2.10' into stable-2.11

  • stable-2.10
    Let the instance's tuple of nodes start with the primary
    Fix the test that checks for the order of instance's nodes
    Revision bump for the 2.10.0 release
    Update NEWS file for 2.10.0
  • stable-2.9...
2977b72c 02/24/2014 01:53 pm Klaus Aehlig

Merge branch 'stable-2.9' into stable-2.10

  • stable-2.9
    Fix net-common script in case of routed mode
    Document hspace's SPN parameters
    Document spindles in TSPEC
    man: Update the manual page of gnt-node FAILOVER
    Make hbal deal with no-LVM storage space properly...
e6e17529 02/20/2014 05:23 pm Hrvoje Ribicic

Improve job status assert affected by race condition

In the sliver of time between choosing a waiting job to be executed and
trying to acquire locks for its execution, the status of the job can be
changed to canceling. An assert checking the job status neglected to...

baaa2ece 01/28/2014 03:17 pm Klaus Aehlig

Merge branch 'stable-2.10' into stable-2.11

  • stable-2.10
    Enable network tags in Haskell code
    Haskell style: explicit errors, visible in the type
    Clarify spacing in record syntax
    Improve the point-free section of the style guide
    Add code style document to documentation...
410945f1 01/27/2014 05:36 pm Klaus Aehlig

Merge branch 'stable-2.9' into stable-2.10

  • stable-2.9
    Revision bump for 2.9.3
    Schedule 2.9.3 release
    Document fix of issue 691 in NEWS
    NEWS: fix typo in 2.8.4 release
    Fix 'hvparams' of '_InstanceStartupMemory' on hypervisors
    Add missing option to gnt-instance documentation...
95a4e33f 01/20/2014 02:11 pm Hrvoje Ribicic

Add job id type assert to jqueue.py

While the changes introduced in previous patches should stop any job
id parameters reaching the queue as strings, add an assertion here to
catch any strings making it through.

Signed-off-by: Hrvoje Ribicic <>...

f3ac6f36 12/19/2013 01:59 pm Klaus Aehlig

Merge branch 'stable-2.10' into master

  • stable-2.10
    Version bump for 2.10.0~rc1
    Update NEWS for 2.10.0 rc1 release
    Fix pylint 0.26.0/Python 2.7 warning
    Update INSTALL and devnotes for 2.10 release
  • stable-2.9
    Bump revision for 2.9.2
    Update NEWS for 2.9.2 release...
4869595d 12/17/2013 10:11 pm Petr Pudlak

Add a Python directory for RPC code to keep it at one place

Move rpc.py to rpc/node.py and modify imports in existing code.

Signed-off-by: Petr Pudlak <>
Reviewed-by: Helga Velroyen <>

c486fb6c 12/16/2013 12:37 pm Thomas Thrainer

Merge branch 'stable-2.9' into stable-2.10

  • stable-2.9
    Bump revision for 2.9.2
    Update NEWS for 2.9.2 release
    Pass hvparams to GetInstanceInfo
    Adapt parameters that moved to instance variables
    Avoid lines longer than 80 chars
    SingleNotifyPipeCondition: don't share pollers...
9cbcb1be 12/05/2013 03:10 pm Hrvoje Ribicic

Fix job error message after unclean master shutdown

According to commit 599ee321eb, any job-related error messages should
be encoded within a Ganeti-specific error and not passed on as a
string, to allow for easier parsing.

For jobs suffering from an undesirable status after an unclean master...

39348887 12/04/2013 05:52 pm Klaus Aehlig

Remove queue inspection from masterd

When masterd was responsible for queue handling, it would, on
restart, pick up all non-started jobs from the queue. Now that
luxid is responsible for handling the queue, make masterd not
do this any more.

Signed-off-by: Klaus Aehlig <>...

704b51ff 11/08/2013 07:00 pm Klaus Aehlig

Make masterd call luxid to write jobs to queue

As the responsibility for writing the job queue changed
to luxid, make masterd call to luxid when it is necessary
to write a job.

Signed-off-by: Klaus Aehlig <>
Reviewed-by: Michele Tartara <>

fcd70b89 11/08/2013 07:00 pm Klaus Aehlig

Factor out functionality to pickup a job from the queue

The current restart procedure for masterd includes functionality
to pick up a job from the queue and restart it, if it hasn't been
started before. Move this functionality into a separate function
to be able to have the enqueuing be done by luxid....

346c3037 10/04/2013 05:11 pm Klaus Aehlig

Provide means of submitting jobs to a drained queue

During an upgrade, the job queue needs to be drained in order to avoid
new jobs coming to the cluster. Nevertheless, the upgrade process
needs to carry out some maintenance, like redistributing the new...

580b1fdd 08/07/2013 10:42 am Jose A. Lopes

Hook h2spy in Makefile.am

  • add rules to Makefile.am to use hs2py to generate the Python opcodes
    from Haskell and update tests to check that Haskell and Python contain
    the same opcodes.
  • split 'opcodes.py' in 'opcodes.py.in_after' and 'opcodes_base.py',...
e0f2bf1e 04/22/2013 02:17 pm Michele Tartara

Add job_id and index to the reason trail

The reason trail will contain an item indicating the job_id and the
index number of the current opcode inside the job queue.

Signed-off-by: Michele Tartara <>
Reviewed-by: Helga Velroyen <>

fe05a931 04/10/2013 06:09 pm Michele Tartara

Fix job queue directory permission problems

If split users are used, the queue directory could only be accessed
by masterd, but also confd needs to be able to read it, e.g. when it
is queried as part of "gnt-job list"

This commit fixes the permissions in such a way to allow proper access rights....

383477e9 12/13/2012 01:27 pm Michael Hanselmann

jqueue: Improve inotify error reporting

This addresses issue 218. When the number of inotify watches is
exhausted, for example by being set too low from the beginning or by
other programs, waiting for a job to change would just report a lost job
(e.g. “Error checking job status: Job with id 7817 lost”)....

be6c403e 12/11/2012 03:37 pm Michael Hanselmann

Replicate queue drain flag across all master candidates

Until now, the flag was unset on a master failover unless the
“$localstatedir/lib/ganeti/queue/drain” file existed.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Guido Trotter <>

3c631ea2 12/05/2012 05:35 pm Michael Hanselmann

jqueue: Don't modify input opcode when changing priority

Commit 4679547 implemented the ability to change job's priority after it
was submitted. The code contained a bug whereby it would modify the
input data for an opcode, something the job queue shouldn't do (logical...

2c9fa1ff 11/20/2012 11:16 am Iustin Pop

Cleanup ht's use of positive/strictpositive

Currently, ht.py uses a bad terminology for positive/non-negative
numbers. Per http://en.wikipedia.org/wiki/Positive_number, this is the
correct terminology:

- A number is positive if it is greater than zero.
- A number is negative if it is less than zero....

4679547e 11/13/2012 09:20 pm Michael Hanselmann

jqueue: Allow changing of job priority

This is due to a feature request. Sometimes one wants to change the
priority of a job after it has been submitted, e.g. after submitting an
important job only to later notice many other pending jobs which will be
processed first. Priority changes only take effect at the next lock...

99fb250b 11/13/2012 09:20 pm Michael Hanselmann

jqueue: Set task ID for jobs added to workerpool

The job ID is re-used as the task ID, as job IDs are unique.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Bernardo Dal Seno <>

e4e59de8 11/08/2012 04:40 pm Michael Hanselmann

jqueue/mcpu: Determine priority using callback

Instead of being given the priority for acquiring locks by means of a
parameter, mcpu will now call back. This is in preparation for
implementing a command to change a job's priority on the fly and allows
to change it while locks are being acquired (taking effect on the next...

8e09e801 11/01/2012 07:44 pm Michael Hanselmann

Merge branch 'devel-2.6'

  • devel-2.6:
    Fix typo in gnt-instance man page
    jqueue: Return jobs to queue when shutting down
    gnt-debug delay: Add "--submit" option
    Make hostname checks uniform between instance rename and add
    Improve logging of new job submissions...
942e2262 11/01/2012 05:48 pm Michael Hanselmann

jqueue: Return jobs to queue when shutting down

When a job is still waiting for locks and the queue is shutting down,
they should be returned and not actually start processing. Until now
jobs which transitioned from “queued” to “waiting” were already
considered to be running as far as the shutdown code was concerned....

aebd0e4e 10/25/2012 03:16 pm Michael Hanselmann

jqueue: Factorize code to modify job

A new function will be added to change a job's priority.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Bernardo Dal Seno <>

bb921668 10/25/2012 03:16 pm Michael Hanselmann

jqueue: Add docstring for _DetermineJobDirectories

Somehow this was missed in commit 0422250e.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Helga Velroyen <>

42d49574 10/25/2012 03:16 pm Michael Hanselmann

jqueue: Fix comments in _SubmitJobUnlocked

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Helga Velroyen <>

0422250e 10/11/2012 12:23 pm Michael Hanselmann

gnt-job: List archived jobs if requested

If requested via a filter or by including the “archived” output,
archived jobs will be loaded and shown. This is significantly slower
than just listing normal jobs, therefore by default they are not loaded
at all....

8a3cd185 10/11/2012 12:23 pm Michael Hanselmann

jqueue: Add new in-memory attribute for archived jobs

This attribute is set to True for jobs which were restored from an
archived file. A new filter will act on this field.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

4c27b231 10/11/2012 12:23 pm Michael Hanselmann

jqueue: Correct docstring

The description was not accurate.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

e4cf42d4 10/05/2012 07:14 pm Michael Hanselmann

jqueue: Look at archived jobs when watching

First: This enables the use of “gnt-job watch $id” for archived jobs.

Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where...

04569469 10/05/2012 06:46 pm Michael Hanselmann

jqueue: Look at archived jobs when watching

First: This enables the use of “gnt-job watch $id” for archived jobs.

Now, the reason for actually making this work is that during
sufficiently large group or node evacuations jobs are archived before
the client gets to poll for their output. This led to situations where...

cffbbae7 09/25/2012 05:28 pm Michael Hanselmann

Implement virtual cluster support in Python code

- pathutils: Prepend node-specific prefix path
- RPC: Use virtual paths (see vcluster.py)
- SSH: Pass environment variables, use destination's node directory when
copying files using scp, use GANETI_HOSTNAME to determine hostname...

e2b4a7ba 09/18/2012 06:09 pm Michael Hanselmann

Migrate lib/{jqueue,jstore}.py from constants to pathutils

File system paths moved from constants to pathutils.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

76b62028 08/07/2012 12:48 pm Iustin Pop

Switch job IDs to numeric

This has been a long-standing cleanup item, which we've always
refrained from doing due to the high estimated effort needed.

In reality, it turned out that after some infrastructure improvements
(the previous patches), the actual job queue-related changes are quite...

1410a389 06/15/2012 02:50 pm Michael Hanselmann

jqueue: Move functions related to job ID to jstore

These don't really need to be in jqueue, and a new function will
be added to convert job IDs to an integer for queries.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

e07f7f7a 03/30/2012 03:03 pm Michael Hanselmann

Add job support to query2 via LUXI

This enables the use of filters through query2 when listing jobs.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

dc2879ea 03/30/2012 03:03 pm Michael Hanselmann

jqueue: Cache prepared field list in _JobChangesChecker

… instead of re-calculating it on every file change.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

a06c6ae8 03/30/2012 03:03 pm Michael Hanselmann

jqueue: Convert GetInfo to query2

This rather inefficient implementation (fields are evaluated on every
call to GetInfo) is not good for WaitForJobChanges and doesn't support
filters, but that will be rectified in later patches.

Signed-off-by: Michael Hanselmann <>...

66abb9ff 03/30/2012 03:02 pm Michael Hanselmann

jqueue._QueuedOpCode: Change a docstring

There was a typo and it's not necessary to repeat the class name.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

aa66c183 12/22/2011 07:16 pm Michael Hanselmann

Merge branch 'devel-2.5'

  • devel-2.5:
    jqueue: Factorize checking job processor's result
    jqueue unittest: Rename simple fake-job class
    jqueue: Fix epylint errors introduced in 37d76f1e4
    jqueue: Fix deadlock between job queue and dependency manager...
4f44e311 12/22/2011 06:41 pm Michael Hanselmann

Merge branch 'stable-2.5' into devel-2.5

  • stable-2.5:
    jqueue: Fix epylint errors introduced in 37d76f1e4
    jqueue: Fix deadlock between job queue and dependency manager

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

df5a5730 12/22/2011 03:19 pm Michael Hanselmann

jqueue: Factorize checking job processor's result

This allows for more unittesting.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

1316ebc2 12/21/2011 06:04 pm Michael Hanselmann

jqueue: Fix epylint errors introduced in 37d76f1e4

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

a182a3ed 12/21/2011 04:55 pm Michael Hanselmann

serializer: Remove JSON indentation and dict key sorting

Serializing to JSON using “simplejson” is significantly slower when
indentation and/or sorting of dictionary keys is used. In simplejson 1.x
the difference isn't that big, but with simplejson 2.x the difference...

37d76f1e 12/21/2011 04:35 pm Michael Hanselmann

jqueue: Fix deadlock between job queue and dependency manager

When an opcode is about to be processed its dependencies are
evaluated using “_JobDependencyManager.CheckAndRegister”. Due
to its nature that function requires a lock on the manager's
internal structures. All of this happens while the job queue...

6d5ea385 11/21/2011 09:36 am Michael Hanselmann

jqueue: Add code to prepare for queue shutdown

Doing so will prevent job submissions (similar to a drained queue),
but won't affect currently running jobs. No further jobs will be
executed.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

0e82dcf9 11/18/2011 01:48 pm Andrea Spadaccini

Merge branch 'devel-2.5'

  • devel-2.5: (24 commits)
    LUInstanceCreate: Release unused node locks
    htools: rework message display construction
    hbal: handle empty node groups
    Document OpNodeMigrate's result for RAPI
    Ensure unused ports return to the free port pool...
c8d0be94 11/17/2011 03:39 pm Michael Hanselmann

jqueue: Factorize code checking for drained queue

This is in preparation for a clean(er) shutdown of masterd.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

719f8fba 10/27/2011 04:26 pm Michael Hanselmann

jqueue: Allow zero jobs to be submitted at once

If cmdlib.LUNodeMigrate was called for a node without primary instances
it would try to submit an empty list of jobs. This was never visible via
CLI as there we check the list of primary instances first.

Signed-off-by: Michael Hanselmann <>...

fb1ffbca 10/26/2011 11:53 am Michael Hanselmann

Convert job queue's RPC to generated code

With these changes job queue RPC will finally show up on the lock
monitor. See below for an example. A job queue-specific class is used to
restrict the use of a static list for name resolution to the job queue.
Further improvements can be made to not re-create the whole RPC client...

17385bd2 08/30/2011 02:01 pm Andrea Spadaccini

Fixes to errors/warnings raised by pylint 0.24

Running pylint 0.24.0 revealed 2 errors and 1 warning. Here is how I
fixed them:

  • jqueue.py: silenced E1101
  • netutils.py: rewrote the list comprehension using extend()
  • watcher/__init__.py: fixed a missing format string parameter...
b459a848 08/30/2011 11:24 am Andrea Spadaccini

DeprecationWarning fixes for pylint

In version 0.21, pylint unified all the disable-* (and enable-*)
directives to disable (resp. enable). This leads to a lot of
DeprecationWarning being emitted even if one uses the recommended
version of pylint (0.21.1, as stated in devnotes.rst)....

cb66225d 08/19/2011 03:11 pm Michael Hanselmann

ensure-dirs: Set permissions on job files in queue

This was a regression from 2.4.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

dfc8824a 08/02/2011 12:56 pm Michael Hanselmann

jqueue: Add short delay before detecting job changes

By sleeping for 100ms after receiving a notification for a changed job
file the job is given some additional time to change again. This
significantly reduces the number of LUXI calls for WaitForJobChanges...

fcb21ad7 07/21/2011 02:55 pm Michael Hanselmann

Export job dependencies through lock monitor

This makes them visible to the user. Example:

$ gnt-debug locks -o name,pending
Name Pending
job/890 job:891,892
job/892 job:894

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

47099cd1 07/21/2011 02:20 pm Michael Hanselmann

Rename *_STATUS_WAITLOCK to …_WAITING

This patch renames the {JOB,OP}_STATUS_WAITLOCK constants to {JOB,OP}_STATUS_WAITING, as per design document for chained jobs.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

75d81fc8 07/21/2011 11:58 am Michael Hanselmann

Fix locking issue with job dependencies

When jobs waiting for a dependency are notified, they're re-added to the
queue. This would require owning the queue lock in exclusive mode, but
since the function doing so is called from within the job/opcode
processor, it only holds the lock in shared mode....

f8a4adfa 07/21/2011 11:56 am Michael Hanselmann

jqueue: Read-only jobs don't need processor lock

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

b247c6fc 07/21/2011 08:33 am Michael Hanselmann

jqueue: Implement submitting multiple jobs with dependencies

With this change users of the “SubmitManyJobs” interface can use
relative job dependencies. Relative job IDs in dependencies are resolved
before handing the job off to the workerpool.

Signed-off-by: Michael Hanselmann <>...

c0f6d0d8 07/20/2011 03:11 pm Michael Hanselmann

jqueue: Add “writable” flag to memory objects

Basically only one instance of the job, the one being processed,
should be serialized to disk and replicated to other nodes. With
this flag assertions can be added in various places.

Signed-off-by: Michael Hanselmann <>...

b95479a5 07/20/2011 03:11 pm Michael Hanselmann

Implement chained jobs

An overview is available in the design document for this change,
doc/design-chained-jobs.rst.

When a job enters the job processor, the current opcode's dependencies
are evaluated. If a referenced job has not yet reached the desired...

45df0793 07/15/2011 08:13 pm Michael Hanselmann

Fix assertion error on unclean master shutdown

Commit 66bd7445 added an assertion to ensure a finalized job has its
“end_timestamp” attribute set. Unfortunately it didn't cover a case when
the queue is recovering from an unclean master shutdown.

Signed-off-by: Michael Hanselmann <>...

b795a775 07/11/2011 05:53 pm Michael Hanselmann

Merge branch 'devel-2.4'

  • devel-2.4:
    ht: Add new check for numbers
    Fix off-by-one bug in job serial generation
    Shorten some unbreakable lines in man pages
    Correct some spelling mistakes
    Fix bug in recreate-disks for DRBD instances
    Fix a lint warning...
3c88bf36 07/11/2011 05:28 pm Michael Hanselmann

Fix off-by-one bug in job serial generation

Commit 009e73d0 (September 2009) changed the job queue to generate
multiple job serials at once. Ever since it would return one more than
requested.

The “serial” file in the job queue directory is defined to contain the...

194c8ca4 06/10/2011 06:48 pm Michael Hanselmann

jqueue: Allow loading of archived jobs

Chained jobs need to look at previous jobs, including archived ones. A
nice side-effect of this change is the ability to look at archived jobs
using “gnt-job info <id>” as long as the ID is known.

Signed-off-by: Michael Hanselmann <>...

07346f28 06/01/2011 08:08 pm Michael Hanselmann

Merge branch 'devel-2.4'

  • devel-2.4:
    jqueue: Fix potential race condition when cancelling queued jobs
    Fix argument order in ReserveLV and ReserveMAC

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

66bd7445 05/31/2011 07:17 pm Michael Hanselmann

jqueue: Fix potential race condition when cancelling queued jobs

When a job was cancelled, its status would be changed and the file
written again. Since this was a final status, the job file could be
moved anytime for archival. If the job was still in the queue, however,...

0aeeb6e3 05/10/2011 06:32 pm Michael Hanselmann

jqueue: Update worker thread name to include opcode summary

With this patch, the worker thread name is updated to include a short
summary of the opcode (basically its OP_ID). The base name of job queue
threads is shortened from “JobQueue” to “Jq”. Logs and the lock monitor...

6a373640 03/25/2011 03:53 pm Michael Hanselmann

Implement submitting jobs from logical units

The design details can be seen in the design document
(doc/design-lu-generated-jobs.rst).

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

98ed5092 03/23/2011 06:17 pm Michael Hanselmann

Add opcode summary to SubmitManyJobs errors

Requested-by: Iustin Pop <>
Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

ff699aa9 02/28/2011 05:26 pm Michael Hanselmann

gnt-cluster master-failover: Undrain queue

- Move functions for drain status (tracked via file) from jqueue to jstore
- Undrain queue on master failover if necessary
- Add QA test

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

30c945d0 12/29/2010 05:48 pm Michael Hanselmann

jqueue: Fix cancelling while in waitlock in queue

Since the recent change to leave jobs in the “waitlock” status (commit
5fd6b6947), cancelling a job while it's back in the queue would break.
This patch handles these cases and adds a unittest.

Signed-off-by: Michael Hanselmann <>...

5fd6b694 12/15/2010 03:42 pm Michael Hanselmann

jqueue: Keep jobs in “waitlock” while returning to queue

Iustin Pop reported that a job's file is updated many times while it
waits for locks held by other thread(s). After an investigation it was
concluded that the reason was a design decision for job priorities to...

9e49dfc5 10/12/2010 03:48 pm Michael Hanselmann

jqueue: Fix bug when cancelling jobs

If a job was cancelled while it was waiting for locks, an assertion
would've failed. This patch fixes the problem and provides a unit
test to check for this situation.

Signed-off-by: Michael Hanselmann <>...

b8802cc4 10/12/2010 03:48 pm Michael Hanselmann

jqueue/gnt-job: Add job priority fields for display

These fields can help with debugging.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

320d1daf 10/12/2010 03:48 pm Michael Hanselmann

jqueue: Resume jobs from “waitlock” status (2nd try)

Commit 5ef699a0e had to roll back an earlier attempt at implementing
this. With the improved job queue processer, this is finally possible.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

86b16e9d 10/07/2010 06:10 pm Michael Hanselmann

jqueue, CancelJob: Check status only once per call

This simplifies the code a bit--the status is only checked once.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

a38e8674 09/24/2010 07:21 pm Michael Hanselmann

Fix docstring typo in jqueue._JobProcessor._MarkWaitlock

epydoc complained:
“File …/ganeti/jqueue.py, line 886, in
ganeti.jqueue._JobProcessor._MarkWaitlock
Warning: Redefinition of type for job”

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

f23db633 09/24/2010 06:18 pm Michael Hanselmann

jqueue: Use priority for acquiring locks

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

26d3fd2f 09/24/2010 06:18 pm Michael Hanselmann

jqueue: Use timeout when acquiring locks

As already noted in the design document, an opcode's priority is
increased when the lock(s) can't be acquired within a certain amount of
time, except at the highest priority, where in such a case a blocking
acquire is used....

b80cc518 09/23/2010 04:06 pm Michael Hanselmann

jqueue: Introduce per-opcode context object

This is better to group per-opcode data.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

03b63608 09/23/2010 12:07 pm Michael Hanselmann

jqueue: Rename current_op to better reflect what it actually is

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

fa4aa6b4 09/23/2010 12:07 pm Michael Hanselmann

jqueue: Separate function for in-memory variables

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

be760ba8 09/20/2010 03:41 pm Michael Hanselmann

jqueue: Change model from per-job to per-opcode processing

In order to support priorities, the processing of jobs needs to be
changed. Instead of processing jobs as a whole, the code is changed to
process one opcode at a time and then return to the queue. See the...

7b5c4a69 09/20/2010 03:41 pm Michael Hanselmann

jqueue: Use priority for worker pool

A small helper function is added to make this easier. Priorities are not
yet used in all necessary places.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

a0d2fe2c 09/20/2010 11:11 am Michael Hanselmann

jqueue: Add missing docstring to _QueuedJob.Cancel

This was forgotten in commit 099b2870b.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

099b2870 09/16/2010 01:25 pm Michael Hanselmann

jqueue: Move CancelJob logic to separate function

Moving the internals of this function will allow it to be used from
unittests in the future. Splitting this into a pure, side-effect free
function and an impure one makes the pure function easily testable....

42e32075 09/16/2010 10:25 am Iustin Pop

Merge branch 'devel-2.2'

  • devel-2.2:
    Fix case of MAC special-values
    Remove mcpu's ReportLocks callback
    Revert "jqueue: Resume jobs from “waitlock” status"

(no conflicts, took LGTM from original commit)

Signed-off-by: Iustin Pop <>...

e71c8147 09/13/2010 06:35 pm Michael Hanselmann

jqueue: Ensure only accepted priorities are allowed for submitting jobs

Quoting the design document: “Submitted opcodes can have one of the priorities
listed below. Other priorities are reserved for internal use”. Submitting jobs
at priority -20 should not be allowed....

8f5c488d 09/13/2010 06:35 pm Michael Hanselmann

Add support for job priority to opcodes and job queue objects

This allows clients to submit opcodes with a priority. Except for being
tracked by the job queue, it is not yet used by any code.

Unittests for jqueue._QueuedOpCode and jqueue._QueuedJob are provided for...

acf931b7 09/13/2010 05:46 pm Michael Hanselmann

Remove mcpu's ReportLocks callback

This is no longer needed with the new lock monitor. One callback is kept to
check for cancelled jobs.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>

5ef699a0 09/13/2010 05:39 pm Michael Hanselmann

Revert "jqueue: Resume jobs from “waitlock” status"

This reverts commit 4008c8edae31a3971fa8c4b200238afc8005d3d4.

While it worked in my initial tests, I've now found cases where this doesn't
work properly as it is. More work is needed and will be done as part of the...

a68fe106 09/10/2010 03:11 pm Michael Hanselmann

Merge branch 'devel-2.2'

  • devel-2.2:
    Fix pylint warning in http/__init__.py
    Allow SSL ciphers to be overridden in HTTP server
    jqueue: Resume jobs from “waitlock” status
    jqueue: Move queue inspection into separate function
    jqueue: Don't update file in MarkUnfinishedOps...
4008c8ed 09/10/2010 02:23 pm Michael Hanselmann

jqueue: Resume jobs from “waitlock” status

After an unclean restart of ganeti-masterd, jobs in the “waitlock” status can
be safely restarted. They hadn't modified the cluster yet.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

de9d02c7 09/10/2010 02:23 pm Michael Hanselmann

jqueue: Move queue inspection into separate function

This makes the init function a lot smaller while not changing
functionality.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: René Nussbaumer <>

747f6113 09/10/2010 02:23 pm Michael Hanselmann

jqueue: Don't update file in MarkUnfinishedOps

This reduced the number of updates to the job files. It's used in two places
while processing a job and the file is updated just afterwards.

Signed-off-by: Michael Hanselmann <>
Reviewed-by: Iustin Pop <>