Bug #3479

Database stack limit reached in retrieving quota for all users

Added by Georgios Tsoukalas about 11 years ago. Updated about 11 years ago.

Status:Closed Start date:03/21/2013
Priority:High Due date:
Assignee:Giorgos Korfiatis % Done:

0%

Category:Astakos Spent time: -
Target version:0.13.0

Description

From the logs:
...astakos.im.management.commands.astakos-quota astakos-quota [ERROR] stack depth limit exceeded
HINT: Increase the configuration parameter "max_stack_depth", after ensuring the platform's stack depth limit is adequate.

The server setting was the default:
#max_stack_depth = 2MB # min 100kB

The query was of the form:
SELECT ... WHERE (a=... AND b=...) OR (a=... AND b=...) ... FOR UPDATE.

The number of OR terms was around 15800.

It seems that during the parsing of the statement the OR-tree is not optimized into a flat structure,
and the resulting depth causes the recursion to reach the stack limit.
The query form is a result of repeated Qall |= Qx Django queryset calls.

The efficient SQL query can be expressed in the form:
SELECT ... WHERE (a, b) in ((..., ...), ...) FOR UPDATE.

However the Django __in notation cannot express the (a, b) part of the syntax
even though it can express the right-hand list of tuples.

Associated revisions

Revision 2f906450
Added by Giorgos Korfiatis about 11 years ago

Simplify query to avoid hitting db stack limit

Refs #3479

Looking up (entity, resource) pairs created a huge query formula
in disjunctive normal form, which apparently brought the db parser
to its limits.

It has been replaced by selecting the holdings by entity only.
This means that in theory we lock more holdings than needed (for any
resource for a given entity), but in practice we would need to
select these holdings anyway.

Revision f6950864
Added by Giorgos Korfiatis about 11 years ago

Sync only quotas that differ

Refs #3479

Database stack limit issue was triggered by syncing quotas for
all users. It is much faster to compare quotas locally and send
for sync to quotaholder only those that differ.

History

#1 Updated by Giorgos Korfiatis about 11 years ago

  • Status changed from New to Resolved

#2 Updated by Georgios Tsoukalas about 11 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF