Bug #3414
AssertionError in Astakos gunicorn
Status: | Closed | Start date: | 03/08/2013 | |
---|---|---|---|---|
Priority: | High | Due date: | ||
Assignee: | Stratos Psomadakis | % Done: | 0% |
|
Category: | Astakos | Spent time: | - | |
Target version: | 0.13.0 |
Description
As seen in production:
2013-03-08 12:58:28,175 [18951] astakos.im.api.backends.lib.django __init__ [ERROR] This event is already used by another greenlet: (<Greenlet at 0x23cf9e0: <bound method GeventWorker.handle of <gunicorn.workers.ggevent.GeventWorker object at 0x11a0390>>(<socket at 0x2b8e7d0 fileno=24>, ('2001:648:2ffc:106::82', 46750, 0, 0))>, SSLError('The read operation timed out',)) Traceback (most recent call last): File "/usr/lib/python2.6/dist-packages/astakos/im/api/backends/lib/django/__init__.py", line 71, in wrapper data = func(self, *args, **kwargs) or () File "/usr/lib/python2.6/dist-packages/astakos/im/api/backends/lib/django/__init__.py", line 241, in get_resource_usage data = get_quota([user]) File "/usr/lib/python2.6/dist-packages/astakos/im/functions.py", line 393, in get_quota return qh_get_quota(users, resources) File "/usr/lib/python2.6/dist-packages/astakos/im/endpoints/qh.py", line 154, in qh_get_quota result = c.get_quota(context={}, clientkey=clientkey, get_quota=payload) File "/usr/lib/python2.6/dist-packages/kamaki/clients/commissioning/callpoint.py", line 84, in call_func return self.make_call(local_call_name, data) File "/usr/lib/python2.6/dist-packages/kamaki/clients/commissioning/callpoint.py", line 161, in make_call data = self.do_make_call(call_name, data) File "/usr/lib/python2.6/dist-packages/kamaki/clients/commissioning_client.py", line 59, in do_make_call resp = method(path, data=json_data, success=(200, 450, 500)) File "/usr/lib/python2.6/dist-packages/kamaki/clients/__init__.py", line 248, in get return self.request('get', path, **kwargs) File "/usr/lib/python2.6/dist-packages/kamaki/clients/__init__.py", line 216, in request recvlog.info('%d %s', r.status_code, r.status) File "/usr/lib/python2.6/dist-packages/kamaki/clients/connection/__init__.py", line 108, in status_code self._get_response() File "/usr/lib/python2.6/dist-packages/kamaki/clients/connection/kamakicon.py", line 58, in _get_response r = self.request.getresponse() File "/usr/lib/python2.6/httplib.py", line 990, in getresponse response.begin() File "/usr/lib/python2.6/httplib.py", line 391, in begin version, status, reason = self._read_status() File "/usr/lib/python2.6/httplib.py", line 349, in _read_status line = self.fp.readline() File "/usr/lib/python2.6/socket.py", line 427, in readline data = recv(1) File "/usr/lib/pymodules/python2.6/gevent/ssl.py", line 232, in recv return self.read(buflen) File "/usr/lib/pymodules/python2.6/gevent/ssl.py", line 116, in read wait_read(self.fileno(), timeout=self.timeout, timeout_exc=_SSLErrorReadTimeout, event=self._read_event) File "/usr/lib/pymodules/python2.6/gevent/socket.py", line 165, in wait_read assert event.arg is None, 'This event is already used by another greenlet: %r' % (event.arg, ) AssertionError: This event is already used by another greenlet: (<Greenlet at 0x23cf9e0: <bound method GeventWorker.handle of <gunicorn.workers.ggevent.GeventWorker object at 0x11a0390>>(<socket at 0x2b8e7d0 fileno=24>, ('2001:648:2ffc:106::82', 46750, 0, 0))>, SSLError('The read operation timed out',))
Could it be an interaction between astakos and qholder?
Astakos/kamaki/qholder using non-monkey-patched socket functionality, giving gevent trouble?
This is a major problem for production, it brings down Astakos.
Related issues
History
#1 Updated by Georgios Tsoukalas about 11 years ago
- Status changed from Assigned to Closed