Bug #1021

The reconcilation mechanism must be overhauled

Added by Vangelis Koukis almost 11 years ago. Updated almost 11 years ago.

Status:Closed Start date:08/24/2011
Priority:Medium Due date:
Assignee:Vangelis Koukis % Done:

0%

Category:Ganeti Spent time: -
Target version:v0.6

Description

The reconciliation mechanism must be overhauled.
Currently it seems to fail:

vkoukis@dev67:~/synnefo [master]$ ./manage.py reconcile --interval=100
Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_manager(settings)
  File "/usr/lib/pymodules/python2.6/django/core/management/__init__.py", line 438, in execute_manager
    utility.execute()
  File "/usr/lib/pymodules/python2.6/django/core/management/__init__.py", line 379, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 191, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/pymodules/python2.6/django/core/management/base.py", line 220, in execute
    output = self.handle(*args, **options)
  File "/home/vkoukis/synnefo/logic/management/commands/reconcile.py", line 70, in handle
    vm_ids = map(lambda x: x.id, not_updated[:to_update])
  File "/usr/lib/pymodules/python2.6/django/db/models/query.py", line 180, in __getitem__
    stop = int(k.stop)
ValueError: invalid literal for int() with base 10: ''

Moreover, it needs to account for:
  • orphan VMs, snf-* in Ganeti, which do not exist in the DB or are marked as deleted
    • ...potentially with an option to kill them, instead of just displaying a big fat warning for them
  • VMs which exist in the database but no longer exist on the Ganeti side
    • ...they should be marked as deleted, somehow the OP_INSTANCE_REMOVE success message was lost for them.

Associated revisions

Revision 9fea53cc
Added by Vangelis Koukis almost 11 years ago

Overhaul reconciliation mamagement command

Detect the following problems: * Detect stale DB servers without corresponding Ganeti instances * Detect orphan Ganeti instances, without corresponding DB entries * Detect out-of-sync operstate for DB entries wrt to Ganeti instances
Fix them as follows: * Simulate server deletion for stale DB servers * Issue RAPI DeleteInstance() call for orphan Ganeti instances * Set operstate in DB as in Ganeti for out-of-sync DB servers

See ./manage.py reconcile --help for more info, refs #1021.

History

#1 Updated by Vangelis Koukis almost 11 years ago

  • Target version changed from v0.5.5 to v0.6

#2 Updated by Vangelis Koukis almost 11 years ago

  • Status changed from New to Closed

New version of reconcile management command pushed to master, closing ticket will re-open if problems arise.

Also available in: Atom PDF