Uploaded image for project: 'Wazo Platform'
  1. WAZO-1905

race condition cause traceback during wazo-chatd initialization

    Details

    • Type: Bug
    • Status: To Do (View workflow)
    • Priority: Medium
    • Resolution: Unresolved
    • Affects versions: None
    • Fix versions: None
    • Labels:
    • Sprint:

      Description

      Given I have a very long initialization process of wazo-chatd
      When I have an DeviceStateChange event received during init
      Then I can have some weird traceback in log:

      2020-08-06 16:08:45,792 [11943] (ERROR) (xivo.pubsub): UPDATE statement on table 'chatd_endpoint' expected to update 1 row(s); 0 were matched.
        File "/usr/lib/python3/dist-packages/xivo/pubsub.py", line 39, in publish_one
          callback(message)
        File "/usr/lib/python3/dist-packages/wazo_chatd/plugins/presences/bus_consume.py", line 219, in _device_state_change
          self._dao.endpoint.update(endpoint)
        File "/usr/lib/python3/dist-packages/wazo_chatd/database/queries/endpoint.py", line 46, in update
          self.session.flush()
        File "/usr/lib/python3/dist-packages/sqlalchemy/orm/session.py", line 2446, in flush
          self._flush(objects)
        File "/usr/lib/python3/dist-packages/sqlalchemy/orm/session.py", line 2584, in _flush
          transaction.rollback(_capture_exception=True)
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 67, in __exit__
          compat.reraise(exc_type, exc_value, exc_tb)
        File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 277, in reraise
          raise value
        File "/usr/lib/python3/dist-packages/sqlalchemy/orm/session.py", line 2544, in _flush
          flush_context.execute()
        File "/usr/lib/python3/dist-packages/sqlalchemy/orm/unitofwork.py", line 416, in execute
          rec.execute(self)
        File "/usr/lib/python3/dist-packages/sqlalchemy/orm/unitofwork.py", line 583, in execute
          uow,
        File "/usr/lib/python3/dist-packages/sqlalchemy/orm/persistence.py", line 236, in save_obj
          update,
        File "/usr/lib/python3/dist-packages/sqlalchemy/orm/persistence.py", line 1008, in _emit_update_statements
          % (table.description, len(records), rows)
      sqlalchemy.orm.exc.StaleDataError: UPDATE statement on table 'chatd_endpoint' expected to update 1 row(s); 0 were matched.

      Expected: no traceback
      Workaround:

      • nothing (because state will be updated on the next event)
      • restart wazo-chatd

      Real scenario: It can happen when you restart your server and the phone change their state during wazo-chatd start and the init takes many seconds

      Ideas:
      Add a lock during init process to avoid to process events. But process this event after init process IF the event is received between the amid response of all devices state AND the end of the init. It will minimize error of state after init

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Fran├žois Blackburn
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Zendesk Support