Sporadic call logs in Wazo Platform 23.09

Description

Given a large volume of call logs (e.g. 16M calls)
Given I place a new call, answer and hangup
Given I wait for a few seconds
When I list the call logs (GET /api/call-logd/cdr)
Then I don’t see the new call in the list

Expected: I see the new call log in the list

This is a regression in Wazo Platform 23.09.

Workaround:

Technical details:

  • SQL operations in wazo-call-logd take much longer in 23.09

  • This causes timeouts between wazo-call-logd and RabbitMQ

  • When the timeout occurs, all previous call logs waiting for processing are abandoned

Zendesk Ticket IDs

None

Activity

Show:

Sébastien Duthil July 24, 2023 at 4:00 PM

This ticket was hotfixed in Wazo Platform 23.10.

Charles Langlois July 20, 2023 at 3:22 PM

to validate fix(sql index) and avoid similar future regression, is being worked on.

Pascal Crausaz July 20, 2023 at 2:37 PM

From: Sébastien Duthil

@Pascal Crausaz we just found a regression in the stack 23.09 (i.e. the already released version, not the one being prepared 23.10) by the support team:

  • Jira ticket: 

  • Spread: stacks with a "large" volume of calls (e.g. 4K calls per day) in 23.09 (already released)

  • Visible impact: call logs are not generated, users can't see the calls they placed or received since the upgrade to 23.09

  • Workaround: Relatively easy to install

  • Amount of work to fix: around 1 day: we already have a probably decent fix (used for workaround), but it needs more testing to evaluate the exact performance gain.

We need a decision between:

  • block release of 23.10 until it's fixed

  • proceed with release of 23.10 and fix it in 23.11

Done

Details

Priority

Assignee

Reporter

Fix versions

Sprint

Zendesk Support

2

Created July 20, 2023 at 10:09 AM
Updated August 9, 2023 at 3:15 PM
Resolved July 24, 2023 at 3:59 PM