migrate wazo-call-logd database to its own DB

Currently wazo-call-logd has its own database with only "call_logd_recording" table
To keep consistency, "call_log" and "call_log_participant" should be migrated.

The only remaining table on the "main" database should be "cel" and "queue_log" (which are not owned by wazo-call-logd, but by asterisk process)

Propositions

1. do not migrate database and write an upgrade notes to re-generate all call-logs

2. On upgrade, post-stop.d, add script to dump database
   Then add wazo-call-logd migration plugin to import data

   • pros:

     • easier to maintain plugin inside wazo-call-logd repo than wazo-upgrade

   • cons:

     • We will need to support all tables version from 19.13 to current

     • Plugin will need to be maintained until bullseye (new debian version)

3. On upgrade, pre-start.d, execute wazo-call-logd migration plugin (script entrypoint) to migrate data. Finally, drop table in the plugin

   • pros:

     • easier to migrate

   • cons

     • Plugin will need to be maintained until bullseye (new debian version)

4. On post-stop.d, set manage-db-skip-db variable, then on pre-start.d, migrate database to 21.02, dump DB, then unset manage-db-skip-db and run alembic HEAD

   • pros:

     • easier to migrate

     • can be mitigate by doing this only on < 20.14

   • cons:

     • can be really risky doing this on all upgrade

Solution (#3)

Workflow:

On upgrade, pre-start.d (to be sure no call is processed),

• add script to start call-logd and wait for it

• call wazo-call-logd-migration-db command

• output:

  • return success if everything is migrated

  • return warning if too much data to process

  • return error if an error occurs

TODO

• wazo-upgrade pre-start script

• wazo-call-logd script entrypoint

  • Allow to be executed multiple time

  • integration tests

  • drop table when everything is migrated

• manual test with big database (xivo-load)

• determine what is “too much” data to migrate

Note

On xivo-load (with 800 000 to 1 000 000 entries for 3 tables (stat_call_on_queue, call_log, call_log_participant)):

• pg_dump of all call-logd tables: ~ 35secondes

• pg_restore (on my dev): ~ 3 minutes