The backlog has been processed and the service is back to normal. We are still investigating the root cause of the issue.
Posted Jul 01, 2020 - 18:08 EEST
Investigating
We are seeing a large build-up of pending transcoder jobs on the queue. It appears that many workers are still stalled on uploads to the master storage due to the previous network disturbances and at the same time we are not getting enough new worker instances to cope with the backlog. We are currently investigating why the system doesn't scale.