The cluster is now operational following the implementation of a fix at 17:13 UTC. Our team is actively monitoring the results. The primary contributing factor was identified as a DNS-related issue. In order to determine the root cause, a thorough investigation will be conducted and a detailed post-mortem will be published.
Posted Nov 14, 2023 - 17:36 UTC
Update
We are continuing our investigation into the ongoing issue affecting EU cluster. Our team is actively working to mitigate the problem. While the cluster is slowly recovering, it has not yet reached a stable state.
Posted Nov 14, 2023 - 16:16 UTC
Investigating
We are currently experiencing an outage affecting API, WebSocket, and Webhook delivery services. Our team is actively investigating the issue
Posted Nov 14, 2023 - 15:27 UTC
Monitoring
All systems are now operational. However, clients utilising fallback transports may still experience intermittent issues. We are actively working to address the remaining issues.
Posted Nov 14, 2023 - 14:31 UTC
Update
Webhook delivery was temporarily affected, resulting in delays for some customers. Delivery is returning to normal.
Posted Nov 14, 2023 - 14:07 UTC
Update
We saw 3 short bursts of errors related to service restarts with limited customer impact in the last 2 hours. We are continuing to investigate the issue.
Posted Nov 14, 2023 - 13:43 UTC
Investigating
We are currently investigating this issue.
Posted Nov 14, 2023 - 12:55 UTC
This incident affected: Channels REST API, Channels WebSocket client API, Channels presence channels, and Channels Webhooks.