Graia team members observed recovered functionality.
Update · UTC
The existing pods that were on the unreachable node were moved away from it.
Update · UTC
The node pool was expanded with an additional node to take over the workload of the unresponsive one.
Identified · UTC
The SRE team discovered the fact that a node is unresponsive by noticing the unusual metrics it is exhibiting.
Investigating · UTC
The problem manifests to users of the portal web administration interface by it failing to load beyond the initial blank screen or to load page-specific content when navigating between menus. Contact center agents also see an error message such as “Failed to get voice configuration”, are unable to receive or initiate calls or are faling to load the web application entirely.