ccaas-status - Undelivered agent messages for select customer using Cisco connector – Incident details

All systems operational

Undelivered agent messages for select customer using Cisco connector

Resolved
Degraded performance
Started over 1 year agoLasted about 9 hours

Affected

Graia Contact Center as a Service

Degraded performance from 8:00 AM to 4:30 PM

CCaaS Custom Connectors to Cisco

Degraded performance from 8:00 AM to 4:30 PM

Updates
  • Update
    UTC
    Update

    Root Cause Due to the doubling of the underlying resources (all app service connectors were duplicated between the two instances), chat conversations were distributed between the two logical servers. A conversation existed on one server, meaning that when that conversation was referenced on the other server, it could not be found. For more details see the RCA document titled "Buzzeasy Root Cause Analysis Report - 2025-01-25 United Group projects, undelivered agent messages.docx"

  • Investigating
    UTC
    Investigating

    Due to a misconfiguration (human error), the scale-out duplicated the connectors between the two instances instead of distributing them. Troubleshooting took unusually long because initial tests were made with short conversations that did not provide opportunities for the issue to manifest.