Notice history

Dec 2024

Resolved
December 17, 2024 at 6:00 PM
Resolved
December 17, 2024 at 6:00 PM
On 17/12/2024 we had a performance issue with our live data processor service that was unable to handle all system messages. This did not affect customer interactions (phone calls, emails, webchat sessions with customers were all delivered to agents), but supervisor reports on Portal Teams report might have shown outdated Workitems (Tasks) on agents' cards or Agent UI showed Agent Performance Dashboard with outdated data. After the internal alerting mechanism triggered an alarm, the wider team got involved and we managed to rectify the fact the there were wrongfully lingering agent tasks on the portal's Teams reports.
Investigating
December 17, 2024 at 1:20 PM
Investigating
December 17, 2024 at 1:20 PM
On 17/12/2024 we had a performance issue with our live data processor service that was unable to handle all system messages. This did not affect customer interactions (phone calls, emails, webchat sessions with customers were all delivered to agents), but supervisor reports on Portal Teams report might have shown outdated Workitems (Tasks) on agents' cards or Agent UI showed Agent Performance Dashboard with outdated data. After the internal alerting mechanism triggered an alarm, the wider team got involved and we managed to rectify the fact the there were wrongfully lingering agent tasks on the portal's Teams reports.

Buzzeasy Security Patching

In progress
December 14, 2024 at 7:59 AM
In progress
December 14, 2024 at 7:59 AM
TThe maintenance has been completed
Planned
December 14, 2024 at 6:00 AM
Planned
December 14, 2024 at 6:00 AM
Dear Buzzeasy Customer, We would like to inform you that on 14-12-2024 there will be a scheduled maintenance window for applying security patches on the Buzzeasy Voice components. Buzzeasy International: 14-12-2024 | 06:00 UTC | 2 h During this time all Buzzeasy components will be operational, but we do anticipate a short voice service interruption during key moments of the update. Thank you for your patience. If you have any questions, please contact us via the known communication channels. Kind Regards, Buzzeasy Team

Nov 2024

Buzzeasy Security Patching

In progress
November 30, 2024 at 7:12 AM
In progress
November 30, 2024 at 7:12 AM
TThe maintenance has been completed
Planned
November 30, 2024 at 6:00 AM
Planned
November 30, 2024 at 6:00 AM
Dear Buzzeasy Customer, We would like to inform you that on 30-11-2024 there will be a scheduled maintenance window for applying security patches on the Buzzeasy Voice components. Buzzeasy International: 30-11-2024 | 06:00 UTC | 2 h During this time all Buzzeasy components will be operational, but we do anticipate a short voice service interruption during a key moment of the update. Thank you for your patience. If you have any questions, please contact us via the known communication channels. Kind Regards, Buzzeasy Team

Buzzeasy Security Patching

In progress
November 09, 2024 at 7:43 AM
In progress
November 09, 2024 at 7:43 AM
TThe maintenance has been completed
Planned
November 09, 2024 at 6:00 AM
Planned
November 09, 2024 at 6:00 AM
Dear Buzzeasy Customer, We would like to inform you that on 09-11-2024 there will be a scheduled maintenance window for applying security patches on the Buzzeasy Voice components. Buzzeasy International: 09-11-2024 | 06:00 UTC | 2 h During this time all Buzzeasy components will be operational, but we do anticipate a short voice service interruption during key moments of the update. Thank you for your patience. If you have any questions, please contact us via the known communication channels. Kind Regards, Buzzeasy Team

Buzzeasy Europe Voice Maintenance

In progress
November 06, 2024 at 3:44 AM
In progress
November 06, 2024 at 3:44 AM
TThe maintenance has been completed
Planned
November 06, 2024 at 3:00 AM
Planned
November 06, 2024 at 3:00 AM
Dear Buzzeasy Customer, We would like to inform you that we have scheduled an urgent maintenance window for 06-11-2024 at 03:00 UTC to perform quality improving maintenance on part of our EU voice systems. We estimate the work to last approximately 1 hour. During this time we anticipate voice service interruption during key phases of the work. Thank you for your patience. If you have any questions, please contact us via the known communication channels. Kind Regards, Buzzeasy Team

Investigating
November 27, 2024 at 10:40 AM
Investigating
November 27, 2024 at 10:40 AM
Relevant DevOps workitem: Incident 119765: Bad call quality on EXT2-3 ENV
Resolved
November 05, 2024 at 1:25 PM
Resolved
November 05, 2024 at 1:25 PM
The incident has been resolved
Investigating
November 05, 2024 at 8:45 AM
Investigating
November 05, 2024 at 8:45 AM
We are observing call quality degradation on one of our voice servers. We are investigating the problem and will come back with updates soon.

Update
November 27, 2024 at 10:40 AM
Update
November 27, 2024 at 10:40 AM
Relevant DevOps workitem: Incident 119765: Bad call quality on EXT2-3 ENV
Investigating
November 04, 2024 at 10:00 AM
Investigating
November 04, 2024 at 10:00 AM
Today, between approx. 10:00 - 12:30 UTC some of our customers observed degraded call quality.

Oct 2024

Buzzeasy notification sender email address change

Planned
October 18, 2024 at 8:00 AM
Planned
October 18, 2024 at 8:00 AM
Dear Buzzeasy Customer, We would like to inform you that as of tomorrow, 18-10-2024, the sender email address from which you will receive these notifications about our product releases and maintenance windows will change: from the currently used serviceuser@geomant.com to the new serviceuser@buzzeasy.com Thank you for your understanding. Kind Regards, Buzzeasy Team

Update
October 18, 2024 at 8:38 AM
Update
October 18, 2024 at 8:38 AM
The outage was caused by Azure App Service resource health problems. The custom solution named Cisco Connector is hosted on Azure App Service resources which was having problems with a backend component. The following is from the Azure-supplied report about the incident: Title: PIR - App Service intermittent availability issues in West Europe Tracking ID: 5NXS-NR8 Event type: Service Issue Status: Resolved Service(s): App Service Region(s): West Europe Start time: 2024-10-17T08:30:00Z Resolve time: 2024-10-17T15:15:00Z Last update time: 2024-10-17T15:15:00Z Impacted subscriptions: dcffb1fb-e926-4e1e-bbdb-1894128838b3 (Buzzeasy - Production) What happened? Between 08:30 UTC and 15:15 UTC on 17 October 2024, a platform issue resulted in an impact to App Service in the West Europe region. Impacted customers may have experienced intermittent HTTP 500- level response codes, timeouts or high latency when trying to access apps hosted in the region. What went wrong and why? We determined that this issue had been caused by a backend component App Service operations rely on reaching an operational threshold, which manifested in the impact described above. How did we respond? 08:05 UTC on 17 October 2024 - Service monitoring detected unhealthy resources. 08:30 UTC on 17 October 2024 – Customer impact began. 11:45 UTC on 17 October 2024 – We determined this issue was caused by a backend component reaching an operational threshold. 15:15 UTC on 17 October 2024 - The Azure Platform self-healed the issue after monitoring efforts.
Investigating
October 17, 2024 at 1:10 PM
Investigating
October 17, 2024 at 1:10 PM
Select customers are experiencing failures with their custom Buzzeasy chatbot solution. Specifically, transfers of the chats to live agents using Cisco agent application are failing.

Buzzeasy Release Deployment - Release 2024-10-12

In progress
October 12, 2024 at 7:01 AM
In progress
October 12, 2024 at 7:01 AM
TThe maintenance has been completed
Planned
October 12, 2024 at 5:00 AM
Planned
October 12, 2024 at 5:00 AM
Dear Buzzeasy Customer, We would like to inform you that on 2024-10-12 there will be a scheduled maintenance window for deploying the new Buzzeasy Release. Buzzeasy International: 2024-10-12 | starting at 05:00 UTC | duration 2 h During this time all Buzzeasy components will be operational, but we do anticipate a short service interruption during key moments of the deployment. After the release, please refresh the agent login page using Ctrl+F5 to benefit immediately from the latest features and fixes. Thank you for your patience. If you have any questions, please contact us via the known communication channels. Kind Regards, Buzzeasy Team

Update
October 10, 2024 at 1:43 PM
Update
October 10, 2024 at 1:43 PM
Relevant DevOps workitem: Incident 118058: web-chat-connector-deployment restart cycle
Update
October 10, 2024 at 1:38 PM
Update
October 10, 2024 at 1:38 PM
According to our findings, the intermittent outage of the Buzzeasy webchat service on 2024-10-09 was due to increased traffic flowing through our webchat-connector microservice. The service had to process enough data to the point that the built-in monitoring system's liveness probe's health checking requests did not get a reply in time. As a result, the service was considered non-responsive which triggered automatic restarts, making the webchat widget temporarily unavailable for web pages that were loaded while a restart was in progress. We addressed the problem by on the one hand, increasing the maximum used memory limit of the service and on the other hand, allowing more time for the liveness probe's requests to get a response before restarts are triggered. This will prevent the service from being restarted prematurely while there is no real fault with the resource but rather there would just be some additional time needed for it to process all requests in its buffer. We will be continuing to monitor closely this system in following period. Furthermore, we will be planning to implement additional redundancies and resources between which the service can load balance its traffic in future.
Investigating
October 09, 2024 at 11:00 AM
Investigating
October 09, 2024 at 11:00 AM
Customers using the Buzzeasy webchat widget reported that the widget is intermittently not loading into the webpage where it's supposed to be embedded.

Buzzeasy Scripting Domain Change

Planned
October 01, 2024 at 9:00 AM
Planned
October 01, 2024 at 9:00 AM
Dear Buzzeasy Customer, This is an announcement to let you know that on the 1st of October 2024 we will be moving our Scripting service to a new domain from geomant.cloud to buzzeasy.com. The service itself will continue working as it has until now, in fact our internal links (from portal.buzzeasy.com to the scripting service) are already using the new domain. There is no action required of you unless you have bookmarked the Script Editor (a.k.a. Script Designer) web page. In this case, please update your bookmark from https://script-designer-prod.geomant.cloud to https://script-designer.buzzeasy.com Kind Regards, Buzzeasy Team

ccaas-status - Notice history

All systems operational

Dec 2024

Nov 2024

Oct 2024