High Response Times
Incident Report for NetDocuments DE
Postmortem

On 14 April at approximately 8:48 am CEST, NetDocuments experienced a disruption with the DE Service.   During the event the Service would have appeared slow in response but remained reachable.

We were alerted to the start of the issue via our automated monitoring infrastructure and were able to begin investigating immediately.  By approximately 11:20 am CEST the Service had been stabilized and operations had returned to normal.

An investigation found the root cause was related to a software defect in the recently deployed 23.7 version of the Service affecting the retrieval of cached information.   Once the cause was identified, steps were taken to roll the Service back to the previous version of 23.6 in order to stabilize the Service and allow for the defect to be corrected before the new version is deployed again.

As a part of ongoing and continuous improvement, NetDocuments is evaluating and adding additional test cases to enhance the Beta and Deployment processes to prevent a similar defect from being able to affect the Service in this way again.

We apologize for any inconvenience this disruption may have caused.

Thank you,

The NetDocuments Team

Posted Apr 18, 2023 - 01:17 CEST

Resolved
This incident is now resolved.

If you are experiencing difficulty, please contact support via the portal or by phone.
https://support.netdocuments.com/

AU customers call +61 2 8310 4319
EU customers call +44 2034.556770
US customers call 801-226-6882 or 866-NETDOCS (638-3627)
Posted Apr 14, 2023 - 11:54 CEST
Monitoring
Our engineering team has implemented a fix that was causing high-response times. We will continue to monitor this closely.

If you are experiencing difficulty, please contact support via the portal or by phone.
https://support.netdocuments.com/

AU customers call +61 2 8310 4319
EU customers call +44 2034.556770
US customers call 801-226-6882 or 866-NETDOCS (638-3627)
Posted Apr 14, 2023 - 10:50 CEST
Identified
Our engineering team has identified an issue causing high-response times. We are currently doing work to help mitigate the issue. We will provide a further update in 15 mins.
Posted Apr 14, 2023 - 10:23 CEST
This incident affected: Platform.