Summary
On Saturday 24 January 2015 at approximately 11:30 PM UTC, an issue in one of Microsoft's datacenters caused some customers in the North American region to experience slowness when accessing CRM Online. The problem was reported by one customer. Microsoft engineers took action to mitigate the situation. The issue was successfully fixed on Sunday 25 January 2015 at 02:35 AM UTC.
Customer Impact
Impacted customers experienced organization load times of over one minute. Only one customer escalated the issue, but other customers sharing the same front end hardware may have experienced a similar performance issue. Less than 10% of customers in the region were impacted.
Incident Start Date and Time
Saturday 24 January 2015 11:30 PM UTC
Date and Time Service was Restored
Sunday 25 January 2015 02:35 AM UTC
Root Cause
A definitive root cause has not yet been determined, but further investigation is still ongoing. Analysis has indicated that the web applications serving a set of customers experienced a spike in SDK requests originating from a few customers in particular. It would not be expected that requests from one or two customers would impact performance of others more broadly, so further investigation and analysis is underway.
Next Step(s)
Issue | Next Step | Team Owner | Timeline |
Root Cause Analysis | Further analysis of data is underway to look for correlation of SDK request spike by one or two customers on the broader system, so that system design changes can be assessed and implemented as appropriate. | Microsoft Dynamics CRM Online Service Engineering | Ongoing |
Monitoring | Assess current monitoring to identify if there are metrics that correlated to this issue; and determine if monitoring or alerting changes should be made. | Microsoft Dynamics CRM Online Service Engineering | Ongoing |