Resolved: Service Degradation from Network Congestion
As of 8 p.m., Thursday, October 16, the issue has been resolved. A solution was implemented to alleviate the degradations, which has since been verified by the vendor.
A software change was made to the back-end servers of one of the web servers in response to the initial incident on Tuesday, October 14. Changes made during Wednesdayâ€™s Maintenance Window (5-7 a.m.) allowed for the network disruptions as a result.
Specifically, an errant interaction between the load balancer and back-end server in response to a specific network request triggered the service degradations.
The network setting configuration has been changed on the back-end servers to resolve the issue.
Staff will continue to monitor the issue, ensuring that the solution continues to prevent future occurrences.