At this time we've caught up on all backlog and the changes we deployed seem to be holding up well. Thanks for your patience while we worked through this issue.
Posted Nov 25, 2015 - 18:21 UTC
Monitoring
We've deployed some changes to the service in question, to help reduce database contention and are monitoring things closely.
Posted Nov 25, 2015 - 17:57 UTC
Identified
We've cleared the containerized build backlog. We've identified some database queries which are causing intermittent slow down in the job status change processing and we are working on improving them.
Posted Nov 25, 2015 - 16:54 UTC
Investigating
The service which processes build job status changes, which drive the elastic capacity of our containerized builds, is having some performance issues. This is leading to delays for public containerized builds. We are investigating why and manually adding additional capacity for containerized builds.