We've finally cleared the remaining backlog of Mac builds hence, we are resolving the incident. Thank you for your patience!
Posted Feb 16, 2018 - 03:19 UTC
Update
At this point the logs are mostly cleared up, and we are now just clearing up remaining backlogs.
Posted Feb 16, 2018 - 01:04 UTC
Update
The backlog has been cleared for Linux, and macOS is making steady progress. Logs are still being processed at this time, so you may not be able to view them right away.
Posted Feb 15, 2018 - 22:06 UTC
Update
The 'sudo: required' backlog has now been cleared, and we are making progress on clearing the remaining backlogs for the Trusty and macOS backlog.
Posted Feb 15, 2018 - 20:38 UTC
Identified
We have proceeded to increase the resources of our RabbitMQ to help us cope with the current incident. The upgrade has improved things on the logs processing front and we are currently going through the accumulated backlog.
Posted Feb 15, 2018 - 20:09 UTC
Investigating
We are continuing to investigate this incident which has unfortunately moved to other components. As a result, we are currently seeing a backlog of public/open source builds on all our infrastructures. Apologies for the inconveniences.
Posted Feb 15, 2018 - 19:31 UTC
Identified
We have identified the cause of this issue, and we're working on resolving it.
Posted Feb 15, 2018 - 18:15 UTC
Investigating
We are currently seeing issues with log delivery. The same issue is also causing a backlog of macOS builds. We are working on fully identifying and resolving these issues.