We managed to process all the remaining queued builds for public repositories on our Mac infrastructure. Thank you for your enduring patience.
Posted Jun 23, 2016 - 04:04 UTC
Update
Just about all of the closed source jobs which were in the job backlog have processed. Open source jobs are still backed up, but processing normally.
Posted Jun 22, 2016 - 07:21 UTC
Monitoring
We've restarted all of our workers and the queue is draining as it is expected to do so. We're monitoring things closely as jobs process.
Posted Jun 22, 2016 - 05:55 UTC
Update
All the stale VMs have cleared! We'll be restarting workers momentarily.
Posted Jun 22, 2016 - 05:29 UTC
Update
We've coordinated with our infrastructure provider to restart our vCenter instance, which had become unresponsive, and it has come back up. We're actively working to continue cleaning up the stale VMs.
Posted Jun 22, 2016 - 04:43 UTC
Update
We're still working on cleaning up the stale VMs. At this time we have stopped all Mac workers in order to help the cleanup go faster.
Posted Jun 22, 2016 - 03:22 UTC
Identified
We're working on cleaning up some stale VMs on our Mac infrastructure, you may experience increased boot times for the time being.