We've caught up on backlog are processing jobs. We're monitoring things closely to ensure they remain stable
Posted Oct 07, 2016 - 04:27 UTC
Update
We have resumed running builds and are quickly ramping up capacity to catch up on all the delayed sudo-enabled jobs
Posted Oct 07, 2016 - 04:08 UTC
Update
We've determined that Instance type exhaustion in one of the zones we're using was resulting in failed boots and causing build delays. We're updating our configs to stop using that zone and should resume builds soon.
Posted Oct 07, 2016 - 04:04 UTC
Identified
We are engaging with Google Cloud support about why all of our VM requests are failing. All sudo-enabled builds are stopped until we can dig into this more.
Posted Oct 07, 2016 - 03:38 UTC
Investigating
We are seeing issues with booting VMs on GCE and looking into why