apt-get update failing on GCE builds (Precise and Trusty)

Incident Report for Travis CI

Resolved

This incident has been resolved.
Posted Dec 14, 2015 - 18:40 UTC

Update

We have shipped a workaround to overwrite the APT sources. Jobs with this workaround will include a fold named "gce.apt_hack" immediately following the "worker_info" fold. Implementation details at https://github.com/travis-ci/travis-build/pull/567/files.
Posted Dec 14, 2015 - 16:25 UTC

Identified

The primary Ubuntu mirror for the GCE us-central1 region has been returning 403s, which is causing apt-get update to fail. We have a workaround in https://github.com/travis-ci/travis-ci/issues/5285#issuecomment-164464865 and are working to apply a run time fix to this, while also reaching out to Canonical about the mirror issue.
Posted Dec 14, 2015 - 15:25 UTC