Slow or stalling downloads inside Docker container

Incident Report for Travis CI

Resolved

This incident has been resolved. We are thanking you all for hanging in there with us. Happy building!
Posted May 07, 2019 - 19:28 UTC

Monitoring

A fix has been implemented and deployed to production. We are monitoring the results.
Posted May 07, 2019 - 16:32 UTC

Identified

The issue has been identified and a fix is being implemented.
Posted May 07, 2019 - 15:17 UTC

Update

We are continuing to investigate this issue.
Posted May 07, 2019 - 14:55 UTC

Investigating

Our test cases have been failing again so we are retracting what we said in the previous update. Sorry for the inconvenience.
Posted May 07, 2019 - 03:12 UTC

Monitoring

We have proceeded with replacing affected NAT instances with new ones in production and network operations from within Docker containers seem to be back to normal. We'll be monitoring things to ensure everything is stable. We are again thanking your for your enduring patience.
Posted May 07, 2019 - 02:45 UTC

Identified

We’ve been able to reproduce and then fix this issue on our staging environment by re-creating some misbehaving NAT instances. We are devising a way to apply the same to our production environment without disruption.
Posted May 06, 2019 - 20:10 UTC

Update

Adding `--network=host` to your `docker` commands should help you get past the current issue e.g.

docker build --network=host [...]

or

docker run --network=host [...]

We are still looking at fixing the problem at the product level.

Thank you for your continued patience.
Posted May 06, 2019 - 18:01 UTC

Update

In the meantime, a workaround that's worth trying to stabilize your builds and prevent the need to restart them manually, is to add `travis_retry` to your failing commands e.g.

travis_retry docker build [...]

This will effectively re-run your command up to 3 times if it fails.
Posted May 06, 2019 - 15:19 UTC

Investigating

We have received reports since last Friday about slow or stalling downloads when done from within a Docker container. We are opening this incident to give visibility about this issue and also post updates on our investigation. Thank you for your patience.
Posted May 06, 2019 - 14:28 UTC