Travis CI
Monitoring - If you restart jobs that completed before 18:00 UTC, you will continue to see the previous job's output or inconsistent logs displayed. We are working on a resolution for this but it may be a few hours before this is done.
Mar 29, 20:01 UTC
Update - We are resuming build processing. New build logs should appear, but recent build logs may be unavailable until further notice.

We also expect some delay in build processing while we catch up with the back logs.
Mar 29, 19:50 UTC
Update - We are continuing our database maintenance.
Mar 29, 19:23 UTC
Update - We are performing emergency database maintenance. We hope to resume processing build requests shortly.
Mar 29, 18:52 UTC
Update - We are stopping accepting builds while we perform emergency maintenance.
Mar 29, 18:22 UTC
Identified - Logs database is having problems with writing logs. We are working with our service provider to resolve this issue.
Mar 29, 18:06 UTC
Investigating - We are investigating an .org logs database problem. Logs are unavailable at this time.
Mar 29, 18:00 UTC
API Operational
Web App Operational
Builds Processing Degraded Performance
Mac Builds ? Degraded Performance
Linux Builds (container-based) ? Degraded Performance
Linux Builds (Trusty and Precise on GCE) ? Degraded Performance
Background Processing Operational
Log Processing Degraded Performance
User Sync ? Operational
Notifications ? Operational
Third Party Services Operational
GitHub Operational
DNSimple Name Servers Operational
Heroku Operational
Pusher Pusher REST API Operational
Pusher WebSocket client API Operational
Pusher Presence channels Operational
Pusher Webhooks Operational
AWS ec2-us-east-1 Operational
Help Scout Email Processing Operational
Help Scout Web App Operational
PagerDuty Notification Delivery Operational
Google Cloud Platform Google Compute Engine Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Scheduled Maintenance
On Sunday April 2nd, 2017, from 17:00 to 21:00 UTC, we will be performing network maintenance on our Mac infrastructure to improve and test the redundancy in our network stack. The maintenance will be performed in two stages, the first starting at 17:00 UTC and the second starting at 20:00 UTC. We're not expecting any user-visible impact apart from brief packet loss as we perform failover tests.
Posted on Mar 24, 09:30 UTC
System Metrics Month Week Day
API Uptime (api.travis-ci.org)
Fetching
API Uptime (api.travis-ci.com)
Fetching
Active Linux Builds for Open Source projects (container-based)
Fetching
Backlog Linux Builds for Open Source projects (container-based)
Fetching
Active Linux Builds for Open Source projects
Fetching
Backlog Linux Builds for Open Source projects
Fetching
Active OS X Builds for Open Source Projects
Fetching
Backlog OS X Builds for Open Source Projects
Fetching
Past Incidents
Mar 29, 2017
Resolved - Systems operating normally. Thanks for bearing with us!
Mar 29, 14:09 UTC
Monitoring - Maintenance has been completed and services appear to be stable. We will continue to monitor.
Mar 29, 13:50 UTC
Identified - We mitigated the issue and API error rates have recovered. We are continuing with maintenance to make sure the API is stable.
Mar 29, 13:26 UTC
Investigating - We are investigating reports of the higher rate of API errors for .org.
Mar 29, 13:04 UTC
Mar 28, 2017
Resolved - This issue has been resolved and logs are working as expected now. Thanks for your patience!
Mar 28, 11:45 UTC
Monitoring - We've deployed a fix for this issue and logs should be rendering now properly. We're currently monitoring. Please email support@travis-ci.com if you notice any log hiccups.
Mar 28, 11:03 UTC
Identified - We’ve identified an issue that is causing some logs for both public and private repositories to not render properly. We’re working on a fix for this.
Mar 28, 10:18 UTC
Investigating - We are investigating reports that logs are not always displayed in web.
Mar 28, 07:52 UTC
Mar 27, 2017
Completed - The scheduled maintenance has been completed.
Mar 27, 21:35 UTC
Verifying - We've completed the update of the API components on both travis-ci.org and travis-ci.com. We are continuing to monitor closely.
Mar 27, 20:52 UTC
Update - We've finished the update of the API components on travis-ci.org and we are monitoring things closely. We shall begin updating travis-ci.com shortly.
Mar 27, 19:43 UTC
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Mar 27, 18:30 UTC
Scheduled - We will be performing a maintenance on our API for more efficient access to our logs database. We will first update the API for travis-ci.org and then travis-ci.com with a least an hour in between for monitoring. We don't expect any visible downtime or user impact during the maintenance. Please contact support [at] travis-ci [dot] com if you see something.
Mar 27, 18:08 UTC
Resolved - Backlog for both open-source-builds and private-builds has now cleared. Thank you for your patience.
Mar 27, 08:26 UTC
Update - The backlog for our open-source-builds has cleared. The private-builds backlog continues to drop.
Mar 27, 08:22 UTC
Monitoring - We have identified the issue, applied a fix, and are monitoring as things recover.
Display of newer logs may be delayed until we process the backlog.
Mar 27, 08:02 UTC
Identified - We have identified the issue, applied a fix, and are monitoring as things recover.
Display of newer logs may be delayed until we process the backlog.
Mar 27, 08:02 UTC
Investigating - We’re investigating issues with logs not being displayed.
Mar 27, 07:40 UTC
Mar 26, 2017

No incidents reported.

Mar 25, 2017

No incidents reported.

Mar 24, 2017
Resolved - Webhook delivery delays have been resolved on GitHub. We have confirmed we're receiving build request events immediately.
Mar 24, 12:36 UTC
Identified - We are experiencing delayed build requests due to webhook delivery delays on GitHub (up to ~3min). We are monitoring the situation. Also see https://status.github.com.
Mar 24, 12:27 UTC
Mar 23, 2017
Resolved - We have confirmed the effectiveness of the short-term fix. We will continue investigating for a longer-term resolution. Thank you for your patience.
Mar 23, 15:32 UTC
Monitoring - We have identified the problematic component. While we continue investigating a long-term fix, we have applied a short-time one to have the logs load correctly.
Mar 23, 14:55 UTC
Investigating - We’re investigating issues displaying public logs while builds are running.
Mar 23, 14:13 UTC
Mar 22, 2017
Resolved - The issue was confirmed fixed by our infrastructure provider, and we're no longer seeing errors while booting instances. Jobs are running normally across all our infrastructures at this time.
Mar 22, 16:59 UTC
Identified - Our infrastructure provider has identified an issue and is working on a fix.
Mar 22, 16:29 UTC
Investigating - We're investigating elevated error rates while booting instances for sudo-enabled Linux builds, which is causing jobs to take longer to start.
Mar 22, 15:46 UTC
Mar 21, 2017

No incidents reported.

Mar 20, 2017
Resolved - Things have been reliable and stable, so we are resolving this issue.
Mar 20, 23:43 UTC
Monitoring - The unresponsive host has been removed from the set of DNS records. The site should be fully reachable again. We continue to monitor the situation.
Mar 20, 16:59 UTC
Identified - We are receiving reports of travis-ci.org being unreachable for some users. One of the 8 edge server IPs appears to be unreachable via TCP. We have escalated the issue to our upstream provider.
Mar 20, 16:29 UTC
Mar 19, 2017

No incidents reported.

Mar 18, 2017

No incidents reported.

Mar 17, 2017
Resolved - User reports of issues have cleared up as of approximately Noon PST. We apologize for the delays in resolving this issue. Please email support@travis-ci.com if you see any further issues.
Mar 17, 02:45 UTC
Update - We are seeing some reports of similar intermittent slowness from both nodejs.org and registry.yarnpkg.com from other users outside of Travis CI's infrastructure, which seems to indicate a potential upstream issue.
Restarting affected builds can help and this is recommended for the moment.
We are continuing to investigate.
Mar 16, 15:30 UTC
Investigating - We’re currently investigating slowness and connectivity issues on `sudo: false`, container-based builds. This seems to be specially affecting Node.js installations via nvm.
Mar 16, 14:20 UTC
Mar 16, 2017
Resolved - At this time things are now running at our current full capacity for public macOS builds. Please email support@travis-ci.com if you have any questions. Thank you for your patience while we worked through some underlying improvements that required reduced capacity today.
Mar 16, 02:40 UTC
Update - We are beginning to add capacity back for public macOS jobs and will provide an update when this process is finished.
Mar 16, 01:29 UTC
Identified - We are in the process of reducing capacity for public macOS builds, in order to support moving some of our hardware capacity for infrastructure improvements. We'll be at reduced capacity for a few hours and we'll provide periodic updates. Please email support@travis-ci.com if you have any questions.
Mar 15, 18:08 UTC
Mar 15, 2017
Resolved - Google has resolved the issue and we're seeing jobs executing as expected. Please email support@travis-ci.com if you continue to see any network issues. Thanks for your patience during this issue.
Mar 15, 22:32 UTC
Update - We're seeing job error rates start to drop, which indicates the issue may be improving, but we're waiting on updates from Google Cloud's status site, https://status.cloud.google.com/incident/compute/17006.
Mar 15, 21:21 UTC
Identified - Our cloud provider for sudo enabled builds, Google Compute Engine (GCE) is currently experiencing network connectivity issues and this affecting public and private builds. We're monitoring their status incident, https://status.cloud.google.com/incident/compute/17006, closely and will provide updates as we get them. During this incident you'll see failures connectivity to external resources like Launchpad PPAs, PyPI, RubyGems, etc.
Mar 15, 21:00 UTC
Resolved - The API is responding normally again.
Mar 15, 10:56 UTC
Investigating - We're investigating elevated error rates for the travis-ci.com API, which may also affect the web UI.
Mar 15, 10:31 UTC
Resolved - At this time we've restarted the errored jobs and they are in our backlog again. We are processing jobs at full capacity, with the usual backlog due to high demand for public macOS builds. We'll be publishing a postmortem of this incident by the end of the week. We thank you for your patience and understanding while we resolved this issue.
Mar 15, 02:45 UTC
Update - A mistake in the order we brought up some of the services resulted in all pending macOS jobs "erroring out" quickly. We are currently working on resetting the state of those jobs so that they'll be queued and run again. We are very sorry for this issue and will post an update when the jobs have been requeued.
Mar 15, 01:52 UTC
Update - We are beginning to ramp up the capacity and are monitoring things closely.
Mar 15, 01:36 UTC
Update - We've been able to restore the backplane to service and we're working on verification and preparing to resume builds and ramp back up to full capacity.
Mar 15, 01:10 UTC
Update - The backplane has come up in an unexpected state and we're escalating with our infrastructure provider, as we'll need their help in resolving this issue.

In the mean time we continue to run public macOS builds with degraded capacity. travis-ci.com jobs are not affected at this time.

Thank you for your patience while we work to resolve this.
Mar 15, 00:47 UTC
Identified - The issue has been identified and a fix is being implemented.
Mar 15, 00:23 UTC
Update - The control plane restart is still in progress, we discovered that the root filesystem partition had filled up and we weren't alerted to this issue. We've cleaned up the filesystem and are still working to get the backplane services started up again. In the mean time we continue to run public macOS builds with degraded capacity. travis-ci.com jobs are not affected at this time.
Mar 15, 00:22 UTC
Update - The "control backplane" for part of our virtualization infrastructure is unstable, so we're initiating a restart of the backplane. In the mean time we continue to run public macOS builds with degraded capacity. travis-ci.com jobs are not affected at this time.
Mar 14, 23:43 UTC
Update - We are investigating some intermittent stability errors with some of our physical servers for this infrastructure and we are working to restore stability to this. At this time we're running public macOS builds with degraded capacity.
Mar 14, 23:27 UTC
Investigating - We are seeing high amounts of errors when trying to start public macOS jobs. This is causing build delays for travis-ci.org macOS builds and we are investigating why.
Mar 14, 22:59 UTC