Get email notifications whenever GitHub creates, updates or resolves an incident.
Get text message notifications whenever GitHub creates or resolves an incident.
Get incident updates and maintenance status messages in Slack.
Subscribe via Slack
By subscribing you acknowledge our Privacy Policy. In addition, you agree to the Atlassian Cloud Terms of Service and acknowledge Atlassian's Privacy Policy.
Get webhook notifications whenever GitHub creates an incident, updates an incident, resolves an incident or changes a component status.
Visit our support site.
All Systems Operational
Git Operations
?
Operational
Webhooks
?
Operational
Visit www.githubstatus.com for more information
Operational
API Requests
?
Operational
Issues
?
Operational
Pull Requests
?
Operational
Actions
?
Operational
Packages
?
Operational
Pages
?
Operational
Codespaces
?
Operational
Copilot
Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Aug 22, 2024
Resolved -
This incident has been resolved.
Aug 22, 17:28 UTC
Aug 22, 17:28 UTC
Update -
We are investigating issues with failed workflow runs due to internal errors. We are seeing signs of recovery and continuing to monitor the situation.
Aug 22, 17:21 UTC
Aug 22, 17:21 UTC
Investigating -
We are investigating reports of degraded performance for Actions
Aug 22, 16:49 UTC
Aug 22, 16:49 UTC
Aug 21, 2024
Resolved -
This incident has been resolved.
Aug 21, 15:11 UTC
Aug 21, 15:11 UTC
Update -
We have seen recovery and Actions workflow runs are now running as expected.
Aug 21, 15:10 UTC
Aug 21, 15:10 UTC
Update -
We are seeing reduced delays for Actions workflow runs to get triggered. We are continuing to investigate how to further reduce impact on customers and recover more quickly.
Aug 21, 14:51 UTC
Aug 21, 14:51 UTC
Update -
We are investigating reports of users seeing delays for Actions workflow runs to get triggered.
Aug 21, 14:15 UTC
Aug 21, 14:15 UTC
Investigating -
We are investigating reports of degraded performance for Actions
Aug 21, 14:09 UTC
Aug 21, 14:09 UTC
Aug 20, 2024
No incidents reported.
Aug 19, 2024
No incidents reported.
Aug 18, 2024
No incidents reported.
Aug 17, 2024
No incidents reported.
Aug 16, 2024
No incidents reported.
Aug 15, 2024
Resolved -
On August 15, 2024, between 13:14 UTC and 13:43 UTC, the Actions service was degraded and resulted in failures to start new workflow runs for customers of github.com. On average, 10% of Actions workflow runs failed to start with the failure rate peaking at 15%. This was due to an infrastructure change that enabled a network proxy for requests between the Actions service and an internal API which caused requests to fail.
We mitigated the incident by rolling back the change. We are working to improve our monitoring and deployment practices to reduce our time to detection and mitigation of issues like this one in the future.
Aug 15, 13:59 UTC
We mitigated the incident by rolling back the change. We are working to improve our monitoring and deployment practices to reduce our time to detection and mitigation of issues like this one in the future.
Aug 15, 13:59 UTC
Update -
Approximately 10-15% of customers may be experiencing problems executing new GitHub Actions runs. The problem is currently being investigated by our teams.
Aug 15, 13:45 UTC
Aug 15, 13:45 UTC
Investigating -
We are investigating reports of degraded performance for Actions
Aug 15, 13:35 UTC
Aug 15, 13:35 UTC
Resolved -
On August 14, 2024 between 23:02 UTC and 23:38 UTC, all GitHub services on GitHub.com were inaccessible for all users.
This was due to a configuration change that impacted traffic routing within our database infrastructure, resulting in critical services unexpectedly losing database connectivity. There was no data loss or corruption during this incident.
At 22:59 UTC an erroneous configuration change rolled out to all GitHub.com databases that impacted the ability of the database to respond to health check pings from the routing service. As a result, the routing service could not detect healthy databases to route application traffic to. This led to widespread impact on GitHub.com starting at 23:02 UTC.
We mitigated the incident by reverting the change and confirming restored connectivity to our databases. At 23:38 UTC, traffic resumed and all services recovered to full health. Out of an abundance of caution, we continued to monitor before resolving the incident at 00:30 UTC on August 15th, 2024.
To prevent recurrence we are implementing additional guardrails in our database change management process. We are also prioritizing several repair items such as faster rollback functionality and more resilience to dependency failures.
Given the severity of this incident, follow-up items are the highest priority work for teams at this time.
Aug 15, 00:30 UTC
This was due to a configuration change that impacted traffic routing within our database infrastructure, resulting in critical services unexpectedly losing database connectivity. There was no data loss or corruption during this incident.
At 22:59 UTC an erroneous configuration change rolled out to all GitHub.com databases that impacted the ability of the database to respond to health check pings from the routing service. As a result, the routing service could not detect healthy databases to route application traffic to. This led to widespread impact on GitHub.com starting at 23:02 UTC.
We mitigated the incident by reverting the change and confirming restored connectivity to our databases. At 23:38 UTC, traffic resumed and all services recovered to full health. Out of an abundance of caution, we continued to monitor before resolving the incident at 00:30 UTC on August 15th, 2024.
To prevent recurrence we are implementing additional guardrails in our database change management process. We are also prioritizing several repair items such as faster rollback functionality and more resilience to dependency failures.
Given the severity of this incident, follow-up items are the highest priority work for teams at this time.
Aug 15, 00:30 UTC
Update -
We have fully rolled-back the changes to database infrastructure and mitigated the impact. All services are now fully operational.
Aug 15, 00:26 UTC
Aug 15, 00:26 UTC
Update -
Git Operations is operating normally.
Aug 15, 00:25 UTC
Aug 15, 00:25 UTC
Update -
Copilot is operating normally.
Aug 15, 00:21 UTC
Aug 15, 00:21 UTC
Update -
Codespaces, Packages and Pages are operating normally.
Aug 15, 00:20 UTC
Aug 15, 00:20 UTC
Update -
Webhooks is operating normally.
Aug 15, 00:19 UTC
Aug 15, 00:19 UTC
Update -
Actions is operating normally.
Aug 15, 00:19 UTC
Aug 15, 00:19 UTC
Update -
Actions is operating normally.
Aug 15, 00:19 UTC
Aug 15, 00:19 UTC
Update -
Actions is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:19 UTC
Aug 15, 00:19 UTC
Update -
Actions is operating normally.
Aug 15, 00:19 UTC
Aug 15, 00:19 UTC
Update -
Pull Requests is operating normally.
Aug 15, 00:19 UTC
Aug 15, 00:19 UTC
Update -
Issues is operating normally.
Aug 15, 00:18 UTC
Aug 15, 00:18 UTC
Update -
Copilot is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:12 UTC
Aug 15, 00:12 UTC
Update -
Git Operations is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:11 UTC
Aug 15, 00:11 UTC
Update -
Codespaces is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:10 UTC
Aug 15, 00:10 UTC
Update -
Webhooks is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:10 UTC
Aug 15, 00:10 UTC
Update -
Actions is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:10 UTC
Aug 15, 00:10 UTC
Update -
Pages is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:09 UTC
Aug 15, 00:09 UTC
Update -
Service health continues to improve, and we are working to stabilize all services. Some services may experience delays in updates and notifications as we work through a backlog of events.
Aug 15, 00:09 UTC
Aug 15, 00:09 UTC
Update -
Packages is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:09 UTC
Aug 15, 00:09 UTC
Update -
Issues is experiencing degraded performance. We are continuing to investigate.
Aug 15, 00:09 UTC
Aug 15, 00:09 UTC
Update -
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Aug 14, 23:50 UTC
Aug 14, 23:50 UTC
Update -
The database infrastructure change is being rolled back. We are seeing improvements in service health and are monitoring for full recovery.
Aug 14, 23:45 UTC
Aug 14, 23:45 UTC
Update -
We are experiencing interruptions in multiple public GitHub services. We suspect the impact is due to a database infrastructure related change that we are working on rolling back.
Aug 14, 23:29 UTC
Aug 14, 23:29 UTC
Update -
Codespaces is experiencing degraded availability. We are continuing to investigate.
Aug 14, 23:22 UTC
Aug 14, 23:22 UTC
Update -
Webhooks is experiencing degraded availability. We are continuing to investigate.
Aug 14, 23:20 UTC
Aug 14, 23:20 UTC
Update -
Issues is experiencing degraded availability. We are continuing to investigate.
Aug 14, 23:19 UTC
Aug 14, 23:19 UTC
Update -
Git Operations is experiencing degraded availability. We are continuing to investigate.
Aug 14, 23:19 UTC
Aug 14, 23:19 UTC
Update -
Packages is experiencing degraded availability. We are continuing to investigate.
Aug 14, 23:18 UTC
Aug 14, 23:18 UTC
Update -
We are investigating reports of issues with GitHub.com and GitHub API. We will continue to keep users updated on progress towards mitigation.
Aug 14, 23:16 UTC
Aug 14, 23:16 UTC
Update -
Copilot is experiencing degraded availability. We are continuing to investigate.
Aug 14, 23:13 UTC
Aug 14, 23:13 UTC
Update -
Pages is experiencing degraded availability. We are continuing to investigate.
Aug 14, 23:12 UTC
Aug 14, 23:12 UTC
Investigating -
We are investigating reports of degraded availability for Actions, Pages and Pull Requests
Aug 14, 23:11 UTC
Aug 14, 23:11 UTC
Aug 14, 2024
Aug 13, 2024
Resolved -
On August 13, 2024, between 13:00 UTC and 13:23 UTC the Copilot service and some parts of the GitHub UI were degraded. This impacted about 25% of GitHub.com users. This was due to a partial rollout of a caching layer for Copilot licensing checks. During the rollout, connections to the caching layer were overwhelmed causing the licensing checks to timeout. Many pages were impacted by this failure due to a lack of resiliency to the timeouts.
We mitigated the incident by reverting the rollout of the caching layer 11 minutes after initial detection. This immediately restored functionality for affected users.
We are working to gracefully degrade experiences during these types of failures and reduce dependencies across services that may cause these types of failures in the future.
Aug 13, 13:23 UTC
We mitigated the incident by reverting the rollout of the caching layer 11 minutes after initial detection. This immediately restored functionality for affected users.
We are working to gracefully degrade experiences during these types of failures and reduce dependencies across services that may cause these types of failures in the future.
Aug 13, 13:23 UTC
Update -
Pull Requests is experiencing degraded performance. We are continuing to investigate.
Aug 13, 13:16 UTC
Aug 13, 13:16 UTC
Investigating -
We are currently investigating this issue.
Aug 13, 13:11 UTC
Aug 13, 13:11 UTC
Aug 12, 2024
Resolved -
On August 12, 2024 from 13:39 to 14:28 UTC some users experienced an elevated rate of errors of up to 0.45% from the GitHub API. Less than 5% of webhooks interactions failed and less than 0.5% of Actions runs were delayed.
This impact was caused by internal networking instances being insufficiently scaled.
We mitigated the incident by provisioning additional instances. We are working to enhance the sizing strategy for the relevant infrastructure to prevent similar issues and to also improve monitoring and processes to reduce our time to detection and mitigation of issues like this one in the future.
Aug 12, 14:41 UTC
This impact was caused by internal networking instances being insufficiently scaled.
We mitigated the incident by provisioning additional instances. We are working to enhance the sizing strategy for the relevant infrastructure to prevent similar issues and to also improve monitoring and processes to reduce our time to detection and mitigation of issues like this one in the future.
Aug 12, 14:41 UTC
Update -
Webhooks is experiencing degraded performance. We are continuing to investigate.
Aug 12, 14:41 UTC
Aug 12, 14:41 UTC
Update -
Webhooks is operating normally.
Aug 12, 14:41 UTC
Aug 12, 14:41 UTC
Update -
Some customers may see failures calling the webhooks API or running queries for enterprise or organization audit logs. We have started mitigating the issue and are watching recovery. We will provide an update within the next 30 minutes.
Aug 12, 14:32 UTC
Aug 12, 14:32 UTC
Investigating -
We are investigating reports of degraded performance for Webhooks
Aug 12, 14:09 UTC
Aug 12, 14:09 UTC
Resolved -
This incident has been resolved.
Aug 12, 14:15 UTC
Aug 12, 14:15 UTC
Investigating -
We are currently investigating this issue.
Aug 12, 14:03 UTC
Aug 12, 14:03 UTC
Aug 11, 2024
No incidents reported.
Aug 10, 2024
No incidents reported.
Aug 9, 2024
No incidents reported.
Aug 8, 2024
No incidents reported.
