Coinbase Identifies Kubernetes Error Behind 50-Minute Outage on July 14

COIN-5.57%
SOL0.40%
Key Takeaways
  • Coinbase published a postmortem on July 21 identifying a Kubernetes resource name collision causing a 50-minute outage on July 14.
  • The outage prevented transfers, Coinbase Card purchases, and onchain services across retail, institutional, and developer platforms.
  • Coinbase is implementing new safeguards including naming conflict checks and separation of recovery tools from managed infrastructure.

Coinbase published a postmortem on July 21 identifying the software error behind a 50-minute outage on July 14 that prevented customers from completing transfers, making Coinbase Card purchases, and accessing onchain services. The disruption was caused by an undetected Kubernetes resource name collision during a routine infrastructure update that modified the wrong network component. The incident affected retail, institutional, and developer platforms from approximately 12:37 p.m. ET to 1:25 p.m. ET, though customer funds were not at risk. The outage occurred as Coinbase expands beyond spot crypto trading into tokenized stocks, perpetual futures, and broader financial services, increasing the stakes when shared infrastructure fails.

Kubernetes Resource Name Collision Triggered Gateway Failure

Engineers were deploying what appeared to be a routine, low-risk update when an undetected Kubernetes resource name collision caused the update to modify the wrong network component, according to the postmortem published on July 21. Kubernetes is software Coinbase uses to organize and run applications across its cloud infrastructure. The resource name collision caused the update to alter part of the company's Istio ingress gateway instead of only the intended component. The gateway acts like a traffic controller, directing requests between Coinbase's internal services. Once the gateway went offline, systems responsible for processing trades, transfers, settlements and payment authorizations could no longer communicate. Coinbase stated that the incident "was caused by an unintended misconfiguration of an important network component as a result of a routine configuration update" and that "at no point were customer funds at risk."

Outage Disrupted Transfers, Card Purchases, and Institutional Settlements

The disruption affected Coinbase's retail, institutional and developer platforms from approximately 12:37 p.m. ET to 1:25 p.m. ET. Some retail customers could not complete deposits, withdrawals or off-platform transactions. Transfers already underway appeared stuck but were processed after service returned. Coinbase Card debit purchases were declined, while credit card purchases continued. The outage also interrupted onchain swaps through Coinbase DEX on Base and Solana. Coinbase Exchange and Prime customers experienced delayed or failed transfers and settlements, while developer platform customers could not onboard users, move funds, or use onramp services. The impact highlights the growing importance of reliability as Coinbase expands beyond spot crypto trading. Its planned "everything exchange" products, including tokenized stocks and perpetual futures, are widening the range of financial activity dependent on the company's infrastructure. Coinbase has separately described plans to make the platform a broader financial hub, with new trading, payments and money-management services spanning stocks, derivatives, stablecoin payments and developer tools.

Coinbase Implements New Safeguards After Manual Recovery

The gateway was restored at 1:20 p.m. ET, and Coinbase declared the incident mitigated three minutes later. Remaining transaction backlogs cleared over the following hours. Recovery took longer because Coinbase's deployment tools relied on the failed gateway. Engineers could not use the standard rollback process and instead manually launched an emergency deployment through the company's cloud provider using temporary privileged access. Coinbase is now adding checks to block naming conflicts, separating recovery tools from the infrastructure they manage and testing emergency access procedures more regularly. Coinbase emphasized that "our goal is always zero downtime" and stated: "We are actively redesigning our infrastructure to ensure that, in the highly unlikely event of another failure like this one, we can swiftly revert and recover."

FAQ

What caused the Coinbase outage on July 14? The outage was caused by an undetected Kubernetes resource name collision during a routine infrastructure update that modified the wrong network component, specifically altering part of Coinbase's Istio ingress gateway instead of the intended component.

How long did the Coinbase outage last and what services were affected? The outage lasted approximately 50 minutes, from 12:37 p.m. ET to 1:25 p.m. ET on July 14. It affected retail customer transfers and Coinbase Card debit purchases, institutional settlements on Coinbase Exchange and Prime, onchain swaps through Coinbase DEX on Base and Solana, and developer platform services including user onboarding and fund movement.

What safeguards is Coinbase implementing after the outage? Coinbase is now adding checks to block naming conflicts, separating recovery tools from the infrastructure they manage, and testing emergency access procedures more regularly to ensure swift reversion and recovery in future incidents.

Disclaimer: The information on this page may come from third-party sources and is for reference only. It does not represent the views or opinions of Gate and does not constitute any financial, investment, or legal advice. Virtual asset trading involves high risk. Please do not rely solely on the information on this page when making decisions. For details, see the Disclaimer.
Comment
0/400
No comments