By Yashin Manraj, CEO — Pvotal Technologies
The Crowdstrike debacle, which impacted 8.5 million devices on July 19, 2024, resulted from a simple software update gone wrong. Yet, before the issue could be resolved, it resulted in an estimated $5.4 billion in direct losses for organizations around the globe.
The event was a wake-up call for the airlines, banks, hospitals, and other organizations that relied on Crowdstrike, as well as those that look to similar pseudo-cybersecurity companies for security support.
Trusting your security to a third party, as many organizations do, is risky. Organizations that choose that route should take steps to ensure the solution they’ve selected is trustworthy.
Why did the Crowdstrike failure occur?
Crowdstrike’s operational approach was the main culprit behind the failure. Leading IT infrastructure companies like AWS, Google, and Microsoft leverage robust automated management and rigorous testing procedures to ensure code updates are delivered effectively. Crowdstrike does not — a shortcoming that led to one of the most significant disruptions to civilian infrastructure in the past decade.
Automated deployment management systems ensure updates are implemented seamlessly without errors. Advanced systems facilitate a gradual rollout, in which phased deployment across regions limits the impact of unforeseen problems. When platforms fail to utilize automation, safeguards, and checks and balances in their release management systems, the type of widespread issues seen in the Crowdstrike event should not be surprising.
How can another Crowdstrike be avoided?
The disruption caused by the Crowdstrike event stands as a stark reminder that applications running with high privileges at the core of the operating system require stringent security procedures. Any platform that has the potential to derail an operating system should utilize deployment protocols similar to those employed by OS or cloud providers like Microsoft, Google, Apple, or open-source Linux.
On the user side of the equation, the Crowdstrike failure should prompt a change in attitude and approach. Crowdstrike offers a technologically sound security solution for all of its clients’ endpoints. However, the company’s marketing success fostered an impression of unparalleled security and stability that led its clients to adopt the system without fully understanding its potential risks.
Organizations in need of infrastructure support should resist equating commercial success with operational competence. Due diligence requires understanding not only what a platform offers but also how providers should maintain their platforms to keep them secure and online.
How should companies prepare for another Crowdstrike?
For companies that are blindly trusting a single vendor to provide critical infrastructure support, major changes will be required to deal with another Crowdstrike efficiently. Those changes should begin with re-evaluating their IT infrastructure, understanding the risks they are assuming by depending on a single vendor, and prioritizing vendor due diligence for those running with local system privileges. Being better prepared would also require protecting vendors' critical infrastructure with secret management, such as disk encryption keys and certificates.
The Crowdstrike failure also underscores the importance of local engineering capabilities, which have become less common as remote work has become more common. Early reports on the Crowdstrike issue indicated that implementing a fix could require having physical access to affected devices. For remote-first companies, having local capabilities available to fix critical company hardware may need to be a new priority.
Overall, the Crowdstrike incident highlights the need for balancing capability and competency in the IT space. Before companies trust third-party providers promising productivity gains, they must ensure that the systems are operationally sound.
– Yashin Manraj, CEO of Pvotal Technologies, has served as a computational chemist in academia, an engineer working on novel challenges at the nanoscale, and a thought leader building more secure systems at the world’s best engineering firms. His deep technical knowledge from product development, design, business insights, and coding provides a unique nexus to identify and solve gaps in the product pipeline. The Pvotal mission is to build sophisticated enterprises with no limits that are built for rapid change, seamless communication, top-notch security, and scalability to infinity.
