In the cybersecurity realm, the reliability of security software are crucial. The recent high-profile outage caused by a bug in CrowdStrike’s (CS) testing software has spotlighted the critical importance of non-functional testing (NFT). This incident disrupted services for many users and underscored the necessity for comprehensive testing strategies beyond functional requirements.
Understanding Non-Functional Testing
Non-functional testing (NFT) focuses on evaluating aspects of a system that do not pertain to specific behaviors or functions. These include performance, scalability, reliability, usability, security, and more. While functional testing answers the question, “Does the software do what it is supposed to do?”, non-functional testing addresses, “How well does the software perform under various conditions?”
The CrowdStrike Incident: A Wake-Up Call
CrowdStrike, a prominent cybersecurity firm, faced a significant outage due to a bug in their testing software. This bug was not related to the core functionality of their security products but impacted the reliability of their systems. The incident caused widespread disruption, affecting numerous clients who rely on CrowdStrike’s services to protect their digital assets.
This outage serves as a stark reminder that even when a software application meets all its functional requirements, failures in non-functional aspects can lead to catastrophic consequences. In CrowdStrike’s case, a seemingly basic oversight in non-functional testing resulted in a massive service disruption. Basic tests, such as rolling out an update to a Windows host to observe if a blue screen appears or simply checking if ICMP ping requests are returned from the host, could have prevented this debacle.
The Gravity of Neglecting NFT
Neglecting non-functional testing can have severe repercussions, as evidenced by the CrowdStrike incident. Clients depending on their services experienced interruptions, potentially exposing them to increased security risks. The incident also damaged CrowdStrike’s reputation, illustrating that even the most reputable firms are vulnerable when non-functional testings are overlooked.
In the context of Crowdstrike, the simple two-step test below could have saved Crowdstrike and its customers a lot of heartache.
- Apply the latest hot fix or patches to a groups of machines running all the current different version of supported OS.
- Observing for a fix period of time e.g. 6 to 12 hours that the test hosts continue to be responsive. A simple test would be to issue ICMP ping requests, and then waiting for the replies as a sign that the test hosts are online and all is well.
This test could have been written by a junior IT executives or computer science intern.
Having said that a comprehensive NFT test plan involves far more than these examples. However, it serves to illustrate the gaps in NFT that could be present in even one of the most respectable technology companies.
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
In Perennial Consultancy, we take pride in Non Functional Test consulting such as Web Application Penetration and Performance testing for our customers. This is what we have been doing for the last 10 years and we have gotten pretty good at it, checkout our penetration test packages or contact us to find out more.
