Earlier this year, our Detect Monitoring System team was faced with increased attacks across our customer base. This aligns with the Anti Phishing Working Group’s (APWG) findings that it observed more phishing attacks in the first quarter of 2016 than in any other three-month span since it began tracking data in 2004. APWG reported that the number of phishing websites it detected jumped a startling 250 percent between October 2015 and March 2016.
While we, just like search engines, social networks, ISPs, hosting providers, URL shortening providers, and others use blacklisting technologies, it is becoming increasingly evident that these backward-looking technologies cannot keep up with the pace of change that phishers are utilizing to improve their odds of success.
We are always looking for ways to improve our detection capability, prioritize and triage events, and remove or disable attacks as soon as possible. And so, we were looking for an edge.
The challenge that we and others face is increased attack load; increased diversity of attacks based on style, structure and geographic distributions; and a myriad of other factors. Our hypothesis was that if we could predict the chances of a URL being malicious based on the characteristics outlined above, instead of relying on historic information (blacklists), we would have a better shot of preventing more phishing than with blacklists alone. This is why we developed Swordphish. Swordphish is a REST API backed by a set of extremely fast machine learning classifiers designed to predict with a high-degree of confidence if a URL or domain is likely to be associated with phishing or malware command and control (C&C).
Check this 2-minute video that explains what Swordphish is all about:
The twin burdens of any large InfoSec, threat intelligence, security monitoring or abuse team are event overload/fatigue and intelligence latency. The first is related to the constant struggle to collect, triage and prioritize events from multiple sources, in multiple formats related to assets of varying degrees of criticality. Building systems and processes to avoid being simply overwhelmed by security and fraud events is a huge challenge in and of itself. The second is related to intelligence latency, otherwise known as the “zero-day problem.” There are fantastic tools available to automate malware detonation and extract IOC data, to query numerous IP/domain reputation services for threat information such as VirusTotal and powerful workflow and orchestration systems to automate the process, but all of these are behind the eight ball because they depend on recognizing already-known malware before kicking off a response. Swordphish can help with both of these issues.
How can Swordphish do this? Let’s unpack some buzzy terminology around “machine learning” and get into some more details. First, while I definitely believe in the steady progression of AI, the upcoming arrival of superintelligence and the march of the robots—we are not there yet. Machine learning technology is incredibly powerful—it is permeating software and consumer electronics and will likely usher in a new age of computing. Machine learning at a fundamental level is a field that explores how to empower software to solve pattern recognition problems as–or more–effectively than humans. Machine learning benefits from the availability of huge data sets and machine learning algorithms, as well as systems such as Random Forest, SVM, k-means clustering and neural networks, among others. These technologies enhance the ability for software to predict future outcomes based upon historical data with a high degree of accuracy. That is the power of machine learning, and of Swordphish.
Swordphish is built upon three machine learning classifiers—phishing, malware C&C and malware DGA (Domain Generating Algorithm). Each classifier has been trained on millions of URLs, both good and bad, and has extracted more than 50 features in the domains and URLs themselves that the classifier considers when making a prediction. The training process is on-going and continual. This allows Swordphish to adjust its predictive capability as attack trends change—in essence to adapt to changing conditions seamlessly.
The resulting phishing model had a F1-Score of 0.94, an accuracy of over 95 percent and showed great stability in the holdout set. The classifiers for malware and malware DGA show similar results. We will do a more technical post in the future to go into more detail.
We encourage you to test it out, share your feedback, and think about how this could help your organization combat the growing phishing problem. Existing solutions are clearly not working. We look forward to working with the community to turn the phishing problem on its head and using the characteristics of phishing sites themselves to help us fight this battle.