How to Use Qumulo’s Detective Controls is the third blog of a four-part series designed to help you take advantage of the security controls and data protection capabilities in Qumulo’s file data platform.
Detective controls are essential when it comes to uncovering malicious activities as early as possible. Ideally, these kinds of activities must be detected before they reach the storage system.
Implementing a holistic security approach that includes network, compute, device and event-monitoring techniques, together with data correlation and analysis, is preferable over siloed solutions that are embedded in the storage system.
The Qumulo File Data Platform supports all major ISV security solutions through its auditing feature. In addition, Qumulo’s API allows you to initiate automated mitigation actions from any attack surface should a malicious activity be detected.
A holistic security approach to malware detection captures data from as many devices as possible to analyze, correlate and automate action on suspicious events.
Audit logging in Qumulo Core, the heart of our file data platform, provides a mechanism for tracking file systems as well as management operations. As connected clients issue requests to the cluster, log messages are generated describing each attempted operation. These log messages are then sent over the network to the remote syslog instance specified by the current audit configuration in compliance with RFC 5424. As the target instances of all these log events are outside of the Qumulo cluster, they cannot be manipulated or deleted afterward.
The advantages of the Qumulo approach for detective controls are:
- Qumulo uses an industry-standard syslog format that can be read, parsed, and indexed by any common Security Information and Event Management Software (SIEM) in the market
- All data access and management tasks are captured
- Validated solutions include Splunk, Elasticsearch, and AWS CloudWatch
The advantage of a centralized SIEM approach is that a common solution for all data center or cloud instances and services exists. Data can be gathered easily, and indexed, filtered, analyzed, searched, and visualized. Automated or semi-automated actions can be triggered when suspicious activities are detected. This is the most effective approach because malware is being identified and stopped before it reaches the storage system.
The first line of antivirus (AV) prevention should be the data center security infrastructure. This can include firewalls, network scanning, server and desktop clients. It is essential to understand that if malware reaches the storage system, the data can be compromised. Nevertheless, we still see constant asks for AV solutions on the storage system and here are some options:
- On-demand-scans: Qumulo supports on-demand scans of all major antivirus solutions in the market. They can be scheduled regularly and preferably run in off-peak hours. On-demand scans on Qumulo can be completed much faster than scale-up NAS because several scanners can be run in parallel against all of the nodes in the cluster simultaneously.
- Client-side scans: this is the preferred method if you need scan-on opens and it is fully supported by Qumulo. In general, this is the best place to invest in security measures since malicious payloads are executed by the client. Qumulo suggests adopting a next-generation antivirus which is not based on signatures (that can be modified easily by advanced attackers) but is based on binary fingerprinting and AI techniques. Also, a proper patching management strategy and a whitelisting approach that allows only legitimate and IT-controlled software to execute, are beneficial in order to reduce the attack surface and the overall risk of an outbreak.
- On-access scans (without AV client SW): some vendors conduct this with the ICAP protocol. Once a file is opened, it is sent to an antivirus SW instance that must also support this protocol. Then, the file is scanned before (scan on open) or after (scan on close) the file is opened. The common issues with this approach are discussed in the next section.
On-access scan issues
The on-access scan (if it is not performed on the client) has shown significant challenges in practice:
- Unacceptable response times: you bought a super fast and scalable scale-out NAS system that delivers typical response times in the range of 0.5-5 ms. If you added an antivirus engine for scan-on opens the response times would typically increase to several seconds (depending on the network, the scan engine, the hardware, and the size of the files). This is usually not acceptable for a high-end solution where users and applications require fast response times.
- An incredibly high amount of scanning servers: On-access scans require a very large server farm for the scanners. Typically 1-2 physical servers per storage node. This isn’t a good approach and the viruses should be captured at the source: desktop computers, servers, network devices.
To avoid the disadvantages of antivirus-scanning on the storage system, Qumulo recommends the following best practices:
- If on-access scanning is required, use client-side agents. They are available for all major platforms and they don’t come with substantial disadvantages.
- Use on-demand or scheduled scans on Qumulo with your antivirus solution of choice. They can be scheduled regularly and in off-peak hours.
- Take regular snapshots (covered in Corrective Controls, the final blog from the white paper).
Download the white paper: Security Architecture and Best Practices to Counter Malware
The goal of a ransomware attack is typically to exfiltrate data and/or encrypt data to force companies to pay for keys to decrypt their data.
An attack typically happens in phases:
- Get access to the network and at least one initial device
- Infect as many additional devices as possible to gather information
- Exfiltrate data
- Ransomware deployment: loading additional modules that; for example, encrypt data.
- Encrypt data for extortion
Typical attack vectors
There are many attack vectors for a ransomware deployment. Some common examples are:
- Spear-phishing emails (that is the number one threat)
- Leverage OS vulnerabilities that are not fixed/patched
- Trojan-horse software
- Man-in-the-middle attacks
- Web-server exploits
- Cross-site scripting
- SQL injection
- Domain spoofing
- Watering hole websites
Efficient detection controls and prevention strategies
An efficient prevention strategy needs to target the start of the infection phase. Looking at the attack vectors, it is obvious that this is not primarily the storage device. Or in other words: don’t think that having a siloed anti-malware solution on your storage system is an efficient strategy. This is definitely not the case. Why? Because during ransomware attacks all kinds of devices get infected: network devices, IoT devices, desktop computers, servers, cameras, printers, basically everything that has an operating system. The malware stays there for quite some time before it becomes active. In the first phase, the intruders continue to gather more information about the infrastructure (users, data flows, network topologies, devices). Then, at the later stage of the attack, they start to exfiltrate data and/or load additional modules to start other threads such as accessing data and encrypting files.
So here it becomes obvious that an efficient prevention strategy needs to start at the early phase of the infection. The storage system is at the very end of the attack phase. The infection needs to be captured and stopped much earlier in the network and any kind of compute devices where the malware lands and runs.
Furthermore, often remediation efforts are followed by a re-infection because not all potential devices have been cleared in the network. This is another strong reason to capture malware at the initial phase, in the network, servers, applications (i.e. email).
Modern security information and event management (SIEM) solutions capture data from all potential devices and offer holistic approaches to detect and prevent malware infections. One important aspect for detective controls is central event capturing, and correlation. In addition, intrusion detection systems (IDS) can detect patterns of dangerous network traffic; for example, anomalous domain name server (DNS) queries used to exfiltrate data, packets that are correlated to an exploit technique.
Many companies are using intrusion prevention systems (IPS) for detection controls with advanced fire walling and exploit-detection capabilities, that can help to fence off some categories of attacks.
Implement automated responses using the Qumulo API
Once the IDS system has detected a suspicious or even malicious activity for a file, the systems can trigger automated events. Qumulo provides a rich REST API which allows automating all kinds of management tasks on the cluster. Following are examples of mitigation tasks in case of a security event:
- Set a quota for a directory or the full system to 0. In this case, any new write activity is prevented (but overwrites might still be possible)
- Set a share to read-only or restrict IP addresses
- Remove privileges for a user
- Take or restore a snapshot
- Start an antivirus on-demand scan
There are multiple ways to leverage the Qumulo API:
- Direct API calls
- Use the Qumulo provided python libraries to simplify API script development
- Use the Qumulo CLI
We recommend using Qumulo’s detective controls in conjunction with security best practices to mitigate risk by detecting cybersecurity breach attempts (“events”) or successful security breaches (“incidents”) while in progress.
In the final blog of this series, we’ll cover corrective controls, or, to jump ahead, download the Security Architecture white paper.
Take a test drive. Demo Qumulo in our interactive Hands-On Labs.
Subscribe to the Qumulo blog for customer stories, technical insights, industry trends and product news.
Stefan Radtke, Field CTO EMEA, has spent his career working in technology and is the principal evangelist of universal-scale storage for Qumulo. He started as employee #1 in EMEA in 2017 as Technical Director where he built a fantastic multi-national technical team. Recently he took over the role of the Field CTO and he is now focusing on building a strong technical team for Cloud Q. He’s a certified AWS Solution Architect Professional and Azure Solution Architect Expert.