In this article, you’ll learn how to use Qumulo Audit to forward events into Azure Monitor and query Azure Log Analytics to secure your data.
Malware attacks are on the rise in quality and quantity, infecting hundreds of thousands of IT systems and causing millions, if not billions of dollars of damage to all kinds of businesses. Having a well thought out security architecture is critical for all IT infrastructure environments on-premises and on the cloud.
The Qumulo File Data Platform includes a broad spectrum of modern technologies and data services to support holistic security architectures. The following sections will show you how to use the Qumulo Audit logging feature, in conjunction with Azure Log Analytics, to track file system operations including users, file paths, permissions, status, changes, and deletes.
Because the Qumulo file system does not send audit events directly into Azure Monitor, we use a syslog server to receive the messages and forward them with the Azure Monitor Agent to Azure Monitor. This article focuses on the ingest process, i.e. how to get the Qumulo events into Azure Monitor and then do some basic queries in a Log Analytics Workspace. In a later blog we’ll focus on active threat hunting using Azure Sentinel.
Azure Monitoring
Azure Monitor is Microsoft’s platform to store all kinds of machine log data and metrics. It provides a rich tool set to analyze, query, visualize and correlate all kinds of data.
Ingesting Qumulo audit events into Azure Monitor allows you to:
- Track any file system operation.
- Create advanced queries and correlate different data sources (for example, logging events from Active Directory or your Firewall). This makes detecting suspicious activity much easier.
- Use Machine Learning techniques with Azure Sentinel to automatically detect unusual behaviour.
- Use automation workbooks to run automated responses.
- Massive or unusual delete activity.
- Unexpected actions from certain users or machines.
- Users and paths with unexpected permissions errors.
For more information on Azure Monitoring please visit Azure Monitor Overview.
Qumulo Auditing
Audit logging in Qumulo Core provides a mechanism for tracking file system operations. As connected clients issue requests to the cluster, log messages are generated describing each attempted operation (read, write, delete, etc). These log messages include all relevant information about the event in the syslog message body: IP address of the client, user name and user id, operation,file name, and ID of the target file or directory. The events are then sent over the network to a remote syslog instance specified by the current audit configuration in compliance with RFC 5424.
For further information on Qumulo Auditing, please visit Qumulo Core Audit Logging.
Qumulo does not parse or analyse these logs. Therefore, you might ingest them into an appropriate tool such as Azure Monitor to query, correlate and visualize them with tools like Azure Log Analytics or Azure Sentinel.
Sending Qumulo Auditing Data to Azure Monitor
There are various options to send log messages to Azure Monitor. For example:
- Use the Azure Monitoring Agent for syslog forwarding
- Use the Log Analytics Agent for syslog forwarding
- The HTTP Data Collector API
In this post we focus on using the Azure Monitoring Agent. As of this publish date (July 2021), the Azure Monitor Agent was in preview status. However, it will replace and consolidate a couple of other agents like the Log Analytics agent and the Telegraph agent. Therefore, we focus on this newest agent (we also tested the Log Analytics Agent, which works fine as well, but is deployed differently).
For more information on the Azure Monitoring agents please visit Overview of Azure Monitoring Agents and the HTTP Data Collector site.
Implementation Steps
The required implementation steps to send Qumulo audit events are relatively straight forward:
- Create a Log Analytics Workspace in you Azure portal
- Install the Azure Arc enabled servers agent (only required if the syslog server is not running in Azure)
- Create a Data Collection Rule (DCR). Azure Monitoring Agent will automatically deploy on the selected target machines.
- Configure Qumulo to send Audit Events to the syslog VM
Upon implementation, all your audit events will be sent to Azure Monitor and you can start querying and analyzing your audit logs.
For detailed implementation steps, please refer to the Qumulo Knowledge Base article: Auditing Qumulo on Azure using Azure Monitor.
Query Log Analytics
To query your audit logs:
- Log into the Azure Portal
- Go to Monitor
- Select your Log Analytics Workspace
- Select Logs
- Enter your query
Azure Log Analytics allows entering all sorts of simple and complex queries with the query language Kusto (for more information go to Getting started with Kusto).
To simply show all syslog messages, simply type: syslog into the search file, and then you’ll see all syslog messages coming in.
| where HostName startswith “du6”
This will list events that have been generated from hosts starting with du6* only.
Extracting field out of the syslog message
The syslog messages are stored in raw format. The format is described here in detail and it contains several fields such as the IPv4 or IPv6, user-ID, protocol, computer, operation, the target file or directory etc.
A sample looks like this:
192.168.1.10,”AD\alice”,nfs,fs_read_data,ok,123,”/.snapshot/1_snapshot1225/dir/”,””
For more complex queries it might be helpful to extract the different values of the syslog message body into separate fields, which can be accomplished by the following query:
Syslog
| extend CSVFields = split(SyslogMessage, ‘,’)
| extend ClientIP = tostring(CSVFields[0])
| extend UserID = tostring(CSVFields[1])
| extend Protocol = tostring(CSVFields[2])
| extend Operation = tostring(CSVFields[3])
| extend ResponseCode = tostring(CSVFields[4])
| extend MessageID = tostring(CSVFields[5])
| extend File = tostring(CSVFields[6])
| extend Status = tostring(CSVFields[7])
Now you have field names which you can use to further detail your queries. Or you can save the query as a function which is easier for reuse and more complex queries. For example, we store the above query in a function called QumuloEvents.
Then, we can see all audit events made on text files in the selected scope:
QumuloEvents
| project EventTime, ClientIP, File, Operation
| where File contains “txt”
Summary
Azure Monitor offers a huge set of data analysis options for various sources and destinations. In this article we provided an overview of how Qumulo Audit events can be sent to Azure Monitor Logs by using the Azure Monitor Agent and how these events can be queried in a Log Analytics Workspace.
In a later blog we’ll discuss how to discover malicious behavior in your environment using active threat hunting techniques and machine learning (ML) features.
Learn more
- Your Path to the Hybrid Cloud: Qumulo on Azure as a Service (QaaS)
- Auditing Qumulo on Azure using Azure Monitor, Implementation Steps
- How to Use Qumulo’s Built-in Security Controls for Data Protection
- How to Use Qumulo’s Preventive Controls Against Malware
- How to Use Qumulo’s Detective Controls Against Data Breaches
- How to Use Qumulo’s Corrective Controls to Minimize Data Loss
Contact us
- Take a test drive. Demo Qumulo’s products in our new, interactive hands-on labs or request a free trial.
- Subscribe to the Qumulo blog for customer stories, technical insights, and product news.
