pythonsecurityprocess-monitoringyarasystem-monitoring

Threat detection with Sysmon .csv log using Sigma Rules

I'm not sure whether my approach is completely wrong, so please let me know if it is.

So basically I have a .csv file that contains sysmon logs in this format:

"TimeCreated","EventId","Message"
"15.08.2024 22:24:12",     "11",    "File created:
RuleName: Downloads
UtcTime: 2024-08-15 20:24:12.886
ProcessGuid: {processGuid}
ProcessId: ID
Image: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
TargetFilename: C:\Users\Downloads\Sysmon\SysmonLogs.csv
CreationUtcTime: 2024-08-15 13:51:58.523
User: Username"

And I have a few Sigma Rules specifically for sysmon, for example:

title: Suspicious Outbound RDP Connections
id: ed74fe75-7594-4b4b-ae38-e38e3fd2eb23
status: experimental
description: Detects Non-Standard Tools Connecting to TCP port 3389 indicating possible lateral movement
references:
    - https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/CVE-2019-0708
author: Markus Neis - Swisscom
date: 2019/05/15
tags:
    - attack.lateral_movement
    - attack.t1210
    - car.2013-07-002
logsource:
    product: windows
    service: sysmon
detection:
    selection:
        EventID: 3
        DestinationPort: 3389
        Initiated: 'true'
    filter:
        Image:
            - '*\mstsc.exe'
            - '*\RTSApp.exe'
            - '*\RTS2App.exe'
            - '*\RDCMan.exe'
            - '*\ws_TunnelService.exe'
            - '*\RSSensor.exe'
            - '*\RemoteDesktopManagerFree.exe'
            - '*\RemoteDesktopManager.exe'
            - '*\RemoteDesktopManager64.exe'
            - '*\mRemoteNG.exe'
            - '*\mRemote.exe'
            - '*\Terminals.exe'
            - '*\spiceworks-finder.exe'
            - '*\FSDiscovery.exe'
            - '*\FSAssessment.exe'
            - '*\MobaRTE.exe'
            - '*\chrome.exe'
            - '*\thor.exe'
            - '*\thor64.exe'
    condition: selection and not filter 
falsepositives:
    - Other Remote Desktop RDP tools
level: high

Now, I wanted to scan my .csv files and find whether there is any match in the logs with any sigma rule that I have.

I tried using pysigma with python, for example:

import os
import pandas as pd
from sigma.collection import SigmaCollection

sigma_rules_dir = 'path_to_rules'

def load_sigma_rules(directory):
    rules = []
    for filename in os.listdir(directory):
        if filename.endswith('.yml') or filename.endswith('.yaml'):
            with open(os.path.join(directory, filename), 'r') as file:
                rule_yaml = file.read()
                sigma_collection = SigmaCollection.from_yaml(rule_yaml)
                rules.extend(sigma_collection.rules)
    return rules

def process_detection_items(detection_items):
    # do detections

# Load the CSV log file
def load_logs(csv_path):
    return pd.read_csv(csv_path)

def evaluate_rule(rule, log_entry):
    # evaluate

def check_matches(df, rules):
    

csv_log_path = 'path_to_csv'
sigma_rules = load_sigma_rules(sigma_rules_dir)
df = load_logs(csv_log_path)

check_matches(df, sigma_rules)

I've been trying hours to get this right but I can't make it work. And what's worse is that I can't find much online either with this approach.

Am I doing this wrongly? For example, how do sandboxes generally do what I'm trying to do? Is there a simpler approach that I am missing for exactly this?

Solution

  • I changed approach. This is the alternative solution that seems to work fine for me: