Detection Engineering Automation: ChatGPT

Ashish Bansal
System Weakness
Published in
4 min readDec 30, 2022

--

ChatGPT is the latest buzzword, but its capabilities go far beyond just answering a few questions.

If you haven’t heard of ChatGPT then you been living under the rocks :-P

I have been constantly inundated with ChatGPT stories on my Linkedin account, so I thought it would be worth showing you how ChatGPT can help automate detection engineering and allow us to generate basic SIEM rules quickly and easily.

Walk with me! Photo by Jason Yuen

Not sure if we should be excited or nervous about the capabilities of ChatGPT.

I am probably the first person on internet talking about this use case of ChatGPT, so not sure if people will love it or if I am been beaten up in public. :-P

~Let’s go step by step

  1. First step is to understand the detection that we would create in our further steps, let’s ask ChatGPT for the required details.
ChatGPT answering about Windows Bruteforce Event Code

2. Let’s get the dummy logs for Windows login failure (Bruteforce), so let me ask my friend ChatGPT.

Windows failure log dummy event generated by ChatGPT

Continuation:

If you want to understand the specific 4625 event log then you can research as per https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4625

I very sure some of you might be thinking just Windows logs?, too basic!
So, lets look at some PaloAlto Firewall logs then.

Yea, it actually generated below few examples for Palo Alto Firewalls.
  1. Traffic Log:
PaloAlto Firewall Log: Traffic Log (ChatGPT)

2. Threat Log:

PaloAlto FW Log: Threat Log (ChatGPT)

3. Configuration Log:

I do understand that you are already too excited and scared both at the same time but let me remind you that we still need humans to infer the context and detection logics :-)

3. Last step would be to actually generate the detection rules for windows brute force , ChatGPT dude! help me.

First Try!

ChatGPT Sigma detection windows bruteforce

Another try:

Check the Sigma format (Not 100% but almost)

2# Continue

Let’s try my favorite “MimiKatz”.

Mimikatz detection using ChatGPT

`Something advance :-) — Read about it \../

Created new Sigma rule using ChatGPT

I created an AWS-specific detection rule that ChatGPT was unable to infer my intent. However, the response is logically correct that I never thought about it before. (Try if it works)

ChatGPT bro you need more brains!

Still not enough? Let’s do Splunking ~ \../

Wow! Hate Me or Love Me! ChatGPT

Another screenshot from my Android Phone >

Splunking with ChatGPT

Something to get started with your own research:

Please note ChatGPT is still in initial stages and it can’t automate things 
entirely for you, so don’t panic & relax.

There are already a few companies that have begun integrating ChatGPT in their solutions but they are still far from reality and it will take few more years to reach the level of required confidence.

If you need any recommendations, don’t hesitate to reach out to me.

~AshishSecDev

--

--