pythonperlbro

Bro IDS searches

I have never used Python or Perl before. I got a project from work due to unforeseen circumstances and am hoping one of you guys can help. We use BRO for an IDS System. I am supposed to write a script that will open a log file (in .gz format), search the log for a list of keywords saved in a text file, exclude any results that have keywords listed in another file, and then output those results into a new .gz. It is supposed to run 3 times per day. I am not sure where to start but any help would be incredibly helpful and greatly appreciated.

Solution

  • ahh after several re-reads I think I start to see the question ... IDS and BRO just are totally irrellevant to the question and actually confuse the matter

    I think I can decompose your question down into several sub-questions

    Question 1.How do I open a gzip encoded file for reading?

    import gzip
with gzip.GzipFile("/path/to/some_log_file.gz") as input_plaintext_filehandle:
     for line in input_plaintext_filehandle:
         print line

    Question 2. How do I write to a gzip file?

    import gzip
with gzip.GzipFile("/path/to/some_log_file.gz","w") as output_file:
     output_file.write("Hello GZIP World")

    Question 3. How Do I Filter out lines based on a wordlist.txt?

    words = set(open("wordlist.txt").read().split())
for line in open_file_handle:
    if words.intersection(line.split()):
       print "Match Found"
    else:
       print "No Match Found"

    I think this should give you enough to get started on solving this problem. feel free to come back once you get stuck and have some code to show