I have a index (7111) in elastic search that contains event log data. One of the fields (_source) contains a Dictionary of data. Is there any way using python to formulate a elastic search query that searches the key and values in the Dictionary?
For example let's say I want to do a query that searching for "ProcessID" = "2060" AND "EventID" = "2002". How would I formulate this query?
Note I am currently using pyes, and the best I have been able to do is return all data in the index:
from pyes import *
conn = ES('dbup:9200')
conn.default_indices=["7111"]
q = TermQuery("_type", "tzworks_evtwalk")
results = conn.search(query = q)
for r in results:
print results
Example json data:
"hits" : [ {
"_index" : "7111",
"_type" : "tzworks_evtwalk",
"_id" : "AU7cz4WnebFiST-VQOSA",
"_score" : 1.0,
"_source":{"ProcessID": "2060", "Time-UTC": " 14:04:14.071", "UserID": "S-1-5-19", "Version": "0", "RelatedActivityID": "0", "ThreadID": "6316", "SettingValue": "04 00 00 00", "ModifyingUser": "S-1-5-80-3088073201-1464728630-1879813800-1107566885-823218052", "Channel": "Microsoft-Windows-Windows Firewall With Advanced Security/Firewall", "guid": ["1e152f7025a311e5b1cf005056c00008"], "eventlog": "/home/xxxxx/xxxxx-dirs/workdir/Collection-070815-141327_7111/C/Windows/System32/winevt/Logs/Microsoft-Windows-Windows Firewall With Advanced Security%4Firewall.evtx", "Date": "05/26/2015", "Origin": "1", "Task": "0", "SettingValueSize": "4", "Name": "Microsoft-Windows-Windows Firewall With Advanced Security", "Level": "4", "Opcode": "0", "Security": "", "EventID": "2002", "Record#": "19937", "xmlns": "http://schemas.microsoft.com/win/2004/08/events/event", "SettingValueDisplay": "Public", "line_md5": "55dd6f2a29a43658c5904e39a2e66fc4", "ActivityID": "86432a0b-3c7d-4ddf-a89c-172faa90485d", "Computer": "xxxxxx-X7001.clients.us.xxxx.xxxxx.com", "ModifyingApplication": "", "SettingType": "2", "Keywords": "0x8000000000000000", "row_id": 9, "Guid": "d1bc9aff-2abf-4d71-9146-ecb2a986eb85", "Qualifiers": "0"}
}, {
"_index" : "7111",
"_type" : "tzworks_evtwalk",
"_id" : "AU7cz4WnebFiST-VQOSF",
"_score" : 1.0,
"_source":{"ProcessID": "2060", "Time-UTC": " 14:05:57.506", "UserID": "S-1-5-19", "Version": "0", "RelatedActivityID": "0", "ThreadID": "5988", "SettingValue": "05 00 00 00", "ModifyingUser": "S-1-5-80-3088073201-1464728630-1879813800-1107566885-823218052", "Channel": "Microsoft-Windows-Windows Firewall With Advanced Security/Firewall", "guid": ["1e152f7025a311e5b1cf005056c00008"], "eventlog": "/home/xxxxx/xxxxx-dirs/workdir/Collection-070815-141327_7111/C/Windows/System32/winevt/Logs/Microsoft-Windows-Windows Firewall With Advanced Security%4Firewall.evtx", "Date": "05/26/2015", "Origin": "1", "Task": "0", "SettingValueSize": "4", "Name": "Microsoft-Windows-Windows Firewall With Advanced Security", "Level": "4", "Opcode": "0", "Security": "", "EventID": "2002", "Record#": "19950", "xmlns": "http://schemas.microsoft.com/win/2004/08/events/event", "SettingValueDisplay": "Domain,Public", "line_md5": "0b20f373041ddb34a6f0fc61926dc5bc", "ActivityID": "0", "Computer": xxxxxxx-X7001.clients.us.xxxx.xxxxx.com", "ModifyingApplication": "", "SettingType": "2", "Keywords": "0x8000000000000000", "row_id": 14, "Guid": "d1bc9aff-2abf-4d71-9146-ecb2a986eb85", "Qualifiers": "0"}
}, {
You can use filtered query in python:
t1 = TermFilter(‘ProcessID’, ‘2060’)
t2 = TermFilter(‘EventID’, ‘2002’)
f = ANDFilter([t1, t2])
q = FilteredQuery(MatchAllQuery(), f)
results = conn.search(q)