marklogicmarklogic-dhfmarklogic-10

How to read log file of all the hosts in a marklogic cluster and search for a particular keyword and count the results

I had a requirement where I need to read the application log files of all host in cluster and search for a particular keyword and get the count of all.

Log file in question is 9005_ErrorLog.txt and will look like below (I am searching for keyword "SERVICE-EXIT: ManchesterUnited")

Host 1 (9005_ErrorLog.txt),

2023-03-15 00:48:01.804 Info: SERVICE-ENTRY: ManchesterUnited
2023-03-15 00:48:01.840 Info: SERVICE-EXIT: ManchesterUnited
2023-03-15 00:53:00.874 Info: SERVICE-ENTRY: ManchesterUnited
2023-03-15 00:53:00.906 Info: SERVICE-EXIT: ManchesterUnited
2023-03-15 01:28:01.946 Info: SERVICE-ENTRY: ManchesterUnited
2023-03-15 01:28:01.954 Info: SERVICE-EXIT: ManchesterUnited

no of counts matching in host 1 9005_ErrorLog.txt = 3

Host 2 (9005_ErrorLog.txt)

2023-03-15 00:48:01.804 Info: SERVICE-ENTRY: ManchesterUnited
2023-03-15 00:48:01.840 Info: SERVICE-EXIT: ManchesterUnited

no of counts matching host 2 9005_ErrorLog.txt = 1

Host 3 (9005_ErrorLog.txt)

2023-03-15 00:48:01.804 Info: SERVICE-ENTRY: ManchesterUnited
2023-03-15 00:48:01.840 Info: SERVICE-EXIT: ManchesterUnited

no of counts matching host 3 9005_ErrorLog.txt = 1

Total Count = 5 ( I need this to be the final result of Xquery code)

I wrote below script to get the count from individual host file

xquery version "1.0-ml";

let $error-log := "/ML-master/Logs/9005_ErrorLog.txt"
let $keyword := "SERVICE-EXIT: ManchesterUnited"


for $line in fn:tokenize(xdmp:filesystem-file($error-log), '\r?\n')
where contains($line, $keyword)

return
$line

Also, to get all the log file from single xquery code in Query console, I am able to do that using below code

xquery version "1.0-ml";
for $hid in xdmp:hosts()
let $h := xdmp:host-name($hid)
return 
xdmp:filesystem-file("file://" || $h || "/"||xdmp:data-directory($hid)||"/Logs/9005_ErrorLog.txt")

How can I integrate both the scripts into one single Xquery module which will search a keyword "SERVICE-EXIT: ManchesterUnited" in all the 3 host 9005_ErrorLog.txt files and count the no of matches and give us the final count of matches = 5.

Solution

  • I also agree with David suggestions. But if you are still looking something as per your code snippet, this should work:

    let $error-log := "/ML-master/Logs/9005_ErrorLog.txt"
return
sum(
  for $hid in xdmp:hosts()
  let $h := xdmp:host-name($hid)
  return 
    count(
      for $line in fn:tokenize(xdmp:filesystem-file("file://" || $h || "/"||xdmp:data-directory($hid)||"/Logs/9005_ErrorLog.txt"), '\r?\n')
      where contains($line, $keyword)
      return $line
    )
)