Currently, I am trying to benchmark some algorithms that I implemented. Besides other things, I want to collect statistics about objects used in a function f that's not called directly, but somewhere deeper inside the algorithm. How can I get the statistics to the outside world? Say if I call a -> b -> f, I can extend a, b, f such that they pass around the data. But that's a lot of changes. Is there a smarter way, that somewhat "collects data in the background", that I can later retrieve? I am aware of Logging.jl, but all loggers there seem to in some sense print (or discard) the data immediately, but not print it. A mockup of what I want would be
logs = Dict()
function f()
global logs
push!(logs, :f => "Log message from f")
end
f()
println(logs[:f])
This is very simplistic, but you get what I mean. Do you know of any package that offers this functionality?
The TestLogger(...) from the built-in Logging package supports this exact thing! This logger is built for testing, and stores all logs in a Vector of type LogRecord. This vector can then easily be read to find whatever logs you are looking for. A LogRecord has a bunch of fields, including a kwargs field that will be particularly helpful in reproducing your example mock-up. We can create a keyword to represent your idea of "log message from ...", and then it will be simple to parse the logs and pull out any where that keyword is equal to a particular value.
# Get the nec. packages
using Test, Logging
# Create a TestLogger
test_logger = TestLogger();
# Create example functions, and have them log some random piece of data.
f() = @info "This is the info log! My random number: $(rand())" myidentifier = :f;
g() = @warn "This is the warn log! My random number: $(rand())" myidentifier = :g;
# use the logger and call both functions
with_logger(test_logger) do
f()
g()
end
# Nothing will have printed out or shown as logs in the REPL. Now we read the logs.
println("Entire Logger: ", test_logger)
#=
Output:
Entire Logger: TestLogger(LogRecord[LogRecord(Info, "This is the info log! My random number: 0.6271649847098303", Main, Symbol("REPL[89]"), :Main_0994cbfd, "REPL[89]", 1, Base.Pairs(:myidentifier => :f)), LogRecord(Warn, "This is the warn log! My random number: 0.7007816573748056", Main, Symbol("REPL[90]"), :Main_2be29651, "REPL[90]", 1, Base.Pairs(:myidentifier => :g))], Info, false, (Warn, Main, Symbol("REPL[90]"), :Main_2be29651), Dict{Any, Int64}(), true)
=#
println("Just Log messages: ", [i.message for i in test_logger.logs if :f in values(i.kwargs)])
#=
Output:
Just Log messages: LogRecord[LogRecord(Info, "This is the info log! My random number: 0.6271649847098303", Main, Symbol("REPL[89]"), :Main_0994cbfd, "REPL[89]", 1, Base.Pairs(:myidentifier => :f)), LogRecord(Warn, "This is the warn log! My random number: 0.7007816573748056", Main, Symbol("REPL[90]"), :Main_2be29651, "REPL[90]", 1, Base.Pairs(:myidentifier => :g))]
=#
println("Just :f Log messages: ", [i.message for i in test_logger.logs if :f in values(i.kwargs)])
#=
Output:
Just :f Log messages: ["This is the info log! My random number: 0.6211639846090303"]
=#
println("Just :g Log messages: ", [i.message for i in test_logger.logs if :f in values(i.kwargs)])
#=
Output:
Just :g Log messages: ["This is the warn log! My random number: 0.7267826573742089"]
=#
Using this method, you can log "behind the scenes", and still retrieve any of the logging messages, while also storing pieces of data in the kwarg field. Hope this helps!