I am trying to parallelize a matrix rotation in Julia. I'm using the Distributed package, but I'm still getting an error. My original (serial) code is as follows:
using Rotations
using ImageTransformations
using CoordinateTransformations
data_arr = rand(25,25)
angle_arr = -80:0.01:80
for a in angle_arr
trfm = recenter(Rotations.RotMatrix(deg2rad(a)),center(data_arr))
rot_arr = warp(data_arr, trfm);
rot_arr[rot_arr.<0].=0
end
After attempting to apply the Distributed
package my code is now this:
using Distributed
@everywhere using Rotations
@everywhere using ImageTransformations
@everywhere using CoordinateTransformations
data_arr = rand(25,25)
angle_arr = -80:0.01:80
addprocs(8)
nheads = @distributed (+) for a in angle_arr
trfm = recenter(RotMatrix(deg2rad(a)),center(data_arr))
rot_arr = warp(data_arr, trfm);
rot_arr[rot_arr.<0].=0
end
which still gives me the following error. I'm a bit confused, since I thought I was defining the Rotations
package containing RotMatrix
everywhere. What am I missing?
ERROR: LoadError: TaskFailedException
Stacktrace:
[1] wait
@ .\task.jl:334 [inlined]
[2] fetch
@ .\task.jl:349 [inlined]
[3] preduce(reducer::Function, f::Function, R::UnitRange{Int64})
@ Distributed C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\macros.jl:274
[4] top-level scope
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\macros.jl:286
[5] eval
@ .\boot.jl:373 [inlined]
[6] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
@ Base .\loading.jl:1196
nested task error: On worker 2:
UndefVarError: RotMatrix not defined
Stacktrace:
[1] top-level scope
@ none:1
[2] eval
@ .\boot.jl:373
[3] #103
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\process_messages.jl:274
[4] run_work_thunk
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\process_messages.jl:63
[5] run_work_thunk
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\process_messages.jl:72
[6] #96
@ .\task.jl:423
...and 8 more exceptions.
Stacktrace:
[1] sync_end
@ .\task.jl:381
[2] macro expansion
@ .\task.jl:400 [inlined]
[3] remotecall_eval
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\macros.jl:219
[4] macro expansion
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\macros.jl:203 [inlined]
[5] macro expansion
@ C:\Git\Repositories\water\src\Swamp.jl:146 [inlined]
[6] #15
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\macros.jl:288
[7] #106
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\process_messages.jl:278
[8] run_work_thunk
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\process_messages.jl:63
[9] macro expansion
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\process_messages.jl:278 [inlined]
[10] #105
@ .\task.jl:423
Stacktrace:
[1] remotecall_fetch(::Function, ::Distributed.Worker, ::Function, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ Distributed C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\remotecall.jl:469
[2] remotecall_fetch
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\remotecall.jl:461 [inlined]
[3] #remotecall_fetch#158
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\remotecall.jl:496 [inlined]
[4] remotecall_fetch
@ C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\remotecall.jl:496 [inlined]
[5] (::Distributed.var"#169#170"{typeof(+), var"#15#16", UnitRange{Int64}, Vector{UnitRange{Int64}}, Int64, Int64})()
@ Distributed C:\Users\apung\AppData\Local\Programs\Julia-1.7.2\share\julia\stdlib\v1.7\Distributed\src\macros.jl:270
in expression starting at C:\Git\Repositories\water\src\Swamp.jl:137
@everywhere
should be called after addprocs
- otherwise you have no workers to load your libraries. The workers added later have no libraries loaded.
using Distributed
addprocs(8)
@everywhere using Rotations
@everywhere using ImageTransformations
@everywhere using CoordinateTransformations
Your loop is not collecting anything - perhaps you want to put rot_arr
in the last line just before end
. In that case the values of rot_arr
would get summed.