juliafor-comprehensionsequence-generators

Why is generator not identical to comprehension in 2D comprehension


What is the logic between one generator producing a 1D array and the other a 2D array in the following expressions (version 0.6.2):

julia> collect((a,b,c) for (a,b) in ((i,j) for i in 1:2 for j in 3:4), c in 5:6)
8-element Array{Tuple{Int64,Int64,Int64},1}:
 (1, 3, 5)
 (1, 4, 5)
 (2, 3, 5)
 (2, 4, 5)
 (1, 3, 6)
 (1, 4, 6)
 (2, 3, 6)
 (2, 4, 6)

julia> collect((a,b,c) for (a,b) in [(i,j) for i in 1:2 for j in 3:4], c in 5:6)
4×2 Array{Tuple{Int64,Int64,Int64},2}:
 (1, 3, 5)  (1, 3, 6)
 (1, 4, 5)  (1, 4, 6)
 (2, 3, 5)  (2, 3, 6)
 (2, 4, 5)  (2, 4, 6)

The only difference is replacing the generator in the first expression with a comprehension in the second expression.


Solution

  • ((i,j) for i in 1:2 for j in 3:4) and [(i,j) for i in 1:2, j in 3:4] are parsed to expressions with a final flatten operation. The expression [(i,j) for i in 1:2, j in 3:4] is collected to a vector giving it iteratorsize HasShape so it behaves nicely in products.

    Base.iteratorsize(f(i,j) for i in 1:2 for j in 3:4) in general is SizeUnknown because we do not know the return type of f and operators in products with SizeUnknown make the entire product SizeUnknown and hence flat when collected.

    You might also be looking for

    julia> collect((a,b,c) for (a,b) in ((i,j) for i in 1:2, j in 3:4), c in 5:6)
    2×2×2 Array{Tuple{Int64,Int64,Int64},3}
    
    julia> collect((a,b,c) for (a,b) in [(i,j) for i in 1:2, j in 3:4], c in 5:6)
    2×2×2 Array{Tuple{Int64,Int64,Int64},3}:
    

    (no flatten involved in forming the first generator and everything goes through fine).

    Edit: I think now Base.iteratorsize(f(i,j) for i in 1:2 for j in 3:4) can be HasShape, I'll take a shot at adding that behaviour to flatten.