apldyalog

Dyalog APL: what is the problem with loops?

I'm trying to process data received via API using a loop, but the loop is very slow. Is this a Dyalog bug?

res←getBinanceSymbols

 res←⍬
 baseToken←'USDT'

 tmpSymbols←(⎕JSON(HttpCommand.Get'api.binance.com/api/v3/exchangeInfo').Data).symbols

 :For sym1 :In tmpSymbols
     ⎕←sym1.symbol
 :EndFor

see here → https://www.youtube.com/watch?v=oUl4oixd3Ds

Anyway, my complete task is to form pairs of the form XXXYYY XXXZZZ ZZZYYY, and I've tried to do this using nested loops (see below), but it takes forever...

 res←getBinanceSymbols

 res←⍬
 baseToken←'USDT'

 tmpSymbols←(⎕JSON(HttpCommand.Get'api.binance.com/api/v3/exchangeInfo').Data).symbols

 :For sym1 :In tmpSymbols
     Sym1_Token1←sym1.baseAsset
     Sym1_Token2←sym1.quoteAsset

     :If Sym1_Token1≡baseToken
         :For sym2 :In tmpSymbols
             Sym2_Token1←sym2.baseAsset
             Sym2_Token2←sym2.quoteAsset

             :If Sym1_Token1≡Sym2_Token2
                 :For sym3 :In tmpSymbols
                     Sym3_Token1←sym3.baseAsset
                     Sym3_Token2←sym3.quoteAsset

                     :If Sym2_Token1≡Sym3_Token1
                     :AndIf Sym3_Token2≡Sym1_Token2
                     :AndIf Sym1_Token1≢Sym1_Token2
                     :AndIf Sym2_Token1≢Sym1_Token1
                     :AndIf Sym2_Token1≢Sym1_Token2

                         res,←⊂((Sym1_Token1⍪Sym1_Token2)(Sym2_Token1⍪Sym1_Token2)(Sym2_Token1⍪Sym1_Token2))

                     :EndIf
                 :EndFor
             :EndIf
         :EndFor
     :EndIf
 :EndFor

Question 1: Why is the loop so slow? I work on a computer with Intel Core i5 11th and 40GB DDR4. It's just unthinkable!

Question 2: How can I solve my problem without resorting to nested loops? In a more the APL way

Solution

  • The reason for unreasonable slowness is what I call a dangling reference. Dyalog objects contain a reference to their parent object. tmpSymbols therefore contains a reference to Data but Data doesn't otherwise live anywhere per se. This continuously causes the garbage collector to kick in and investigate whether Data can be expunged, only to find, every time, that it cannot, because tmpSymbols needs it to remain in existence. It is this contant garbage collection activity that is the cause of the slowdown.

    So, how do you work around this? Keep a direct reference to Data, as then there's no question that it doesn't need to be expunged:

     res←getBinanceSymbols

 res←⍬
 baseToken←'USDT'

⍝ Insert "data←" ↓
 tmpSymbols←(data←⎕JSON(HttpCommand.Get'api.binance.com/api/v3/exchangeInfo').Data).symbols

 :For sym1 :In tmpSymbols
     ⎕←sym1.symbol
 :EndFor