Very slow `@astable` macro outside a function about dataframesmeta.jl

[pdeffebach]

Thank you for your bug report!

It is true that @astable will be slower outside of a function. The reason is that DataFramesMeta.jl creates anonymous functions which and calling them requires specialization on input and output type. For the @astable behavior, the output type is complicated, as the names and values are stored in type information of a NamedTuple.

That said, I can't replicate your issue on version 1.9. I get roughly comparable timings.

julia> @time @chain df begin
               @rtransform @astable begin
                       b = :a * 10
                       c = mean(b)
                       :d = b - c
               end
       end;
  0.052417 seconds (64.17 k allocations: 3.484 MiB, 97.60% compilation time)

julia> @time @chain df begin
               @rtransform(:b = :a * 10)
               @rtransform(:c = mean(:b))
               @rtransform(:d = :b - :c)
               @select(:a, :d)
       end;
  0.045937 seconds (45.52 k allocations: 3.869 MiB, 96.93% compilation time)

Can you double check your measurements and give me your version info?

[mbataillou]

So I double checked the results and found something potentially interesting!

In the REPL I get a similar timing just as you, but the above timings came from running inside a Pluto notebook.
To triple check I ran the above in a Pluto notebook, and there you get the 40x slow down again.

Seems like the issue is using @astable outside a function in a Pluto notebook.

Here is my version info

Julia Version 1.9.3
Commit bed2cd540a1 (2023-08-24 14:43 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: macOS (x86_64-apple-darwin22.4.0)
  CPU: 12 × Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 6 on 12 virtual cores
Environment:
  JULIA_DEPOT_PATH = /Users/xxx/.julia:/Applications/Julia-1.9.app/Contents/Resources/julia/local/share/julia:/Applications/Julia-1.9.app/Contents/Resources/julia/share/julia
  JULIA_LOAD_PATH = @:@v#.#:@stdlib
  JULIA_REVISE_WORKER_ONLY = 1

[pdeffebach]

That's pretty frustrating. Can you post an issue with Pluto.jl? (Or their internal tracker or whatever they use) and I can cross-link with this issue?

[pdeffebach]

Bumping this @mbataillou did you ever file an issue with Pluto.jl?