Error `.*` not defined with `@by` about dataframesmeta.jl

[pdeffebach]

Thank you for the bug report!

I think I understand what's going on but it will take a few days to look into it more fully.

[bkamins]

@pdeffebach The issue is caused by the automatic-composition mechanism you have recently introduced (that avoids creation of anonymous functions):

julia> @macroexpand @by(
           df,
           :x,
           :a = maximum(:y .* :z)
       )
quote
    #= C:\Users\bogum\.julia\packages\DataFramesMeta\kWkhX\src\macros.jl:2056 =#
    (DataFrames).combine((DataFrames.groupby)(df, :x), DataFramesMeta.make_source_concrete([:y, :z]) => ((∘)(maximum, .*) => :a))
end

[pdeffebach]

It's more mysterious than that, though. On 1.7.2

julia> @by(
           df,
           :x,
           :a = maximum(:y .* :z)
       )
ERROR: UndefVarError: .* not defined
Stacktrace:
 [1] top-level scope
   @ ~/.julia/packages/DataFramesMeta/kWkhX/src/macros.jl:2056

julia> (DataFrames).combine((DataFrames.groupby)(df, :x), DataFramesMeta.make_source_concrete([:y, :z]) => ((∘)(maximum, .*) => :a))
3×2 DataFrame
 Row │ x      a
     │ Int64  Bool
─────┼──────────────
   1 │     1   true
   2 │     2   true
   3 │     3  false

[bkamins]

They display the same, but are parsed differently. The @macroexpand version in the crucial part is:

Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: ∘ (function of type typeof(∘))
    2: Symbol maximum
    3: Symbol .*

while the manually passed version is parsed as:

Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol ∘
    2: Symbol maximum
    3: Expr
      head: Symbol .
      args: Array{Any}((1,))
        1: Symbol *

[pdeffebach]

Okay, so one solution would be to de-construct .* etc. to Expr(:., :*)

I will file an issue in Julia Base so people are aware of this, which I think is a bug.

[bkamins]

But probably it will take a long time to fix it in Base (this is unfortunately my experience), so we need some work-around in DataFramesMeta.jl anyway.

[tcovert]

is this still busted? what is a sensible work around?

[pdeffebach]

Sorry for not addressing this.

A workaround, which isn't really sensible, is to add "junk" in the expression

julia> @by(
           df,
           :x,
           :a = begin 1; maximum(:y .* :z) end
       )

I apologize for this. I need to be more focused on getting these parsing errors fixed.

[tcovert]

totally understand, just making sure I hadn't missed another solution somewhere.

FYI the problem exists in @combine() expressions after a groupby as well.

[pdeffebach]

Yes, it should exist for all macros.

[pdeffebach]

I'm having a hard time making an MWE

julia> macro make_composed(x, y)
           esc(:(∘($x, $y)))
       end;

julia> function make_composed2_helper(x, y)
           Expr(:call, ∘, x, y)
       end;

julia> macro make_composed2(x, y)
           esc(make_composed2_helper(x, y))
       end
@make_composed2 (macro with 1 method)

julia> MacroTools.@macroexpand(@make_composed(first, .*))
:(first ∘ (.*))

julia> MacroTools.@macroexpand(@make_composed2(first, .*))
:((∘)(first, (.*)))

julia> MacroTools.@macroexpand(@make_composed2(first, .*)) |> dump
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: ∘ (function of type typeof(∘))
    2: Symbol first
    3: Expr
      head: Symbol .
      args: Array{Any}((1,))
        1: Symbol *

julia> MacroTools.@macroexpand(@make_composed(first, .*)) |> dump
Expr
  head: Symbol call
  args: Array{Any}((3,))
    1: Symbol ∘
    2: Symbol first
    3: Expr
      head: Symbol .
      args: Array{Any}((1,))
        1: Symbol *

[pdeffebach]

It's really hard to tell the rules about when .* survives and when it becomes Expr(., *). I will have to explore more.