Feature: use an existing list of names for new variables about dataframesmeta.jl

[pdeffebach]

I got annoyed by this in dplyr again today and want to think about a way we could do this.

We could see if a variable is defined in the local scope of the select function, and then change behavior based on that.

macro isdefined(var)
           quote
               try
                   local _ = $(esc(var))
                   true
               catch err
                   isa(err, UndefVarError) ? false : rethrow(err)
               end
           end
       end

[pdeffebach]

Or you could use some version of the ^(x) syntax that is on the RHS of expressions, and expand it so it also works on the LHS of expressions.

However I should really do more research into dplyr to make sure this approach isn't the exact same as what they are doing.

[davidagold]

dplyr leans heavily on R's ability to examine enclosing environments. You won't be able to emulate what it does, precisely. Look at vignette("programming") from the dplyr package. You will find how to do what (I think) you are trying to do:

> library(tidyverse)

> titanic <- Titanic %>% as.tibble

> head(titanic)
# A tibble: 6 x 5
  Class Sex    Age   Survived     n
  <chr> <chr>  <chr> <chr>    <dbl>
1 1st   Male   Child No          0.
2 2nd   Male   Child No          0.
3 3rd   Male   Child No         35.
4 Crew  Male   Child No          0.
5 1st   Female Child No          0.
6 2nd   Female Child No          0.

> map(colnames(titanic), quo) %>%
+   map(~ transmute(titanic, !!quo_name(.) := 5)) %>%
+   bind_cols %>%
+   head
# A tibble: 6 x 5
  Class   Sex   Age Survived     n
  <dbl> <dbl> <dbl>    <dbl> <dbl>
1    5.    5.    5.       5.    5.
2    5.    5.    5.       5.    5.
3    5.    5.    5.       5.    5.
4    5.    5.    5.       5.    5.
5    5.    5.    5.       5.    5.
6    5.    5.    5.       5.    5.

[davidagold]

Also, @pdeffebach, I think your second suggestion is much preferable to your first. IMHO, trying to guess what a user intends based on what's defined in the local environment is generally a bad idea.

recedes back into woodwork

[pdeffebach]

I appreciate that dplyr does this, and I understand how to use it more or less, but I am always frustrated with having to remember the syntax that I end up just using $ in the end, or avoid functional programming at all.

It seems to be that Julia could be able to do this better. I'm curious if the devs think that adding a ^(x) syntax is something thats easy to do, because it seems like it just means adding an
if head(expr) == ^ somewhere in one of the many helper functions.

[tshort]

It's doable. I wouldn't vote for inclusion, though. Seems like a low priority for increased code complexity. Seems like just doing the loop or writing a function to do the operation would be cleaner. That said, one of the other maintainers might want to include it.

Or, if it turns out the implementation is simple enough, that might convince me (or others).

[pdeffebach]

Fair enough. I could live with using df[...] for functions. But it would be nice!

One thing to note is that in dplyr, tibbles and data.frames have slightly different behavior in functions as well, with tibbles return dataframes more frequently than data.frames. Julia doesn't have this annoyance.

[davidagold]

One thing to note is that in dplyr, tibbles and data.frames have slightly different behavior in functions as well, with tibbles return dataframes more frequently than data.frames. Julia doesn't have this annoyance.

Personally, I try never to return a data.frame, but that's just me =p

[pdeffebach]

I spent some time on this last night. I think there are two issues that are at play. Two things are impossible to with DataFramesMeta but for different reasons.

1. Modifying an existing variable with @transform. This should be feasible with the _I_ function.

julia> t = :x3
:x3

julia> @transform(df, _I_(t) = 100)
ERROR: syntax: keyword argument is not a symbol: "_I_(t)"

This can probably be resolved by fixing an overly restrictive error message somewhere.

2. Creating a new variable based on an old name. This will require a new call to the onearg function somewhere, probably here. We need some syntax that goes from

var = y
g(var) -> y #rather than
_I_(var) -> :y

[pdeffebach]

I think this is more trouble then its worth.

@tranform(df, _T_(var) = _I_(xx) + 100) is both ugly and goes against the rest of Julia's rules for assignment.

This is a perfectly feasible workflow. _I_() is pretty ugly but I understand why its necessary.

 function makedata(df, y, xx)
       df[y] = @with df begin
       _I_(xx) + 100
       end
end