thautwarm / mlstyle.jl

Julia functional programming infrastructures and metaprogramming facilities

MLStyle.jl

Providing intuitive, fast, consistent and extensible functional programming infrastructures, and metaprogramming facilities.

Check everything in the documentation website.

Preview

using MLStyle

@data Shape begin # Define an algebraic data type Shape
    Rock
    Paper
    Scissors
end

# Determine who wins a game of rock paper scissors with pattern matching
play(a::Shape, b::Shape) = @match (a, b) begin
    (Paper,    Rock)      => "Paper Wins!";
    (Rock,     Scissors)  => "Rock Wins!";
    (Scissors, Paper)     => "Scissors Wins!";
    (a, b)                => a == b ? "Tie!" : play(b, a)
end

P.S: When preferring Base.@enum than MLStyle.@data, you need this to pattern match on Julia Base.@enum.

Benchmarks

Arrays

• code: matrix-benchmark/bench-array.jl

Tuples

• code: matrix-benchmark/bench-tuple.jl

Data Types

• code: matrix-benchmark/bench-datatype.jl

Extracting Struct Definitions

• code: matrix-benchmark/bench-structfields.jl

Misc

• code: matrix-benchmark/bench-misc.jl

An Example from Match.jl Documentation

• code: matrix-benchmark/bench-vs-match.jl

Acknowledgements

Thanks to all individuals referred in Acknowledgements!

mlstyle.jl's Issues

Data.* design

In Data.List:

module List
<define abstract type List!>
<define ADT of List>
end

Just as the implementation of List, when implementing other data structures, use this template:

module <Name of Data>
<define abstract type Name!>
<define ADT of Data>
end

Error message for non-exhaustive patterns

Below might be a common problem with people new to pattern matching (or rusty with it like me, apparently). It would be really nice to have a clear error message that the problem here is with a non-exhaustive pattern match :)

julia> @match :(sin(x)) quote
           :($f($(args...))) => args
       end
ERROR: LoadError: MethodError: no method matching length(::Nothing)
Closest candidates are:
  length(::Core.SimpleVector) at essentials.jl:561
  length(::Base.MethodList) at reflection.jl:801
  length(::Core.MethodTable) at reflection.jl:875
  ...
Stacktrace:
 [1] _similar_for(::UnitRange{Int64}, ::Type{Any}, ::Nothing, ::Base.HasLength) at ./array.jl:517
 [2] _collect(::UnitRange{Int64}, ::Nothing, ::Base.HasEltype, ::Base.HasLength) at ./array.jl:550
 [3] collect(::Nothing) at ./array.jl:544
 [4] (::getfield(MLStyle.MatchCore, Symbol("##50#52")){Expr,Symbol,Module})(::LineNumberNode) at /home/chad/.julia/packages/MLStyle/ksIXg/src/MatchCore.jl:249
 [5] (::getfield(MLStyle.Toolz, Symbol("##1#2")){getfield(MLStyle.MatchCore, Symbol("##50#52")){Expr,Symbol,Module}})(::MLStyle.MatchCore.config) at /home/chad/.julia/packages/MLStyle/ksIXg/src/Internal/Toolz.jl:25
 [6] $(::getfield(MLStyle.Toolz, Symbol("##1#2")){getfield(MLStyle.MatchCore, Symbol("##50#52")){Expr,Symbol,Module}}, ::MLStyle.MatchCore.config) at /home/chad/.julia/packages/MLStyle/ksIXg/src/Internal/Toolz.jl:8
 [7] (::getfield(MLStyle.Toolz, Symbol("##1#2")){getfield(MLStyle.MatchCore, Symbol("##48#49")){Expr,Module}})(::MLStyle.MatchCore.config) at /home/chad/.julia/packages/MLStyle/ksIXg/src/Internal/Toolz.jl:25
 [8] $(::getfield(MLStyle.Toolz, Symbol("##1#2")){getfield(MLStyle.MatchCore, Symbol("##48#49")){Expr,Module}}, ::MLStyle.MatchCore.config) at /home/chad/.julia/packages/MLStyle/ksIXg/src/Internal/Toolz.jl:8
 [9] gen_match(::Expr, ::Expr, ::Module) at /home/chad/.julia/packages/MLStyle/ksIXg/src/MatchCore.jl:220
 [10] @match(::LineNumberNode, ::Module, ::Any, ::Any) at /home/chad/.julia/packages/MLStyle/ksIXg/src/MatchCore.jl:238
in expression starting at REPL[62]:1

Expr template pattern?

In MacroTools there is @capture that accept an expression template and match the template. Here is a simple proposal to support expression pattern matching:

@match expr begin
    name_::Int => "this is a type annotation"
    f_{x_}(vars__) => "this is a function accepts one type parameter"
    # etc...
end

The template syntax will just follow @capture, _ means a variable, __ means a list.

Or in Match.jl, there is

@match expr begin
       Expr(:(::), [name, typename]) => (name, typename)
       _ => "failed"
end

This fails in MLStyle

Introduce a syntax extension system

@ext.+ [ext.pattern_matching(case)] begin
    case(value) do
           (a, b) => a + b
           (a, b, c) => a + b + c
           _ => 0
    end
end

@ext.define my_common_extensions [ext.pattern_matching(case)]
@ext.+ [my_common_extensions] module Mod
     case(value) do
         ...
     end
     @ext.- [my_common_extensions] begin
          case(f, a) = f(a)
          case(1) do x
                x => 1
          end # produces `1 => 1`
     end 
end

Error tagging new release

The tag name "0.1.3" is not of the appropriate SemVer form (vX.Y.Z).
cc: @thautwarm

else branch of @when

Currently @when can only handle with a single branch with multiple destructuring, but can do nothing to the scenarios an else is needed. It turns out @when is not an equivalent of if-let in Rust.

The enhancement should be performed as soon as possible, but the obstacle is the syntax design.
I don't have an idea about how to make an elegant and concise one as if-let equivalent.

Any suggestions are welcome.

Referential transparency

This is not specific to MLStyle, but if you have time I hope you might be able to help. There's a sort of "design pattern" I see you using that I'd like to better understand.

Macros interpret their arguments as expressions, so @foo x will try to do manipulations on the x as a symbol, ignoring any value x is bound to.

For things like match, you somehow pass an expression, and any non-expression gets looked up in the current local context. This seems to me a much better use of macros (or at least better suited to my use case).

I'd love to see a minimal example of this approach. I've looked at the source code for match, but I'm not sure which parts are essential to making things work this way.

The symbol nothing is regarded as capturing

@match 1 begin
  nothing => 1
end # => 1

The reason for this is, a symbol named nothing is binded instead of comparing with the literal nothing. I used to think all symbols named nothing should be parsed into nothing literals.
That's an unexpected bug. Should be patched as soon as possible.

match dot expression

JuliaLang/julia#6104 (comment)

There is a problem that QuoteNode would be automatically evaluated, so when source ast contains QuoteNode, pattern matching compilation generates the same QuoteNode and inserts it into the generated ast. The QuoteNode generated by pattern compilation would then be auto evaluated and finally we got a a != QuoteNode(a), in another words, matching fails.

To solve this problem we introduce a QuotePattern to avoid auto evaluation of QuoteNode generated by patterns.

Following the knowledge given by the the first referenced issue, this technically works.

Feature Proposal

The @def macro in the Incoming Features is very interesting:

@def f {
   ((a, b), true)   => <body1>
   (nothing, false) => <body2>

   _                => <body3>
}

I'm wondering if you can also support state sharing between different method implementations? Like bellow:

@def f {
    let x = 1
       ((a, b), true)   => <body1>
       (nothing, false) => <body2>

       _                => <body3>
}

Thanks!

update documentation and resolve compatibility(v0.1 and v0.2) issues

Since the syntax of some builtin patterns have changed(| -> ||, & -> &&, as-pattern has been removed and you can use <pat1> && a to perform capturing), documentation and compatibility should be taken into consideration.

Strings cannot be compared via `===` in Julia v1.1-

Tests on Julia v1.0 failed

Allow destructuring normal structs

While Match.jl provides such functionalities, in order to support users' redefinition of the way to destructure a struct, we disabled the default way to destructure normal structs like:

struct A
  a_1
  a_2
end

@match A(1, 2) begin
    A(1, 2) => ...
end

I have an idea to solve this: register the structs to somewhere which marks all the structs that can be destructured.

struct A
  a_1
  a_2
end

@default_pattern A
 
@match A(1, 2) begin
    A(1, 2) => ...
end

Optional ast match ?

@thautwarm Sorry that's more a basic question than an issue. I'm playing a bit with the package and it looks super interesting. I'm trying to change the ast match example from the doc to include where statements and return type annotation:

ast = quote
    function f(a::Int, b::Float16, d::T)::Int128 where {T}
      let d = a + b + c, e = x -> 2x + d
          e(d)
      end
    end
end

@match ast begin
    quote
        $(::LineNumberNode)
        function $funcname($(args...))::$(out) where {$(cov)}
            $(block...)
        end
    end =>
         Dict(:funcname => funcname,
              :args     => args,
              :out      => out,
              :cov      => cov,
        )
end

Is it possible to say that the where {$(cov)} for example is optional ? Or do I need to make patterns for each subcases ?

Thanks!

macro capture

To make use of the fast speed of this package, a capture macro like MacroTools would be preferred. And instead of underscore, we could use current match's expr template style.

@matchast wildcard matching might cause misunderstanding

In @matchast, the wildcard in first layer of quotation is not the true wildcard, it is gonna matching a symbol :_.

@matchast :(1 + 1) begin
    _ => 1
end # raise exception

You can use wildcard in @matchast via $_.

@matchast :(1 + 1) begin
    $_ => 1
end # => 1

use = instead of in for the as-pattern?

It might make sense to use = or := for the as-pattern instead of in. I mean, this is essentially assignment inside a pattern, right? Could look like eg:

@match (1, 2) begin
    c := (a, b) => c[1] == a && c[2] == b
end

about benchmark

I've already made a comparison between MacroTools.jl and MLStyle.jl, and the latter seems to be 3 times faster in dev branch with 1/5 space cost and 5 times faster in pattern-to-inline-function branch with 1/5 space cost(a bit less than dev).

The test snippet(check "benchmark.jl" in the root directory) I used comes from the README document of MacroTools.jl, and the gap could be much larger when the case becomes more complex.

I'm planning to make a thorough benchmark experiment to give a rational evaluation about the performance gain of MLStyle.jl.

Dict pattern failed when value is nothing

Following case failed for a some reason. A quick fix would be performed at once.

x = Dict(:a => nothing)
@math x begin
    Dict(:a => a) => 1
end

`if let` support

Just like that in Rust.

if let (a, b) = data {
    # do something if the case is matched.
}

perspective syntax proposals:

@when (a, b) = data begin
    # do something if the case is matched.
end
# or 
@if_let (a, b) = data begin
   # do something if the case is matched.
end

feel free to make a supplementary.

rm Manifest

just add it to .gitignore this is not necessary , but you can keep it if you want anyway...

can not use @match inside another macro

MWE: copying what's in here: https://github.com/thautwarm/MLStyle-Playground/blob/master/StaticCapturing.jl

to a module. The scoping is not correct.

A sub-package to provide some easy-to-use tools

e.g., #37

Array pattern performance issues

MLStyle.jl/src/Pervasives.jl

Line 306 in 47fc37b

function check_generic_array(body)

Current one makes a trade-off for the performance of expr template patterns and normal array patterns.

Following one could be faster when it comes to normal array destructing.

 function check_generic_array(body)
            @format [Array, SubArray, UnitRange, Tuple, Int] quote

                @inline __L__ function NAME(TARGET :: Array{T, 1}) where {T}
                    body
                end

              @inline __L__ function NAME(TARGET :: SubArray{T, 1, Array{T, 1}, Tuple{UnitRange{Int}}, true}) where {T}
                    body
                end

                @inline __L__ function NAME(_)
                    failed
                end

                NAME(tag)

            end
end

clarify package motivation, purpose, and scope in the readme

It would be great if you could write a paragraph or two in the README about what motivated this package, what it does (broadly), and what the underlying design principle is.

Is nil(State) a typo?

MLStyle.jl/src/toolz.jl

Line 44 in 5344f63

v = nil(State)

Error with pattern match guards

julia> using MLStyle
[ Info: Precompiling MLStyle [816b5f4a-9e2c-11e8-0bf8-3d61c8eadc99]

julia>         capture_match(x) = @match x begin
                           x{x > 0} => x + 1
                           x{x < 0} => x - 1
                          _ => 0
                  end
capture_match (generic function with 1 method)

julia> capture_match(3)
ERROR: UndefVarError: ret not defined
Stacktrace:
 [1] macro expansion at /mnt/d/workspace/github/MLStyle.jl/src/Match.jl:113 [inlined]
 [2] macro expansion at ./REPL[5]:2 [inlined]
 [3] macro expansion at /mnt/d/workspace/github/MLStyle.jl/src/Match.jl:179 [inlined]
 [4] capture_match(::Int64) at ./REPL[5]:1
 [5] top-level scope at none:0

Currently several test errors like above are skipped(there are 5 test cases are broken as you can see in the test result). You may change @test_skip into @test in the match.jl file after the error get fixed.

Test Summary: | Pass  Broken  Total
match         |   19       5     24
Test Summary: | Pass  Total
pattern       |    1      1
Test Summary: | Pass  Total
adt           |    3      3
Test Summary: | Pass  Total
@case         |    3      3
   Testing MLStyle tests passed 

your work is well appreciated

This matters, and your effort drives other's active imagination as software is designed.

Best,
Jeffrey Sarnoff

Introduce Cassette.jl into codegen to achieve higher performance and simplify the generated codes

Literal functions don't @match as expected

Hi,

Today I was trying to match a "literal function" (like x -> f(x)), and getting some strange results:

julia> @match :(a -> a+1) begin
           :($a -> $b) => "ok"
           x           => "nope"
       end
"nope"

Is this a bug? Or maybe I'm just doing it wrong.

Folding over an AST

I'd like to be able to do some simple fold operations over an AST. Here's my attempt:

julia> function fold(ast, leaf, branch) 
           function go(ast)
               @match ast begin
                   :(Expr(:call, $f, $args...)) => branch(f, map(go, args))
                   x                            => leaf(x)
               end
           end
       
           return go(ast)
       end
fold (generic function with 1 method)

julia> fold(ast, x -> :(f($x)), function(f,args) :($f($args...)) end)
:(f(2 * (4x - 1) + 7))

I would expect the first pattern to match, but it doesn't.

Beyond this, there are some general things I don't understand:

• when to use @match vs @matchast
• when do patterns need to be quoted (:(Expr(:call, $f, $args...)) above, though seem to indicate unquoted should work
• why/how does @match ast ... do anything at all? I would expect it to require $ast

I'd greatly appreciate any help or suggestions! :)

Error tagging new release

The tag name "0.2" is not of the appropriate SemVer form (vX.Y.Z).
cc: @thautwarm

Access patterns via qualified ways

@match expr begin
   Module.Pattern(x) => x
end

Access local variable names

JuliaLang/julia#29366

This super useful one can help with making a much more convenient ast interpolation method:

@fmt quote
      a + b + c
end

instead of

@fmt [a, b, c] quote
   a + b + c
end

julia-flavored guards

We've tried refinement-type-like syntax, where suffix, and now it's the turn of if <cond> end syntax.

@match value begin
    <capture> && if <cond using captured> end => ...

I think it much more Julian than the previous alternatives, @Roger-luo any arguments? I'm planning to release the new version with fast expression template matching(the AST patterns).

priority issue of lambda cases

a && b -> c is partitioned as a && (b -> c).

Error with @def

julia> @def ff begin
       x => x+1
       (:reset, x) => x
       end
ERROR: LoadError: MethodError: no method matching pattern_match(::QuoteNode, ::Nothing, ::Expr, ::Module)
Closest candidates are:
  pattern_match(::Expr, ::Any, ::Any, ::Module) at /home/tj/.julia/packages/MLStyle/1KA9W/src/Match.jl:264
  pattern_match(::Number, ::Any, ::Any, ::Module) at /home/tj/.julia/packages/MLStyle/1KA9W/src/MatchExt.jl:24
  pattern_match(::AbstractString, ::Any, ::Any, ::Module) at /home/tj/.julia/packages/MLStyle/1KA9W/src/MatchExt.jl:31
  ...

String literal patterns got broken in many distribution versions

See JuliaLang/julia#32842 .

I think we cannot wait for the resolution from Julia developers, and now I'm planning to compile string literal patterns to comparisons by == instead of ===.

And we should release v0.3.1 AS SOON AS POSSIBLE!!!!

Directly matching macrocall expressions

Now, as the line number node contained in macrocall expressions

julia> dump(:(@f 1))
Expr
  head: Symbol macrocall
  args: Array{Any}((3,))
    1: Symbol @f
    2: LineNumberNode
      line: Int64 1
      file: Symbol REPL[1]
    3: Int64 1

We cannot use

@match :(@f 1) begin
   :(@f 1) => :okay
end

However, we can instead use

@match :(@f 1) begin
   :(@f $(::LineNumberNode) 1) => :okay
end

This should be a tip put in docs.

Requests for new tutorials: OO.jl and ParameterizedModule.jl

一般市民 might prefer the syntax of so-called object oriented programming, I think it could be leveraged to make MLStyle.jl popular.

Parameterized modules from ocaml could help with the organization of code base and polymorphisms.

Use `UnitRange` instead of `..`

For ranged pattern

@match num begin
    1..10  in x => "$x in [1, 10]"
    11..20 in x => "$x in [11, 20]"
    21..30 in x => "$x in [21, 30]"
end

does not looks Julian, use x in 1:10 instead?

An introduction of For patterns

After considering for extremely a long time, now I think there is a useful case of patterns look like set comprehension.

d = [(i, i + 1) for i = 1:5]
@match d begin
   [(i, i + 1) for i in seq] => seq
end # [1, 2, 3, 4, 5]

Which is to say, (i, i + 1) matches the element of d, thus we can do a reversed comprehension, where for i in seq produces a [i for (i, i+1) in d].

As a result,

d = [(i, i + 1) for i = 1:5]
@match d begin
   [i for i in seq] => seq
end # [(1, 1 + 1), (2, 2+1), (3, 3+1), (4, 4+1), (5, 5+1)]

You might think it a bit useless, however,

x :: Vector{Tuple{String, Term}}
eval_term :: Term => Expr

@active Eval(x) begin
    eval_term(x)
end

@match x begin
   [(a, Eval(b)) for (a, b) in seq] => seq
end # typed Vector{Tuple{String, Expr}}

AWSL!

Active patterns for conveninently making custom patterns

Although one can define her/his custom patterns without much efforts, some impl details could be exposed from this behavior, which might not be that good. The further optimizations might change some concrete infrastructures so we should export a more stable way to define custom patterns.

An Active pattern in F# is super useful and deserved to be referenced here.

https://docs.microsoft.com/en-us/dotnet/fsharp/language-reference/active-patterns

The implementation is trivial, and the key is about becoming more Julian.

I'd present some prospective syntaxes here about active patterns:

@active (Nomral || Singleton) function (x)
    if x > 10
        Singleton
    else 
        Normal(x)
    end
end

@match 10 begin
    Normal(_)   => true
    Singleton   => false
end

# regex pattern could be implemented through following one
@active Parametric(v) function(x)
      if x > 0 
         return x / v
     end
end 

@match 15 begin
    Parametric(5)(x) => @assert x === 3
    _ => ...
end

Pattern matching with substrings

First off thanks for the package it works great. I'm not sure if this is and bug or the intended semantics (if so feel free to close) but when pattern matching on a String type the following works fine

x = ["a","b","c"]


@match x begin
    ["a", z, "c"] => :okay
    _ => :nomatch
end 
#:okay

But the same pattern fails on a SubString

y = split("a b c")

@match y begin
    ["a", z, "c"] => :okay
    _ => :nomatch
end
#nomatch

Split non-primitive pattern implementations into a dedicated directory

As the proliferation of forthcoming new patterns, hold them with a single file StandPatterns.jl is becoming lack of maintenance and readability.

Just as a feasible solution, I'm thinking about to split them this way:

• MLStyle.jl
  • src
    • StandardPatterns
      • ActivePatterns.jl
      • ForPatterns.jl
      • ...
    • Internal
    • Modules

Extensions to support both view patterns and capturing via capitalized symbols

module M1
using MLStyle
@use Enum 
@active IsEven(x) begin
    x % 2 === 0
end
@match 4 begin
    IsEven => :ok
end # :ok  
end

module M2
using MLStyle
@use CapitalizedCapturing
@match 2 begin
   IsEven => IsEven + 1
end # 3
end

And these 2 extensions cannot be enabled in the same module:

module M3
using MLStyle
@use CapitalizedCapturing
@use Enum
end
ERROR: "Cannot use extensions `CapitalizedCapturing` and `Enum` simultaneously."

Error tagging new release

The tag name "0.1" is not of the appropriate SemVer form (vX.Y.Z).
cc: @thautwarm

Add Documenter badge

I've just set up the support for Documenter.jl. You'll need to manualy add ssh keys following the instructions here

lambda case in v0.2

map((@λ  C(s) => s), [C(s)])

map((@λ begin
   1 => 2
   2 => 3
   4 => 1
   3 => 2
   _ => 0
end), [1, 2, 3, 4])

subtyping of abstract data types

Like,

abstract type A <: SomeAbs end
@data A begin
    A1(Int)
    A2(Int)
    A3(Int)
end