samber / lo Goto Github PK
View Code? Open in Web Editor NEW💥 A Lodash-style Go library based on Go 1.18+ Generics (map, filter, contains, find...)
Home Page: https://pkg.go.dev/github.com/samber/lo
License: MIT License
💥 A Lodash-style Go library based on Go 1.18+ Generics (map, filter, contains, find...)
Home Page: https://pkg.go.dev/github.com/samber/lo
License: MIT License
Let's talk about a new suite of helpers for manipulating channels.
// ToChannel returns a read-only channels of collection elements.
func ToChannel[T any](collection []T) <-chan T
// Generator implements the generator design pattern.
func Generator[T any](bufferSize int, generator func(int64) T) <-chan T
// Batch creates a slice of n elements from a channel. Returns the slice and the slice length.
func Batch[T any](ch <-chan T, size int) (collection []T, length int)
// BatchWithTimeout creates a slice of n elements from a channel, with timeout. Returns the slice and the slice length.
func BatchWithTimeout[T any](ch <-chan T, size int, timeout time.Duration) (collection []T, length int)
Some thoughts about BatchXXX functions:
ok bool
?duration time.Duration
?<-chan []T
instead of []T
?Line 42 in 1dbdfc2
It would be useful to have functions like that:
func UnpackT2[A any, B any](tuple Tuple2[A, B]) (A, B){
return tuple.A, tuple.B
}
That would allow to extract variables from tuples which would make the code more readable in some situations.
I can provide a PR for this.
Quite frequently, I want to do something like lo.Map(slice, Transformer)
but Transformer
returns (R, error)
.
I can write a function literal and handle the error, but it's still not ergonomic as there's no proper way to return the error.
So, I propose (and I can submit a PR for this) a variant of Map (and maybe others that make sense) that looks like this:
func Map[T any, R any](collection []T, iteratee func(T, int) (R, error)) ([]R, error)
Where, if the iteratee returns an error, the entire iteration stops, returns an empty list and an error (or maybe returns what it has so far)
It would be great to be able to add context to the panic message in Must
.
That would be especially useful in the case where Must
validates a boolean as at the moment it panics with the message "not ok".
I was thinking we could add the parameter msgAndArgs ...interface{}
at the end of each Must
function like it is done in the assert functions of the testify
package.
We could then call it like that for example:
lo.Must0(strings.Contains(myString, requiredSubString)), "'%s' must always contain '%s'", myString, requiredSubString)
Note:
One issue with the solution described above is that the compiler will not detect when the number of values returned by the function used in Must
doesn't match the expected number of parameters of Must
.
One solution for that could be to create new function like MustF
& MustFX
which take a string instead of msgAndArgs ...interface{}
For the same example we would then have:
lo.MustF0(strings.Contains(myString, requiredSubString)), fmt.Sprintf("'%s' must always contain '%s'", myString, requiredSubString))
Current implementation spawn as many go-routines as passed collection size (for example as passed to Map
) method in parallel
package. While this is perfectly okay to do for up to a few thousand go-routines, it could become problematic for larger values. For example, for a map
ping operation that makes any networking call, this is almost equivalent to DOS
sing the end service.
Suggestion is to allow configuring concurrency for such parallel operations. Default would be equal to runtime.NumCPU()
, whereas, if anything <= 0 is specified, it would automatically switch to current default behavior. Anything greater than 0 would override previous behaviors specified.
Sample implementation with above in mind for Map
method can be found here (this diff is against current implementation): https://www.diffchecker.com/4pLAUeMh
Let me know if this looks good. I can work on the PR.
Best wishes...
We could add a function that execute a function in a goroutine and returns the result in a channel:
func Async[T any](f func() T) chan T {
channel := make(chan T)
go func() {
channel <- f()
}()
return channel
}
Similar to https://github.com/luraim/fun#filtermap, performing filtering and mapping in a single call.
Some languages like Typescript or Swift offer a very cool syntactic sugar a?.b?.c?.d?.e
.
In Go, we need to write a condition similar to: a != nil && a.b != nil && a.b.c != nil
.
I create this issue for discussing a new helper.
In PR #106, I suggest an implementation called Safe
:
type a struct {
foo *string
}
type b struct {
a *a
}
type c struct {
b *b
}
v := &c{
b: &b{
a: nil,
},
}
foo, ok := lo.Safe(func() string { return *v.b.a.foo })
// "", false
Calling *v.b.a.foo
will panic, but this nil pointer error will be caught by lo.Safe
. Other exception won't be caught.
This implementation is much more "typesafe" than something like lo.Safe(a, ".b.c.d")
WDYT?
Needed to read the description to get what it does. Naming it Merge would make it obvious. What do you think?
Currently, description of Empty()
function is missing in README.
Line 48 in 75ed9bb
In: Intersect[int]([]int{0, 6, 0}, []int{0, 1, 2, 3, 5, 5, 6, 6})
Out: []int{0, 6, 6}
In: Difference[int]([]int{0, 1, 2, 3, 3, 4, 4}, []int{0, 1, 2, 2, 3, 5, 5})
Out: []int{4, 4}, []int{5, 5}
In: Union[int]([]int{0, 1, 2, 3, 3, 5, 6, 6}, []int{0, 1, 2, 4, 4, 5, 5, 6})
Out: []int{0, 1, 2, 3, 3, 5, 6, 6, 4, 4}
Hello, these functions are dealing with duplicate values differently. Is there a rule that the collections disallow duplicates?
Invert
is nondeterministic when a map contains duplicate values because map iteration is randomized by the go compiler.
I don't think this function really should exist. But I'm curious if anyone is using/relying on this, and what the use case is.
Such function could be named like ValuesF
which means Values Filtered
or may be you suggest another naming.
The implementation is just an extension of the std lib slices.Values
function:
ValuesF[K comparable, V any](m ~map[K]V, keys []K) []V {
r := make([]V, 0, len(m))
for k, v := range m {
if slices.Contains(keys, k) {
r = append(r, v)
}
}
return r
}
It seems that lo can't do any chain opertaions like
type Poo struct {
Id int
Name string
}
poos := []Poo{
{1, "A"},
{2, "B"},
{3, "C"},
}
// it dosen't work
lo.Map[Poo, int](poos, func(x Poo, _ int) int {
return x.Id
}).Filter[int](lo.Map[Poo, int](poos, func(x int, _ int) bool {
return x%2 == 0
})
// it works
even := lo.Filter[int](lo.Map[Poo, int](poos, func(x Poo, _ int) int {
return x.Id
}), func(x int, _ int) bool {
return x%2 == 0
})
It will be supported in the future?
May be we can introduce a middle data stream like the stream operation in Java
Line 139 in 4e339ad
Your naive shuffle algorithm suffers the common over shuffling problem
https://possiblywrong.wordpress.com/2014/12/01/card-shuffling-algorithms-good-and-bad/
Go uses the https://en.wikipedia.org/wiki/Fisher–Yates_shuffle standard algorithm.
But you can just use Go's implementation from rand.
func Shuffle[T any](collection []T) []T {
rand.Shuffle(len(collection), func(i, j int) {
collection[i], collection[j] = collection[j], collection[i]
})
return collection
}
See https://play.golang.com/p/YgPwRAtSQdb
package main
import (
"fmt"
"github.com/samber/lo"
)
func main() {
ints := []int{0,1,2}
fmt.Println(lo.Nth(ints, -1)) // prints 2, as expected
ints = []int{0}
fmt.Println(lo.Nth(ints, -1)) // returns error "nth: -1 out of slice bounds" - but shouldn't
}
Additionally, I think this function should panic instead of raising an error when out of bounds (but I'm open to be convinced otherwise):
I'll submit a PR with a fix - and my proposed change, but wanted to file this issue first.
It would be useful to have:
func Duplicate[T comparable](collection []T) []T
func DuplicateBy[T any, U comparable](collection []T, iteratee func(T) U) []T
That would only return the duplicated values of the collection
.
This is a request for a new function: Except
. This would work similar to Difference
except that it documents (assumes) that list1 is a superset of list2. This is the logical inverse of the Intersect()
.
I understand that Difference actually has this behavior:
https://go.dev/play/p/WxfAXrK70wX
From a user experience perspective, it does a bit more than I need (I have to throw away the second array). And then there's this:
I'm happy to create a PR. Please let me know if you would like this. I believe that Except
would be slightly faster, and with a slightly smaller memory footprint as well.
Thank you for this awesome library!!
All[T any](collection []T, predicate func (T) bool) bool
would return true
if all the elements pass the predicate and short-circuit to false
otherwise.
Any[T any](collection []T, predicate func (T) bool) bool
would return false
if none the elements pass the predicate and short-circuit to true
otherwise.
These are common functions in Rust, Haskell, many other languages.
Sometimes if the collection is too big, it will make too many goroutines in one time when using parallel
methods. That will significantly increase resource usage, especially when the iteratee
function is computationally-Intensive, io-Intensive, or both.
An example that requests every URL in a huge list, will start so many requests at the same time:
hugeUrlList := []string{ /*...*/ }
parallel.ForEach(hugeUrlList, request)
I think there should have an option to set the max concurrency count, which means up to configured number iteratee
calling at the same time:
hugeUrlList := []string{ /*...*/ }
parallel.ForEach(hugeUrlList, request, parallel.Option().Concurrency(100))
I will enjoy implementing it.
I use this function personally. With this, you can save dozens of lines with if err != nil
. Execute all simple logics (doSimpleThing
) first, and check the error later. The error will be the first occured error or nil.
func Delay[T any](target *error) func(v T, err error) T {
a := func(v T, err error) T {
if *target == nil && err != nil {
*target = err
}
return v
}
return a
}
Example:
func doSimpleThing1() (int, error){...}
func doSimpleThing2() (string, error){...}
var err error
dInt := Delay[int](&err)
dString := Delay[string](&err)
_ = complexStruct{
id: dInt(doSimpleThing1()),
name: dString(doSimpleThing2()),
email: dString(doSimpleThing2()),
...
}
// check error later
if err != nil {
panic(err)
}
I think this function is appropriate for this package if there's a good name for it. Do you have a good idea?
var someTime = Must(time.Parse("2006-01-02", "2022-01-15"))
Must
allows converting unexpected errors into panics, very useful when initializing global variables. This is a pretty common pattern and lot of libraries ship their own implementation of it eg:
Must
in "text/template"
Must
in "html/template"
MustParse
in "github.com/google/uuid"
MustCompile
in "regexp"
Lots more examples as well.
Create function func MaxBy[T any](collection []T, comparison func (T,T) bool) T
that returns the max using the given comparison function.
Create function func MinBy[T any](collection []T, comparison func (T,T) bool) T
that returns the min using the given comparison function.
when calling min with an empty slice it should return the maximum value
when calling max with an empty slice it should return the minimum value
It would be useful to have functions ToTupleX to simplify the creation of Tuples.
It would be especially useful to convert the return of a function with multiple values into a single Tuple.
It would be nice if there something like https://www.geeksforgeeks.org/lodash-_-pipeline-method.
Let's say there's an iteratee:
func iteratee(a int) int {
return a + 1
}
Whenever I use functions in lo
, it requires me to write another iteratee to wrap it:
slice = lo.Map(slice, func(i int, _ int) int {
return iteratee(i)
})
Well, it's acceptable, but not elegant. If lo
could provide a function like this:
func WrapIndex[T any, R any](f func(T) R) func(T, int) R {
return func(t T, _ int) R {
return f(t)
}
}
Then the function could be written in this way:
slice = lo.Map(slice, WrapIndex(iteratee))
It's seldom we use an indexed version iteratee, most of the time we just throw away the index.
However, providing an unindexed version for all functions would be tedious. Instead, providing a wrapper function to throw away the index should be better.
BTW, I don't know whether I used the words 'indexed' and 'unindexed' correctly. Hope you could understand. XD
It might be convenient to have a slice function similar to Find()
, but that you can pass a default value to return if no match was found. Signature could look like so:
FindDefault[T any](collection []T, fallback T, predicate func(T) bool) T
Say you have a collection and want to find the element that minimizes or maximizes a function.
For example, here I have a slice of person
structs and am looking for the one for which the computeAge
method returns the lowest value:
type person struct {
name string
birthday time.Time
}
func (p *person) computeAge() int {
age := time.Now().Year() - p.birthday.Year() - 1
if p.birthday.YearDay() < time.Now().YearDay() {
age += 1
}
return age
}
func main() {
people := []person{
{"Jordan", time.Date(1994, time.October, 2, 0, 0, 0, 0, time.UTC)},
{"Emily", time.Date(1868, time.March, 26, 0, 0, 0, 0, time.UTC)},
{"Micheal", time.Date(1999, time.June, 5, 0, 0, 0, 0, time.UTC)},
{"Nikola", time.Date(2001, time.November, 16, 0, 0, 0, 0, time.UTC)},
{"Rachel", time.Date(1993, time.April, 1, 0, 0, 0, 0, time.UTC)},
}
youngest := lo.MinBy(people, func(a, b person) bool {
return a.computeAge() < b.computeAge()
})
fmt.Println(youngest.name)
}
Using MinBy
like this results in 8 calls to computeAge
where 5 should have sufficed. Using a custom keying function with MinBy
results in many more calls than necessary.
I propose adding a MinByKey
and MaxByKey
functions that apply the custom keying function to all elements and then return the one that minimizes/maximizes it.
The signatures would be:
func MinByKey[T any, K constraints.Ordered](collection []T, key func(T) K) T
func MaxByKey[T any, K constraints.Ordered](collection []T, key func(T) K) T
Would this contribution be appreciated @samber ?
The Find
function exists. The IndexOf
function exists. What would be super helpful is a FindIndex
function; similar in mechanics to Find
, except it returns the slice index of the first slice member which satisfies the Find
function (or -1 if none are found).
Awesome project, by the way!
Additionally to error returns, you also often stumble upon boolean returns.
Fictional example:
func Parse(raw string) (out ConcreteType, ok bool)
It is possible to use type switches within generic function, so we could switch for error and bool alike.
Allowing the Must() function to become slightly more generic than it is right now without breaking anything.
like this
type Foo struct {
ID string
}
value := []*Foo{{ID: "a"}, {ID: "b"}, {ID: "c"}}
pipe := lo.NewPipeline(value)
result := pipe.Map(func(v *Foo) string { return v.ID }).Filter(func(v *Foo) bool { return v.ID == "a" }).Uniq().Result()
Create function Count[T any](collection []T, predicate func (T) bool) int
would return the number of elements for which the predicate returns true.
I just learned about this project (been using RxGo), and the API and helpers look really useful.
I mostly work on streaming-based systems/APIs in Go so was wondering if you're also considering adding support for such use cases?
E.g. filter and map based on a stream (Go channel?) of Order
structs coming from a messaging system (Kafka, Rabbit, Redis) or HTTP (gRPC) poll/push API.
func MinBy[T any](collection []T, comparison func(T, T) bool) T {
var min T
if len(collection) == 0 {
return min
}
min = collection[0]
for i := 1; i < len(collection); i++ {
item := collection[i]
if comparison(item, min) {
min = item
}
}
return min
}
func MaxBy[T any](collection []T, comparison func(T, T) bool) T {
var max T
if len(collection) == 0 {
return max
}
max = collection[0]
for i := 1; i < len(collection); i++ {
item := collection[i]
if comparison(item, max) {
max = item
}
}
return max
}
comparison in MinBy is less
,while in MaxBy is greater
. In most of the generics library, just need one comparison function, less, such as C++ STL. This minimizes user misuse.
@samber I may suggest a PR if you agreed with this proposal.
Can we get some more common string operations, such as the classic chop (force one rune truncation) and chomp (remove trailing runes only on CR, LF, or CRLF match)?
Function retrieves any slices and a callback which returns 2 values: map key and map value.
Possible implementation:
func ToMap[S, V any, K comparable, M ~map[K]V](s []S, fn func(S) (K, V)) M {
m := make(map[K]V, len(s))
for _, elem := range s {
key, value := fn(elem)
m[key] = value
}
return m
}
Lodash describes this operation as returning unique entries
https://lodash.com/docs/#intersection
Also described in SQL:
https://www.postgresql.org/docs/9.4/queries-union.html
INTERSECT returns all rows that are both in the result of query1 and in the result of query2. Duplicate rows are eliminated unless INTERSECT ALL is used.
EXCEPT returns all rows that are in the result of query1 but not in the result of query2. (This is sometimes called the difference between two queries.) Again, duplicates are eliminated unless EXCEPT ALL is used.
Hi,
Moving from go-funk
to lo
, just noticed go-funk
is still included in my dependency graph, probably due to it being a dependency for the benchmark suite in lo
.
Not sure, but is there any way to remove it from the 'release' lib?
Example:
even, odd := lo.FilterXXX[int]([]int{1, 2, 3, 4}, func(x int, _ int) bool {
return x%2 == 0
})
I would name it lo.Filter2
but it looks weird. Does anyone have a better idea?
A lo.RejectXXX
helper would be excellent too.
I'm new to Go lang community, so this is most a discussion about if it'd be a good practice than a suggestion by itself. Coming from the Javascript world, it is common to see situations where this expression is used:
function (arg1, arg2) {
return arg1 ?? arg2 ?? "[default value]"
}
In SQL databases it is also common to find the Coalesce function that solves a similar problem:
SELECT COALESCE(column1, column2, '[default value]')
FROM xyz
But I didn't find in Go a simple way to handle nil values and use a default value instead, needing to use the Ternary()
or Find()
functions as a solution, but still they don't seem to be the most idiomatic way to solve the problem. Following examples above, it might be interesting to have functions like:
Coalesce():
Returns the first non-nil argument passed to the function
NullIf():
Returns nil if both arguments are equal, otherwise returns the first
Is this project going to be actively maintained?
I see that people would like to contribute, but issues and PR's review and acceptance depends only on @samber and his free time.
May be such bus-factor can be reduced somehow?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.