jucardi / go-streams Goto Github PK

Hi,

I really loved your library! However I couldn't find java .flatMap or go youthlin-stream .FlatMap equivalent.

Example:
A stream of different size string arrays into stream of these arrays content:
{ [a], [b, c], [d, e, f] } into { a, b, c, d, e, f }

Legend:
{} <- stream
[] <- non empty array that may contain min 1 string
a,b,c,... <- different length (in my case) strings

I need to call FlatMap in the middle of parallel executing stream, before I have 2 filter and 3 map calls, after 2 Map and 2 Filter calls.
I operate on 87 sets of data between 30 to 2'000'000 records that's why I really need parallel execution. (for each record i have regex, comparison to other string, etc. ) I shouldn't create additional middle datasets because it'd be huge memory leak

Thanks in advance :)

Please add a license file as Go website requires it.

Hi!

With Java 8+ it's possible to read file using streams instead of using loop.
This is so cool, as file isn't read entirely to memory.

Now I'm facing this issue with go.
I need to read hundreds of text files with millions of records and process them.

Thus I need reader and writer that will be compatible with this library.
I tried to do so but it's hard considering implementing generic interface IIterator

The strength of java in this case is the fact that streams became standard.

Do you think it's possible to include such io.reader or bufio.scanner to this library?

Hi, great lib! Could you please add version 2 definition in go.mod?

Cannot import module with generics:
github.com/jucardi/[email protected]: invalid version: module contains a go.mod file, so module path must match major version ("github.com/jucardi/go-streams/v2")

Thanks in advance

Why does version 2 not support Map anymore?
I'm really looking forward to using map operations in version 2!
eg

func main() {
	streams.From[int]([]int{1,2,3,4,5,6}).Map[int,int](func(item int) int{
		return item * 2
	})
}

Is seems straightforward to obain an array as an output of stream operations (using ToArray()). Is it also possible to obtain a map?

I'm getting the following error when I try the sample code:

var fruitArray = []string{"peach", "apple", "pear", "plum", "pineapple", "banana", "kiwi", "orange"}
	fruitsThatStartWithP := streams.

		// Creates a stream from the given array
		From[string](fruitArray).

		// Adds a filter for strings that start with 'p'
		Filter(func(v string) bool {
			return strings.HasPrefix(v, "p")
		}).

		// Sorts alphabetically
		//Sort(strings.Compare).

		// Converts back to an array
		ToArray()

invalid operation: cannot index streams.From (value of type func(set interface{}, threads ...int) streams.IStream)

Go 1.20.2 on Ubuntu.

If I change From[string] to just From, then that particular error goes away.... but then I also have to change func v string to func (v interface{})

Do you think it would make sense to open discussion on golang/go if streams should become standard like in Java 8+?

If Combined with IO file doesn't need to be read entirely to memory. This would for instance help fix #10

$ go version
go version go1.20.6 darwin/arm64

• step1

$ mkdir teststream && cd teststream  &&  go mod init github.com/voidint/teststream && touch main.go

• step2

package main

import (
	"fmt"
	"strings"

	"github.com/jucardi/go-streams/streams/v2"
)

func main() {
	var fruitArray = []string{"peach", "apple", "pear", "plum", "pineapple", "banana", "kiwi", "orange"}

	fmt.Println(streams.
		// Creates a stream from the given array
		From[string](fruitArray).
		// Adds a filter for strings that start with 'p'
		Filter(func(v string) bool {
			return strings.HasPrefix(v, "p")
		}).
		// Sorts alphabetically
		Sort(strings.Compare).
		// Converts back to an array
		ToArray(),
	)
}

• step3

$ go get github.com/jucardi/go-streams/streams/v2@latest
$ go mod tidy 
go: finding module for package github.com/jucardi/go-streams/streams/v2
github.com/voidint/teststream imports
        github.com/jucardi/go-streams/streams/v2: module github.com/jucardi/go-streams@latest found (v1.0.3), but does not contain package github.com/jucardi/go-streams/streams/v2

$ cat go.mod
module github.com/voidint/teststream

go 1.20

require github.com/jucardi/go-streams/v2 v2.0.1 // indirect

Maybe I'm doing something wrong here, but I want an empty list instead of a panic.

	arr := []string{"x", "y", "z"}
	newArr := streams.FromArray(arr).
		Filter(func(str interface{}) bool {
			return len(str.(string)) > 1
		}).
		Map(func(str interface{}) interface{} {
			return strings.ToUpper(str.(string))
		}).
		ToArray().([]string)

changing 1 to 0 gives the correct result...
I'm trying to do something similar with incoming json data and sometimes the filter gets rid of all the entries.