lotusirous / go-concurrency-patterns

Concurrency patterns in Go

Go Concurrency Patterns

This repository collects common concurrency patterns in Golang

Materials

• Concurrency is not parallelism
• Go Concurrency Patterns and source
• Advanced Go Concurrency Patterns
• Rethinking classical concurrency pattern
• Go Concurrency Patterns: Pipelines and cancellation
• Go Concurrency Patterns: Timing out, moving on
• Complex Concurrency Patterns with Go

Context:

• Go Concurrency Patterns: Context
• How to correctly use package context
• justforfunc #9: The Context Package
• Contexts and structs

Name	Description	Playground
1-boring	A hello world to goroutine	play
2-chan	A hello world to go channel	play
3-generator	A python-liked generator	play
4-fanin	Fan in pattern	play
5-restore-sequence	Restore sequence	play
6-select-timeout	Add Timeout to a goroutine	play
7-quit-signal	Quit signal	play
8-daisy-chan	Daisy chan pattern	play
9-google1.0	Build a concurrent google search from the ground-up	play
10-google2.0	Build a concurrent google search from the ground-up	play
11-google2.1	Build a concurrent google search from the ground-up	play
12-google3.0	Build a concurrent google search from the ground-up	play
13-adv-pingpong	A sample ping-pong table implemented in goroutine	play
14-adv-subscription	Subscription	play
15-bounded-parallelism	Bounded parallelism	play
16-context	How to user context in HTTP client and server	play
17-ring-buffer-channel	Ring buffer channel	play
18-worker-pool	worker pool pattern	play

go-concurrency-patterns's Issues

The main in item 14 is not implemented

The main in item 14 https://github.com/lotusirous/go-concurrency-patterns/blob/main/14-adv-subscription/main.go is not implemented.

timeout example is unsuitable

go-concurrency-patterns/6-select-timeout/main.go

Lines 25 to 35 in 2c16586

	// timeout for the whole conversation
	timeout := time.After(5 * time.Second)
	for {
	select {
	case s := <-c:
	fmt.Println(s)
	case <-timeout:
	fmt.Println("You talk too much.")
	return
	}
	}

We always think timeout as the time that one service has on response. In this case, timeout 's meaning should be ** we don't receive any message from c channel for 5 seconds.

so maybe the right code is :

timeout := time.After(5 * time.Second)
for {
       select {
       case s := <-c:
               fmt.Println(s)
               timeout = time.After(5 * time.Second) // update timeout time
       case <-timeout:
               fmt.Println("channel has no response for 5 seconds")
               return
       }
}

the origin code seems that you want to receive from every channel for same time, like linux cfs scheduler. so maybe in that case we should use ticker

package main

import (
        "fmt"
        "time"
)

func boring(id int) <-chan string {
        c := make(chan string)
        go func() {
                 c <- "" // skip the first time
                for i := 0; ; i++ {
                        c <- fmt.Sprintf("%d, %s", id, time.Now().Format("2006-01-02 15:04:05"))
                        time.Sleep(1 * time.Second)
                }
        }()
        return c
}

func main() {
        timeout := time.NewTicker(5 * time.Second)
        c1 := boring(1)
        c2 := boring(2)
        jobchannels := []<-chan string{c1, c2}
        i := 0
        for {
                select {
                case s := <-jobchannels[i]:
                        fmt.Println(s)
                case <-timeout.C:
                        fmt.Printf("%d has talk for 5 secs\n", i+1)
                        i = (i + 1) % len(jobchannels)
                }
        }
}

and it's output is

1, 2021-03-17 11:27:03
1, 2021-03-17 11:27:04
1, 2021-03-17 11:27:05
1, 2021-03-17 11:27:06
1, 2021-03-17 11:27:07
1 has talk for 5 secs

2, 2021-03-17 11:27:08
2, 2021-03-17 11:27:09
2, 2021-03-17 11:27:10
2, 2021-03-17 11:27:11
2, 2021-03-17 11:27:12
2 has talk for 5 secs
1, 2021-03-17 11:27:08
1, 2021-03-17 11:27:14
1, 2021-03-17 11:27:15
1, 2021-03-17 11:27:16
....

Different approach for ring buffer

go-concurrency-patterns/17-ring-buffer-channel/main.go

Line 28 in 82c2313

r.outCh <- v

This is a bug surely? Another routine could write between these two calls, and then this write would block.

A safer approach would be to drop the current write AND one from the ring buffer. This would ensure that no code ever blocks, but you are still guaranteed new data (which IMO is a big use-case for ring buffers).

This would ensure no back-pressure. It would make starvation technically possible very rarely by some complex timing and workload patterns if you have too many writers for the size of the ring buffer, but this is trivial to avoid by sizing the ring buffer appropriately.

So as long as your ring buffer isn't too small, I think it would behave more like what people would expect.

about 18-worker-pool question

why does one worker complete eight tasks?

The current implementation of the ring buffer is not very clear.

Hello, I have tested your ring buffer, and the output consisted of 5 values:

2024/04/03 07:39:49 5
2024/04/03 07:39:49 6
2024/04/03 07:39:49 7
2024/04/03 07:39:49 8
2024/04/03 07:39:49 9

However, the buffer for the channel was set to 4. Therefore, I find this response somewhat unclear.