[BUG] Fix non thread safe token cache behavior about flyte

[runllm]

The current implementation of the token cache in Flyte is not thread-safe, which can cause issues when multiple parallel attempts are made to get a token. This is evident in the TokenCacheInMemoryProvider class, which does not use any synchronization mechanisms to ensure thread safety.

To make the token cache thread-safe, you can use synchronization primitives like mutexes. Here's an example of how you can modify the TokenCacheInMemoryProvider to be thread-safe:

package cache

import (
	"fmt"
	"sync"

	"golang.org/x/oauth2"
)

type TokenCacheInMemoryProvider struct {
	token *oauth2.Token
	mu    sync.Mutex
}

func (t *TokenCacheInMemoryProvider) SaveToken(token *oauth2.Token) error {
	t.mu.Lock()
	defer t.mu.Unlock()
	t.token = token
	return nil
}

func (t *TokenCacheInMemoryProvider) GetToken() (*oauth2.Token, error) {
	t.mu.Lock()
	defer t.mu.Unlock()
	if t.token == nil {
		return nil, fmt.Errorf("cannot find token in cache")
	}
	return t.token, nil
}

Would you like more details on this implementation?

Sources:

• Does Flyte support the caching of M2M authorization tokens?
• Hi there we ve been having some challenges with the token ca
• Core feature pyflyte should store tokens after DeviceFlow auth

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

[pingsutw]

@pmahindrakar-oss, does #5407 fix it? If so, could we close the issue?

[pmahindrakar-oss]

Yes we can close this issue