GithubHelp home page GithubHelp logo

mikaelemmmm / go-zero-looklook Goto Github PK

View Code? Open in Web Editor NEW
4.1K 32.0 771.0 102.97 MB

🔥基于go-zero(go zero) 微服务全技术栈开发最佳实践项目。Develop best practice projects based on the full technology stack of go zero (go zero) microservices.

Home Page: https://go-zero.dev

License: MIT License

Go 98.15% Shell 1.85%
go-zero golang go-queue goctl gozero microservices asynq dtm modd go-stash

go-zero-looklook's Introduction

Hi there 👋

info

  • 🌱 I like coding
  • ⚡I am from Shenzhen, China
  • 📫 You can contact me by email

go-zero-looklook's People

Contributors

aflybird0 avatar bllli avatar cexll avatar ctra-wang avatar cuishuang avatar cy414 avatar cyb0225 avatar hp66722667 avatar kabochar avatar kevwan avatar kpkym avatar lunay001 avatar mikaelemmmm avatar mrmm avatar pittyao avatar rocklau avatar shaojunda avatar shapohun avatar sixwaaaay avatar spectatornan avatar sunray97 avatar waltcow avatar yangyang880402 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-zero-looklook's Issues

windows 运行 goctl api go -api *.api -dir ../ --style goZero 出错

api目录下user.api已经定义好了,运行

goctl api go -api *.api -dir ../  --style goZero  

出错,错误提示为

Error: open *.api: The filename, directory name, or volume label syntax is incorrect.
Usage:
  goctl api go [flags]

Flags:
      --api string      The api file
      --branch string   The branch of the remote repo, it does work with --remote
      --dir string      The target dir
  -h, --help            help for go
      --home string     The goctl home path of the template, --home and --remote cannot be set at the same time, if they are, --remote has higher priority
      --remote string   The remote git repo of the template, --home and --remote cannot be set at the same time, if they are, --remote has higher priority
                        The git repo directory must be consistent with the https://github.com/zeromicro/go-zero-template directory structure
      --style string    The file naming format, see [https://github.com/zeromicro/go-zero/blob/master/tools/goctl/config/readme.md] (default "gozero")

open *.api: The filename, directory name, or volume label syntax is incorrect.

指定生成文件就成功,命令如下:

goctl api go -api user.api -dir ../  --style goZero

关于token过期自动刷新的问题

今天在群里问了下万总,go zero自带的jwt认证是不会实现自动刷新token的,但是我看你好像没有go zero自带的Auth Handler,而是使用自己的token认证,不知道是怎么实现token的自动刷新的。

English documentation

Hello,

Thank you so much for dedicating the time to create this project, the issue is that it is very hard to work with go-zero because there are so few English resources available.

Is there a way to create the documentation in English also so that it can reach more of the community 🙏

Please let me know how I can help

日志收集

可以不用zero自带日志功能,自定义日志组件比如uber zap吗?这个支持直接讲日志发送到es或者kafka

lyumikael/gomodd:v1.0.0 镜像不支持 go 1.8

lyumikael/gomodd:v1.0.0 不支持 go1.8

然后我自己做了个Dockerfile

FROM golang AS builder

ENV GOPROXY=https://goproxy.cn,direct
ENV GO111MODULE=on

RUN go version
RUN go install github.com/cortesi/modd/cmd/modd@latest

CMD [ "modd" ]

我这个 docker 镜像运行后 文件改动后 modd 不会重启应用,请问你的 Dockerfile 是怎么实现的呢?

token鉴权的问题

hello,想问下什么情况下会调用nginx的/auth的路由? 可以大概举个例子么,方法我是看懂了,但是实际的应用场景可能还不清楚,谢谢

关于cosmtrek/air 使用的疑问

在开发过程中遇到rpc的service 容器recreate的情况,这时候容器会重新下载一遍依赖,
加上了 GOPROXY: https://goproxy.cn,direct 一样很慢
这个能不能做一下缓存策略

image

一对多和多对多查询

你好,go-zero里面的sqlx组件进行一对多和多对多查询要怎么处理呀,我看sqlx的issues有提过同样的问题,尝试了一下没有绑定数据到结构体里面。
sqlx 一对多issues地址 jmoiron/sqlx#636
查询代码如下:
RsUser struct {
Id int64 db:"id" // 唯一标示
...
}
RsRole struct {
Id int64 db:"id" // 主键
...
}
RsMenu struct {
Id int64 db:"id" // 主键
...
}
CurrentLoginUser struct {
RsUser db:"RsUser"
Role []RsRole db:"RsRole"
Menu []RsMenu db:"RsMenu"
}
//联表查询当前登录用户信息
queryOp := squirrel.Select("u.id AS 'RsUser.id'",
...
"r.id as 'RsRole.id'",
...
"m.id as 'RsMenu.id'",
...
).From("rs_user u ").LeftJoin("re_user_role ur ON ur.user_id = u.id ").LeftJoin("rs_role r ON r.id = ur.role_id AND r.del_flag = 0 ").LeftJoin("re_role_menu rm ON rm.role_id = r.id ").LeftJoin("rs_menu m ON m.id = rm.menu_id AND m.del_flag = 0 ").Where("u.id = ? AND u.del_flag = 0")
query, _, err := queryOp.ToSql()
if err != nil {
return nil, err
}
//查询与缓存
resp := &[]CurrentLoginUser{}
err = c.conn.QueryRowsCtx(ctx, resp, query, id)
return *resp, err

查询结果数据绑定失败:
image

我进行1对1绑定查询是可以查询到数据的:
CurrentLoginUser struct {
RsUser
RsRole
RsMenu
}
查询数据绑定成功:
image

win11无法收集日志数据。

win11系统无法获取得到日志,要改什么配置吗?求帮忙~

file

Go

Go

Go

Go

系统参数:
版本 Windows 11 专业版
版本 21H2
操作系统版本 22000.795

【安装环境】导入mysql数据执行出错

环境:
python -V -> Python 3.7.10
python3 -V -> Python 3.5.2
redis-cli 6.2.6
etcd Version: 3.5.2
Docker version 18.09.7, build 2d0083d
mysql Ver 14.14 Distrib 5.7.33
Linux Version: Ubuntu 16.04

我在操作3.2导入mysql数据的时候, 执行:docker exec -it mysql mysql -uroot -p 失败。
执行 docker logs --tail 50 --follow --timestamps 2f48f1f5d8b8 注: 2f48f1f5d8b8 是容器id
查看docker日志发现:
[Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead.
[Server] /usr/sbin/mysqld (mysqld 8.0.28) initializing of server in progress as process 16
[Server] Can't find error-message file '/usr/share/mysql-8.0/errmsg.sys'. Check error-message file location and 'lc-messages-dir' configuration directive.
[Server] Can't change data directory owner to mysql
[Server] The designated data directory /var/lib/mysql/ is unusable. You can remove all files that the server added to it.
[Server] Aborting
[Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.28) MySQL Community Server - GPL.

请问这个如何处理?

elasticsearch 启动占用cpu过高

你好, 第一步用docker-compose启动所有镜像后, 其中elasticsearch 这个镜像cpu占用达到了189%,是配置不对吗?
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4145 lightho+ 20 0 3121992 646488 22416 S 189.7 16.7 0:09.52 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -

Asynq Monitor questions

I have successfully run your mqueue service,
【questions1】
in redis ,server data is
image
but in Asynq Monitoring,No servers found at this time.
image
Why wasn't it detected,and delay queue not work after timetout

docker部署出错 dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:39899->127.0.0.11:53: i/o timeout

启动报错,docker中日志:

looklook         | 19:58:53: prep: go build -o data/server/payment-rpc  -v app/payment/cmd/rpc/payment.go
looklook         | go: downloading github.com/zeromicro/go-zero v1.3.4
looklook         | go: downloading google.golang.org/grpc v1.46.2
looklook         | go: downloading github.com/zeromicro/go-queue v1.1.5
looklook         | go: downloading google.golang.org/protobuf v1.28.0
looklook         | go: downloading github.com/pkg/errors v0.9.1
looklook         | go: downloading github.com/jinzhu/copier v0.3.2
looklook         | go: downloading github.com/Masterminds/squirrel v1.5.2
looklook         | go: downloading github.com/sony/sonyflake v1.0.0
looklook         | go: downloading github.com/shopspring/decimal v1.3.1
looklook         | app/payment/model/thirdPaymentModel.go:10:2: github.com/Masterminds/[email protected]: Get "https://goproxy.cn/github.com/%21masterminds/squirrel/@v/v1.5.2.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:35491->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/internal/logic/getPaymentBySnLogic.go:11:2: github.com/jinzhu/[email protected]: Get "https://goproxy.cn/github.com/jinzhu/copier/@v/v0.3.2.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:35491->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/internal/logic/createPaymentLogic.go:11:2: github.com/pkg/[email protected]: Get "https://goproxy.cn/github.com/pkg/errors/@v/v0.9.1.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:39899->127.0.0.11:53: i/o timeout
looklook         | common/tool/coinconvert.go:3:8: github.com/shopspring/[email protected]: Get "https://goproxy.cn/github.com/shopspring/decimal/@v/v1.3.1.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:35491->127.0.0.11:53: i/o timeout
looklook         | common/uniqueid/uniqueid.go:4:2: github.com/sony/[email protected]: Get "https://goproxy.cn/github.com/sony/sonyflake/@v/v1.0.0.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:35491->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/internal/svc/serviceContext.go:4:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-queue/@v/v1.1.5.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:39899->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/payment.go:13:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/internal/logic/createPaymentLogic.go:12:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/payment.go:14:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/model/thirdPaymentModel_gen.go:15:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/internal/config/config.go:4:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/model/thirdPaymentModel_gen.go:17:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/internal/svc/serviceContext.go:8:2: github.com/zeromicro/[email protected]: Get 
"https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/model/thirdPaymentModel_gen.go:19:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/payment.go:15:2: github.com/zeromicro/[email protected]: Get "https://goproxy.cn/github.com/zeromicro/go-zero/@v/v1.3.4.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/payment.go:16:2: google.golang.org/[email protected]: Get "https://goproxy.cn/google.golang.org/grpc/@v/v1.46.2.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/pb/payment.pb.go:12:2: google.golang.org/[email protected]: Get "https://goproxy.cn/google.golang.org/grpc/@v/v1.46.2.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/payment.go:17:2: google.golang.org/[email protected]: Get "https://goproxy.cn/google.golang.org/grpc/@v/v1.46.2.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/pb/payment.pb.go:13:2: google.golang.org/[email protected]: Get "https://goproxy.cn/google.golang.org/grpc/@v/v1.46.2.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:57976->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/pb/payment.pb.go:14:2: google.golang.org/[email protected]: Get "https://goproxy.cn/google.golang.org/protobuf/@v/v1.28.0.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:39899->127.0.0.11:53: i/o timeout
looklook         | app/payment/cmd/rpc/pb/payment.pb.go:15:2: google.golang.org/[email protected]: Get "https://goproxy.cn/google.golang.org/protobuf/@v/v1.28.0.zip": dial tcp: lookup goproxy.cn on 127.0.0.11:53: read udp 127.0.0.1:39899->127.0.0.11:53: i/o timeout
looklook         | exit status 1

windows WSL2 mysql容器一直重启,报错信息...

[ERROR] [MY-010326] [Server] Fatal error: Can't open and lock privilege tables: Table 'mysql.user' doesn't exist

[ERROR] [MY-010952] [Server] The privilege system failed to initialize correctly. For complete instructions on how to upgrade MySQL to a new version please see the 'Upgrading MySQL' section from the MySQL manual.

[ERROR] [MY-010119] [Server] Aborting

image
这个文件里面是空的

go-stash配置es密码

不知道是否正确

Output:
      ElasticSearch:
        Hosts:
          - "http://elasticsearch:9200"
        Index: "dev-{{yyyy-MM-dd}}"
        User: "elastic"
        Password: "elastic"

filebeat收集不到日志

请检查下docker的版本
centos默认安装的版本是1.13(此版本不行)
本人测试的是18.03.1-ce(可以)

【安装环境】查看服务环境 Grafana 失败

环境:
python -V -> Python 3.7.10
python3 -V -> Python 3.5.2
redis-cli 6.2.6
etcd Version: 3.5.2
Docker version 18.09.7, build 2d0083d
mysql Ver 14.14 Distrib 5.7.33
Linux Version: Ubuntu 16.04

在查看服务环境的时候, Elastic,jaeger等都正确,但是查看Grafana: http://127.0.0.1:3001/ 出错了,
查看日志 docker logs grafana :
t=2022-04-26T17:12:50+0800 lvl=eror msg="Failed to get settings" logger=context userId=0 orgId=0 uname= error="unable to open database file: no such file or directory"

t=2022-04-26T17:12:50+0800 lvl=eror msg="Request Completed" logger=context userId=0 orgId=0 uname= method=GET path=/login status=500 remote_addr=172.18.0.1 time_ms=12 size=1728 referer=http://127.0.0.1:3001/login

浏览器输出:
GET http://127.0.0.1:3001/login 500 (Internal Server Error)

看了项目的issues以及Google,还是没解决,请问有小伙伴遇到过吗?

casbin验权的问题

线上的casbin,一般是怎么应用的?
参考looklook项目,想加入casbin的验权功能,有几个问题想听听大佬的分析?

  1. 如果各个独立的微服务的api里面使用中间件引入casbin,这样做需要各个app需要使用同一个数据库里面的casbin_rule表,与一个app一个库文件背道而驰了?有什么好的方法?
  2. 或者在identity-api里面加casbin_rulle中间件,在nginx的auth_request这个层面就把RBAC验权给办了?
  3. 重要业务的微服务app如何单独验权?比如说订单服务app,需要验证这个订单的所属用户ID;验证订单所属商家租户的id等等?

1.3.0-cli/model/insert.tpl中的一个缺省,导致模板替换无法生成modle内容

func (m *default{{.upperStartCamelObject}}Model) Insert(session sqlx.Session, data *{{.upperStartCamelObject}}) (sql.Result,error) {
{{if .withCache}}{{if .containsIndexCache}}{{.keys}}
return m.Exec(func(conn sqlx.SqlConn) (result sql.Result, err error) {
//@todo self edit value , because change table field is trouble in here , so self fix field is easy
//query := fmt.Sprintf("insert into %s (...) values (...) m.table)
query := fmt.Sprintf("insert into %s (...) values (...)", m.table)

	if session != nil{
		//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
		return session.Exec(query,{{.expressionValues}})
	}
	//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
	return conn.Exec(query, {{.expressionValues}})
}, {{.keyValues}}){{else}}
//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
query := fmt.Sprintf("insert into %s (....) values (...)", m.table)
if session != nil{
	//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
   return session.Exec(query, {{.expressionValues}})
}
//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
return m.ExecNoCache(query, {{.expressionValues}})
{{end}}{{else}}
//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
query := fmt.Sprintf("insert into .... (%s) values ...", m.table)
if session != nil{
	//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
	return session.Exec(query,{{.expressionValues}})
}
//@todo self edit  value , because change table field is trouble in here , so self fix field is easy
return m.conn.Exec(query, {{.expressionValues}})
{{end}}

}

【安装环境】项目所依赖的环境命令执行出错

环境:
python -V -> Python 3.7.10
python3 -V -> Python 3.5.2
redis-cli 6.2.6
etcd Version: 3.5.2
Docker version 18.09.7, build 2d0083d
mysql Ver 14.14 Distrib 5.7.33
Linux Version: Ubuntu 16.04

在执行时,提示:docker-compose -f docker-compose-env.yml up -d

The program 'docker-compose' is currently not installed. You can install it by typing:
sudo apt install docker-compose

然后使用命令 sudo apt install docker-compose 安装 docker-compose。

安装完毕后执行: docker-compose -f docker-compose-env.yml up -d , 报错:
$ docker-compose -f docker-compose-env.yml up -d
/home/.local/lib/python3.7/site-packages/pkg_resources/init.py:119: PkgResourcesDeprecationWarning: 0.14ubuntu0.2 is an invalid version and will not be supported in a future release
PkgResourcesDeprecationWarning,
Traceback (most recent call last):
File "/usr/bin/docker-compose", line 5, in
from pkg_resources import load_entry_point

使用命令 sudo apt-get remove docker-compose 移除后,使用下述步骤安装:

  1. sudo curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
  2. sudo chmod +x /usr/local/bin/docker-compose
  3. sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
  4. $ docker-compose --version
    /bin/sh: /tmp/_MEIcXgwby/libtinfo.so.5: no version information available (required by /bin/sh)
    docker-compose version 1.21.0, build 5920eb0

然后再执行: docker-compose -f docker-compose-env.yml up -d

搭建开发环境常见问题解决方案

创建阶段,起docker-compose-env.yml容器
Grafana 报错You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migrate-to-v51-or-later
mkdir: can't create directory '/var/lib/grafana/plugins': Permission denied
因权限问题导致,可在docker-compose-env.yml中grafana部分加入user: root

filebeat容器启动报错Exiting: error loading config file: config file ("filebeat.yml") must be owned by the user identifier (uid=0) or root
因文件所有者不同导致(我在普通用户下clone的项目),filebeat的配置文件所有者必须为root,需修改sudo chown root deploy/filebeat/conf/filebeat.yml

elasticsearch容器启动报错ElasticsearchException[failed to bind service]; nested: AccessDeniedException[/usr/share/elasticsearch/data/nodes];
Likely root cause: java.nio.file.AccessDeniedException: /usr/share/elasticsearch/data/nodes
报错原因es没有权限操作挂载目录,无法绑定节点,解决方法,修改权限sudo chmod 777 data/elasticsearch/data (不知道es是哪个用户启动的,所以硬改了777)

jaeger依赖于elasticsearch,且没有失败自动重启

环境配置问题go-stash: connect: connection refused: no Elasticsearch node available

ubuntu16.04, 参考looklook搭建自己服务环境,现在使用docker logs查看 kafka日志,发现错误:
[2022-05-20 12:04:14,070] WARN [Controller id=1001, targetBrokerId=1001] Connection to node 1001 (cloudxr-kafka/172.80.0.8:49092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)

端口我用49092,改回9092, 就可以了。 但是我我的go-stash容器一直没有启动成功,错误日志如下:
2022/05/20 15:13:23 stash.go:57 health check timeout: Head "http://127.0.0.1:9200": dial tcp 127.0.0.1:9200: connect: connection refused: no Elasticsearch node available
2022/05/20 15:13:23 {"@timestamp":"2022-05-20T15:13:23.464+08","level":"fatal","content":"stash.go:57 health check timeout: Head "http://127.0.0.1:9200\": dial tcp 127.0.0.1:9200: connect: connection refused: no Elasticsearch node available"}。

我的配置如下:

#zookeeper是kafka的依赖
zookeeper:
image: wurstmeister/zookeeper
container_name: cloudxr-zookeeper
environment:
# 时区上海
TZ: Asia/Shanghai
restart: always
ports:
- 42181:2181
networks:
- cloudxr

#消息队列
kafka:
image: wurstmeister/kafka
container_name: cloudxr-kafka
ports:
- 9092:9092 # TCP
environment:
- KAFKA_ADVERTISED_HOST_NAME=cloudxr-kafka
- KAFKA_ZOOKEEPER_CONNECT=cloudxr-zookeeper:2181
- KAFKA_AUTO_CREATE_TOPICS_ENABLE=false
- TZ=Asia/Shanghai
restart: always
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- cloudxr
depends_on:
- zookeeper

#收集业务数据
filebeat:
image: elastic/filebeat:7.13.4
container_name: cloudxr-filebeat
environment:
# 时区上海
TZ: Asia/Shanghai
user: root
restart: always
entrypoint: "filebeat -e -strict.perms=false" #解决配置文件权限问题
volumes:
- ./deploy/filebeat/conf/filebeat.yml:/usr/share/filebeat/filebeat.yml
# 此处需指定docker的containers目录,取决于你docker的配置
# 如snap安装的docker,则为/var/snap/docker/common/var-lib-docker/containers
# - /var/snap/docker/common/var-lib-docker/containers:/var/lib/docker/containers
- /var/lib/docker/containers:/var/lib/docker/containers
networks:
- cloudxr
depends_on:
- kafka

#搜集kafka业务日志、存储prometheus监控数据
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.13.4
container_name: cloudxr-elasticsearch
user: root
environment:
- discovery.type=single-node
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- TZ=Asia/Shanghai
volumes:
- ./data/elasticsearch/data:/usr/share/elasticsearch/data
restart: always
ports:
- 9200:9200
- 9300:9300
networks:
- cloudxr

#消费kafka中filebeat收集的数据输出到es
go-stash:
image: kevinwan/go-stash:1.0 # if you "macOs intel" or "linux amd"
container_name: cloudxr-go-stash
environment:
# 时区上海
TZ: Asia/Shanghai
user: root
restart: always
volumes:
- ./deploy/go-stash/etc:/app/etc
networks:
- cloudxr
depends_on:
- elasticsearch
- kafka

#查看elasticsearch数据
kibana:
image: docker.elastic.co/kibana/kibana:7.13.4
container_name: cloudxr-kibana
environment:
- elasticsearch.hosts=http://127.0.0.1:9200
- TZ=Asia/Shanghai
restart: always
networks:
- cloudxr
ports:
- "45601:5601"
depends_on:
- elasticsearch

deploy/go-stash/etc/config.yaml:
Clusters:

  • Input:
    Kafka:
    Name: gostash
    Brokers:
    - "cloudxr-kafka:9092"
    Topics:
    - cloudxr-log
    Group: pro
    Consumers: 16
    Filters:
    • Action: drop
      Conditions:
      • Key: k8s_container_name
        Value: "-rpc"
        Type: contains
      • Key: level
        Value: info
        Type: match
        Op: and
    • Action: remove_field
      Fields:
      #- message
      • _source
      • _type
      • _score
      • _id
      • "@Version"
      • topic
      • index
      • beat
      • docker_container
      • offset
      • prospector
      • source
      • stream
      • "@metadata"
    • Action: transfer
      Field: message
      Target: data
      Output:
      ElasticSearch:
      Hosts:

代码版本管理问题

look项目文档中涉及到的identity不在main,在分支add-license,要不要合并下

Request header does not carry JWT token runtime error: invalid memory address or nil

请求 /identity/v1/verify/token 接口若未携带jwttoken, error: invalid memory address

tokenHandler.go

		l := verify.NewTokenLogic(r.Context(), ctx)
		resp, err := l.Token(req, r)
		if err == nil && !resp.Ok {      // 由于resp为nil 时在 if !resp.Ok 则发生错误
			err = errors.Wrapf(ErrTokenExpireError, "jwtAuthHandler JWT Auth no err , userId is zero , req:%+v,resp:%+v", req, resp)
		}

go-stash Group Coordinator Not Available

基于docker-compose-env.yaml 中 完成 filebeat -- kafka -- go-stash -- es --kibana 日志收集部署,容器启动后发现go-stash 报错, 日志如下:

{"@timestamp":"2022-02-20T17:25:15.412+08","level":"error","content":"queue.go:170 Error on reading message, \"[15] Group Coordinator Not Available: the broker returns this error code for group coordinator requests, offset commits, and most group management requests if the offsets topic has not yet been created, or if the group coordinator is not active\""}

日志显示kafka group coordinator 存在问题

[Question] 请问api和rpc业务功能上设计的考虑?

Question

请问在设计上,哪些logic要放在api,哪些logic要放在rpc呢?
travel api为例,

  1. 对于其他路径, 处理逻辑都是自己读db或计算
  2. 对于/homestayDetail路径,它会转发给travel rpc处理,然后travel rpc会读db

我比较疑惑的是处理逻辑放在api还是rpc是怎么考虑的呢?

看调用拓扑图,rpc还会被其他api调用,那么是将那些可能被其他api所调用的功能放在rpc来复用吗? 可是判断哪些功能会被复用,以及后续的增减似乎并不轻松.

Background

我此前的理解是所有服务都使用rpc来编写,同时使用grpc-gateway来生成grpc的http代理,可以一个rpc server一个grpc gateway,然后使用一个nginx来反向代理所有的gateway,也可以多个rpc server一个grpc gateway,gateway本身就充当nginx网关的反向代理功能.

这样的好处是对于rpchttp rest,我只需要维护一个proto文件就可以了. 在gozero中,api和pb是需要分别定义的,但在事实实践上,api的定义和pb的定义又基本是一致的,但却需要定义两次,所以这个会让我比较困惑.

不过看了本仓库,倒是解决了我这个困惑,那就是api并不是rpc的一个协议转换代理,但也引出了上面的第一个问题,哪些logic要放在api,哪些logic要放在rpc呢?

感谢解答!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.