idoubi / goz Goto Github PK
View Code? Open in Web Editor NEWA fantastic HTTP request libarary used in Golang.
License: MIT License
A fantastic HTTP request libarary used in Golang.
License: MIT License
http.Request Close 参数没有设置
httpClient 会默认在下一次请求时复用同一个 tcp链接
然而此时对方已经关闭,可能会出现 EOF 错误
参考:https://stackoverflow.com/questions/28046100/golang-http-concurrent-requests-post-eof
使用fmt.Println(response.Raw)
获取,得到如下结果
&{200 OK 200 HTTP/1.1 1 1 map[Cache-Control:[private, must-revalidate] Connection:[keep-alive] Content-Type:[text/html; charset=UTF-8] Date:[Thu, 30 May 2019 06:35:03 GMT] Expires:[-1] Pragma:[no-cache] Server:[nginx] Set-Cookie:[XSRF-TOKEN=eyJpdiI6ImVZOThBVFlRZ3MzZmNsaVJmczd6Z3c9PSIsInZhbHVlIjoiSUdiSU5MblF0XC83N25iUXliMjNsNUlqTFE0MXpJQ3RLdU1LeGMrR1dmWG9pT25cL2hrd2I2ZG5iQlVnQVNTblJzZENCcm1LcTlcL3RVelZ3RFNmOWtkYlE9PSIsIm1hYyI6IjU2ZWU2MGY5NGE1OWJhNjBiZjM5YTFhNDgwMzJkNjViNDkxYmI2NGEyZmUzZGQ0N2NkNjhmZDQ5NDg2YTcyOGIifQ%3D%3D; expires=Thu, 13-Jun-2019 06:35:03 GMT; Max-Age=1209600; path=/ laravel_session=eyJpdiI6ImpPMTI0MHI2WnM2OGw2RERpN1NOaWc9PSIsInZhbHVlIjoiSjhkUUJ4TFdIN2VGSkFQd0NkcUZGXC90dkFZOG1oSE1wWGwwVnFLTm1qejVqN1VBSWlrNWdlck9ZQnQxbHRHMmVyZGdwUmlcL1hxXC9qaTY5QlFaTTNUNEE9PSIsIm1hYyI6IjI3NzA4OTZiOTcxNzFiZjM0MTgzNGNlYmMyMjc0NmY2MDJhNTUxNzRmM2YwMWNjNDQ0ODBkOTQ0NjJlMTU0Y2MifQ%3D%3D; expires=Thu, 13-Jun-2019 06:35:03 GMT; Max-Age=1209600; path=/; HttpOnly] Vary:[Accept-Encoding] X-Content-Type-Options:[nosniff]] 0xc0002349e0 -1 [chunked] false true map[] 0xc000110000 0xc0000d02c0}
使用fmt.Println(response.Headers)
获取,得到如下结果
map[Cache-Control:private, must-revalidate Connection:keep-alive Content-Type:text/html; charset=UTF-8 Date:Thu, 30 May 2019 06:35:03 GMT Expires:-1 Pragma:no-cache Server:nginx Set-Cookie:XSRF-TOKEN=eyJpdiI6ImVZOThBVFlRZ3MzZmNsaVJmczd6Z3c9PSIsInZhbHVlIjoiSUdiSU5MblF0XC83N25iUXliMjNsNUlqTFE0MXpJQ3RLdU1LeGMrR1dmWG9pT25cL2hrd2I2ZG5iQlVnQVNTblJzZENCcm1LcTlcL3RVelZ3RFNmOWtkYlE9PSIsIm1hYyI6IjU2ZWU2MGY5NGE1OWJhNjBiZjM5YTFhNDgwMzJkNjViNDkxYmI2NGEyZmUzZGQ0N2NkNjhmZDQ5NDg2YTcyOGIifQ%3D%3D; expires=Thu, 13-Jun-2019 06:35:03 GMT; Max-Age=1209600; path=/ Vary:Accept-Encoding X-Content-Type-Options:nosniff]
可以发现,laravel_session
消失了。
应该是下面这段代码的问题:
for k, v := range this.Raw.Header {
headers[k] = v[0]
}
如题.
优秀的项目 加油
RT.
go get 下来的代码不是最新的~
同时在多个 goroutine 中使用 goz 下载文件时,会发生段错误
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x40 pc=0x67d7b9]
goroutine 20255 [running]:
github.com/idoubi/goz.(*Response).GetBody(0xc012042a60, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/gs/go/pkg/mod/github.com/idoubi/[email protected]/response.go:46 +0x59
main.DownloadFile(0xc011180240, 0x3d, 0xc010990740, 0x34, 0x2)
/home/gs/face/douban/douban.go:403 +0x35a
main.DownloadPhotos.func1(0xc011180240, 0x3d, 0xc0119fe240, 0xc00b104000)
/home/gs/face/douban/douban.go:86 +0x12b
created by main.DownloadPhotos
/home/gs/face/douban/douban.go:83 +0x433
douban.go:403 行的代码是
body, err := response.GetBody()
也就是在调用 GetBody() 的时候会出错
所以 goz 是 goroutine 安全的吗?如果不是的话还建议在 README 中说明,以便用户自己做并发控制
在较新版本的curl还支持一个为例子显示下:–resolve的选项,可以直接用来指定对url的解析
curl --resolve aqzt.com:80:127.0.0.1 http://aqzt.com
我这边以为BaseURI就类似axios里面的BaseURI,设置完了调用Get啥的可以不需要接url,但是,发现,不行,好像根本没拼接进去,大佬能说哈咋用吗?
// 看一下这段代码,我已经调试完毕,下载文件使用的代码,看看怎么融合到你的这个包
func Down(resource_url string,sava_path string) {
uri, err := url.ParseRequestURI(resource_url)
if err != nil {
panic("网址错误")
}
filename := path.Base(uri.Path)
client := &http.Client{};
client.Timeout = time.Second * 600 //设置超时时间
resp, err := client.Get(resource_url)
if err != nil {
panic(err)
}
if resp.ContentLength <= 0 {
log.Println("[*] Destination server does not support breakpoint download.")
}
raw := resp.Body
defer raw.Close()
reader := bufio.NewReaderSize(raw, 1024*50); //相当于一个临时缓冲区(设置为可以单次存储50M的文件),每次读取以后就把原始数据重新加载一份,等待下一次读取
//根据目录创建镜像文件,等待数据写入
if !strings.HasSuffix(sava_path,"/"){
sava_path+="/"
}
fmt.Println(sava_path+filename)
file,err:=os.OpenFile(sava_path+filename,os.O_WRONLY |os.O_CREATE,0777)
if err != nil {
log.Panic("创建镜像文件失败,无法进行后续的写入操作"+err.Error())
}
writer := bufio.NewWriter(file)
buff := make([]byte, 32*1024)
for {
curr_read_size, reader_err := reader.Read(buff)
if curr_read_size > 0 {
write_size, write_err := writer.Write(buff[0:curr_read_size])
if write_err != nil {
log.Panic("写入发生错误"+write_err.Error(),"写入长度:",write_size)
break
}
}
// 读取结束
if reader_err == io.EOF {
writer.Flush()
break
}
}
fmt.Println("下载完毕!")
}
如果用户输入为
Query: map[string]interface{}{
"iReportID": 20,
"iModuleID": 1,
},
转换string 那也会有报错
我有个定时任务,使用curl发出请求的。
发现curl产生的openfiels数量越来越大,是不是请求后没有主动关闭
兄得,你的 0.1.0 发布版本里错误咋 panic 了捏。
只使用了
url := "http://www.baidu.com"
resp, err := req.SetUrl(url).Get()
无论url是什么都会显示
Get http://www.baidu.com: read tcp xxx.xxx.xxx.xxx: xxxx ->115.239.211.112:80: i/o timeout
rt,go
的httpClient没有自动设置请求超时,希望可以设置r/w的超时时间
// can be adjusted to
func (r *Response) GetReasonPhrase() string {
status := r.resp.Status
return status[strings.Index(status, " ")+1:]
}
step3 step4发起的step2请求,都可以正常解析,step1发起的无法解析表单数据
package main
import (
"encoding/json"
"fmt"
"github.com/gin-gonic/gin"
"github.com/mikemintang/go-curl"
"github.com/rifflock/lfshook"
"github.com/sirupsen/logrus"
"io/ioutil"
"net/http"
"net/url"
"os"
"strings"
)
func main() {
pathMap := lfshook.PathMap{
logrus.InfoLevel: "./info.log",
logrus.ErrorLevel: "./error.log",
}
Log := logrus.New()
Log.Hooks.Add(lfshook.NewHook(pathMap, &logrus.JSONFormatter{}));
router := gin.New()
router.GET("/step1", func(c *gin.Context) {
header := map[string]string{
"Content-Type": "application/x-www-form-urlencoded",
//"X-Requested-With": "XMLHttpRequest",
}
url := "http://localhost:81/step2";
request := curl.NewRequest()
postData := map[string]interface{}{
"name": "zhangsan",
"age": 18,
};
resp, err := request.SetUrl(url).SetHeaders(header).SetPostData(postData).Post();
if err != nil {
Log.Error("发生错误 => ", err);
}
Log.Info("请求成功,响应内容为 => ", resp.Body)
c.String(http.StatusOK, resp.Body);
});
router.POST("/step2", func(c *gin.Context) {
// 获取请求数据
header := c.Request.Header;
header_tmp, _ := json.Marshal(header)
header_str := string(header_tmp)
form := c.Request.Form;
name := c.PostForm("name");
age := c.Request.PostFormValue("age");
fmt.Fprintln(os.Stdout, "form name数据 => ", name);
fmt.Fprintln(os.Stdout, "form age数据 => ", age);
form_tmp, _ := json.Marshal(form);
form_str := string(form_tmp)
c.String(http.StatusOK, "请求方式 => "+c.Request.Method)
c.String(http.StatusOK, "请求头 => "+header_str)
c.String(http.StatusOK, "form数据 => "+form_str)
c.String(http.StatusOK, "接收到的name数据 => "+name)
c.String(http.StatusOK, "接收到的age数据 => "+age)
age2, _ := c.GetPostForm("age");
c.String(http.StatusOK, "接收到的age数据 => "+age2)
})
router.GET("/step3", func(c *gin.Context) {
client := http.Client{}
url := "http://localhost:81/step2";
postdata_str := "name=zhangsan&age=18";
postdata := strings.NewReader(postdata_str);
request, _ := http.NewRequest("POST", url, postdata);
request.Header.Add("Content-Type", "application/x-www-form-urlencoded;");
//request.Header.Add("X-Requested-With", "XMLHttpRequest");
resp, err := client.Do(request);
if err != nil {
Log.Error("发生错误 => ", err);
}
defer resp.Body.Close();
Log.Info("请求成功,响应内容为 => ", resp.Body)
body, _ := ioutil.ReadAll(resp.Body)
c.String(http.StatusOK, string(body));
});
router.GET("/step4", func(c *gin.Context) {
client := &http.Client{}
postValues := url.Values{}
postValues.Add("name", "zhangsan")
postValues.Add("age", "18")
url := "http://localhost:81/step2";
resp, err := client.PostForm(url, postValues)
defer resp.Body.Close()
if err != nil {
fmt.Println(err.Error())
}
if resp.StatusCode == 200 {
body, _ := ioutil.ReadAll(resp.Body)
c.String(http.StatusOK, string(body));
}
});
router.Run(":81");
}
师兄的前身是写php的吧?
哈哈,一看代码格式就有一种php的感觉。
request.go 文件中 panic 没有处理 ~ 客户端引用很容易出问题希望能直接返回err 而不是夹着 panic . 谢谢
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.