Go语言HTTP测试及程序性能调优

这篇要讲的东西，主要是HTTP,WebSocket的测试及如何调优Go程序的一些方法.

分下面几个内容:

一.httptest测试包

二.性能测试

三.怎么利用参数分析和调优程序

四.在运行中实时监控调优

一.httptest测试包

对于HTTP和WebSocket测试，Go标准库有一个HTTP测试框架.在"http/httptest"包下.

go1.5.1\go\src\net\http\httptest

怎么用可以在源码目录看例子，也可以上官网看看这个例子:

https://golang.org/src/net/http/request_test.go

里面各种用法还是很全的.

如果想亲自动手试试. https://golang.org/doc/articles/wiki/ 有个很完整的Go Web的例子。

可以搭建起来，再跑一下测试.

比如Post,大致测试程序是这样的:

func TestPost(t *testing.T) {
	req, err := NewRequest("POST", "http://localhost:8080/edit/nn", strings.NewReader("body=xcl"))
	if err != nil {
		t.Errorf("%v", err)
	}
	defer req.Body.Close()
	req.Header.Set("Content-Type", "application/x-www-form-urlencoded; param=value")

	if q := req.FormValue("body"); q != "xcl" {
		t.Errorf(`req.FormValue("body") = %q, want "xcl"`, q)
	} 
}

二.性能测试

除了传统的AB,有一个Go写的好工具叫 boom.

例子:

boom -n 1000 -c 100 https://google.com

命令说明:

Usage: boom [options...] <url>

Options:
  -n  Number of requests to run.
  -c  Number of requests to run concurrently. Total number of requests cannot
      be smaller than the concurency level.
  -q  Rate limit, in seconds (QPS).
  -o  Output type. If none provided, a summary is printed.
      "csv" is the only supported alternative. Dumps the response
      metrics in comma-seperated values format.

  -m  HTTP method, one of GET, POST, PUT, DELETE, HEAD, OPTIONS.
  -h  Custom HTTP headers, name1:value1;name2:value2.
  -t  Timeout in ms.
  -A  HTTP Accept header.
  -d  HTTP request body.
  -T  Content-type, defaults to "text/html".
  -a  Basic authentication, username:password.
  -x  HTTP Proxy address as host:port.

  -readall              Consumes the entire request body.
  -allow-insecure       Allow bad/expired TLS/SSL certificates.
  -disable-compression  Disable compression.
  -disable-keepalive    Disable keep-alive, prevents re-use of TCP
                        connections between different HTTP requests.
  -cpus                 Number of used cpu cores.
                        (default for current machine is 1 cores)

但这个只对HTTP接口之类好使，但像我那种基于WebSocket,使用自定义协议的情况。

还是需要自己定制。

我目前的做法，客户端测试直接参考boom的源码架构，改成支持WebSocket协议。

并加上指定的自定义协议做业务逻辑模拟,用起来还不错。不过有个地方要注意，如果一下并发

连接太大,WebSocket的TCP连接建立可能会超时，在定制时可以扩大下等待时长.

代码如下:

//.......
	 client, err := net.DialTimeout("tcp", serverIP, waiteDial*time.Second)
	if err != nil {
		log.Printf("[testConnect] 用户(%s) net.DialTimeout error:%s", userName, err)
		return
	}
	defer client.Close()

	config, _ := websocket.NewConfig(serverAddr, origin)
	ws, err := websocket.NewClient(config, client)
	if err != nil {
		log.Printf("[testConnect] 用户(%s)连接服务器失败! err:%s", userName, err)
		return
	}
   //.......

三.怎么利用参数分析和调优程序

/*
调优程序例子

go build main.go
main.exe -cpuprofile=cpu.pprof
go tool pprof main.exe  cpu.pprof

Author:xcl
Date: 2015-11-22
*/

package main

import (
  "flag"
  "fmt"
  "os"
  "runtime/pprof"
)

var cpuprofile = flag.String("cpuprofile", "", "write cpu profile to file")

func main() {
  flag.Parse()
  if *cpuprofile != "" {
    f, err := os.Create(*cpuprofile)
    if err != nil {
      fmt.Println("Error: ", err)
    }
    pprof.StartCPUProfile(f)
    defer pprof.StopCPUProfile()
  }

  t1()

}

func t1() {
  for i := 0; i < 10000; i++ {
    fmt.Sprintf("%d", i)
  }
}

/*




E:\GOtest\testing\testpprof3>go build main.go

E:\GOtest\testing\testpprof3>main.exe -cpuprofile=cpu.pprof

E:\GOtest\testing\testpprof3>dir
 驱动器 E 中的卷是 doc
 卷的序列号是 0E3D-2A1F

 E:\GOtest\testing\testpprof3 的目录

2015/11/22  18:39    <DIR>          .
2015/11/22  18:39    <DIR>          ..
2015/11/22  18:39               168 cpu.pprof
2015/11/22  18:38         2,826,752 main.exe
2015/11/22  18:38             1,604 main.go
               3 个文件      2,828,524 字节
               2 个目录 15,171,936,256 可用字节

E:\GOtest\testing\testpprof3>go tool pprof main.exe  cpu.pprof
Entering interactive mode (type "help" for commands)
(pprof) top10
10ms of 10ms total (  100%)
Showing top 10 nodes out of 11 (cum >= 10ms)
      flat  flat%   sum%        cum   cum%
      10ms   100%   100%       10ms   100%  runtime.memmove
         0     0%   100%       10ms   100%  fmt.(*fmt).integer
         0     0%   100%       10ms   100%  fmt.(*fmt).pad
         0     0%   100%       10ms   100%  fmt.(*pp).doPrintf
         0     0%   100%       10ms   100%  fmt.(*pp).fmtInt64
         0     0%   100%       10ms   100%  fmt.(*pp).printArg
         0     0%   100%       10ms   100%  fmt.Sprintf
         0     0%   100%       10ms   100%  main.main
         0     0%   100%       10ms   100%  main.t1
         0     0%   100%       10ms   100%  runtime.goexit
(pprof)

*/

四.在运行中实时监控调优

其实这个，可以参考源码下的 net/http/pprof/pprof.go

/*
调优程序例子

go build main.go

在浏览器边上查看
http://127.0.0.1:7081/debug/pprof/
http://127.0.0.1:7081/debug/pprof/profile

Author:xcl
Date: 2015-11-22
*/

package main

import (
  "fmt"
  "net/http"
  "net/http/pprof"
  "os"
  "time"
)

func main() {
  go func() {
    if err := StartAdminHttp("127.0.0.1:7081"); err != nil {
      os.Exit(-1)
    }
  }()
  t1()
}

func StartAdminHttp(webaddr string) error {
  adminServeMux := http.NewServeMux()
  adminServeMux.HandleFunc("/debug/pprof/", pprof.Index)
  adminServeMux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
  adminServeMux.HandleFunc("/debug/pprof/profile", pprof.Profile)
  adminServeMux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
  err := http.ListenAndServe(webaddr, adminServeMux)
  if err != nil {
    x := fmt.Sprintf("http.ListenAdServe(\"%s\") failed (%s)", webaddr, err.Error())
    fmt.Println(x)
    return err
  }
  return nil
}

func t1() {
  for i := 0; i < 10000; i++ {
    fmt.Sprintf("%d", i)
    time.Sleep(1 * time.Second)
  }
}

另外一种做法:

在程序运行中，在命令行窗口，执行"go tool pprof url... "会也成相应的文档.

以profile为例，会等待30s,然后在 \pprof\ 生成相关文档。然后可使用pprof相关命令来调优。

C:\Users\XCL>go tool pprof http://127.0.0.1:7081/debug/pprof/profile
Fetching profile from http://127.0.0.1:7081/debug/pprof/profile
Please wait... (30s)
Saved profile in \pprof\pprof.127.0.0.1:7081.samples.cpu.001.pb.gz
Entering interactive mode (type "help" for commands)
(pprof)
(pprof)
(pprof) top10
10ms of 10ms total (  100%)
      flat  flat%   sum%        cum   cum%
      10ms   100%   100%       10ms   100%  runtime.runqput
         0     0%   100%       10ms   100%  runtime.goready.func1
         0     0%   100%       10ms   100%  runtime.ready
         0     0%   100%       10ms   100%  runtime.startTheWorldWithSema
         0     0%   100%       10ms   100%  runtime.systemstack
(pprof)

参考文档:

https://golang.org/cmd/go/#hdr-Description_of_testing_flags

http://saml.rilspace.org/profiling-and-creating-call-graphs-for-go-programs-with-go-tool-pprof

http://www.cnblogs.com/yjf512/archive/2012/12/27/2835331.html

BLOG: http://blog.csdn.net/xcl168

Go语言HTTP测试及程序性能调优

继续阅读

golang结构体tag编译错误的问题

Golang结构体中的tag

GO语言笔记--结构体1.说明2.工厂函数

Go语言之map与结构体初入map结构体初入（场景map的value通常为struct）

Go初探 (5) – 结构体与接口

go(7)结构体，方法，接口

Go学习笔记: 结构体、方法、接口

Go 结构体方法指针和值的区别

06结构体、接口结构体接口

go语言笔记（结构体、方法、接口）go语言笔记（结构体、方法、接口）

Go中结构体和接口的定义

Rich Domain Model

【Electron】Error: A dynamic link library (DLL) initialization routine failed.

Golang如何写能进入文档的测试先看看效果吧可测试的Example结语

Linux之父警告全球程序员：我刚发布的5.12内核有bug，你们千万别用

最小化DevOps自动化流程(Golang)