bytes.makeSlice问题

程序内存一直在飙升,然后使用golang的pprof做分析,发现bytes.makeSlice占用内存最多,执行:go tool pprof --text http://localhost:6060/debug/pprof/heap ,结果如下,大家帮忙分析下问题,纠结了好久,不知道问题在什么地方。

83.94MB of 83.94MB total (  100%)
Dropped 282 nodes (cum <= 0.42MB)
      flat  flat%   sum%        cum   cum%
   78.39MB 93.39% 93.39%    78.39MB 93.39%  bytes.makeSlice
       1MB  1.19% 94.58%        1MB  1.19%  github.com/astaxie/beego/orm.newFieldInfo
       1MB  1.19% 95.77%        1MB  1.19%  reflect.Value.call
    0.52MB  0.62% 96.40%     0.52MB  0.62%  github.com/go-sql-driver/mysql.(*mysqlConn).writeCommandPacketStr
    0.52MB  0.61% 97.01%     0.52MB  0.61%  regexp.(*bitState).reset
    0.51MB   0.6% 97.62%     0.51MB   0.6%  fmt.(*fmt).padString
    0.50MB   0.6% 98.21%     0.50MB   0.6%  text/template.addValueFuncs
    0.50MB   0.6% 98.81%     0.50MB   0.6%  github.com/astaxie/beego/orm.(*fields).Add
    0.50MB   0.6% 99.40%     0.50MB   0.6%  text/template/parse.(*Tree).newField
    0.50MB   0.6%   100%        1MB  1.19%  text/template/parse.(*Tree).textOrAction
         0     0%   100%    78.39MB 93.39%  bytes.(*Buffer).Write
         0     0%   100%    78.39MB 93.39%  bytes.(*Buffer).grow
         0     0%   100%     0.52MB  0.62%  database/sql.(*DB).Query
         0     0%   100%     0.52MB  0.62%  database/sql.(*DB).query
         0     0%   100%     0.52MB  0.62%  database/sql.(*DB).queryConn
         0     0%   100%     0.51MB   0.6%  fmt.(*fmt).fmt_s
         0     0%   100%     0.51MB   0.6%  fmt.(*pp).doPrint
         0     0%   100%     0.51MB   0.6%  fmt.(*pp).fmtString
         0     0%   100%     0.51MB   0.6%  fmt.(*pp).printArg
         0     0%   100%     0.51MB   0.6%  fmt.Fprint
         0     0%   100%     2.53MB  3.01%  github.com/astaxie/beego.(*Controller).Render
         0     0%   100%     2.53MB  3.01%  github.com/astaxie/beego.(*Controller).RenderBytes
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego.(*Controller).renderTemplate
         0     0%   100%     0.52MB  0.61%  github.com/astaxie/beego.(*ControllerRegister).FindRouter
         0     0%   100%    80.41MB 95.80%  github.com/astaxie/beego.(*ControllerRegister).ServeHTTP
         0     0%   100%     0.52MB  0.61%  github.com/astaxie/beego.(*Tree).Match
         0     0%   100%     0.52MB  0.61%  github.com/astaxie/beego.(*Tree).match
         0     0%   100%     0.52MB  0.61%  github.com/astaxie/beego.(*leafInfo).match
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego.BuildTemplate
         0     0%   100%     2.53MB  3.01%  github.com/astaxie/beego.ExecuteTemplate
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego.Run
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego.getTemplate
         0     0%   100%        1MB  1.19%  github.com/astaxie/beego.getTplDeep
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego.initBeforeHTTPRun
         0     0%   100%    77.37MB 92.18%  github.com/astaxie/beego.openFile
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego.registerTemplate
         0     0%   100%    77.37MB 92.18%  github.com/astaxie/beego.serverStaticRouter
         0     0%   100%    77.37MB 92.18%  github.com/astaxie/beego/context.(*nopResetWriter).Write
         0     0%   100%    77.37MB 92.18%  github.com/astaxie/beego/context.WriteFile
         0     0%   100%    77.37MB 92.18%  github.com/astaxie/beego/context.writeLevel
         0     0%   100%     0.52MB  0.62%  github.com/astaxie/beego/orm.(*rawSet).Values
         0     0%   100%     0.52MB  0.62%  github.com/astaxie/beego/orm.(*rawSet).readValues
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego/orm.RegisterModelWithPrefix
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego/orm.addModelFields
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego/orm.newModelInfo
         0     0%   100%     1.50MB  1.79%  github.com/astaxie/beego/orm.registerModel
         0     0%   100%     0.52MB  0.62%  github.com/go-sql-driver/mysql.(*mysqlConn).Query
         0     0%   100%     2.53MB  3.01%  html/template.(*Template).ExecuteTemplate
         0     0%   100%     0.50MB   0.6%  html/template.(*Template).Funcs
         0     0%   100%        1MB  1.19%  html/template.(*Template).Parse
         0     0%   100%    77.37MB 92.18%  io.Copy
         0     0%   100%    77.37MB 92.18%  io.copyBuffer
         0     0%   100%     1.50MB  1.79%  main.init
         0     0%   100%     1.50MB  1.79%  main.init.1
         0     0%   100%     1.50MB  1.79%  main.main
         0     0%   100%    79.91MB 95.20%  net/http.(*conn).serve
         0     0%   100%    80.41MB 95.80%  net/http.serverHandler.ServeHTTP
         0     0%   100%        1MB  1.19%  reflect.Value.Call
         0     0%   100%     0.52MB  0.61%  regexp.(*Regexp).MatchString
         0     0%   100%     0.52MB  0.61%  regexp.(*Regexp).doExecute
         0     0%   100%     0.52MB  0.61%  regexp.(*machine).backtrack
         0     0%   100%    83.44MB 99.40%  runtime.goexit
         0     0%   100%        3MB  3.58%  runtime.main
         0     0%   100%     2.53MB  3.01%  text/template.(*Template).Execute
         0     0%   100%     0.50MB   0.6%  text/template.(*Template).Funcs
         0     0%   100%        1MB  1.19%  text/template.(*Template).Parse
         0     0%   100%     2.53MB  3.01%  text/template.(*Template).execute
         0     0%   100%     0.50MB   0.6%  text/template.(*state).evalArg
         0     0%   100%        1MB  1.19%  text/template.(*state).evalCall
         0     0%   100%        1MB  1.19%  text/template.(*state).evalCommand
         0     0%   100%        1MB  1.19%  text/template.(*state).evalFunction
         0     0%   100%        1MB  1.19%  text/template.(*state).evalPipeline
         0     0%   100%     0.51MB   0.6%  text/template.(*state).printValue
         0     0%   100%     2.53MB  3.01%  text/template.(*state).walk
         0     0%   100%     1.52MB  1.81%  text/template.(*state).walkRange
         0     0%   100%     1.52MB  1.81%  text/template.(*state).walkRange.func1
         0     0%   100%     0.52MB  0.61%  text/template.(*state).walkTemplate
         0     0%   100%        1MB  1.19%  text/template/parse.(*Tree).Parse
         0     0%   100%        1MB  1.19%  text/template/parse.(*Tree).action
         0     0%   100%     0.50MB   0.6%  text/template/parse.(*Tree).command
         0     0%   100%     0.50MB   0.6%  text/template/parse.(*Tree).ifControl
         0     0%   100%     0.50MB   0.6%  text/template/parse.(*Tree).itemList
         0     0%   100%     0.50MB   0.6%  text/template/parse.(*Tree).operand
         0     0%   100%        1MB  1.19%  text/template/parse.(*Tree).parse
         0     0%   100%     0.50MB   0.6%  text/template/parse.(*Tree).parseControl
         0     0%   100%     0.50MB   0.6%  text/template/parse.(*Tree).pipeline
         0     0%   100%        1MB  1.19%  text/template/parse.Parse
         0     0%   100%     0.52MB  0.62%  wangpan/models.(*Share).GetByIds
         0     0%   100%     1.50MB  1.79%  wangpan/models.Init
         0     0%   100%     1.50MB  1.79%  wangpan/models.RegisterDB
         0     0%   100%     0.52MB  0.62%  wangpan/models.SitemapUpdate
         0     0%   100%     0.52MB  0.62%  wangpan/models.SitemapUpdate.func1
         0     0%   100%     0.52MB  0.62%  wangpan/models.Tmp2data
         0     0%   100%     0.52MB  0.62%  wangpan/models.sitemapBuild
已邀请:

stevewang

赞同来自:

这两天我也遇到同样的问题,CPU都飙满了。 后来检查发现是程序逻辑里的一个低级错误,原因是内存不够时申请了缓冲区,但是没有把创建的缓冲区保存下来,导致重复不断申请内存。

cctamlulu

赞同来自:

上面这个问题是底层bytes.makeSlice申请了很多内存,我遇到的例子是json.Marshal---最终在makeSlice上申请很多内存,但是在我例子里后面很多内存都会被回收

huhuyou2 - fish

赞同来自:

1、检查一下全局变量的使用 2、检查一下申请空间时候是否指定了合理的初始大小

pathbox - https://pathbox.github.io/

赞同来自:

bytes.makeSlice 一般情况都是消耗内存最多的。 就像楼上说的, 你的全局变量 比如数组 map,缓存等是否一直增长 导致内存泄漏,随着程序运行时间增长,内存逐渐增加而没有被GC回收

youzhengchuan - Perl Python Golang

赞同来自:

  • 如果使用了bytes包的Buffer对象,那么你每次使用之后,都要记得回收,否则可能会导致内存不能释放。
  • 如果使用了bytes.Buffer,建议使用sync.pool对创建的对象进行管理,例子如下:
import (
    "sync"
    "bytes"
)
var bufPool = sync.Pool{New: func() interface{} { return new(bytes.Buffer) }}

func getBuf() *bytes.Buffer {
        if bb, ok := bufPool.Get().(*bytes.Buffer); ok {
                return bb
        }
        return new(bytes.Buffer)
}

func putBuf(b *bytes.Buffer) {
        b.Reset()
        bufPool.Put(b)
}

func  Example()  {
        b := getBuf()
        defer putBuf(b)
        for _, item := range items {
                b.WriteString("Example String ...")
                b.WriteByte('\n')
        }
        b.Reset()
}

lrita

赞同来自:

用火焰图看,能最直观的看出谁触发的bytes.makeSlice次数最多

要回复问题请先登录注册