你使用的Go版本是什么( go version
)?
go version go1.11 linux/amd64
这个问题在最新版本中是否会重现?
是的。
你正在使用什么操作系统和处理器架构( go env
)?
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/kivikakk/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/kivikakk/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build515123667=/tmp/go-build -gno-record-gcc-switches"
你做了什么?
package main
/*
#include <string.h>
#include <stdlib.h>
char* s() {
return strdup("hello");
}
*/
import "C"
import "unsafe"
func main() {
s := C.s()
C.GoString(s)
C.free(unsafe.Pointer(s))
}
$ go build
$ valgrind ./sscce
==11241== Memcheck, a memory error detector
==11241== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==11241== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==11241== Command: ./sscce
==11241==
==11241== Warning: ignored attempt to set SIGRT32 handler in sigaction();
==11241== the SIGRT32 signal is used internally by Valgrind
==11241== Warning: ignored attempt to set SIGRT32 handler in sigaction();
==11241== the SIGRT32 signal is used internally by Valgrind
==11241== Warning: client switching stacks? SP change: 0xfff0001b0 --> 0xc0000367d8
==11241== to suppress, use: --max-stackframe=755931244072 or greater
==11241== Warning: client switching stacks? SP change: 0xc000036790 --> 0xfff000260
==11241== to suppress, use: --max-stackframe=755931243824 or greater
==11241== Warning: client switching stacks? SP change: 0xfff000260 --> 0xc000036790
==11241== to suppress, use: --max-stackframe=755931243824 or greater
==11241== further instances of this message will not be shown.
==11241== Conditional jump or move depends on uninitialised value(s)
==11241== at 0x40265B: indexbytebody (/usr/local/go/src/internal/bytealg/indexbyte_amd64.s:151)
==11241==
==11241==
==11241== HEAP SUMMARY:
==11241== in use at exit: 1,200 bytes in 6 blocks
==11241== total heap usage: 10 allocs, 4 frees, 1,310 bytes allocated
==11241==
==11241== LEAK SUMMARY:
==11241== definitely lost: 0 bytes in 0 blocks
==11241== indirectly lost: 0 bytes in 0 blocks
==11241== possibly lost: 1,152 bytes in 4 blocks
==11241== still reachable: 48 bytes in 2 blocks
==11241== suppressed: 0 bytes in 0 blocks
==11241== Rerun with --leak-check=full to see details of leaked memory
==11241==
==11241== For counts of detected and suppressed errors, rerun with: -v
==11241== Use --track-origins=yes to see where uninitialised values come from
==11241== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
(忽略"可能丢失"的块;它们是由Go运行时启动的pthreads。)
你期望看到什么?
没有基于未初始化的值的条件跳转/移动。
你看到了什么?
一个基于未初始化的值的条件跳转/移动。
如果你用 --partial-loads-ok=no
运行Valgrind,问题的本质会更加明显:
$ valgrind --partial-loads-ok=no ./sscce
==11376== Memcheck, a memory error detector
==11376== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==11376== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==11376== Command: ./sscce
==11376==
==11376== Warning: ignored attempt to set SIGRT32 handler in sigaction();
==11376== the SIGRT32 signal is used internally by Valgrind
==11376== Warning: ignored attempt to set SIGRT32 handler in sigaction();
==11376== the SIGRT32 signal is used internally by Valgrind
==11376== Warning: client switching stacks? SP change: 0xfff0001b0 --> 0xc0000367d8
==11376== to suppress, use: --max-stackframe=755931244072 or greater
==11376== Warning: client switching stacks? SP change: 0xc000036790 --> 0xfff000260
==11376== to suppress, use: --max-stackframe=755931243824 or greater
==11376== Warning: client switching stacks? SP change: 0xfff000260 --> 0xc000036790
==11376== to suppress, use: --max-stackframe=755931243824 or greater
==11376== further instances of this message will not be shown.
==11376== Invalid read of size 32
==11376== at 0x40264E: indexbytebody (/usr/local/go/src/internal/bytealg/indexbyte_amd64.s:148)
==11376== Address 0x53f47c0 is 0 bytes inside a block of size 12 alloc'd
==11376== at 0x4C2BBAF: malloc (vg_replace_malloc.c:299)
==11376== by 0x45165D: s (main.go:7)
==11376== by 0x4516A5: _cgo_a004886745c9_Cfunc_s (cgo-gcc-prolog:54)
==11376== by 0x44A0DF: runtime.asmcgocall (/usr/local/go/src/runtime/asm_amd64.s:637)
==11376== by 0x7: ???
==11376== by 0x6C287F: ??? (in /home/kivikakk/sscce/sscce)
==11376== by 0xFFF00024F: ???
==11376== by 0x4462B1: runtime.(*mcache).nextFree.func1 (/usr/local/go/src/runtime/malloc.go:749)
==11376== by 0x448905: runtime.systemstack (/usr/local/go/src/runtime/asm_amd64.s:351)
==11376== by 0x4283BF: ??? (/usr/local/go/src/runtime/proc.go:1146)
==11376== by 0x448798: runtime.rt0_go (/usr/local/go/src/runtime/asm_amd64.s:201)
==11376== by 0x451DEF: ??? (in /home/kivikakk/sscce/sscce)
==11376==
==11376==
==11376== HEAP SUMMARY:
==11376== in use at exit: 1,200 bytes in 6 blocks
==11376== total heap usage: 10 allocs, 4 frees, 1,316 bytes allocated
==11376==
==11376== LEAK SUMMARY:
==11376== definitely lost: 0 bytes in 0 blocks
==11376== indirectly lost: 0 bytes in 0 blocks
==11376== possibly lost: 1,152 bytes in 4 blocks
==11376== still reachable: 48 bytes in 2 blocks
==11376== suppressed: 0 bytes in 0 blocks
==11376== Rerun with --leak-check=full to see details of leaked memory
==11376==
==11376== For counts of detected and suppressed errors, rerun with: -v
==11376== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
我知道已经做了一些工作,以确保 IndexByte
不会跨越页面边界( #24206 ),因此这不太可能产生负面影响。我应该只是添加一个抑制并就此了事吗?
{
indexbytebody_loves_to_read
Memcheck:Addr32
fun:indexbytebody
}
8条答案
按热度按时间ua4mk5z41#
感谢您提交了一个问题。
cc @TocarIP
zbwhf8kr2#
在
IndexByte
中存在一段代码,用于避免跨页边界加载,这可能会导致运行时检测器(如valgrind)出错。但您看到的错误并不是可能发生的那种情况。它发生在32字节以上的部分,我们从未在传入的范围内读取过外部数据。产生越界读取的地方是src/runtime/string.go:findnull。我们这样做:
// pageSize is the unit we scan at a time looking for NULL.
// It must be the minimum page size for any architecture Go
// runs on. It's okay (just a minor performance loss) if the
// actual system page size is larger than this value.
const pageSize = 4096
基本上,我们一次查找一个页面的空值。我们将范围传递给IndexByte,该范围可能超出一个小分配的范围。从虚拟内存的Angular 来看它是安全的,因为我们没有偏离页面,但我肯定能理解它会如何使Valgrind感到困惑。
ki0zmccv3#
确实,它在不跑出页面的情况下是安全的,但它确实在读取未初始化的内存(即该部分页面尚未被
malloc
或类似工具分配给用户空间),而Valgrind正在捕获这一点。我理解它是安全的,我们在这里没有问题;只是遗憾这种事情会出现并玷污Valgrind的结果。话虽如此,上述提到的pthread“可能丢失”的分配也是Valgrind的噪音。也许我们可以记录抑制/包含一个样本抑制文件?我怀疑还有许多我没有遇到的,尽管如此。
utugiqy64#
是的,如果我们有一个干净的Valgrind运行,或者有一种方法可以通过配置文件或其他方式诱导一个,那就太好了。我们在Go运行时的一些地方进行了过度读取。通常,过度读取发生在Go堆中,而Valgrind可能并不关心这一点。我认为
findnull
可能是在C分配的内存上发生的最常见操作。但是,如果有人创建了一个指向C分配内存的字符串或切片,那么在涉及这些字符串/切片的操作中也可能发生这种情况。C是如何处理这个问题的呢?我想C库函数,如
strlen
,做的和我们一样的事情。njthzxwz5#
是的,这很有道理。我还没有深入研究过这个问题,但我确实认为C库也做了同样的事情,而且Valgrind有自己的
strlen
,它不会像这样陷入困境。(这看起来像是Valgrind在了解这些优化之前发布的报告:https://stackoverflow.com/a/3246124/499609)ukdjmx9f6#
我认为Go标准库中的所有案例都会被Valgrind选项
--partial-loads-ok=yes
处理,而且我认为自2015年以来一直是这样。你能检查一下看看是否有区别吗?你的Valgrind看起来像是2015年的版本。
xuo3flqw7#
我使用的是3.12.0版本(2016年10月),刚刚尝试了最新的稳定版本3.13.0,结果仍然是一样的:
hfwmuf9z8#
看起来我们不应该仅仅为了让valgrind满意而改变Go库的行为。我们需要一种方法来告诉valgrind,这个特定的未初始化的内存读取是OK的。有没有办法做到这一点?