Commit Graph

6138 Commits

Author SHA1 Message Date
Michael Anthony Knyszek b513bd808f runtime: bias the pacer's cons/mark smoothing against noise
Currently the pacer is designed to pace against the edge. Specifically,
it tries to find the sweet spot at which there are zero assists, but
simultaneously finishes each GC perfectly on time.

This pretty much works, despite the noisiness of the measurement of the
cons/mark ratio, which is central to the pacer's function. (And this
noise is basically a given; the cons/mark ratio is used as a prediction
under a steady-state assumption.) Typically, this means that the GC
might assist a little bit more because it started the GC late, or it
might execute more GC cycles because it started early. In many cases the
magnitude of this variation is small.

However, we can't possibly control for all sources of noise, especially
since some noise can come from the underlying system. Furthermore, there
are inputs to the measurement that have effectively no restrictions on
how they vary, and the pacer needs to assume that they're essentially
static when they might not be in some applications (i.e. goroutine
stacks).

The result of high noise is that the variation in when a GC starts is
much higher, leading to a significant amount of assists in some GC
cycles. While the GC cycle frequency basically averages out in the
steady-state in the face of this variation, starting a GC late has the
significant drawback of reducing application latencies.

This CL thus biases the pacer toward avoiding assists by picking a
cons/mark smoothing function that takes the maximum measured cons/mark
over 5 cycles total. I picked 5 cycles because empirically this was the
best trade-off between window size and smoothness for a uniformly
distributed jitter in the cons/mark signal. The cost here is that if
there's a significant phase change in the application that makes it less
active with the GC, then we'll be using a stale cons/mark measurement
for 5 cycles. I suspect this is fine precisely because this only happens
when the application becomes less active, i.e. when latency matters
less.

Another good reason for this particular bias is that even though the GC
might start earlier and end earlier on average, resulting in more
frequent GC cycles and potentially worse throughput, it also means that
it uses less memory used on average. As a result, there's a reasonable
workaround in just turning GOGC up slightly to reduce GC cycle
frequency and bringing memory (and hopefully throughput) levels back to
the same baseline. Meanwhile, there should still be fewer assists than
before which is just a clear improvement to latency.

Lastly, this CL updates the GC pacer tests to capture this bias against
assists and toward GC cycles starting earlier in the face of noise.

Sweet benchmarks didn't show any meaningful difference, but real
production applications showed a reduction in tail latencies of up
to 45%.

Updates #56966.

Change-Id: I8f03d793f9a1c6e7ef3524d18294dbc0d7de6122
Reviewed-on: https://go-review.googlesource.com/c/go/+/467875
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-03-21 19:27:18 +00:00
Austin Clements 9eba17ff90 runtime: for deep stacks, print both the top 50 and bottom 50 frames
This is relatively easy using the new traceback iterator.

Ancestor tracebacks are now limited to 50 frames. We could keep that
at 100, but the fact that it used 100 before seemed arbitrary and
unnecessary.

Fixes #7181
Updates #54466

Change-Id: If693045881d84848f17e568df275a5105b6f1cb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/475960
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-21 19:14:14 +00:00
Tobias Klauser 6be7fd3f9b runtime: let (*sigctxt).r9 return the correct register value on netbsd
Fixes #59152

Change-Id: I7f89e2b48db9ff120d7cd094b72ded865874e961
Reviewed-on: https://go-review.googlesource.com/c/go/+/478135
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2023-03-21 18:51:53 +00:00
WANG Xuerui 6f2ef5ec98 cmd/asm, runtime: remove the RSB register from loong64
It was carryover from the mips64 port (where it represented the platform
GP register) but LoongArch platform ABI doesn't have the GP concept.

Change-Id: Iea326ae13676e95b040b52aaadc08d311b507bd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/475815
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-03-21 06:51:28 +00:00
Achille Roussel dac514ea37 runtime: reuse freed memory blocks on wasm
When compiling Go programs to WebAssembly, the memory allocation
strategy was neither releasing memory to the OS nor reusing blocks freed
by calls to runtime.sysFreeOS.

This CL unifies the plan9 and wasm memory management strategy
since both platforms use a linear memory space and do not have a
mechanism for returning memory blocks to the OS.

Fixes #59061

Change-Id: I282ba93c0fe1a0961a31c0825b2a7e0478b8713d
GitHub-Last-Rev: 1c485be4fb
GitHub-Pull-Request: golang/go#59065
Reviewed-on: https://go-review.googlesource.com/c/go/+/476717
Reviewed-by: Julien Fabre <ju.pryz@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Evan Phoenix <evan@phx.io>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-21 02:45:06 +00:00
Damien Neil b98c1b22bd all: gofmt
Change-Id: I926388ee5aeeff11f765cbd4558b66645d1bbc08
Reviewed-on: https://go-review.googlesource.com/c/go/+/477836
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-20 22:54:22 +00:00
Joel Sing 6827f0d772 runtime: provide and use a GO_PPC64X_HAS_FUNCDESC define
Rather than implying that all ppc64 GOARCHs use function descriptors,
provide a define for platforms that make use of function descriptors.
Condition on GO_PPC64X_HAS_FUNCDESC when choosing whether or not
to load the entry address from the first slot of the function
descriptor.

Updates #56001.

Change-Id: I9cdc788f2de70a1262c17d8485b555383d1374b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/476117
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-18 06:28:23 +00:00
Joel Sing 602e6aa979 runtime: remove arm64 non-register ABI fallback code
This presumably got missed in CL 393875.

Change-Id: I4f2de00ebd6ec405d5e289a7f8c2fc781607260b
Reviewed-on: https://go-review.googlesource.com/c/go/+/475617
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-03-17 09:07:40 +00:00
Guoqi Chen c8402cd330 runtime: remove the fake mstart caller in systemstack on linux/loong64
The backtrace knows to stop in the system stack due to writing to the SP,
so here the fake mstart caller in the system stack is no longer needed and
can be removed

ref. CL 288799

Change-Id: I0841e75fd515cf6a0d98abe4cffc3f63fc275e0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/416035
Auto-Submit: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: WANG Xuerui <git@xen0n.name>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-17 06:57:16 +00:00
Leonard Wang 192814985d runtime/trace: record HeapGoal when StartTrace
Fixes #52517

Change-Id: I06aa6112f14f264360c3bb0ffd4e1cd54ad22514
Reviewed-on: https://go-review.googlesource.com/c/go/+/401777
Reviewed-by: hopehook <hopehook@golangcn.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-03-16 19:44:32 +00:00
Austin Clements 5f62ba621e runtime: fix callee tracking in traceback printing
In CL 466099, we accidentally stopped tracking callees while unwinding
inlined frames during traceback printing. The effect is that if you
have a call stack like:

  f -> wrapper -> inlined into wrapper -> panic

when considering whether to print the frame for "wrapper", we'll think
that wrapper called panic, rather than the inlined function.

Fix this in the traceback code and add a test.

Change-Id: I30ec836cc316846ce93de94e28a650e23dca184e
Reviewed-on: https://go-review.googlesource.com/c/go/+/476579
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-15 23:53:51 +00:00
Michael Pratt 07a25a9f6e runtime: pass M to traceReleaseBuffer
This change is a no-op, but makes the acquire-release pair
traceAcquireBuffer / traceReleaseBuffer more explicit, since the former
does acquirem and the latter releasm.

Change-Id: If8a5b1ba8709bf6f39c8ff27b2d3e0c0b0da0e58
Reviewed-on: https://go-review.googlesource.com/c/go/+/476575
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
2023-03-15 18:32:37 +00:00
Ian Lance Taylor f5eef58e43 syscall: restore original NOFILE rlimit in child process
If we increased the NOFILE rlimit when starting the program,
restore the original rlimit when forking a child process.

For #46279

Change-Id: Ia5d2af9ef435e5932965c15eec2e428d2130d230
Reviewed-on: https://go-review.googlesource.com/c/go/+/476097
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@google.com>
2023-03-15 17:21:30 +00:00
Paul E. Murphy a8cd10f2ab runtime: add arg storage to ppc64le/linux runtime.sigprofNonGoWrapper frame
CL 475935 fixed the the ELFv2 ABI violations, but in the process created a
Go ABI violation by failing to allocate stack space for arguments.

Allocate this space while keeping the frame 16 byte aligned.

Updates #58953

Change-Id: I9942d9a433118b391ef8cd7bcea5808695cf94d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/476296
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Archana Ravindar <aravind5@in.ibm.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-15 13:30:09 +00:00
WANG Xuerui 7789966286 cmd/compile/internal, runtime: use NOOP for hardware NOPs on loong64
The canonical LoongArch NOP instruction form is "andi r0, r0, 0", as
described in the LoongArch Reference Manual Volume 1, Section 2.2.1.10.
We currently use NOR instead, which may or may not change anything (e.g.
performance on less capable micro-architectures) but is deviation from
upstream standards nevertheless. Fix them to use the explicit hardware
NOP which happens to be supported as `NOOP`.

Change-Id: I0a799a1da959e9c3b582feb88202df2bab0ab23a
Reviewed-on: https://go-review.googlesource.com/c/go/+/475615
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-15 07:18:43 +00:00
Guoqi Chen b5c8ae9816 runtime: mark morestack_noctxt SPWRITE for linux/loong64
ref. CL 425396

Updates #54332.

Change-Id: I1a235b0cca4dbf79cf61cf5f40b594fc2d940857
Reviewed-on: https://go-review.googlesource.com/c/go/+/446895
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: WANG Xuerui <git@xen0n.name>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: xiaodong liu <teaofmoli@gmail.com>
2023-03-15 07:17:43 +00:00
Guoqi Chen 574c620901 runtime: refactor the linux/loong64 entrypoint
Remove the meaningless jump, and add the missing NOFRAME flag to _rt0_loong64_linux.

Change-Id: I1aec68c556615b42042684bd176dfc2a8af094d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/420977
Run-TryBot: M Zhuo <mzh@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: xiaodong liu <teaofmoli@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: WANG Xuerui <git@xen0n.name>
2023-03-15 07:15:46 +00:00
Paul E. Murphy 46bc9a6e1e runtime,runtime/cgo: fix tsan/pprof/cgo issues
The tsan13 test highlighted a few bugs.

The first being runtime.sigprofNonGoWrapper was being
called from C code and violating the C ABI.

The second was a missed tsan acquire/release after
thread creation.

The third was runtime.cgoSigtramp violating ELFv2
ABI constraints when loading g. It is reworked to
avoid clobbering R30 and R31 via runtime.load_g.

Change-Id: Ib2d98047fa1b4e72b8045767e86457a8ddfe492e
Reviewed-on: https://go-review.googlesource.com/c/go/+/475935
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Archana Ravindar <aravind5@in.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-14 16:42:25 +00:00
Wayne Zuo dd009b2f1a runtime/internal/sys: remove 386 assembly code
Change-Id: Ia4f8638d626c30b91729caa0c58a96451c627f67
Reviewed-on: https://go-review.googlesource.com/c/go/+/475736
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-14 08:10:45 +00:00
Tianon Gravi 7ec69abf05 runtime/debug: replace "GOARM64" with "GOARM" in docs
The former does not exist (yet) but the latter very much exists and is in active use.

Change-Id: Ic8812b98a10f01061a211a0de4caa9e9cfff36fa
GitHub-Last-Rev: 0dab9f524c
GitHub-Pull-Request: golang/go#59014
Reviewed-on: https://go-review.googlesource.com/c/go/+/476016
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-14 04:30:59 +00:00
cui fliter 50ba90bce5 all: fix some problematic comments
Change-Id: Ia110d19fe5ff3adc8bbf86dd2112f9702164d495
Reviewed-on: https://go-review.googlesource.com/c/go/+/475515
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-03-13 16:49:45 +00:00
qmuntal 43f911b0b6 runtime: remove NOFRAME from asmcgocall, systemstack and mcall
This CL removes the NOFRAME flag from runtime.asmcgocall,
runtime.systemstack and runtime.mcall so the compiler can place
the frame pointer on the stack.

This will help unwinding cgo stack frames, and might be all what's
needed for tools that only use the frame pointer to unwind the stack.
That's not the case for gdb, which uses DWARF CFI, and windbg,
which uses SEH. Yet, having the frame pointer correctly set lays
the foundation for supporting cgo unwinding with DWARF CFI and SEH.

Updates #58378

Change-Id: I7655363b3fb619acccd9d5a7f0e3d3dec953cd52
Reviewed-on: https://go-review.googlesource.com/c/go/+/472195
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-13 15:28:25 +00:00
qmuntal b6f29d2365 runtime: implement high resolution timer on windows arm/arm64
This CL moves the usleep2HighRes from assembly to good old Go.
This is safe because since CL 288793 usleep is always called with
a g, else one wold have to call usleep_no_g. This condition was
not enforced when high resolution timers were first implemented
on Windows (CL 248699), so the implementation was done in assembly.

Other than removing a bunch of obscure assembly code, this CL makes
high resolution timers work on windows arm/arm64 by free, as the
system calls are the same in all windows platforms.

Change-Id: I41ecf78026fd7e11e85258a411ae074a77e8c7fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/471142
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
2023-03-13 14:20:41 +00:00
Bryan C. Mills d12fe60004 all: skip tests that fail on android/arm64
Many of the tests skipped platforms that build PIE binaries by
default, but (still) lack a central function to report which platforms
those are.

Some of the tests assumed (but did not check for) internal linking
support, or invoked `go tool link` directly without properly
configuring the external linker.

A few of the tests seem to be triggering latent bugs in the linker.

For #58806.
For #58807.
For #58794.

Change-Id: Ie4d06b1597f404590ad2abf978d4c363647407ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/472455
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 20:38:29 +00:00
Austin Clements 3790ceca97 runtime: delete gentraceback
Printing is the only remaining functionality of gentraceback. Move
this into the traceback printing code and eliminate gentraceback. This
lets us simplify the logic, which fixes at least one minor bug:
previously, if inline unwinding pushed the total printed count over
_TracebackMaxFrames, we would print extra frames and then fail to
print "additional frames elided".

The cumulative performance effect of the series of changes starting
with "add a benchmark of Callers" (CL 472956) is:

goos: linux
goarch: amd64
pkg: runtime
cpu: Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
                       │  baseline   │              unwinder               │
                       │   sec/op    │   sec/op     vs base                │
Callers/cached-48        1.464µ ± 1%   1.684µ ± 1%  +15.03% (p=0.000 n=20)
Callers/inlined-48       1.391µ ± 1%   1.536µ ± 1%  +10.42% (p=0.000 n=20)
Callers/no-cache-48      10.50µ ± 1%   11.11µ ± 0%   +5.82% (p=0.000 n=20)
StackCopyPtr-48          88.74m ± 1%   81.22m ± 2%   -8.48% (p=0.000 n=20)
StackCopy-48             80.90m ± 1%   70.56m ± 1%  -12.78% (p=0.000 n=20)
StackCopyNoCache-48      2.458m ± 1%   2.209m ± 1%  -10.15% (p=0.000 n=20)
StackCopyWithStkobj-48   26.81m ± 1%   25.66m ± 1%   -4.28% (p=0.000 n=20)
geomean                  518.8µ        512.9µ        -1.14%

The performance impact of intermediate CLs in this sequence varies a
lot as we went through many refactorings. The slowdown in Callers
comes primarily from the introduction of unwinder because that doesn't
get inlined and results in somewhat worse code generation in code
that's extremely hot in those microbenchmarks. The performance gains
on stack copying come mostly from replacing callbacks with direct use
of the unwinder.

Updates #54466.
Fixes #32383.

Change-Id: I4970603b2861633eecec30545e852688bc7cc9a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/468301
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 17:59:39 +00:00
Austin Clements 3e360b035f runtime: new API for filling PC traceback buffers
Currently, filling PC traceback buffers is one of the jobs of
gentraceback. This moves it into a new function, tracebackPCs, with a
simple API built around unwinder, and changes all callers to use this
new API.

Updates #54466.

Change-Id: Id2038bded81bf533a5a4e71178a7c014904d938c
Reviewed-on: https://go-review.googlesource.com/c/go/+/468300
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
2023-03-10 17:59:37 +00:00
Austin Clements 9872428a71 runtime: move cgo traceback into unwinder
Currently, gentraceback's loop ends with a call to tracebackCgoContext
to process cgo frames. This requires spreading various parts of the
printing and pcbuf logic across these two functions.

Clean this up by moving cgo unwinding into unwinder and then lifting
the printing and pcbuf logic from tracebackCgoContext into
gentraceback along with the other printing and pcbuf logic.

Updates #54466.

Change-Id: Ic71afaa5ae110c0ea5be9409e267e4284e36a8c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/468299
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 17:59:36 +00:00
Austin Clements c9b2da3f28 runtime: simplify traceback PC back-up logic
Updates #54466.

Change-Id: If070cf3f484e3e02b8e586bff466e0018b1a1845
Reviewed-on: https://go-review.googlesource.com/c/go/+/468298
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 17:59:34 +00:00
Austin Clements 2d99109cfc runtime: replace all callback uses of gentraceback with unwinder
This is a really nice simplification for all of these call sites.

It also achieves a nice performance improvement for stack copying:

goos: linux
goarch: amd64
pkg: runtime
cpu: Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
                       │   before    │                after                │
                       │   sec/op    │   sec/op     vs base                │
StackCopyPtr-48          89.25m ± 1%   79.78m ± 1%  -10.62% (p=0.000 n=20)
StackCopy-48             83.48m ± 2%   71.88m ± 1%  -13.90% (p=0.000 n=20)
StackCopyNoCache-48      2.504m ± 2%   2.195m ± 1%  -12.32% (p=0.000 n=20)
StackCopyWithStkobj-48   21.66m ± 1%   21.02m ± 2%   -2.95% (p=0.000 n=20)
geomean                  25.21m        22.68m       -10.04%

Updates #54466.

Change-Id: I31715b7b6efd65726940041d3052bb1c0a1186f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/468297
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-10 17:59:32 +00:00
Austin Clements da384766a0 runtime: make unsafe.Slice usable from nowritebarrierrec
Many compiler-generated panics are dynamically changed to a "throw"
when they happen in the runtime. One effect of this is that they are
allowed in nowritebarrierrec contexts. Currently, the unsafe.Slice
panics don't have this treatment.

We're about to expose more code that uses unsafe.Slice to the write
barrier checker (it's actually already there and it just can't see
through an indirect call), so give these panics the dynamic check.

Very indirectly updates #54466.

Change-Id: I65cb96fa17eb751041e4fa25a1c1bd03246c82ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/468296
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-10 17:59:31 +00:00
Austin Clements f758d648b0 runtime: implement traceback iterator
Currently, all stack walking logic is in one venerable, large, and
very, very complicated function: runtime.gentraceback. This function
has three distinct operating modes: printing, populating a PC buffer,
or invoking a callback. And it has three different modes of unwinding:
physical Go frames, inlined Go frames, and cgo frames. It also has
several flags. All of this logic is very interwoven.

This CL reimplements the monolithic gentraceback function as an
"unwinder" type with an iterator API. It moves all of the logic for
stack walking into this new type, and gentraceback is now a
much-simplified wrapper around the new unwinder type that still
implements printing, populating a PC buffer, and invoking a callback.
Follow-up CLs will replace uses of gentraceback with direct uses of
unwinder.

Exposing traceback functionality as an iterator API will enable a lot
of follow-up work such as simplifying the open-coded defer
implementation (which should in turn help with #26813 and #37233),
printing the bottom of deep stacks (#7181), and eliminating the small
limit on CPU stacks in profiles (#56029).

Fixes #54466.

Change-Id: I36e046dc423c9429c4f286d47162af61aff49a0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/458218
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
2023-03-10 17:59:29 +00:00
Austin Clements 86b69ef329 runtime: replace cgoCtxt slice with index in traceback
Currently, gentraceback consumes the gp.cgoCtxt slice by copying the
slice header and then sub-slicing it as it unwinds. The code for this
is nice and clear, but we're about to lift this state into a structure
and mutating it is going to introduce write barriers that are
disallowed in gentraceback.

This CL replaces the mutable slice header with an index into
gp.cgoCtxt.

For #54466.

Change-Id: I6b701bb67d657290a784baaca34ed02d8247ede2
Reviewed-on: https://go-review.googlesource.com/c/go/+/466863
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 17:59:27 +00:00
Austin Clements ec319d6d43 runtime: skip TestTracebackInlined if inlining is disabled
This should fix the noopt builders.

Change-Id: I49aa374f4d372803599cd2d2a7a29833b379ce1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/475376
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Austin Clements <austin@google.com>
2023-03-10 17:46:38 +00:00
Austin Clements 166e5ee4f2 runtime: use inlineUnwinder
This converts all places in the runtime that perform inline expansion
to use the new inlineUnwinder abstraction.

For #54466.

Change-Id: I48d996fb6263ed5225bd21d30914a27ae434528d
Reviewed-on: https://go-review.googlesource.com/c/go/+/466099
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-10 17:18:34 +00:00
Austin Clements 59d0de16e4 runtime: add tests of printing inlined frames in tracebacks
We're about to rewrite this code and it has almost no test coverage
right now.

This test is also more complete than the existing
TestTracebackInlineExcluded, so we delete that test.

For #54466.

Change-Id: I144154282dac5eb3798f7d332b806f44c4a0bdf6
Reviewed-on: https://go-review.googlesource.com/c/go/+/466098
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-10 17:18:31 +00:00
Austin Clements d829b62681 runtime: use srcFunc for showframe
Since srcFunc can represent information for either an real text
function or an inlined function, this means we no longer have to
synthesize a fake _func just to call showframe on an inlined frame.

This is cleaner and also eliminates the one case where _func values
live in the heap. This will let us mark them NotInHeap, which will in
turn eliminate pesky write barriers in the traceback rewrite.

For #54466.

Change-Id: Ibf5e24d01ee4bf384c825e1a4e2922ef444a438e
Reviewed-on: https://go-review.googlesource.com/c/go/+/466097
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 17:18:29 +00:00
Austin Clements f52bede354 runtime: create an API for unwinding inlined frames
We've replicated the code to expand inlined frames in many places in
the runtime at this point. This CL adds a simple iterator API that
abstracts this out.

We also use this to try out a new idea for structuring tests of
runtime internals: rather than exporting this whole internal data type
and API, we write the test in package runtime and import the few bits
of std we need. The idea is that, for tests of internals, it's easier
to inject public APIs from std than it is to export non-public APIs
from runtime. This is discussed more in #55108.

For #54466.

Change-Id: Iebccc04ff59a1509694a8ac0e0d3984e49121339
Reviewed-on: https://go-review.googlesource.com/c/go/+/466096
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Austin Clements <austin@google.com>
2023-03-10 17:18:27 +00:00
Austin Clements dcb4c1c1aa runtime: dedup function name logic into moduledata method
For #54466.

Change-Id: I4d8e1953703b6c763e5bd53024da43efcc993489
Reviewed-on: https://go-review.googlesource.com/c/go/+/466095
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Austin Clements <austin@google.com>
2023-03-10 17:18:26 +00:00
Austin Clements 6f22d42c74 runtime: resolve caller funcInfo after processing current frame
Currently, gentraceback resolves the funcInfo of the caller prior to
processing the current frame (calling the callback, printing it, etc).
As a result, if this lookup fails in a verbose context, it will print
the failure before printing the frame that it's already resolved.

To fix this, move the resolution of LR to a funcInfo to after current
frame processing.

This also has the advantage that we can reduce the scope of "flr" (the
caller's funcInfo) to only the post-frame part of the loop, which will
make it easier to stack-rip gentraceback into an iterator.

For #54466.

Change-Id: I8be44d4eac598a686c32936ab37018b8aa97c00b
Reviewed-on: https://go-review.googlesource.com/c/go/+/458217
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
2023-03-10 17:18:25 +00:00
Austin Clements 5acd2d658e runtime: eliminate waspanic from gentraceback
gentraceback also tracks the funcID of the callee, which is more
general. Fix this up to happen in all cases and eliminate waspanic in
favor of checking the funcID of the caller.

For #54466.

Change-Id: Idc98365a6f05022db18ddcd5b3ed8684a6872a88
Reviewed-on: https://go-review.googlesource.com/c/go/+/458216
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 17:18:23 +00:00
Austin Clements 4f132b7ef8 runtime: don't track stack separately in gentraceback
Currently, gentraceback keeps a copy of the stack bounds of the stack
it's walking in the "stack" variable. Now that "gp" always refers to
the G whose stack it's walking, we can simply use gp.stack instead of
keeping a separate copy.

For #54466.

Change-Id: I68256e5dff6212cfcf14eda615487e66a92d4914
Reviewed-on: https://go-review.googlesource.com/c/go/+/458215
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-10 17:18:22 +00:00
Austin Clements 2d93805171 runtime: add a benchmark of Callers
We're about to make major changes to tracebacks. We have benchmarks of
stack copying, but not of PC buffer filling, so add some that we can
track through these changes.

For #54466.

Change-Id: I3ed61d75144ba03b61517cd9834eeb71c99d74df
Reviewed-on: https://go-review.googlesource.com/c/go/+/472956
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-10 17:18:20 +00:00
Guoqi Chen f92a69f907 runtime: implement cputicks with the stable counter on loong64
The stable counter is described in Section 2.2.10.4, LoongArch Reference Manual Volume 1.

Ref: https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html

Change-Id: I160b695a8c0e38ef49b21fb8b41460fd23d9538c
Reviewed-on: https://go-review.googlesource.com/c/go/+/421656
Reviewed-by: Meidan Li <limeidan@loongson.cn>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: WANG Xuerui <git@xen0n.name>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-03-10 12:47:42 +00:00
Keith Randall 642542cb3c Revert "cmd/link: establish dependable package initialization order"
This reverts commit ce2a609909.
aka CL 462035

Reason for revert: this CL is causing some problems in some internal Google programs.

Change-Id: I4476b8d8d2c3d7b5703d1d85c93baebb4b4e5d26
Reviewed-on: https://go-review.googlesource.com/c/go/+/474976
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-03-09 19:19:41 +00:00
Nick Ripley 4460291e0e runtime/trace: update outdated Task and Region documentation
A previous iteration of the tracer's user annotation API had different
names for tasks and regions, and used to return functions for ending
them rather than types with End methods. This CL updates the doc
comments to reflect those changes, and also fixes up the internal
documentation of the events (similar to go.dev/cl/465335, the stack
argument was in the wrong place in the list).

The User Log event internal documentation might also look wrong since
the value argument follows the stack argument. However, the User Log
event is a special case where the log message is appended immediately
following the normal event, including the stack argument. There isn't
much room to clarify this next to the event type definitions, so this CL
clarifies the comment where the event is encoded.

Change-Id: I846c709f6026ef01c0a272557d6390b2c17074e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/472955
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Nick Ripley <nick.ripley@datadoghq.com>
2023-03-09 19:10:52 +00:00
Ian Lance Taylor 618fb4ab06 runtime/cgo: add tsan sync for traceback function
Change-Id: Ifb8d64f18b67c8b712feec29ffb6719c6e9718ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/474198
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-03-08 20:11:59 +00:00
Revolution 600b982df9 runtime: fix comment mismatch for currentConsMark
Change-Id: Ie0ed83e17be180100f144ce61bbd2c72a64d857b
GitHub-Last-Rev: 9db7a90a95
GitHub-Pull-Request: golang/go#58910
Reviewed-on: https://go-review.googlesource.com/c/go/+/473820
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-03-07 22:05:44 +00:00
Alex Brainman f6cbc1da05 runtime: allow for 5 more threads in TestWindowsStackMemory*
Original version of TestWindowsStackMemory did not consider sysmon and
other threads running during the test. Allow for 5 extra threads in this
test - this should cover any new threads in the future.

Fixes #58570

Change-Id: I215790f9b94ff40a32ddd7aa54af715d1dc391c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/473415
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-07 21:32:17 +00:00
David Chase c20d959163 cmd/compile: experimental loop iterator capture semantics change
Adds:
GOEXPERIMENT=loopvar (expected way of invoking)
-d=loopvar={-1,0,1,2,11,12} (for per-package control and/or logging)
-d=loopvarhash=... (for hash debugging)

loopvar=11,12 are for testing, benchmarking, and debugging.

If enabled,for loops of the form `for x,y := range thing`, if x and/or
y are addressed or captured by a closure, are transformed by renaming
x/y to a temporary and prepending an assignment to the body of the
loop x := tmp_x.  This changes the loop semantics by making each
iteration's instance of x be distinct from the others (currently they
are all aliased, and when this matters, it is almost always a bug).

3-range with captured iteration variables are also transformed,
though it is a more complex transformation.

"Optimized" to do a simpler transformation for
3-clause for where the increment is empty.

(Prior optimization of address-taking under Return disabled, because
it was incorrect; returns can have loops for children.  Restored in
a later CL.)

Includes support for -d=loopvarhash=<binary string> intended for use
with hash search and GOCOMPILEDEBUG=loopvarhash=<binary string>
(use `gossahash -e loopvarhash command-that-fails`).

Minor feature upgrades to hash-triggered features; clients can specify
that file-position hashes use only the most-inline position, and/or that
they use only the basenames of source files (not the full directory path).
Most-inlined is the right choice for debugging loop-iteration change
once the semantics are linked to the package across inlining; basename-only
makes it tractable to write tests (which, otherwise, depend on the full
pathname of the source file and thus vary).

Updates #57969.

Change-Id: I180a51a3f8d4173f6210c861f10de23de8a1b1db
Reviewed-on: https://go-review.googlesource.com/c/go/+/411904
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-06 18:34:24 +00:00
Keith Randall ce2a609909 cmd/link: establish dependable package initialization order
As described here:

https://github.com/golang/go/issues/31636#issuecomment-493271830

"Find the lexically earliest package that is not initialized yet,
but has had all its dependencies initialized, initialize that package,
 and repeat."

Simplify the runtime a bit, by just computing the ordering required
in the linker and giving a list to the runtime.

Update #31636
Fixes #57411

RELNOTE=yes

Change-Id: I1e4d3878ebe6e8953527aedb730824971d722cac
Reviewed-on: https://go-review.googlesource.com/c/go/+/462035
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-03 23:11:37 +00:00