Commit Graph

5414 Commits

Author SHA1 Message Date
eric fang eee6f9f825 runtime: unify C->Go ABI transitions on arm64
There are several of places that save and restore the C callee-saved registers,
the operation is the same everywhere, so this CL defines several macros
to do this, which will help reduce code redundancy and unify the operation.

This CL also replaced consecutive MOVD instructions with STP and LDP instructions
in several places where these macros do not apply.

Change-Id: I815f39fe484a9ab9b6bd157dfcbc8ad99c1420fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/374397
Trust: Eric Fang <eric.fang@arm.com>
Run-TryBot: Eric Fang <eric.fang@arm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-30 01:28:43 +00:00
Meng Zhuo 8fefeabb35 runtime: add runtime changes for register ABI on riscv64
This CL adds
- spill functions used by runtime
- ABIInternal to functions

Adding new stubs_riscv64 file to eliminate vet issues while compiling.

Change-Id: I2a9f6088a1cd2d9708f26b2d97895b4e5f9f87e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/360296
Trust: mzh <mzh@golangcn.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-30 01:12:57 +00:00
Paul E. Murphy dac0220841 runtime: avoid potential hang in TestGdbAutotmpTypes
If a GC related task preempts between hitting the main.main breakpoint
and stepping, the test program may halt forever waiting on a GC
operation. This happens if gdb is configured to halt other threads
while executing a step.

Configure gdb to continue running all threads during a step by
setting the scheduler-locking option to off.

Fixes #49852

Change-Id: Iacc9732cbd23526bde0a295e6fa8a0d90f733f59
Reviewed-on: https://go-review.googlesource.com/c/go/+/370775
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Cherry Mui <cherryyz@google.com>
2022-03-29 22:06:31 +00:00
Ian Lance Taylor a2baae6851 all: use new "unix" build tag where appropriate
For #20322
For #51572

Change-Id: Id0b4799d097d01128e98ba4cc0092298357bca45
Reviewed-on: https://go-review.googlesource.com/c/go/+/389935
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2022-03-29 16:24:51 +00:00
Cherry Mui 0a69c98214 all: delete PPC64 non-register ABI fallback path
Change-Id: Ie058c0549167b256ad943a0134907df3aca4a69f
Reviewed-on: https://go-review.googlesource.com/c/go/+/394215
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-28 18:20:56 +00:00
Dmitry Vyukov d6a1ffd624 runtime/race: update runtime (v3)
New tsan runtime (v3) built on llvm commit 1784fe0532a6.

The new runtime features:
 - 2x smaller shadow memory (2x of app memory)
 - faster fully vectorized (on x86) race detection
 - small fixed-size vector clocks (512b)
 - fast vectorized vector clock operations
 - unlimited number of alive threads/goroutines

Some random subset of benchmarks:

encoding/json:
CodeEncoder-72           20.3ms ± 6%     11.7ms ± 4%   -42.02%  (p=0.000 n=10+8)
CodeMarshal-72           22.3ms ±11%     12.7ms ±11%   -43.28%  (p=0.000 n=10+10)
CodeDecoder-72           46.1ms ±42%     20.2ms ± 7%   -56.18%  (p=0.000 n=10+10)
CodeUnmarshal-72         16.7ms ±14%     13.2ms ± 9%   -20.93%  (p=0.000 n=10+10)
CodeUnmarshalReuse-72    17.7ms ±17%     12.8ms ± 8%   -27.58%  (p=0.000 n=10+10)

net/http:
ClientServerParallel4-72         914µs ±18%      72µs ± 5%  -92.16%  (p=0.000 n=20+18)
ClientServerParallel64-72       1.77ms ±16%    0.12ms ±29%  -93.43%  (p=0.000 n=20+17)
ClientServerParallelTLS4-72     1.99ms ±33%    0.20ms ± 5%  -89.83%  (p=0.000 n=19+17)
ClientServerParallelTLS64-72    2.72ms ±26%    0.25ms ±16%  -90.96%  (p=0.000 n=20+16)

compress/flate:
Decode/Digits/Huffman/1e4-72        2.53ms ± 3%     1.47ms ± 4%   -41.72%  (p=0.000 n=9+10)
Decode/Digits/Speed/1e4-72          2.59ms ± 5%     1.52ms ± 3%   -41.44%  (p=0.000 n=8+9)
Decode/Digits/Default/1e4-72        2.56ms ± 6%     1.51ms ± 4%   -40.96%  (p=0.000 n=10+10)
Decode/Digits/Compression/1e4-72    2.54ms ± 6%     1.52ms ± 2%   -40.14%  (p=0.000 n=10+9)
Decode/Newton/Huffman/1e4-72        2.65ms ± 3%     1.58ms ± 4%   -40.45%  (p=0.000 n=10+10)
Decode/Newton/Speed/1e4-72          2.16ms ± 9%     1.28ms ± 5%   -40.59%  (p=0.000 n=10+10)
Decode/Newton/Default/1e4-72        2.01ms ± 8%     1.16ms ± 6%   -42.11%  (p=0.000 n=10+10)
Decode/Newton/Compression/1e4-72    1.99ms ± 6%     1.17ms ± 3%   -41.05%  (p=0.000 n=9+10)
Encode/Digits/Huffman/1e4-72        2.05ms ± 2%     0.75ms ± 5%   -63.32%  (p=0.000 n=10+10)
Encode/Digits/Speed/1e4-72          2.89ms ± 2%     1.37ms ± 2%   -52.56%  (p=0.000 n=9+10)
Encode/Digits/Default/1e4-72        7.55ms ± 2%     3.86ms ± 4%   -48.93%  (p=0.000 n=8+10)
Encode/Digits/Compression/1e4-72    7.46ms ± 4%     3.88ms ± 4%   -48.07%  (p=0.000 n=9+9)
Encode/Newton/Huffman/1e4-72        2.20ms ± 4%     0.90ms ± 6%   -59.01%  (p=0.000 n=10+10)
Encode/Newton/Speed/1e4-72          2.62ms ± 2%     1.30ms ± 4%   -50.52%  (p=0.000 n=9+10)
Encode/Newton/Default/1e4-72        7.40ms ± 5%     3.72ms ± 2%   -49.65%  (p=0.000 n=10+9)
Encode/Newton/Compression/1e4-72    7.67ms ± 9%     3.85ms ± 4%   -49.87%  (p=0.000 n=10+10)

encoding/json short tests:
Time     2.34sec ± 6%   1.67sec ±11%  -28.85%  (p=0.000 n=10+10)
Memory     266MB ± 1%     190MB ± 9%  -28.78%  (p=0.000 n=10+10)

compress/flate short tests:
Time      4.69sec ± 4%   2.78sec ± 3%  -40.61%  (p=0.000 n=10+10)
Memory      284MB ± 5%     181MB ± 4%  -36.04%  (p=0.000 n=10+9)

sync short tests:
Time     4.87sec ± 4%  0.87sec ± 6%  -82.21%  (p=0.000 n=10+10)
Memory     147MB ± 3%     99MB ± 8%  -32.47%  (p=0.000 n=10+9)

Fixes #47056
Fixes #38184

Change-Id: I0cf228f2e4cac7778d34d33e46df7c081645f5d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/333529
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Trust: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-28 16:52:35 +00:00
mzh ad646b33c9 Revert "reflect, runtime: add reflect support for regabi on riscv64"
This reverts commit 56400fc706.

Reason for revert: this CL requires CL360296 be merged

Change-Id: I4c48c4d23b73b6e892cf86cbbc864698ebc5c992
Reviewed-on: https://go-review.googlesource.com/c/go/+/396076
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: mzh <mzh@golangcn.org>
Run-TryBot: mzh <mzh@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-28 01:10:35 +00:00
Meng Zhuo 56400fc706 reflect, runtime: add reflect support for regabi on riscv64
This CL adds regabi support needed for reflect.

Change-Id: Ib78f8c7765f03e3a7b46e8b115bf8870b8076e6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/360994
Trust: mzh <mzh@golangcn.org>
Run-TryBot: mzh <mzh@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-27 12:55:16 +00:00
Romanos Skiadas b2643c6739 runtime: update framepointer_enabled doc
Change-Id: I69e64ebf8c11145ce32aa4c11178e3a47d22fb84
Reviewed-on: https://go-review.googlesource.com/c/go/+/394915
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
2022-03-24 15:51:28 +00:00
Rhys Hiltner 212bda0669 runtime/pprof: rerun magnitude test on failure
Restructure TestCPUProfileMultithreadMagnitude so it will run again with
a longer duration on failure. Log the split between the user vs system
CPU time that rusage reports.

For #50232

Change-Id: Ice5b38ee7594dbee1eaa5686d32b968c306e3e85
Reviewed-on: https://go-review.googlesource.com/c/go/+/393934
Run-TryBot: Rhys Hiltner <rhys@justin.tv>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
2022-03-22 21:31:40 +00:00
Paul E. Murphy 946167906e runtime: make static/dynamic startup detection work with musl on ppc64le
The glibc loader explicitly sets the first doubleword on the stack (R1)
to $0 to indicate it was dynamically loaded.

An ELFv2 ABI compliant loader will set R3/R4 to argc/argv when starting
the process, and R13 to TLS. musl is not compliant. Instead it passes
argc/argv like the kernel, but R3/R4 are in an undefined state and R13
is valid.

With the knowledge above, the startup code can be modified to
dynamically handle all three cases when linked internally.

Fixes #51787

Change-Id: I5de33862c161900d9161817388bbc13a65fdc69c
Reviewed-on: https://go-review.googlesource.com/c/go/+/394654
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Trust: Paul Murphy <murp@ibm.com>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2022-03-22 20:56:39 +00:00
eric fang 817d6ea2b3 runtime: delete useless TPIDR macro on arm64
The TPIDR macro in tls_arm64.h is not used anywhere, so remove it
to reduce confusion.

Change-Id: I04aa5e64ee30753f28f43bc67b44559d81d093c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/373357
Trust: Eric Fang <eric.fang@arm.com>
Run-TryBot: Eric Fang <eric.fang@arm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2022-03-22 20:46:22 +00:00
Jakub Ciolek 86c8075675 runtime: combine wbuf checks in tryGetFast and putFast
Less text and improves codegen a bit.

compilecmp on ARM64:

runtime
(*gcWork).putFast 160 -> 144  (-10.00%)
(*gcWork).tryGetFast 144 -> 128  (-11.11%)
scanobject 784 -> 752  (-4.08%)
greyobject 800 -> 784  (-2.00%)

AMD64:

runtime
greyobject 765 -> 748  (-2.22%)
(*gcWork).tryGetFast 102 -> 85  (-16.67%)
scanobject 837 -> 820  (-2.03%)
(*gcWork).putFast 102 -> 89  (-12.75%)

Change-Id: I6bb508afe1ba416823775c0bfc08ea9dc21de8a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/393754
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
2022-03-21 22:42:59 +00:00
Bryan C. Mills 58631ba54f internal/testenv: add GOROOT and use it to fix tests broken with -trimpath
This fixes many (but not all) of the tests that currently fail
(due to a bogus path reported by runtime.GOROOT) when run with
'go test -trimpath std cmd'.

Updates #51461

Change-Id: Ia2cc05705529c4859e7928f32eeceed647f2e986
Reviewed-on: https://go-review.googlesource.com/c/go/+/391806
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-18 21:55:52 +00:00
Cherry Mui 9f252a0462 all: delete ARM64 non-register ABI fallback path
Change-Id: I3996fb31789a1f8559348e059cf371774e548a8d
Reviewed-on: https://go-review.googlesource.com/c/go/+/393875
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-03-18 18:26:13 +00:00
Cherry Mui d8bee94be2 reflect, runtime: drop RegabiArgs conditions
With the previous CL, internal/abi.IntArgRegs and FloatArgRegs
is controlled by RegabiArgs (or always enabled), so there is no
need to check for that goexperiment.

There are a few places we guard register-ABI specific code and
tests with the RegabiArgs flag. Switch to checking for the number
of argument registers instead.

Change-Id: I79fff9fd1e919684ffaf73aba9e7e85d5a9e1629
Reviewed-on: https://go-review.googlesource.com/c/go/+/393363
Trust: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-18 15:17:37 +00:00
Cherry Mui 3684abbf6c all: delete regabireflect goexperiment
regabireflect goexperiment was helpful in the register ABI
development, to control code paths for reflect calls, before the
compiler can generate register ABI everywhere. It is not necessary
for now. Drop it.

Change-Id: I2731197d2f496e29616c426a01045c9b685946a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/393362
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-03-18 14:45:56 +00:00
Nuno Cruces f02108636c runtime: allow TestCtrlHandler to run in ConPTY
Fixes #51602. Previous test would not run in a pseudo-console (ConPTY).

New test avoids taskkill entirely by having the child request its own
console window be closed.

Verified that this runs locally (within a real console), over SSH
(within a pseudo-console), and that it breaks if #41884 were reverted.

Change-Id: If868b92ec36647e5d0e4107e29a2a6e048d35ced
GitHub-Last-Rev: b1421e4bed
GitHub-Pull-Request: golang/go#51681
Reviewed-on: https://go-review.googlesource.com/c/go/+/392874
Reviewed-by: Bryan Mills <bcmills@google.com>
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
2022-03-18 07:37:53 +00:00
Tobias Klauser adfee1e1e7 runtime/cgo: remove memset in _cgo_sys_thread_start on freebsd/arm
pthread_attr_init on freebsd properly initializes the pthread_attr,
there is no need to zero it before the call. The comment and code were
probably copied from the linux/arm implementation.

This aligns the implementation on freebsd/arm with the implementation on
other freebsd architectures.

Fixes #44248

Change-Id: If82ebb115b877b6c6f4862018a9419ba8d870f12
Reviewed-on: https://go-review.googlesource.com/c/go/+/393617
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Hajime Hoshi <hajimehoshi@gmail.com>
Trust: Hajime Hoshi <hajimehoshi@gmail.com>
2022-03-18 06:03:19 +00:00
Ian Lance Taylor 2d32594396 runtime: call testenv.MustHaveCGO in a couple of tests
Fixes #51695

Change-Id: Icfe9d26ecc28a7db9040d50d4661cf9e8245471e
Reviewed-on: https://go-review.googlesource.com/c/go/+/392916
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-03-18 00:49:08 +00:00
Bryan C. Mills c6244b5909 runtime/debug: do not require a GOROOT/src prefix in TestStack
When paths are trimmed, the reported file locations begin with the
package import path (not GOROOT/src).

Updates #51461.

Change-Id: Ia6814f970aee11f3d933e75c75136d679d19e220
Reviewed-on: https://go-review.googlesource.com/c/go/+/391815
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-17 19:24:19 +00:00
Bryan C. Mills 1a2f726195 runtime/pprof: do not require a GOROOT/src prefix in tests
When paths are trimmed, the reported file locations begin with the
package import path (not GOROOT/src).

Updates #51461

Change-Id: Idbd408a02e8d03329d10e30b0b08263e69e66285
Reviewed-on: https://go-review.googlesource.com/c/go/+/391812
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-16 15:20:22 +00:00
Russ Cox 1178255f85 all: untab /* */ doc comments
A long time ago, gofmt insisted on inserting tabs in /* */ comments
at the top level of the file, like this:

	/*
		Package doc comment.
	*/
	package p

Gofmt still insists on the tab for comments not at top level,
but it has relaxed the rules about top-level comments.
A few very old doc comments are indented, left over from the old rule.

We are considering formatting doc comments, and so to make
everything consistent, standardize on unindented doc comments
by removing tabs in the few doc comments that are still indented this way.

Also update some cmd/gofmt testdata to match.

Change-Id: I293742e39b52f8a48ec41f72ca4acdafa7ce43bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/384261
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-15 17:17:30 +00:00
Tobias Klauser 3c2e73c8c3 runtime/pprof: use syscall.RUSAGE_SELF
Change-Id: Idc37429de5a48e708eda868ca7fa26b28620bac0
Reviewed-on: https://go-review.googlesource.com/c/go/+/391854
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-13 21:31:06 +00:00
Ian Lance Taylor 842d37ee5f syscall: add race annotations to Pread and Pwrite
Fixes #51618

Change-Id: Ife894d8c313dce8c4929f40fa0ac90a069f77a89
Reviewed-on: https://go-review.googlesource.com/c/go/+/391954
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2022-03-12 23:32:36 +00:00
Tobias Klauser c1f22134f2 runtime/pprof, syscall: report MaxRSS on all unix platforms
All unix platforms currently supported by Go provide the getrusage
syscall. On aix and solaris the Getrusage syscall wrapper is not
available yet, so add and use it to report MaxRSS in memory profiles.

Change-Id: Ie880a3058171031fd2e12ccf9adfb85ce18858b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/391434
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-11 06:03:26 +00:00
Michael Pratt 1cf67709be runtime: fix SegvInCgo skip check
CL 390034 changed this throw message to add the goid, breaking the
match.

For #50979.

Change-Id: I52d97695484938701e5b7c269e2caf0c87d44d7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/391139
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-03-10 16:06:29 +00:00
Meng Zhuo 29b968e766 runtime,cmd/compile: change reg duff{zero,copy} for regabi riscv64
As CL 356519 require, X8-X23 will be argument register, however X10, X11
is used by duff device.

This CL changes X10, X11 into X24, X25 to meet the prerequisite.

Update #40724

Change-Id: Ie9b899afbba7e9a51bb7dacd89e49ca1c1fc33ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/357976
Trust: mzh <mzh@golangcn.org>
Run-TryBot: mzh <mzh@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
2022-03-10 06:26:28 +00:00
Rhys Hiltner 0d33a99675 runtime/pprof: fix pcDeck's frame indexing
When building the inlining deck, correctly identify which is the last
frame in the deck. Otherwise, when some forms of inlining cause a PC to
expand to multiple frames, the length of the deck's two slices will
diverge.

Fixes #51567

Change-Id: I24e7ba32cb16b167f4307178b3f03c29e5362c4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/391134
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Than McIntosh <thanm@google.com>
2022-03-09 17:07:49 +00:00
Rhys Hiltner bd77d6e240 runtime/pprof: check if PC is reused for inlining
When describing call stacks that include inlined function calls, the
runtime uses "fake" PCs to represent the frames that inlining removed.
Those PCs correspond to real NOP instructions that the compiler inserts
for this purpose.

Describing the call stack in a protobuf-formatted profile requires the
runtime/pprof package to collapse any sequences of fake call sites back
into single PCs, removing the NOPs but retaining their line info.

But because the NOP instructions are part of the function, they can
appear as leaf nodes in a CPU profile. That results in an address that
should sometimes be ignored (when it appears as a call site) and that
sometimes should be present in the profile (when it is observed
consuming CPU time).

When processing a PC address, consider it first as a fake PC to add to
the current inlining deck, and then as a previously-seen (real) PC.

Fixes #50996

Change-Id: I80802369978bd7ac9969839ecfc9995ea4f84ab4
Reviewed-on: https://go-review.googlesource.com/c/go/+/384239
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2022-03-08 21:23:02 +00:00
Tobias Klauser c3c74777bc runtime, syscall: implement syscall.Pipe using syscall.Pipe2 on solaris
All other platforms providing the pipe2 syscall already implement it
that way. Do so as well on solaris, which allows to drop
runtime.syscall_pipe and its dependencies as well.

Change-Id: Icf04777f21d1804da74325d173fefdc87caa42eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/390716
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Matt Layher <mdlayher@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-08 21:16:53 +00:00
Michael Pratt cc9d3f548a runtime: print goid when throwing in gentraceback
This makes it easier to figure out where the crash is occurring.

Change-Id: Ie1f78a360367090dcd61c61b2a55c34f3e2ff2eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/390034
Trust: David Chase <drchase@google.com>
Reviewed-by: David Chase <drchase@google.com>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-03-07 16:24:54 +00:00
eric fang 81767e23c2 runtime: support cgo traceback on linux arm64
Code essentially mirrors AMD64 implementation.

Change-Id: Ie97627a3041d1858fb1a30d2fc500302ab4011b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/373363
Trust: Eric Fang <eric.fang@arm.com>
Run-TryBot: Eric Fang <eric.fang@arm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-04 01:18:57 +00:00
Tobias Klauser 9d34fc5108 runtime: remove fallback to pipe on platforms with pipe2
On Linux, the minimum required kernel version for Go 1.18 was be changed
to 2.6.32, see #45964. The pipe2 syscall was added in 2.6.27.

All other platforms already provide the pipe2 syscall in the minimum
supported version:
- DragonFly BSD added it in version 4.2, see
  https://www.dragonflybsd.org/release42/
- FreeBSD added it in version 10.0, see
  https://www.freebsd.org/cgi/man.cgi?pipe(2)#end
- NetBSD added it in version 6.0, see
  https://man.netbsd.org/pipe2.2#HISTORY
- OpenBSD added it in version 5.7, see
  https://man.openbsd.org/pipe.2#HISTORY
- Illumos supports it since 2013, see
  https://www.illumos.org/issues/3714
- Solaris supports it since 11.4

This also allows to remove setNonblock which was only used in the pipe
fallback path on these platforms.

Change-Id: I1f40d32fd3065d74e22af77b9ff2292b9cf66706
Reviewed-on: https://go-review.googlesource.com/c/go/+/389354
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-03-03 20:47:17 +00:00
Cherry Mui 58804ea67a runtime: count spill slot for frame size at finalizer call
The finalizer is called using reflectcall. When register ABI is
used, the finalizer's argument is passed in register(s). But the
frame size calculation does not include the spill slot. When the
argument actually spills, it may clobber the caller's stack frame.
This CL fixes it.

Change-Id: Ibcc7507c518ba65c1c5a7759e5cab0ae3fc7efce
Reviewed-on: https://go-review.googlesource.com/c/go/+/389574
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-03-03 18:04:51 +00:00
Paul E. Murphy 78070ec3d4 syscall, runtime/internal/syscall: always return 0 in r2 on ppc64{,le} linux syscalls
Both endians perform syscalls similarly. Only CR0S0 and R3 hold
the resultant status of a syscall. A random value may be stored into
the second return value (r2) result in some cases. Always set it to
zero.

Fixes #51192

Change-Id: Ida6a5692578d2cdadf3099af28478b3bc364f623
Reviewed-on: https://go-review.googlesource.com/c/go/+/385796
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Trust: Paul Murphy <murp@ibm.com>
2022-03-03 17:28:58 +00:00
Paul E. Murphy d82c294da7 runtime: fix 32B backward copy on ppc64x
The test to enter the 32b copy loop always fails, and execution
falls back to a single 8B/iteration copy loop for copies of more
than 7 bytes. Likewise, the 32B loop has SRC/DST args mixed,
and fails to truncate DWORDS after completing.

Fix these, and unroll the 8B/iteration loop as it will only
execute 1-3 times if reached.

POWER10 benchmarks:

name                             old speed      new speed       delta
MemmoveOverlap/32                5.28GB/s ± 0%  10.37GB/s ± 0%   +96.22%
MemmoveOverlap/64                5.97GB/s ± 0%  18.15GB/s ± 0%  +203.95%
MemmoveOverlap/128               7.67GB/s ± 0%  24.35GB/s ± 0%  +217.41%
MemmoveOverlap/256               14.1GB/s ± 0%   25.0GB/s ± 0%   +77.48%
MemmoveOverlap/512               14.2GB/s ± 0%   30.9GB/s ± 0%  +118.19%
MemmoveOverlap/1024              12.3GB/s ± 0%   36.4GB/s ± 0%  +194.75%
MemmoveOverlap/2048              13.7GB/s ± 0%   48.8GB/s ± 0%  +255.24%
MemmoveOverlap/4096              14.1GB/s ± 0%   43.4GB/s ± 0%  +208.80%
MemmoveUnalignedDstOverlap/32    5.07GB/s ± 0%   3.78GB/s ± 0%   -25.33%
MemmoveUnalignedDstOverlap/64    6.00GB/s ± 0%   9.59GB/s ± 0%   +59.78%
MemmoveUnalignedDstOverlap/128   7.66GB/s ± 0%  13.51GB/s ± 0%   +76.42%
MemmoveUnalignedDstOverlap/256   13.4GB/s ± 0%   24.3GB/s ± 0%   +80.92%
MemmoveUnalignedDstOverlap/512   13.9GB/s ± 0%   30.3GB/s ± 0%  +118.29%
MemmoveUnalignedDstOverlap/1024  12.3GB/s ± 0%   37.3GB/s ± 0%  +203.07%
MemmoveUnalignedDstOverlap/2048  13.7GB/s ± 0%   45.9GB/s ± 0%  +235.39%
MemmoveUnalignedDstOverlap/4096  13.9GB/s ± 0%   41.2GB/s ± 0%  +196.34%
MemmoveUnalignedSrcOverlap/32    5.13GB/s ± 0%   5.18GB/s ± 0%    +0.98%
MemmoveUnalignedSrcOverlap/64    6.26GB/s ± 0%   9.53GB/s ± 0%   +52.29%
MemmoveUnalignedSrcOverlap/128   7.94GB/s ± 0%  18.40GB/s ± 0%  +131.76%
MemmoveUnalignedSrcOverlap/256   14.1GB/s ± 0%   25.5GB/s ± 0%   +81.40%
MemmoveUnalignedSrcOverlap/512   14.2GB/s ± 0%   30.9GB/s ± 0%  +116.76%
MemmoveUnalignedSrcOverlap/1024  12.4GB/s ± 0%   46.4GB/s ± 0%  +275.22%
MemmoveUnalignedSrcOverlap/2048  13.7GB/s ± 0%   48.7GB/s ± 0%  +255.16%
MemmoveUnalignedSrcOverlap/4096  14.0GB/s ± 0%   43.2GB/s ± 0%  +208.89%

Change-Id: I9fc6956ff454a2856d56077d1014388fb74c1f52
Reviewed-on: https://go-review.googlesource.com/c/go/+/384074
Trust: Paul Murphy <murp@ibm.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-03 14:39:51 +00:00
eric fang 2e9facbdd4 runtime: use stp/ldp to save and restore all registers on arm64
Async preemption needs to save and restore almost all of the registers,
currently this is done by ldr and str on arm64. We can do it with ldp
and stp as they are more efficient.

Change-Id: Ida5a6f0a8d825a56af607ba2c2cd91fdc2e8f67f
Reviewed-on: https://go-review.googlesource.com/c/go/+/379715
Reviewed-by: Cherry Mui <cherryyz@google.com>
Trust: Eric Fang <eric.fang@arm.com>
Run-TryBot: Eric Fang <eric.fang@arm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-03 01:58:56 +00:00
Keith Randall 510ad4561f runtime: improve work stealing randomness
For certain values of GOMAXPROCS, the current code is less random than
it looks. For example with GOMAXPROCS=12, there are 4 coprimes: 1 5 7 11.
That's bad, as 12 and 4 are not relatively prime. So if pos == 2, then we
always pick 7 as the inc. We want to pick pos and inc independently
at random.

Change-Id: I5c7e4f01f9223cbc2db12a685dc0bced2cf39abf
Reviewed-on: https://go-review.googlesource.com/c/go/+/369976
Run-TryBot: Keith Randall <khr@golang.org>
Trust: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2022-03-01 18:43:08 +00:00
Cherry Mui 57e3809884 runtime: avoid cgo_unsafe_args for syscall.syscall functions on darwin/arm64
Currently, syscall.syscall-like functions are defined as
cgo_unsafe_args, which makes them ABI0, as it takes the address of
the argument area based on ABI0 layout. Those functions are
linkname'd to the syscall package. When compiling the syscall
package, the compiler doesn't know they are ABI0 therefore
generate an ABIInternal call, which will use the wrapper. As some
of the functions (e.g. syscall6) has many arguments, the wrapper
would take a good amount of stack space. And those functions must
be nosplit. This causes nosplit overflow when building with -N -l
and -race.

Avoid that by rewriting the functions to not use cgo_unsafe_args.
Instead, make a struct locally and pass the address of that
struct. This way the functions are ABIInternal and the call will
not use the wrapper.

Fixes #51247.

Change-Id: I76c1ab86b9d28664fa7d5b9c7928fbb2fd8d1417
Reviewed-on: https://go-review.googlesource.com/c/go/+/386719
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-02-28 17:51:12 +00:00
Cherry Mui 26999cfd84 runtime/internal/atomic: set SP delta correctly for 64-bit atomic functions on ARM
64-bit atomic functions on ARM have the following structure:
- check if the address is 64-bit aligned, if not, prepare a frame
  and call panicUnaligned
- tail call armXXX or goXXX depending on GOARM

The alignment check calls panicUnaligned after preparing a frame,
so the stack can be unwound. The call never returns, so the SP is
not set back. However, the assembler assigns SP delta following
the instruction stream order, not the control flow. So it leaves
a nonzero SP delta after the check, to the tail call instructions,
which is wrong because when they are executed the SP is not
decremented. This CL fixes this by adding the SP back (the
instruction never executes, just tells the assembler to set the
SP delta back).

Should fix #51353.

Change-Id: I976cb1cfb0e9008b13538765cbc7eea0c19c7130
Reviewed-on: https://go-review.googlesource.com/c/go/+/388014
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2022-02-25 19:54:17 +00:00
Russ Cox d17b65ff54 crypto/x509, runtime: fix occasional spurious “certificate is expired”
As documented in #51209, we have been seeing a low-rate failure
on macOS builders caused by spurious x509 “certificate is expired” errors.

The root cause is that CFDateCreate takes a float64, but it is being
passed a uintptr instead. That is, we're not even putting CFDateCreate's
argument in the right register during the call. Luckily, having just
computed the argument by calling time.Duration.Seconds, which
returns a float64, most of the time the argument we want is still
in the right floating point register, somewhat accidentally.

The only time the lucky accident doesn't happen is when the goroutine
is rescheduled between calling time.Duration.Seconds and calling
into CFDateCreate *and* the rescheduling smashes the floating point
register, which can happen during various block memory moves,
since the floating point registers are also the SIMD registers.

Passing the float64 through explicitly eliminates the problem.
It is difficult to write a test for this that is suitable for inclusion
in the standard library. We will have to rely on the builders to
start flaking again if somehow this problem is reintroduced.

For future reference, there is a standalone test that used to fail
every few seconds at https://go.dev/play/p/OWfDpxgnW9g.

Fixes #51209.

Change-Id: I8b334a51e41f406b13f37270e9175c64fe6f55ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/387255
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-22 15:23:59 +00:00
hopehook 0261fa616a testdata: fix typo in comment
Change-Id: If3d5884d9f3f32606c510af5597529b832a8f4a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/386934
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2022-02-19 16:23:54 +00:00
Cherry Mui d27248c52f runtime: save some stack space for racecall on ARM64
To avoid nosplit overflow in -race -N -l build.

For #51247.

Change-Id: I48426bbd4adefd18eaf26ed51b4113c6a28305b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/386716
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-02-18 18:47:47 +00:00
Cherry Mui d93cc8cb96 runtime: define racefuncenter and racefuncexit as ABIInternal
They are called from compiler instrumented code as ABIInternal.
Define them as ABIInternal to avoid the wrappers and save some
stack space, to avoid nosplit overflow in -race -N -l build.

For #51247.

Change-Id: Iadad7d6da8ac03780a7b02b03b004c52d34e020a
Reviewed-on: https://go-review.googlesource.com/c/go/+/386715
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-02-18 18:46:44 +00:00
Michael Pratt b5af5c0834 runtime: enable sigPerThreadSyscall handling on android
CL 383434 forgot to enable these paths for android, which is still linux
just not via GOOS.

Fixes #51213.

Change-Id: I102e53e8671403ded6edb4ba04789154d7a0730b
Reviewed-on: https://go-review.googlesource.com/c/go/+/385954
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-15 19:04:00 +00:00
Michael Pratt 0a5fae2a0e runtime, syscall: reimplement AllThreadsSyscall using only signals.
In issue 50113, we see that a thread blocked in a system call can result
in a hang of AllThreadsSyscall. To resolve this, we must send a signal
to these threads to knock them out of the system call long enough to run
the per-thread syscall.

Stepping back, if we need to send signals anyway, it should be possible
to implement this entire mechanism on top of signals. This CL does so,
vastly simplifying the mechanism, both as a direct result of
newly-unnecessary code as well as some ancillary simplifications to make
things simpler to follow.

Major changes:

* The rest of the mechanism is moved to os_linux.go, with fields in mOS
  instead of m itself.
* 'Fixup' fields and functions are renamed to 'perThreadSyscall' so they
  are more precise about their purpose.
* Rather than getting passed a closure, doAllThreadsSyscall takes the
  syscall number and arguments. This avoids a lot of hairy behavior:
    * The closure may potentially only be live in fields in the M,
      hidden from the GC. Not necessary with no closure.
    * The need to loan out the race context. A direct RawSyscall6 call
      does not require any race context.
    * The closure previously conditionally panicked in strange
      locations, like a signal handler. Now we simply throw.
* All manual fixup synchronization with mPark, sysmon, templateThread,
  sigqueue, etc is gone. The core approach is much simpler:
  doAllThreadsSyscall sends a signal to every thread in allm, which
  executes the system call from the signal handler. We use (SIGRTMIN +
  1), aka SIGSETXID, the same signal used by glibc for this purpose. As
  such, we are careful to only handle this signal on non-cgo binaries.

Synchronization with thread creation is a key part of this CL. The
comment near the top of doAllThreadsSyscall describes the required
synchronization semantics and how they are achieved.

Note that current use of allocmLock protects the state mutations of allm
that are also protected by sched.lock. allocmLock is used instead of
sched.lock simply to avoid holding sched.lock for so long.

Fixes #50113

Change-Id: Ic7ea856dc66cf711731540a54996e08fc986ce84
Reviewed-on: https://go-review.googlesource.com/c/go/+/383434
Reviewed-by: Austin Clements <austin@google.com>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-15 15:40:35 +00:00
Michael Pratt 0b321c9a7c runtime/internal/syscall: new package for linux
Add a generic syscall package for use by the runtime. Eventually we'd
like to clean up system calls in the runtime to use more code generation
and be moved out of the main runtime package.

The implementations of the assembly functions are based on copies of
syscall.RawSyscall6, modified slightly for more consistency between
arches. e.g., renamed trap to num, always set syscall num register
first.

For now, this package is just the bare minimum needed for
doAllThreadsSyscall to make an arbitrary syscall.

For #51087.
For #50113.

Change-Id: Ibecb5e6303279ce15286759e1cd6a2ddc52f7c72
Reviewed-on: https://go-review.googlesource.com/c/go/+/383999
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2022-02-15 15:40:29 +00:00
Michael Pratt 7a132d6f4e runtime: move doAllThreadsSyscall to os_linux.go
syscall_runtime_doAllThreadsSyscall is only used on Linux. In
preparation of a follow-up CL that will modify the function to use other
Linux-only functions, move it to os_linux.go with no changes.

For #50113.

Change-Id: I348b6130038603aa0a917be1f1debbca5a5a073f
Reviewed-on: https://go-review.googlesource.com/c/go/+/383996
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Andrew G. Morgan <agm@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-15 15:40:09 +00:00
Cherry Mui bcee121ae4 cmd/compile, runtime: use unwrapped PC for goroutine creation tracing
With the switch to the register ABI, we now generate wrapper
functions for go statements in many cases. A new goroutine's start
PC now points to the wrapper function. This does not affect
execution, but the runtime tracer uses the start PC and the
function name as the name/label of that goroutine. If the start
function is a named function, using the name of the wrapper loses
that information. Furthur, the tracer's goroutine view groups
goroutines by start PC. For multiple go statements with the same
callee, they are grouped together. With the wrappers, which is
context-dependent as it is a closure, they are no longer grouped.

This CL fixes the problem by providing the underlying unwrapped
PC for tracing. The compiler emits metadata to link the unwrapped
PC to the wrapper function. And the runtime reads that metadata
and record that unwrapped PC for tracing.

(This doesn't work for shared buildmode. Unfortunate.)

TODO: is there a way to test?

Fixes #50622.

Change-Id: Iaa20e1b544111c0255eb0fc04427aab7a5e3b877
Reviewed-on: https://go-review.googlesource.com/c/go/+/384158
Trust: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-11 20:01:24 +00:00