Commit Graph

4953 Commits

Author SHA1 Message Date
sivchari ba0f8ce50f runtime: gofmt proc_test.go
Change-Id: I09a2be64e96fe85d84560728814af74b234d7210
GitHub-Last-Rev: bc881ea002
GitHub-Pull-Request: golang/go#45929
Reviewed-on: https://go-review.googlesource.com/c/go/+/316409
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
2021-05-06 12:33:02 +00:00
Andy Pan 3b304ce7fe runtime: implement runqdrain() for GC mark worker goroutines
Revive CL 310149

Change-Id: Ib4714ea5b2ade32c0f66edff841a79d8212bd79a
Reviewed-on: https://go-review.googlesource.com/c/go/+/313009
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
2021-05-05 15:53:20 +00:00
Joel Sing 4c9791299d runtime: remove redundant save_g call in mcall for arm
The setg call a few lines earlier has already performed the same iscgo check
and called save_g if necessary.

Change-Id: I6e7c44cef4e0397d6001a3d5b7e334cdfbc3ce22
Reviewed-on: https://go-review.googlesource.com/c/go/+/316929
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-05 09:00:06 +00:00
Michael Anthony Knyszek cd28ca3a4e runtime: abort when receiving a signal on a non-Go thread on Windows
Currently if a signal lands on a non-Go thread that's handled by the Go
handler, Go will emit a message. However, unlike everywhere else in the
runtime, Go will not abort the process after, and the signal handler
will try to continue executing.

This leads to cascading failures and possibly even memory corruption.

For #45638.

Change-Id: I546f4e82f339d555bed295528d819ac883b92bc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/316809
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-05-04 21:16:16 +00:00
David Chase 8a4b7294af cmd/compile: fix possible nil deref added in CL 270943
In the event allocSpan returned a nil, this would crash.
Cleaned up the code and comments slightly, too.

Change-Id: I6231d4b4c14218e6956b4a97a205adc3206f59ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/316429
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-05-03 18:17:42 +00:00
Dan Scales d75fbac54d cmd/compile: add edge from lock rank edge from forceGC to traceStackTab
This edge can happen when forcegchelper() calls
goparkunlock(&forcegc.lock, ...) while holding the forcegc lock.
goparkunlock() eventually calls park_m(). In park_m(), traceGoPark()
(which leads to (*traceStackTable).put() and acquires the traceStackTab
lock) can be called before the forcegc lock is released.

Fixes #45774

Change-Id: If0fceab596712eb9ec0b9b47326778bc0ff80913
Reviewed-on: https://go-review.googlesource.com/c/go/+/316029
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Trust: Carlos Amedee <carlos@golang.org>
2021-05-03 16:25:05 +00:00
Matthew Dempsky fadad851a3 cmd/compile: implement unsafe.Add and unsafe.Slice
Updates #19367.
Updates #40481.

Change-Id: Iabd2afdd0d520e5d68fd9e6dedd013335a4b3886
Reviewed-on: https://go-review.googlesource.com/c/go/+/312214
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2021-05-02 20:38:13 +00:00
Joel Sing 8e91458b19 runtime,syscall: convert syscall on openbsd/386 to libc
Convert the syscall package on openbsd/386 to use libc rather than performing
direct system calls.

Updates #36435

Change-Id: Ifcfbca0e6b933762596a564243caa850dac01442
Reviewed-on: https://go-review.googlesource.com/c/go/+/287654
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-30 20:00:36 +00:00
Joel Sing faff49aae6 runtime: switch runtime to libc for openbsd/386
Use libc rather than performing direct system calls for the runtime on
openbsd/386.

Updates #36435

Change-Id: I0cd65368bc824c81f5f98ea24e4f82db5468b170
Reviewed-on: https://go-review.googlesource.com/c/go/+/287653
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-30 19:59:10 +00:00
David Chase 0bbfc5c31e runtime: break up large calls to memclrNoHeapPointers to allow preemption
If something "huge" is allocated, and the zeroing is trivial (no pointers
involved) then zero it by chunks in a loop so that preemption can occur,
not all in a single non-preemptible call.

Benchmarking suggests that 256K is the best chunk size.

Updates #42642.

Change-Id: I94015e467eaa098c59870e479d6d83bc88efbfb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/270943
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-30 19:41:02 +00:00
Joel Sing c23a32284a runtime: work around vet asmdecl checks for openbsd/386 mstart_stub
Include a NOP with the SP in order to disable/bypass vet asmdecl checks
for runtime.mstart_stub on openbsd/386. Without this we get:

runtime/sys_openbsd_386.s:33:1: [386] mstart_stub: use of 32(SP) points beyond argument frame

Change-Id: I834ae3dbceffcb5776481b076ec2afe3700671cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/315789
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
2021-04-30 19:26:14 +00:00
Joel Sing c3b2b04156 runtime: switch openbsd/386 locking to libc
Switch openbsd/386 to locking via libc, rather than performing direct
system calls.

Update #36435

Change-Id: I8198171e21f9acf28846ad723ea9ff48f7c8a69d
Reviewed-on: https://go-review.googlesource.com/c/go/+/287652
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-30 18:34:40 +00:00
Joel Sing 4893eee9dc runtime: switch openbsd/386 to pthreads
This switches openbsd/386 to thread creation via pthreads, rather than doing
direct system calls.

Update #36435

Change-Id: I000a815fc0edd0272c3285954f3f007229bc60a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/250577
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-30 18:34:10 +00:00
Cherry Zhang 02ab8d1a1d cmd/compile, runtime: emit only GC data for stack objects
Currently, for stack objects, the compiler emits metadata that
includes the offset and type descriptor for each object. The type
descriptor symbol has many fields, and it references many other
symbols, e.g. field/element types, equality functions, names.

Observe that what we actually need at runtime is only the GC
metadata that are needed to scan the object, and the GC metadata
are "leaf" symbols (which doesn't reference other symbols). Emit
only the GC data instead. This avoids bringing live the type
descriptor as well as things referenced by it (if it is not
otherwise live).

This reduces binary sizes:

                     old          new
hello (println)    1187776      1133856 (-4.5%)
hello (fmt)        1902448      1844416 (-3.1%)
cmd/compile       22670432     22438576 (-1.0%)
cmd/link           6346272      6225408 (-1.9%)

No significant change in compiler speed.

name        old time/op       new time/op       delta
Template          184ms ± 2%        186ms ± 5%    ~     (p=0.905 n=9+10)
Unicode          78.4ms ± 5%       76.3ms ± 3%  -2.60%  (p=0.009 n=10+10)
GoTypes           1.09s ± 1%        1.08s ± 1%  -0.73%  (p=0.027 n=10+8)
Compiler         85.6ms ± 3%       84.6ms ± 4%    ~     (p=0.143 n=10+10)
SSA               7.23s ± 1%        7.25s ± 1%    ~     (p=0.780 n=10+9)
Flate             116ms ± 5%        115ms ± 6%    ~     (p=0.912 n=10+10)
GoParser          201ms ± 4%        195ms ± 1%    ~     (p=0.089 n=10+10)
Reflect           455ms ± 1%        458ms ± 2%    ~     (p=0.050 n=9+9)
Tar               155ms ± 2%        155ms ± 3%    ~     (p=0.436 n=10+10)
XML               202ms ± 2%        200ms ± 2%    ~     (p=0.053 n=10+9)

Change-Id: I33a7f383d79afba1a482cac6da0cf5b7de9c0ec4
Reviewed-on: https://go-review.googlesource.com/c/go/+/313514
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-04-30 00:08:50 +00:00
Michael Anthony Knyszek 897baae953 runtime/metrics: add additional allocation metrics
This change adds four additional metrics to the runtime/metrics package
to fill in a few gaps with runtime.MemStats that were overlooked. The
biggest one is TotalAlloc, which is impossible to find with the
runtime/metrics package, but also add a few others for convenience and
clarity. For instance, the total number of objects allocated and freed
are technically available via allocs-by-size and frees-by-size, but it's
onerous to get them (one needs to sum the sample counts in the
histograms).

The four additional metrics are:
- /gc/heap/allocs:bytes   -- total bytes allocated (TotalAlloc)
- /gc/heap/allocs:objects -- total objects allocated (Mallocs - [tiny])
- /gc/heap/frees:bytes    -- total bytes frees (TotalAlloc-HeapAlloc)
- /gc/heap/frees:objects  -- total objects freed (Frees - [tiny])

This change also updates the descriptions of allocs-by-size and
frees-by-size to be more precise.

Change-Id: Iec8c1797a584491e3484b198f2e7f325b68954a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/312431
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-29 21:54:05 +00:00
Keith Randall 2c05ba4ae0 runtime: top align tinyallocs in race mode
Top align allocations in tinyalloc buckets when in race mode.
This will make checkptr checks more reliable, because any code
that modifies a pointer past the end of the object will trigger
a checkptr error.

No test, because we need -race for this to actually kick in.  We could
add it to the race detector tests, but the race detector tests are all
geared towards race detector reports, not checkptr reports. Mucking
with parsing reports is more than a test is worth.

Fixes #38872

Change-Id: Ie56f0fbd1a9385539f6631fd1ac40c3de5600154
Reviewed-on: https://go-review.googlesource.com/c/go/+/315029
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-29 17:39:31 +00:00
Michael Anthony Knyszek e03cca6407 runtime: use 4 MiB heap arenas on iOS
iOS arm64 is a 64-bit platform but with a strictly 32-bit address space
(technically 33 bits, but the bottom half is unavailable to the
application). Since address space is limited, use 4 MiB arenas instead
of 64 MiB arenas. No changes are needed to the arena index because it's
still relatively small; this change just brings iOS more in line with
32-bit platforms.

Change-Id: I484e2d273d896fd0a57cd5c25012df0aef160290
Reviewed-on: https://go-review.googlesource.com/c/go/+/270538
Trust: Michael Knyszek <mknyszek@google.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-29 17:08:05 +00:00
Ian Lance Taylor b15372f39b runtime: remove linux-amd64 walltime function
It's never called.

Change-Id: I8956743b21301816b5f37a9b34e3f50ef7b2e70a
Reviewed-on: https://go-review.googlesource.com/c/go/+/314771
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-29 14:16:19 +00:00
Ian Lance Taylor 862ddb37b4 runtime: rename walltime1 to walltime
Change-Id: Iec9de5ca56eb68d524bbaa0668515dbd09ad38a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/314770
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-29 14:15:04 +00:00
Ian Lance Taylor 42953bc9f5 runtime: remove walltime function
There was only one meaningful caller, which changes to call time_now.

This clearly separates systems that use walltime1 to be just those
that use the stub version of time_now. That is to say, those that do
not provide an assembler version of time_now.

Change-Id: I14c06cc402070bd705f953af6f9966785015e2a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/314769
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-29 04:19:20 +00:00
Ian Lance Taylor d09947522d runtime: implement time.now in assembly for linux-amd64
name                                                       old time/op      new time/op      delta
AfterFunc-12                                                   66.7µs ± 3%      66.8µs ± 4%     ~     (p=0.836 n=20+20)
After-12                                                       99.4µs ± 4%      98.1µs ± 3%   -1.31%  (p=0.013 n=20+20)
Stop-12                                                        66.1µs ±12%      65.4µs ±10%     ~     (p=0.602 n=20+20)
SimultaneousAfterFunc-12                                        110µs ± 1%       114µs ± 2%   +3.98%  (p=0.000 n=19+18)
StartStop-12                                                   32.1µs ±15%      32.2µs ±13%     ~     (p=0.620 n=20+20)
Reset-12                                                       3.66µs ± 2%      3.63µs ± 5%   -0.92%  (p=0.018 n=20+17)
Sleep-12                                                        134µs ± 1%       139µs ± 4%   +3.97%  (p=0.000 n=20+18)
Ticker-12                                                      32.8µs ± 1%      32.6µs ± 2%   -0.63%  (p=0.017 n=18+20)
TickerReset-12                                                 3.72µs ± 3%      3.71µs ± 3%     ~     (p=0.753 n=20+20)
TickerResetNaive-12                                            68.9µs ±11%      65.8µs ± 8%   -4.44%  (p=0.008 n=20+20)
Now-12                                                         33.3ns ± 1%      29.6ns ± 0%  -11.06%  (p=0.000 n=18+16)
NowUnixNano-12                                                 34.6ns ± 0%      31.2ns ± 0%   -9.94%  (p=0.000 n=20+17)
NowUnixMilli-12                                                35.0ns ± 1%      30.9ns ± 0%  -11.75%  (p=0.000 n=19+15)
NowUnixMicro-12                                                35.0ns ± 0%      30.9ns ± 0%  -11.85%  (p=0.000 n=20+19)
Format-12                                                       302ns ± 3%       306ns ± 3%   +1.22%  (p=0.009 n=20+20)
FormatNow-12                                                    184ns ± 5%       187ns ± 2%   +1.25%  (p=0.046 n=20+19)
MarshalJSON-12                                                  262ns ± 2%       270ns ± 3%   +2.99%  (p=0.000 n=20+20)
MarshalText-12                                                  262ns ± 3%       268ns ± 3%   +2.37%  (p=0.000 n=19+19)
Parse-12                                                        145ns ± 1%       148ns ± 0%   +2.27%  (p=0.000 n=18+19)
ParseDuration-12                                               82.3ns ± 1%      79.7ns ± 1%   -3.06%  (p=0.000 n=20+20)
Hour-12                                                        4.48ns ± 1%      4.42ns ± 1%   -1.32%  (p=0.000 n=19+19)
Second-12                                                      4.44ns ± 1%      4.42ns ± 1%   -0.39%  (p=0.000 n=20+18)
Year-12                                                        11.2ns ± 1%      11.1ns ± 1%     ~     (p=0.193 n=20+20)
Day-12                                                         14.8ns ± 0%      14.8ns ± 1%     ~     (p=0.873 n=19+20)
ISOWeek-12                                                     17.2ns ± 0%      17.2ns ± 0%     ~     (p=0.605 n=18+20)

name                                                       old avg-late-ns  new avg-late-ns  delta
ParallelTimerLatency-12                                          375k ± 3%        377k ± 3%     ~     (p=0.445 n=20+20)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=1-12         136k ± 2%        137k ± 2%     ~     (p=0.242 n=20+20)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=2-12        97.4k ±11%       96.4k ±10%     ~     (p=0.336 n=19+20)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=3-12        74.8k ± 3%       74.2k ± 3%     ~     (p=0.158 n=20+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=4-12        70.7k ± 7%       70.4k ± 6%     ~     (p=0.879 n=20+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=5-12        65.8k ± 9%       66.3k ±14%     ~     (p=0.594 n=17+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=6-12        55.5k ±29%       56.7k ±30%     ~     (p=0.758 n=20+20)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=7-12        45.3k ±29%       43.6k ±33%     ~     (p=0.212 n=19+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=8-12        64.7k ±46%       65.2k ±78%     ~     (p=0.480 n=18+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=9-12         147k ±88%        119k ±83%     ~     (p=0.092 n=19+18)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=10-12       2.63M ±29%       2.70M ±59%     ~     (p=0.989 n=19+20)
StaggeredTickerLatency/work-dur=2ms/tickers-per-P=1-12          81.4k ± 4%       80.2k ± 3%   -1.55%  (p=0.009 n=17+18)

name                                                       old max-late-ns  new max-late-ns  delta
ParallelTimerLatency-12                                        7.66M ±102%      6.98M ±131%     ~     (p=0.445 n=20+20)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=1-12         381k ±12%        382k ±17%     ~     (p=0.901 n=17+16)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=2-12         388k ±69%        356k ±10%     ~     (p=0.363 n=17+16)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=3-12         350k ±17%        347k ±25%     ~     (p=0.538 n=19+18)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=4-12         378k ±52%        341k ±30%     ~     (p=0.153 n=18+17)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=5-12         392k ±54%        410k ±78%     ~     (p=0.730 n=18+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=6-12         467k ±80%       527k ±129%     ~     (p=0.616 n=17+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=7-12        915k ±138%      1023k ±227%     ~     (p=0.696 n=20+18)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=8-12       1.84M ±155%      1.74M ±158%     ~     (p=0.893 n=18+19)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=9-12        6.95M ±92%       7.66M ±91%     ~     (p=0.687 n=19+20)
StaggeredTickerLatency/work-dur=300µs/tickers-per-P=10-12       18.6M ±22%       16.2M ±28%  -12.78%  (p=0.003 n=19+19)
StaggeredTickerLatency/work-dur=2ms/tickers-per-P=1-12          1.01M ± 8%       1.04M ±10%     ~     (p=0.111 n=19+18)

Change-Id: I96aa2e0206a6e9286bcbfc8be372e84608ed4e2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/314277
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-29 04:18:39 +00:00
Ian Lance Taylor c96fec9036 runtime: use a single definition of time_now for faketime
Build other definitions with the !faketime build tag.

This makes it easy for us to add new assembly implementations of time.now.

Change-Id: I4e48e41a4a04ab001030e6d1cdd9cebfa0161b0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/314274
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-28 20:17:20 +00:00
Ian Lance Taylor b36596b14f runtime: move Windows time.now implementations into separate files
This is a step toward separating whether time.now is implemented in
assembly from whether we are using faketime.

Change-Id: I8bf059b44a103b034835e3d3b799319cc05e9552
Reviewed-on: https://go-review.googlesource.com/c/go/+/314273
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-28 20:16:36 +00:00
Ian Lance Taylor e51246c881 runtime: consolidate Windows time constants into single copy
Change-Id: I1a583d3da9cca4ac51f3fec9b508b7638b452d60
Reviewed-on: https://go-review.googlesource.com/c/go/+/314270
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-28 15:58:43 +00:00
Joel Sing f68878f0fc cmd/dist,runtime: support cgo on openbsd/mips64
Add support for cgo on openbsd/mips64.

Fixes #43005

Change-Id: I2386204f53fa984a01a9d89f0b6c96455768f326
Reviewed-on: https://go-review.googlesource.com/c/go/+/275896
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-28 12:51:42 +00:00
Felix Geisendörfer 0ae9c3b98b runtime/pprof: fix block profile bias
Block profiles were biased towards infrequent long events over frequent
short events. This fix corrects the bias by aggregating shorter events
as longer but less frequent in the profiles. As a result their
cumulative duration will be accurately represented in the profile
without skewing their sample mean (duration/count).

Credit to @dvyukov for suggesting to adjust the count in the
saveblockevent function.

Fixes #44192.

Change-Id: I71a99d7f6ebdb2d484d44890a2517863cceb4004
Reviewed-on: https://go-review.googlesource.com/c/go/+/299991
Trust: Michael Pratt <mpratt@google.com>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-04-27 18:54:49 +00:00
Michael Anthony Knyszek 0b9ca4d907 runtime/metrics: add tiny allocs metric
Currently tiny allocations are not represented in either MemStats or
runtime/metrics, but they're represented in MemStats (indirectly) via
Mallocs. Add them to runtime/metrics by first merging
memstats.tinyallocs into consistentHeapStats (just for simplicity; it's
monotonic so metrics would still be self-consistent if we just read it
atomically) and then adding /gc/heap/tiny/allocs:objects to the list of
supported metrics.

Change-Id: Ie478006ab942a3e877b4a79065ffa43569722f3d
Reviewed-on: https://go-review.googlesource.com/c/go/+/312909
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-04-27 13:59:22 +00:00
Alex Brainman e0815d041c runtime: replace --buildmode with -buildmode in tests
While debugging issue #45638, I discovered that some tests were using
--buildmode command line parameter instead of -buildmode.

The --buildmode parameter is handled properly by the flag package - it
is read as -buildmode. But we should correct code anyway.

Updates #45638

Change-Id: I75cf95c7d11dcdf4aeccf568b2dea77bd8942352
Reviewed-on: https://go-review.googlesource.com/c/go/+/313351
Trust: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-27 08:39:25 +00:00
Alex Brainman ca8e8317be runtime: add missing import "C" in TestLibraryCtrlHandler
CL 211139 added TestLibraryCtrlHandler. But the CL left out import "C"
line in the test file that is supposed to be build with Cgo.

While debugging issue #45638, I discovered that the DLL built during
TestLibraryCtrlHandler does not have Dummy function. Adding import "C"
makes Dummy function appear in DLL function list.

TestLibraryCtrlHandler does not actually calls Dummy function. So I
don't see how this change affects issue #45638, but still let's make
this code correct.

Updates #45638

Change-Id: Ibab8fed29ef2ae446d0815842cf0bd040a5fb943
Reviewed-on: https://go-review.googlesource.com/c/go/+/313350
Trust: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-27 08:36:54 +00:00
Cherry Zhang c430313992 runtime: use abi.FuncPCABI0 for libc syscall wrappers
In CL 288092 we made Darwin syscall wrappers as ABIInternal, so
their addresses taken from Go using funcPC are the actual function
entries, not the wrappers.

As we introduced internal/abi.FuncPCABIxxx intrinsics, use that.
And change the assembly functions back to ABI0.

Change-Id: I50645af74883e2d5dfcd67a5e8c739222c6f645b
Reviewed-on: https://go-review.googlesource.com/c/go/+/313250
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-26 18:44:31 +00:00
Cherry Zhang 8ff1da0b81 runtime: fix offset in comment
Change-Id: I2770195cb53220948081a6265f891ef064e4f763
Reviewed-on: https://go-review.googlesource.com/c/go/+/313629
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-26 18:42:12 +00:00
Michael Anthony Knyszek 14ade57ab8 runtime: fix scavenge min fraction constant floor division
Currently there's a minor bug where the constant for the min fraction of
time spent scavenging is rounded down to zero. I don't think this
affects anything in practice because this case is exceedingly rare and
extreme, but currently it doesn't properly prevent the pacing parameters
from getting out of hand in these extreme cases.

Fixes #44036.

Change-Id: I7de644ab0ecac33765c337a736482a0966882780
Reviewed-on: https://go-review.googlesource.com/c/go/+/313249
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-26 18:15:31 +00:00
Michael Anthony Knyszek e7db792fc5 runtime: simplify TestSelectStackAdjust
Currently this test attempts to trigger a concurrent GC in a very
indirect way, but the way it does so is extremely error-prone. This test
is virtually always prone to flaking based on test order. For example if
the test that executed immediately before this one made a big heap but
didn't clean it up, then this test could easily fail to trigger a GC.
I was able to prove this with a small reproducer.

This roundabout way of triggering a GC is also way overkill for this
test. It just wants to get goroutines in a select and shrink their
stacks. Every GC will schedule a stack for shrinking if it can.

Replace all the complicated machinery with a single runtime.GC call.
I've confirmed that the test consistently triggers a stack shrink,
noting that both shrinkstack's copystack call is made and that
syncadjustsudogs (the relevant function that's being indirectly tested)
are both called.

Fixes #44610.

Change-Id: Ib1c091e0d1475bf6c596f56dc9b85eaea366fc73
Reviewed-on: https://go-review.googlesource.com/c/go/+/313109
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-04-23 22:31:20 +00:00
Russ Cox bf9216055b runtime/testdata: fix testprogcgo for windows/arm64
Our toolchain does not like -mnop-fun-dllimport.

Change-Id: Iaaee01fe0f4b0959406a35eb13aefa390116b483
Reviewed-on: https://go-review.googlesource.com/c/go/+/312043
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-23 21:43:10 +00:00
Russ Cox a25d1d45d9 runtime: fix windows/arm64 callbackasm1 for frame pointer hack
Fixes the previously failing TestStdcallAndCDeclCallbacks
for the 9+ argument case.

The last time this code passed, the invisible frame pointer
below SP was apparently not enabled on windows/arm64.

Change-Id: Ifc3064e894b2f39d6410f3be51c17309ebab08a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/312042
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-23 21:43:08 +00:00
Russ Cox 16330817aa runtime: add windows/arm64 cgo-linking code
This code is needed for use with cgo proper
(as opposed to hand-written DLL calls, which
we always use but only exercise cgo execution,
not cgo linking).

Change-Id: Iddc31d9c1c924d83d032b80dca65ddfda6624046
Reviewed-on: https://go-review.googlesource.com/c/go/+/312041
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-04-23 21:43:07 +00:00
Cherry Zhang 1421516973 cmd/compile, internal/abi: add FuncPCABIxxx intrinsics
When ABI wrappers are used, there are cases where in Go code we
need the PC of the defined function instead of the ABI wrapper.
Currently we work around this by define such functions as
ABIInternal, even if they do not actually follow the internal ABI.

This CL introduces internal/abi.FuncPCABIxxx functions as compiler
intrinsics, which return the underlying defined function's entry
PC if the argument is a direct reference of a function of the
expected ABI, and reject it if it is of a different ABI.

As a proof of concept, change runtime.goexit back to ABI0 and use
internal/abi.FuncPCABI0 to retrieve its PC.

Updates #44065.

Change-Id: I02286f0f9d99e6a3090f9e8169dbafc6804a2da6
Reviewed-on: https://go-review.googlesource.com/c/go/+/304232
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-04-23 21:05:39 +00:00
Michael Pratt bedfeed54a runtime,runtime/metrics: add metric to track scheduling latencies
This change adds a metric to track scheduling latencies, defined as the
cumulative amount of time a goroutine spends being runnable before
running again. The metric is an approximations and samples instead of
trying to record every goroutine scheduling latency.

This change was primarily authored by mknyszek@google.com.

Change-Id: Ie0be7e6e7be421572eb2317d3dd8dd6f3d6aa152
Reviewed-on: https://go-review.googlesource.com/c/go/+/308933
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-23 13:48:10 +00:00
Michael Pratt ecfce58965 runtime: skip work recheck for non-spinning Ms
When an M transitions from spinning to non-spinning state, it must
recheck most sources of work to avoid missing work submitted between its
initial check and decrementing sched.nmspinning (see "delicate dance"
comment).

Ever since the scheduler rewrite in Go 1.1 (golang.org/cl/7314062), we
have performed this recheck on all Ms before stopping, regardless of
whether or not they were spinning.

Unfortunately, there is a problem with this approach: non-spinning Ms
are not eligible to steal work (note the skip over the stealWork block),
but can detect work during the recheck. If there is work available, this
non-spinning M will jump to top, skip stealing, land in recheck again,
and repeat. i.e., it will spin uselessly.

The spin is bounded. This can only occur if there is another spinning M,
which will either take the work, allowing this M to stop, or take some
other work, allowing this M to upgrade to spinning. But the spinning is
ultimately just a fancy spin-wait.

golang.org/issue/43997 discusses several ways to address this. This CL
takes the simplest approach: skipping the recheck on non-spinning Ms and
allowing them to go to stop.

Results for scheduler-relevant runtime and time benchmarks can be found
at https://perf.golang.org/search?q=upload:20210420.5.

The new BenchmarkCreateGoroutinesSingle is a characteristic example
workload that hits this issue hard. A single M readies lots of work
without itself parking. Other Ms must spin to steal work, which is very
short-lived, forcing those Ms to spin again. Some of the Ms will be
non-spinning and hit the above bug.

With this fixed, that benchmark drops in CPU usage by a massive 68%, and
wall time 24%. BenchmarkNetpollBreak shows similar drops because it is
unintentionally almost the same benchmark (create short-living Gs in a
loop). Typical well-behaved programs show little change.

We also measure scheduling latency (time from goready to execute). Note
that many of these benchmarks are very noisy because they don't involve
much scheduling. Those that do, like CreateGoroutinesSingle, are
expected to increase as we are replacing unintentional spin waiting with
a real park.

Fixes #43997

Change-Id: Ie1d1e1800f393cee1792455412caaa5865d13562
Reviewed-on: https://go-review.googlesource.com/c/go/+/310850
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-22 20:45:37 +00:00
Cherry Zhang 537cde0b4b cmd/compile, runtime: add metadata for argument printing in traceback
Currently, when the runtime printing a stack track (at panic, or
when runtime.Stack is called), it prints the function arguments
as words in memory. With a register-based calling convention,
the layout of argument area of the memory changes, so the
printing also needs to change. In particular, the memory order
and the syntax order of the arguments may differ. To address
that, this CL lets the compiler to emit some metadata about the
memory layout of the arguments, and the runtime will use this
information to print arguments in syntax order.

Previously we print the memory contents of the results along with
the arguments. The results are likely uninitialized when the
traceback is taken, so that information is rarely useful. Also,
with a register-based calling convention the results may not
have corresponding locations in memory. This CL changes it to not
print results.

Previously the runtime simply prints the memory contents as
pointer-sized words. With a register-based calling convention,
as the layout changes, arguments that were packed in one word
may no longer be in one word. Also, as the spill slots are not
always initialized, it is possible that some part of a word
contains useful informationwhile the rest contains garbage.
Instead of letting the runtime recreating the ABI0 layout and
print them as words, we now print each component separately.
Aggregate-typed argument/component is surrounded by "{}".

For example, for a function

F(int, [3]byte, byte) int

when called as F(1, [3]byte{2, 3, 4}, 5), it used to print

F(0x1, 0x5040302, 0xXXXXXXXX) // assuming little endian, 0xXXXXXXXX is uninitilized result

Now prints

F(0x1, {0x2, 0x3, 0x4}, 0x5).

Note: the liveness tracking of the spill splots has not been
implemented in this CL. Currently the runtime just assumes all
the slots are live and print them all.

Increase binary sizes by ~1.5%.

                     old          new
hello (println)    1171328      1187712 (+1.4%)
hello (fmt)        1877024      1901600 (+1.3%)
cmd/compile       22326928     22662800 (+1.5%)
cmd/go            13505024     13726208 (+1.6%)

Updates #40724.

Change-Id: I351e0bf497f99bdbb3f91df2fb17e3c2c5c316dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/304470
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-22 17:47:59 +00:00
Cherry Zhang d3853fb4e6 runtime: call cgocallbackg indirectly
cgocallback calls cgocallbackg after switching the stack. Call it
indirectly to bypass the linker's nosplit check.

Apparently (at least on Windows) cgocallbackg can use quite a bit
stack space in a nosplit chain. We have been running over the
nosplit limit, or very close to the limit. Since it switches
stack in cgocallback, it is not meaningful to count frames above
cgocallback and below cgocallbackg together. Bypass the check.

For #45658.

Change-Id: Ie22017e3f82d2c1fcc37336696f2d02757856399
Reviewed-on: https://go-review.googlesource.com/c/go/+/312669
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-22 16:15:44 +00:00
Ian Lance Taylor 02a8e83661 runtime: don't run TestCrashDumpsAllThreads in parallel
It sometimes seems to time out on slow systems, perhaps due to
being run at the same time as a lot of other work.

Also move the code to testdata/testprog, so that we don't have to
build it separately.

I hope that this
Fixes #35356

Change-Id: I875b858fa23836513ae14d3116461e22fffd5352
Reviewed-on: https://go-review.googlesource.com/c/go/+/312510
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-22 00:34:25 +00:00
Michael Anthony Knyszek 6639bb894d runtime: call nanotimeQPC from nanotime1 normally
Currently we call runtimeQPC as ABIInternal because it shaves off 24
bytes by not having an extra wrapper, and at the time we were exceeding
the nosplit stack limit in some cases.

However, this code was written before we had the regabiargs GOEXPERIMENT
flag, and wasn't properly flagged. Naturally, with regabiargs enabled,
it leads to garbage being returned, because it needs to store
runtimeQPC's result to the stack.

We didn't notice this because today runtimeQPC is only used in Wine, not
on any native Windows platform.

Back when I wrote this code, it appeared to be necessary on even native
Windows, but it turns out that's not true anymore. Turn it back into a
native call through a wrapper.

For #40724.

Change-Id: Ia2e5901965ef46c5f299daccef49952026854fe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/312429
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-21 23:01:42 +00:00
Andrew G. Morgan 7e97e4e8cc syscall: syscall.AllThreadsSyscall signal handling fixes
The runtime support for syscall.AllThreadsSyscall() functions had
some corner case deadlock issues when signal handling was in use.
This was observed in at least 3 build test failures on ppc64 and
amd64 architecture CGO_ENABLED=0 builds over the last few months.

The fixes involve more controlled handling of signals while the
AllThreads mechanism is being executed. Further details are
discussed in bug #44193.

The all-threads syscall support is new in go1.16, so earlier
releases are not affected by this bug.

Fixes #44193

Change-Id: I01ba8508a6e1bb2d872751f50da86dd07911a41d
Reviewed-on: https://go-review.googlesource.com/c/go/+/305149
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Pratt <mpratt@google.com>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-21 21:25:26 +00:00
Michael Pratt 4d56576ec0 runtime: move timer recheck after GC recheck
When rechecking for work after transitioning from a spinning to
non-spinning M, checking timers before GC isn't useful. That is, if
there is GC work available, it will run immediately and the updated
pollUntil is unused.

Move this check to just before netpoll, where pollUntil is used. While
this technically improves efficiency in the (rare) case that we find
GC work in this block, the primary motivation is simply to improve
clarity by moving the update closer to use.

For #43997

Change-Id: Ibc7fb308ac4a582875c200659c9e272121a89f3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/308654
Trust: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-21 13:38:59 +00:00
Ian Lance Taylor c33ced6d8a runtime: don't test sig.inuse in sigsend
Signals can be delivered on a different thread. There is no necessary
happens-before relationship between setting sig.inuse in signal_enable
and checking it in sigsend. It is theoretically possible, if unlikely,
that sig.inuse is set by thread 1, thread 2 receives a signal, does not
see that sig.inuse is set, and discards the signal. This could happen
if the signal is received immediately after the first call to signal_enable.

For #33174

Change-Id: Idb0f1c77847b7d4d418bd139e801c0c4460531d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/312131
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-04-21 02:39:25 +00:00
Josh Bleecher Snyder faa4fa1a6e cmd/compile: allow conversion from slice to array ptr
Panic if the slice is too short.

Updates #395

Change-Id: I90f4bff2da5d8f3148ba06d2482084f32b25c29a
Reviewed-on: https://go-review.googlesource.com/c/go/+/301650
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-04-21 00:53:48 +00:00
Ian Lance Taylor 48e3d92454 Revert "runtime: implement runqdrain() for GC mark worker goroutines"
This reverts CL 310149.

Reason for revert: Breaks longtest builders:

https://build.golang.org/log/6af9fb147fa3101154db10e7ce055e8267cd4c93
https://build.golang.org/log/172ed6e1ec3bb503370333ee421c590fd2a72d0a

Change-Id: Iaf5a8b9eec51d0517311e050d0b0f7569759d292
Reviewed-on: https://go-review.googlesource.com/c/go/+/312129
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-20 22:47:50 +00:00
Michael Pratt dbade774c3 runtime: refactor findrunnable spinning recheck
Break the main components of the findrunnable spinning -> non-spinning
recheck out into their own functions, which simplifies both findrunnable
and the new functions, which can make use of fancy features like early
returns.

This CL should have no functional changes.

For #43997
For #44313

Change-Id: I6d3060fcecda9920a3471ff338f73d53b1d848a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/307914
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-20 20:58:52 +00:00
Andy Pan fbb600b283 runtime: implement runqdrain() for GC mark worker goroutines
Change-Id: Ida44a2e07f277bee8806538ecee4beee3474cf3d
Reviewed-on: https://go-review.googlesource.com/c/go/+/310149
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-20 19:48:29 +00:00