Commit Graph

4459 Commits

Author SHA1 Message Date
Cherry Zhang a413908dd0 all: add GOOS=ios
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin"
build tag, like GOOS=android matches "linux" and GOOS=illumos
matches "solaris". Only ios/arm64 is supported (ios/amd64 is
not).

GOOS=ios and GOOS=darwin remain essentially the same at this
point. They will diverge at later time, to differentiate macOS
and iOS.

Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"),
except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"),
it remains GOOS=="darwin".

Updates #38485.

Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c
Reviewed-on: https://go-review.googlesource.com/c/go/+/254740
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-09-23 18:12:59 +00:00
Michael Pratt d42b32e321 runtime: add sched.lock assertions
Functions that require holding sched.lock now have an assertion.

A few places with missing locks have been fixed in this CL:

Additionally, locking is added around the call to procresize in
schedinit. This doesn't technically need a lock since the program is
still starting (thus no concurrency) when this is called, but lock held
checking doesn't know that.

Updates #40677

Change-Id: I198d3cbaa727f7088e4d55ba8fa989cf1ee8f9cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/250261
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-09-22 15:14:09 +00:00
Michael Pratt 53c9b9588a runtime: check held locks with staticlockranking
When lock ranking is enabled, we can now assert that lock preconditions
are met by checking that the caller holds required locks on function
entry.

This change adds the infrastructure to add assertions. Actual assertions
will be added for various locks in subsequent changes.

Some functions are protected by locks that are not directly accessible
in the function. In that case, we can use assertRankHeld to check that
any lock with the rank is held. This is less precise, but it avoids
requiring passing the lock into the functions.

Updates #40677

Change-Id: I843c6874867f975e90a063f087b6e2ffc147877b
Reviewed-on: https://go-review.googlesource.com/c/go/+/245484
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-09-22 15:13:57 +00:00
Michael Pratt 02ff8b8ce4 runtime: expand gopark documentation
unlockf is called after the G is put into _Gwaiting, meaning another G
may have readied this one before unlockf is called.

This is implied by the current doc, but add additional notes to call out
this behavior, as it can be quite surprising.

Updates #40641

Change-Id: I60b1ccc6a4dd9ced8ad2aa1f729cb2e973100b59
Reviewed-on: https://go-review.googlesource.com/c/go/+/256058
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-09-21 15:08:44 +00:00
Michael Anthony Knyszek eb3c6a93c3 runtime: disable stack shrinking in activeStackChans race window
Currently activeStackChans is set before a goroutine blocks on a channel
operation in an unlockf passed to gopark. The trouble is that the
unlockf is called *after* the G's status is changed, and the G's status
is what is used by a concurrent mark worker (calling suspendG) to
determine that a G has successfully been suspended. In this window
between the status change and unlockf, the mark worker could try to
shrink the G's stack, and in particular observe that activeStackChans is
false. This observation will cause the mark worker to *not* synchronize
with concurrent channel operations when it should, and so updating
pointers in the sudog for the blocked goroutine (which may point to the
goroutine's stack) races with channel operations which may also
manipulate the pointer (read it, dereference it, update it, etc.).

Fix the problem by adding a new atomically-updated flag to the g struct
called parkingOnChan, which is non-zero in the race window above. Then,
in isShrinkStackSafe, check if parkingOnChan is zero. The race is
resolved like so:

* Blocking G sets parkingOnChan, then changes status in gopark.
* Mark worker successfully suspends blocking G.
* If the mark worker observes parkingOnChan is non-zero when checking
  isShrinkStackSafe, then it's not safe to shrink (we're in the race
  window).
* If the mark worker observes parkingOnChan as zero, then because
  the mark worker observed the G status change, it can be sure that
  gopark's unlockf completed, and gp.activeStackChans will be correct.

The risk of this change is low, since although it reduces the number of
places that stack shrinking is allowed, the window here is incredibly
small. Essentially, every place that it might crash now is replaced with
no shrink.

This change adds a test, but the race window is so small that it's hard
to trigger without a well-placed sleep in park_m. Also, this change
fixes stackGrowRecursive in proc_test.go to actually allocate a 128-byte
stack frame. It turns out the compiler was destructuring the "pad" field
and only allocating one uint64 on the stack.

Fixes #40641.

Change-Id: I7dfbe7d460f6972b8956116b137bc13bc24464e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/247050
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
2020-09-21 14:34:33 +00:00
Paschalis Tsilias 331614c4da runtime: improve error messages after allocating a stack that is too big
In the current implementation, we can observe crashes after calling
debug.SetMaxStack and allocating a stack larger than 4GB since
stackalloc works with 32-bit sizes. To avoid this, we define an upper
limit as the largest feasible point we can grow a stack to and provide a
better error message when we get a stack overflow.

Fixes #41228

Change-Id: I55fb0a824f47ed9fb1fcc2445a4dfd57da9ef8d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/255997
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Giovanni Bajo <rasky@develer.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-20 09:54:44 +00:00
Keith Randall 4c4a376736 runtime/debug: skip fault address test on unsupported platforms
Change-Id: I6a6fe616365fa542218fbc9ff61805ff70a1ef63
Reviewed-on: https://go-review.googlesource.com/c/go/+/255999
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Keith Randall <khr@golang.org>
2020-09-18 22:09:45 +00:00
Keith Randall 8925290cf7 reflect: use zero buffer to back the Value returned by Zero
In the common case (<1KB types), no allocation is required
by reflect.Zero.

Also use memclr instead of memmove in Set when the source
is known to be zero.

Fixes #33136

Change-Id: Ic66871930fbb53328032e587153ebd12995ccf55
Reviewed-on: https://go-review.googlesource.com/c/go/+/192331
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
2020-09-18 21:47:28 +00:00
Keith Randall d91d0762c7 runtime/debug: provide Addr method for errors from SetPanicOnFault
When we're building a panic that's triggered by a memory fault when
SetPanicOnFault has been called, include an Addr method. This
method reports the address at which the fault occurred.

Fixes #37023

RELNOTE=yes

Change-Id: Idff144587d6b75070fdc861a36efec76f4ec7384
Reviewed-on: https://go-review.googlesource.com/c/go/+/249677
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Keith Randall <khr@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2020-09-18 17:21:06 +00:00
Cherry Zhang 1ee30d25c5 runtime: correctly log stderr in TestFakeTime
Change-Id: Iaf122ce7a8b8fb431199399aeed67b128a34d20b
Reviewed-on: https://go-review.googlesource.com/c/go/+/255720
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-18 01:02:39 +00:00
Michael Anthony Knyszek 10dfb1dd3d runtime: actually fix locking in BenchmarkMSpanCountAlloc
I just submitted CL 255297 which mostly fixed this problem, but totally
forgot to actually acquire/release the heap lock. Oops.

Updates #41391.

Change-Id: I45b42f20a9fc765c4de52476db3654d4bfe9feb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/255298
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-16 17:36:58 +00:00
Michael Anthony Knyszek 2ae2a94857 runtime: fix leak and locking in BenchmarkMSpanCountAlloc
CL 249917 made the mspan in MSpanCountAlloc no longer stack-allocated
(for good reason), but then allocated an mspan on each call and did not
free it, resulting in a leak. That allocation was also not protected by
the heap lock, which could lead to data corruption of mheap fields and
the spanalloc.

To fix this, export some functions to allocate/free dummy mspans from
spanalloc (with proper locking) and allocate just one up-front for the
benchmark, freeing it at the end. Then, update MSpanCountAlloc to accept
a dummy mspan.

Note that we need to allocate the dummy mspan up-front otherwise we
measure things like heap locking and fixalloc performance instead of
what we actually want to measure: how fast we can do a popcount on the
mark bits.

Fixes #41391.

Change-Id: If6629a6ec1ece639c7fb78532045837a8c872c04
Reviewed-on: https://go-review.googlesource.com/c/go/+/255297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
2020-09-16 17:07:38 +00:00
Martin Möhrmann 790fa1c546 cmd/compile: unify reflect, string and slice copy runtime functions
Use a common runtime slicecopy function to copy strings or slices
into slices. This deduplicates similar code previously used in
reflect.slicecopy and runtime.stringslicecopy.

Change-Id: I09572ff0647a9e12bb5c6989689ce1c43f16b7f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/254658
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-16 04:37:14 +00:00
Martin Möhrmann 14c7caae50 runtime: add 24 byte allocation size class
This CL introduces a 24 byte allocation size class which
fits 3 pointers on 64 bit and 6 pointers on 32 bit architectures.

Notably this new size class fits a slice header on 64 bit
architectures exactly while previously a 32 byte size class
would have been used for allocating a slice header on the heap.

The main complexity added with this CL is that heapBitsSetType
needs to handle objects that aren't 16-byte aligned but contain
more than a single pointer on 64-bit architectures.

Due to having a non 16 byte aligned size class on 32 bit a
h.shift of 2 is now possible which means a heap bitmap byte might
only be partially written. Due to this already having been
possible on 64 bit before the heap bitmap code only needed
minor adjustments for 32 bit doublecheck code paths.

Note that this CL changes the slice capacity allocated by append
for slice growth to a target capacity of 17 to 24 bytes.

On 64 bit architectures the capacity of the slice returned by
append([]byte{}, make([]byte, 24)...)) is 32 bytes before and
24 bytes after this CL. Depending on allocation patterns of the
specific Go program this can increase the number of total
alloctions as subsequent appends to the slice can trigger slice
growth earlier than before. On the other side if the slice is
never appended to again above its capacity this will lower heap
usage by 8 bytes.

This CL changes the set of size classes reported in the
runtime.MemStats.BySize array due to it being limited to a
total of 61 size classes. The new 24 byte size class is now
included and the 20480 byte size class is not included anymore.

Fixes #8885

name                      old time/op       new time/op       delta
Template                        196ms ± 3%        194ms ± 2%    ~     (p=0.247 n=10+10)
Unicode                        85.6ms ±16%       88.1ms ± 1%    ~     (p=0.165 n=10+10)
GoTypes                         673ms ± 2%        668ms ± 2%    ~     (p=0.258 n=9+9)
Compiler                        3.14s ± 6%        3.08s ± 1%    ~     (p=0.243 n=10+9)
SSA                             6.82s ± 1%        6.76s ± 1%  -0.87%  (p=0.006 n=9+10)
Flate                           128ms ± 7%        127ms ± 3%    ~     (p=0.739 n=10+10)
GoParser                        154ms ± 3%        153ms ± 4%    ~     (p=0.730 n=9+9)
Reflect                         404ms ± 1%        412ms ± 4%  +1.99%  (p=0.022 n=9+10)
Tar                             172ms ± 4%        170ms ± 4%    ~     (p=0.065 n=10+9)
XML                             231ms ± 4%        230ms ± 3%    ~     (p=0.912 n=10+10)
LinkCompiler                    341ms ± 1%        339ms ± 1%    ~     (p=0.243 n=9+10)
ExternalLinkCompiler            1.72s ± 1%        1.72s ± 1%    ~     (p=0.661 n=9+10)
LinkWithoutDebugCompiler        221ms ± 2%        221ms ± 2%    ~     (p=0.529 n=10+10)
StdCmd                          18.4s ± 3%        18.2s ± 1%    ~     (p=0.515 n=10+8)

name                      old user-time/op  new user-time/op  delta
Template                        238ms ± 4%        243ms ± 6%    ~     (p=0.661 n=9+10)
Unicode                         116ms ± 6%        113ms ± 3%  -3.37%  (p=0.035 n=9+10)
GoTypes                         854ms ± 2%        848ms ± 2%    ~     (p=0.604 n=9+10)
Compiler                        4.10s ± 1%        4.11s ± 1%    ~     (p=0.481 n=8+9)
SSA                             9.49s ± 1%        9.41s ± 1%  -0.92%  (p=0.001 n=9+10)
Flate                           149ms ± 6%        151ms ± 7%    ~     (p=0.481 n=10+10)
GoParser                        189ms ± 2%        190ms ± 2%    ~     (p=0.497 n=9+10)
Reflect                         511ms ± 2%        508ms ± 2%    ~     (p=0.211 n=9+10)
Tar                             215ms ± 4%        212ms ± 3%    ~     (p=0.105 n=10+10)
XML                             288ms ± 2%        288ms ± 2%    ~     (p=0.971 n=10+10)
LinkCompiler                    559ms ± 4%        557ms ± 1%    ~     (p=0.968 n=9+10)
ExternalLinkCompiler            1.78s ± 1%        1.77s ± 1%    ~     (p=0.055 n=8+10)
LinkWithoutDebugCompiler        245ms ± 3%        245ms ± 2%    ~     (p=0.684 n=10+10)

name                      old alloc/op      new alloc/op      delta
Template                       34.8MB ± 0%       34.4MB ± 0%  -0.95%  (p=0.000 n=9+10)
Unicode                        28.6MB ± 0%       28.3MB ± 0%  -0.95%  (p=0.000 n=10+10)
GoTypes                         115MB ± 0%        114MB ± 0%  -1.02%  (p=0.000 n=10+9)
Compiler                        554MB ± 0%        549MB ± 0%  -0.86%  (p=0.000 n=9+10)
SSA                            1.28GB ± 0%       1.27GB ± 0%  -0.83%  (p=0.000 n=10+10)
Flate                          21.8MB ± 0%       21.6MB ± 0%  -0.87%  (p=0.000 n=8+10)
GoParser                       26.7MB ± 0%       26.4MB ± 0%  -0.97%  (p=0.000 n=10+9)
Reflect                        75.0MB ± 0%       74.1MB ± 0%  -1.18%  (p=0.000 n=10+10)
Tar                            32.6MB ± 0%       32.3MB ± 0%  -0.94%  (p=0.000 n=10+7)
XML                            41.5MB ± 0%       41.2MB ± 0%  -0.90%  (p=0.000 n=10+8)
LinkCompiler                    105MB ± 0%        104MB ± 0%  -0.94%  (p=0.000 n=10+10)
ExternalLinkCompiler            153MB ± 0%        152MB ± 0%  -0.69%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler       63.7MB ± 0%       63.6MB ± 0%  -0.13%  (p=0.000 n=10+10)

name                      old allocs/op     new allocs/op     delta
Template                         336k ± 0%         336k ± 0%  +0.02%  (p=0.002 n=10+10)
Unicode                          332k ± 0%         332k ± 0%    ~     (p=0.447 n=10+10)
GoTypes                         1.16M ± 0%        1.16M ± 0%  +0.01%  (p=0.001 n=10+10)
Compiler                        4.92M ± 0%        4.92M ± 0%  +0.01%  (p=0.000 n=10+10)
SSA                             11.9M ± 0%        11.9M ± 0%  +0.02%  (p=0.000 n=9+10)
Flate                            214k ± 0%         214k ± 0%  +0.02%  (p=0.032 n=10+8)
GoParser                         270k ± 0%         270k ± 0%  +0.02%  (p=0.004 n=10+9)
Reflect                          877k ± 0%         877k ± 0%  +0.01%  (p=0.000 n=10+10)
Tar                              313k ± 0%         313k ± 0%    ~     (p=0.075 n=9+10)
XML                              387k ± 0%         387k ± 0%  +0.02%  (p=0.007 n=10+10)
LinkCompiler                     455k ± 0%         456k ± 0%  +0.08%  (p=0.000 n=10+9)
ExternalLinkCompiler             670k ± 0%         671k ± 0%  +0.06%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler         113k ± 0%         113k ± 0%    ~     (p=0.149 n=10+10)

name                      old maxRSS/op     new maxRSS/op     delta
Template                        34.1M ± 1%        34.1M ± 1%    ~     (p=0.853 n=10+10)
Unicode                         35.1M ± 1%        34.6M ± 1%  -1.43%  (p=0.000 n=10+10)
GoTypes                         72.8M ± 3%        73.3M ± 2%    ~     (p=0.724 n=10+10)
Compiler                         288M ± 3%         295M ± 4%    ~     (p=0.393 n=10+10)
SSA                              630M ± 1%         622M ± 1%  -1.18%  (p=0.001 n=10+10)
Flate                           26.0M ± 1%        26.2M ± 2%    ~     (p=0.493 n=10+10)
GoParser                        28.6M ± 1%        28.5M ± 2%    ~     (p=0.256 n=10+10)
Reflect                         55.5M ± 2%        55.4M ± 1%    ~     (p=0.436 n=10+10)
Tar                             33.0M ± 1%        32.8M ± 2%    ~     (p=0.075 n=10+10)
XML                             38.7M ± 1%        39.0M ± 1%    ~     (p=0.053 n=9+10)
LinkCompiler                     164M ± 1%         164M ± 1%  -0.27%  (p=0.029 n=10+10)
ExternalLinkCompiler             174M ± 0%         173M ± 0%  -0.33%  (p=0.002 n=9+10)
LinkWithoutDebugCompiler         137M ± 0%         136M ± 2%    ~     (p=0.825 n=9+10)

Change-Id: I9ecf2a10024513abef8fbfbe519e44e0b29b6167
Reviewed-on: https://go-review.googlesource.com/c/go/+/242258
Trust: Martin Möhrmann <moehrmann@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-14 19:21:56 +00:00
Cherry Zhang d277a36123 runtime: remove darwin/arm specifc code
That port is gone.

Change-Id: I212d435e290d1890d6cd5531be98bb692650595e
Reviewed-on: https://go-review.googlesource.com/c/go/+/254077
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-11 02:58:36 +00:00
ipriver 8098dbb30e runtime: update docs for GOMAXPROCS
https://github.com/golang/go/blob/master/doc/effective_go.html#L3211 is used to update the docs comment for `GOMAXPROCS` function.

Fixes #41275

Change-Id: I39f58e93a267c6e9f3ac6638ed51acbe5284ada2
GitHub-Last-Rev: e45c8ac587
GitHub-Pull-Request: golang/go#41276
Reviewed-on: https://go-review.googlesource.com/c/go/+/253537
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-10 14:44:25 +00:00
Ian Lance Taylor 2556eb76c8 runtime: ignore SIGPROF if profiling disable for thread
This avoids a deadlock on prof.signalLock between setcpuprofilerate
and cpuprof.add if a SIGPROF is delivered to the thread between the
call to setThreadCPUProfiler and acquiring prof.signalLock.

Fixes #41014

Change-Id: Ie825e8594f93a19fb1a6320ed640f4e631553596
Reviewed-on: https://go-review.googlesource.com/c/go/+/253758
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-09 18:04:10 +00:00
Michael Anthony Knyszek 34835df048 runtime: fix ReadMemStatsSlow's and CheckScavengedBits' chunk iteration
Both ReadMemStatsSlow and CheckScavengedBits iterate over the page
allocator's chunks but don't actually check if they exist. During the
development process the chunks index became sparse, so now this was a
possibility. If the runtime tests' heap is sparse we might end up
segfaulting in either one of these functions, though this will generally
be very rare.

The pattern here to return nil for a nonexistent chunk is also useful
elsewhere, so this change introduces tryChunkOf which won't throw, but
might return nil. It also updates the documentation of chunkOf.

Fixes #41296.

Change-Id: Id5ae0ca3234480de1724fdf2e3677eeedcf76fa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/253777
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-09 17:48:56 +00:00
chainhelen 565ad134c9 runtime: make PCDATA_RegMapUnsafe more clear and remove magic number
Change-Id: Ibf3ee755c3fbec03a9396840dc92ce148c49d9f7
GitHub-Last-Rev: 945d8aaa13
GitHub-Pull-Request: golang/go#41262
Reviewed-on: https://go-review.googlesource.com/c/go/+/253377
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-08 15:07:20 +00:00
Martin Möhrmann f98f3b0c30 runtime: remove outdated comment in mkduff.go about usage of STOSQ
Change-Id: I71966cc5def4615d64876165872e5e7f2956b270
Reviewed-on: https://go-review.googlesource.com/c/go/+/253397
Run-TryBot: Martin Möhrmann <martisch@uos.de>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-07 19:01:17 +00:00
Cuong Manh Le 1a119edd8b runtime: rotate map key seed on clearing up maps
Same thing as CL 253020 did for map clear idiom.

name                        old time/op  new time/op  delta
MapDelete/Int32/100-12      30.0ns ± 1%  30.7ns ± 3%   ~     (p=0.400 n=3+3)
MapDelete/Int32/1000-12     26.6ns ± 2%  28.1ns ± 3%   ~     (p=0.100 n=3+3)
MapDelete/Int32/10000-12    28.6ns ± 1%  31.9ns ± 1%   ~     (p=0.100 n=3+3)
MapDelete/Int64/100-12      30.2ns ± 0%  32.1ns ± 3%   ~     (p=0.100 n=3+3)
MapDelete/Int64/1000-12     26.5ns ± 1%  27.5ns ± 3%   ~     (p=0.100 n=3+3)
MapDelete/Int64/10000-12    29.6ns ± 1%  29.3ns ± 1%   ~     (p=0.300 n=3+3)
MapDelete/Str/100-12        19.5ns ± 3%  19.6ns ± 2%   ~     (p=0.800 n=3+3)
MapDelete/Str/1000-12       31.6ns ± 1%  31.4ns ± 1%   ~     (p=0.500 n=3+3)
MapDelete/Str/10000-12      37.8ns ± 1%  37.1ns ± 1%   ~     (p=0.100 n=3+3)
MapDelete/Pointer/100-12    15.9ns ± 1%  16.8ns ± 9%   ~     (p=0.200 n=3+3)
MapDelete/Pointer/1000-12   26.9ns ± 1%  26.2ns ± 2%   ~     (p=0.200 n=3+3)
MapDelete/Pointer/10000-12  30.6ns ± 1%  30.7ns ± 4%   ~     (p=0.700 n=3+3)

Fixes #25237

Change-Id: I353cf44a2f6158549f0ef563d867f0844fec7095
Reviewed-on: https://go-review.googlesource.com/c/go/+/252940
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-07 02:46:24 +00:00
Benjamin Barenblat 93810ac1f4 runtime: opportunistically rotate map key seed
When clearing a map, reinitialize the hash seed with random data. This
makes it more difficult for attackers to trigger pathological
performance via repeated hash collisions.

The extra reinitialization causes no statistically significant slowdown:

name                              old time/op  new time/op  delta
GoMapClear/Reflexive/1-12         18.3ns ± 0%  20.0ns ± 0%   ~             (p=1.000 n=1+1)
GoMapClear/Reflexive/10-12        18.2ns ± 0%  19.8ns ± 0%   ~             (p=1.000 n=1+1)
GoMapClear/Reflexive/100-12       44.6ns ± 0%  46.1ns ± 0%   ~             (p=1.000 n=1+1)
GoMapClear/Reflexive/1000-12       592ns ± 0%   592ns ± 0%   ~     (all samples are equal)
GoMapClear/Reflexive/10000-12     3.88µs ± 0%  3.88µs ± 0%   ~             (p=1.000 n=1+1)
GoMapClear/NonReflexive/1-12      62.7ns ± 0%  63.9ns ± 0%   ~             (p=1.000 n=1+1)
GoMapClear/NonReflexive/10-12     75.0ns ± 0%  76.1ns ± 0%   ~             (p=1.000 n=1+1)
GoMapClear/NonReflexive/100-12     203ns ± 0%   206ns ± 0%   ~             (p=1.000 n=1+1)
GoMapClear/NonReflexive/1000-12   2.33µs ± 0%  2.33µs ± 0%   ~     (all samples are equal)
GoMapClear/NonReflexive/10000-12  18.1µs ± 0%  18.1µs ± 0%   ~             (p=1.000 n=1+1)

Fixes #25237

Change-Id: I629a79dd7c562ba18bd94159673c3b9b653da643
Reviewed-on: https://go-review.googlesource.com/c/go/+/253020
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-03 22:08:58 +00:00
Heisenberg e6583dc953 runtime: add file copyright header declaration
Some files have no copyright notice.
The copyright time is the earliest modification record of the file.

Change-Id: I5698bae16b6b73543e074415877a03348f792951
Reviewed-on: https://go-review.googlesource.com/c/go/+/246378
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-02 10:09:21 +00:00
chainhelen b246c0e12f runtime: fix panic if newstack at runtime.acquireLockRank
Process may crash becaues acquireLockRank and releaseLockRank may
be called in nosplit context. With optimizations and inlining
disabled, these functions won't get inlined or have their morestack
calls eliminated.
Nosplit is not strictly required for lockWithRank, unlockWithRank
and lockWithRankMayAcquire, just keep consistency with lockrank_on.go
here.

Fixes #40843

Change-Id: I5824119f98a1da66d767cdb9a60dffe768f13c81
GitHub-Last-Rev: 38fd3ccf6e
GitHub-Pull-Request: golang/go#40844
Reviewed-on: https://go-review.googlesource.com/c/go/+/248878
Reviewed-by: Dan Scales <danscales@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-01 23:19:30 +00:00
Heisenberg 4d89b3231d runtime: remove remnants of signal stack workaround
Updates #35979

Change-Id: Ic3a6e1b5e9d544979a3c8d909a36a55efa3b9c9d
Reviewed-on: https://go-review.googlesource.com/c/go/+/251757
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-09-01 13:23:55 +00:00
Cuong Manh Le 27a30186ab cmd/compile,runtime: skip zero'ing order array for select statements
The order array was zero initialized by the compiler, but ends up being
overwritten by the runtime anyway.

So let the runtime takes full responsibility for initializing, save us
one instruction per select.

Fixes #40399

Change-Id: Iec1eca27ad7180d4fcb3cc9ef97348206b7fe6b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/251517
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-08-29 08:02:52 +00:00
Keith Randall 5c2c6d3fbf runtime: framepointers are no longer an experiment - hard code them
I think they are no longer experimental status. Might as well promote
them to permanent.

Change-Id: Id1259601b3dd2061dd60df86ee48080bfb575d2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/249857
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2020-08-27 21:15:47 +00:00
Michael Pratt 2013f70256 runtime: add lock partial order edge (fin -> wbufSpans)
runfinq may have write barriers, thus it may need to take wbufSpans on
any write.

Fixes #41021

Change-Id: Ib69e20994b5d7d1526ad53d6ddb5e2e83bf2ed00
Reviewed-on: https://go-review.googlesource.com/c/go/+/250464
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2020-08-27 19:42:42 +00:00
Keith Randall d9a6bdf7ef cmd/compile: don't allow go:notinheap on the heap or stack
Right now we just prevent such types from being on the heap. This CL
makes it so they cannot appear on the stack either. The distinction
between heap and stack is pretty vague at the language level (e.g. it
is affected by -N), and we don't need the flexibility anyway.

Once go:notinheap types cannot be in either place, we don't need to
consider pointers to such types to be pointers, at least according to
the garbage collector and stack copying. (This is the big win of this
CL, in my opinion.)

The distinction between HasPointers and HasHeapPointer no longer
exists. There is only HasPointers.

This CL is cleanup before possible use of go:notinheap to fix #40954.

Update #13386

Change-Id: Ibd895aadf001c0385078a6d4809c3f374991231a
Reviewed-on: https://go-review.googlesource.com/c/go/+/249917
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2020-08-25 01:46:05 +00:00
Cuong Manh Le aa49a0b372 runtime: add test for StorepNoWB param leaking
CL 249962 added wasm StorepNoWB implementation in assembly, it's now
like all other architectures. This CL adds a general test that the
second param of StorepNoWB must be force to escape.

Fixes #40975

Change-Id: I1eccc7e50a3ec742a1912d65f25b15f9f5ad9241
Reviewed-on: https://go-review.googlesource.com/c/go/+/249761
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-08-24 03:55:18 +00:00
Keith Randall c0602603b2 runtime: implement StorepNoWB for wasm in assembly
The second argument of StorepNoWB must be forced to escape.
The current Go code does not explicitly enforce that property.
By implementing in assembly, and not using go:noescape, we
force the issue.

Test is in CL 249761. Issue #40975.

This CL is needed for CL 249917, which changes how go:notinheap
works and breaks the previous StorepNoWB wasm code.

I checked for other possible errors like this. This is the only
go:notinheap that isn't in the runtime itself.

Change-Id: I43400a806662655727c4a3baa8902b63bdc9fa57
Reviewed-on: https://go-review.googlesource.com/c/go/+/249962
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-08-23 20:18:56 +00:00
Matthew Dempsky e94544cf01 cmd/compile: fix checkptr handling of &^
checkptr has code to recognize &^ expressions, but it didn't take into
account that "p &^ x" gets rewritten to "p & ^x" during walk, which
resulted in false positive diagnostics.

This CL changes walkexpr to mark OANDNOT expressions with Implicit
when they're rewritten to OAND, so that walkCheckPtrArithmetic can
still recognize them later.

It would be slightly more idiomatic to instead mark the OBITNOT
expression as Implicit (as it's a compiler-generated Node), but the
OBITNOT expression might get constant folded. It's not worth the extra
complexity/subtlety of relying on n.Right.Orig, so we set Implicit on
the OAND node instead.

To atone for this transgression, I add documentation for nodeImplicit.

Fixes #40917.

Change-Id: I386304171ad299c530e151e5924f179e9a5fd5b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/249477
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2020-08-20 17:48:29 +00:00
Ian Lance Taylor 6b420169d7 os, internal/poll: loop on EINTR for all file syscalls
When using a FUSE file system, any system call that touches the file
system can return EINTR.

Fixes #40846

Change-Id: I25d32da22cec08dea81ab297291a85ad72db2df7
Reviewed-on: https://go-review.googlesource.com/c/go/+/249178
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-08-19 21:49:56 +00:00
Ian Lance Taylor bd519d0c87 runtime: don't call setitimer for each thread
Previously, on Unix systems, when the profiler was enabled or disabled,
we called setitimer once per thread. With this change we instead call
it once per process.

Change-Id: I90f0189b562e11232816390dc7d55ed154bd836d
Reviewed-on: https://go-review.googlesource.com/c/go/+/240003
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-08-19 03:49:06 +00:00
cui 0941fc3f9f runtime: reduce syscall when call runtime.clone
Change-Id: I3ea398fd86aae4c86557dd6fff65d90a6f756890
GitHub-Last-Rev: 4c295388f7
GitHub-Pull-Request: golang/go#40392
Reviewed-on: https://go-review.googlesource.com/c/go/+/244626
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-19 00:02:38 +00:00
Matthew Dempsky fe23ba4a14 runtime: eliminate scase.kind field
Currently, we include a "kind" field on scase to distinguish the three
kinds of cases in a select statement: sends, receives, and defaults.

This commit removes by kind field by instead arranging for the
compiler to always place sends before receives, and to provide their
counts separately. It also passes an explicit "block bool" parameter
to avoid needing to include a default case in the array.

It's safe to shuffle cases like this because the runtime will
randomize the order they're polled in anyway.

Fixes #40410.

Change-Id: Iaeaed4cf7bddd576d78f2c863bd91a03a5c82df2
Reviewed-on: https://go-review.googlesource.com/c/go/+/245125
Reviewed-by: Keith Randall <khr@golang.org>
2020-08-18 20:06:33 +00:00
Matthew Dempsky d36bc7d78a runtime: split PCs out of scase
Per-case PCs are only needed for race detector builds, so this allows
skipping allocating stack space for them for non-race builds.

It's possible to arrange the PCs and order arrays consecutively in
memory so that we could just reuse the order0 pointer to identify
both. However, there's more risk of that silently going wrong, so this
commit passes them as separate arguments for now. We can revisit this
in the future.

Updates #40410.

Change-Id: I8468bc25749e559891cb0cb007d1cc4a40fdd0f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/245124
Reviewed-by: Keith Randall <khr@golang.org>
2020-08-18 20:06:16 +00:00
Matthew Dempsky 8a984e8e3f runtime: omit nil-channel cases from selectgo's orders
Currently, selectgo does an initial pass over the cases array to look
for entries with nil channels, so they can be easily recognized and
skipped later on. But this still involves actually visiting the cases.

This commit changes selectgo to omit cases with nil channels when
constructing pollorder, so that they'll be skipped over entirely later
on. It also checks for caseDefault up front, which will facilitate
changing it to use a "block bool" parameter instead.

Updates #40410.

Change-Id: Icaebcb8f08df03cc33b6d8087616fb5585f7fedd
Reviewed-on: https://go-review.googlesource.com/c/go/+/245123
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-08-18 20:05:47 +00:00
Matthew Dempsky 78a1064d5d runtime: remove scase.releasetime field
selectgo will report at most one block event, so there's no need to
keep a releasetime for every select case. It suffices to simply track
the releasetime of the case responsible for the wakeup.

Updates #40410.

Change-Id: I72679cd43dde80d7e6dbab21a78952a4372d1e79
Reviewed-on: https://go-review.googlesource.com/c/go/+/245122
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-08-18 20:05:41 +00:00
Matthew Dempsky 30a68bfb80 runtime: add "success" field to sudog
The current wakeup protocol for channel communications is that the
second goroutine sets gp.param to the sudog when a value is
successfully communicated over the channel, and to nil when the wakeup
is due to closing the channel.

Setting nil to indicate channel closure works okay for chansend and
chanrecv, because they're only communicating with one channel, so they
know it must be the channel that was closed. However, it means
selectgo has to re-poll all of the channels to figure out which one
was closed.

This commit adds a "success" field to sudog, and changes the wakeup
protocol to always set gp.param to sg, and to use sg.success to
indicate successful communication vs channel closure.

While here, this also reorganizes the chansend code slightly so that
the sudog is still released to the pool if the send blocks and then is
awoken because the channel closed.

Updates #40410.

Change-Id: I6cd9a20ebf9febe370a15af1b8afe24c5539efc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/245019
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-08-18 20:05:33 +00:00
Cherry Zhang b58d297416 cmd/compile, runtime: mark R12 clobbered for write barrier call on PPC64
When external linking, for large binaries, the external linker
may insert a trampoline for the write barrier call, which looks

0000000005a98cc8 <__long_branch_runtime.gcWriteBarrier>:
 5a98cc8:       86 01 82 3d     addis   r12,r2,390
 5a98ccc:       d8 bd 8c e9     ld      r12,-16936(r12)
 5a98cd0:       a6 03 89 7d     mtctr   r12
 5a98cd4:       20 04 80 4e     bctr

It clobbers R12 (and CTR, which is never live across a call).

As at compile time we don't know whether the binary is big and
what link mode will be used, I think we need to mark R12 as
clobbered for write barrier call. For extra safety (future-proof)
we mark caller-saved register that cannot be used for function
arguments, which includes R11, as potentially clobbered as well.

Fixes #40851.

Change-Id: Iedd901c5072f1127cc59b0a48cfeb4aaec81b519
Reviewed-on: https://go-review.googlesource.com/c/go/+/248917
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-08-18 15:50:52 +00:00
Tobias Klauser 4149493443 runtime: move startupRandomData declaration to os_linux.go
startupRandomData is only used in sysauxv and getRandomData on linux,
thus move it closer to where it is used. Also adjust its godoc comment.

Change-Id: Ice51d579ec33436adbfdf247caf4ba00bae865e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/248761
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-18 08:47:48 +00:00
Cholerae Hu 613388315e runtime: reduce critical path in injectglist
Change-Id: Ia3fb30ac9add39c803f11f69d967c6604fdeacf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/233217
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-18 04:22:33 +00:00
lihaowei 7fbd8c75c6 all: fix spelling mistakes
Change-Id: I7d512281d8442d306594b57b5deaecd132b5ea9e
GitHub-Last-Rev: 251e1d6857
GitHub-Pull-Request: golang/go#40793
Reviewed-on: https://go-review.googlesource.com/c/go/+/248441
Reviewed-by: Dave Cheney <dave@cheney.net>
2020-08-18 03:28:52 +00:00
Keith Randall 8b8f926fc3 runtime: bit parallel implementation of findBitRange64
Use a bit-parallel implementation of findBitRange64.
It uses a repeated shift-'N-and technique to erase all the
free marks that are too small for the allocation.

Also some small improvements to find1.

name                                             old time/op  new time/op  delta
FindBitRange64/Pattern00Size2-16                 4.19ns ± 0%  2.26ns ± 0%   -46.04%  (p=0.000 n=10+8)
FindBitRange64/Pattern00Size8-16                 4.19ns ± 0%  2.12ns ± 0%   -49.35%  (p=0.000 n=9+10)
FindBitRange64/Pattern00Size32-16                4.20ns ± 0%  2.12ns ± 0%   -49.49%  (p=0.000 n=10+8)
FindBitRange64/PatternFFFFFFFFFFFFFFFFSize2-16   2.13ns ± 0%  2.27ns ± 0%    +6.28%  (p=0.000 n=10+10)
FindBitRange64/PatternFFFFFFFFFFFFFFFFSize8-16   2.13ns ± 0%  4.46ns ± 0%  +109.39%  (p=0.000 n=10+9)
FindBitRange64/PatternFFFFFFFFFFFFFFFFSize32-16  2.13ns ± 1%  5.58ns ± 0%  +162.37%  (p=0.000 n=10+9)
FindBitRange64/PatternAASize2-16                 22.2ns ± 0%   2.3ns ± 0%   -89.82%  (p=0.000 n=9+8)
FindBitRange64/PatternAASize8-16                 22.2ns ± 0%   2.1ns ± 1%   -90.41%  (p=0.000 n=9+10)
FindBitRange64/PatternAASize32-16                22.2ns ± 0%   2.1ns ± 1%   -90.43%  (p=0.000 n=10+10)
FindBitRange64/PatternAAAAAAAAAAAAAAAASize2-16    156ns ± 1%     2ns ± 0%   -98.54%  (p=0.000 n=10+10)
FindBitRange64/PatternAAAAAAAAAAAAAAAASize8-16    155ns ± 1%     2ns ± 0%   -98.63%  (p=0.000 n=10+8)
FindBitRange64/PatternAAAAAAAAAAAAAAAASize32-16   155ns ± 0%     2ns ± 1%   -98.63%  (p=0.000 n=8+10)
FindBitRange64/Pattern80000000AAAAAAAASize2-16   81.2ns ± 0%   2.3ns ± 1%   -97.21%  (p=0.000 n=10+10)
FindBitRange64/Pattern80000000AAAAAAAASize8-16   81.1ns ± 0%   2.1ns ± 0%   -97.39%  (p=0.000 n=10+9)
FindBitRange64/Pattern80000000AAAAAAAASize32-16  81.1ns ± 0%   2.1ns ± 0%   -97.38%  (p=0.000 n=10+10)
FindBitRange64/PatternAAAAAAAA00000001Size2-16   76.8ns ± 1%   2.3ns ± 0%   -97.05%  (p=0.000 n=10+10)
FindBitRange64/PatternAAAAAAAA00000001Size8-16   76.6ns ± 0%   2.1ns ± 0%   -97.23%  (p=0.000 n=8+10)
FindBitRange64/PatternAAAAAAAA00000001Size32-16  76.7ns ± 0%   2.1ns ± 0%   -97.23%  (p=0.000 n=9+9)
FindBitRange64/PatternBBBBBBBBBBBBBBBBSize2-16   2.13ns ± 0%  2.27ns ± 0%    +6.57%  (p=0.000 n=8+8)
FindBitRange64/PatternBBBBBBBBBBBBBBBBSize8-16   76.7ns ± 0%   2.9ns ± 0%   -96.20%  (p=0.000 n=9+10)
FindBitRange64/PatternBBBBBBBBBBBBBBBBSize32-16  76.7ns ± 0%   2.9ns ± 0%   -96.20%  (p=0.000 n=10+10)
FindBitRange64/Pattern80000000BBBBBBBBSize2-16   2.12ns ± 0%  2.27ns ± 1%    +6.74%  (p=0.000 n=10+10)
FindBitRange64/Pattern80000000BBBBBBBBSize8-16   44.8ns ± 0%   2.9ns ± 0%   -93.49%  (p=0.000 n=9+10)
FindBitRange64/Pattern80000000BBBBBBBBSize32-16  44.9ns ± 0%   2.9ns ± 0%   -93.49%  (p=0.000 n=10+8)
FindBitRange64/PatternBBBBBBBB00000001Size2-16   4.20ns ± 1%  2.27ns ± 1%   -46.02%  (p=0.000 n=10+10)
FindBitRange64/PatternBBBBBBBB00000001Size8-16   44.9ns ± 0%   2.9ns ± 1%   -93.51%  (p=0.000 n=10+9)
FindBitRange64/PatternBBBBBBBB00000001Size32-16  44.9ns ± 0%   2.9ns ± 0%   -93.51%  (p=0.000 n=10+9)
FindBitRange64/PatternCCCCCCCCCCCCCCCCSize2-16   4.19ns ± 0%  2.26ns ± 0%   -46.10%  (p=0.000 n=10+10)
FindBitRange64/PatternCCCCCCCCCCCCCCCCSize8-16   76.5ns ± 0%   2.9ns ± 0%   -96.19%  (p=0.000 n=8+7)
FindBitRange64/PatternCCCCCCCCCCCCCCCCSize32-16  76.5ns ± 0%   2.9ns ± 0%   -96.19%  (p=0.000 n=10+8)
FindBitRange64/Pattern4444444444444444Size2-16   76.4ns ± 0%   2.3ns ± 0%   -97.04%  (p=0.000 n=8+10)
FindBitRange64/Pattern4444444444444444Size8-16   76.5ns ± 0%   2.1ns ± 0%   -97.23%  (p=0.000 n=9+10)
FindBitRange64/Pattern4444444444444444Size32-16  76.5ns ± 0%   2.1ns ± 0%   -97.23%  (p=0.000 n=8+10)
FindBitRange64/Pattern4040404040404040Size2-16   40.3ns ± 0%   2.3ns ± 0%   -94.38%  (p=0.000 n=7+10)
FindBitRange64/Pattern4040404040404040Size8-16   40.2ns ± 0%   2.1ns ± 0%   -94.75%  (p=0.000 n=10+10)
FindBitRange64/Pattern4040404040404040Size32-16  40.2ns ± 0%   2.1ns ± 0%   -94.76%  (p=0.000 n=10+6)
FindBitRange64/Pattern4000400040004000Size2-16   22.2ns ± 0%   2.2ns ± 0%   -89.86%  (p=0.001 n=8+9)
FindBitRange64/Pattern4000400040004000Size8-16   22.2ns ± 0%   2.1ns ± 0%   -90.52%  (p=0.000 n=8+10)
FindBitRange64/Pattern4000400040004000Size32-16  22.2ns ± 1%   2.1ns ± 0%   -90.50%  (p=0.000 n=10+10)

The cases that slow down aren't really that slow, and those inputs
never actually occur (there's a short circuit before the call to
findBitRange64 for that case).

Change-Id: I50fae62915098032d8ce7fa57ef29eee9deb01ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/241279
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-08-17 22:00:17 +00:00
Keith Randall 4e5ed83e8d runtime: use bit-parallel operations to compute heap bit summaries
The new implementation is much faster in all cases.

name                                             old time/op  new time/op  delta
PallocBitsSummarize/Unpacked00-16                 142ns ± 1%     7ns ± 2%  -94.75%  (p=0.000 n=10+9)
PallocBitsSummarize/UnpackedFFFFFFFFFFFFFFFF-16   172ns ± 0%    24ns ± 0%  -86.02%  (p=0.000 n=9+9)
PallocBitsSummarize/UnpackedAA-16                 145ns ± 0%    32ns ± 0%  -78.16%  (p=0.000 n=8+10)
PallocBitsSummarize/UnpackedAAAAAAAAAAAAAAAA-16   172ns ± 0%    33ns ± 0%  -80.95%  (p=0.000 n=9+9)
PallocBitsSummarize/Unpacked80000000AAAAAAAA-16   162ns ± 1%    60ns ± 0%  -62.69%  (p=0.000 n=10+9)
PallocBitsSummarize/UnpackedAAAAAAAA00000001-16   163ns ± 0%    68ns ± 1%  -58.47%  (p=0.000 n=8+10)
PallocBitsSummarize/UnpackedBBBBBBBBBBBBBBBB-16   172ns ± 0%    35ns ± 0%  -79.70%  (p=0.000 n=9+9)
PallocBitsSummarize/Unpacked80000000BBBBBBBB-16   161ns ± 0%    63ns ± 0%  -60.61%  (p=0.000 n=8+10)
PallocBitsSummarize/UnpackedBBBBBBBB00000001-16   163ns ± 0%    60ns ± 0%  -63.14%  (p=0.000 n=9+10)
PallocBitsSummarize/UnpackedCCCCCCCCCCCCCCCC-16   172ns ± 0%    39ns ± 0%  -77.41%  (p=0.000 n=7+10)
PallocBitsSummarize/Unpacked4444444444444444-16   172ns ± 0%    39ns ± 0%  -77.42%  (p=0.000 n=7+10)
PallocBitsSummarize/Unpacked4040404040404040-16   173ns ± 2%    51ns ± 1%  -70.55%  (p=0.000 n=10+10)
PallocBitsSummarize/Unpacked4000400040004000-16   160ns ± 1%    53ns ± 0%  -66.78%  (p=0.000 n=10+10)
PallocBitsSummarize/Unpacked1000404044CCAAFF-16   169ns ± 1%    59ns ± 1%  -65.28%  (p=0.000 n=10+10)

Change-Id: I94daa645b76a9cf9c93edeb2058d7132216fcb72
Reviewed-on: https://go-review.googlesource.com/c/go/+/240900
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-08-17 21:58:31 +00:00
Keith Randall 88c094c96a runtime: print faulting instruction on a SIGFPE
Just like SIGILL, it might be useful to see what the instruction
that generated the SIGFPE is.

Update #39816

Change-Id: I8b2ff692998f0b770289339537dceab96b09d1ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/239999
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-17 21:57:05 +00:00
liu-xuewen ba97be4b58 runtime: remove tracebackinit and unused skipPC
CL [152537](https://go-review.googlesource.com/c/go/+/152537/) changed the way inlined frames are represented in tracebacks to no longer use skipPC

Change-Id: I42386fdcc5cf72f3c122e789b6af9cbd0c6bed4b
GitHub-Last-Rev: 79c26dcd53
GitHub-Pull-Request: golang/go#39829
Reviewed-on: https://go-review.googlesource.com/c/go/+/239701
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-08-17 21:05:19 +00:00
Heisenberg a61a3c378d runtime: use the CBZ instruction in the assembler
Use CBZ to replace the comparison and branch of arm64 and the zero instruction in the assembly file.

Change-Id: Id6c03e9af13aadafc3ad3953f82d2ffa29c12926
Reviewed-on: https://go-review.googlesource.com/c/go/+/237497
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-17 20:59:59 +00:00
Michael Anthony Knyszek e6d0bd2b89 runtime: clean up old mcentral code
This change deletes the old mcentral implementation from the code base
and the newMCentralImpl feature flag along with it.

Updates #37487.

Change-Id: Ibca8f722665f0865051f649ffe699cbdbfdcfcf2
Reviewed-on: https://go-review.googlesource.com/c/go/+/221184
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2020-08-17 20:06:49 +00:00