go/src/sync
Paul E. Murphy 15ead857db cmd/compiler,cmd/go,sync: add internal {LoadAcq,StoreRel}64 on ppc64
Add an internal atomic intrinsic for load with acquire semantics
(extending LoadAcq to 64b) and add LoadAcquintptr for internal
use within the sync package.  For other arches, this remaps to the
appropriate atomic.Load{,64} intrinsic which should not alter code
generation.

Similarly, add StoreRel{uintptr,64} for consistency, and inline.

Finally, add an exception to allow sync to directly use the
runtime/internal/atomic package which avoids more convoluted
workarounds (contributed by Lynn Boger).

In an extreme example, sync.(*Pool).pin consumes 20% of wall time
during fmt tests.  This is reduced to 5% on ppc64le/power9.

From the fmt benchmarks on ppc64le:

name                           old time/op  new time/op  delta
SprintfPadding                  468ns ± 0%   451ns ± 0%   -3.63%
SprintfEmpty                   73.3ns ± 0%  51.9ns ± 0%  -29.20%
SprintfString                   135ns ± 0%   122ns ± 0%   -9.63%
SprintfTruncateString           232ns ± 0%   214ns ± 0%   -7.76%
SprintfTruncateBytes            216ns ± 0%   202ns ± 0%   -6.48%
SprintfSlowParsingPath          162ns ± 0%   142ns ± 0%  -12.35%
SprintfQuoteString             1.00µs ± 0%  0.99µs ± 0%   -1.39%
SprintfInt                      117ns ± 0%   104ns ± 0%  -11.11%
SprintfIntInt                   190ns ± 0%   175ns ± 0%   -7.89%
SprintfPrefixedInt              232ns ± 0%   212ns ± 0%   -8.62%
SprintfFloat                    270ns ± 0%   255ns ± 0%   -5.56%
SprintfComplex                 1.01µs ± 0%  0.99µs ± 0%   -1.68%
SprintfBoolean                  127ns ± 0%   111ns ± 0%  -12.60%
SprintfHexString                220ns ± 0%   198ns ± 0%  -10.00%
SprintfHexBytes                 261ns ± 0%   252ns ± 0%   -3.45%
SprintfBytes                    600ns ± 0%   590ns ± 0%   -1.67%
SprintfStringer                 684ns ± 0%   658ns ± 0%   -3.80%
SprintfStructure               2.57µs ± 0%  2.57µs ± 0%   -0.12%
ManyArgs                        669ns ± 0%   646ns ± 0%   -3.44%
FprintInt                       140ns ± 0%   136ns ± 0%   -2.86%
FprintfBytes                    184ns ± 0%   181ns ± 0%   -1.63%
FprintIntNoAlloc                140ns ± 0%   136ns ± 0%   -2.86%
ScanInts                        929µs ± 0%   921µs ± 0%   -0.79%
ScanRecursiveInt                122ms ± 0%   121ms ± 0%   -0.11%
ScanRecursiveIntReaderWrapper   122ms ± 0%   122ms ± 0%   -0.18%

Change-Id: I4d66780261b57b06ef600229e475462e7313f0d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/253748
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-21 14:34:44 +00:00
..
atomic runtime/internal/atomic: panic nicely on unaligned 64-bit atomics 2020-10-16 17:31:17 +00:00
cond.go cmd/vet,sync: check lock values more precisely 2018-07-14 06:48:21 +00:00
cond_test.go all: add empty line between copyright header and package clause 2020-08-17 09:45:44 +00:00
example_pool_test.go
example_test.go
export_test.go sync: fix pool wrap-around test 2019-06-26 19:48:39 +00:00
map.go sync: delete dirty keys inside Map.LoadAndDelete 2020-08-24 20:05:41 +00:00
map_bench_test.go sync: add new Map method LoadAndDelete 2020-02-25 14:31:55 +00:00
map_reference_test.go sync: add new Map method LoadAndDelete 2020-02-25 14:31:55 +00:00
map_test.go sync: delete dirty keys inside Map.LoadAndDelete 2020-08-24 20:05:41 +00:00
mutex.go sync: yield to the waiter when unlocking a starving mutex 2019-11-09 19:31:32 +00:00
mutex_test.go sync: fix goroutine leak for when TestMutexFairness times out 2020-08-14 23:04:16 +00:00
once.go sync: document implementation of Once.Do 2019-07-01 14:45:49 +00:00
once_test.go
pool.go cmd/compiler,cmd/go,sync: add internal {LoadAcq,StoreRel}64 on ppc64 2020-10-21 14:34:44 +00:00
pool_test.go sync: only check for successful PopHeads in long mode 2019-06-26 19:48:42 +00:00
poolqueue.go sync: fix typo in pooldequeue 2020-10-14 13:38:41 +00:00
runtime.go runtime: static lock ranking for the runtime (enabled by GOEXPERIMENT) 2020-04-07 21:51:03 +00:00
runtime2.go runtime, sync: add copyright headers to new files 2020-07-25 03:26:17 +00:00
runtime2_lockrank.go runtime, sync: add copyright headers to new files 2020-07-25 03:26:17 +00:00
runtime_sema_test.go sync: allow inlining the Mutex.Unlock fast path 2019-03-05 14:59:31 +00:00
rwmutex.go sync: allow inlining the RWMutex.RUnlock fast path 2019-03-09 16:34:17 +00:00
rwmutex_test.go net/textproto, sync: unlock mutexes appropriately before panics 2020-04-08 16:19:51 +00:00
waitgroup.go sync: allow inlining the Mutex.Unlock fast path 2019-03-05 14:59:31 +00:00
waitgroup_test.go sync: deflake TestWaitGroupMisuse3 2019-12-06 05:12:15 +00:00