Changes Darwin semaphore support from using pthread mutexes and
condition variables to using dispatch semaphores. Signaling a dispatch
semaphore is async-signal-safe.
Fixes#31264
Change-Id: If0ce47623501db13e3804b14ace5f4d8eaef461e
Reviewed-on: https://go-review.googlesource.com/c/go/+/182258
Reviewed-by: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Keith Randall <khr@golang.org>
This reverts CL 180761
Reason for revert: Reinstate the stack-allocated defer CL.
There was nothing wrong with the CL proper, but stack allocation of defers exposed two other issues.
Issue #32477: Fix has been submitted as CL 181258.
Issue #32498: Possible fix is CL 181377 (not submitted yet).
Change-Id: I32b3365d5026600069291b068bbba6cb15295eb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/181378
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 179862 introduced go:linkname directives to create ABI wrappers for
Store and Store64 on s390x, but a concurrent change (CL 180439)
replaced the Go definitions of these functions with assembly
definitions. This resulted in conflicting definitions for the ABI0
symbols, which led to a bootstrap linking failure.
Fix this by removing the now-incorrect go:linkname directives for
Store and Store64. This should fix the linux-s390x builders.
Updates #31230.
Change-Id: I8de8c03c23412fc217d428c0018cc56eb2f9996f
Reviewed-on: https://go-review.googlesource.com/c/go/+/181078
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Somehow I missed these two functions in CL 179863. This should fix the
linux-arm builders.
Updates #31230.
Change-Id: I3f8bef3fac331b505a55c0850b0fbc799b7c06c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/181077
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The built-in Go resolver works significantly better.
In particular, the use of res_search does not support
CNAME or PTR queries and may not even be thread-safe.
This CL is essentially a revert of CL 166297 plus fixes,
including CL 180842.
See CL 180842 for additional notes about problems
with this approach.
Fixes#31705.
Change-Id: I0a30a0de2fbd04f6c461520fd34378c84aadf66c
Reviewed-on: https://go-review.googlesource.com/c/go/+/180843
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
It matters whether we are calling a function that would
return a 32-bit or 64-bit -1 on error. A few sites were wrong
and this key detail was omitted from syscall/syscallX docs.
Change-Id: I48a421b6cc4d2d2b5e58f790cc947e3cb2f98940
Reviewed-on: https://go-review.googlesource.com/c/go/+/180841
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This marks all Go symbols called from assembly in other packages with
"go:linkname" directives to ensure they get ABI wrappers.
Now that we have this go:linkname convention, this also removes the
abi0Syms definition in the runtime, which was used to give morestackc
an ABI0 wrapper. Instead, we now just mark morestackc with a
go:linkname directive.
This was tested with buildall.bash in the default configuration, with
-race, and with -gcflags=all=-d=ssa/intrinsics/off. Since I couldn't
test cgo on non-Linux configurations, I manually grepped for runtime
symbols in runtime/cgo.
Updates #31230.
Change-Id: I6c8aa56be2ca6802dfa2bf159e49c411b9071bf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/179862
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The z/Architecture does not guarantee that a load following a store
will not be reordered with that store, unless they access the same
address. Therefore if we want to ensure the sequential consistency
of atomic loads and stores we need to perform serialization
operations after atomic stores.
We do not need to serialize in the runtime when using StoreRel[ease]
and LoadAcq[uire]. The z/Architecture already provides sufficient
ordering guarantees for these operations.
name old time/op new time/op delta
AtomicLoad64-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal)
AtomicStore64-16 0.51ns ± 0% 0.60ns ± 9% +16.47% (p=0.000 n=17+20)
AtomicLoad-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal)
AtomicStore-16 0.51ns ± 0% 0.60ns ± 9% +16.50% (p=0.000 n=18+20)
Fixes#32428.
Change-Id: I88d19a4010c46070e4fff4b41587efe4c628d4d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/180439
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
At least one libc call we make
(res_search, which calls _mdns_query and then mdns_item_call)
pushes a 64 kB stack frame onto the stack.
Then it faults on the guard page.
Use the default system stack size, under the assumption
that the C code being called is compatible with that stack size.
For #31705.
Change-Id: I1b0bfc2e54043c49f0709255988ef920ce30ee82
Reviewed-on: https://go-review.googlesource.com/c/go/+/180779
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts commit fff4f599fe.
Reason for revert: Seems to still have issues around GC.
Fixes#32452
Change-Id: Ibe7af629f9ad6a3d5312acd7b066123f484da7f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/180761
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
When a defer is executed at most once in a function body,
we can allocate the defer record for it on the stack instead
of on the heap.
This should make defers like this (which are very common) faster.
This optimization applies to 363 out of the 370 static defer sites
in the cmd/go binary.
name old time/op new time/op delta
Defer-4 52.2ns ± 5% 36.2ns ± 3% -30.70% (p=0.000 n=10+10)
Fixes#6980
Update #14939
Change-Id: I697109dd7aeef9e97a9eeba2ef65ff53d3ee1004
Reviewed-on: https://go-review.googlesource.com/c/go/+/171758
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
ARM64's R19-R29 and F8-F15 are callee saved registers, which
should be saved in the beginning of sigtramp, and restored at
the end.
fixes#31827
Change-Id: I622e03f1a13fec969d3a11b6a303a8a492e02bcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/177045
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Normally, reflect.makeFuncStub records the context value at a known
point in the stack frame, so that the runtime can get the argument map
for reflect.makeFuncStub from that known location.
This doesn't work for defers or goroutines that haven't started yet,
because they haven't allocated a frame or run an instruction yet. The
argument map must be extracted from the context value. We already do
this for defers (the non-nil ctxt arg to getArgInfo), we just need to
do it for unstarted goroutines as well.
When we traceback a goroutine, remember the context value from
g.sched. Use it for the first frame we find.
(We never need it for deeper frames, because we normally don't stop at
the start of reflect.makeFuncStub, as it is nosplit. With this CL we
could allow makeFuncStub to no longer be nosplit.)
Fixes#25897
Change-Id: I427abf332a741a80728cdc0b8412aa8f37e7c418
Reviewed-on: https://go-review.googlesource.com/c/go/+/180258
Reviewed-by: Cherry Zhang <cherryyz@google.com>
These tests assume that it is OK to switch between time implementations,
but the clock_gettime call uses CLOCK_MONOTONIC and the fallback call,
gettimeofday, uses CLOCK_REALTIME. Disabling the clock_gettime call means
that calls to nanotime will start returning very different values.
This breaks the new timer code, which assumes that nanotime will return
a consistently increasing value.
This test is not very useful in any case as it doesn't check the results.
Removing this file also removes BenchmarkTimeNow, which is a duplicate
of BenchmarkNow in the time package.
Updates #27707Fixes#32109
Change-Id: I6a884af07f75822d724193c5eed94742f524f07d
Reviewed-on: https://go-review.googlesource.com/c/go/+/174679
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently there's an invariant in the runtime wherein the heap lock
can only be acquired on the system stack, otherwise a self-deadlock
could occur if the stack grows while the lock is held.
This invariant is upheld and documented in a number of situations (e.g.
allocManual, freeManual) but there are other places where the invariant
is either not maintained at all which risks self-deadlock (e.g.
setGCPercent, gcResetMarkState, allocmcache) or is maintained but
undocumented (e.g. gcSweep, readGCStats_m).
This change adds go:systemstack to any function that acquires the heap
lock or adds a systemstack(func() { ... }) around the critical section,
where appropriate. It also documents the invariant on (*mheap).lock
directly and updates repetitive documentation to refer to that comment.
Fixes#32105.
Change-Id: I702b1290709c118b837389c78efde25c51a2cafb
Reviewed-on: https://go-review.googlesource.com/c/go/+/177857
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Shorten some of the longest tests that run during all.bash.
Removes 7r 50u 21s from all.bash.
After this change, all.bash is under 5 minutes again on my laptop.
For #26473.
Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79
Reviewed-on: https://go-review.googlesource.com/c/go/+/177559
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
First, remove the randomization of initialization order.
Then, revert to source code order instead of sorted package path order.
This restores the behavior that was in 1.12.
A larger change which will implement the suggestion in #31636 will
wait for 1.14. It's too complicated for 1.13 at this point (it has
tricky interactions with plugins).
Fixes#31636
Change-Id: I35b48e8cc21cf9f93c0973edd9193d2eac197628
Reviewed-on: https://go-review.googlesource.com/c/go/+/178297
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
That way we will never have to look up the file/line for the frame
that's next to be returned when the user stops calling Next.
For the benchmark from #32093:
name old time/op new time/op delta
Helper-4 948ns ± 1% 836ns ± 3% -11.89% (p=0.000 n=9+9)
(#32093 was fixed with a more specific, and better, fix, but this
fix is much more general.)
Change-Id: I89e796f80c9706706d8d8b30eb14be3a8a442846
Reviewed-on: https://go-review.googlesource.com/c/go/+/178077
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, this test allocates many objects and relies on heap-growth
scavenging to happen unconditionally on heap-growth. However with the
new pacing system for the scavenging, this is no longer true and the
test is flaky.
So, this change overhauls TestPhysicalMemoryUtilization to check the
same aspect of the runtime, but in a much more robust way.
Firstly, it sets up a much more constrained scenario: only 5 objects are
allocated total with a maximum worst-case (i.e. the test fails) memory
footprint of about 16 MiB. The test is now aware that scavenging will
only happen if the heap growth causes us to push way past our scavenge
goal, which is based on the heap goal. So, it makes the holes in the
test much bigger and the actual retained allocations much smaller to
keep the heap goal at the heap's minimum size. It does this twice to
create exactly two unscavenged holes. Because the ratio between the size
of the "saved" objects and the "condemned" object is so small, two holes
are sufficient to create a consistent test.
Then, the test allocates one enormous object (the size of the 4 other
objects allocated, combined) with the intent that heap-growth scavenging
should kick in and scavenge the holes. The heap goal will rise after
this object is allocated, so it's very important we do all the
scavenging in a single allocation that exceeds the heap goal because
otherwise the rising heap goal could foil our test.
Finally, we check memory use relative to HeapAlloc as before. Since the
runtime should scavenge the entirety of the remaining holes,
theoretically there should be no more free and unscavenged memory.
However due to other allocations that may happen during the test we may
still see unscavenged memory, so we need to have some threshold. We keep
the current 10% threshold which, while arbitrary, is very conservative
and should easily account for any other allocations the test makes.
Before, we also had to ensure the allocations we were making looked
large relative to the size of a heap arena since newly-mapped memory was
considered unscavenged, and so that could significantly skew the test.
However, thanks to the fix for #32012 we were able to reduce memory use
to 16 MiB in the worst case.
Fixes#32010.
Change-Id: Ia38130481e292f581da7fa3289c98c99dc5394ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/177237
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
A timespec on netbsd/amd64 is int64/int64, not int64/int32.
This bug appears to have been introduced in 7777bac6e4.
Spotted by Cherry Zhang while reviewing https://go-review.googlesource.com/c/go/+/177120.
Change-Id: I163c55d926965defd981bdbfd2511de7d9d4c542
Reviewed-on: https://go-review.googlesource.com/c/go/+/177637
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
On most platforms newly-mapped memory is untouched, meaning the pages
backing the region haven't been faulted in yet. However, we mark this
memory as unscavenged which means the background scavenger
aggressively "returns" this memory to the OS if the heap is small.
The only platform where newly-mapped memory is actually unscavenged (and
counts toward the application's RSS) is on Windows, since
(*mheap).sysAlloc commits the reservation. Instead of making a special
case for Windows, I change the requirements a bit for a sysReserve'd
region. It must now be both sysMap'd and sysUsed'd, with sysMap being a
no-op on Windows. Comments about memory allocation have been updated to
include a more up-to-date mental model of which states a region of memory
may be in (at a very low level) and how to transition between these
states.
Now this means we can correctly mark newly-mapped heap memory as
scavenged on every platform, reducing the load on the background
scavenger early on in the application for small heaps. As a result,
heap-growth scavenging is no longer necessary, since any actual RSS
growth will be accounted for on the allocation codepath.
Finally, this change also cleans up grow a little bit to avoid
pretending that it's freeing an in-use span and just does the necessary
operations directly.
Fixes#32012.
Fixes#31966.
Updates #26473.
Change-Id: Ie06061eb638162e0560cdeb0b8993d94cfb4d290
Reviewed-on: https://go-review.googlesource.com/c/go/+/177097
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This updates the Go function declarations to match race_amd64.s.
Change-Id: I2b541a6b335ce732f4c31652aa615240ce7bb1c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/177397
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The memory might not be synchronized in a thread being woken up after a
semasleep. Using atomic instructions in noteclear function will force
this synchronisation.
Fixes#30189
Change-Id: If7432f29b2a1a56288231822db52f3f8d1d6dbfe
Reviewed-on: https://go-review.googlesource.com/c/go/+/163624
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This was left over from the C->Go transition.
Change-Id: I52494af3d49a388dc45b57210ba68292ae01cf84
Reviewed-on: https://go-review.googlesource.com/c/go/+/176897
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
I forgot that in Go assembly, x+16(SP) is not the same as 16(SP).
The former is the virtual stack pointer (one word below FP on x86)
while the latter is the actual stack pointer.
Change-Id: Ibb7012bb97261949f5e1a0dc70869d9a6f50aa99
Reviewed-on: https://go-review.googlesource.com/c/go/+/176557
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The new prototypes of duffzero and duffcopy must be
accompanied by functions. Otherwise buildmode=shared
(in particular, misc/cgo/testshared) has missing symbols.
The right fix, of course, is to implement these on s390x.
For #31916.
Change-Id: I3efff5e3011956341e1b26223a4847a8a91a0453
Reviewed-on: https://go-review.googlesource.com/c/go/+/176397
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for nacl/*,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I6adb4a7b0c2b03d901fba37f9c05e74e5b7b6691
Reviewed-on: https://go-review.googlesource.com/c/go/+/176107
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for windows/*,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: If37ab2b3f6fca4696b8a6afb2ef11ba6c4fb42e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/176106
Reviewed-by: Austin Clements <austin@google.com>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOSes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I63c4805bdd44b301072da66c77086940e2a2765e
Reviewed-on: https://go-review.googlesource.com/c/go/+/176105
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: Ic64f7f4034695dd4c32c9b7f258960faf3742a83
Reviewed-on: https://go-review.googlesource.com/c/go/+/176103
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
except for an unresolved complaint on mips/mipsle that is a bug in vet,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I6ef7e982a2fdbbfbc22cee876ca37ac54d8109e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176102
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "GOOS=linux GOARCH=arm go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: Ifae75b832320b5356ac8773cf85055bfb2bd7214
Reviewed-on: https://go-review.googlesource.com/c/go/+/176101
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "GOOS=linux GOARCH=amd64 go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I4ca1acb02f4666b102d25fcc55fac96b8f80379a
Reviewed-on: https://go-review.googlesource.com/c/go/+/176100
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "GOOS=linux GOARCH=386 go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I3e5586a7ff6e359357350d0602c2259493280ded
Reviewed-on: https://go-review.googlesource.com/c/go/+/176099
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Trybots started failing on js/wasm after golang.org/cl/175797 landed,
but it seemed completely unrelated. It would fail very consistently on
the heapsampling.go test.
Digging deeper it was very difficult to ascertain what was going wrong,
but clearly m.locks for some m was non-zero when calling into the
scheduler.
The failure comes from the fact that lock calls into gosched, but it's
unclear how exactly we got there in the first place; there should be no
preemption in this single-threaded context.
Furthermore, lock shouldn't be calling gosched_m at all because in a
single-threaded context, the thread shouldn't be preempted until it
actually unlocks.
But, digging further it turns out the implementation in lock_js.go never
incremented or decremented m.locks. This is definitely wrong because
many parts of the runtime depend on that value being set correctly.
So, this change removes the loop which calls into gosched_m (which
should be unnecessary) and increments and decrements m.locks
appropriately. This appears to fix the aforementioned failure.
Change-Id: Id214c0762c3fb2b405ff55543d7e2a78c17443c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/176297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change makes it so that during scavenging we split spans when the
span we have next for scavenging is larger than the amount of work we
have left to do.
The purpose of this change is to improve the worst-case behavior of the
scavenger: currently, if the scavenger only has a little bit of work to
do but sees a very large free span, it will scavenge the whole thing,
spending a lot of time to get way ahead of the scavenge pacing for no
reason.
With this change the scavenger should follow the pacing more closely,
but may still over-scavenge by up to a physical huge page since the
splitting behavior avoids breaking up huge pages in free spans.
This change is also the culmination of the scavenging improvements, so
we also include benchmark results for this series (starting from
"runtime: merge all treaps into one implementation" until this patch).
This patch stack results in average and peak RSS reductions (up to 11%
and 7% respectively) for some benchmarks, with mostly minimal
performance degredation (3-4% for some benchmarks, ~0% geomean). Each of
these benchmarks was executed with GODEBUG=madvdontneed=1 on Linux; the
performance degredation is even smaller when MADV_FREE may be used, but
the impact on RSS is much harder to measure. Applications that generally
maintain a steady heap size for the most part show no change in
application performance.
These benchmarks are taken from an experimental benchmarking suite
representing a variety of open-source Go packages, the raw results may
be found here:
https://perf.golang.org/search?q=upload:20190509.1
For #30333.
Change-Id: I618a48534d2d6ce5f656bb66825e3c383ab1ffba
Reviewed-on: https://go-review.googlesource.com/c/go/+/175797
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a background scavenging goroutine whose pacing is
determined when the heap goal changes. The scavenger is paced to use
at most 1% of the mutator's time for most systems. Furthermore, the
scavenger's pacing is computed based on the estimated number of
scavengable huge pages to take advantage of optimizations provided by
the OS.
The purpose of this scavenger is to deal with a shrinking heap: if the
heap goal is falling over time, the scavenger should kick in and start
returning free pages from the heap to the OS.
Also, now that we have a pacing system, the credit system used by
scavengeLocked has become redundant. Replace it with a mechanism which
only scavenges on the allocation path if it makes sense to do so with
respect to the new pacing system.
Fixes#30333.
Change-Id: I6203f8dc84affb26c3ab04528889dd9663530edc
Reviewed-on: https://go-review.googlesource.com/c/go/+/142960
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change removes the periodic scavenger which goes over every span
in the heap and scavenges it if it hasn't been used for 5 minutes. It
should no longer be necessary if we have background scavenging
(follow-up).
For #30333.
Change-Id: Ic3a1a4e85409dc25719ba4593a3b60273a4c71e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/143157
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The vetall builder runs vet straight out of golang.org/x/tools,
so submiting CL 176097 in that repo will break the builder
by making all these whitelist entries stale.
Submiting this CL will fix it, by removing them.
The addition of the gcWriteBarrier declaration in runtime/stubs.go
is necessary because the diagnostic is no longer emitted on arm,
so it must be removed from all.txt. Adding it to runtime is better
than adding it to every-other-goarch.txt.
For #31916.
Change-Id: I432f6049cd3ee5a467add5066c440be8616d9d54
Reviewed-on: https://go-review.googlesource.com/c/go/+/176177
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a new sysHugePage function to provide the equivalent of
Linux's madvise(MADV_HUGEPAGE) support to the runtime. It then uses
sysHugePage to mark a newly-coalesced free span as backable by huge
pages to make the freeHugePages approximation a bit more accurate.
The problem being solved here is that if a large free span is composed
of many small spans which were coalesced together, then there's a chance
that they have had madvise(MADV_NOHUGEPAGE) called on them at some point,
which makes freeHugePages less accurate.
For #30333.
Change-Id: Idd4b02567619fc8d45647d9abd18da42f96f0522
Reviewed-on: https://go-review.googlesource.com/c/go/+/173338
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Now that the treap is first-fit, we can make a nice optimization.
Mainly, since we know that span splitting doesn't modify the relative
position of a span in a treap, we can actually modify a span in-place
on the treap. The only caveat is that we need to update the relevant
metadata.
To enable this optimization, this change introduces a mutate method on
the iterator which takes a callback that is passed the iterator's span.
The method records some properties of the span before it calls into the
callback and then uses those records to see what changed and update
treap metadata appropriately.
Change-Id: I74f7d2ee172800828434ba0194d3d78d3942acf2
Reviewed-on: https://go-review.googlesource.com/c/go/+/174879
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This change tracks the number of potential free and unscavenged huge
pages which will be used to inform the rate at which scavenging should
occur.
For #30333.
Change-Id: I47663e5ffb64cac44ffa10db158486783f707479
Reviewed-on: https://go-review.googlesource.com/c/go/+/170860
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds two new treap iteration types: one for large
unscavenged spans (contain at least one huge page) and one for small
unscavenged spans. This allows us to scavenge the huge spans first by
first iterating over the large ones, then the small ones.
Also, since we now depend on physHugePageSize being a power of two,
ensure that that's the case when it's retrieved from the OS.
For #30333.
Change-Id: I51662740205ad5e4905404a0856f5f2b2d2a5680
Reviewed-on: https://go-review.googlesource.com/c/go/+/174399
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change introduces a treapIterFilter type which represents the
power set of states described by a treapIterType.
This change then adds a treapIterFilter field to each treap node
indicating the types of spans that live in that subtree. The field is
maintained via the same mechanism used to maintain maxPages. This allows
pred, succ, start, and end to be judicious about which subtrees it will
visit, ensuring that iteration avoids traversing irrelevant territory.
Without this change, repeated scavenging attempts can end up being N^2
as the scavenger walks over what it already scavenged before finding new
spans available for scavenging.
Finally, this change also only scavenges a span once it is removed from
the treap. There was always an invariant that spans owned by the treap
may not be mutated in-place, but with this change violating that
invariant can cause issues with scavenging.
For #30333.
Change-Id: I8040b997e21c94a8d3d9c8c6accfe23cebe0c3d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174878
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change modifies the treap implementation to support holding all
spans in a single treap, instead of keeping them all in separate treaps.
This improves ergonomics for nearly all treap-related callsites.
With that said, iteration is now more expensive, but it never occurs on
the fast path, only on scavenging-related paths.
This change opens up the opportunity for further optimizations, such as
splitting spans without treap removal (taking treap removal off the span
allocator's critical path) as well as improvements to treap iteration
(building linked lists for each iteration type and managing them on
insert/removal, since those operations should be less frequent).
For #30333.
Change-Id: I3dac97afd3682a37fda09ae8656a770e1369d0a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174398
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Those print statements are not a good debug helpers
and only clutter the code.
Change-Id: Ifbf450a04e6fa538af68e6352c016728edb4119a
Reviewed-on: https://go-review.googlesource.com/c/go/+/160537
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
sys.HugePageSize was superceded in the last commit by physHugePageSize
which is determined dynamically by querying the operating system.
For #30333.
Change-Id: I827bfca8bdb347e989cead31564a8fffe56c66ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/173757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds the global physHugePageSize which is initialized in
osinit(). physHugePageSize contains the system's transparent huge page
(or superpage) size in bytes.
For #30333.
Change-Id: I2f0198c40729dbbe6e6f2676cef1d57dd107562c
Reviewed-on: https://go-review.googlesource.com/c/go/+/170858
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This repairs one of the several causes of pauses uncovered
by a GC microbenchmark. A pause can occur when a goroutine's
quantum expires "at the same time" a GC is needed. The
current M switches to running a GC worker, which means that
the amount of available work has expanded by one. The GC
worker, however, does not call ready, and does not itself
conditionally wake a P (a "normal" thread would do this).
This is also true if M switches to a traceReader.
This is problem 4 in this list:
https://github.com/golang/go/issues/27732#issuecomment-423301252
Updates #27732.
Change-Id: I6905365cac8504cde6faab2420f4421536551f0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/146817
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change modifies the treap implementation to be address-ordered
instead of size-ordered, and further augments it so it may be used for
allocation. It then modifies the find method to implement a first-fit
allocation policy.
This change to the treap implementation consequently makes it so that
spans are scavenged in highest-address-first order without any
additional changes to the scavenging code. Because the treap itself is
now address ordered, and the scavenging code iterates over it in
reverse, the highest address is now chosen instead of the largest span.
This change also renames the now wrongly-named "scavengeLargest" method
on mheap to just "scavengeLocked" and also fixes up logic in that method
which made assumptions about size.
For #30333.
Change-Id: I94b6f3209211cc1bfdc8cdaea04152a232cfbbb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/164101
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The spec carefully and consistently uses "key" and "element"
as map terminology. The implementation, not so much.
This change attempts to make the implementation consistently
hew to the spec's terminology. Beyond consistency, this has
the advantage of avoid some confusion and naming collisions,
since v and value are very generic and commonly used terms.
I believe that I found all everything, but there are a lot of
non-obvious places for these to hide, and grepping for them is hard.
Hopefully this change changes enough of them that we will start using
elem going forward. Any remaining hidden cases can be removed ad hoc
as they are discovered.
The only externally-facing part of this change is in package reflect,
where there is a minor doc change and a function parameter name change.
Updates #27167
Change-Id: I2f2d78f16c360dc39007b9966d5c2046a29d3701
Reviewed-on: https://go-review.googlesource.com/c/go/+/174523
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The return values are 32 bit, not 64 bit.
I don't think this would be the cause of any problems, but
it can't hurt to fix it.
Change-Id: Icdd50606360ab9d74070271f9d1721d5fe640bc7
Reviewed-on: https://go-review.googlesource.com/c/go/+/174518
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Like GOOS=android which implies the "linux" build tag, GOOS=illumos
implies the "solaris" build tag. This lets the existing ecosystem of
packages still work on illumos, but still permits packages to start
differentiating between solaris and illumos.
Fixes#20603
Change-Id: I8f4eabf1a66060538dca15d7658c1fbc6c826622
Reviewed-on: https://go-review.googlesource.com/c/go/+/174457
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
They were not needed when Go only produced binaries with cgo suppport.
Now that Go is about to run self-hosted on iOS we do need these.
Updates #31722
Change-Id: If233aa2b31edc7b1c2dcac68974f9fba0604f9a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174300
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The address space of WebAssembly's linear memory is contiguous, so
requesting specific addresses is not supported. Do not use heap arena
hints so we do not have unused memory ranges.
This fixes go1 benchmarks on wasm which ran out of memory since
https://golang.org/cl/170950.
Change-Id: I70115b18dbe43abe16dd5f57996343d97bf94760
Reviewed-on: https://go-review.googlesource.com/c/go/+/174203
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Updates #31656.
Change-Id: Ide6f829282fcdf20c67998b766a201a6a92c3035
Reviewed-on: https://go-review.googlesource.com/c/go/+/174132
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When a callback runs on a different thread in Windows, as in the
runtime package test TestCallbackInAnotherThread, it will use the
extra M. That can cause the test in checkdead to fail incorrectly.
Check whether there actually is an extra M before expecting it.
I think this is a general problem unrelated to timers. I think the test
was passing previously because the timer goroutine was using an M.
But I haven't proved that. This change seems correct, and it avoids
the test failure when using the new timers on Windows.
Updates #27707
Change-Id: Ieb31c04ff0354d6fae7e173b59bcfadb8b0464cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174037
Reviewed-by: Keith Randall <khr@golang.org>
Most of the platforms are returning 0 instead of -1 when mmap syscalls
is failing. This patch corrects it for AIX in order to fix
TestMmapErrorSign and to improve AIX compatibility.
Change-Id: I1dad88d0e69163ad55c504b2b4a997892fd876cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174297
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When running Go programs on Corellium virtual iPhones, the Info.plist
files might not exist. Ignore the error.
Updates #31722
Change-Id: Id2e315c09346b69dda9e10cf29fb5dba6743aac4
Reviewed-on: https://go-review.googlesource.com/c/go/+/174202
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
For example, can use `goroutine all bt` to dump all goroutines'
information.
Change-Id: I51b547c2b837913e4bdabf0f45b28f09250a3e34
GitHub-Last-Rev: d04dcd4f58
GitHub-Pull-Request: golang/go#26283
Reviewed-on: https://go-review.googlesource.com/c/go/+/122589
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
OpenBSD does not provide auxv, however we still need to initialise cpu.HWCap.
For now initialise it to the bare minimum, until some form of CPU capability
detection is implemented or becomes available - see issue #31746.
Updates #31656
Change-Id: I68c3c069319fe60dc873f46def2a67c9f3d937d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/174129
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Whitelists functions debugCall32 through debugCall65536 in
runtime.debugCallCheck so that any instruction inside those functions
is considered a safe point.
This is useful for implementing nested function calls.
For example when evaluating:
f(g(x))
The debugger should:
1. initiate the call to 'f' until the entry point of 'f',
2. complete the call to 'g(x)'
3. copy the return value of 'g(x)' in the arguments of 'f'
4. complete the call to 'f'
Similarly for:
f().amethod()
The debugger should initiate the call to '.amethod()', then initiate
and complete the call to f(), copy the return value to the arguments
of '.amethod()' and finish its call.
However in this example, unlike the other example, it may be
impossible to determine the entry point of '.amethod()' until after
'f()' is evaluated, which means that the call to 'f()' needs to be
initiated while stopped inside a debugCall... function.
Change-Id: I575c23542709cedb1a171d63576f7e11069c7674
Reviewed-on: https://go-review.googlesource.com/c/go/+/161137
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
The kqueue system call takes no arguments, hence there should be no need
to zero the registers used for the first syscall arguments.
Change-Id: Ia79b2d4f4d568bb6795cb885e1464cf1fc2bf7c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/174128
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
The notdead argument to sys___threxit() is a pointer, hence requires a 64-bit
move rather than a 32-bit one.
Updates #31656
Change-Id: I52ad31ed5afaf43ccc3d934025288216e8052528
Reviewed-on: https://go-review.googlesource.com/c/go/+/174124
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Testing whether cgo is enabled in go/build is not the same as testing
whether the go tool supports cgo. They differ, for example, when using
GOARCH=386 on an amd64 system, as for a cross-build cgo is disabled by default.
Change-Id: Ib59106c92a3131b73ac6a91c0f7658a1769acf73
Reviewed-on: https://go-review.googlesource.com/c/go/+/174098
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
AIX doesn't allow to mmap an address range which is already mmap.
Therefore, once the region has been allocated, it must munmap before
being able to play with it.
Change-Id: I1547782f0379024f57869f1dda8c1c9bb12d831f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174059
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ps are strictly numbered from 0 to GOMAXPROCS-1, so if procresize
happens to be running on a P that's being destroyed, it moves itself
to P 0.
However, currently procresize destroys the unused Ps *before* moving
itself to P 0. This means it may briefly run on a destroyed P. This is
basically harmless, but has at least one very confusing consequence:
since destroying a P has write barriers, it may enqueue pointers to a
destroyed write barrier buffer. As far as I can tell, there are no
negative consequences of this, but this seems really fragile.
This CL swaps the order of things, so now procresize moves itself to P
0 if necessary before destroying Ps. This ensures it always has a
valid P.
This is part of refactoring for #10958 and #24543, but is a good
cleanup regardless.
Change-Id: I91a23dd6ed27e372f8d6291feec9bc991bcf9812
Reviewed-on: https://go-review.googlesource.com/c/go/+/173941
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
We're about to change some of these rules, so it's about time we wrote
them down!
For #10958, #24543.
Change-Id: I3efce0c44b53bfb6f31ce2d299809b2b4eb329f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/172857
Reviewed-by: Russ Cox <rsc@golang.org>
This change modifies the implementation of mTreap.find to find the
best-fit span with the lowest possible base address.
Fixes#31616.
Change-Id: Ib4bda0f85d7d0590326f939a243a6e4665f37d3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/173479
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This is one small step to force people to not depend on the order of
initialization of packages which are not explicitly ordered by import
directives. Similar to randomizing map iteration order, this makes
sure people aren't depending on the behavior of the current release,
so that we can change the order in future releases without breaking
everyone.
Maybe one day we can randomize always, but for now we do it just in
race mode. (We would need to measure the impact on startup time before
we enabled it always.)
RELNOTE=yes
Change-Id: I99026394796125974c5f2c3660a88becb92c9df3
Reviewed-on: https://go-review.googlesource.com/c/go/+/170318
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
It is easier to ensure that the symbol is always present
if we move it to package runtime. Avoids init-time work.
Also moves it next to buildVersion, the other similar symbol.
Setting up for "go version <binary>".
For #31624.
Change-Id: I943724469ce6992153e701257eb6f12da88c8e4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/173341
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Setting environment variables for go command configuration
is too difficult and system-specific. This CL adds go env -w,
to change the default settings more easily, in a portable way.
It also adds go env -u, to unset those changes.
See https://golang.org/design/30411-env for details.
Fixes#30411.
Change-Id: I36e83f55b666459f8f7f482432a4a6ee015da71d
Reviewed-on: https://go-review.googlesource.com/c/go/+/171137
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
It will use the full names that appear in netbsd's /usr/include/sys/syscall.h names.
This adds some compat-goo (sys_sigprocmask->SYS_sigprocmask14), which might not be pretty, but the information about whether the compat version is used is probably important, as Go will keep using interfaces even after they are considered compatibility, which has caused problems in the past.
also, the same names appear in ktrace (with the numbers).
Change-Id: Idc1bb254ee33757a39ba224d91e8fbb0331e2149
GitHub-Last-Rev: b915e8f8a3
GitHub-Pull-Request: golang/go#31594
Reviewed-on: https://go-review.googlesource.com/c/go/+/173158
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some of the comments were unclear or outdated.
Change-Id: I02e01bf60def0074c1fa760e94aa992e9e4969b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/172987
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
g.m is an muintptr, but we want to print it in hex like a pointer.
Change-Id: Ifc48ed77fb2e93cff7a49d98adc7b9679d26c3b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/172988
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This works well enough to run some code natively on arm64, but not well enough for more complicated code. I've been suggested to start a pull request anyway.
Updates #30824
Change-Id: Ib4f63e0e8a9edfc862cf65b5f1b0fbf9a8a1628e
GitHub-Last-Rev: b01b105e04
GitHub-Pull-Request: golang/go#29398
Reviewed-on: https://go-review.googlesource.com/c/go/+/155739
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Allows us to stop whitelisting this error on many OS/arch combinations
XXX I'm not sure I am running vet correctly, and testing all platforms right.
Change-Id: I29f548bd5f4a63bd13c4d0667d4209c75c886fd9
GitHub-Last-Rev: 52f6ff4a6b
GitHub-Pull-Request: golang/go#31583
Reviewed-on: https://go-review.googlesource.com/c/go/+/173157
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Pass "set print thread-events off" to gdb to suppress thread
event prints, like "[New Thread 0xe7b83b40 (LWP 18609)]". We
don't check them, and the extra output may confuse our other
checks, in particular, checkCleanBacktrace.
Hopefully fixes#31569.
Change-Id: I6549e1280da7afa1d2e38da2b2fa7cc18c2f0373
Reviewed-on: https://go-review.googlesource.com/c/go/+/172980
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The pclntab encoding supports writing only some PCDATA and FUNCDATA values.
However, the encoding is dense: The max index in use determines the space used.
We should thus choose a numbering in which frequently used indices are smaller.
This change re-orders the PCDATA and FUNCDATA indices using that principle,
using a quick and dirty instrumentation to measure index frequency.
It shrinks binaries by about 0.5%.
Updates #6853
file before after Δ %
go 14745044 14671316 -73728 -0.500%
addr2line 4305128 4280552 -24576 -0.571%
api 6095800 6058936 -36864 -0.605%
asm 4930928 4906352 -24576 -0.498%
buildid 2881520 2861040 -20480 -0.711%
cgo 4896584 4867912 -28672 -0.586%
compile 25868408 25770104 -98304 -0.380%
cover 5319656 5286888 -32768 -0.616%
dist 3654528 3634048 -20480 -0.560%
doc 4719672 4691000 -28672 -0.607%
fix 3418312 3393736 -24576 -0.719%
link 6137952 6109280 -28672 -0.467%
nm 4250536 4225960 -24576 -0.578%
objdump 4665192 4636520 -28672 -0.615%
pack 2297488 2285200 -12288 -0.535%
pprof 14735332 14657508 -77824 -0.528%
test2json 2834952 2818568 -16384 -0.578%
trace 11679964 11618524 -61440 -0.526%
vet 8452696 8403544 -49152 -0.581%
Change-Id: I30665dce57ec7a52e7d3c6718560b3aa5b83dd0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/171760
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Fixes#31554
Updates #12032 (also originally about plan9, but later openbsd/arm)
Change-Id: Ib9f35d27a2304f38bf271c38c0b9153d210d8f95
Reviewed-on: https://go-review.googlesource.com/c/go/+/172837
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
As the stackguard was increased on AIX by CL 157117, every syscalls can
now have libcall directly on the stack. This fixes some concurrency bugs
which seems to occur when semasleep is interrupted by a SIGPROF signal.
Change-Id: I905a9618d13ef227dad6f8328b0f958f2f917a5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/172359
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
variable setg_gcc in runtime/cgo/*.c should be static, otherwise it
will be mixed with the function of the same name in runtime/asm_*.s or
tls_*.s, which causes an error when building PIE with internal linking
mode.
Fixes#31485
Change-Id: I79b311ffcaf450984328db65397840ae7d85e65d
Reviewed-on: https://go-review.googlesource.com/c/go/+/172498
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The //go:noescape directive says that arguments don't leak at all,
which is too aggressive of a claim for functions that return pointers
derived from their parameters.
Remove the directive for now. Long term fix will require a new
directive that allows more fine-grained control over escape analysis
information supplied for functions implemented in assembly.
Also, update the BAD comments in the test cases for Loadp: we really
want that *ptr leaks to the result parameter, not that *ptr leaks to
the heap.
Updates #31525.
Change-Id: Ibfa61f2b70daa7ed3223056b57eeee777eef2e31
Reviewed-on: https://go-review.googlesource.com/c/go/+/172578
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Though there is variation in the spelling of canceled,
cancellation is always spelled with a double l.
Reference: https://www.grammarly.com/blog/canceled-vs-cancelled/
Change-Id: I240f1a297776c8e27e74f3eca566d2bc4c856f2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/170060
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Same as CL 170323, but for s390x instead of wasm.
Fixes#31495.
Change-Id: Ie39f649f5e33690375a8bcb1bc3b92d912ca4398
Reviewed-on: https://go-review.googlesource.com/c/go/+/172417
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
sigaction is called as part of library mode initializers
(_rt0_*_lib). Sigaction in turn calls getg, but on Android the TLS
offset for g has not been initialized and getg might return garbage.
Add a check for initialization before calling getg.
Fixes the golang.org/x/mobile/bind/java tests on amd64 and 386.
Fixes#31476
Change-Id: Id2c41fdc983239eca039b49a54b8853c5669d127
Reviewed-on: https://go-review.googlesource.com/c/go/+/172158
Reviewed-by: Ian Lance Taylor <iant@golang.org>
WebAssembly's memory is contiguous. Allocating memory at a high address
also allocates all memory up to that address. This change reduces
the initial memory allocated on wasm from 1GB to 16MB by using multiple
heap arenas and reducing the size of a heap arena.
Fixes#27462.
Change-Id: Ic941e6edcadd411e65a14cb2f9fd6c8eae02fc7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/170950
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
On Ubuntu 18.04 I am seeing GDB fail to restore the stack pointer
during this test because stack unwinding can't find the PC. This CL
is essentially a partial revert of CL 23940 and fixes the issue on
s390x.
Change-Id: Ib4c41162dc85dc882eb6e248330f4082c3fa94c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/169857
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This CL adds a new attribute, TOPFRAME, which can be used to mark
functions that should be treated as being at the top of the call
stack. The function `runtime.goexit` has been marked this way on
architectures that use a link register.
This will stop programs that use DWARF to unwind the call stack
from unwinding past `runtime.goexit` on architectures that use a
link register. For example, it eliminates "corrupt stack?"
warnings when generating a backtrace that hits `runtime.goexit`
in GDB on s390x.
Similar code should be added for non-link-register architectures
(i.e. amd64, 386). They mark the top of the call stack slightly
differently to link register architectures so I haven't added
that code (they need to mark "rip" as undefined).
Fixes#24385.
Change-Id: I15b4c69ac75b491daa0acf0d981cb80eb06488de
Reviewed-on: https://go-review.googlesource.com/c/go/+/169726
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
On AIX, syscallX functions are using the first argument to retrieve the
next arguments when calling asmcgocall. Therefore,//go:cgo_unsafe_args
is needed.
Change-Id: I7fe0fbf0c961250a6573c66a8b0eb897dff94bfe
Reviewed-on: https://go-review.googlesource.com/c/go/+/171723
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Inside syscall_syscall6 function, libcall can directly be on the stack.
This is first function called with //go:nosplit, unlike runtime syscalls
which can be called during the sigtramp or by others //go:nosplit
functions.
Change-Id: Icc28def1a63e525850ec3bfb8184b995dfeaa736
Reviewed-on: https://go-review.googlesource.com/c/go/+/171338
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
procresize is rather ungainly because it includes all of the code to
initialize new Ps and tear down unused Ps. Pull these out into their
own methods on the p type. This also tweaks the initialization loop in
procresize so we only initialize new Ps (and Ps we previously
destroyed) rather than asking each P "have you been initialized?"
This is for #10958 and #24543, but it's also just a nice cleanup.
Change-Id: Ic1242066f572c94a23cea8ea4dc47c918e31d559
Reviewed-on: https://go-review.googlesource.com/c/go/+/171762
Reviewed-by: Michael Knyszek <mknyszek@google.com>
For a failed interface conversion not in ",ok" form, getitab
calls itab.init to get the name of the missing method for the
panic message. itab.init will try to find the methods, populate
the method table as it goes. When some method is missing, it sets
itab.fun[0] to 0 before return. There is a small window that
itab.fun[0] could be non-zero.
If concurrently, another goroutine tries to do the same interface
conversion, it will read the same itab's fun[0]. If this happens
in the small window, it sees a non-zero fun[0] and thinks the
conversion succeeded, which is bad.
Fix the race by setting fun[0] to non-zero only when we know the
conversion succeeds. While here, also simplify the syntax
slightly.
Fixes#31419.
Change-Id: Ied34d3043079eb933e330c5877b85e13f98f1916
Reviewed-on: https://go-review.googlesource.com/c/go/+/171759
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Currently which treap a span should be inserted into/removed from is
checked by looking at the span's properties. This logic is repeated in
four places. As this logic gets more complex, it makes sense to
de-duplicate this, so introduce treapForSpan instead which captures this
logic by returning the appropriate treap for the span.
For #30333.
Change-Id: I4bd933d93dc50c5fc7c7c7f56ceb95194dcbfbcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/170857
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change exports the runtime mTreap in export_test.go and then adds a
series of tests which check that the invariants of the treap are
maintained under different operations. These tests also include tests
for the treap iterator type.
Also, we note that the find() operation on the treap never actually was
best-fit, so the tests just ensure that it returns an appropriately
sized span.
For #30333.
Change-Id: If81f7c746dda6677ebca925cb0a940134701b894
Reviewed-on: https://go-review.googlesource.com/c/go/+/164100
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently when coalescing if two adjacent spans are scavenged, we
subtract their sizes from memstats and re-scavenge the new combined
span. This is wasteful however, since the realignment semantics make
this case of having to re-scavenge impossible.
In realign() inside of coalesce(), there was also a bug: on systems
where physPageSize > pageSize, we wouldn't realign because a condition
had the wrong sign. This wasteful re-scavenging has been masking this
bug this whole time. So, this change fixes that first.
Then this change gets rid of the needsScavenge logic and instead checks
explicitly for the possibility of unscavenged pages near the physical
page boundary. If the possibility exists, it throws. The intent of
throwing here is to catch changes to the runtime which cause this
invariant to no longer hold, at which point it would likely be
appropriate to scavenge the additional pages (and only the additional
pages) at that point.
Change-Id: I185e3d7b53e36e90cf9ace5fa297a9e8008d75f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/158377
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
CL 170955 set tlsg to the Android Q free TLS slot offset in the linker
data (16 on amd64, 8 on 386), offsetting all TLS relative access.
We need the 0'th slot (TLS_SLOT_SELF) at initialization, so
compensate with a corresponding negative offset.
Fixes the android/386 and android/amd64 builders broken by CL 170955.
Change-Id: I9882088c0c8bc6a777d2aabc9404cb76f02b6cea
Reviewed-on: https://go-review.googlesource.com/c/go/+/170956
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
A goroutine should be preempted if it runs for 10ms without blocking.
We found that this doesn't work for goroutines which call short system calls.
For example, the next program can stuck for seconds without this fix:
$ cat main.go
package main
import (
"runtime"
"syscall"
)
func main() {
runtime.GOMAXPROCS(1)
c := make(chan int)
go func() {
c <- 1
for {
t := syscall.Timespec{
Nsec: 300,
}
if true {
syscall.Nanosleep(&t, nil)
}
}
}()
<-c
}
$ time go run main.go
real 0m8.796s
user 0m0.367s
sys 0m0.893s
Updates #10958
Change-Id: Id3be54d3779cc28bfc8b33fe578f13778f1ae2a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/170138
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Android Q frees a static TLS slot for us to use. Use the offset of
that slot as the default for our TLS offset.
As a result, runtime/cgo is no more a requirement for Android Q and
newer.
Updates #31343
Updates #29674
Change-Id: I759049b2e2865bd3d4fdc05a8cfc6db8b0da1f5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/170955
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Right now the mTreap structure exposes the treapNode structure through
only one interface: find. There's no reason (performance or otherwise)
for exposing this, and we get a cleaner abstraction through the
iterators this way. This change also makes it easier to make changes to
the mTreap implementation without violating its interface.
Change-Id: I5ef86b8ac81a47d05d8404df65af9ec5f419dc40
Reviewed-on: https://go-review.googlesource.com/c/go/+/164098
Reviewed-by: Austin Clements <austin@google.com>
This change just makes the code in scavengeLargest easier to reason
about by reducing the number of exit points to the method. It should
still be correct either way because the condition checked at the end
(released > nbytes) will always be false if we return, but this just
makes the code a little easier to maintain.
Change-Id: If60da7696aca3fab3b5ddfc795d600d87c988238
Reviewed-on: https://go-review.googlesource.com/c/go/+/160617
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently, shrinkstack will free the stack if the goroutine is dead.
There are only two places that call shrinkstack: scanstack, which will
never call it if the goroutine is dead; and markrootFreeGStacks, which
only calls it on dead goroutines.
Clean this up by separating stack freeing out of shrinkstack.
Change-Id: I7d7891e620550c32a2220833923a025704986681
Reviewed-on: https://go-review.googlesource.com/c/go/+/170890
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
We've copy-pasted the pattern of releasem in many places. This CL
replaces almost everywhere that manipulates g.m.locks and g.preempt
with calls to acquirem/releasem. There are a few where we do something
more complicated, like where exitsyscall has to restore the stack
bound differently depending on the preempt flag, which this CL leaves
alone.
Change-Id: Ia7a46c261daea6e7802b80e7eb9227499f460433
Reviewed-on: https://go-review.googlesource.com/c/go/+/170064
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 3660 replaced m.gcing with m.preemptoff, but unintentionally
reversed the sense of part of a sanity check in notetsleep.
Originally, notetsleep required that it be called from g0 or with
preemption disabled (specifically from within the garbage collector).
CL 3660 made it require that it be called from g0 or that preemption
be *enabled*.
I'm not sure why it had the original exception for being called from a
user g within the garbage collector, but the current garbage collector
certainly doesn't need that, and the new condition is completely wrong.
Make the sanity check just require that it's called on g0.
Change-Id: I6980d44f5a4461935e10b1b33a981e32b1b7b0c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/170063
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds an internal runtime debug log. It uses per-M time-stamped
ring buffers of binary log records. On panic, these buffers are
collected, interleaved, and printed.
The entry-point to the debug log is a new "dlog" function. dlog is
designed so it can be used even from very constrained corners of the
runtime such as signal handlers or inside the write barrier.
The facility is only enabled if the debuglog build tag is set.
Otherwise, it compiles away to a no-op implementation.
The debug log format is also designed so it would be reasonable to
decode from a core dump, though this hasn't been implemented.
Change-Id: I6e2737c286358e97a0d8091826498070b95b66a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/157997
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This change adds directives to link the res_search function in libSystem.
The corresponding Go function is then used in `lookup_darwin.go` for
resolution when cgo is disabled. This makes DNS resolution logic more
reliable as macOS has some unique quirks such as the `/etc/resolver/`
directory for specifying nameservers.
Fixes#12524
Change-Id: I367263c4951383965b3ef6491196152f78e614b1
GitHub-Last-Rev: 3c3ff6bfa7
GitHub-Pull-Request: golang/go#30686
Reviewed-on: https://go-review.googlesource.com/c/go/+/166297
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Getdirentries is implemented with the __getdirentries64 function
in libSystem.dylib. That function works, but it's on Apple's
can't-be-used-in-an-app-store-application list.
Implement Getdirentries using the underlying fdopendir/readdir_r/closedir.
The simulation isn't faithful, and could be slow, but it should handle
common cases.
Don't use Getdirentries in the stdlib, use fdopendir/readdir_r/closedir
instead (via (*os.File).readdirnames).
Fixes#30933
Update #28984
RELNOTE=yes
Change-Id: Ia6b5d003e5bfe43ba54b1e1d9cfa792cc6511717
Reviewed-on: https://go-review.googlesource.com/c/go/+/168479
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The presence of the android_get_device_api_level symbol is used to
detect Android Q or later. Use the suggestion by Ryan Prichard and
look for it in libc.so and not in the entire program where someone
else might have defined it.
Manually tested on an Android Q amd64 emulator and arm64 Pixel.
Updates #29674
Change-Id: Iaef35d8f8910037b3690aa21f319e216a05a9a73
Reviewed-on: https://go-review.googlesource.com/c/go/+/170127
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Package unsafe's safety rules require that pointers converted to
uintptr must be converted back to pointer-type before being stored
into memory. In particular, storing a pointer into a non-pointer-typed
expression does not guarantee the pointer stays valid, even if the
expression refers to a pointer-typed variable.
wasm's StorepNoWB implementation violates these rules by storing a
pointer through a uintptr-typed expression.
This happens to work today because esc.go is lenient in its
implementation of package unsafe's rules, but my escape analysis
rewrite follows them more rigorously, which causes val to be treated
as a non-leaking parameter.
This CL fixes the issue by using a *T-typed expression, where T is
marked //go:notinheap so that the compiler still omits the write
barrier as appropriate.
Updates #23109.
Change-Id: I49bc5474dbaa95729e5c93201493afe692591bc8
Reviewed-on: https://go-review.googlesource.com/c/go/+/170323
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Currently the shift amount is only masked on x86. Change it so it
is masked on all architectures. In the worst case we generate a
couple of extra instructions to perform the masking and in the best
case we can elide overflow checks.
This particular shift could also be replaced with a rotate
instruction during optimization which would remove both the masking
instructions and overflow checks on all architectures.
Fixes#31165.
Change-Id: I16b7a8800b4ba8813dc83735dfc59564e661d3b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/170122
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
In CL 169960 I didn't realize that we also have race detector support for arm64.
Change-Id: If77bfb0f700a04c04416dad61ef11e27b1c98e07
Reviewed-on: https://go-review.googlesource.com/c/go/+/170105
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Android assumes pthread tls keys correspond to some offset from the
TLS base. This is about to change in a future version of Android.
Fortunately, Android Q leaves a slot open for use to use, TLS_SLOT_APP.
Fixes#29674
Change-Id: Id6ba19afacdfed9b262453714715435e2544185f
Reviewed-on: https://go-review.googlesource.com/c/go/+/170117
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We're going to need a different TLS offset for Android Q, so the static
offsets used for 386 and amd64 are no longer viable on Android.
Introduce runtime·tls_g and use that for indexing into TLS storage. As
an added benefit, we can then merge the TLS setup code for all android
GOARCHs.
While we're at it, remove a bunch of android special cases no longer
needed.
Updates #29674
Updates #29249 (perhaps fixes it)
Change-Id: I77c7385aec7de8f1f6a4da7c9c79999157e39572
Reviewed-on: https://go-review.googlesource.com/c/go/+/169817
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Both g and p had a racectx field, but they held different kinds of values.
The g field held ThreadState values while the p field held Processor values
(to use the names used in the C++ code in the compiler_rt support library).
Rename the p field to raceprocctx to reduce potential confusion.
Change-Id: Iefba0e259d240171e973054c452c3c15bf3f8f8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/169960
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When run with GOPATH=/dev/null, go build fails:
$ GOPATH=/dev/null go test -run=TestLldbPython -v -count=1 runtime
=== RUN TestLldbPython
--- FAIL: TestLldbPython (0.21s)
runtime-lldb_test.go:169: building source exit status 1
go: failed to create cache directory /dev/null/pkg/mod/cache: mkdir /dev/null: not a directory
FAIL
FAIL runtime 0.220s
But run.bash sets GOPATH=/dev/null.
Fix this by setting GOPATH to the empty string before passing to 'go build'.
Fixes#31100
Change-Id: I573c4755d209e0c3eb26c20d4f7870c2961f2782
Reviewed-on: https://go-review.googlesource.com/c/go/+/169918
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
The specialized functions set up the g register using the pthread
API instead of setg_gcc, but the inittls functions have already
made sure setg_gcc works.
Updates #29674
Change-Id: Ie67c068d638af8b5823978ee839f6b61b2228996
Reviewed-on: https://go-review.googlesource.com/c/go/+/169797
Reviewed-by: Ian Lance Taylor <iant@golang.org>
As .init_array section aren't available on AIX, the Go runtime
initialization is made with gcc constructor attribute.
However, as cgo tool is building a binary in order to get imported
C symbols, Go symbols imported for this initilization must be ignored.
-Wl,-berok is mandatory otherwize ld will fail to create this binary,
_rt0_aix_ppc64_lib and runtime_rt0_go aren't defined in runtime/cgo.
These two symbols must also be ignored when creating _cgo_import.go.
Change-Id: Icf2e0282f5b50de5fa82007439a428e6147efef1
Reviewed-on: https://go-review.googlesource.com/c/go/+/169118
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Now the net package is back to no longer depending on unicode. And lock that in
with a test.
Fixes#30440
Change-Id: I18b89b02f7d96488783adc07308da990f505affd
Reviewed-on: https://go-review.googlesource.com/c/go/+/169137
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
R14, R15 must be saved in sigtramp because they might be modified by Go
code when a SIGPROF occurs.
Fixes#28555
Change-Id: I573541f108d7d6aac8e60d33c649e5db943f3ef5
Reviewed-on: https://go-review.googlesource.com/c/go/+/169117
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We already have the ptrdata field in a type, which encodes exactly
the same information that kindNoPointers does.
My problem with kindNoPointers is that it often leads to
double-negative code like:
t.kind & kindNoPointers != 0
Much clearer is:
t.ptrdata == 0
Update #27167
Change-Id: I92307d7f018a6bbe3daca4a4abb4225e359349b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/169157
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Instrumenting make.bash reveals that almost half (49.54%)
of the >16 million calls to growslice for
pointer-containing slices are
growing from an empty to a non-empty slice.
In that case, there is no need to call the write barrier,
which does some work before discovering that no pointers need shading.
Change-Id: Ide741468d8dee7ad43ea0bfbea6ccdf680030a0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/168959
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
It seems like we need to pay special attention to capturing error
condition on the event port of SmartOS. The previous attempt CL 167777
works on Oracle Solaris but doesn't work on SmartOS for the uncertain
reason. It's better to disable the reporting for now.
Updates #30624.
Fixes#30840.
Change-Id: Ieca5dac4fceb7e8c9cb4db149bb4c2e79691588c
Reviewed-on: https://go-review.googlesource.com/c/go/+/167782
Run-TryBot: Mikio Hara <mikioh.public.networking@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit removes spaces which were wrongly added in
//go:cgo_export_static during CL 164010.
Change-Id: Iadd18efdde9ff32e907d793a72ef0f9efda35fe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/168317
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit fixes runtime.sigfwd for aix/ppc64. fn is a function
descriptor and not a function. R2 must be saved and restored.
Change-Id: Ie506b0bdde562ca37202d19973ba1d537c3d64e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/164015
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change makes the runtime-integrated network poller report only
critical event scanning errors.
In the previous attempt, CL 166497, we treated any combination of error
events as event scanning errors and it caused false positives in event
waiters because platform-dependent event notification mechanisms allow
event originators to use various combination of events.
To avoid false positives, this change makes the poller treat an
individual error event as a critical event scanning error by the
convention of event notification mechanism implementations.
Updates #30624.
Fixes#30817.
Fixes#30840.
Change-Id: I906c9e83864527ff73f636fd02bab854d54684ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/167777
Run-TryBot: Mikio Hara <mikioh.public.networking@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: Icc42843adb15c2aba1cfea854fad049c6704344b
Reviewed-on: https://go-review.googlesource.com/c/go/+/164014
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit disables new cgo pprof tests and adds an handler in sigtramp
to refuse SIGPROF signal.
Updates #28555
Change-Id: I152a871f8636e93328d411329104c6f023bd1691
Reviewed-on: https://go-review.googlesource.com/c/go/+/164013
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit fixes TestSigStackSwapping by increasing the signal stack
size. This is needed because SIGSTKSZ is too small when VMX is used on
AIX.
Change-Id: Ic2b5faa65745228d0768383b3d6ebd4b6f9f532c
Reviewed-on: https://go-review.googlesource.com/c/go/+/164012
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
C trampolines are made by fixup CSECTS which are added between two
symbols. If such CSECTS is added inside Go functions, all method
offsets stored in moduledatas will be wrong.
In order to prevent this, every C code is moved at the end of the
executable and long calls are created for GO functions called by C
code.
The main function can't longer be made in Go as AIX __start isn't using
a long call to branch on it. Therefore, a main is defined on
runtime/cgo.
Change-Id: I214b18decdb83107cf7325b298609eef9f9d1330
Reviewed-on: https://go-review.googlesource.com/c/go/+/164010
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Some of the registers in which indexes + length were supposed to
be passed were wrong.
Update #30116
Change-Id: I1089366b7429c1e0ecad9219b847db069ce6b5d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/168041
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Instead of writing an init function per package that does the same
thing for every package, just write that implementation once in the
runtime. Change the compiler to generate a data structure that encodes
the required initialization operations.
Reduces cmd/go binary size by 0.3%+. Most of the init code is gone,
including all the corresponding stack map info. The .inittask
structures that replace them are quite a bit smaller.
Most usefully to me, there is no longer an init function in every -S output.
(There is an .inittask global there, but it's much less distracting.)
After this CL we could change the name of the "init.ializers" function
back to just "init".
Update #6853
R=go1.13
Change-Id: Iec82b205cc52fe3ade4d36406933c97dbc9c01b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/161337
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
A few examples (for accessing a slice of length 3):
s[-1] runtime error: index out of range [-1]
s[3] runtime error: index out of range [3] with length 3
s[-1:0] runtime error: slice bounds out of range [-1:]
s[3:0] runtime error: slice bounds out of range [3:0]
s[3:-1] runtime error: slice bounds out of range [:-1]
s[3:4] runtime error: slice bounds out of range [:4] with capacity 3
s[0:3:4] runtime error: slice bounds out of range [::4] with capacity 3
Note that in cases where there are multiple things wrong with the
indexes (e.g. s[3:-1]), we report one of those errors kind of
arbitrarily, currently the rightmost one.
An exhaustive set of examples is in issue30116[u].out in the CL.
The message text has the same prefix as the old message text. That
leads to slightly awkward phrasing but hopefully minimizes the chance
that code depending on the error text will break.
Increases the size of the go binary by 0.5% (amd64). The panic functions
take arguments in registers in order to keep the size of the compiled code
as small as possible.
Fixes#30116
Change-Id: Idb99a827b7888822ca34c240eca87b7e44a04fdd
Reviewed-on: https://go-review.googlesource.com/c/go/+/161477
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Divisions are generally slow. The compiler can optimize a division
to use a sequence of faster multiplies and shifts (magic constants)
if the divisor is not know at compile time.
The value of the span element size in mcentral.grow is not known at
compile time but magic constants to compute n / span.elementsize
are already stored in class_to_divmagic and mspan.
They however need to be adjusted to work for
(0 <= n <= span.npages * pagesize) instead of
(0 <= n < span.npages * pagesize).
Change-Id: Ieea59f1c94525a88d012f2557d43691967900deb
Reviewed-on: https://go-review.googlesource.com/c/go/+/148057
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
The current wasm write barrier implementation incorrectly implements
the "deletion" part of the barrier. It correctly greys the new value
of the pointer, but rather than also greying the old value of the
pointer, it greys the object containing the slot (which, since the old
value was just overwritten, is not going to contain the old value).
This can lead to unmarked, reachable objects.
Often, this is masked by other marking activity, but one specific
sequence that can lead to an unmarked object because of this bug is:
1. Initially, GC is off, object A is reachable from just one pointer
in the heap.
2. GC starts and scans the stack of goroutine G.
3. G copies the pointer to A on to its stack and overwrites the
pointer to A in the heap. (Now A is reachable only from G's stack.)
4. GC finishes while A is still reachable from G's stack.
With a functioning deletion barrier, step 3 causes A to be greyed.
Without a functioning deletion barrier, nothing causes A to be greyed,
so A will be freed even though it's still reachable from G's stack.
This CL fixes the wasm write barrier.
Fixes#30871.
Change-Id: I8a74ee517facd3aa9ad606e5424bcf8f0d78e754
Reviewed-on: https://go-review.googlesource.com/c/go/+/167743
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The general code for setting a timespec value sometimes used set_nsec
and sometimes used a combination of set_sec and set_nsec. Standardize
on a setNsec function that takes a number of nanoseconds and splits
them up to set the tv_sec and tv_nsec fields. Consistently mark
setNsec as go:nosplit, since it has to be that way on some systems
including Darwin and GNU/Linux. Consistently use timediv on 32-bit
systems to help stay within split-stack limits on processors that
don't have a 64-bit division instruction.
Change-Id: I6396bb7ddbef171a96876bdeaf7a1c585a6d725b
Reviewed-on: https://go-review.googlesource.com/c/go/+/167389
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
As of CL 163747, we can write arbitrary length strings
in assembly DATA instructions.
Make use of it here to improve readability.
Change-Id: I556279ca893f527874e3b26112c43573834ccd9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/167386
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We already have a pure go code sequence that is compiled into single load.
Just use it everywhere, instead of pointer hackery.
Passes toolstash-check.
Change-Id: I0c42b5532fa9a5665da3385913609c6d42aaff27
Reviewed-on: https://go-review.googlesource.com/c/go/+/118568
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change makes it possible the runtime-integrated network poller and
APIs in the package internal/poll to report an event scanning error on a
read event.
The latest Go releases open up the way of the manipulation of the poller
for users. On the other hand, it starts misleading users into believing
that the poller accepts any user-configured file or socket perfectly
because of not reporting any error on event scanning, as mentioned in
issue 30426. The initial implementation of the poller was designed for
just well-configured, validated sockets produced by the package net.
However, the assumption is now obsolete.
Fixes#30624.
Benchmark results on linux/amd64:
benchmark old ns/op new ns/op delta
BenchmarkTCP4OneShot-4 24649 23979 -2.72%
BenchmarkTCP4OneShotTimeout-4 25742 24411 -5.17%
BenchmarkTCP4Persistent-4 5139 5222 +1.62%
BenchmarkTCP4PersistentTimeout-4 4919 4892 -0.55%
BenchmarkTCP6OneShot-4 21182 20767 -1.96%
BenchmarkTCP6OneShotTimeout-4 23364 22305 -4.53%
BenchmarkTCP6Persistent-4 4351 4366 +0.34%
BenchmarkTCP6PersistentTimeout-4 4227 4255 +0.66%
BenchmarkTCP4ConcurrentReadWrite-4 2309 1839 -20.36%
BenchmarkTCP6ConcurrentReadWrite-4 2180 1791 -17.84%
benchmark old allocs new allocs delta
BenchmarkTCP4OneShot-4 26 26 +0.00%
BenchmarkTCP4OneShotTimeout-4 26 26 +0.00%
BenchmarkTCP4Persistent-4 0 0 +0.00%
BenchmarkTCP4PersistentTimeout-4 0 0 +0.00%
BenchmarkTCP6OneShot-4 26 26 +0.00%
BenchmarkTCP6OneShotTimeout-4 26 26 +0.00%
BenchmarkTCP6Persistent-4 0 0 +0.00%
BenchmarkTCP6PersistentTimeout-4 0 0 +0.00%
BenchmarkTCP4ConcurrentReadWrite-4 0 0 +0.00%
BenchmarkTCP6ConcurrentReadWrite-4 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkTCP4OneShot-4 2000 2000 +0.00%
BenchmarkTCP4OneShotTimeout-4 2000 2000 +0.00%
BenchmarkTCP4Persistent-4 0 0 +0.00%
BenchmarkTCP4PersistentTimeout-4 0 0 +0.00%
BenchmarkTCP6OneShot-4 2144 2144 +0.00%
BenchmarkTCP6OneShotTimeout-4 2144 2145 +0.05%
BenchmarkTCP6Persistent-4 0 0 +0.00%
BenchmarkTCP6PersistentTimeout-4 0 0 +0.00%
BenchmarkTCP4ConcurrentReadWrite-4 0 0 +0.00%
BenchmarkTCP6ConcurrentReadWrite-4 0 0 +0.00%
Change-Id: Iab60e504dff5639e688dc5420d852f336508c0af
Reviewed-on: https://go-review.googlesource.com/c/go/+/166497
Run-TryBot: Mikio Hara <mikioh.public.networking@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
While many other call sites have been moved to using the proper
higher-level system loading, these areas were left out. This prevents
DLL directory injection attacks. This includes both the runtime load
calls (using LoadLibrary prior) and the implicitly linked ones via
cgo_import_dynamic, which we move to our LoadLibraryEx. The goal is to
only loosely load kernel32.dll and strictly load all others.
Meanwhile we make sure that we never fallback to insecure loading on
older or unpatched systems.
This is CVE-2019-9634.
Fixes#14959Fixes#28978Fixes#30642
Change-Id: I401a13ed8db248ab1bb5039bf2d31915cac72b93
Reviewed-on: https://go-review.googlesource.com/c/go/+/165798
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
It turns out not to be necessary. Russ expressed a preference for
avoiding module fetches over making 'go mod tidy' work within std and
cmd right away, so for now we will make the loader use the vendor
directory for the standard library even if '-mod=vendor' is not set
explicitly.
Updates #30228
Change-Id: Idf7208e63da8cb7bfe281b93ec21b61d40334947
Reviewed-on: https://go-review.googlesource.com/c/go/+/166357
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
If a for loop has a simple condition and begins with a simple
"if x { break; }"; we can simply add "!x" to the loop's condition.
While at it, simplify a few assignments to use the common pattern
"x := staticDefault; if cond { x = otherValue(); }".
Finally, simplify a couple of var declarations.
Change-Id: I413982c6abd32905adc85a9a666cb3819139c19f
Reviewed-on: https://go-review.googlesource.com/c/go/+/165342
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently only CPU profile utilizes tag information.
This change documents that fact
Updates #23458
Change-Id: Ic893e85f63af0da9100d8cba7d3328c294e8c810
GitHub-Last-Rev: be99a12629
GitHub-Pull-Request: golang/go#27198
Reviewed-on: https://go-review.googlesource.com/c/go/+/131275
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This CL removes runtime code working around missing ARM processor capability
information in the auxiliary vector in older FreeBSD versions.
As announced in the Go 1.12 release notes Go 1.13 will require FreeBSD 11.2+
or FreeBSD 12.0+. These FreeBSD versions support CPU capability detection
through AT_HWCAP and AT_HWCAP2 values stored in the auxiliary vector.
Updates #27619
Change-Id: I2a457b578d35101a7a5fd56ae9b81b300ad17da4
Reviewed-on: https://go-review.googlesource.com/c/go/+/165799
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Relative imports do not work in module mode. Use a fully-qualified
import path instead.
Updates #30228
Change-Id: I0a42ffa521a7b513395e7e1788022d24cbb1f31a
Reviewed-on: https://go-review.googlesource.com/c/go/+/165817
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
This commit adds external linking in cmd/link for aix/ppc64.
As relocations on .text data aren't possible on AIX, Segrelrodata is
used to move all these datas to .data section.
Change-Id: I4d1361c1fc9290e11e6f5560864460c76551dbeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/164003
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TestGdbAutotmpTypes takes more than one minute due to gdb performances.
Therefore, it must be skipped in short mode.
Change-Id: I253ebce62264cc7367c9b0f6ce9c5088a9994641
Reviewed-on: https://go-review.googlesource.com/c/go/+/164339
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This commit add port of runtime/cgo for aix/ppc64.
AIX assembly is different from Linux assembly, therefore gcc_ppc64.S
must be redone for AIX.
Change-Id: I780ebab4ef9c4ce912f4c4d521d8c135b1eebf6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/164002
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, the pacer assumes the goal growth ratio is always exactly
GOGC/100. But sometimes this isn't the case, like when the heap is
very small (limited by heapminimum). So to placate the pacer, we lie
about the value of heap_marked in such situations.
Right now, these two lies make a truth, but GOGC is about to get more
complicated with the introduction of heap limits.
Rather than introduce more lies into the system to handle this,
introduce the concept of an "effective GOGC", which is the GOGC we're
actually using for pacing (we'll need this concept anyway with heap
limits). This commit changes the pacer to use the effective GOGC
rather than the user-set GOGC. This way, we no longer need to lie
about heap_marked because its true value is incorporated into the
effective GOGC.
Change-Id: I5b005258f937ab184ffcb5e76053abd798d542bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/66092
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, the minimum sweep distance is 1MB * GOGC/100. It's been
this way since it was introduced in CL 13043 with no justification.
Since there seems to be no good reason to scale the minimum sweep
distance by GOGC, make the minimum sweep distance just 1MB.
Change-Id: I5320574a23c0eec641e346282aab08a3bbb057da
Reviewed-on: https://go-review.googlesource.com/c/go/+/66091
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
This slightly rearranges gcSetTriggerRatio to compute the goal before
computing the other controls. This will simplify implementing the heap
limit, which needs to control the absolute goal and flow the rest of
the control parameters from this.
For #16843.
Change-Id: I46b7c1f8b6e4edbee78930fb093b60bd1a03d75e
Reviewed-on: https://go-review.googlesource.com/c/go/+/46750
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
This was used during the implementation of concurrent runtime.GC() but
now there's nothing that triggers GC unconditionally. Remove this
trigger type and simplify (gcTrigger).test() accordingly.
Change-Id: I17a893c2ed1f661b8146d7783d529f71735c9105
Reviewed-on: https://go-review.googlesource.com/c/go/+/66090
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
With cgo, some syscalls will be called with g == nil or m == nil.
SyscallX functions cannot handle them so they call an equivalent
function in sys_aix_ppc64.s which will directly call this syscall.
Change-Id: I6508ec772b304111330e6833e7db729200af547c
Reviewed-on: https://go-review.googlesource.com/c/go/+/164001
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I had been finding these over a year or so, but none were big enough
changes to warrant CLs. They're a handful now, so clean them all up in a
single commit.
The smaller bodies get a bit simpler, but most importantly, the larger
bodies get unindented.
Change-Id: I5707a6fee27d4c9ff9efd3d363af575d7a4bf2aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/165340
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The commit fixes asmcgocall in order to use the AIX C ABI.
Change-Id: I2a44914a65557a841ea1e12991938af26ad7fd1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/164000
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently on darwin we use MADV_FREE, which unfortunately doesn't result
in a change in the process's RSS until pages actually get kicked out,
which the OS is free to do lazily (e.g. until it finds itself under
memory pressure).
To remedy this, we instead use MADV_FREE_REUSABLE which has similar
semantics, except that it also sets a reusable bit on each page so the
process's RSS gets reported more accurately. The one caveat is for every
time we call MADV_FREE_REUSABLE on a region we must call MADV_FREE_REUSE
to keep the kernel's accounting updated.
Also, because this change requires adding new constants that only exist
on darwin, it splits mem_bsd.go into mem_bsd.go and mem_darwin.go.
Fixes#29844.
Change-Id: Idb6421698511138a430807bcbbd1516cd57557c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/159117
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The names of some instructions have been updated in the WebAssembly
specification to be more consistent, see
994591e51c.
This change to the spec is possible because it is still in a draft
state.
Go's support for WebAssembly is still experimental and thus excempt from
the compatibility promise. Being consistent with the spec should
warrant this breaking change to the assembly instruction names.
Change-Id: Iafb8b18ee7f55dd0e23c6c7824aa1fad43117ef1
Reviewed-on: https://go-review.googlesource.com/c/153797
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
gogrep found only one such case with the pattern below:
$tmp := $x; $x = $y; $y = $tmp
R=1.13
Change-Id: I6e46fb5ef2887f24fa9fc451323a8cef272e2886
Reviewed-on: https://go-review.googlesource.com/c/151200
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
With stack objects, when we scan the stack, it scans defers with
tracebackdefers, but it seems to me that tracebackdefers doesn't
include the func value itself, which could be a stack allocated
closure. Scan it explicitly.
Alternatively, we can change tracebackdefers to include the func
value, which in turn needs to change the type of stkframe.
Fixes#30453.
Change-Id: I55a6e43264d6952ab2fa5c638bebb89fdc410e2b
Reviewed-on: https://go-review.googlesource.com/c/164118
Reviewed-by: Keith Randall <khr@golang.org>
to be used by errors package for checking assignability
and setting error values in As.
Updates #29934.
Change-Id: I8c1d02a2c6efa0919d54b286cfe8b4edc26da059
Reviewed-on: https://go-review.googlesource.com/c/161759
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
The number of CPUs reported by the hw.ncpu sysctl is twice as high as
the actual number of CPUs running on OpenBSD 6.4. with hyperthreading
disabled (hw.smt=0). Try hw.cpuonline first and fall back to hw.ncpu
in case it fails (which is the case on older OpenBSD before 6.4).
Fixes#30127
Change-Id: Id091234b8038cc9f7c40519d039fc1a05437c40d
Reviewed-on: https://go-review.googlesource.com/c/161757
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
syscall.NewCallback mistakenly used MOVL even for windows/amd64,
which only returned the lower 32 bits regardless of the architecture.
This was due to a copy and paste after porting from windows/386.
The code now uses MOVQ, which will return all the available bits.
Also adjust TestReturnAfterStackGrowInCallback to ensure we never
regress.
Fixes#29331
Change-Id: I4f5c8021c33f234c2bb7baa9ef7a6b4870172509
Reviewed-on: https://go-review.googlesource.com/c/159579
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
In module mode, building the current directory requires a go.mod file
(in order to determine the import path of the package).
Change the tests to pass explicit file arguments instead, since those
can be built in module mode without defining a module.
Updates #30228
Change-Id: I680c658d1f79645f73ad4d1e88189ea50a4852e9
Reviewed-on: https://go-review.googlesource.com/c/162837
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Allow shifts by signed amounts. Panic if the shift amount is negative.
TODO: We end up doing two compares per shift, see Ian's comment
https://github.com/golang/go/issues/19113#issuecomment-443241799 that
we could do it with a single comparison in the normal case.
The prove pass mostly handles this code well. For instance, it removes the
<0 check for cases like this:
if s >= 0 { _ = x << s }
_ = x << len(a)
This case isn't handled well yet:
_ = x << (y & 0xf)
I'll do followon CLs for unhandled cases as needed.
Update #19113
R=go1.13
Change-Id: I839a5933d94b54ab04deb9dd5149f32c51c90fa1
Reviewed-on: https://go-review.googlesource.com/c/158719
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
In runtime.gopanic, the _panic object p is stack allocated and
referenced from gp._panic. With stack objects, p on stack is dead
at the point preprintpanics runs. gp._panic points to p, but
stack scan doesn't look at gp. Heap scan of gp does look at
gp._panic, but it stops and ignores the pointer as it points to
the stack. So whatever p points to may be collected and clobbered.
We need to scan gp._panic explicitly during stack scan.
To test it reliably, we introduce a GODEBUG mode "clobberfree",
which clobbers the memory content when the GC frees an object.
Fixes#30150.
Change-Id: I11128298f03a89f817faa221421a9d332b41dced
Reviewed-on: https://go-review.googlesource.com/c/161778
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
When scavenging small amounts it's possible we over-scavenge by a
significant margin since we choose to scavenge the largest spans first.
This over-scavenging is never accounted for.
With this change, we add a scavenge credit pool, similar to the reclaim
credit pool. Any time scavenging triggered by RSS growth starts up, it
checks if it can cash in some credit first. If after using all the
credit it still needs to scavenge, then any extra it does it adds back
into the credit pool.
This change mitigates the performance impact of golang.org/cl/159500 on
the Garbage benchmark. On Go1 it suggests some improvements, but most of
that is within the realm of noise (Revcomp seems very sensitive to
GC-related changes, both postively and negatively).
Garbage: https://perf.golang.org/search?q=upload:20190131.5
Go1: https://perf.golang.org/search?q=upload:20190131.4
Performance change with both changes:
Garbage: https://perf.golang.org/search?q=upload:20190131.7
Go1: https://perf.golang.org/search?q=upload:20190131.6
Change-Id: I87bd3c183e71656fdafef94714194b9fdbb77aa2
Reviewed-on: https://go-review.googlesource.com/c/160297
Reviewed-by: Austin Clements <austin@google.com>
Because scavenged and unscavenged spans no longer coalesce, memory that
is freed no longer has a high likelihood of being re-scavenged. As a
result, if an application is allocating at a fast rate, it may work fast
enough to undo all the scavenging work performed by the runtime's
current scavenging mechanisms. This behavior is exacerbated by the
global best-fit allocation policy the runtime uses, since scavenged
spans are just as likely to be chosen as unscavenged spans on average.
To remedy that, we treat each allocation of scavenged space as a heap
growth, and scavenge other memory to make up for the allocation.
This change makes performance of the runtime slightly worse, as now
we're scavenging more often during allocation. The regression is
particularly obvious with the garbage benchmark (3%) but most of the Go1
benchmarks are within the margin of noise. A follow-up change should
help.
Garbage: https://perf.golang.org/search?q=upload:20190131.3
Go1: https://perf.golang.org/search?q=upload:20190131.2
Updates #14045.
Change-Id: I44a7e6586eca33b5f97b6d40418db53a8a7ae715
Reviewed-on: https://go-review.googlesource.com/c/159500
Reviewed-by: Austin Clements <austin@google.com>
Remove an unnecessary check on the heap sampling code that forced sampling
of all heap allocations larger than the sampling rate. This need to follow
a poisson process so that they can be correctly unsampled. Maintain a check
for MemProfileRate==1 to provide a mechanism for full sampling, as
documented in https://golang.org/pkg/runtime/#pkg-variables.
Additional testing for this change is on cl/129117.
Fixes#26618
Change-Id: I7802bde2afc655cf42cffac34af9bafeb3361957
GitHub-Last-Rev: 471f747af8
GitHub-Pull-Request: golang/go#29791
Reviewed-on: https://go-review.googlesource.com/c/158337
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
As a result of changes earlier in Go 1.12, the scavenger became much
more aggressive. In particular, when scavenged and unscavenged spans
coalesced, they would always become scavenged. This resulted in most
spans becoming scavenged over time. While this is good for keeping the
RSS of the program low, it also causes many more undue page faults and
many more calls to madvise.
For most applications, the impact of this was negligible. But for
applications that repeatedly grow and shrink the heap by large amounts,
the overhead can be significant. The overhead was especially obvious on
older versions of Linux where MADV_FREE isn't available and
MADV_DONTNEED must be used.
This change makes it so that scavenged spans will never coalesce with
unscavenged spans. This results in fewer page faults overall. Aside
from this, the expected impact of this change is more heap growths on
average, as span allocations will be less likely to be fulfilled. To
mitigate this slightly, this change also coalesces spans eagerly after
scavenging, to at least ensure that all scavenged spans and all
unscavenged spans are coalesced with each other.
Also, this change adds additional logic in the case where two adjacent
spans cannot coalesce. In this case, on platforms where the physical
page size is larger than the runtime's page size, we realign the
boundary between the two adjacent spans to a physical page boundary. The
advantage of this approach is that "unscavengable" spans, that is, spans
which cannot be scavenged because they don't cover at least a single
physical page are grown to a size where they have a higher likelihood of
being discovered by the runtime's scavenging mechanisms when they border
a scavenged span. This helps prevent the runtime from accruing pockets
of "unscavengable" memory in between scavenged spans, preventing them
from coalescing.
We specifically choose to apply this logic to all spans because it
simplifies the code, even though it isn't strictly necessary. The
expectation is that this change will result in a slight loss in
performance on platforms where the physical page size is larger than the
runtime page size.
Update #14045.
Change-Id: I64fd43eac1d6de6f51d7a2ecb72670f10bb12589
Reviewed-on: https://go-review.googlesource.com/c/158078
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently the code surrounding coalescing is duplicated between merging
with the span before the span considered for coalescing and merging with
the span after. This change factors out the shared portions of these
codepaths into a local closure which acts as a helper.
Change-Id: I7919fbed3f9a833eafb324a21a4beaa81f2eaa91
Reviewed-on: https://go-review.googlesource.com/c/158077
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The coalescing process is complex and in a follow-up change we'll need
to do it in more than one place, so this change factors out the
coalescing code in freeSpanLocked into a method on mheap.
Change-Id: Ia266b6cb1157c1b8d3d8a4287b42fbcc032bbf3a
Reviewed-on: https://go-review.googlesource.com/c/157838
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reuse the strict mechanism from FileLine for FuncForPC, so we don't
crash when asking the pcln table about bad pcs.
Fixes#29735
Change-Id: Iaffb32498b8586ecf4eae03823e8aecef841aa68
Reviewed-on: https://go-review.googlesource.com/c/157799
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change makes mTreap's iterator type, treapIter, bidirectional
instead of unidirectional. This change helps support moving the find
operation on a treap to return an iterator instead of a treapNode, in
order to hide the details of the treap when accessing elements.
For #28479.
Change-Id: I5dbea4fd4fb9bede6e81bfd089f2368886f98943
Reviewed-on: https://go-review.googlesource.com/c/156918
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This commit allows to cross-compiling aix/ppc64. The nosplit limit must
twice as large as on others platforms because of AIX syscalls.
The stack limit, especially stackGuardMultiplier, was set by cmd/dist
during the bootstrap and doesn't depend on GOOS/GOARCH target.
Fixes#29572
Change-Id: Id51e38885e1978d981aa9e14972eaec17294322e
Reviewed-on: https://go-review.googlesource.com/c/157117
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Returning the innermost frame instead of the outermost
makes code that walks the results of runtime.Caller{,s}
still work correctly in the presence of mid-stack inlining.
Fixes#29582
Change-Id: I2392e3dd5636eb8c6f58620a61cef2194fe660a7
Reviewed-on: https://go-review.googlesource.com/c/156364
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In 1.11 we stored "return addresses" in the result of runtime.Callers.
I changed that behavior in CL 152537 to store an address in the call
instruction itself. This CL reverts that part of 152537.
The change in 152537 was made because we now store pcs of inline marks
in the result of runtime.Callers as well. This CL will now store the
address of the inline mark + 1 in the results of runtime.Callers, so
that the subsequent -1 done in CallersFrames will pick out the correct
inline mark instruction.
This CL means that the results of runtime.Callers can be passed to
runtime.FuncForPC as they were before. There are a bunch of packages
in the wild that take the results of runtime.Callers, subtract 1, and
then call FuncForPC. This CL keeps that pattern working as it did in
1.11.
The changes to runtime/pprof in this CL are exactly a revert of the
changes to that package in 152537 (except the locForPC comment).
Update #29582
Change-Id: I04d232000fb482f0f0ff6277f8d7b9c72e97eb48
Reviewed-on: https://go-review.googlesource.com/c/156657
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The in-tree GDB is too old (6.1.1) on all the builders except the
FreeBSD 12.0 one, where it was removed from the base system.
Update #29508
Change-Id: Ib6091cd86440ea005f3f903549a0223a96621a6f
Reviewed-on: https://go-review.googlesource.com/c/156717
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
With gccgo, if a profiling signal arrives in certain time during
traceback, it may crash or hang. The fix is CL 156037 and
CL 156038. This CL adds a test.
Updates #29448.
Change-Id: Idb36af176b4865b8fb31a85cad185ed4c07ade0c
Reviewed-on: https://go-review.googlesource.com/c/156018
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We still don't understand what's causing there to be remaining GC work
when we enter mark termination, but in order to move forward on this
issue, this CL implements a work-around for the problem.
If debugCachedWork is false, this CL does a second check for remaining
GC work as soon as it stops the world for mark termination. If it
finds any work, it starts the world again and re-enters concurrent
mark. This will increase STW time by a small amount proportional to
GOMAXPROCS, but fixes a serious correctness issue.
This works-around #27993.
Change-Id: Ia23b85dd6c792ee8d623428bd1a3115631e387b8
Reviewed-on: https://go-review.googlesource.com/c/156140
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
As a followon to CL 152537, modify the panic-printing traceback
to also handle mid-stack inlining correctly.
Also declare -fm functions (aka method functions) as wrappers, so that
they get elided during traceback. This fixes part 2 of #26839.
Fixes#28640Fixes#24488
Update #26839
Change-Id: I1c535a9b87a9a1ea699621be1e6526877b696c21
Reviewed-on: https://go-review.googlesource.com/c/153477
Reviewed-by: David Chase <drchase@google.com>
Use the length of the bitmap to decide how much to pass to the
write barrier, not the total length of the arguments.
The test needs enough arguments so that two distinct bitmaps
get interpreted as a single longer bitmap.
Update #29362
Change-Id: I78f3f7f9ec89c2ad4678f0c52d3d3def9cac8e72
Reviewed-on: https://go-review.googlesource.com/c/156123
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
After CL 31455, "go fun(n)" may put "n" to write barrier buffer
when there are no pointers in fun's arguments.
Fixes#29362
Change-Id: Icfa42b8759ce8ad9267dcb3859c626feb6fda381
Reviewed-on: https://go-review.googlesource.com/c/155779
Reviewed-by: Keith Randall <khr@golang.org>
In the twoNonZero function in hash_test, the buffer is sliced as [:] three times. This change deletes them.
Change-Id: I0701d0c810b4f3e267f80133a0dcdb4ed81fe356
Reviewed-on: https://go-review.googlesource.com/c/156138
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently it's possible for the runtime to deadlock if checkPut is
called in a non-preemptible context. In this case, checkPut may spin,
so it won't leave the non-preemptible context, but the thread running
gcMarkDone needs to preempt all of the goroutines before it can
release the checkPut spin loops.
Fix this by returning from checkPut if it's called under any of the
conditions that would prevent gcMarkDone from preempting it. In this
case, it leaves a note behind that this happened; if the runtime does
later detect left-over work it can at least indicate that it was
unable to catch it in the act.
For #27993.
Updates #29385 (may fix it).
Change-Id: Ic71c10701229febb4ddf8c104fb10e06d84b122e
Reviewed-on: https://go-review.googlesource.com/c/156017
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
A notetsleepg may get stuck if its timeout callback gets invoked
exactly on its deadline due to low precision of nanotime. This change
fixes the comparison so it also resolves the note if the timestamps are
equal.
Updates #28975
Change-Id: I045d2f48b7f41cea0caec19b56876e9de01dcd6c
Reviewed-on: https://go-review.googlesource.com/c/153558
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Use EnumTimeFormatsEx() to test panics across callback boundaries
instead of EnumWindows(). EnumWindows() is incompatible with Go's panic
unwinding mechanism. See the associated issue for more information.
Updates #26148
Change-Id: If1dd70885d9c418b980b6827942cb1fd16c73803
Reviewed-on: https://go-review.googlesource.com/c/155923
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reorg map flags a bit so we don't need any extra space for the extra flag.
Fixes#23734
Change-Id: I436812156240ae90de53d0943fe1aabf3ea37417
Reviewed-on: https://go-review.googlesource.com/c/155918
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Work involved in getting a stack trace is divided between
runtime.Callers and runtime.CallersFrames.
Before this CL, runtime.Callers returns a pc per runtime frame.
runtime.CallersFrames is responsible for expanding a runtime frame
into potentially multiple user frames.
After this CL, runtime.Callers returns a pc per user frame.
runtime.CallersFrames just maps those to user frame info.
Entries in the result of runtime.Callers are now pcs
of the calls (or of the inline marks), not of the instruction
just after the call.
Fixes#29007Fixes#28640
Update #26320
Change-Id: I1c9567596ff73dc73271311005097a9188c3406f
Reviewed-on: https://go-review.googlesource.com/c/152537
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This change splits a testprog out of TestLockOSThreadExit and makes it
its own test. Then, this change makes the testprog exit prematurely with
a special message if unshare fails with EPERM because not all of the
builders allow the user to call the unshare syscall.
Also, do some minor cleanup on the TestLockOSThread* tests.
Fixes#29366.
Change-Id: Id8a9f6c4b16e26af92ed2916b90b0249ba226dbe
Reviewed-on: https://go-review.googlesource.com/c/155437
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When a locked M has its G exit without calling UnlockOSThread, then
lockedExt on it was getting cleared. Unfortunately, this meant that
during P handoff, if a new M was started, it might get forked (on
most OSs besides Windows) from the locked M, which could have kernel
state attached to it.
To solve this, just don't clear lockedExt. At the point where the
locked M has its G exit, it will also exit in accordance with the
LockOSThread API. So, we can safely assume that it's lockedExt state
will no longer be used. For the case of the main thread where it just
gets wedged instead of exiting, it's probably better for it to keep
the locked marker since it more accurately represents its state.
Fixed#28979.
Change-Id: I7d3d71dd65bcb873e9758086d2cbcb9a06429b0f
Reviewed-on: https://go-review.googlesource.com/c/153078
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This change disables the test TestArenaCollision on Darwin in race mode
to deal with the fact that Darwin 10.10 must use MAP_FIXED in race mode
to ensure we retain our heap in a particular portion of the address
space which the race detector needs. The test specifically checks to
make sure a manually mapped region's space isn't re-used, which is
definitely possible with MAP_FIXED because it replaces whatever mapping
already exists at a given address.
This change then also makes it so that MAP_FIXED is only used in race
mode and on Darwin, not all BSDs, because using MAP_FIXED breaks this
test for FreeBSD in addition to Darwin.
Updates #26475.
Fixes#29340.
Change-Id: I1c59349408ccd7eeb30c4bf2593f48316b23ab2f
Reviewed-on: https://go-review.googlesource.com/c/155097
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reverts change https://golang.org/cl/154758.
Restore the previous implementations of nanotime and time.now, which
are sufficiently high resolution and more efficient than
QueryPerformanceCounter. The intent of the change was to improve
resolution of tracing timestamps, but the change was overly broad
as it was only necessary to fix cputicks(). cputicks() is fixed in
a subsequent change.
Updates #26148
Change-Id: Ib9883d02fe1af2cc4940e866d8f6dc7622d47781
Reviewed-on: https://go-review.googlesource.com/c/154761
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
startpanic_m could be called correctly in a context where there's a
valid G, a valid M, but no P, for example in a signal handler which
panics. Currently, startpanic_m has write barriers enabled because
write barriers are permitted if a G's M is dying. However, all the
current write barrier implementations assume the current G has a P.
Therefore, in this change we disable write barriers in startpanic_m,
remove the only pointer write which clears g.writebuf, and fix up gwrite
to ignore the writebuf if the current G's M is dying, rather than
relying on it being nil in the dying case.
Fixes#26575.
Change-Id: I9b29e6b9edf00d8e99ffc71770c287142ebae086
Reviewed-on: https://go-review.googlesource.com/c/154837
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The previous implementation of nanotime and time.now used a time source
that was updated on the system clock tick, which has a maximum
resolution of about 1ms. On 386 and amd64, this time source maps to
the system performance counter, so has much higher resolution.
On ARM, use QueryPerformanceCounter() to get a high resolution timestamp.
Updates #26148
Change-Id: I1abc99baf927a95b472ac05020a7788626c71d08
Reviewed-on: https://go-review.googlesource.com/c/154758
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change makes it so that reserving more of the address space for the
heap calls mmap with MAP_FIXED in race mode. Race mode requires certain
guarantees on where the heap is located in the address space, and on
Darwin 10.10 it appears that the kernel may end up ignoring the hint
quite often (#26475). Using MAP_FIXED is relatively OK in race mode
because nothing else should be mapped in the memory region provided by
the initial hints.
Fixes#26475.
Change-Id: Id7ac1534ee74f6de491bc04441f27dbda09f0285
Reviewed-on: https://go-review.googlesource.com/c/153897
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This commit fixes backtrace if a crash or an exit signal is received
during a C syscall on aix/ppc64.
This is similar to Solaris, Darwin or Windows implementation.
Change-Id: I6040c0b1577a9f5b298f58bd4ee6556258a135ef
Reviewed-on: https://go-review.googlesource.com/c/154718
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, we flush the write barrier buffer on every write barrier
once throwOnGCWork is set, but not during the mark completion
algorithm itself. As seen in recent failures like
https://build.golang.org/log/317369853b803b4ee762b27653f367e1aa445ac1
by the time we actually catch a late gcWork put, the write barrier
buffer is full-size again.
As a result, we're probably not catching the actual problematic write
barrier, which is probably somewhere in the buffer.
Fix this by using the gcWork pause generation to also keep the write
barrier buffer small between the mark completion flushes it and when
mark completion is done.
For #27993.
Change-Id: I77618169441d42a7d562fb2a998cfaa89891edb2
Reviewed-on: https://go-review.googlesource.com/c/154638
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Add support for cgo on openbsd/arm.The gcc shipped with base OpenBSD armv7
is old/inadequate, so use clang by default.
Change-Id: I945a26d369378952d357727718e69249411e1127
Reviewed-on: https://go-review.googlesource.com/c/154381
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change modifies the behavior of span allocations to no longer
prefer the free treap over the scavenged treap.
While there is an additional cost to allocating out of the scavenged
treap, the current behavior of preferring the unscavenged spans can
lead to unbounded growth of a program's virtual memory footprint.
In small programs (low # of Ps, low resident set size, low allocation
rate) this behavior isn't really apparent and is difficult to
reproduce.
However, in relatively large, long-running programs we see this
unbounded growth in free spans, and an unbounded amount of heap
growths.
It still remains unclear how this policy change actually ends up
increasing the number of heap growths over time, but switching the
policy back to best-fit does indeed solve the problem.
Change-Id: Ibb88d24f9ef6766baaa7f12b411974cc03341e7b
Reviewed-on: https://go-review.googlesource.com/c/148979
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds the treapIter type which provides an iterator
abstraction for walking over an mTreap. In particular, the mTreap type
now has iter() and rev() for iterating both forwards (smallest to
largest) and backwards (largest to smallest). It also has an erase()
method for erasing elements at the iterator's current position.
For #28479.
While the expectation is that this change will slow down Go programs,
the impact on Go1 and Garbage is negligible.
Go1: https://perf.golang.org/search?q=upload:20181214.6
Garbage: https://perf.golang.org/search?q=upload:20181214.11
Change-Id: I60dbebbbe73cbbe7b78d45d2093cec12cc0bc649
Reviewed-on: https://go-review.googlesource.com/c/151537
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
sysUsed on Windows cares about the result from the VirtualAlloc syscall
returning exactly the address that was passed to it. However,
VirtualAlloc aligns the address its given to the kernel's allocation
granularity, so the returned address may not be the same.
Note that this wasn't an issue in the past because we only sysUsed
regions owned by spans, and spans are always a multiple of 8K, which
is a multiple of the allocation granularity on most Windows machines.
Change-Id: I3f5ccd63c6bbbd8b7995945ecedee17573b31667
Reviewed-on: https://go-review.googlesource.com/c/153677
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
This captures the stack trace where mark completion observed that each
P had no work, and then dumps this if that P later discovers more
work. Hopefully this will help bound where the work was created.
For #27993.
Change-Id: I4f29202880d22c433482dc1463fb50ab693b6de6
Reviewed-on: https://go-review.googlesource.com/c/154599
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Many of the crashes observed in #27993 involve committing the new
_defer object at the end of newdefer. It would be helpful to know if
the _defer was just allocated or was retrieved from the defer pool. In
order to indicate this in the traceback, this CL duplicates the tail
of newdefer so that the PC/line number will tell us whether d is new
or not.
For #27993.
Change-Id: Icd3e23dbcf00461877bb082b6f18df701149a607
Reviewed-on: https://go-review.googlesource.com/c/154598
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently we only know the slot address and the value being written in
the throwOnGCWork crash tracebacks, and we have to infer the old value
from what's dumped by gcWork.checkPut. Sometimes these old values
don't make sense, like when we see a write of a nil pointer to a
freshly-allocated object, yet we observe marking a value (where did
that pointer come from?).
This CL adds the old value of the slot and the first two pointers in
the buffer to the traceback.
For #27993.
Change-Id: Ib70eead1afb9c06e8099e520172c3a2acaa45f80
Reviewed-on: https://go-review.googlesource.com/c/154597
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently we reset the write barrier buffer before processing the
pointers in it. As a result, if there were any write barriers in the
code that processes the buffer, it would corrupt the write barrier
buffer and cause us to mark objects without later scanning them.
As far as I can tell, this shouldn't be happening, but rather than
relying on hope (and incomplete static analysis), this CL changes
wbBufFlush1 to poison the write barrier buffer while processing it,
and only reset it once it's done.
Updates #27993. (Unlike many of the other changes for this issue,
there's no need to roll back this CL. It's a good change in its own
right.)
Change-Id: I6d2d9f1b69b89438438b9ee624f3fff9f009e29d
Reviewed-on: https://go-review.googlesource.com/c/154537
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This fixes a few different issues that led to hangs and general
flakiness in the TestDebugCall* tests.
1. This fixes missing wake-ups in two error paths of the SIGTRAP
signal handler. If the goroutine was in an unknown state, or if
there was an unknown debug call status, we currently don't wake the
injection coordinator. These are terminal states, so this resulted
in a hang.
2. This adds a retry if the target goroutine is in a transient state
that prevents us from injecting a call. The most common failure
mode here is that the target goroutine is in _Grunnable, but this
was previously masked because it deadlocked the test.
3. Related to 2, this switches the "ready" signal from the target
goroutine from a blocking channel send to a non-blocking channel
send. This makes it much less likely that we'll catch this
goroutine while it's in the runtime performing that send.
4. This increases GOMAXPROCS from 2 to 8 during these tests. With the
current setting of 2, we can have at most the non-preemptible
goroutine we're injecting a call in to and the goroutine that's
trying to make it exit. If anything else comes along, it can
deadlock. One particular case I observed was in TestDebugCallGC,
where runtime.GC() returns before the forEachP that prepares
sweeping on all goroutines has finished. When this happens, the
forEachP blocks on the non-preemptible loop, which means we now
have at least three goroutines that need to run.
Fixes#25519.
Updates #29124.
Change-Id: I7bc41dc0b865b7d0bb379cb654f9a1218bc37428
Reviewed-on: https://go-review.googlesource.com/c/154112
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Replaces //go:linkname by assembly functions for syscall
functions on aix/ppc64.
Since the new runtime internal ABI, this was triggering an error if
syscall.Syscall6 was called by others packages like x/sys/unix.
This commit should remove every future occurences of this problem.
Fixes#28769
Change-Id: I6a4bf77472ee1e974bdb76b27e74275e568f5a76
Reviewed-on: https://go-review.googlesource.com/c/153997
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
The name "Callback" does not fit to all use cases of js.Callback.
This commit changes its name to Func. Accordingly NewCallback
gets renamed to FuncOf, which matches ValueOf and TypedArrayOf.
The package syscall/js is currently exempt from Go's compatibility
promise and js.Callback is already affected by a breaking change in
this release cycle. See #28711 for details.
Fixes#28711
Change-Id: I2c380970c3822bed6a3893909672c15d0cbe9da3
Reviewed-on: https://go-review.googlesource.com/c/153559
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Update sigcontext and siginfo structs to match those currently in use by OpenBSD armv7.
Also correct the offset of the fault address field in the siginfo struct, which moved
due to the switch to EABI.
Change-Id: Icdd95222346239fcc04b95ae0fcefae09b7aa044
Reviewed-on: https://go-review.googlesource.com/c/154077
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The getdirentries syscall is considered private API on iOS and is
rejected by the App Store submission checks. Replace it with the
fdopendir/readdir_r/closedir syscalls.
Fixes#28984
Change-Id: I73341b124310e9cb34834a95f946769f337ec5b7
Reviewed-on: https://go-review.googlesource.com/c/153338
Reviewed-by: Keith Randall <khr@golang.org>
TestCPUProfileLabel was failing on windows/arm because the link register
was not being passed to sigprof(). The link register is required to
generate a correct traceback. With this change, all tests in runtime.pprof
are now passing.
Updates #26148
Change-Id: Ia693b34278dc08a98023751ff1a922d9eee8fdd6
Reviewed-on: https://go-review.googlesource.com/c/153839
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This test is currently too flakey on openbsd/arm - ignore failures for the time
being.
Change-Id: Ia334d188f505167e691177ebe2c7a2df54bf556a
Reviewed-on: https://go-review.googlesource.com/c/153579
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Use smaller heap on Windows/ARM, which generally does not have
page file enabled and therefore has limited virtual address space.
Updates #26148
Signed-off-by: Jordan Rhee <jordanrh@microsoft.com>
Change-Id: I4030be24a10fab7b9b659e3736b7e83f10710bfa
Reviewed-on: https://go-review.googlesource.com/c/153719
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fix profiling handler to get the correct g for the m being profiled.
Store a pointer to the TLS slot holding g in the thread's m. This
enables the profiling handler to get the current g for the thread,
even if the thread is executing external code or system code.
Updates #26148
Signed-off-by: Jordan Rhee <jordanrh@microsoft.com>
Change-Id: Ie061284c12341c76c7d96cc0c2d5bac969230829
Reviewed-on: https://go-review.googlesource.com/c/153718
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The OpenBSD arm port switched to EABI in September 2016 - this revises the layout
of the runtime definitions to match what the kernel currently uses.
Change-Id: I1bca7de56979f576862a7c280631e835f7ae4278
Reviewed-on: https://go-review.googlesource.com/c/153577
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Without this, each additional C frame found via SetCgoTraceback will
cause a frame to be dropped from the bottom of the traceback stack.
Fixes#29034
Change-Id: I90aa6b2a1dced90c69b64c5dd565fe64a25724a3
Reviewed-on: https://go-review.googlesource.com/c/151917
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
After a recent change to runtime-gdb_test.go the ppc64le builder
has had intermittent failures. The failures occur when trying to
invoke the goroutineCmd function to display the backtrace for
a selected goroutine. There is nothing wrong with the testcase
but it seems to intermittently leave goroutines in a state
where an error can occur.
The error message indicates that the problem occurs when trying
to change the sp back to the original after displaying the
stacktrace for the goroutine.
gdb.error: Attempt to assign to an unmodifiable value.
After some searching I found that this error message can happen
if the sp register is changed when on a frame that is not the
top-most frame. To fix the problem, frame 0 is selected before
changing the value of sp. This fixes the problem in my
reproducer environment, and hopefully will fix the problem on
the builder.
Updates #28679
Change-Id: I329bc95b30f8c95acfb161b0d9cfdcbd917a1954
Reviewed-on: https://go-review.googlesource.com/c/152540
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A mark worker goroutine may attempt to preempt the mark termination
goroutine to scan its stack while the mark termination goroutine is
trying to preempt that worker to flush its work buffer, in rare
cases.
This change makes it so that, like a worker goroutine, the mark
termination goroutine stack is preemptible while it is on the
system stack, attempting to preempt others.
Fixes#28695.
Change-Id: I23bbb191f4fdad293e8a70befd51c9175f8a1171
Reviewed-on: https://go-review.googlesource.com/c/153077
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
In order to further diagnose #27993, I need to see exactly what
pointers are being added to the gcWork buffer too late.
Change-Id: I8d92113426ffbc6e55d819c39e7ab5eafa68668d
Reviewed-on: https://go-review.googlesource.com/c/152957
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The expression passed into isAbortPC call was written specifically
for windows/amd64 and windows/386 runtime.abort implementation.
Adjust the code, so it also works for windows/arm.
Fixes#29050
Change-Id: I3dc8ddd08031f34115396429eff512827264826f
Reviewed-on: https://go-review.googlesource.com/c/152357
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change fixes an error in HACKING.md which claims all pointers
which live in unmanaged memory but point to the heap must be marked
as GC roots explicitly by runtime.markroot. This isn't technically
necessary if the pointer is accessible through a global variable.
Change-Id: I632b25272fdb2f789c5259dd1685d517f45fd435
Reviewed-on: https://go-review.googlesource.com/c/151539
Reviewed-by: Rick Hudson <rlh@golang.org>
This adds several new checks to help debug #27993. It adds a mechanism
for freezing write barriers and gcWork puts during the mark completion
algorithm. This way, if we do detect mark completion, we can catch any
puts that happened during the completion algorithm. Based on build
dashboard failures, this seems to be the window of time when these are
happening.
This also double-checks that all work buffers are empty immediately
upon entering mark termination (much earlier than the current check).
This is unlikely to trigger based on the current failures, but is a
good safety net.
Change-Id: I03f56c48c4322069e28c50fbc3c15b2fee2130c2
Reviewed-on: https://go-review.googlesource.com/c/151797
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Adjust mTreap ordering logic to reflect the description of mTreap ordering.
Before it was using unsafe.Pointer in order to gather the base address of
a span. This has been changed to use base, which is the startAddress of a
span as the description is telling us in mgclarge.go.
Fixes: golang/go#28805
Change-Id: Ib3cd94a0757e23d135b5d41830f38fc08bcf16a3
GitHub-Last-Rev: 93f749b670
GitHub-Pull-Request: golang/go#28973
Reviewed-on: https://go-review.googlesource.com/c/151499
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
On XCOFF, it is forbidden relocation of a DATA pointer to a text
section. It happens when a RODATA symbol needs a DATA symbol's address.
This commit moves every RODATA symbols with a R_ADDR on a data symbol to
.data sections to avoid these relocations.
Change-Id: I7f34d8e0ebdc8352a74e6b40e4c893d8d9419f4d
Reviewed-on: https://go-review.googlesource.com/c/146977
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In assembly free packages (aka "complete" or "pure go"), allow
bodyless functions if they are linkname'd to something else.
Presumably the thing the function is linkname'd to has a definition.
If not, the linker will complain. And linkname is unsafe, so we expect
users to know what they are doing.
Note this handles only one direction, where the linkname directive
is in the local package. If the linkname directive is in the remote
package, this CL won't help. (See os/signal/sig.s for an example.)
Fixes#23311
Change-Id: I824361b4b582ee05976d94812e5b0e8b0f7a18a6
Reviewed-on: https://go-review.googlesource.com/c/151318
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The exception handler modifies the stack and continuation context so
it looks like the faulting code calls sigpanic() directly. The call was
not set up correctly on ARM, because it did not handle the link register
correctly. This change handles the link register correctly for ARM.
Updates #28854
Change-Id: I7ccf838adfc05cd968a5edd7d19ebba6a2478360
Reviewed-on: https://go-review.googlesource.com/c/150957
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This commit allows the runtime to handle 64bits addresses returned by
mmap syscall on AIX.
Mmap syscall returns addresses on 59bits on AIX. But the Arena
implementation only allows addresses with less than 48 bits.
This commit increases the arena size up to 1<<60 for aix/ppc64.
Update: #25893
Change-Id: Iea72e8a944d10d4f00be915785e33ae82dd6329e
Reviewed-on: https://go-review.googlesource.com/c/138736
Reviewed-by: Austin Clements <austin@google.com>
This adds a debug check to throw immediately if any pointers are added
to the gcWork buffer after the mark completion barrier. The intent is
to catch the source of the cached GC work that occasionally produces
"P has cached GC work at end of mark termination" failures.
The result should be that we get "throwOnGCWork" throws instead of "P
has cached GC work at end of mark termination" throws, but with useful
stack traces.
This should be reverted before the release. I've been unable to
reproduce this issue locally, but this issue appears fairly regularly
on the builders, so the intent is to catch it on the builders.
This probably slows down the GC slightly.
For #27993.
Change-Id: I5035e14058ad313bfbd3d68c41ec05179147a85c
Reviewed-on: https://go-review.googlesource.com/c/149969
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In race mode, these functions are defined and declared in
different packages, which therefore don't have implicit arg maps.
When they are defer'd, and the stack needs to move, the runtime
fails with missing stack maps. This CL adds arg maps (FUNCDATA)
to them.
Updates #28848
Change-Id: I0271563b7e78e7797ce2990c303dced957efaa86
Reviewed-on: https://go-review.googlesource.com/c/150457
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We should be able to build docker after this get applied.
Updates #13192
Change-Id: I5378d3518fac52d6bd4c97828884c1b382b7ace5
GitHub-Last-Rev: 210b7bc2e1
GitHub-Pull-Request: golang/go#28546
Reviewed-on: https://go-review.googlesource.com/c/146898
Reviewed-by: Jiang Ma <ma.jiang@zte.com.cn>
Reviewed-by: Clément Chigot <clement.chigot@atos.net>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 138675 added a call to runtime.save_g which uses thread local
storage to store g. On iOS however, that storage was not initialized
yet. Move the call to below _cgo_init where it is set up.
Change-Id: I14538d3e7d56ff35a6fa02c47bca306d24c38010
Reviewed-on: https://go-review.googlesource.com/c/150157
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The old whole-page reclaimer was the only thing that used the busy
span lists. Remove them so nothing uses them any more.
Change-Id: I4007dd2be08b9ef41bfdb0c387215c73c392cc4c
Reviewed-on: https://go-review.googlesource.com/c/138960
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
When we attempt to allocate an N page span (either for a large
allocation or when an mcentral runs dry), we first try to sweep spans
to release N pages. Currently, this can be extremely expensive:
sweeping a span to emptiness is the hardest thing to ask for and the
sweeper generally doesn't know where to even look for potentially
fruitful results. Since this is on the critical path of many
allocations, this is unfortunate.
This CL changes how we reclaim empty spans. Instead of trying lots of
spans and hoping for the best, it uses the newly introduced span marks
to efficiently find empty spans. The span marks (and in-use bits) are
in a dense bitmap, so these spans can be found with an efficient
sequential memory scan. This approach can scan for unmarked spans at
about 300 GB/ms and can free unmarked spans at about 32 MB/ms. We
could probably significantly improve the rate at which is can free
unmarked spans, but that's a separate issue.
Like the current reclaimer, this is still linear in the number of
spans that are swept, but the constant factor is now so vanishingly
small that it doesn't matter.
The benchmark in #18155 demonstrates both significant page reclaiming
delays, and object reclaiming delays. With "-retain-count=20000000
-preallocate=true -loop-count=3", the benchmark demonstrates several
page reclaiming delays on the order of 40ms. After this change, the
page reclaims are insignificant. The longest sweeps are still ~150ms,
but are object reclaiming delays. We'll address those in the next
several CLs.
Updates #18155.
Fixes#21378 by completely replacing the logic that had that bug.
Change-Id: Iad80eec11d7fc262d02c8f0761ac6998425c4064
Reviewed-on: https://go-review.googlesource.com/c/138959
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This adds a bitmap indexed by page number that marks the starts of
in-use spans. This will be used to quickly find in-use spans with no
marked objects for sweeping.
For #18155.
Change-Id: Icee56f029cde502447193e136fa54a74c74326dd
Reviewed-on: https://go-review.googlesource.com/c/138957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently, there's no efficient way to iterate over the Go heap. We're
going to need this for fast free page sweeping, so this CL adds a
slice of all allocated heap arenas. This will also be useful for
generational GC.
For #18155.
Change-Id: I58d126cfb9c3f61b3125d80b74ccb1b2169efbcc
Reviewed-on: https://go-review.googlesource.com/c/138076
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Change internal/cpu feature configuration to use
GODEBUG=cpu.feature1=value,cpu.feature2=value...
instead of GODEBUGCPU=feature1=value,feature2=value... .
This is not a backwards compatibility breaking change
since GODEBUGCPU was introduced in go1.11 as an
undocumented compiler experiment.
Fixes#28757
Change-Id: Ib21b3fed2334baeeb061a722ab1eb513d1137e87
Reviewed-on: https://go-review.googlesource.com/c/149578
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The current support_XXX variables are specific for the
amd64 and 386 platforms.
Prefix processor capability variables by architecture to have a
consistent naming scheme and avoid reuse of the existing
variables for new platforms.
This also aligns naming of runtime variables closer with internal/cpu
processor capability variable names.
Change-Id: I3eabb29a03874678851376185d3a62e73c1aff1d
Reviewed-on: https://go-review.googlesource.com/c/91435
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
When we set an explicit argmap, we may want only a prefix of that
argmap. Argmap is set when the function is reflect.makeFuncStub or
reflect.methodValueCall. In this case, arglen specifies how much of
the args section is actually live. (It could be either all the args +
results, or just the args.)
Fixes#28750
Change-Id: Idf060607f15a298ac591016994e58e22f7f92d83
Reviewed-on: https://go-review.googlesource.com/c/149217
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
When we delete an element, and it was the last element in the bucket,
update the slots between the new last element and the old last element
with the marker that says "no more elements beyond here".
Change-Id: I8efeeddf4c9b9fc491c678f84220a5a5094c9c76
Reviewed-on: https://go-review.googlesource.com/c/142438
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
When module is enabled, the go tool embeds build information
related to the module in the binary including the dependencies
and the replace information (See
src/cmd/go/internal/modload.PackageBuildInfo).
The newly introduced ReadBuildInfo reads the information and
makes it accessible programmatically.
Update #26404
Change-Id: Ide37022d609b4a8fb6b5ce02afabb73f04fbb532
Reviewed-on: https://go-review.googlesource.com/c/144220
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Changes include:
1. enable compiler option -race for arm64
2. add runtime/race_arm64.s to manage the calls from Go to the compiler-rt runtime
3. change racewalk.go to call racefuncenterfp instead of racefuncenter on arm64 to
allow the caller pc to be obtained in the asm code before calling the tsan version
4. race_linux_arm64.syso comes from compiler-rt which just supports 48bit VA, compiler-rt
is fetched from master branch which latest commit is 3aa2b775d08f903f804246af10b
Fixes#25682
Change-Id: I04364c580b8157fd117deecae74a4656ba16e005
Reviewed-on: https://go-review.googlesource.com/c/138675
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This adds the includes for errno.h to the windows stubs
for runtime/cgo so that "errno" is properly declared.
Due to "errno" not being properly declared, the compiler is
forced to assume it's an external which leaves it up to the
linker. This is an issue in some implementations as errno
might be a macro which results in an unresolved symbol error
during linking.
runtime/cgo/gcc_libinit_windows.c: added include
runtime/cgo/gcc_windows_386.c: added include
runtime/cgo/gcc_windows_amd64.c: added include
Change-Id: I77167d02f7409462979135efc55cf50bbc6bd363
GitHub-Last-Rev: 90da06ee3c
GitHub-Pull-Request: golang/go#28747
Reviewed-on: https://go-review.googlesource.com/c/149118
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There are three cases where we don't currently have the visibility to
get the ABIs of runtime symbols right, which this CL fixes:
1. For Go functions referenced from non-Go code in other packages.
This is runtime.morestackc (which is referenced from function
prologues) and a few syscall symbols. For these we need to generate
ABI0 wrappers, so this CL adds dummy calls in the assembly code to
force wrapper generation. There are many other cross-package
references to runtime and runtime/internal/atomic, but these are
handled specially by cmd/go.
2. For calls generated by the compiler to runtime Go functions, there
are a few symbols that aren't declared in builtins.go because we've
never needed their type information before. Now we at least need
their ABI information, so these are added to builtins.go.
3. For calls generated by the compiler to runtime assembly functions,
the compiler is going to assume the internal ABI is available, so
we add Go stubs to the runtime to trigger wrapper generation. For
these we're probably going to want to provide internal ABI
definitions directly in the assembly for performance, but for now
the ABIs are the same so it doesn't matter.
For #27539.
Change-Id: I9c224e7408d2ef4dd9b0e4c9d7e962ddfe111245
Reviewed-on: https://go-review.googlesource.com/c/146822
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The symbol runtime._cgo_panic_internal is defined both as a function
in package runtime and as a (linknamed) variable in package
runtime/cgo. Since we're introducing function ABIs, this is going to
cause problems with resolving the ABI-marked function symbol with the
unmarked data symbol. It's also confusing.
Fix this by declaring runtime._cgo_panic_internal as a function in
runtime/cgo as well and extracting the PC from the function object.
For #27539.
Change-Id: I148a458a600cf9e57791cf4cbe92e79bddbf58d4
Reviewed-on: https://go-review.googlesource.com/c/146821
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The internal/bytealg package defines several symbols in the runtime,
bytes, and strings packages in assembly, and the runtime package
defines symbols in reflect and sync/atomic. Currently, there's no
corresponding Go prototype for these symbols in the defining package.
We're going to start depending on Go prototypes in the same package as
their assembly definitions in order to provide ABI wrappers. Plus,
these are good documentation and colocate type information with
definitions, which could be useful for vet if it learned a little
about linkname.
This CL adds linknamed Go prototypes for all pushed symbols in
internal/bytealg and runtime.
For #27539.
Change-Id: I9b0c12d935a75bb6af46b6761180d451c00f11b8
Reviewed-on: https://go-review.googlesource.com/c/146820
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently, package runtime contains the definition of reflect.call,
even though it's just a jump to runtime.reflectcall. This "push"
symbol is confusing, since it's not clear where the definition of
reflect.call comes from when you're in the reflect package.
Replace this with a "pull" symbol: the runtime now defines only
runtime.reflectcall and package reflect uses a go:linkname to access
this symbol directly. This makes it clear where reflect.call is coming
from without any spooky action at a distance and eliminates all of the
definitions of reflect.call in the runtime.
Change-Id: I3ec73cd394efe9df8d3061a57c73aece2e7048dd
Reviewed-on: https://go-review.googlesource.com/c/148657
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This change fixes incorrect formatting in mheap.go (the result of my
previous heap scavenging changes) and map_test.go.
Change-Id: I2963687504abdc4f0cdf2f0c558174b3bc0ed2df
Reviewed-on: https://go-review.googlesource.com/c/148977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When a goroutine enters a syscall, its M unwires from its P to allow
the P to be retaken by another M if the syscall is slow. The M retains a
reference to its old P, however, so that if its old P has not been
retaken when the syscall returns, it can quickly reacquire that P.
The implementation, however, was confusing, as it left the reference to
the potentially-retaken P in m.p, which implied that the P was still
wired.
Make the code clearer by enforcing the invariant that m.p is never
stale. entersyscall now moves m.p to m.oldp and sets m.p to 0;
exitsyscall does the reverse, provided m.oldp has not been retaken.
With this scheme in place, the issue described in #27660 (assertion
failures in the race detector) would have resulted in a clean segfault
instead of silently corrupting memory.
Change-Id: Ib3e03623ebed4f410e852a716919fe4538858f0a
Reviewed-on: https://go-review.googlesource.com/c/148899
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
With this change, callbacks returned by syscall/js.NewCallback
get executed synchronously. This is necessary for the APIs of
many JavaScript libraries.
A callback triggered during a call from Go to JavaScript gets executed
on the same goroutine. A callback triggered by JavaScript's event loop
gets executed on an extra goroutine.
Fixes#26045Fixes#27441
Change-Id: I591b9e85ab851cef0c746c18eba95fb02ea9e85b
Reviewed-on: https://go-review.googlesource.com/c/142004
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This issue was found by the new vet's nilness check. _defer was already
checked against nil, so don't check it again.
Change-Id: I78725eaec7234b262b3c941e06441ca57f82bdd9
Reviewed-on: https://go-review.googlesource.com/c/148917
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
cgocall could previously invoke the race detector on an M whose P had
been retaken. The race detector would attempt to use the P-local state
from this stale P, racing with the thread that was actually wired to
that P. The result was memory corruption of ThreadSanitizer's internal
data structures that presented as hard-to-understand assertion failures
and segfaults.
Reorder cgocall so that it always acquires a P before invoking the race
detector, and add a test that stresses the interaction between cgo and
the race detector to protect against future bugs of this kind.
Fixes#27660.
Change-Id: Ide93f96a23490314d6647547140e0a412a97f0d4
Reviewed-on: https://go-review.googlesource.com/c/148717
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
This change fixes a bug wherein freeing a scavenged span that didn't
coalesce with any neighboring spans would result in that span getting
scavenged again. This case may actually be a common occurance because
"freeing" span trimmings and newly-grown spans end up using the same
codepath. On systems where madvise is relatively expensive, this can
have a large performance impact.
This change also cleans up some of this logic in freeSpanLocked since
a number of factors made the coalescing code somewhat difficult to
reason about with respect to scavenging. Notably, the way the
needsScavenge boolean is handled could be better expressed and the
inverted conditions (e.g. !after.released) can make things even more
confusing.
Fixes#28595.
Change-Id: I75228dba70b6596b90853020b7c24fbe7ab937cf
Reviewed-on: https://go-review.googlesource.com/c/147559
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This change introduces two optimizations together,
one for recursive and one for non-recursive stacks.
For recursive stacks, we introduce the new entry
at the beginning of the cache, so it can be found first.
This adds an extra read and write.
While we're here, switch from fastrandn, which does a multiply,
to fastrand % n, which does a shift.
For non-recursive stacks, split the cache from [16]pcvalueCacheEnt
into [2][8]pcvalueCacheEnt, and add a very cheap associative lookup.
name old time/op new time/op delta
StackCopyPtr-8 118ms ± 1% 106ms ± 2% -9.56% (p=0.000 n=17+18)
StackCopy-8 95.8ms ± 1% 87.0ms ± 3% -9.11% (p=0.000 n=19+20)
StackCopyNoCache-8 135ms ± 2% 139ms ± 1% +3.06% (p=0.000 n=19+18)
During make.bash, the association function used has this return distribution:
percent count return value
53.23% 678797 1
46.74% 596094 0
It is definitely not perfect, but it is pretty good,
and that's all we need.
Change-Id: I2cabb1d26b99c5111bc28f427016a2a5e6c620fd
Reviewed-on: https://go-review.googlesource.com/c/110564
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This is a simple tweak to allow a bit more mid-stack inlining.
In cases like this:
func f() {
g()
}
We'd really like to inline f into its callers. It can't hurt.
We implement this optimization by making calls a bit cheaper, enough
to afford a single call in the function body, but not 2.
The remaining budget allows for some argument modification, or perhaps
a wrapping conditional:
func f(x int) {
g(x, 0)
}
func f(x int) {
if x > 0 {
g()
}
}
Update #19348
Change-Id: Ifb1ea0dd1db216c3fd5c453c31c3355561fe406f
Reviewed-on: https://go-review.googlesource.com/c/147361
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
Add unexported unlinkat, openat, and fstatat calls, so that
the internal/syscall/unix package can use them.
Change-Id: I1df81ecae6427211dd392ec68c9f020fe131a526
Reviewed-on: https://go-review.googlesource.com/c/148457
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
There are still some references to the bare Syscall functions
in the stdlib. I will root those out in a following CL.
(This CL is big enough as it is.)
Most are in vendor directories:
cmd/vendor/golang.org/x/sys/unix/
vendor/golang_org/x/net/route/syscall.go
syscall/bpf_bsd.go
syscall/exec_unix.go
syscall/flock.go
Update #17490
Change-Id: I69ab707811530c26b652b291cadee92f5bf5c1a4
Reviewed-on: https://go-review.googlesource.com/c/141639
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Elias Naur <elias.naur@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change cleans up references to MSpan, MCache, and MCentral in the
docs via a bunch of sed invocations to better reflect the Go names for
the equivalent structures (i.e. mspan, mcache, mcentral) and their
methods (i.e. MSpan_Sweep -> mspan.sweep).
Change-Id: Ie911ac975a24bd25200a273086dd835ab78b1711
Reviewed-on: https://go-review.googlesource.com/c/147557
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TestTracebackAncestors has a ~0.1% chance of failing with more
goroutines in the traceback than expected. This happens because
there's a window between each goroutine starting its child and that
goroutine actually exiting. The test captures its own stack trace
after everything is "done", but if this happens during that window, it
will include the goroutine that's in the process of being torn down.
Here's an example of such a failure:
https://build.golang.org/log/fad10d0625295eb79fa879f53b8b32b9d0596af8
This CL fixes this by recording the goroutines that are expected to
exit and removing them from the stack trace. With this fix, this test
passed 15,000 times with no failures.
Change-Id: I71e7c6282987a15e8b74188b9c585aa2ca97cbcd
Reviewed-on: https://go-review.googlesource.com/c/147517
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
As of 07e738e all spans are allocated out of a treap, and not just
large spans or spans for large objects. Also, now we have a separate
treap for spans that have been scavenged.
Change-Id: I9c2cb7b6798fc536bbd34835da2e888224fd7ed4
Reviewed-on: https://go-review.googlesource.com/c/142958
Reviewed-by: Austin Clements <austin@google.com>
zgoos_aix.go is missing GoosJs, the order of GoosAndroid and GoosAix is
mixed up in all files and GoosHurd was added after CL 146023 introduced
GOOS=hurd.
Change-Id: I7e2f5a15645272e9020cfca86e44c364fc072a2b
Reviewed-on: https://go-review.googlesource.com/c/147397
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The kernel on some Samsung S9+ models reports support for arm64 8.1
atomics, but in reality only some of the cores support them. Go
programs scheduled to cores without support will crash with SIGILL.
This change unconditionally disables the optimization on Android.
A better fix is to precisely detect the offending chipset.
Fixes#28431
Change-Id: I35a1273e5660603824d30ebef2ce7e429241bf1f
Reviewed-on: https://go-review.googlesource.com/c/147377
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Go documentation style for boolean funcs is to say:
// Foo reports whether ...
func Foo() bool
(rather than "returns true if")
This CL also replaces 4 uses of "iff" with the same "reports whether"
wording, which doesn't lose any meaning, and will prevent people from
sending typo fixes when they don't realize it's "if and only if". In
the past I think we've had the typo CLs updated to just say "reports
whether". So do them all at once.
(Inspired by the addition of another "returns true if" in CL 146938
in fd_plan9.go)
Created with:
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor)
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor)
Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f
Reviewed-on: https://go-review.googlesource.com/c/147037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There are some OpenGL functions that take more than 15 arguments.
This CL adds Syscall18 to enable to call such functions on Windows
via syscall functions.
Fixes#28434
Change-Id: Ic7e37dda9cadf4516183e98166bfc52844ad2bbe
Reviewed-on: https://go-review.googlesource.com/c/147117
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
This commit skips tests which aren't yet supported on AIX.
nosplit.go is disabled because stackGuardMultiplier is increased for
syscalls.
Change-Id: Ib5ff9a4539c7646bcb6caee159f105ff8a160ad7
Reviewed-on: https://go-review.googlesource.com/c/146939
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
This commit changes poll.PollDescriptor by poll.IsPollDescriptor. This
is needed for OS like AIX which have more than one FD using inside their
netpoll implementation.
Change-Id: I49e12a8d74045c501e19fdd8527cf166a3c64850
Reviewed-on: https://go-review.googlesource.com/c/146938
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We only need the memory barrier from these stores,
and we only store nil over nil or over a static function value.
The write barrier is unnecessary.
name old time/op new time/op delta
TCP4OneShotTimeout-6 17.0µs ± 0% 17.0µs ± 0% -0.43% (p=0.032 n=5+5)
SetReadDeadline-6 205ns ± 1% 205ns ± 1% ~ (p=0.683 n=5+5)
Update #25729
Change-Id: I66c097a1db7188697ddfc381f31acec053dfed2c
Reviewed-on: https://go-review.googlesource.com/c/146345
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtimeNano is slower than nanotime, so pass the duration
to runtime_pollSetDeadline as is. netpoll can add nanotime itself.
Arguably a bit simpler because, say, a negative duration
clearly represents already expired timer, no need to compare to
nanotime again.
This may also fix an obscure corner case when a deadline in past
which happens to be nanotime 0 is confused with no deadline at all,
which are radically different things.
Also don't compute any durations and times if Time is zero
(currently we first compute everything and then reset d back to 0,
which is wasteful).
name old time/op new time/op delta
TCP4OneShotTimeout-6 17.1µs ± 0% 17.0µs ± 0% ~ (p=0.421 n=5+5)
SetReadDeadline-6 230ns ± 0% 205ns ± 1% -10.63% (p=0.008 n=5+5)
Change-Id: I2aad699270289a5b9ead68f5e44ec4ec6d96baa0
Reviewed-on: https://go-review.googlesource.com/c/146344
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
We only need the memory barrier in poll_runtime_pollSetDeadline only
when one of the timers has fired, which is not the expected case.
Memory barrier can be somewhat expensive on some archs,
so execute it only if one of the timers has in fact fired.
name old time/op new time/op delta
TCP4OneShotTimeout-6 17.0µs ± 0% 17.1µs ± 0% +0.35% (p=0.032 n=5+5)
SetReadDeadline-6 232ns ± 0% 230ns ± 0% -1.03% (p=0.000 n=4+5)
Update #25729
Change-Id: Ifce6f505b9e7ba3717bad8f454077a2e94ea6e75
Reviewed-on: https://go-review.googlesource.com/c/146343
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Move startNano from runtime to time package.
In preparation for a subsequent change that speeds up Since and Until.
This also makes code simpler as we have less assembly as the result,
monotonic time handling is better localized in time package.
This changes values returned from nanotime on windows
(it does not account for startNano anymore), current comments state
that it's important, but it's unclear how it can be important
since no other OS does this.
Update #25729
Change-Id: I2275d57b7b5ed8fd0d53eb0f19d55a86136cc555
Reviewed-on: https://go-review.googlesource.com/c/146340
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently when netpoll deadline is incrementally prolonged,
we delete and re-add timer each time.
Add modtimer function that does both and use it when we need
to modify an existing netpoll timer to avoid unnecessary lock/unlock.
TCP4OneShotTimeout-6 17.2µs ± 0% 17.0µs ± 0% -0.82% (p=0.008 n=5+5)
SetReadDeadline-6 274ns ± 2% 261ns ± 0% -4.89% (p=0.008 n=5+5)
Update #25729
Change-Id: I08b89dbbc1785dd180e967a37b0aa23b0c4613a8
Reviewed-on: https://go-review.googlesource.com/c/146339
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently we always delete both read and write timers and then
add them again. However, if user setups read and write deadline
separately, then we don't need to touch the other one.
name old time/op new time/op delta
TCP4OneShotTimeout-6 17.2µs ± 0% 17.2µs ± 0% ~ (p=0.310 n=5+5)
SetReadDeadline-6 319ns ± 1% 274ns ± 2% -13.94% (p=0.008 n=5+5)
Update #25729
Change-Id: I4c869c3083521de6d0cd6ca99a7609d4dd84b4e4
Reviewed-on: https://go-review.googlesource.com/c/146338
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It's not always necessary to wake timerproc even if we add
a new timer to the top of the heap. Since we don't wake and
reset timerproc when we remove timers, it still can be sleeping
with shorter timeout. It such case it's more profitable to let it
sleep and then update timeout when it wakes on its own rather than
proactively wake it, let it update timeout and go to sleep again.
name old time/op new time/op delta
TCP4OneShotTimeout-6 18.6µs ± 1% 17.2µs ± 0% -7.66% (p=0.008 n=5+5)
SetReadDeadline-6 562ns ± 5% 319ns ± 1% -43.27% (p=0.008 n=5+5)
Update #25729
Change-Id: Iec8eacb8563dbc574a82358b3bac7ac479c16826
Reviewed-on: https://go-review.googlesource.com/c/146337
Reviewed-by: Ian Lance Taylor <iant@golang.org>
As discussed in golang.org/cl/28499:
Only test that all expected variables are listed in 'info locals' since
different versions of gdb print variables in different order and with
differing amount of information and formats.
Fixes#28499
Change-Id: I76627351170b5fdf2bf8cbf143e54f628b45dc4e
Reviewed-on: https://go-review.googlesource.com/c/146598
Reviewed-by: Heschi Kreinick <heschi@google.com>
AIX doesn't have HWCAP/HWCAP2 variables like Linux. Therefore, it relies on
getsystemcfg syscall which can provide some information about the CPU.
Change-Id: Ic0dc927e80890d4bf8f0bdfb43fad1e2b890d7a0
Reviewed-on: https://go-review.googlesource.com/c/144959
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
This lets []byte->string conversions which are used as arguments to
strings.IndexByte and friends have their backing store allocated on
the stack.
It only prevents allocation when the string is small enough (32
bytes), so it isn't perfect. But reusing the []byte backing store
directly requires a bunch more compiler analysis (see #2205 and
related issues).
Fixes#25864.
Change-Id: Ie52430422196e3c91e5529d6e56a8435ced1fc4c
Reviewed-on: https://go-review.googlesource.com/c/146018
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change introduces a test to malloc_test which checks for overuse
of physical memory in the large object treap. Due to fragmentation,
there may be many pages of physical memory that are sitting unused in
large-object space.
For #14045.
Change-Id: I3722468f45063b11246dde6301c7ad02ae34be55
Reviewed-on: https://go-review.googlesource.com/c/138918
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change scavenges the largest spans before growing the heap for
physical pages to "make up" for the newly-mapped space which,
presumably, will be touched.
In theory, this approach to scavenging helps reduce the RSS of an
application by marking fragments in memory as reclaimable to the OS
more eagerly than before. In practice this may not necessarily be
true, depending on how sysUnused is implemented for each platform.
Fixes#14045.
Change-Id: Iab60790be05935865fc71f793cb9323ab00a18bd
Reviewed-on: https://go-review.googlesource.com/c/139719
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently, we mark a whole span as sysUsed before trimming, but this
unnecessarily tells the OS that the trimmed section from the span is
used when it may have been scavenged, if s was scavenged. Overall,
this just makes invocations of sysUsed a little more fine-grained.
It does come with the caveat that now heap_released needs to be managed
a little more carefully in allocSpanLocked. In this case, we choose to
(like before this change) negate any effect the span has on
heap_released before trimming, then add it back if the trimmed part is
scavengable.
For #14045.
Change-Id: Ifa384d989611398bfad3ca39d3bb595a5962a3ea
Reviewed-on: https://go-review.googlesource.com/c/140198
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change extends the test function ReadMemStatsSlow to re-compute
the HeapReleased statistic such that it is checked in testing to be
consistent with the bookkeeping done in the runtime.
Change-Id: I49f5c2620f5731edea8e9f768744cf997dcd7c22
Reviewed-on: https://go-review.googlesource.com/c/142397
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change removes npreleased from mspan since spans may now either be
scavenged or not scavenged; how many of its pages were actually scavenged
doesn't matter. It saves some space in mpsan overhead too, as the boolean
fits into what would otherwise be struct padding.
For #14045.
Change-Id: I63f25a4d98658f5fe21c6a466fc38c59bfc5d0f5
Reviewed-on: https://go-review.googlesource.com/c/139737
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a new treap to mheap which contains scavenged (i.e.
its physical pages were returned to the OS) spans.
As of this change, spans may no longer be partially scavenged.
For #14045.
Change-Id: I0d428a255c6d3f710b9214b378f841b997df0993
Reviewed-on: https://go-review.googlesource.com/c/139298
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a method for computing a treap node's successor
to the treap, which will simplify the implementation of algorithms
used for heap growth scavenging.
For #14045.
Change-Id: If2af3f2707dbcbef5fb6e42cb2712061f9da5129
Reviewed-on: https://go-review.googlesource.com/c/144718
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a method for computing a treap node's predecessor
to the treap, which will simplify the implementation of algorithms
used for heap growth scavenging.
For #14045.
Change-Id: Id203e4bd246db3504f2f0c5163ec36f4579167df
Reviewed-on: https://go-review.googlesource.com/c/144717
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Only return a pointer p to the new slices backing array from makeslice.
Makeslice callers then construct sliceheader{p, len, cap} explictly
instead of makeslice returning the slice.
Reduces go binary size by ~0.2%.
Removes 92 (~3.5%) panicindex calls from go binary.
Change-Id: I29b7c3b5fe8b9dcec96e2c43730575071cfe8a94
Reviewed-on: https://go-review.googlesource.com/c/141822
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Adds AIX, DragonFly BSD, FreeBSD, NetBSD, OpenBSD and Solaris
to the list of operating systems where the GODEBUGCPU environment
variable will be parsed and interal/cpu features can be enabled
and disabled.
Updates #27218
Change-Id: I9cd99142e2a5147cb00ca57b581f049ea6ce8508
Reviewed-on: https://go-review.googlesource.com/c/145281
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This instruction was linked with a new stack layout which might be
needed for AIX. This change might not be taken finally. So, this
instruction must be removed.
See https://go-review.googlesource.com/c/go/+/138733
Change-Id: Ic4a2566e2882696b437eb817d980b7c4bfc03b18
Reviewed-on: https://go-review.googlesource.com/c/144957
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This changes the runtime asm code that loads iscgo to use MOVBZ
instead of MOVB, avoiding an unnecessary sign extension. This is most
significant in runtime.save_g, reducing the size from 8 to 7
instructions.
Change-Id: Iaa2121464b5309e1f27fd91b19b5603c7aaf619d
Reviewed-on: https://go-review.googlesource.com/c/144217
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This change creates the infrastructure for new lightweight atomics
primitives in runtime/internal/atomic:
- LoadAcq, for load-acquire
- StoreRel, for store-release
- CasRel, for Compare-and-Swap-release
and implements them for ppc64x. There is visible performance improvement
in producer-consumer scenarios, like BenchmarkChanProdCons*:
benchmark old ns/op new ns/op delta
BenchmarkChanProdCons0-48 2034 2034 +0.00%
BenchmarkChanProdCons10-48 1798 1608 -10.57%
BenchmarkChanProdCons100-48 1596 1585 -0.69%
BenchmarkChanProdConsWork0-48 2084 2046 -1.82%
BenchmarkChanProdConsWork10-48 1829 1668 -8.80%
BenchmarkChanProdConsWork100-48 1650 1650 +0.00%
Fixes#21348
Change-Id: I1f6ce377e4a0fe4bd7f5f775e8036f50070ad8db
Reviewed-on: https://go-review.googlesource.com/c/142277
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
All uses of these have been converted to use runtime/internal/math
functions for overflow checking.
Fixes#21588
Change-Id: I0ba57028e471803dc7d445e66d77a8f87edfdafb
Reviewed-on: https://go-review.googlesource.com/c/144037
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This improves performance for e.g. maps with a bucket size
(key+value*8 bytes) larger than 32 bytes and removes loading
a value from the maxElems array for smaller bucket sizes.
name old time/op new time/op delta
MakeMap/[Byte]Byte 95.5ns ± 1% 94.7ns ± 1% -0.78% (p=0.013 n=9+9)
MakeMap/[Int]Int 128ns ± 0% 121ns ± 2% -5.63% (p=0.000 n=6+10)
Updates #21588
Change-Id: I7d9eb7d49150c399c15dcab675e24bc97ff97852
Reviewed-on: https://go-review.googlesource.com/c/143997
Reviewed-by: Keith Randall <khr@golang.org>
This improves performance for maps with a bucket size
(key+value*8 bytes) larger than 32 bytes and removes loading
a value from the maxElems array for smaller bucket sizes.
name old time/op new time/op delta
MakeMap/[Byte]Byte 93.5ns ± 1% 91.8ns ± 1% -1.83% (p=0.000 n=10+10)
MakeMap/[Int]Int 134ns ± 1% 127ns ± 2% -5.61% (p=0.000 n=9+10)
Updates #21588
Change-Id: I53f77186769c4bd0f2b90f3c6c17df643b060e39
Reviewed-on: https://go-review.googlesource.com/c/143797
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit adds the change on asm_ppc64.s and tls_ppc64.s files for AIX
operating system.
R2 does not need to be set for aix/ppc64 since it should remain valid
througout Go execution, except after a call to a C function.
Moreover, g must always be saved on the tls as syscalls are made with
C functions.
Some modifications on asm_ppc64.s are due to AIX stack layout.
It also removes a useless part in asmcgocall which was done twice.
Change-Id: Ie037ab73da00562bb978f2d0f17fcdabd4a40aa2
Reviewed-on: https://go-review.googlesource.com/c/138735
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, span scavenging was done nearly identically in two different
locations. This change deduplicates that into one shared routine.
For #14045.
Change-Id: I15006b2c9af0e70b7a9eae9abb4168d3adca3860
Reviewed-on: https://go-review.googlesource.com/c/139297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
x = map[string(byteslice)] is already optimized by the compiler to avoid a
string allocation. This CL generalizes this optimization to:
x = map[T1{ ... Tn{..., string(byteslice), ...} ... }]
where T1 to Tn is a nesting of struct and array literals.
Found in a hot code path that used a struct of strings made from []byte
slices to make a map lookup.
There are no uses of the more generalized optimization in the standard library.
Passes toolstash -cmp.
MapStringConversion/32/simple 21.9ns ± 2% 21.9ns ± 3% ~ (p=0.995 n=17+20)
MapStringConversion/32/struct 28.8ns ± 3% 22.0ns ± 2% -23.80% (p=0.000 n=20+20)
MapStringConversion/32/array 28.5ns ± 2% 21.9ns ± 2% -23.14% (p=0.000 n=19+16)
MapStringConversion/64/simple 21.0ns ± 2% 21.1ns ± 3% ~ (p=0.072 n=19+18)
MapStringConversion/64/struct 72.4ns ± 3% 21.3ns ± 2% -70.53% (p=0.000 n=20+20)
MapStringConversion/64/array 72.8ns ± 1% 21.0ns ± 2% -71.13% (p=0.000 n=17+19)
name old allocs/op new allocs/op delta
MapStringConversion/32/simple 0.00 0.00 ~ (all equal)
MapStringConversion/32/struct 0.00 0.00 ~ (all equal)
MapStringConversion/32/array 0.00 0.00 ~ (all equal)
MapStringConversion/64/simple 0.00 0.00 ~ (all equal)
MapStringConversion/64/struct 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20)
MapStringConversion/64/array 1.00 ± 0% 0.00 -100.00% (p=0.000 n=20+20)
Change-Id: I483b4d84d8d74b1025b62c954da9a365e79b7a3a
Reviewed-on: https://go-review.googlesource.com/c/116275
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add generic, 386 and amd64 specific ops and SSA rules for multiplication
with overflow and branching based on overflow flags. Use these to intrinsify
runtime/internal/math.MulUinptr.
On amd64
mul, overflow := math.MulUintptr(a, b)
if overflow {
is lowered to two instructions:
MULQ SI
JO 0x10ee35c
No codegen tests as codegen can not currently test unexported internal runtime
functions.
amd64:
name old time/op new time/op delta
MulUintptr/small 1.16ns ± 5% 0.88ns ± 6% -24.36% (p=0.000 n=19+20)
MulUintptr/large 10.7ns ± 1% 1.1ns ± 1% -89.28% (p=0.000 n=17+19)
Change-Id: If60739a86f820e5044d677276c21df90d3c7a86a
Reviewed-on: https://go-review.googlesource.com/c/141820
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This CL adds a new internal math package for use by the runtime.
The new package exports a MulUintptr function with uintptr arguments
a and b and returns uintptr(a*b) and whether the full-width product
x*y does overflow the uintptr value range (uintptr(x*y) != x*y).
Uses of MulUinptr in the runtime and intrinsics for performance
will be added in followup CLs.
Updates #21588
Change-Id: Ia5a02eeabc955249118e4edf68c67d9fc0858058
Reviewed-on: https://go-review.googlesource.com/c/91755
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
When we pass these types by reference, we usually have to allocate
temporaries on the stack, initialize them, then pass their address
to the conversion functions. It's simpler to pass these types
directly by value.
This particularly applies to conversions needed for fmt.Printf
(to interface{} for constructing a [...]interface{}).
func f(a, b, c string) {
fmt.Printf("%s %s\n", a, b)
fmt.Printf("%s %s\n", b, c)
}
This function's stack frame shrinks from 200 to 136 bytes, and
its code shrinks from 535 to 453 bytes.
The go binary shrinks 0.3%.
Update #24286
Aside: for this function f, we don't really need to allocate
temporaries for the convT2E function. We could use the address
of a, b, and c directly. That might get similar (or maybe better?)
improvements. I investigated a bit, but it seemed complicated
to do it safely. This change was much easier.
Change-Id: I78cbe51b501fb41e1e324ce4203f0de56a1db82d
Reviewed-on: https://go-review.googlesource.com/c/135377
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Enabling GODEBUGCPU without the need to set GOEXPERIMENT=debugcpu enables
trybots and builders to run tests for GODEBUGCPU features in upcoming CLs
that will implement the new syntax and features for non-experimental
GODEBUGCPU support from proposal golang.org/issue/27218.
Updates #27218
Change-Id: Icc69e51e736711a86b02b46bd441ffc28423beba
Reviewed-on: https://go-review.googlesource.com/c/141817
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit adds AIX operating system to syscall package for ppc64
architecture.
It also adds the file syscall_aix.go in the runtime package for
syscalls needed during fork and exec.
Updates: #25893
Change-Id: I301b1051b178a3efb7bbc39cdbd8e00b594d65ef
Reviewed-on: https://go-review.googlesource.com/c/138720
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
AIX and Solaris both requires libc to make any syscalls and their
implementation is really similar.
Therefore, Solaris files reused by AIX have their name changed to *_libc.
exec_libc.go is also adapted to AIX.
Updates: #25893
Change-Id: I50d1d7b964831637013d5e64799187cd9565c42b
Reviewed-on: https://go-review.googlesource.com/c/138719
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gosweepdone is another anachronism from the time when the sweeper was
implemented in C. Rename it to "isSweepDone" for the modern era.
Change-Id: I8472aa6f52478459c3f2edc8a4b2761e73c4c2dd
Reviewed-on: https://go-review.googlesource.com/c/138658
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
gosweepone just switches to the system stack and calls sweepone.
sweepone doesn't need to run on the system stack, so this is pretty
pointless.
Historically, this was necessary because the sweeper was written in C
and hence needed to run on the system stack. gosweepone was the
function that Go code (specifically, bgsweep) used to call into the C
sweeper implementation. This probably became unnecessary in 2014 with
CL golang.org/cl/167540043, which ported the sweeper to Go.
This CL changes all callers of gosweepone to call sweepone and
eliminates gosweepone.
Change-Id: I26b8ef0c7d060b4c0c5dedbb25ecfc936acc7269
Reviewed-on: https://go-review.googlesource.com/c/138657
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
For unclear reasons, cacheSpan and uncacheSpan compute the number of
elements in a span by dividing its size by the element size. This
number is simply available in the mspan structure, so just use it.
Change-Id: If2e5de6ecec39befd3324bf1da4a275ad000932f
Reviewed-on: https://go-review.googlesource.com/c/138656
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Lazy mcache flushing (golang.org/cl/134783) made it so that moving a
span from an mcache to an mcentral was sometimes responsible for
sweeping the span. However, it did a "preserving" sweep, which meant
it retained ownership, even if the sweeper swept all objects in the
span. As a result, we could put a completely unused span back in the
mcentral.
Fix this by first taking back ownership of the span into the mcentral
and moving it to the right mcentral list, and then doing a
non-preserving sweep. The non-preserving sweep will move the span to
the heap if it sweeps all objects.
Change-Id: I244b1893b44b8c00264f0928ac9239449775f617
Reviewed-on: https://go-review.googlesource.com/c/140597
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Lazy mcache flushing (golang.org/cl/134783) introduced a second value
for sweepgen that indicates a span has been swept. I missed adding
this case to a sanity check in sweepone, so it can now panic if it
finds a non-in-use spans that's been swept *and* put in an mcache.
Fix this by adding the second sweepgen case to this check.
Fixes#27997.
Change-Id: I568d9f2cc8923396ca897a37d154cd2c859c7bef
Reviewed-on: https://go-review.googlesource.com/c/140697
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Since atomic.Or8 is now an intrinsic (and has been for some time),
markBits.setMarked is inlinable. Undo the manual inlining of it.
Change-Id: I8e37ccf0851ad1d3088d9c8ae0f6f0c439d7eb2d
Reviewed-on: https://go-review.googlesource.com/c/138659
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
freeSpan currently takes a mysterious "acct int32" argument. This is
really just a boolean and actually just needs to match the "large"
argument to alloc in order to balance out accounting.
To make this clearer, replace acct with a "large bool" argument that
must match the call to mheap.alloc.
Change-Id: Ibc81faefdf9f0583114e1953fcfb362e9c3c76de
Reviewed-on: https://go-review.googlesource.com/c/138655
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reverts commit d217004061.
Reason for revert: It broke all the darwin builders; it's also not
obvious how the weird darwin versions (900, 1000) relate to the > 3.9
requisite, so I'm not sure how to decide about skipping in a robust
way. It's better to revert the check for now.
Fixes#28028
Change-Id: Ibbcb7bf7cd2136e0851ebd097a2bc4dec9f0ee18
Reviewed-on: https://go-review.googlesource.com/c/140217
Reviewed-by: Keith Randall <khr@golang.org>
This is similar to CL 136816 for x/sys/unix, changing the FreeBSD ABI to use 64-bit inodes in
Stat_t, Statfs_t, and Dirent types.
The changes are forward compatible, that is FreeBSD 10.x, 11.x continue to use their current sysnum numbers.
The affected types are converted to the new layout (with some overhead).
Thus the same statically linked binary should work using the native sysnums (without any conversion) on FreeBSD 12.
Breaking API changes in package syscall are:
Mknod takes a uint64 (C dev_t) instead of int.
Stat_t: Dev, Ino, Nlink, Rdev, Gen became uint64.
Atimespec, Mtimespec, Ctimespec, Birthtimespec renamed to Atim, Mtim, Ctim, Birthtim respectively.
Statfs_t: Mntonname and Mntfromname changed from [88]int8 to [1024]int8 arrays.
Dirent: Fileno became uint64, Namlen uint16 and an additional field Off int64 (currently unused) was added.
The following commands were run to generate ztypes_* and zsyscall_* on FreeBSD-12.0-ALPHA6 systems (GOARCH=386 were run on the same amd64 host):
GOOS=freebsd GOARCH=amd64 ./mksyscall.pl -tags freebsd,amd64 syscall_bsd.go syscall_freebsd.go syscall_freebsd_amd64.go |gofmt >zsyscall_freebsd_amd64.go
GOOS=freebsd GOARCH=amd64 go tool cgo -godefs types_freebsd.go | GOOS=freebsd GOARCH=amd64 go run mkpost.go >ztypes_freebsd_amd64.go
GOOS=freebsd GOARCH=386 ./mksyscall.pl -l32 -tags freebsd,386 syscall_bsd.go syscall_freebsd.go syscall_freebsd_386.go |gofmt >zsyscall_freebsd_386.go
GOOS=freebsd GOARCH=386 go tool cgo -godefs types_freebsd.go | GOOS=freebsd GOARCH=386 go run mkpost.go >ztypes_freebsd_386.go
GOOS=freebsd GOARCH=arm ./mksyscall.pl -l32 -arm -tags freebsd,arm syscall_bsd.go syscall_freebsd.go syscall_freebsd_arm.go |gofmt >zsyscall_freebsd_arm.go
GOOS=freebsd GOARCH=arm go tool cgo -godefs -- -fsigned-char types_freebsd.go | GOOS=freebsd GOARCH=arm go run mkpost.go >ztypes_freebsd_arm.go
The Kevent struct was changed to use the FREEBSD_COMPAT11 version always (requiring the COMPAT_FREEBSD11 kernel option FreeBSD-12, this is the default).
The definitions of ifData were not updated, their functionality in has have been replaced by vendored golang.org/x/net/route.
freebsdVersion initialization was dropped from init() in favor of a sync.Once based wrapper - supportsABI().
Updates #22448.
Change-Id: I359b756e2849c036d7ed7f75dbd6ec836e0b90b4
Reviewed-on: https://go-review.googlesource.com/c/138595
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
runtime.armArch is unused on linux/arm since CL 93637. The new code in
runtime/internal/atomic (added by CL 111315) only checks runtime.goarm.
Change-Id: Ic097ee6750e39abb20f45770a1c7c2d925f02408
Reviewed-on: https://go-review.googlesource.com/c/140077
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The alloc_m documentation refers to concepts that don't exist (and
maybe never did?). alloc_m is also not the API entry point to span
allocation.
Hence, rewrite the documentation for alloc and alloc_m. While we're
here, document why alloc_m must run on the system stack and replace
alloc_m's hand-implemented system stack check with a go:systemstack
annotation.
Change-Id: I30e263d8e53c2774a6614e1b44df5464838cef09
Reviewed-on: https://go-review.googlesource.com/c/139459
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The TestLldbPython test is known to fail with very old lldb releases
(3.8 and older). Skip the test when the lldb found on the system is
too old.
Fixes#22299
Change-Id: I8f78d6c0d995118f806dae87f3f04a9726473116
Reviewed-on: https://go-review.googlesource.com/c/139397
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently the table of arena sizes mixes the number of entries in the
L1 with the size of the L2. While the size of the L2 is important,
this makes it hard to see what's actually going on because there's an
implicit factor of sys.PtrSize.
This changes the L2 column to say both the number of entries and the
size that results in. This should hopefully make the relations between
the columns of the table clearer, since they can now be plugged
directly into the given formula.
Change-Id: Ie677adaef763b893a2f620bd4fc3b8db314b3a1e
Reviewed-on: https://go-review.googlesource.com/c/139697
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit adds AIX operating system to runtime package for ppc64
architecture.
Only new files and minor changes are in this commit. Others
modifications in files like asm_ppc64.s will be in separate commits.
Updates: #25893
Change-Id: I9c5e073f5f3debb43b004ad1167694a5afd31cfd
Reviewed-on: https://go-review.googlesource.com/c/138716
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, the Windows profile loop isn't robust against racing with
unminit. For example,
T1 is running profileloop1, T2 is another thread
T1: thread := atomic.Loaduintptr(&T2.thread)
T2: calls unminit, which does CloseHandle(T2.thread)
T1: attempts to suspends T2
In this case the SuspendThread will fail, but currently we ignore this
failure and forge ahead, which will cause further failures and
probably bad profile data.
Handle this race by defending against SuspendThread failing. If
SuspendThread succeeds, then we know the thread is no longer going
anywhere.
Change-Id: I4726553239b17f05ca07a0cf7df49631e0cb550d
Reviewed-on: https://go-review.googlesource.com/c/129685
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
The debug call injection tests will freeze when run under a debugger
because they depend on catching SIGTRAP, which is usually swallowed by
a debugger.
Change-Id: If6b86ca279b0489182990dd513444ca3062973f1
Reviewed-on: https://go-review.googlesource.com/c/139437
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
When a function triggers a signal (like a segfault which translates to
a nil pointer exception) during execution, a sigpanic handler is just
below it on the stack. The function itself did not stop at a
safepoint, so we have to figure out what safepoint we should use to
scan its stack frame.
Previously we used the site of the most recent defer to get the live
variables at the signal site. That answer is not quite correct, as
explained in #27518. Instead, use the site of a deferreturn call.
It has all the right variables marked as live (no args, all the return
values, except those that escape to the heap, in which case the
corresponding PAUTOHEAP variables will be live instead).
This CL requires stack objects, so that all the local variables
and args referenced by the deferred closures keep the right variables alive.
Fixes#27518
Change-Id: Id45d8a8666759986c203181090b962e2981e48ca
Reviewed-on: https://go-review.googlesource.com/c/134637
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The previous CL introduced stack objects. This CL removes the old
ambiguously live liveness analysis. After this CL we're relying
on stack objects exclusively.
Update a bunch of liveness tests to reflect the new world.
Fixes#22350
Change-Id: I739b26e015882231011ce6bc1a7f426049e59f31
Reviewed-on: https://go-review.googlesource.com/c/134156
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Rework how the compiler+runtime handles stack-allocated variables
whose address is taken.
Direct references to such variables work as before. References through
pointers, however, use a new mechanism. The new mechanism is more
precise than the old "ambiguously live" mechanism. It computes liveness
at runtime based on the actual references among objects on the stack.
Each function records all of its address-taken objects in a FUNCDATA.
These are called "stack objects". The runtime then uses that
information while scanning a stack to find all of the stack objects on
a stack. It then does a mark phase on the stack objects, using all the
pointers found on the stack (and ancillary structures, like defer
records) as the root set. Only stack objects which are found to be
live during this mark phase will be scanned and thus retain any heap
objects they point to.
A subsequent CL will remove all the "ambiguously live" logic from
the compiler, so that the stack object tracing will be required.
For this CL, the stack tracing is all redundant with the current
ambiguously live logic.
Update #22350
Change-Id: Ide19f1f71a5b6ec8c4d54f8f66f0e9a98344772f
Reviewed-on: https://go-review.googlesource.com/c/134155
Reviewed-by: Austin Clements <austin@google.com>
Don't worry, this patch just remove trailing whitespace from
assembly files, and does not touch any logical changes.
Change-Id: Ia724ac0b1abf8bc1e41454bdc79289ef317c165d
Reviewed-on: https://go-review.googlesource.com/c/113595
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Now that we do no mark work during mark termination, we no longer need
the gchelper mechanism.
Updates #26903.
Updates #17503.
Change-Id: Ie94e5c0f918cfa047e88cae1028fece106955c1b
Reviewed-on: https://go-review.googlesource.com/c/134785
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Before STW and concurrent GC were unified, there could be either one
or two root marking passes per GC cycle. There were several tasks we
had to make sure happened once and only once (whether that was at the
beginning of concurrent mark for concurrent GC or during mark
termination for STW GC). We kept track of this in work.markrootdone.
Now that STW and concurrent GC both use the concurrent marking code
and we've eliminated all work done by the second root marking pass, we
only ever need a single root marking pass. Hence, we can eliminate
work.markrootdone and all of the code that's conditional on it.
Updates #26903.
Change-Id: I654a0f5e21b9322279525560a31e64b8d33b790f
Reviewed-on: https://go-review.googlesource.com/c/134784
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, all mcaches are flushed during STW mark termination as a
root marking job. This is currently necessary because all spans must
be out of these caches before sweeping begins to avoid races with
allocation and to ensure the spans are in the state expected by
sweeping. We do it as a root marking job because mcache flushing is
somewhat expensive and O(GOMAXPROCS) and this parallelizes the work
across the Ps. However, it's also the last remaining root marking job
performed during mark termination.
This CL moves mcache flushing out of mark termination and performs it
lazily. We keep track of the last sweepgen at which each mcache was
flushed and as each P is woken from STW, it observes that its mcache
is out-of-date and flushes it.
The introduces a complication for spans cached in stale mcaches. These
may now be observed by background or proportional sweeping or when
attempting to add a finalizer, but aren't in a stable state. For
example, they are likely to be on the wrong mcentral list. To fix
this, this CL extends the sweepgen protocol to also capture whether a
span is cached and, if so, whether or not its cache is stale. This
protocol blocks asynchronous sweeping from touching cached spans and
makes it the responsibility of mcache flushing to sweep the flushed
spans.
This eliminates the last mark termination root marking job, which
means we can now eliminate that entire infrastructure.
Updates #26903. This implements lazy mcache flushing.
Change-Id: Iadda7aabe540b2026cffc5195da7be37d5b4125e
Reviewed-on: https://go-review.googlesource.com/c/134783
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Now work.helperDrainBlock is always false, so we can remove it and
code paths that only ran when it was true. That means we no longer use
the gcDrainBlock mode of gcDrain, so we can eliminate that. That means
we no longer use gcWork.get, so we can eliminate that. That means we
no longer use getfull, so we can eliminate that.
Updates #26903. This is a follow-up to unifying STW GC and concurrent GC.
Change-Id: I8dbcf8ce24861df0a6149e0b7c5cd0eadb5c13f6
Reviewed-on: https://go-review.googlesource.com/c/134782
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Now that STW GC marking is unified with concurrent marking, there
should never be mark work remaining in mark termination. Hence, we can
make that check unconditional.
Updates #26903. This is a follow-up to unifying STW GC and concurrent GC.
Change-Id: I43a21df5577635ab379c397a7405ada68d331e03
Reviewed-on: https://go-review.googlesource.com/c/134781
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, STW GC works very differently from concurrent GC. The
largest differences in that in concurrent GC, all marking work is done
by background mark workers during the mark phase, while in STW GC, all
marking work is done by gchelper during the mark termination phase.
This is a consequence of the evolution of Go's GC from a STW GC by
incrementally moving work from STW mark termination into concurrent
mark. However, at this point, the STW code paths exist only as a
debugging mode. Having separate code paths for this increases the
maintenance burden and complexity of the garbage collector. At the
same time, these code paths aren't tested nearly as well, making it
far more likely that they will bit-rot.
This CL reverses the relationship between STW GC, by re-implementing
STW GC in terms of concurrent GC.
This builds on the new scheduled support for disabling user goroutine
scheduling. During sweep termination, it disables user scheduling, so
when the GC starts the world again for concurrent mark, it's really
only "concurrent" with itself.
There are several code paths that were specific to STW GC that are now
vestigial. We'll remove these in the follow-up CLs.
Updates #26903.
Change-Id: Ia3883d2fcf7ab1d89bdc9c8ee54bf9bffb32c096
Reviewed-on: https://go-review.googlesource.com/c/134780
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This adds support for disabling the scheduling of user goroutines
while allowing system goroutines like the garbage collector to
continue running. User goroutines pass through the usual state
transitions, but if we attempt to actually schedule one, it will get
put on a deferred scheduling list.
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Updates #25578. This same mechanism can form the basis for disabling
all but a single user goroutine for the purposes of debugger function
call injection.
Change-Id: Ib72a808e00c25613fe6982f5528160d3de3dbbc6
Reviewed-on: https://go-review.googlesource.com/c/134779
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, isSystemGoroutine varies on whether it considers the
finalizer goroutine a user goroutine or a system goroutine. For the
next CL, we're going to want to always consider the finalier goroutine
a user goroutine, so add a flag that indicates that.
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Change-Id: Iafc92e519c13d9f8d879332cb5f0d12164104c33
Reviewed-on: https://go-review.googlesource.com/c/134778
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, setting GODEBUG=gcrescanstacks=1 enables a debugging mode
where the garbage collector re-scans goroutine stacks during mark
termination. This was introduced in Go 1.8 to debug the hybrid write
barrier, but I don't think we ever used it.
Now it's one of the last sources of mark work during mark termination.
This CL removes it.
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Updates #17503.
Change-Id: I6ae04d3738aa9c448e6e206e21857a33ecd12acf
Reviewed-on: https://go-review.googlesource.com/c/134777
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, checkmarks mode uses the full STW GC infrastructure to
perform mark checking. We're about to remove that infrastructure and,
furthermore, since checkmarks is about doing the simplest thing
possible to check concurrent GC, it's valuable for it to be simpler.
Hence, this CL makes checkmarks even simpler by making it non-parallel
and divorcing it from the STW GC infrastructure (including the
gchelper mechanism).
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Change-Id: Iad21158123e025e3f97d7986d577315e994bd43e
Reviewed-on: https://go-review.googlesource.com/c/134776
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This argument is always gcBackgroundMode since only
debug.gcstoptheworld can trigger a STW GC at this point. Remove the
unnecessary argument.
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Change-Id: Icb4ba8f10f80c2b69cf51a21e04fa2c761b71c94
Reviewed-on: https://go-review.googlesource.com/c/134775
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, we disable GC work caching during mark termination. This is
no longer necessary with the new mark completion detection because
1. There's no way for any of the GC mark termination helpers to have
any real work queued and,
2. Mark termination has to explicitly flush every P's buffers anyway
in order to flush Ps that didn't run a GC mark termination helper.
Hence, remove the code that disposes gcWork buffers during mark
termination.
Updates #26903. This is a follow-up to eliminating mark 2.
Change-Id: I81f002ee25d5c10f42afd39767774636519007f9
Reviewed-on: https://go-review.googlesource.com/c/134320
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Now that there is no mark 2 phase, gcBlackenPromptly is no longer
used.
Updates #26903. This is a follow-up to eliminating mark 2.
Change-Id: Ib9c534f21b36b8416fcf3cab667f186167b827f8
Reviewed-on: https://go-review.googlesource.com/c/134319
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
The mark 2 phase was originally introduced as a way to reduce the
chance of entering STW mark termination while there was still marking
work to do. It works by flushing and disabling all local work caches
so that all enqueued work becomes immediately globally visible.
However, mark 2 is not only slow–disabling caches makes marking and
the write barrier both much more expensive–but also imperfect. There
is still a rare but possible race (~once per all.bash) that can cause
GC to enter mark termination while there is still marking work. This
race is detailed at
https://github.com/golang/proposal/blob/master/design/17503-eliminate-rescan.md#appendix-mark-completion-race
The effect of this is that mark termination must still cope with the
possibility that there may be work remaining after a concurrent mark
phase. Dealing with this increases STW pause time and increases the
complexity of mark termination.
Furthermore, a similar but far more likely race can cause early
transition from mark 1 to mark 2. This is unfortunate because it
causes performance instability because of the cost of mark 2.
This CL fixes this by replacing mark 2 with a distributed termination
detection algorithm. This algorithm is correct, so it eliminates the
mark termination race, and doesn't require disabling local caches. It
ensures that there are no grey objects upon entering mark termination.
With this change, we're one step closer to eliminating marking from
mark termination entirely (it's still used by STW GC and checkmarks
mode).
This CL does not eliminate the gcBlackenPromptly global flag, though
it is always set to false now. It will be removed in a cleanup CL.
This led to only minor variations in the go1 benchmarks
(https://perf.golang.org/search?q=upload:20180909.1) and compilebench
benchmarks (https://perf.golang.org/search?q=upload:20180910.2).
This significantly improves performance of the garbage benchmark, with
no impact on STW times:
name old time/op new time/op delta
Garbage/benchmem-MB=64-12 2.21ms ± 1% 2.05ms ± 1% -7.38% (p=0.000 n=18+19)
Garbage/benchmem-MB=1024-12 2.30ms ±16% 2.20ms ± 7% -4.51% (p=0.001 n=20+20)
name old STW-ns/GC new STW-ns/GC delta
Garbage/benchmem-MB=64-12 138k ±44% 141k ±23% ~ (p=0.309 n=19+20)
Garbage/benchmem-MB=1024-12 159k ±25% 178k ±98% ~ (p=0.798 n=16+18)
name old STW-ns/op new STW-ns/op delta
Garbage/benchmem-MB=64-12 4.42k ±44% 4.24k ±23% ~ (p=0.531 n=19+20)
Garbage/benchmem-MB=1024-12 591 ±24% 636 ±111% ~ (p=0.309 n=16+18)
(https://perf.golang.org/search?q=upload:20180910.1)
Updates #26903.
Updates #17503.
Change-Id: Icbd1e12b7a12a76f423c9bf033b13cb363e4cd19
Reviewed-on: https://go-review.googlesource.com/c/134318
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Nothing currently consumes the flag, but we'll use it in the
distributed termination detection algorithm.
Updates #26903. This is preparation for eliminating mark 2.
Change-Id: I5e149a05b1c878fe1009150da21f8bd8ae2b9b6a
Reviewed-on: https://go-review.googlesource.com/c/134317
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
It turns out if you set GODEBUG=gctrace=2, it enables an obscure
debugging mode that, in addition to printing gctrace statistics, also
does a second STW GC following each regular GC. This debugging mode
has long since lost its value (you could maybe use it to analyze
floating garbage, except that we don't print the gctrace line on the
second GC), and it interferes substantially with the operation of the
GC by messing up the statistics used to schedule GCs.
It's also a source of mark termination GC work when we're in
concurrent GC mode, so it's going to interfere with eliminating mark
2. And it's going to get in the way of unifying STW and concurrent GC.
This CL removes this debugging mode.
Updates #26903. This is preparation for eliminating mark 2 and
unifying STW GC and concurrent GC.
Change-Id: Ib5bce05d8c4d5b6559c89a65165d49532165df07
Reviewed-on: https://go-review.googlesource.com/c/134316
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, if the gcWork runs out of work, we'll fall out of the GC
worker, even though flushing the write barrier buffer could produce
more work. While this is not a correctness issue, it can lead to
premature mark 2 or mark termination.
Fix this by flushing the write barrier buffer if the local gcWork runs
out of work and then checking the local gcWork again.
This reduces the number of premature mark terminations during all.bash
by about a factor of 10.
Updates #26903. This is preparation for eliminating mark 2.
Change-Id: I48577220b90c86bfd28d498e8603bc379a8cd617
Reviewed-on: https://go-review.googlesource.com/c/134315
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Previously, some of output from gdb matched with literal string, while
gdb v8.2 print the address of variable (e.g. map key and value) in
output.
This commit fix the regex in testing the output.
Fixes#27608
Change-Id: Ic3fe8280b9f93fda2799116804822616caa66beb
Reviewed-on: https://go-review.googlesource.com/135055
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
During a call to a reflect-generated function or method (via
makeFuncStub or methodValueCall), when should we scan the return
values?
When we're starting a reflect call, the space on the stack for the
return values is not initialized yet, as it contains whatever junk was
on the stack of the caller at the time. The return space must not be
scanned during a GC.
When we're finishing a reflect call, the return values are
initialized, and must be scanned during a GC to make sure that any
pointers in the return values are found and their referents retained.
When the GC stack walk comes across a reflect call in progress on the
stack, it needs to know whether to scan the results or not. It doesn't
know the progress of the reflect call, so it can't decide by
itself. The reflect package needs to tell it.
This CL adds another slot in the frame of makeFuncStub and
methodValueCall so we can put a boolean in there which tells the
runtime whether to scan the results or not.
This CL also adds the args length to reflectMethodValue so the
runtime can restrict its scanning to only the args section (not the
results) if the reflect package says the results aren't ready yet.
Do a delicate dance in the reflect package to set the "results are
valid" bit. We need to make sure we set the bit only after we've
copied the results back to the stack. But we must set the bit before
we drop reflect's copy of the results. Otherwise, we might have a
state where (temporarily) no one has a live copy of the results.
That's the state we were observing in issue #27695 before this CL.
The bitmap used by the runtime currently contains only the args.
(Actually, it contains all the bits, but the size is set so we use
only the args portion.) This is safe for early in a reflect call, but
unsafe late in a reflect call. The test issue27695.go demonstrates
this unsafety. We change the bitmap to always include both args
and results, and decide at runtime which portion to use.
issue27695.go only has a test for method calls. Function calls were ok
because there wasn't a safepoint between when reflect dropped its copy
of the return values and when the caller is resumed. This may change
when we introduce safepoints everywhere.
This truncate-to-only-the-args was part of CL 9888 (in 2015). That
part of the CL fixed the problem demonstrated in issue27695b.go but
introduced the problem demonstrated in issue27695.go.
TODO, in another CL: simplify FuncLayout and its test. stack return
value is now identical to frametype.ptrdata + frametype.gcdata.
Fixes#27695
Change-Id: I2d49b34e34a82c6328b34f02610587a291b25c5f
Reviewed-on: https://go-review.googlesource.com/137440
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
mcache.refill doesn't need to run on the system stack; it just needs
to be non-preemptible. Its only caller, mcache.nextFree, also needs to
be non-preemptible, so we can remove the unnecessary systemstack
switch.
Change-Id: Iba5b3f4444855f1dc134485ba588efff3b54c426
Reviewed-on: https://go-review.googlesource.com/138196
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
mcache.refill acquires g.m.locks, which is pointless because the
caller itself absolutely must have done so already to prevent
ownership of mcache from shifting.
Also, mcache.refill's documentation is generally a bit out-of-date, so
this cleans this up.
Change-Id: Idc8de666fcaf3c3d96006bd23a8f307539587d6c
Reviewed-on: https://go-review.googlesource.com/138195
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This is the first commit of a series that will add AIX as an
operating system target for ppc64 architecture.
Updates #25893
Change-Id: I865b67a9c98277c11c1a56107be404ac5253277d
Reviewed-on: https://go-review.googlesource.com/138115
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ending a loop with a break is confusing. Rewrite the loop so the
default behavior is to loop and then do the "post-loop" work outside
of the loop.
Change-Id: Ie49b4132541dfb5124c31a8163f2c883aa4abc75
Reviewed-on: https://go-review.googlesource.com/138155
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)
Updates #27864
Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fix the code to use write barriers on heap memory, and no
write barriers on stack memory.
These errors were discoverd as part of fixing #27695. They may
have something to do with that issue, but hard to be sure.
The core cause is different, so this fix is a separate CL.
Update #27695
Change-Id: Ib005f6b3308de340be83c3d07d049d5e316b1e3c
Reviewed-on: https://go-review.googlesource.com/137438
Reviewed-by: Austin Clements <austin@google.com>
We already aliased mSpanInUse to _MSpanInUse. The dual constants are
getting annoying, so fix all of these to use the mSpan* naming
convention.
This was done automatically with:
sed -i -re 's/_?MSpan(Dead|InUse|Manual|Free)/mSpan\1/g' *.go
plus deleting the existing definition of mSpanInUse.
Change-Id: I09979d9d491d06c10689cea625dc57faa9cc6767
Reviewed-on: https://go-review.googlesource.com/137875
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Updates #26148
Change-Id: I8f68b2c926c7b11dc86c9664ed7ff2d2f78b64b4
Reviewed-on: https://go-review.googlesource.com/128715
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
They aren't really races, or at least they don't have any
observable effect. The spec is silent on whether these are actually
races or not.
Fix this problem by not using the address of len (or of cap)
as the location where channel operations are recorded to occur.
Use a random other field of hchan for that.
I'm not 100% sure we should in fact fix this. Opinions welcome.
Fixes#27070
Change-Id: Ib4efd4b62e0d1ef32fa51e373035ef207a655084
Reviewed-on: https://go-review.googlesource.com/135698
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
On Linux, sysUnused currently uses madvise(MADV_DONTNEED) to signal the
kernel that a range of allocated memory contains unneeded data. After a
successful call, the range (but not the data it contained before the
call to madvise) is still available but the first access to that range
will unconditionally incur a page fault (needed to 0-fill the range).
A faster alternative is MADV_FREE, available since Linux 4.5. The
mechanism is very similar, but the page fault will only be incurred if
the kernel, between the call to madvise and the first access, decides to
reuse that memory for something else.
In sysUnused, test whether MADV_FREE is supported and fall back to
MADV_DONTNEED in case it isn't. This requires making the return value of
the madvise syscall available to the caller, so change runtime.madvise
to return it.
Fixes#23687
Change-Id: I962c3429000dd9f4a00846461ad128b71201bb04
Reviewed-on: https://go-review.googlesource.com/135395
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The cmd/compile/internal/ld/go.go file not exist, actually cmd/link/internal/ld/go.go.
Also, write line number is not good because it changes every commit of the file.
Change-Id: Id2b9f2c9904390adb011dab357716ee8e2fe84fc
Reviewed-on: https://go-review.googlesource.com/135516
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This makes TestRuntimePanic keep most of the existing environment,
just as the other runtime tests do.
Change-Id: I7944abfeee292d41716dca14483134a50d75f081
Reviewed-on: https://go-review.googlesource.com/135376
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 135015 added TestSpuriousWakeupsNeverHangSemasleep.
However, this test is failing on Plan 9 because
syscall.SIGIO is not defined.
This change excludes semasleep_test.go on Plan 9
Fixes#27662.
Change-Id: I52f9f0fe9ec3c70da5d2f586a95debbc1fe568a1
Reviewed-on: https://go-review.googlesource.com/135315
Run-TryBot: David du Colombier <0intro@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This failure occurs randomly on arm64.
13:10:32 --- FAIL: TestGcSys (0.06s)
13:10:32 gc_test.go:30: expected "OK\n", but got "using too much memory: 71401472 bytes\n"
13:10:32 FAIL
Updates #27636
Change-Id: Ifd4cfce167d8054dc6f037bd34368d63c7f68ed4
Reviewed-on: https://go-review.googlesource.com/135155
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A regression test in which: for a program that invokes semasleep,
we send non-terminal signals such as SIGIO.
Since the signal wakes up pthread_cond_timedwait_relative_np,
after CL 133655, we should only re-spin for the amount of
time left, instead of re-spinning with the original duration
which would cause an indefinite spin.
Updates #27520
Change-Id: I744a6d04cf8923bc4e13649446aff5e42b7de5d8
Reviewed-on: https://go-review.googlesource.com/135015
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The implementations of the functions in mem_darwin.go is identical to
the ones defined in mem_bsd.go for all other BSD-like GOOSes. Also use
them on Darwin.
Change-Id: Ie7c170c1a50666475e79599471081cd85f0837ad
Reviewed-on: https://go-review.googlesource.com/134875
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
At the very beginning of timediv, inside a for loop,
we reduce the base value by at most (1<<31)-1, while
incrementing the quotient result by 1<<uint(bit).
However, since the quotient value was 0 to begin with,
we are essentially just doing bitsets.
This change is in the hot path of various concurrency and
scheduling operations that require sleeping, waiting
on mutexes and futexes etc. On the following OSes:
* Dragonfly
* FreeBSD
* Linux
* NetBSD
* OpenBSD
* Plan9
* Windows
and paired with architectures that provide the BTS instruction, this
change shaves off a couple of nanoseconds per invocation of timediv.
Fixes#27529
Change-Id: Ia2fea5022c1109e02d86d1f962a3b0bd70967aa6
Reviewed-on: https://go-review.googlesource.com/134231
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
raise uses tkill to send a signal to the current thread. For this use,
tgkill is functionally equivalent to tkill expect that it also takes the
pid as the first argument.
Using tgkill makes it simpler to run a Go program in a strict sandbox.
With kill and tgkill, the sandbox policy (e.g., seccomp) can prevent the
program from sending signals to other processes by checking that the
first argument == getpid().
With tkill, the policy must whitelist all tids in the process, which is
effectively impossible given Go's dynamic thread creation.
Fixes#27548
Change-Id: I8ed282ef1f7215b02ef46de144493e36454029ea
Reviewed-on: https://go-review.googlesource.com/133975
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When pthread_cond_timedwait_relative_np gets a spurious wakeup
(due to a signal, typically), we used to retry with the same
relative timeout. That's incorrect, we should lower the timeout
by the time we've spent in this function so far.
In the worst case, signals come in and cause spurious wakeups
faster than the timeout, causing semasleep to never time out.
Also fix nacl and netbsd while we're here. They have similar issues.
Fixes#27520
Change-Id: I6601e120e44a4b8ef436eef75a1e7c8cf1d39e39
Reviewed-on: https://go-review.googlesource.com/133655
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
If binary file of a running program was deleted or moved, maps
file (/proc/pid/maps) will contain lines that have this binary
filename suffixed with "(deleted)" string. This suffix stayed
as a part of the filename and made remote profiling slightly more
difficult by requiring from a user to rename binary file to
include this suffix.
This change cleans up the filename and removes this suffix and
thus simplify debugging.
Fixes#25740
Change-Id: Ib3c8c3b9ef536c2ac037fcc14e8037fa5c960036
Reviewed-on: https://go-review.googlesource.com/116395
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
In the CPU profile tests for gccgo, check to make sure that the
runtime's sigprof handler itself doesn't appear in the profile. Add a
"skip if gccgo" guard to one testpoint.
Updates #26595
Change-Id: I92a44161d61f17b9305ce09532134edd229745a7
Reviewed-on: https://go-review.googlesource.com/126316
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change adds support for VDSO on ppc64x, making it possible to
avoid a syscall in walltime and nanotime.
BenchmarkClockVDSOAndFallbackPaths/vDSO-192 20000000 66.0 ns/op
BenchmarkClockVDSOAndFallbackPaths/Fallback-192 1000000 1456 ns/op
Change-Id: I3373bd804b6f122961de3ae9d034e6ccf35748e6
Reviewed-on: https://go-review.googlesource.com/131135
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Fixes#26638.
Change-Id: I3c18d1298d99af8ea8c00916303efd2b5a5effc7
Reviewed-on: https://go-review.googlesource.com/126336
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This makes the runtime.support_sse2 variable unused
so it is removed in this CL too.
Change-Id: Ia8b9ffee7ac97128179f74ef244b10315e44c234
Reviewed-on: https://go-review.googlesource.com/131455
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The sigInitIgnored function can be called by initsig before a shared
library is initialized, before the runtime is initialized.
Fixes#27183
Change-Id: I7073767938fc011879d47ea951d63a14d1cce878
Reviewed-on: https://go-review.googlesource.com/131277
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add an "offset_" prefix to all cpu feature variable offset constants to
signify that they are not boolean cpu feature variables.
Remove _ from offset constant names.
Change-Id: I6e22a79ebcbe6e2ae54c4ac8764f9260bb3223ff
Reviewed-on: https://go-review.googlesource.com/131215
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
sys here is runtime/internal/sys.
Replace uses of sys.CacheLineSize for padding by
cpu.CacheLinePad or cpu.CacheLinePadSize.
Replace other uses of sys.CacheLineSize by cpu.CacheLineSize.
Remove now unused sys.CacheLineSize.
Updates #25203
Change-Id: I1daf410fe8f6c0493471c2ceccb9ca0a5a75ed8f
Reviewed-on: https://go-review.googlesource.com/126601
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Assumes mandatory VFP and VFPv3 support to be present by default
but not IDIVA if AT_HWCAP is not available.
Adds GODEBUGCPU options to disable the use of code paths in the runtime
that use hardware support for division.
Change-Id: Ida02311bd9b9701de3fc120697e69445bf6c0853
Reviewed-on: https://go-review.googlesource.com/114826
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The new slice created in growslice is cleared during malloc for
element types containing pointers and therefore can only contain
nil pointers. This change avoids executing write barriers for these
nil pointers by adding and using a special bulkBarrierPreWriteSrcOnly
function that does not enqueue pointers to slots in dst to the write
barrier buffer.
Change-Id: If9b18248bfeeb6a874b0132d19520adea593bfc4
Reviewed-on: https://go-review.googlesource.com/115996
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Using internal/cpu variables has the benefit of avoiding false sharing
(as those are padded) and allows memory and cache usage for these variables
to be shared by multiple packages.
Change-Id: I2bf68d03091bf52b466cf689230d5d25d5950037
Reviewed-on: https://go-review.googlesource.com/126599
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
gentraceback handles system stack transitions, but only when they're
done by systemstack(). Handle morestack() too.
I tried to do this generically but systemstack and morestack are
actually *very* different functions. Most notably, systemstack returns
"normally", just messes with $sp along the way. morestack never
returns at all -- it calls newstack, and newstack then jumps both
stacks and functions back to whoever called morestack. I couldn't
think of a way to handle both of them generically. So don't.
The current implementation does not include systemstack on the generated
traceback. That's partly because I don't know how to find its stack frame
reliably, and partly because the current structure of the code wants to
do the transition before the call, not after. If we're willing to
assume that morestack's stack frame is 0 size, this could probably be
fixed.
For posterity's sake, alternatives tried:
- Have morestack put a dummy function into schedbuf, like systemstack
does. This actually worked (see patchset 1) but more by a series of
coincidences than by deliberate design. The biggest coincidence was that
because morestack_switch was a RET and its stack was 0 size, it actually
worked to jump back to it at the end of newstack -- it would just return
to the caller of morestack. Way too subtle for me, and also a little
slower than just jumping directly.
- Put morestack's PC and SP into schedbuf, so that gentraceback could
treat it like a normal function except for the changing SP. This was a
terrible idea and caused newstack to reenter morestack in a completely
unreasonable state.
To make testing possible I did a small redesign of testCPUProfile to
take a callback that defines how to check if the conditions pass to it
are satisfied. This seemed better than making the syntax of the "need"
strings even more complicated.
Updates #25943
Change-Id: I9271a30a976f80a093a3d4d1c7e9ec226faf74b4
Reviewed-on: https://go-review.googlesource.com/126795
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
gentraceback gets the currently running g to do some sanity checks, but
should use gp everywhere to do its actual work. Some noncritical checks
later accidentally used g instead of gp. This seems like it could be a
problem in many different contexts, but I noticed in Windows profiling,
where profilem calls gentraceback on a goroutine from a different
thread.
Change-Id: I3da27a43e833b257f6411ee6893bdece45a9323f
Reviewed-on: https://go-review.googlesource.com/128895
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Austin Clements <austin@google.com>
If two goroutines are racing on a map, one of them will exit
cleanly, clearing the hashWriting bit, and the other will
likely notice and panic. If we use XOR instead of OR to
set the bit in the first place, even numbers of racers will
hopefully all see the bit cleared and panic simultaneously,
giving the full set of available stacks. If a third racer
sneaks in, we are no worse than the current code, and
the generated code should be no more expensive.
In practice, this catches most racing goroutines even in
very tight races. See the demonstration program posted
on https://github.com/golang/go/issues/26703 for an example.
Fixes#26703
Change-Id: Idad17841a3127c24bd0a659b754734f70e307434
Reviewed-on: https://go-review.googlesource.com/126936
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This is causing failures on TryBots and BuildBots:
--- FAIL: TestGcSys (0.06s)
gc_test.go:27: expected "OK\n", but got "using too much memory: 39882752 bytes\n"
FAIL
Updates #27156
Change-Id: I418bbec89002574cd583c97422e433f042c07492
Reviewed-on: https://go-review.googlesource.com/130875
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Example of use:
iter := reflect.ValueOf(m).MapRange()
for iter.Next() {
k := iter.Key()
v := iter.Value()
...
}
See issue golang/go#11104
Q. Are there any benchmarks that would exercise the new calls to
copyval in existing code?
Change-Id: Ic469fcab5f1d9d853e76225f89bde01ee1d36e7a
Reviewed-on: https://go-review.googlesource.com/33572
Reviewed-by: Keith Randall <khr@golang.org>
I merged CL 115835 without testing it after a rebase. My bad.
Change-Id: I0acc6ed78ea7d718ac2df11d509cfcf4364dfaee
Reviewed-on: https://go-review.googlesource.com/130815
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
Replaces legacy Go syntax for pointer struct member access
with more modern auto-deref alternative.
Found using https://go-critic.github.io/overview#underef-ref
Change-Id: I71a3c424126c4ff5d89f9e4bacb6cc01c6fa2ddf
Reviewed-on: https://go-review.googlesource.com/122895
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Replace `x[:]` where x is a slice with just `x`.
Found using https://go-critic.github.io/overview.html#unslice-ref
Change-Id: Ib0ee16e1d49b2a875b6b92a770049acc33208362
Reviewed-on: https://go-review.googlesource.com/123375
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The workthegc function was being inlined, and the slice did not
escape, so there was no memory allocation. Use a sink variable to
force memory allocation, at least for now.
Fixes#23343
Change-Id: I02f4618e343c8b6cb552cb4e9f272e112785f7cf
Reviewed-on: https://go-review.googlesource.com/122576
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The implementation of cputicks has been wrong for ppc64x. The
previous code sequence is for 32 bit, not 64 bit.
Change-Id: I308ae6cf9131f53a0100cd3f8ae4e16601f2d553
Reviewed-on: https://go-review.googlesource.com/129595
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The hasprefix function is redundant and can be removed since it has
the same implementation as hasPrefix modulo variable names.
Fixes#25688
Change-Id: I499cc24a2b5c38d1301718a4e66f555fd138386f
Reviewed-on: https://go-review.googlesource.com/115835
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
The runtime package already imports the internal/cpu package, so there
is no reason for it to use go:linkname comments to refer to
internal/cpu functions and variables. Since internal/cpu is internal,
we can just export those names. Removing the obscurity of go:linkname
outweighs the minor additional complexity added to the internal/cpu API.
Change-Id: Id89951b7f3fc67cd9bce67ac6d01d44a647a10ad
Reviewed-on: https://go-review.googlesource.com/128755
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
The updated list is taken from "src/go/build/syslist.go".
Reason: one should not do web search to know the possible values of GOOS
and GOARCH. The search result point to stackoverflow page which
reference the above source and documentation on installation page [1].
It should available offline (as in local godoc), as part of package
documentation.
[1] https://golang.org/doc/install/source#environment
Change-Id: I736804b8ef4dc11e0260fa862999212ab3f7b3fd
Reviewed-on: https://go-review.googlesource.com/129935
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
netpoll is perhaps one of the most confusing uses of G lists currently
since it passes around many lists as bare *g values right now.
Switching to gList makes it much clearer what's an individual g and
what's a list.
Change-Id: I8d8993c4967c5bae049c7a094aad3a657928ba6c
Reviewed-on: https://go-review.googlesource.com/129397
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
There are two manually managed G dequeues. Abstract these both into a
shared gQueue type. This also introduces a gList type, which we'll use
to replace several manually-managed G lists in follow-up CLs.
This makes the code more readable and maintainable. gcFlushBgCredit in
particular becomes much easier to follow. It also makes it easier to
introduce more G queues in the future. Finally, the gList type clearly
distinguishes between lists of Gs and individual Gs; currently both
are represented by a *g, which can easily lead to confusion and bugs.
Change-Id: Ic7798841b405d311fc8b6aa5a958ffa4c7993c6c
Reviewed-on: https://go-review.googlesource.com/129396
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
the function libc_errno returns a pointer to a signed-32 bit quantity,
not a 64-bit quantity.
Fixes#27004
Change-Id: I0623835ee34fd9655532251f096022a5accb58cd
Reviewed-on: https://go-review.googlesource.com/129475
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
See the added comment for the reasoning behind the workaround.
Fixes#26868.
Change-Id: Idede020ec88a49595dc233d9a1346b12691186f4
Reviewed-on: https://go-review.googlesource.com/128815
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gcWriteBarrier and wbBufFlush assume that not writing to an argument
variable is sufficient to not clobber the corresponding argument slot.
This assumption lets us simplify the write barrier assembly code,
speed up the flush path, and reduce the stack usage of the write
barrier.
But it is an assumption, so this CL documents it to make this clear.
Alternatively, we could separate the register spill slots from the
argument slots in the write barrier, but that loses the advantages
above. On the other hand, it's extremely unlikely that we'll change
the behavior of the compiler to start clobbering argument slots (if
anything, we'd probably change it to *not* clobber argument slots even
if you wrote to the arguments).
Fixes#25512.
Change-Id: Ib2cf29c0d90956ca02b997ef6e7fa56fc8044efe
Reviewed-on: https://go-review.googlesource.com/127815
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The linux syscall functions used in runtime are designed around the calling
convention of returning errors as negative numbers. On some other systems
(like mips and ppc) the actual syscalls signal errors in other ways. This
means that the assembly implementations of the syscall functions on these
platforms need to transform the return values in the error cases to match
the expected negative errno values. This was addressed for certain syscalls
in https://golang.org/cl/19455 and https://golang.org/cl/89235. This patch
handles the rest of the syscall functions in sys_linux_*.s that return any
value for mips/mips64/ppc64.
Fixes#23446
Change-Id: I302100261231f76d5850ab2c2ea080170d7dba72
GitHub-Last-Rev: e358e2b08c
GitHub-Pull-Request: golang/go#26606
Reviewed-on: https://go-review.googlesource.com/125895
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This avoids problems when running under QEMU. It seems that at least
some QEMU versions turn the sigaction implementation into a call to
the C library sigaction function. The C library function will reject
attempts to set the signal handler for signals 32 and 33. Ignore
errors in that case.
Change-Id: Id443a9a32f6fb0ceef5c59a398e7ede30bf71646
Reviewed-on: https://go-review.googlesource.com/125955
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If we're in a libc call and get a trap, don't try to traceback the libc call.
Start from the state we had at entry to libc.
If there are multiple libc calls outstanding, remember the outermost one.
Fixes#26393
Change-Id: Icfe8794b95bf3bfd1a0679b456dcde2481dcabf3
Reviewed-on: https://go-review.googlesource.com/124195
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TSAN for Go only supports heap address in the range [0x00c000000000,
0x00e000000000). However, we currently create heap hints of the form
0xXXc000000000 for XX between 0x00 and 0x7f. Even for XX=0x01, this
hint is outside TSAN's supported heap address range.
Fix this by creating a slightly different set of hints in race mode,
all of which fall inside TSAN's heap address range.
This should fix TestArenaCollision flakes. That test forces the
runtime to use later heap hints. Currently, this always results in
TSAN "failed to allocate" failures on Windows (which happens to have a
slightly more constrained TSAN layout than non-Windows). Most of the
time we don't notice these failures, but sometimes it crashes TSAN,
leading to a test failure.
Fixes#25698.
Change-Id: I8926cd61f0ee5ee00efa77b283f7b809c555be46
Reviewed-on: https://go-review.googlesource.com/123780
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This reverts commit c99300229d.
The original CL skipped the lldb test if it couldn't read compressed
DWARF, but lldb can never read compressed DWARF, so this effectively
disabled this test unconditionally.
The previous commit disabled DWARF compression for this test, so the
test now works on its own merits again. This CL reverts the change to
skip the test so we don't simply mask lldb failures.
Updates #25925.
Change-Id: I3e1c787b658257b542c3c70807065dde9cfe05ee
Reviewed-on: https://go-review.googlesource.com/124386
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
lldb doesn't support compressed DWARF, so right now we're just always
skipping the lldb test. This CL makes the test run again by disabling
compressed DWARF just for this test.
Updates #25925.
Change-Id: Ib9ddc442305fe6d37060d48f36bc4458b6fd8c86
Reviewed-on: https://go-review.googlesource.com/124385
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Profile's Mapping field is currently populated by reading /proc/self/maps.
On systems where /proc/self/maps is not available, the profile generated
by Go's runtime will not have any Mapping entry. Pprof command then adds
a fake entry and links all Location entries in the profile with the fake
entry to be used during symbolization.
a8644067d5/internal/driver/fetch.go (L437)
The fake entry is not enough to suppress the error or warning messages
pprof command produces. We need to tell pprof that Location entries are
symbolized already by Go runtime and pprof does not have to attempt to
perform further symbolization.
In #25743, we made Go runtime mark Mapping entries with HasFunctions=true
when all Location entries from the Mapping entries are successfully
symbolized. This change makes the Go runtime add a fake mapping entry,
otherwise the pprof command tool would add, and set the HasFunctions=true
following the same logic taken when the real mapping information is
available.
Updates #19790.
Fixes#26255. Tested pprof doesn't report the error message any more
for pure Go program.
Change-Id: Ib12b62e15073f5d6c80967e44b3e8709277c11bd
Reviewed-on: https://go-review.googlesource.com/123779
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Also, remove some test code that was trying to work on XP and fix up
some comments referencing XP.
Fixes#26191
Updates #23380
Change-Id: I0b7319fe1954afddb22d396e5ec91d8c960268d8
Reviewed-on: https://go-review.googlesource.com/123415
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix the panic message produced for an interface conversion error to
only say "types from different packages" if they are definitely from
different packges. If they may be from the same package, say "types
from different scopes."
Updates #18911Fixes#26094
Change-Id: I0cea50ba31007d88e70c067b4680009ede69bab9
Reviewed-on: https://go-review.googlesource.com/123395
Reviewed-by: Austin Clements <austin@google.com>
Based on Dmitry Vyukov's comments in CL 65210.
Change-Id: I5dce7286b0d180cd43cad3aaf70f537fafcda588
Reviewed-on: https://go-review.googlesource.com/123275
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
OpenBSD 6.4 will require the stack pointer to be pointing at an area that is
marked as MAP_STACK when entering and exiting syscalls. Adjust the stack pointer
used for a new thread such that it points within the stack, not at the top of
it (i.e. outside).
Fixes#26142
Change-Id: I905bd8e5be3dfc325392e7ac490fb56a7c71b3aa
Reviewed-on: https://go-review.googlesource.com/122735
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Recognize NetBSD in:
- go/internal/work/init.go
- race.bash
- runtime/race/race.go
Add __ps_strings symbol in runtime/cgo/netbsd.go as this is
used internally in the TSan library for NetBSD and used for
ReExec().
Tested on NetBSD/amd64 v. 8.99.12.
Around 98% tests are passing for the ./race.bash target.
Updates #19273
Change-Id: Ic0e48d2fb159a7868aab5e17156eeaca1225e513
GitHub-Last-Rev: d6e082707b
GitHub-Pull-Request: golang/go#24322
Reviewed-on: https://go-review.googlesource.com/99835
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This test is skipped on Linux and should be skipped on Android for the
same reason.
Change-Id: I753c4788d935bd58874554b455c0d5be2315b794
Reviewed-on: https://go-review.googlesource.com/122585
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 122515 tightened TestAbort to look for breakpoint exceptions and
not just general signal crashes, but this only applies on x86 arches.
On non-x86 arches we use a nil pointer dereference to abort, so the
test is now failing.
This CL re-loosens TestAbort on non-x86 arches to only expect a signal
traceback.
Should fix the build on linux/arm, linux/arm64, linux/ppc64, and
linux/s390x.
Change-Id: I1065341180ab5ab4da63b406c641dcde93c9490b
Reviewed-on: https://go-review.googlesource.com/122580
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Since CL 122515, TestAbort is failing on Plan 9
because there is no SIGTRAP signal on Plan 9,
but a note containing the "sys: breakpoint" string.
This change fixes the TestAbort test by handling
the Plan 9 case.
Fixes#26265.
Change-Id: I2fae00130bcee1cf946d8cc9d147a77f951be390
Reviewed-on: https://go-review.googlesource.com/122464
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently, if the runtime overflows the g0 stack on Windows, it leads
to an infinite recursion:
1. Something overflows the g0 stack bounds and calls morestack.
2. morestack determines it's on the g0 stack and hence cannot grow the
stack, so it calls badmorestackg0 (which prints "fatal: morestack on
g0") followed by abort.
3. abort performs an INT $3, which turns into a Windows
_EXCEPTION_BREAKPOINT exception.
4. This enters the Windows sigtramp, which ensures we're on the g0
stack and calls exceptionhandler.
5. exceptionhandler has a stack check prologue, so it determines that
it's out of stack and calls morestack.
6. goto 2
Fix this by making the exception handler avoid stack checks until it
has ruled out an abort and by blowing away the stack bounds in
lastcontinuehandler before we print the final fatal traceback (which
itself involves a lot of stack bounds checks).
Fixes#21382.
Change-Id: Ie66e91f708e18d131d97f22b43f9ac26f3aece5a
Reviewed-on: https://go-review.googlesource.com/120857
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Windows includes an 8K guard in system-allocated thread stacks, which
we currently don't account for when setting the g0 stack bounds. As a
result, if we do overflow the g0 stack bounds, we'll get a
STATUS_GUARD_PAGE_VIOLATION exception, which we're not expecting.
Fix the g0 stack bounds to include a total of 16K of slop to account
for this 8K guard.
Updates #21382.
Change-Id: Ia89b741b1413328e4681a237f5a7ee645531fe16
Reviewed-on: https://go-review.googlesource.com/122516
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
On Windows, the IP recorded in the breakpoint exception caused by
runtime.abort is actually one byte after the INT3, unlike on UNIX
OSes. Account for this in isgoexception.
It turns out TestAbort was "passing" on Windows anyway because abort
still caused a fatal panic, just not the one we were expecting. This
CL tightens this test to check that the runtime specifically reports a
breakpoint exception.
Fixing this is related to #21382, since we use runtime.abort in
reporting g0 stack overflows, and it's important that we detect this
and not try to handle it.
Change-Id: I66120944d138eb80f839346b157a3759c1019e34
Reviewed-on: https://go-review.googlesource.com/122515
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
The implementation is mostly copied from the commit that added
linux/amd64 support for this feature (https://golang.org/cl/17761).
Change-Id: I3f482167620a7a3daf50a48087f8849a30d713bd
Reviewed-on: https://go-review.googlesource.com/102438
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, on Windows, the thread stack size is set or assumed in many
different places. In non-cgo binaries, both the Go linker and the
runtime have a copy of the stack size, the Go linker sets the size of
the main thread stack, and the runtime sets the size of other thread
stacks. In cgo binaries, the external linker sets the main thread
stack size, the runtime assumes the size of the main thread stack will
be the same as used by the Go linker, and the cgo entry code assumes
the same.
Furthermore, users can change the main thread stack size using
editbin, so the runtime doesn't even really know what size it is, and
user C code can create threads with unknown thread stack sizes, which
we also assume have the same default stack size.
This is all a mess.
Fix the corner cases of this and the duplication of knowledge between
the linker and the runtime by querying the OS for the stack bounds
during thread setup. Furthermore, we unify all of this into just
runtime.minit for both cgo and non-cgo binaries and for the main
thread, other runtime-created threads, and C-created threads.
Updates #20975.
Change-Id: I45dbee2b5ea2ae721a85a27680737ff046f9d464
Reviewed-on: https://go-review.googlesource.com/120336
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, we allocate 1MB or 2MB thread stacks on Windows, but in
non-cgo binaries still set the g0 stack bounds assuming only 64k is
available. While this is fine in pure Go binaries, a non-cgo Go binary
on Windows can use the syscall package to call arbitrary DLLs, which
may call back into Go. If a DLL function uses more than 64k of stack
and then calls back into Go, the Go runtime will believe that it's out
of stack space and crash.
Fix this by plumbing the correct stack size into the g0 stacks of
non-cgo binaries. Cgo binaries already use the correct size because
their g0 stack sizes are set by a different code path.
Fixes#20975.
Change-Id: Id6fb559cfe1e1ea0dfac56d4654865c20dccf68d
Reviewed-on: https://go-review.googlesource.com/120195
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The OpenBSD sysctl code has been copy-pasted three times now. Abstract
it.
Change-Id: Ia5558927f0bc2b218b5af425dab368b5485d266c
Reviewed-on: https://go-review.googlesource.com/121775
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
If the runtime code panics due to a bad index or slice expression,
then throw instead of panicing. This will skip calls to recover and dump
the entire runtime stack trace. The runtime should never panic due to
an out of bounds index, and this will help with debugging if it does.
For #24991
Updates #25201
Change-Id: I85a9feded8f0de914ee1558425931853223c0514
Reviewed-on: https://go-review.googlesource.com/121515
Reviewed-by: Austin Clements <austin@google.com>
OpenBSD 6.4 is going to start requiring that the SP points to memory
that was mapped with MAP_STACK on system call entry, traps, and when
switching to the alternate signal stack [1]. Currently, Go doesn't map
any memory MAP_STACK, so the kernel quickly kills Go processes.
Fix this by remapping the memory that backs stack spans with
MAP_STACK, and re-remapping it without MAP_STACK when it's returned to
the heap.
[1] http://openbsd-archive.7691.n7.nabble.com/stack-register-checking-td338238.htmlFixes#26142.
Change-Id: I656eb84385a22833445d49328bb304f8cdd0e225
Reviewed-on: https://go-review.googlesource.com/121657
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reword the package summary to be a little easier to read.
Change-Id: I84a9301a02e228b46165410a429548b3774762d5
GitHub-Last-Rev: 1342c7219f
GitHub-Pull-Request: golang/go#26052
Reviewed-on: https://go-review.googlesource.com/120795
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
semasleep on Darwin was refactored in https://golang.org/cl/118736 to
use the pthread_cond_timedwait function from libc. The new code
incorrectly assumed that pthread_cond_timedwait took a timeout relative
to the current time, when it in fact it takes a timeout specified in
absolute time. semasleep thus specified a timeout well in the past,
causing it to immediately exceed the timeout and spin hot. This was the
source of a large performance hit to CockroachDB (#26019).
Adjust semasleep to instead call pthread_cond_timedwait_relative_np,
which properly interprets its timeout parameter as relative to the
current time.
pthread_cond_timedwait_relative_np is non-portable, but using
pthread_cond_timedwait correctly would require two calls to
gettimeofday: one in the runtime package to convert the relative timeout
to absolute time, then another in the pthread library to convert back to
a relative offset [0], as the Darwin kernel expects a relative offset.
[0]: https://opensource.apple.com/source/libpthread/libpthread-301.30.1/src/pthread_cond.c.auto.htmlFix#26019.
Change-Id: I1a8c2429f79513b43d2b256365cd9166d235af8b
Reviewed-on: https://go-review.googlesource.com/120635
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Code comments should have a space between comments characters and
actual words.
Change-Id: I6274baf1fc09b37a32ec6c69ddbb8edca9eb5469
Reviewed-on: https://go-review.googlesource.com/120475
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, if lock or unlock calls throw because the g.m.lock count is
corrupted, we're unlikely to get a stack trace because startpanic_m
will itself attempt to acquire a lock, causing a recursive failure.
Avoid this by forcing the g.m.locks count to a sane value if it's
currently bad.
This might be enough to get a stack trace from #25128.
Change-Id: I52d7bd4717ffae94a821f4249585f3eb6cd5aa41
Reviewed-on: https://go-review.googlesource.com/120416
Reviewed-by: Ian Lance Taylor <iant@golang.org>
ARMv8.1 has added new instruction (LDADDAL) for atomic memory operations. This
CL improves existing atomic add intrinsics with the new instruction. Since the
new instruction is only guaranteed to be present after ARMv8.1, we guard its
usage with a conditional on CPU feature.
Performance result on ARMv8.1 machine:
name old time/op new time/op delta
Xadd-224 1.05µs ± 6% 0.02µs ± 4% -98.06% (p=0.000 n=10+8)
Xadd64-224 1.05µs ± 3% 0.02µs ±13% -98.10% (p=0.000 n=9+10)
[Geo mean] 1.05µs 0.02µs -98.08%
Performance result on ARMv8.0 machine:
name old time/op new time/op delta
Xadd-46 538ns ± 1% 541ns ± 1% +0.62% (p=0.000 n=9+9)
Xadd64-46 505ns ± 1% 508ns ± 0% +0.48% (p=0.003 n=9+8)
[Geo mean] 521ns 524ns +0.55%
Change-Id: If4b5d8d0e2d6f84fe1492a4f5de0789910ad0ee9
Reviewed-on: https://go-review.googlesource.com/81877
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
num cpu unit test fixes for FreeBSD.
cpuset -g can possibly output more
data than expected.
Fixes#25924
Change-Id: Iec45a919df68648759331da7cd1fa3b9f3ca4241
GitHub-Last-Rev: 4cc275b519
GitHub-Pull-Request: golang/go#25931
Reviewed-on: https://go-review.googlesource.com/119376
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Most (all?) released versions of lldb don't support compressed DWARF.
For now, skip the test if lldb can't find where to put the breakpoint.
This is the best I could think of -- there is no explicit error that I
can find that indicates it couldn't load the DWARF.
Fixes#25925.
Change-Id: Ib8fa486a04940cee5959ba7aab7bdbbaa3b2974e
Reviewed-on: https://go-review.googlesource.com/119535
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
TestMapping assumed that there was only one mapping entry corresponding
to /exe/main, but that is not always true.
This CL changes the test logic to examine whether all referenced mappings
are symbolized. Based on the result, the test determines whether the
corresponding mapping entries' HasFunctions fields to be true or false.
I initially attempted to create two mappings for referenced locations
(one for symbolized and another for unsymbolized) as described in the
TODO in proto.go as part of fixing this bug. But that change requires
non-trivial modification in the upstream profile package so I decided
to just fix the test for now.
Fixes#25891
Change-Id: Id27a5b07bb5b59e133755a0f863bf56c0a4f7f2b
Reviewed-on: https://go-review.googlesource.com/119455
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL removes the last of the direct system calls in the runtime package.
This is the last CL for 1.11.
Use libcCall instead of asmcgocall in a few places I accidentally used
the wrong one.
For 1.12, we need to think about whether/how the syscall package
should be moved over to libc.
Update #17490
Change-Id: I4f0bd9cd6023f662f2e29588266fdfae5233898f
Reviewed-on: https://go-review.googlesource.com/118736
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit enables vet/all for the js/wasm architecture. It got
skipped initially because the codebase did not fully compile yet
for js/wasm, which made vet/all fail.
startTimer and stopTimer are not needed in the syscall package.
Removed their assembly code since their Go stubs were already gone.
Change-Id: Icaeb6d903876e51ceb1edff7631f715a98c28696
Reviewed-on: https://go-review.googlesource.com/118657
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.
JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.
Updates #18892
Updates #25506
Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
Reviewed-on: https://go-review.googlesource.com/114197
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The TestMapping test invokes the go tool in an exec.Command by
directly hard-coding a "go" string for the command. This can cause
test failures on systems where the "go" command points to an old
toolchain where the test is not supposed to work.
Use testenv.GoToolPath instead.
Also call 'go run' directly on the mappingtest/main.go file instead of
go-running the directory.
Change-Id: Ib91877c021209cbf4da50a561737d7a9d42c6adc
Reviewed-on: https://go-review.googlesource.com/118662
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
kqueue, kevent, closeonexec, setitimer, with sysctl and fcntl helpers.
TODO:arm,arm64
Change-Id: I9386f377186d6ac7cb99064c524a67e0c8282eba
Reviewed-on: https://go-review.googlesource.com/118561
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
The test requires cgo
Change-Id: I1bffee5f187afcf4b7e27516451c56ddfc263a26
Reviewed-on: https://go-review.googlesource.com/118638
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The pprof tool utilizes attributes of mapping entries
such as HasFunctions to determine whether the profile
includes necessary symbol information.
If none of the attributes is set, pprof tool tries to
read the corresponding binary to use for local symbolization.
If the binary doesn't exist, it prints out error messages.
Go runtime generated profiles without any of the attributes
set so the pprof tool always printed out the error messages.
The error messages became more obvious with the new
terminal support that uses red color for error messages.
Go runtime can symbolize all Go symbols and generate
self-contained profile for pure Go program. Thus, there
is no reason for the pprof tool to look for the copy of
the binary. So, this CL sets one of the attributes
(HasFunctions) true if all PCs in samples look fully
symbolized.
For non-pure Go program, however, it's possible that
symbolization of non-Go PCs is incomplete. In this case,
we need to leave the attributes all false so pprof can attempt
to symbolize using the local copy of the binary if available.
It's hard to determine whether a mapping includes non-Go
code. Instead, this CL checks PCs from collected samples.
If unsuccessful symbolization is observed, it skips setting
the HasFunctions attribute.
Fixes#25743
Change-Id: I5108be45bbc37ab486d145fa03e7ce37d88fad50
Reviewed-on: https://go-review.googlesource.com/118275
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Also:
- Add extra SystemStack space for darwin/arm64 just
like for darwin/arm.
- Removed redundant stack alignment; the arm64 hardware enforces
the 16 byte alignment.
- Save and restore the g registers at library initialization.
- Zero g registers since libpreinit can call libc functions
that in turn use asmcgocall. asmcgocall requires an initialized g.
- Change asmcgocall to work even if no g is set. The change mimics
amd64.
Change-Id: I1b8c63b07cfec23b909c0d215b50dc229f8adbc8
Reviewed-on: https://go-review.googlesource.com/117176
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
sigaction, sigprocmask, sigaltstack, and raiseproc.
Fix bug in mstart_stub where we weren't saving callee-saved registers,
so if an m finished the pthread library calling mstart_stub would
sometimes fail.
Update #17490
Update #22805
Change-Id: Ie297ede0997910aa956834e49e85711b90cdfaa7
Reviewed-on: https://go-review.googlesource.com/116875
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
For memmove/memclr using jump tables only reduces overall
function performance for both amd64 and 386.
Benchmarks for 32-bit memclr:
name old time/op new time/op delta
Memclr/5-8 8.01ns ± 0% 8.94ns ± 2% +11.59% (p=0.000 n=9+9)
Memclr/16-8 9.05ns ± 0% 9.49ns ± 0% +4.81% (p=0.000 n=8+8)
Memclr/64-8 9.15ns ± 0% 9.49ns ± 0% +3.76% (p=0.000 n=9+10)
Memclr/256-8 16.6ns ± 0% 16.6ns ± 0% ~ (p=1.140 n=10+9)
Memclr/4096-8 179ns ± 0% 166ns ± 0% -7.26% (p=0.000 n=9+8)
Memclr/65536-8 3.36µs ± 1% 3.31µs ± 1% -1.48% (p=0.000 n=10+9)
Memclr/1M-8 59.5µs ± 3% 60.5µs ± 2% +1.67% (p=0.009 n=10+10)
Memclr/4M-8 239µs ± 3% 245µs ± 0% +2.49% (p=0.004 n=10+8)
Memclr/8M-8 618µs ± 2% 614µs ± 1% ~ (p=0.315 n=10+8)
Memclr/16M-8 1.49ms ± 2% 1.47ms ± 1% -1.11% (p=0.029 n=10+10)
Memclr/64M-8 7.06ms ± 1% 7.05ms ± 0% ~ (p=0.573 n=10+8)
[Geo mean] 3.36µs 3.39µs +1.14%
For less predictable data, like loop iteration dependant sizes,
branch table still shows 2-5% worse results.
It also makes code slightly more complicated.
This CL removes TODO note that directly suggest trying this
optimization out. That encourages people to spend their time
in a quite hopeless endeavour.
The code used to implement branch table used a 32/64-entry table
with pointers to TEXT blocks that implemented every associated
label work. Most last entries point to "loop" code that is
a fallthrough for all other sizes that do not map into specialized
routines. The only inefficiency is extra MOVL/MOVQ required
to fetch table pointer itself as MOVL $sym<>(SB)(AX*4) is not valid
in Go asm (it works in other assemblers):
TEXT ·memclrNew(SB), NOSPLIT, $0-8
MOVL ptr+0(FP), DI
MOVL n+4(FP), BX
// Handle 0 separately.
TESTL BX, BX
JEQ _0
LEAL -1(BX), CX // n-1
BSRL CX, CX
// AX or X0 zeroed inside every text block.
MOVL $memclrTable<>(SB), AX
JMP (AX)(CX*4)
_0:
RET
Change-Id: I4f706931b8127f85a8439b95834d5c2485a5d1bf
Reviewed-on: https://go-review.googlesource.com/115678
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This adds the support to enable the race detector for ppc64le.
Added runtime/race_ppc64le.s to manage the calls from Go to the
LLVM tsan functions, mostly converting from the Go ABI to the
PPC64 ABI expected by Clang generated code.
Changed racewalk.go to call racefuncenterfp instead of racefuncenter
on ppc64le to allow the caller pc to be obtained in the asm code
before calling the tsan version.
Changed the set up code for racecallbackthunk so it doesn't use
the autogenerated save and restore of the link register since that
sequence uses registers inconsistent with the normal ppc64 ABI.
Made various changes to recognize that race is supported for
ppc64le.
Ensured that tls_g is updated and accessible from race_linux_ppc64le.s
so that the race ctx can be obtained and passed to tsan functions.
This enables the race tests for ppc64le in cmd/dist/test.go and
increases the timeout when running the benchmarks with the -race
option to avoid timing out.
Updates #24354, #23731
Change-Id: Ib97dc7ac313e6313c836dc7d2fb698f9d8fba3ef
Reviewed-on: https://go-review.googlesource.com/107935
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When using plugins with goroutines calling cgo, we hit a case where
an intermittent SIGSEGV occurs when referencing an address that is based
on r2 (TOC address). When the failure can be generated in gdb, the
contents of r2 is wrong even though the value in the current stack's
slot for r2 is correct. So that means it somehow switched to start
running the code in this function without passing through the beginning
of the function which had the correct value of r2 and stored it there.
It was noted that in runtime.gogo when the state is restored from
gobuf, r2 is not restored from its slot on the stack. Adding the
instruction to restore r2 prevents the SIGSEGV.
This adds a testcase under testplugin which reproduces the problem if
the program is run multiple times. The team who reported this problem
has verified it fixes the issue on their larger, more complex
application.
Fixes#25756
Change-Id: I6028b6f1f8775d5c23f4ebb57ae273330a28eb8f
Reviewed-on: https://go-review.googlesource.com/117515
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
On 32-bit architectures without native 64-bit atomic instructions,
64-bit atomics are emulated using spinlocks. However,
the sigprof handling code expects to be able to perform
64-bit atomic operations in signal handlers. Spinning on an
acquired spinlock in a signal handler leads to a livelock.
This is issue #20146.
The original fix for #20146 did not include arm in
the list of architectures that need to work around portability
issues in the sigprof handler code. The unit test designed to
catch this issue does not fail on arm builds because arm uses
striped spinlocks, and thus the livelock takes many minutes
to reproduce. This is issue #24260. (This patch doesn't completely
fix#24260 on go1.10.2 due to issue #25785, which is probably
related to the arm cas kernel helpers. Those have been removed
at tip.)
With this patch applied, I was able to run the reproducer for
issue #24260 for more than 90 minutes without reproducing the
livelock. Without this patch, the livelock took as little as
8 minutes to reproduce.
Fixes#20146
Updates #24260
Change-Id: I64bf53a14d53c4932367d919ac55e17c99d87484
Reviewed-on: https://go-review.googlesource.com/117057
Run-TryBot: Philip Hofer <phofer@umich.edu>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
They are unused since CL 114799.
Also remove consts _CTL_KERN and _KERN_OSRELEASE previously used by
getDarwinVersion.
Change-Id: I51b701e8effbe4dd4301b0e6d52e8885469032f4
Reviewed-on: https://go-review.googlesource.com/116955
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This line of the inlining tuning experiment
https://go-review.googlesource.com/c/go/+/109918/1/src/cmd/compile/internal/gc/inl.go#347
was incorrectly rewritten in a later patch to use the call
cost, not the panic cost, and thus the inlining of panic
didn't occur when it should. I discovered this when I
realized that tests should have failed, but didn't.
Fix is to make the correct change, and also to modify the
tests that this causes to fail. One test now asserts the
new normal, the other calls "ppanic" instead which is
designed to behave like panic but not be inlined.
Change-Id: I423bb7f08bd66a70d999826dd9b87027abf34cdf
Reviewed-on: https://go-review.googlesource.com/116656
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Adds an extra M in mstartm0 and accounts for it in checkdead. This allows Windows callbacks created with syscall.NewCallback and syscall.NewCallbackCDecl to be called on a non-Go thread.
Fixes#6751
Change-Id: I57626bc009a6370b9ca0827ab64b14b01dec39d4
GitHub-Last-Rev: d429e3eed9
GitHub-Pull-Request: golang/go#25575
Reviewed-on: https://go-review.googlesource.com/114802
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If we run into data corruption due to the program accessing timers in
a racy way, do a normal panic rather than a hard crash with "panic
holding locks". The hope is to make the problem less confusing for users.
Fixes#25686
Change-Id: I863417adf21f7f8c088675b67a3acf49a0cdef41
Reviewed-on: https://go-review.googlesource.com/115815
Reviewed-by: Austin Clements <austin@google.com>
This commit adds the js/wasm architecture to the net package.
The net package is not supported by js/wasm, but a simple fake
networking is available so tests of other packages that require
basic TCP sockets can pass. The tests of the net package itself
are mostly disabled.
Updates #18892
Change-Id: Id287200c39f0a3e23d20ef17260ca15ccdcca032
Reviewed-on: https://go-review.googlesource.com/109995
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
These were generated using the racebuild configuration from
https://golang.org/cl/115375, with the LLVM compiler-rt repository at
commit fe2c72c59aa7f4afa45e3f65a5d16a374b6cce26 for most platforms.
The Windows build is from an older compiler-rt revision, because the
compiler-rt build script for the Go race detector has been broken
since January 2017 (https://reviews.llvm.org/D28596).
Updates #24354.
Change-Id: Ica05a5d0545de61172f52ab97e7f8f57fb73dbfd
Reviewed-on: https://go-review.googlesource.com/112896
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Each URL was manually verified to ensure it did not serve up incorrect
content.
Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This is a documentation error.
Change-Id: I083021f151f7e80a0b9083b98452ae1f5920640d
Reviewed-on: https://go-review.googlesource.com/115598
Reviewed-by: Peter Weinberger <pjw@google.com>
This commit adds the js/wasm architecture to the os package.
Access to the actual file system is supported through Node.js.
Updates #18892
Change-Id: I6fa642fb294ca020b2c545649d4324d981aa0408
Reviewed-on: https://go-review.googlesource.com/109977
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The profiler reports "ExternalCode" when a profiler interrupt happens
while in libc code. Instead, keep track of the most recent Go frame
for the profiler to use.
There is a test for this using time.Now (runtime.TestTimePprof),
which will work once time.Now is moved to using libc (my next CL).
Change-Id: I940ea83edada482a482e2ab103d3a65589979464
Reviewed-on: https://go-review.googlesource.com/114798
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In golang.org/cl/102355 I mistakenly used 26 instead of 25 as the AT_HWCAP value.
26 is AT_HWCAP2. While experimenting with FreeBSD-11.2-BETA3 (where both values are
being supplied in the auxv), the AT_HWCAP2 value read is 0 which triggers the error again:
runtime: this CPU has no floating point hardware, so it cannot run this GOARM=7 binary. Recompile using GOARM=5.
Updates #24507.
Change-Id: Ide04b7365d8f10e4650edf4e188dd58bdf42cc26
Reviewed-on: https://go-review.googlesource.com/114822
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This also allows the GODEBUGCPU options to change the
support_* runtime cpu feature variable values.
Change-Id: I884c5f03993afc7e3344ff2fd471a2c6cfde43d4
Reviewed-on: https://go-review.googlesource.com/114615
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TestMutexProfile and TestEmptyCallStack couldn't run multiple times
because they mutate state in runtime (mutex profile counters and
a user-defined profile type) and test whether the state
matches what it is supposed to be after the very first run.
We fix TestMutexProfile by relaxing the expected state condition.
We fix TestEmptyCallStack by creating a new profile with a different
name every time the test runs.
For #25520
Change-Id: I8e50cd9526eb650c8989457495ff90a24ce07863
Reviewed-on: https://go-review.googlesource.com/114495
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TestDebugCall* uses atomic spin loops and hence can deadlock if the
garbage collector is enabled (because of #10958; ironically,
implementing debugger call injection is closely related to fixing this
exact issue, but we're not there yet).
Fix this by disabling the garbage collector during these tests.
Updates #25519 (might fix it, though I suspect not)
Change-Id: If1e454b9cdea8e4b1cd82509b762c75b6acd8476
Reviewed-on: https://go-review.googlesource.com/114086
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Traceback matches the defer stack with the function call stack using
the SP recorded in defer frames when the defer frame is created.
However, on LR machines this is ambiguous: if function A pushes a
defer and then calls function B, where B is a leaf function with a
zero-sized frame, then both A and B have the same SP and will *both*
match the defer on the defer stack. Since traceback unwinds through B
first, it will incorrectly match up the defer with B's frame instead
of A's frame.
Where this goes particularly wrong is if function B causes a signal
that turns into a panic (e.g., a nil pointer dereference). In order to
handle the fact that we may not have a liveness map at the location
that caused the signal and injected a sigpanic call, traceback has
logic to unwind the panicking frame's continuation PC to the PC where
the most recent defer was pushed (this is safe because the frame is
dead other than any defers it pushed). However, if traceback
mis-matches the defer stack, it winds up reporting the B's
continuation PC is in A. If the runtime then uses this continuation PC
to look up PCDATA in B, it will panic because the PC is out of range
for B. This failure mode can be seen in
sync/atomic/atomic_test.go:TestNilDeref. An example failure is:
https://build.golang.org/log/8e07a762487839252af902355f6b1379dbd463c5
This CL fixes all of this by recognizing that a function that pushes a
defer must also have a non-zero-sized frame and using this fact to
refine the defer matching logic.
Fixes the build for arm64, mips, mipsle, ppc64, ppc64le, and s390x.
Fixes#25499.
Change-Id: Iff7c01d08ad42f3de22b3a73658cc2f674900101
Reviewed-on: https://go-review.googlesource.com/114078
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Needs the go compiler to be build with GOEXPERIMENT=debugcpu to be active.
The GODEBUGCPU environment variable can be used to disable usage of
specific processor features in the Go standard library.
This is useful for testing and benchmarking different code paths that
are guarded by internal/cpu variable checks.
Use of processor features can not be enabled through GODEBUGCPU.
To disable usage of AVX and SSE41 cpu features on GOARCH amd64 use:
GODEBUGCPU=avx=0,sse41=0
The special "all" option can be used to disable all options:
GODEBUGCPU=all=0
Updates #12805
Updates #15403
Change-Id: I699c2e6f74d98472b6fb4b1e5ffbf29b15697aab
Reviewed-on: https://go-review.googlesource.com/91737
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds a mechanism for debuggers to safely inject calls to Go
functions on amd64. Debuggers must participate in a protocol with the
runtime, and need to know how to lay out a call frame, but the runtime
support takes care of the details of handling live pointers in
registers, stack growth, and detecting the trickier conditions when it
is unsafe to inject a user function call.
Fixes#21678.
Updates derekparker/delve#119.
Change-Id: I56d8ca67700f1f77e19d89e7fc92ab337b228834
Reviewed-on: https://go-review.googlesource.com/109699
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Now that raise on darwin targets the current thread, we can remove
the workaround in dieFromSignal.
Change-Id: I1e468dc05e49403ee0bbe0a3a85e764c81fec4f2
Reviewed-on: https://go-review.googlesource.com/110476
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This CL is the darwin/arm and darwin/arm64 equivalent to CL 108679,
110215, 110437, 110438, 111258, 110655.
Updates #17490
Change-Id: Ia95b27b38f9c3535012c566f17a44b4ed26b9db6
Reviewed-on: https://go-review.googlesource.com/111015
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
pthread_self and pthread_kill are not safe to call from a signal
handler. In particular, pthread_self fails in iOS when called from
a signal handler context.
Use raise instead; it is signal handler safe and simpler.
Change-Id: I0cbfe25151aed245f55d7b76719ce06dc78c6a75
Reviewed-on: https://go-review.googlesource.com/113877
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
When an object spans heap arenas, its bitmap is discontiguous, so
heapBitsSetType unrolls the bitmap into the object itself and then
copies it out to the real heap bitmap. Unfortunately, since this code
path is rare, it had two unnoticed bugs related to the head and tail
of the bitmap:
1. At the head of the object, we were using hbitp as the destination
bitmap pointer rather than h.bitp, but hbitp points into the
*temporary* bitmap space (that is, the object itself), so we were
failing to copy the partial bitmap byte at the head of an object.
2. The core copying loop copied all of the full bitmap bytes, but
always drove the remaining word count down to 0, even if there was a
partial bitmap byte for the tail of the object. As a result, we never
wrote partial bitmap bytes at the tail of an object.
I found these by enabling out-of-place unrolling all the time. To
improve our chances of detecting these sorts of bugs in the future,
this CL mimics this by enabling out-of-place mode 50% of the time when
doubleCheck is enabled so that we test both in-place and out-of-place
mode.
Change-Id: I69e5d829fb3444be4cf11f4c6d8462c26dc467e8
Reviewed-on: https://go-review.googlesource.com/110995
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
The function signatures in the comments used a C-like style. Using
Go function signatures is cleaner.
Change-Id: I1a093ed8fe5df59f3697c613cf3fce58bba4f5c1
Reviewed-on: https://go-review.googlesource.com/113876
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Use mach_absolute_time and mach_timebase_info to get nanosecond-level
timing information from libc on Darwin.
The conversion code from Apple's arbitrary time unit to nanoseconds is
really annoying. It would be nice if we could replace the internal
runtime "time" with arbitrary units and put the conversion to nanoseconds
only in the places that really need it (so it isn't in every nanotime call).
It's especially annoying because numer==denom==1 for all the machines
I tried. Makes it hard to test the conversion code :(
Update #17490
Change-Id: I6c5d602a802f5c24e35184e33d5e8194aa7afa86
Reviewed-on: https://go-review.googlesource.com/110655
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A few libc_ calls were missing stack switches.
Unfortunately, adding the stack switches revealed a deeper problem.
systemstack() is fundamentally flawed because when you do
systemstack(func() { ... })
There's no way to mark the anonymous function as nosplit. At first I
thought it didn't matter, as that function runs on the g0 stack. But
nosplit is still required, because some syscalls are done when stack
bounds are not set up correctly (e.g. in a signal handler, which runs
on the g0 stack, but g is still pointing at the g stack). Instead use
asmcgocall and funcPC, so we can be nosplit all the way down.
Mid-stack inlining now pushes darwin over the nosplit limit also.
Leaving that as a TODO.
Update #23168
This might fix the cause of occasional darwin hangs.
Update #25181
Update #17490
Change-Id: If9c3ef052822c7679f5a1dd192443f714483327e
Reviewed-on: https://go-review.googlesource.com/111258
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This block of code once was commented by the original author, but commenting
code looks a little annoying. However, the debugSelect flag is just for the
situation that debug code will be compiled when debuging, when release this
code will be eliminated by the compiler.
Change-Id: I7b94297e368b515116ef44a36058214ddddf9adb
Reviewed-on: https://go-review.googlesource.com/113395
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When a variable of type int is compared with sizeof's return
value, gcc warns:
comparison between signed and unsigned integer expressions
Change the type of a couple loop indices that looped over sizeof from
int to size_t to silence the warnings.
Fixes#25411
Change-Id: I2c7858f84237e77945651c7b1b6a75b97edcef65
Reviewed-on: https://go-review.googlesource.com/113335
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently we have two nearly identical copies of the code that fetches
the locals and arguments liveness maps for a frame, plus a third
that's a poor knock-off. Unify these all into a single function.
Change-Id: Ibce7926a0b0e3d23182112da4e25df899579a585
Reviewed-on: https://go-review.googlesource.com/109698
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
replace map clears of the form:
for k := range m {
delete(m, k)
}
(where m is map with key type that is reflexive for ==)
with a new runtime function that clears the maps backing
array with a memclr and reinitializes the hmap struct.
Map key types that for example contain floats are not
replaced by this optimization since NaN keys cannot
be deleted from maps using delete.
name old time/op new time/op delta
GoMapClear/Reflexive/1 92.2ns ± 1% 47.1ns ± 2% -48.89% (p=0.000 n=9+9)
GoMapClear/Reflexive/10 108ns ± 1% 48ns ± 2% -55.68% (p=0.000 n=10+10)
GoMapClear/Reflexive/100 303ns ± 2% 110ns ± 3% -63.56% (p=0.000 n=10+10)
GoMapClear/Reflexive/1000 3.58µs ± 3% 1.23µs ± 2% -65.49% (p=0.000 n=9+10)
GoMapClear/Reflexive/10000 28.2µs ± 3% 10.3µs ± 2% -63.55% (p=0.000 n=9+10)
GoMapClear/NonReflexive/1 121ns ± 2% 124ns ± 7% ~ (p=0.097 n=10+10)
GoMapClear/NonReflexive/10 137ns ± 2% 139ns ± 3% +1.53% (p=0.033 n=10+10)
GoMapClear/NonReflexive/100 331ns ± 3% 334ns ± 2% ~ (p=0.342 n=10+10)
GoMapClear/NonReflexive/1000 3.64µs ± 3% 3.64µs ± 2% ~ (p=0.887 n=9+10)
GoMapClear/NonReflexive/10000 28.1µs ± 2% 28.4µs ± 3% ~ (p=0.247 n=10+10)
Fixes#20138
Change-Id: I181332a8ef434a4f0d89659f492d8711db3f3213
Reviewed-on: https://go-review.googlesource.com/110055
Reviewed-by: Keith Randall <khr@golang.org>
Currently isSystemGoroutine has a hard-coded list of known entry
points into system goroutines. This list is annoying to maintain. For
example, it's missing the ensureSigM goroutine.
Replace it with a check that simply looks for any goroutine with
runtime function as its entry point, with a few exceptions. This also
matches the definition recently added to the trace viewer (CL 81315).
Change-Id: Iaed723d4a6e8c2ffb7c0c48fbac1688b00b30f01
Reviewed-on: https://go-review.googlesource.com/81655
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The hmap field in the maptype is only used by the runtime to check the sizes of
the hmap structure created by the compiler and runtime agree.
Comments are already present about the hmap structure definitions in the
compiler and runtime needing to be in sync.
Add a test that checks the runtimes hmap size is as expected to detect
when the compilers and runtimes hmap sizes diverge instead of checking
this at runtime when a map is created.
Change-Id: I974945ebfdb66883a896386a17bbcae62a18cf2a
Reviewed-on: https://go-review.googlesource.com/91796
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
The Go runtime registers a handler for every signal. This prevents Go
binaries from working on QEMU in user-emulation mode, since the hacky
way QEMU implements signals on Linux assumes that no-one uses signal
64 (SIGRTMAX).
In the past, we had a workaround in the runtime to prevent crashes on
start-up when running on QEMU:
golang.org/cl/124900043
golang.org/cl/16853
but it went lost during the 1.11 dev cycle. More precisely, the test
for SIGRTMAX was dropped in CL 18150 when we stopped testing the
result of sigaction in the Linux implementation of setsig. That change
was made to avoid a stack split overflow because code started calling
setsig from nosplit functions. Then in CL 99077 we started testing the
result of sigaction again, this time using systemstack to avoid to
stack split overflow. When this test was added back, we did not bring
back the test of SIGRTMAX.
As a result, Go1.10 binaries work on QEMU, while 1.11 binaries
immediately crash on startup.
This change restores the QEMU workaround.
Updates #24656
Change-Id: I46380b1e1b4bf47db7bc7b3d313f00c4e4c11ea3
Reviewed-on: https://go-review.googlesource.com/111176
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 108095 goes to some length inorder to keep the stack usage of getHPETTimecounter code paths bellow a limit
being checked by the linker analysis. That limit is spurious, when running on the system or signal stack.
In a similar scenario, cgocallback_gofunc performs an indirect call through AX to hide the call from the linker analysis.
Here instead, mark getHPETTimecounter //go:systemstack and call it appropriately.
Change-Id: I80bec5e4974eee3c564d94f6e1142f322df88b2f
Reviewed-on: https://go-review.googlesource.com/111495
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is a follow-up of CL 93637. There, when we redirect sync/atomic
to runtime/internal/atomic, a few good implementations of ARM atomics
were lost. This CL brings most of them back, with some improvements.
- Change atomic Store to a plain store with memory barrier, as we
already changed atomic Load to plain load with memory barrier.
- Use native 64-bit atomics on ARMv7, jump to Go implementations
on older machines. But drop the kernel helper. In particular,
for Load64, just do loads, not using Cas on the address being
load from, so it works also for read-only memory (since we have
already fixed 32-bit Load).
Change-Id: I725cd65cf945ae5200db81a35be3f251c9f7af14
Reviewed-on: https://go-review.googlesource.com/111315
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This gets us around the kernel helpers on ARMv7.
It is slightly faster than using the kernel helper.
name old time/op new time/op delta
AtomicLoad-4 72.5ns ± 0% 69.5ns ± 0% -4.08% (p=0.000 n=9+9)
AtomicStore-4 57.6ns ± 1% 54.4ns ± 0% -5.58% (p=0.000 n=10+9)
[Geo mean] 64.6ns 61.5ns -4.83%
If performance is really critical, we can even do compiler intrinsics
on GOARM=7.
Fixes#23792.
Change-Id: I36497d880890b26bdf01e048b542bd5fd7b17d23
Reviewed-on: https://go-review.googlesource.com/94076
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The implementation of atomics are inherently tricky. It would
be good to have them implemented in a single place, instead of
multiple copies.
Mostly a simple redirect.
On 386, some functions in sync/atomic have better implementations,
which are moved to runtime/internal/atomic.
On ARM, some functions in sync/atomic have better implementations.
They are dropped by this CL, but restored with an improved
version in a follow-up CL. On linux/arm, 64-bit CAS kernel helper
is dropped, as we're trying to move away from kernel helpers.
Fixes#23778.
Change-Id: Icb9e1039acc92adbb2a371c34baaf0b79551c3ea
Reviewed-on: https://go-review.googlesource.com/93637
Reviewed-by: Austin Clements <austin@google.com>
Every time I poke at #14921, the g.waitreason string
pointer writes show up.
They're not particularly important performance-wise,
but it'd be nice to clear the noise away.
And it does open up a few extra bytes in the g struct
for some future use.
This is a re-roll of CL 99078, which was rolled
back because of failures on s390x.
Those failures were apparently due to an old version of gdb.
Change-Id: Icc2c12f449b2934063fd61e272e06237625ed589
Reviewed-on: https://go-review.googlesource.com/111256
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Changes include:
1. open compilation option -msan for arm64
2. modify doc to explain -msan is also supported on linux/arm64
3. wrap msan lib API in msan_arm64.s
4. use libc for sigaction syscalls when cgo is enabled
5. use libc for mmap syscalls when cgo is enabled
Change-Id: I26ebe61ff7ce1906125f54a0182a720f9d58ec11
Reviewed-on: https://go-review.googlesource.com/109255
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Go runtime currently only populates hwcap for ppc64 and arm64.
While the interpretation of hwcap is platform specific the hwcap
information is generally available on linux.
Changing the runtime variable name to cpu_hwcap for cpu.hwcap makes it
consistent with the general naming of runtime variables that are linked
to other packages.
Change-Id: I1e1f932a73ed624a219b9298faafbb6355e47ada
Reviewed-on: https://go-review.googlesource.com/94757
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
R11 is only used as a temporary by a very small set of instructions
(DIV, MOD, MULH and extended MVC/XC instructions). By marking these
instructions as clobbering R11 we can allocate R11 in the general
case.
Change-Id: I0d4ffe80e57c164d42a5ea5ef6308756a5b0f742
Reviewed-on: https://go-review.googlesource.com/110255
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This is the scanstack analog of CL 104737,
which made a similar change for copystack.
name old time/op new time/op delta
ScanStack-8 41.1ms ± 6% 38.9ms ± 5% -5.52% (p=0.000 n=50+48)
Change-Id: I7427151dea2895ed3934f8a0f61d96b568019217
Reviewed-on: https://go-review.googlesource.com/105536
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
There are many possible stack scanning benchmarks,
but this one is at least a start.
cpuprofiling shows about 75% of CPU in func scanstack.
Change-Id: I906b0493966f2165c1920636c4e057d16d6447e0
Reviewed-on: https://go-review.googlesource.com/105535
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Moving mmap, munmap, madvise, usleep.
Also introduce __error function to get at libc's errno variable.
Change-Id: Ic47ac1d9eb71c64ba2668ce304644dd7e5bdfb5a
Reviewed-on: https://go-review.googlesource.com/110437
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Make selectgo return recvOK as a result parameter instead.
Change-Id: Iffd436371d360bf666b76d4d7503e7c3037a9f1d
Reviewed-on: https://go-review.googlesource.com/37935
Reviewed-by: Austin Clements <austin@google.com>
Now the registration phase looks like:
var cases [4]runtime.scases
var order [8]uint16
selectsend(&cases[0], c1, &v1)
selectrecv(&cases[1], c2, &v2, nil)
selectrecv(&cases[2], c3, &v3, &ok)
selectdefault(&cases[3])
chosen := selectgo(&cases[0], &order[0], 4)
Primarily, this is just preparation for having the compiler open-code
selectsend, selectrecv, and selectdefault.
As a minor benefit, order can now be layed out separately on the stack
in the pointer-free segment, so it won't take up space in the
function's stack pointer maps.
Change-Id: I5552ba594201efd31fcb40084da20b42ea569a45
Reviewed-on: https://go-review.googlesource.com/37933
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Convert raise from raw syscalls to using the system pthread library.
As a bonus, raise will now target the current thread instead of the
process.
Updates #17490
Change-Id: I2e44f2000bf870e99a5b4dc5ff5e0799fba91bde
Reviewed-on: https://go-review.googlesource.com/110475
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Now we no longer need to mess with TLS on Darwin 386/amd64, we always
rely on the pthread library to set it up. We now just use one entry
in the TLS for the G.
Return from mstart to let the pthread library clean up the OS thread.
Change-Id: Iccf58049d545515d9b1d090b161f420e40ffd244
Reviewed-on: https://go-review.googlesource.com/110215
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 93658 moved stack trace printing inside a systemstack call to
sidestep complexity in case the runtime is in a inconsistent state.
Unfortunately, debuggers generating backtraces for a Go panic
will be confused and come up with a technical correct but useless
stack. This CL moves just the crash performing - typically a SIGABRT
signal - outside the systemstack call to improve backtraces.
Unfortunately, the crash function now needs to be marked nosplit and
that triggers the no split stackoverflow check. To work around that,
split fatalpanic in two: fatalthrow for runtime.throw and fatalpanic for
runtime.gopanic. Only Go panics really needs crashes on the right stack
and there is enough stack for gopanic.
Example program:
package main
import "runtime/debug"
func main() {
debug.SetTraceback("crash")
crash()
}
func crash() {
panic("panic!")
}
Before:
(lldb) bt
* thread #1, name = 'simple', stop reason = signal SIGABRT
* frame #0: 0x000000000044ffe4 simple`runtime.raise at <autogenerated>:1
frame #1: 0x0000000000438cfb simple`runtime.dieFromSignal(sig=<unavailable>) at signal_unix.go:424
frame #2: 0x0000000000438ec9 simple`runtime.crash at signal_unix.go:525
frame #3: 0x00000000004268f5 simple`runtime.dopanic_m(gp=<unavailable>, pc=<unavailable>, sp=<unavailable>) at panic.go:758
frame #4: 0x000000000044bead simple`runtime.fatalpanic.func1 at panic.go:657
frame #5: 0x000000000044d066 simple`runtime.systemstack at <autogenerated>:1
frame #6: 0x000000000042a980 simple at proc.go:1094
frame #7: 0x0000000000438ec9 simple`runtime.crash at signal_unix.go:525
frame #8: 0x00000000004268f5 simple`runtime.dopanic_m(gp=<unavailable>, pc=<unavailable>, sp=<unavailable>) at panic.go:758
frame #9: 0x000000000044bead simple`runtime.fatalpanic.func1 at panic.go:657
frame #10: 0x000000000044d066 simple`runtime.systemstack at <autogenerated>:1
frame #11: 0x000000000042a980 simple at proc.go:1094
frame #12: 0x00000000004268f5 simple`runtime.dopanic_m(gp=<unavailable>, pc=<unavailable>, sp=<unavailable>) at panic.go:758
frame #13: 0x000000000044bead simple`runtime.fatalpanic.func1 at panic.go:657
frame #14: 0x000000000044d066 simple`runtime.systemstack at <autogenerated>:1
frame #15: 0x000000000042a980 simple at proc.go:1094
frame #16: 0x000000000044bead simple`runtime.fatalpanic.func1 at panic.go:657
frame #17: 0x000000000044d066 simple`runtime.systemstack at <autogenerated>:1
After:
(lldb) bt
* thread #7, stop reason = signal SIGABRT
* frame #0: 0x0000000000450024 simple`runtime.raise at <autogenerated>:1
frame #1: 0x0000000000438d1b simple`runtime.dieFromSignal(sig=<unavailable>) at signal_unix.go:424
frame #2: 0x0000000000438ee9 simple`runtime.crash at signal_unix.go:525
frame #3: 0x00000000004264e3 simple`runtime.fatalpanic(msgs=<unavailable>) at panic.go:664
frame #4: 0x0000000000425f1b simple`runtime.gopanic(e=<unavailable>) at panic.go:537
frame #5: 0x0000000000470c62 simple`main.crash at simple.go:11
frame #6: 0x0000000000470c00 simple`main.main at simple.go:6
frame #7: 0x0000000000427be7 simple`runtime.main at proc.go:198
frame #8: 0x000000000044ef91 simple`runtime.goexit at <autogenerated>:1
Updates #22716
Change-Id: Ib5fa35c13662c1dac2f1eac8b59c4a5824b98d92
Reviewed-on: https://go-review.googlesource.com/110065
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The general policy for the current state of js/wasm is that it only
has to support tests that are also supported by nacl.
The test nilptr3.go makes assumptions about which nil checks can be
removed. Since WebAssembly does not signal on reading a null pointer,
all nil checks have to be explicit.
Updates #18892
Change-Id: I06a687860b8d22ae26b1c391499c0f5183e4c485
Reviewed-on: https://go-review.googlesource.com/110096
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Missed conversion of newosproc for the parts of darwin that
weren't affected by my previous change.
Update #25181
Change-Id: I81a2935e192b6d0df358c59b7e785eb03c504c23
Reviewed-on: https://go-review.googlesource.com/110123
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Some of the comments relative paths do not exist and
reflect does not define its own hmap structure.
Correct paths and consistently reference paths starting from the
go src directory.
Change-Id: I5204a3a98f77d65f17dcde98b847378cea05ad8a
Reviewed-on: https://go-review.googlesource.com/94758
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Replace thread creation with calls to the pthread
library in libc.
Update #17490
Change-Id: I1e19965c45255deb849b059231252fc6a7861d6c
Reviewed-on: https://go-review.googlesource.com/108679
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There are several things combined in this change.
First, eliminate the gobitvector type in favor
of adding a ptrbit method to bitvector.
In non-performance-critical code, use that method.
In performance critical code, though, load the bitvector data
one byte at a time and iterate only over set bits.
To support that, add and use sys.Ctz8.
name old time/op new time/op delta
StackCopyPtr-8 81.8ms ± 5% 78.9ms ± 3% -3.58% (p=0.000 n=97+96)
StackCopy-8 65.9ms ± 3% 62.8ms ± 3% -4.67% (p=0.000 n=96+92)
StackCopyNoCache-8 105ms ± 3% 102ms ± 3% -3.38% (p=0.000 n=96+95)
Change-Id: I00b80f45612708bd440b1a411a57fa6dfa24aa74
Reviewed-on: https://go-review.googlesource.com/109716
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Ian Lance Taylor
Keith Randall
Austin Clements
Russ Cox
Michael Munday
Keith Randall
LE Manh Cuong
Ben Shi
Keith Randall
Cherry Zhang
Michael Anthony Knyszek
Joel Sing
Joey
Clément Chigot
Tamir Duberstein
Lorenz Nickel
Iskander Sharipov
David Chase
Josh Bleecher Snyder
Joshua M. Clulow
Elias Naur
Richard Musiol
Haosdent Huang
Alessandro Arzilli
Maya Rashish
Brad Fitzpatrick
erifan01
Matthew Dempsky
Nigel Tao
Andrei Vagin
Jingnan Si
grant
Mikio Hara
Martin Möhrmann
Ilya Tocar
Jason A. Donenfeld
Carlo Alberto Ferraris
Bryan C. Mills
Daniel Martí
Philipp Stephani
Komu Wairagu
Marcel van Lohuizen
Leon Klingele
Tobias Klauser
Alex Brainman
Raul Silvera
Yuval Pavel Zholkover
Inada Naoki
Stepan Shabalin
Daniel Ingram
Michael McLoughlin
Jordan Rhee
Elias Naur
Mark Pulford
Lynn Boger
Gergely Brautigam
majiang
Hana Kim
Fangming.Fang
Ali Rizvi-Santiago
Nikhil Benesch
Ainar Garipov
Hajime Hoshi
Dmitry Vyukov
Carlos Eduardo Seo
Ivan Sharavuev
Muhammad Falak R Wani
Igor Zhilianin
Alberto Donizetti
Zhou Peng
Shulhan
Koichi Shiraishi
David du Colombier
Emmanuel T Odeke
Michael Pratt
Marko Kevac
Charles Kenney
Than McIntosh
Alexey Alexandrov
Zheng Xu
Heschi Kreinick
Kazuhiro Sera
Zachary Amsden
Alan Donovan
Iskander Sharipov
Thanabodee Charoenpiriyakij
Roland Illig
Shivansh Rai
David Wimmer
Jeet Parekh
Xia Bin
Kamil Rytarowski
cch123
Vladimir Kuzmin
Jeremy Jackins
Wei Xiao
David Carlier
Hana Kim
philhofer
Bill Zissimopoulos
Robert Griesemer
Tim Cooper
ChrisALiles