Unlike function calls, when processing instructions that directly
fault we must not subtract 1 from the pc before looking up the
file/line information.
Since the file/line lookup unconditionally subtracts 1, add 1 to
the faulting instruction PCs to compensate.
Fixes#34123
Change-Id: Ie7361e3d2f84a0d4f48d97e5a9e74f6291ba7a8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/196962
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This adds pipe/pipe2 on Solaris as they exist on other Unix systems.
They were not added previously because Solaris does not need them
for netpollBreak. They are added now in preparation for using pipes
in TestSignalM.
Updates #35276
Change-Id: I53dfdf077430153155f0a79715af98b0972a841c
Reviewed-on: https://go-review.googlesource.com/c/go/+/206077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Without this CL, one of the TestDebugCall tests would fail 1% to 2% of
the time on the android-amd64-emu gomote. With this CL, I ran the
tests for 1000 iterations with no failures.
Fixes#32985
Change-Id: I541268a2a0c10d0cd7604f0b2dbd15c1d18e5730
Reviewed-on: https://go-review.googlesource.com/c/go/+/205248
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
This change adds a per-p free page cache which the page allocator may
allocate out of without a lock. The change also introduces a completely
lockless page allocator fast path.
Although the cache contains at most 64 pages (and usually less), the
vast majority (85%+) of page allocations are exactly 1 page in size.
Updates #35112.
Change-Id: I170bf0a9375873e7e3230845eb1df7e5cf741b78
Reviewed-on: https://go-review.googlesource.com/c/go/+/195701
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a page cache structure which owns a chunk of free pages
at a given base address. It also adds code to allocate to this cache
from the page allocator. Finally, it adds tests for both.
Notably this change does not yet integrate the code into the runtime,
just into runtime tests.
Updates #35112.
Change-Id: Ibe121498d5c3be40390fab58a3816295601670df
Reviewed-on: https://go-review.googlesource.com/c/go/+/196643
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a per-p mspan object cache similar to the sudog cache.
Unfortunately this cache can't quite operate like the sudog cache, since
it is used in contexts where write barriers are disallowed (i.e.
allocation codepaths), so rather than managing an array and a slice,
it's just an array and a length. A little bit more unsafe, but avoids
any write barriers.
The purpose of this change is to reduce the number of operations which
require the heap lock in allocation, paving the way for a lockless fast
path.
Updates #35112.
Change-Id: I32cfdcd8528fb7be985640e4f3a13cb98ffb7865
Reviewed-on: https://go-review.googlesource.com/c/go/+/196642
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change combines the functionality of allocSpanLocked, allocManual,
and alloc_m into a new method called allocSpan. While these methods'
abstraction boundaries are OK when the heap lock is held throughout,
they start to break down when we want finer-grained locking in the page
allocator.
allocSpan does just that, and only locks the heap when it absolutely has
to. Piggy-backing off of work in previous CLs to make more of span
initialization lockless, this change makes span initialization entirely
lockless as part of the reorganization.
Ultimately this change will enable us to add a lockless fast path to
allocSpan.
Updates #35112.
Change-Id: I99875939d75fb4e958a67ac99e4a7cda44f06864
Reviewed-on: https://go-review.googlesource.com/c/go/+/196641
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently gcSweepBuf guarantees that push operations may be performed
concurrently with each other and that block operations may be performed
concurrently with push operations as well.
Unfortunately, this isn't quite true. The existing code allows push
operations to happen concurrently with each other, but block operations
may return blocks with nil entries. The way this can happen is if two
concurrent pushers grab a slot to push to, and the first one (the one
with the earlier slot in the buffer) doesn't quite write a span value
when the block is called. The existing code in block only checks if the
very last value in the block is nil, when really an arbitrary number of
the last few values in the block may or may not be nil.
Today, this case can't actually happen because when push operations
happen concurrently during a GC (which is the only time block is
called), they only ever happen during an allocation with the heap lock
held, effectively serializing them. A block operation may happen
concurrently with one of these pushes, but its callers will never see a
nil mspan. Outside of a GC, this isn't a problem because although push
operations from allocations can run concurrently with push operations
from sweeping, block operations will never run.
In essence, the real concurrency guarantees provided by gcSweepBuf are
that block operations may happen concurrently with push operations, but
that push operations may not be concurrent with each other if there are
any block operations.
To fix this, and to prepare for push operations happening without the
heap lock held in a future CL, we update the documentation for block to
correctly state that there may be nil entries in the returned slice.
While we're here, make the mspan writes into the buffer atomic to avoid
a block user racing on a nil check, and document that the user should
load mspan values from the returned slice atomically. Finally, we make
all callers of block adhere to the new rules.
We choose to allow nil values rather than filter them out because the
only caller of block is markrootSpans, and if it catches a nil entry,
then there wasn't anything to mark in there anyway since the span is
just being created.
Updates #35112.
Change-Id: I6450aab15f51690d7a000ba5b3d529cf2ca5da1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/203318
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change makes it so that allocation and free related page sweeper
metadata operations (e.g. pageInUse and pagesInUse) are atomic rather
than protected by the heap lock. This will help in reducing the length
of the critical path with the heap lock held in future changes.
Updates #35112.
Change-Id: Ie82bff024204dd17c4c671af63350a7a41add354
Reviewed-on: https://go-review.googlesource.com/c/go/+/196640
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This CL adds support of call injection and async preemption on
PPC64.
For the injected call to return to the preempted PC, we have to
clobber either LR or CTR. For reasons mentioned in previous CLs,
we choose CTR. Previous CLs have marked code sequences that use
CTR async-nonpreemtible.
Change-Id: Ia642b5f06a890dd52476f45023b2a830c522eee0
Reviewed-on: https://go-review.googlesource.com/c/go/+/203824
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
mheap_.alloc currently accepts both a spanClass and a "large" parameter
indicating whether the allocation is large. These are redundant, since
spanClass.sizeclass() == 0 is an equivalent way to determine this and is
already used in mheap_.alloc. There are no places in the runtime where
the size class could be non-zero and large == true.
Updates #35112.
Change-Id: Ie66facf8f0faca6f4cd3d20a8ac4bc259e11823d
Reviewed-on: https://go-review.googlesource.com/c/go/+/196639
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change defines a maximum supported physical and huge page size in
the runtime based on the new page allocator's implementation, and uses
them where appropriate.
Furthemore, if the system exceeds the maximum supported huge page
size, we simply ignore it silently.
It also fixes a huge-page-related test which is only triggered by a
condition which is definitely wrong.
Finally, it adds a few TODOs related to code clean-up and supporting
larger huge page sizes.
Updates #35112.
Fixes#35431.
Change-Id: Ie4348afb6bf047cce2c1433576d1514720d8230f
Reviewed-on: https://go-review.googlesource.com/c/go/+/205937
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
For the most part, heap memstats are already updated atomically when
passed down to OS-level memory functions (e.g. sysMap). Elsewhere,
however, they're updated with the heap lock.
In order to facilitate holding the heap lock for less time during
allocation paths, this change more consistently makes the update of
these statistics atomic by calling mSysStat{Inc,Dec} appropriately
instead of simply adding or subtracting. It also ensures these values
are loaded atomically.
Furthermore, an undocumented but safe update condition for these
memstats is during STW, at which point using atomics is unnecessary.
This change also documents this condition in mstats.go.
Updates #35112.
Change-Id: I87d0b6c27b98c88099acd2563ea23f8da1239b66
Reviewed-on: https://go-review.googlesource.com/c/go/+/196638
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change removes useless additional heap_objects accounting for large
objects. heap_objects is computed from scratch at ReadMemStats time
(which stops the world) by using nlargealloc and nlargefree, so mutating
heap_objects turns out to be pointless.
As a result, the "large" parameter on "mheap_.freeSpan" is no longer
necessary and so this change cleans that up too.
Change-Id: I7d6b486d9b57c018e3db46221d81b55fe4c1b021
Reviewed-on: https://go-review.googlesource.com/c/go/+/196637
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
In preparation for a lockless fast path in the page allocator, this
change makes it so that checking if an allocation needs to be zeroed may
be done atomically.
Unfortunately, this means there is a CAS-loop to ensure monotonicity of
the zeroedBase value in heapArena. This CAS-loop exits if an allocator
acquiring memory further on in the arena wins or if it succeeds. The
CAS-loop should have a relatively small amount of contention because of
this monotonicity, though it would be ideal if we could just have
CAS-ers with the greatest value always win. The CAS-loop is unnecessary
in the steady-state, but should bring some start-up performance gains as
it's likely cheaper than the additional zeroing required, especially for
large allocations.
For very large allocations that span arenas, the CAS-loop should be
completely uncontended for most of the arenas it touches, it may only
encounter contention on the first and last arena.
Updates #35112.
Change-Id: If3d19198b33f1b1387b71e1ce5902d39a5c0f98e
Reviewed-on: https://go-review.googlesource.com/c/go/+/203859
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This test is failing consistently in the longtest builders,
potentially masking regressions in other packages.
Updates #35271
Change-Id: Idc03171c0109b5c8d4913e0af2078c1115666897
Reviewed-on: https://go-review.googlesource.com/c/go/+/206098
Reviewed-by: Carlos Amedee <carlos@golang.org>
The pprof profile proto message expects inlined functions of a PC
to be encoded in one Location entry using multiple Line entries.
https://github.com/google/pprof/blob/5e96527/proto/profile.proto#L177-L184
runtime/pprof has encoded the symbolization information by creating
a Location for each PC found in the stack trace and including info
from all the frames expanded from the PC using runtime.CallersFrames.
This assumes inlined functions are represented as a single PC in the
stack trace. (https://go-review.googlesource.com/41256)
In the recent years, behavior around inlining and the traceback
changed significantly (e.g. https://golang.org/cl/152537,
https://golang.org/issue/29582, and many changes). Now the PCs
in the stack trace represent user frames even including inline
marks. As a result, the profile proto started to allocate a Location
entry for each user frame, lose the inline information (so pprof
presented incorrect results when inlined functions are involved),
and confuse the pprof tool with those PCs made up for inline marks.
This CL attempts to detect inlined call frames from the stack traces
of CPU profiles, and organize the Location information as intended.
Currently, runtime does not provide a reliable and convenient way to
detect inlined call frames and expand user frames from a given externally
recognizable PCs. So we use heuristics to recover the groups
- inlined call frames have nil Func field
- inlined call frames will have the same Entry point
- but must be careful with recursive functions that have the
same Entry point by definition, and non-Go functions that
may lack most of the fields of Frame.
The followup CL will address the issue with other profile types.
Change-Id: I0c9667ab016a3e898d648f31c3f82d84c15398db
Reviewed-on: https://go-review.googlesource.com/c/go/+/204636
Reviewed-by: Keith Randall <khr@golang.org>
This change removes the old page allocator from the runtime.
Updates #35112.
Change-Id: Ib20e1c030f869b6318cd6f4288a9befdbae1b771
Reviewed-on: https://go-review.googlesource.com/c/go/+/195700
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change flips the oldPageAllocator constant enabling the new page
allocator in the Go runtime.
Updates #35112.
Change-Id: I7fc8332af9fd0e43ce28dd5ebc1c1ce519ce6d0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/201765
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This CL adds support of call injection and async preemption on
MIPS and MIPS64.
Like ARM64, we need to clobber one register (REGTMP) for
returning from the injected call. Previous CLs have marked code
sequences that use REGTMP async-nonpreemtible.
It seems on MIPS/MIPS64, a CALL instruction is not "atomic" (!).
If a signal is delivered right at the CALL instruction, we may
see an updated LR with a not-yet-updated PC. In some cases this
may lead to failed stack unwinding. Don't preempt in this case.
Change-Id: I99437b2d05869ded5c0c8cb55265dbfc933aedab
Reviewed-on: https://go-review.googlesource.com/c/go/+/203720
Reviewed-by: Keith Randall <khr@golang.org>
This change adds the allocNeedZero method to mheap which uses the new
heapArena field zeroedBase to determine whether a new allocation needs
zeroing. The purpose of this work is to avoid zeroing memory that is
fresh from the OS in the context of the new allocator, where we no
longer have the concept of a free span to track this information.
The new field in heapArena, zeroedBase, is small, which runs counter to
the advice in the doc comment for heapArena. Since heapArenas are
already not a multiple of the system page size, this advice seems stale,
and we're OK with using an extra physical page for a heapArena. So, this
change also deletes the comment with that advice.
Updates #35112.
Change-Id: I688cd9fd3c57a98a6d43c45cf699543ce16697e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/203858
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This CL adds support of call injection and async preemption on
S390X.
Like ARM64, we need to clobber one register (REGTMP) for
returning from the injected call. Previous CLs have marked code
sequences that use REGTMP async-nonpreemtible.
Change-Id: I78adbc5fd70ca245da390f6266623385b45c9dfc
Reviewed-on: https://go-review.googlesource.com/c/go/+/204106
Reviewed-by: Keith Randall <khr@golang.org>
This change integrates all the bits and pieces of the new page allocator
into the runtime, behind a global constant.
Updates #35112.
Change-Id: I6696bde7bab098a498ab37ed2a2caad2a05d30ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/201764
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently the runtime background scavenger is paced externally,
controlled by a collection of variables which together describe a line
that we'd like to stay under.
However, the line to stay under is computed as a function of the number
of free and unscavenged huge pages in the heap at the end of the last
GC. Aside from this number being inaccurate (which is still acceptable),
the scavenging system also makes an order-of-magnitude assumption as to
how expensive scavenging a single page actually is.
This change simplifies the scavenger in preparation for making it
operate on bitmaps. It makes it so that the scavenger paces itself, by
measuring the amount of time it takes to scavenge a single page. The
scavenging methods on mheap already avoid breaking huge pages, so if we
scavenge a real huge page, then we'll have paced correctly, otherwise
we'll sleep for longer to avoid using more than scavengePercent wall
clock time.
Unfortunately, all this involves measuring time, which is quite tricky.
Currently we don't directly account for long process sleeps or OS-level
context switches (which is quite difficult to do in general), but we do
account for Go scheduler overhead and variations in it by maintaining an
EWMA of the ratio of time spent scavenging to the time spent sleeping.
This ratio, as well as the sleep time, are bounded in order to deal with
the aforementioned OS-related anomalies.
Updates #35112.
Change-Id: Ieca8b088fdfca2bebb06bcde25ef14a42fd5216b
Reviewed-on: https://go-review.googlesource.com/c/go/+/201763
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This CL adds support of call injection and async preemption on
ARM64.
There seems no way to return from the injected call without
clobbering *any* register. So we have to clobber one, which is
chosen to be REGTMP. Previous CLs have marked code sequences
that use REGTMP async-nonpreemtible.
Change-Id: Ieca4e3ba5557adf3d0f5d923bce5f1769b58e30b
Reviewed-on: https://go-review.googlesource.com/c/go/+/203461
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a "locked" parameter to scavenge() and scavengeone()
which allows these methods to be run with the heap lock acquired, and
synchronously with respect to others which acquire the heap lock.
This mode is necessary for both heap-growth scavenging (multiple
asynchronous scavengers here could be problematic) and
debug.FreeOSMemory.
Updates #35112.
Change-Id: I24eea8e40f971760999c980981893676b4c9b666
Reviewed-on: https://go-review.googlesource.com/c/go/+/195699
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This change makes it so that the new page allocator returns the number
of pages that are scavenged in a new allocation so that mheap can update
memstats appropriately.
The accounting could be embedded into pageAlloc, but that would make
the new allocator more difficult to test.
Updates #35112.
Change-Id: I0f94f563d7af2458e6d534f589d2e7dd6af26d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/195698
Reviewed-by: Austin Clements <austin@google.com>
This change adds a scavenger for the new page allocator along with
tests. The scavenger walks over the heap backwards once per GC, looking
for memory to scavenge. It walks across the heap without any lock held,
searching optimistically. If it finds what appears to be a scavenging
candidate it acquires the heap lock and attempts to verify it. Upon
verification it then scavenges.
Notably, unlike the old scavenger, it doesn't show any preference for
huge pages and instead follows a more strict last-page-first policy.
Updates #35112.
Change-Id: I0621ef73c999a471843eab2d1307ae5679dd18d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/195697
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a new bitmap-based allocator to the runtime with tests.
It does not yet integrate the page allocator into the runtime and thus
this change is almost purely additive.
Updates #35112.
Change-Id: Ic3d024c28abee8be8797d3918116a80f901cc2bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/190622
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change ensures js-wasm returns page-aligned memory. While today
its lack of alignment doesn't cause problems, this is an invariant of
sysAlloc which is documented in HACKING.md but isn't upheld by js-wasm.
Any code that calls sysAlloc directly for small structures expects a
certain alignment (e.g. debuglog, tracebufs) but this is not maintained
by js-wasm's sysAlloc.
Where sysReserve comes into play is that sysAlloc is implemented in
terms of sysReserve on js-wasm. Also, the documentation of sysReserve
says that the returned memory is "OS-aligned" which on most platforms
means page-aligned, but the "OS-alignment" on js-wasm is effectively 1,
which doesn't seem right either.
The expected impact of this change is increased memory use on wasm,
since there's no way to decommit memory, and any small structures
allocated with sysAlloc won't be packed quite as tightly. However, any
memory increase should be minimal. Most calls to sysReserve and sysAlloc
already aligned their request to physPageSize before calling it; there
are only a few circumstances where this is not true, and they involve
allocating an amount of memory returned by unsafe.Sizeof where it's
actually quite important that we get the alignment right.
Updates #35112.
Change-Id: I9ca171e507ff3bd186326ccf611b35b9ebea1bfe
Reviewed-on: https://go-review.googlesource.com/c/go/+/205277
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Richard Musiol <neelance@gmail.com>
This change adds the concept of summaries and of summarizing a set of
pallocBits, a core concept in the new page allocator. These summaries
are really just three integers packed into a uint64. This change also
adds tests and a benchmark for generating these summaries.
Updates #35112.
Change-Id: I69686316086c820c792b7a54235859c2105e5fee
Reviewed-on: https://go-review.googlesource.com/c/go/+/190621
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This change adds a per-chunk bitmap for page allocation called
pallocBits with algorithms for allocating and freeing pages out of the
bitmap. This change also adds tests for pallocBits, but does not yet
integrate it into the runtime.
Updates #35112.
Change-Id: I479006ed9f1609c80eedfff0580d5426b064b0ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/190620
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change is the first of a series of changes which replace the
current page allocator (which is based on the contents of mgclarge.go
and some of mheap.go) with one based on free/used bitmaps.
It adds in the key constants for the page allocator as well as a comment
describing the implementation.
Updates #35112.
Change-Id: I839d3a07f46842ad379701d27aa691885afdba63
Reviewed-on: https://go-review.googlesource.com/c/go/+/190619
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This changes makes it so that sysReserve, which creates a PROT_NONE
mapping, maps that memory as NORESERVE. Before this change, relatively
large PROT_NONE mappings could cause fork to fail with ENOMEM, reported
as "not enough space". Presumably this refers to swap space, since
adding this flag causes the failures to go away.
This helps unblock page allocator work, since it allows us to make large
PROT_NONE mappings on solaris safely.
Updates #35112.
Change-Id: Ic3cba310c626e93d5db0f27269e2569bb7bc393e
Reviewed-on: https://go-review.googlesource.com/c/go/+/205759
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When we have already assigned the semaphore ticket to a specific
waiter, we want to get the waiter running as fast as possible since
no other G waiting on the semaphore can acquire it optimistically.
The net effect is that, when a sync.Mutex is contented, the code in
the critical section guarded by the Mutex gets a priority boost.
Fixes#33747
Change-Id: I9967f0f763c25504010651bdd7f944ee0189cd45
Reviewed-on: https://go-review.googlesource.com/c/go/+/200577
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
On iOS, the address space is not 48 bits as one might believe, since
it's arm64 hardware. In fact, all pointers are truncated to 33 bits, and
the OS only gives applications access to the range [1<<32, 2<<32).
While today this has no effect on the Go runtime, future changes which
care about address space size need this to be correct.
Updates #35112.
Change-Id: Id518a2298080f7e3d31cf7d909506a37748cc49a
Reviewed-on: https://go-review.googlesource.com/c/go/+/205758
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This change removes a hack which was added to deal with Darwin 10.10's
weird ignorance of mapping hints which would cause race mode to fail
since it requires the heap to live within a certain address range.
We no longer support 10.10, and this is potentially causing problems
related to the page allocator, so drop this code.
Updates #26475.
Updates #35112.
Change-Id: I0e1c6f8c924afe715a2aceb659a969d7c7b6f749
Reviewed-on: https://go-review.googlesource.com/c/go/+/205757
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The test deliberately constructs an invalid pointer, so don't check it.
Fixes#35379
Change-Id: Ifeff3484740786b0470de3a4d2d4103d91e06f5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/205717
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Otherwise, we can get into a deadlock: sysmon takes the scheduler lock
and calls timeSleepUntil which takes each P's timer lock. Simultaneously,
some P calls runtimer (holding the P's own timer lock) which wakes up
the scavenger, calling goready, calling wakep, calling startm, getting
the scheduler lock. Now the sysmon thread is holding the scheduler lock
and trying to get a P's timer lock, while some other thread running on
that P is holding the P's timer lock and trying to get the scheduler lock.
So change sysmon to call timeSleepUntil without holding the scheduler
lock, and change timeSleepUntil to use allpLock, which is only held for
limited periods of time and should never compete with timer locks.
This hopefully
Fixes#35375
At least it should fix the linux-arm64-packet builder problems,
which occurred more reliably as that system has GOMAXPROCS == 96,
giving a lot more scope for this deadlock.
Change-Id: I7a7917daf7a4882e0b27ca416e4f6300cfaaa774
Reviewed-on: https://go-review.googlesource.com/c/go/+/205558
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
WaitForSigusr1 registers a callback to be called on SIGUSR1 directly
from the runtime signal handler. Currently, this callback has a write
barrier in it, which can crash with a nil P if the GC is active and
the signal arrives on an M that doesn't have a P.
Fix this by recording the ID of the M that receives the signal instead
of the M itself, since that's all we needed anyway. To make sure there
are no other problems, this also lifts the callback into a package
function and marks it "go:nowritebarrierrec".
Fixes#35248.
Updates #35276, since in principle a write barrier at exactly the
wrong time while entering the scheduler could cause issues, though I
suspect that bug is unrelated.
Change-Id: I47b4bc73782efbb613785a93e381d8aaf6850826
Reviewed-on: https://go-review.googlesource.com/c/go/+/204620
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The check is not relevant anymore.
The comment claims that go run does not rebuild packages,
but this is not true. And we use go build anyway.
We may have added the check because without caching
rebuilding everything starting from runtime for each test
takes a while. But now we have caching.
So from every side this check just adds code and pain.
Change-Id: Ifbbb643724100622e5f9db884339b67cde4ba729
Reviewed-on: https://go-review.googlesource.com/c/go/+/202450
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The hash is used in type switches. However, compiler statically generates itab's
for all interface/type pairs used in switches (which are added to itabTable
in itabsinit). The dynamically-generated itab's never participate in type switches,
and thus the hash is irrelevant.
Change-Id: I4f6e37be31b8f5605cca7a1806cb04708e948cea
Reviewed-on: https://go-review.googlesource.com/c/go/+/202448
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
In the dev.link branch we implemented the new object file format
and (part of) the linker improvements described in
https://golang.org/s/better-linker
The new object file is index-based and provides random access.
The linker maps the object files into read-only memory, and
access symbols on-demand using indices, as opposed to reading
all object files sequentially into the heap with the old format.
The linker carries symbol informations using indices (as opposed
to Symbol data structure). Symbols are created after the
reachability analysis, and only created for reachable symbols.
This reduces the linker's memory usage.
Linking cmd/compile, it creates ~25% fewer Symbols, and reduces
memory usage (inuse_space) by ~15%. (More results from Than.)
Currently, both the old and new object file formats are supported.
The old format is used by default. The new format can be turned
on by using the compiler/assembler/linker's -newobj flag. Note
that the flag needs to be specified consistently to all
compilations, i.e.
go build -gcflags=all=-newobj -asmflags=all=-newobj -ldflags=-newobj
Change-Id: Ia0e35306b5b9b5b19fdc7fa7c602d4ce36fa6abd
When using cgo, we save G to TLS, and when a signal happens, we
load G from TLS in sigtramp. This should give us a valid G. Don't
try to fetch from the signal stack. In particular, C code may
change the signal stack or call our signal handler directly (e.g.
TSAN), so we are not necessarily running on the original gsignal
stack where we saved G.
Also skip saving G on the signal stack when using cgo.
Updates #35249.
Change-Id: I40749ce6682709bd4ebfdfd9f23bd0f317fc197d
Reviewed-on: https://go-review.googlesource.com/c/go/+/204519
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In the normal case, sigFetchG just returns the G register. But in
the case that sigFetchG fetches the G from somewhere else, the G
register still holding an invalid value. Setg here to make sure
they match.
This is particularly useful because setGsignalStack, called by
adjustSignalStack from sigtrampgo before setg to gsignal,
accesses the G register.
Should fix#35249.
Change-Id: I64c85143cb05cdb2ecca7f9936dbd8bfec186c2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/204441
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Before this CL adjustTimers left timers being moved in an inconsistent
state: status timerWaiting but not on a P. Simplify the code by
leaving the timers in timerMoving status until they are actually moved.
Other functions (deltimer, modtimer) will wait until the move is complete
before changing anything on the timer. This does leave timers in timerMoving
state for longer, but still not all that long.
Fixes#35367
Change-Id: I31851002fb4053bd6914139125b4c82a68bf6fb2
Reviewed-on: https://go-review.googlesource.com/c/go/+/205418
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Once defined, a stack slot holding an open-coded defer arg should always be marked
live, since it may be used at any time if there is a panic. These stack slots are
typically kept live naturally by the open-defer code inlined at each return/exit point.
However, we need to do extra work to make sure that they are kept live if a
function has an infinite loop or a panic exit.
For this fix, only in the case of a function that is using open-coded defers, we
compute the set of blocks (most often empty) that cannot reach a return or a
BlockExit (panic) because of an infinite loop. Then, for each block b which
cannot reach a return or BlockExit or is a BlockExit block, we mark each defer arg
slot as live, as long as the definition of the defer arg slot dominates block b.
For this change, had to export (*Func).sdom (-> Sdom) and SparseTree.isAncestorEq
(-> IsAncestorEq)
Updates #35277
Change-Id: I7b53c9bd38ba384a3794386dd0eb94e4cbde4eb1
Reviewed-on: https://go-review.googlesource.com/c/go/+/204802
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Frameless function is an interesting case for call injection
espcially for LR architectures. Extend the test for this case.
Change-Id: I074090d09eeaf642e71e3f44fea216f66d39b817
Reviewed-on: https://go-review.googlesource.com/c/go/+/202339
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This CL adds support of call injection and async preemption on
ARM.
Injected call, like sigpanic, has special frame layout. Teach
traceback to handle it.
Change-Id: I887e90134fbf8a676b73c26321c50b3c4762dba4
Reviewed-on: https://go-review.googlesource.com/c/go/+/202338
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Programs should always check the error return of Close for a file opened
for writing. Update the example code in the comment to mention this.
Change-Id: I2ff6866ff1fe23b47c54268ac8e182210cc876c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/202137
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
As a side-effect ensure that netpollinited only reports true when
netpoll initialization is complete.
Fixes#35282
Updates #35353
Change-Id: I21f08a04fcf229e0de5e6b5ad89c990426ae9b89
Reviewed-on: https://go-review.googlesource.com/c/go/+/204937
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL extends cmd/compile's experimental libFuzzer support with
calls to __sanitizer_cov_trace_{,const_}cmp{1,2,4,8}. This allows much
more efficient fuzzing of comparisons.
Only supports amd64 and arm64 for now.
Updates #14565.
Change-Id: Ibf82a8d9658f2bc50d955bdb1ae26723a3f0584d
Reviewed-on: https://go-review.googlesource.com/c/go/+/203887
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This change renames the "round" function to the more appropriately named
"alignUp" which rounds an integer up to the next multiple of a power of
two.
This change also adds the alignDown function, which is almost like
alignUp but rounds down to the previous multiple of a power of two.
With these two functions, we also go and replace manual rounding code
with it where we can.
Change-Id: Ie1487366280484dcb2662972b01b4f7135f72fec
Reviewed-on: https://go-review.googlesource.com/c/go/+/190618
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This change makes it so that the GC pacer's trigger ratio can never fall
below 0.6. Upcoming changes to the allocator make it significantly more
scalable and thus much faster in certain cases, creating a large gap
between the performance of allocation and scanning. The consequence of
this is that the trigger ratio can drop very low (0.07 was observed) in
order to drop GC utilization. A low trigger ratio like this results in a
high amount of black allocations, which causes the live heap to appear
larger, and thus the heap, and RSS, grows to a much higher stable point.
This change alleviates the problem by placing a lower bound on the
trigger ratio. The expected (and confirmed) effect of this is that
utilization in certain scenarios will no longer converge to the expected
25%, and may go higher. As a result of this artificially high trigger
ratio, more time will also be spent doing GC assists compared to
dedicated mark workers, since the GC will be on for an artifically short
fraction of time (artificial with respect to the pacer). The biggest
concern of this change is that allocation latency will suffer as a
result, since there will now be more assists. But, upcoming changes to
the allocator reduce the latency enough to outweigh the expected
increase in latency from this change, without the blowup in RSS observed
from the changes to the allocator.
Updates #35112.
Change-Id: Idd7c94fa974d0de673304c4397e716e89bfbf09b
Reviewed-on: https://go-review.googlesource.com/c/go/+/200439
Reviewed-by: Austin Clements <austin@google.com>
When dropping a P, if it has any timers, and if some thread is
sleeping in the netpoller, wake the netpoller to run the P's timers.
This mitigates races between the netpoller deciding how long to sleep
and a new timer being added.
In sysmon, if all P's are idle, check the timers to decide how long to sleep.
This avoids oversleeping if no thread is using the netpoller.
This can happen in particular if some threads use runtime.LockOSThread,
as those threads do not block in the netpoller.
Also, print the number of timers per P for GODEBUG=scheddetail=1.
Before this CL, TestLockedDeadlock2 would fail about 1% of the time.
With this CL, I ran it 150,000 times with no failures.
Updates #6239
Updates #27707Fixes#35274Fixes#35288
Change-Id: I7e5193e6c885e567f0b1ee023664aa3e2902fcd1
Reviewed-on: https://go-review.googlesource.com/c/go/+/204800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
CL 200958 adds skipping empty init function feature without any tests
for it. A codegen test sounds ideal, but it's unlikely that we can make
one for now, so use a program to manipulate runtime/proc.go:initTask
directly.
Updates #34869
Change-Id: I2683b9a1ace36af6861af02a3a9fb18b3110b282
Reviewed-on: https://go-review.googlesource.com/c/go/+/204217
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Speed up nanotime1 and walltime1 on MIPS64 with vDSO, just like the
other vDSO-enabled targets.
Benchmark numbers on Loongson 3A3000 (GOARCH=mips64le, 1.4GHz) against
current master:
benchmark old ns/op new ns/op delta
BenchmarkNow 868 293 -66.24%
BenchmarkNowUnixNano 851 296 -65.22%
Performance hit on fallback case, tested by using a wrong vDSO symbol name:
benchmark old ns/op new ns/op delta
BenchmarkNow 868 889 +2.42%
BenchmarkNowUnixNano 851 893 +4.94%
Change-Id: Ibfb48893cd060536359863ffee7624c00def646b
GitHub-Last-Rev: 03a58ac2e4
GitHub-Pull-Request: golang/go#35181
Reviewed-on: https://go-review.googlesource.com/c/go/+/203578
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When we do a successful recover of a panic, we resume normal execution by
returning from the frame that had the deferred call that did the recover (after
executing any remaining deferred calls in that frame).
However, suppose we have called runtime.Goexit and there is a panic during one of the
deferred calls run by the Goexit. Further assume that there is a deferred call in
the frame of the Goexit or a parent frame that does a recover. Then the recovery
process will actually resume normal execution above the Goexit frame and hence
abort the Goexit. We will not terminate the thread as expected, but continue
running in the frame above the Goexit.
To fix this, we explicitly create a _panic object for a Goexit call. We then
change the "abort" behavior for Goexits, but not panics. After a recovery, if the
top-level panic is actually a Goexit that is marked to be aborted, then we return
to the Goexit defer-processing loop, so that the Goexit is not actually aborted.
Actual code changes are just panic.go, runtime2.go, and funcid.go. Adjusted the
test related to the new Goexit behavior (TestRecoverBeforePanicAfterGoexit) and
added several new tests of aborted panics (whose behavior has not changed).
Fixes#29226
Change-Id: Ib13cb0074f5acc2567a28db7ca6912cfc47eecb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/200081
Run-TryBot: Dan Scales <danscales@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
We don't async preempt assembly functions. We do that by checking
whether the function has a local pointer map, and assume it is
an assembly (or, non-Go) function if there isn't one. However,
assembly functions marked with NO_LOCAL_POINTERS still have local
pointer maps, and we wouldn't identify them. For them, check for
the special pointer map runtime.no_pointers_stackmap as well, and
treat them as not async preemptible.
Change-Id: I1301e3b4d35893c31c4c5a5147a0d775987bd6f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/202337
Reviewed-by: Austin Clements <austin@google.com>
This adds a test of preempting a loop containing no synchronous safe
points for STW and stack scanning.
We couldn't add this test earlier because it requires scheduler, STW,
and stack scanning preemption to all be working.
For #10958, #24543.
Change-Id: I73292db78ca3d14aab11bdafd26d03986920ef0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/201777
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This adds signal-based preemption to preemptone.
Since STW and forEachP ultimately use preemptone, this also makes
these work with async preemption.
This also makes freezetheworld more robust so tracebacks from fatal
panics should be far less likely to report "goroutine running on other
thread; stack unavailable".
For #10958, #24543. (This doesn't fix it yet because asynchronous
preemption only works on POSIX platforms on 386 and amd64 right now.)
Change-Id: If776181dd5a9b3026a7b89a1b5266521b95a5f61
Reviewed-on: https://go-review.googlesource.com/c/go/+/201762
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This adds support for pausing a running G by sending a signal to its
M.
The main complication is that we want to target a G, but can only send
a signal to an M. Hence, the protocol we use is to simply mark the G
for preemption (which we already do) and send the M a "wake up and
look around" signal. The signal checks if it's running a G with a
preemption request and stops it if so in the same way that stack check
preemptions stop Gs. Since the preemption may fail (the G could be
moved or the signal could arrive at an unsafe point), we keep a count
of the number of received preemption signals. This lets stopG detect
if its request failed and should be retried without an explicit
channel back to suspendG.
For #10958, #24543.
Change-Id: I3e1538d5ea5200aeb434374abb5d5fdc56107e53
Reviewed-on: https://go-review.googlesource.com/c/go/+/201760
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This adds support for scanning the stack when a goroutine is stopped
at an async safe point. This is not yet lit up because asyncPreempt is
not yet injected, but prepares us for that.
This works by conservatively scanning the registers dumped in the
frame of asyncPreempt and its parent frame, which was stopped at an
asynchronous safe point.
Conservative scanning works by only marking words that are pointers to
valid, allocated heap objects. One complication is pointers to stack
objects. In this case, we can't determine if the stack object is still
"allocated" or if it was freed by an earlier GC. Hence, we need to
propagate the conservative-ness of scanning stack objects: if all
pointers found to a stack object were found via conservative scanning,
then the stack object itself needs to be scanned conservatively, since
its pointers may point to dead objects.
For #10958, #24543.
Change-Id: I7ff84b058c37cde3de8a982da07002eaba126fd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/201761
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This adds asynchronous preemption function for amd64 and 386. These
functions spill and restore all register state that can be used by
user Go code.
For the moment we stub out the other arches.
For #10958, #24543.
Change-Id: I6f93fabe9875f4834922a5712362e79045c00aca
Reviewed-on: https://go-review.googlesource.com/c/go/+/201759
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This adds a sigctxt.pushCall method that pushes a call at the signaled
site. We'll use this to inject asynchronous preemptions and in some
places we use it to clean up preparePanic.
For the moment this only works on 386 and amd64. We stub it out on
other platforms and will avoid calling the stubbed version.
For #10958, #24543.
Change-Id: I49e0e853f935d32dd67a70c6cafbae44ee68af8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/201758
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This doesn't do anything yet, but it will provide a way to disable
non-cooperative preemption.
For #10958, #24543.
Change-Id: Ifdef303f103eabd0922ced8d9bebbd5f0aa2cda4
Reviewed-on: https://go-review.googlesource.com/c/go/+/201757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
On illumos systems, and at least historically on Solaris systems, it is
possible for port_getn(3C) calls to return some number of events and
then fail with error ETIME.
Generally we expect this to happen if the caller passes an nget value
larger than 1 and calls with a timeout; if less than the requested
number of events accumulate the system will still return them after
timeout failure so the caller must check the updated nget value in the
ETIME case. Note that although less likely this can still happen even
when requesting just 1 event, especially with a short timeout value or
on a busy system.
Fixes#35261
Change-Id: I0d83251b69a2fadc64c4e8e280aa596e2e1548ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/204801
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If multiple goroutines call time.(*Timer).Reset then the timer will go
from timerWaiting to timerDeleted to timerModifying to timerModifiedLater.
The timer can be on a different P, meaning that simultaneously cleantimers
could change it from timerDeleted to timerRemoving to timerRemoved.
If Reset sees timerRemoved, it was doing an atomic.Store of timerWaiting,
meaning that it did not necessarily see the other values set in the timer,
so the timer could appear to be in an inconsistent state. Use atomic.Cas
to avoid that possibility.
Updates #6239
Updates #27707Fixes#35272
Change-Id: I1d59a13dc4f2ff4af110fc6e032c8c9d59cfc270
Reviewed-on: https://go-review.googlesource.com/c/go/+/204717
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
I was doing some testing with GODEBUG=schedtrace=1,scheddetail=1 and I
noticed that the program hung after a throw with "all goroutines are
asleep". This is because when doing a throw or fatal panic with schedtrace
the panic code does a final schedtrace, which needs to acquire the
scheduler lock. The checkdead function is always called with the scheduler
lock held. So checkdead would throw with the scheduler lock held, then
the panic code would call schedtrace, which would block trying to acquire
the scheduler lock.
This problem will only happen for people debugging the runtime, but
it's easy to avoid by having checkdead unlock the scheduler lock before
it throws. I only did this for the throws that can happen for a normal
program, not for throws that indicate some corruption in the scheduler data.
Change-Id: Ic62277b3ca6bee6f0fca8d5eb516c59cb67855cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/204778
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change turns off the scavenger if there's less than one physical
page of work to do. If there's less than one phyiscal page of work
today, then the computed time for the work to be done will be zero,
resulting in a floating point division by zero.
This is bad on two accounts. On the one hand it could cause a fault on
some systems. On the other hand, it could cause the pacing computations
done by the scavenger to be nonsense. While this is generally harmless
in the case where there's a very small amount of work to do anyway (the
scavenger might just back off expontentially forever, or do some work
and immediately sleep, because there's not much of it to do), it causes
problems for the deadlock checker. On platforms with a larger physical
page size, such as 64 KiB, we might hit this path in a deadlock
scenario, in which case the deadlock checker will never fire and we'll
just hang.
Specifically, this happens on ppc64 trybot tests, which is where the
issue was discovered.
Fixes#34575.
Change-Id: I8677db539447b2f0e75b8cfcbe33932244e1508c
Reviewed-on: https://go-review.googlesource.com/c/go/+/203517
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
When everything is working correctly, any pointer the garbage
collector encounters can only point into a fully initialized heap
span, since the span must have been initialized before that pointer
could escape the heap allocator and become visible to the GC.
However, in various cases, we try to be defensive against bad
pointers. In findObject, this is just a sanity check: we never expect
to find a bad pointer, but programming errors can lead to them. In
spanOfHeap, we don't necessarily trust the pointer and we're trying to
check if it really does point to the heap, though it should always
point to something. Conservative scanning takes this to a new level,
since it can only guess that a word may be a pointer and verify this.
In all of these cases, we have a problem that the span lookup and
check can race with span initialization, since the span becomes
visible to lookups before it's fully initialized.
Furthermore, we're about to start initializing the span without the
heap lock held, which is going to introduce races where accesses were
previously protected by the heap lock.
To address this, this CL makes accesses to mspan.state atomic, and
ensures that the span is fully initialized before setting the state to
mSpanInUse. All loads are now atomic, and in any case where we don't
trust the pointer, it first atomically loads the span state and checks
that it's mSpanInUse, after which it will have synchronized with span
initialization and can safely check the other span fields.
For #10958, #24543, but a good fix in general.
Change-Id: I518b7c63555b02064b98aa5f802c92b758fef853
Reviewed-on: https://go-review.googlesource.com/c/go/+/203286
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently, several important fields of a heap span are set by
heapBits.initSpan, which happens after the span has already been
published and returned from the locked region of alloc_m. In
particular, allocBits is set very late, which makes mspan.isFree
unsafe even if you were to lock the heap because it tries to access
allocBits.
This CL fixes this by populating these fields in alloc_m. The next CL
builds on this to only publish the span once it is fully initialized.
Together, they'll make it safe to check allocBits even if there is a
race with alloc_m.
For #10958, #24543, but a good fix in general.
Change-Id: I7fde90023af0f497e826b637efa4d19c32840c08
Reviewed-on: https://go-review.googlesource.com/c/go/+/203285
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Add some simple unit tests for these atomic operations. These can't
catch all the bugs that are possible with these operations but at
least they provide some coverage.
Change-Id: I94b9f451fcc9fecdb2a1448c5357b019563ad275
Reviewed-on: https://go-review.googlesource.com/c/go/+/204317
Run-TryBot: Michael Munday <mike.munday@ibm.com>
Reviewed-by: Austin Clements <austin@google.com>
DWARF constant DIE symbols are not referenced by any other symbol,
but are needed by the DWARF pass, where they get linked to the
compilation unit.
Reenable gdb constant test.
Change-Id: If77a0d379d9a6f1591939345bc31b027c2567f22
Reviewed-on: https://go-review.googlesource.com/c/go/+/204397
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Fcntl can't be called using syscall.Syscall as it doesn't work on AIX.
Moreover, fcntl isn't exported by syscall package.
However, it can be accessed by exporting it from runtime package
using export_aix_test.go.
Change-Id: Ib6af66d9d7eacb9ca0525ebc4cd4c92951735f1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/204059
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I2629711ce02d935130fb2aab29f9028b62ba9fe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/204318
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This fixes the Plan 9 support for the new timer code.
Updates #6239
Updates #27707
Change-Id: Ia498c399b8924910b97fcde07545fae3588aad47
Reviewed-on: https://go-review.googlesource.com/c/go/+/204045
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The C fnctl takes all int parameters, so consistently use int32.
We already used int32 on Darwin.
Change-Id: I69a012145d012771d7308d705d133159fc1aceaa
Reviewed-on: https://go-review.googlesource.com/c/go/+/204101
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
On some platforms (currently ARM and ARM64), when calling into
VDSO we store the G to the gsignal stack, if there is one, so if
we receive a signal during VDSO we can find the G.
When an M exits, it frees the gsignal stack. But m.gsignal.stack
still points to that stack. When we call nanotime on this M, we
will write to the already freed gsignal stack, which is bad.
Prevent this by unlinking the freed stack from the M.
Should fix#35235.
Change-Id: I338b1fc8ec62aae036f38afaca3484687e11a40d
Reviewed-on: https://go-review.googlesource.com/c/go/+/204158
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In my experimentation, I had found that most non-SSAable expressions were
converted to autotmp variables during AST evaluation. However, this was not true
generally, as witnessed by issue #35213, which has a non-SSAable field reference
of a struct that is not converted to an autotmp. So, I fixed openDeferSave() to
handle non-SSAable nodes more generally, and make sure that these non-SSAable
expressions are not evaluated more than once (which could incorrectly repeat side
effects).
Fixes#35213
Change-Id: I8043d5576b455e94163599e930ca0275e550d594
Reviewed-on: https://go-review.googlesource.com/c/go/+/203888
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Netpoll must be always be initialized when TestNetpollBreak is launched.
However, when it is run in standalone, it won't be the case, so it must
be forced.
Updates: #27707
Change-Id: I28147f3834f3d6aca982c6a298feadc09b55f66e
Reviewed-on: https://go-review.googlesource.com/c/go/+/204058
Run-TryBot: Clément Chigot <clement.chigot%atos.net@gtempaccount.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TestPreemptM doesn't test preemptM, it tests signalM. Rename it and
co-locate it with the other tests related to signals.
Change-Id: I7b95f2ba96530c49cfa8d5bf33282946b5f2d9af
Reviewed-on: https://go-review.googlesource.com/c/go/+/203891
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TestPreemptM tests signal delivery using SIGUSR1, but (for unknown
reasons) SIGUSR1 is blocked by default on android/arm and
android/arm64, causing the test to fail.
This fixes the test by ensuring that SIGUSR1 is unblocked for this
test.
Updates #10958, #24543.
Change-Id: I9f81fbab53f96c74622aabcb6f5276f79e2b6d33
Reviewed-on: https://go-review.googlesource.com/c/go/+/203957
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We already have Load8, And8, and Or8.
For #10958, #24543, but makes sense on its own.
Change-Id: I478529fc643edc57efdeccaae413c99edd19b2eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/203283
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
On illumos systems, check for the "zone.cpu-cap" resource control when
determining how many usable CPUs are available. If the resource control
is not set, or we are unable to read it, ignore the failure and return
the value we used to return; i.e., the CPU count from
sysconf(_SC_NPROCESSORS_ONLN).
Fixesgolang/go#35199
Change-Id: Ic8a408f84cd140d544d128f1281baad527fb5e35
Reviewed-on: https://go-review.googlesource.com/c/go/+/203758
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If a compilation has multiple text sections, code in
textOff must compare the offset argument against the range
for each text section to determine which one it is in.
The comparison looks like this:
if uintptr(off) >= sectaddr && uintptr(off) <= sectaddr+sectlen
If the off value being compared is equal to sectaddr+sectlen then it
is not within the range of the text section but after it. The
comparison should be just '<'.
Updates #35207
Change-Id: I114633fd734563d38f4e842dd884c6c239f73c95
Reviewed-on: https://go-review.googlesource.com/c/go/+/203817
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Based on work by Mikaël Urankar (@MikaelUrankar).
Updates #24715
Updates #35197
Change-Id: I91144101043d67d3f8444bf8389c9606abe2a66c
Reviewed-on: https://go-review.googlesource.com/c/go/+/199919
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
_defer.fn can be nil, so we need to add a check when dumping
_defer.fn.fn.
Fixes#35172
Change-Id: Ic1138be5ec9dce915a87467cfa51ff83acc6e3a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/203697
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
ARM's R4-R8 & R10-R11 are callee-save registers, and R9
may be callee-save or not. This CL saves them at the beginning
of sigtramp and restores them in the end.
fixes#32738
Change-Id: Ib7eb80836bc074e2e6a46ae4602ba8a3b96c5456
Reviewed-on: https://go-review.googlesource.com/c/go/+/183777
Reviewed-by: Cherry Zhang <cherryyz@google.com>
For these, we split up the existing runtime.raise assembly
implementation into its constituent "get thread ID" and "signal
thread" parts. This lets us implement signalM and reimplement raise in
pure Go. (NetBSD conveniently already had lwp_self.)
We also change minit to store the procid directly, rather than
depending on newosproc to do so. This is because newosproc isn't
called for the bootstrap M, but we need a procid for every M. This is
also simpler overall.
For #10958, #24543.
Change-Id: Ie5f1fcada6a33046375066bcbe054d1f784d39c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/201402
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
We'll add a test once all of the POSIX platforms are done.
For #10958, #24543.
Change-Id: If7e3f14e8391791364877629bf415d9f8e788b0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/201401
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
findObject takes the pointer argument as uintptr. If the pointer is to
the local stack and calling findObject happens to require the stack to
be reallocated, then spanOf is called for the old pointer.
Marking findObject as nosplit fixes the issue.
Fixes#35068
Change-Id: I029d36f9c23f91812f18f98839edf02e0ba4082e
Reviewed-on: https://go-review.googlesource.com/c/go/+/202798
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Factor out case s == nil, make the code cleaner and easier to read.
Change-Id: I63f52e14351c0a0d20a611b1fe10fdc0d4947d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/202498
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
We check whether an M is preemptible in a surprising number of places.
Put it in one function.
For #10958, #24543.
Change-Id: I305090fdb1ea7f7a55ffe25851c1e35012d0d06c
Reviewed-on: https://go-review.googlesource.com/c/go/+/201439
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
We're about to introduce asynchronous safe points, where we won't have
precise pointer maps for all stack frames. That's okay for scanning
the stack (conservatively), but not for shrinking the stack.
Hence, this CL prepares for this by only shrinking the stack as part
of the stack scan if the goroutine is stopped at a synchronous safe
point. Otherwise, it queues up the stack shrink for the next
synchronous safe point.
We already have one condition under which we can't shrink the stack
for very similar reasons: syscalls. Currently, we just give up on
shrinking the stack if it's in a syscall. But with this mechanism, we
defer that stack shrink until the next synchronous safe point.
For #10958, #24543.
Change-Id: Ifa1dec6f33fdf30f9067be2ce3f7ab8a7f62ce38
Reviewed-on: https://go-review.googlesource.com/c/go/+/201438
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When we copy a stack of a goroutine blocked in a channel operation, we
have to be very careful because other goroutines may be writing to
that goroutine's stack. To handle this, stack copying acquires the
locks for the channels a goroutine is waiting on.
One complication is that stack growth may happen while a goroutine
holds these locks, in which case stack copying must *not* acquire
these locks because that would self-deadlock.
Currently, stack growth never acquires these locks because stack
growth only happens when a goroutine is running, which means it's
either not blocking on a channel or it's holding the channel locks
already. Stack shrinking always acquires these locks because shrinking
happens asynchronously, so the goroutine is never running, so there
are either no locks or they've been released by the goroutine.
However, we're about to change when stack shrinking can happen, which
is going to break the current rules. Rather than find a new way to
derive whether to acquire these locks or not, this CL simply adds a
flag to the g struct that indicates that stack copying should acquire
channel locks. This flag is set while the goroutine is blocked on a
channel op.
For #10958, #24543.
Change-Id: Ia2ac8831b1bfda98d39bb30285e144c4f7eaf9ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/172982
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently, gcscanvalid is used to resolve a race between attempts to
scan a stack. Now that there's a clear owner of the stack scan
operation, there's no longer any danger of racing or attempting to
scan a stack more than once, so this CL eliminates gcscanvalid.
I double-checked my reasoning by first adding a throw if gcscanvalid
was set in scanstack and verifying that all.bash still passed.
For #10958, #24543.
Fixes#24363.
Change-Id: I76794a5fcda325ed7cfc2b545e2a839b8b3bc713
Reviewed-on: https://go-review.googlesource.com/c/go/+/201139
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Currently, the process of suspending a goroutine is tied to stack
scanning. In preparation for non-cooperative preemption, this CL
abstracts this into general purpose suspendG/resumeG functions.
suspendG and resumeG closely follow the existing scang and restartg
functions with one exception: the addition of a _Gpreempted status.
Currently, preemption tasks (stack scanning) are carried out by the
target goroutine if it's in _Grunning. In this new approach, the task
is always carried out by the goroutine that called suspendG. Thus, we
need a reliable way to drive the target goroutine out of _Grunning
until the requesting goroutine is ready to resume it. The new
_Gpreempted state provides the handshake: when a runnable goroutine
responds to a preemption request, it now parks itself and enters
_Gpreempted. The requesting goroutine races to put it in _Gwaiting,
which gives it ownership, but also the responsibility to start it
again.
This CL adds several TODOs about improving the synchronization on the
G status. The existing code already has these problems; we're just
taking note of them.
The next CL will remove the now-dead scang and preemptscan.
For #10958, #24543.
Change-Id: I16dbf87bea9d50399cc86719c156f48e67198f16
Reviewed-on: https://go-review.googlesource.com/c/go/+/201137
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
CL 198544 broke the linux/arm64 build because it declares emptyfunc for
GOARCH=arm64, but only freebsd/arm64 defines it. Make it a static
assembly function specific for freebsd/arm64 and remove the stub.
Fixes#35160
Change-Id: I5fd94249b60c6fd259c251407b6eccc8fa512934
Reviewed-on: https://go-review.googlesource.com/c/go/+/203418
Reviewed-by: Bryan C. Mills <bcmills@google.com>
We already claim on the documentation for _Grunning that this is case,
but execute transitions to _Grunning before assigning g.m. Fix this
and make the documentation even more explicit.
For #10958, #24543, but also a good cleanup.
Change-Id: I1eb0108e7762f55cfb0282aca624af1c0a15fe56
Reviewed-on: https://go-review.googlesource.com/c/go/+/201440
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Currently, shrinkstack will not shrink a stack on Windows if
gp.m.libcallsp != 0. In general, we can't shrink stacks in syscalls
because the syscall may hold pointers into the stack, and in principle
this is supposed to be preventing that for libcall-based syscalls
(which are direct syscalls from the runtime). But this test is
actually broken and has been for a long time. That turns out to be
okay because it also appears it's not necessary.
This test is racy. g.m points to whatever M the G was last running on,
even if the G is in a blocked state, and that M could be doing
anything, including making libcalls. Hence, observing that libcallsp
== 0 at one moment in shrinkstack is no guarantee that it won't become
non-zero while we're shrinking the stack, and vice-versa.
It's also weird that this check is only performed on Windows, given
that we now use libcalls on macOS, Solaris, and AIX.
This check was added when stack shrinking was first implemented in CL
69580044. The history of that CL (though not the final version)
suggests this was necessary for libcalls that happened on Go user
stacks, which we never do now because of the limited stack space.
It could also be defending against user stack pointers passed to
libcall system calls from blocked Gs. But the runtime isn't allowed to
keep pointers into the user stack for blocked Gs on any OS, so it's
not clear this would be of any value.
Hence, this checks seems to be simply unnecessary.
Rather than simply remove it, this CL makes it defensive. We can't do
anything about blocked Gs, since it doesn't even make sense to look at
their M, but if a G tries to shrink its own stack while in a libcall,
that indicates a bug in the libcall code. This CL makes shrinkstack
panic in this case.
For #10958, #24543, since those are going to rearrange how we decide
that it's safe to shrink a stack.
Change-Id: Ia865e1f6340cff26637f8d513970f9ebb4735c6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/173724
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Generate inline code at defer time to save the args of defer calls to unique
(autotmp) stack slots, and generate inline code at exit time to check which defer
calls were made and make the associated function/method/interface calls. We
remember that a particular defer statement was reached by storing in the deferBits
variable (always stored on the stack). At exit time, we check the bits of the
deferBits variable to determine which defer function calls to make (in reverse
order). These low-cost defers are only used for functions where no defers
appear in loops. In addition, we don't do these low-cost defers if there are too
many defer statements or too many exits in a function (to limit code increase).
When a function uses open-coded defers, we produce extra
FUNCDATA_OpenCodedDeferInfo information that specifies the number of defers, and
for each defer, the stack slots where the closure and associated args have been
stored. The funcdata also includes the location of the deferBits variable.
Therefore, for panics, we can use this funcdata to determine exactly which defers
are active, and call the appropriate functions/methods/closures with the correct
arguments for each active defer.
In order to unwind the stack correctly after a recover(), we need to add an extra
code segment to functions with open-coded defers that simply calls deferreturn()
and returns. This segment is not reachable by the normal function, but is returned
to by the runtime during recovery. We set the liveness information of this
deferreturn() to be the same as the liveness at the first function call during the
last defer exit code (so all return values and all stack slots needed by the defer
calls will be live).
I needed to increase the stackguard constant from 880 to 896, because of a small
amount of new code in deferreturn().
The -N flag disables open-coded defers. '-d defer' prints out the kind of defer
being used at each defer statement (heap-allocated, stack-allocated, or
open-coded).
Cost of defer statement [ go test -run NONE -bench BenchmarkDefer$ runtime ]
With normal (stack-allocated) defers only: 35.4 ns/op
With open-coded defers: 5.6 ns/op
Cost of function call alone (remove defer keyword): 4.4 ns/op
Text size increase (including funcdata) for go binary without/with open-coded defers: 0.09%
The average size increase (including funcdata) for only the functions that use
open-coded defers is 1.1%.
The cost of a panic followed by a recover got noticeably slower, since panic
processing now requires a scan of the stack for open-coded defer frames. This scan
is required, even if no frames are using open-coded defers:
Cost of panic and recover [ go test -run NONE -bench BenchmarkPanicRecover runtime ]
Without open-coded defers: 62.0 ns/op
With open-coded defers: 255 ns/op
A CGO Go-to-C-to-Go benchmark got noticeably faster because of open-coded defers:
CGO Go-to-C-to-Go benchmark [cd misc/cgo/test; go test -run NONE -bench BenchmarkCGoCallback ]
Without open-coded defers: 443 ns/op
With open-coded defers: 347 ns/op
Updates #14939 (defer performance)
Updates #34481 (design doc)
Change-Id: I63b1a60d1ebf28126f55ee9fd7ecffe9cb23d1ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/202340
Reviewed-by: Austin Clements <austin@google.com>
On ARM and ARM64, during a VDSO call, the g register may be
temporarily clobbered by the VDSO code. If a signal is received
during the execution of VDSO code, we may not find a valid g
reading the g register. In CL 192937, we conservatively assume
g is nil. But this approach has a problem: we cannot handle
the signal in this case. Further, if the signal is not a
profiling signal, we'll call badsignal, which calls needm, which
wants to get an extra m, but we don't have one in a non-cgo
binary, which cuases the program to hang.
This is even more of a problem with async preemption, where we
will receive more signals than before. I ran into this problem
while working on async preemption support on ARM64.
In this CL, before making a VDSO call, we save the g on the
gsignal stack. When we receive a signal, we will be running on
the gsignal stack, so we can fetch the g from there and move on.
We probably want to do the same for PPC64. Currently we rely on
that the VDSO code doesn't actually clobber the g register, but
this is not guaranteed and we don't have control with.
Idea from discussion with Dan Cross and Austin.
Should fix#34391.
Change-Id: Idbefc5e4c2f4373192c2be797be0140ae08b26e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/202759
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Report the value returned by kevent, not the previously set errno which
is 0.
Found while debugging CL 198544
Change-Id: I854f5418f8ed8e083d909d328501355496c67a53
Reviewed-on: https://go-review.googlesource.com/c/go/+/202777
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Since the new timers run on g0, which does not have a race context,
we add a race context field to the P, and use that for timer functions.
This works since all timer functions are in the standard library.
Updates #27707
Change-Id: I8a5b727b4ddc8ca6fc60eb6d6f5e9819245e395b
Reviewed-on: https://go-review.googlesource.com/c/go/+/171882
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Since timers are now on a P, rather than having a G running timerproc,
timejump changes to return a P rather than a G.
Updates #27707
Change-Id: I3d05af2d664409a0fd906e709fdecbbcbe00b9a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/171880
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This CL extends checkptrBase to recognize pointers into the stack and
data/bss sections. I was meaning to do this eventually anyway, but
it's also an easy way to workaround #35068.
Updates #35068.
Change-Id: Ib47f0aa800473a4fbc249da52ff03bec32c3ebe2
Reviewed-on: https://go-review.googlesource.com/c/go/+/202639
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The adjusttimers function is where we check the adjustTimers field in
the P struct to see if we need to resort the heap. We walk forward in
the heap and find and resort timers that have been modified, until we
find all the timers that were modified to run earlier. Along the way
we remove deleted timers.
Updates #27707
Change-Id: I1cba7fe77b8112b7e9a9dba80b5dfb08fcc7c568
Reviewed-on: https://go-review.googlesource.com/c/go/+/171877
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Also add a skeleton of the runOneTimer function.
Updates #27707
Change-Id: Ic6a0279354a57295f823093704b7e152ce5d769d
Reviewed-on: https://go-review.googlesource.com/c/go/+/171835
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
They're still lacking in details, but at least better than being
printed as raw interface values.
Updates #22218.
Change-Id: I4fd813253afdd6455c0c9b5a05c61659805abad1
Reviewed-on: https://go-review.googlesource.com/c/go/+/202677
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The dodeltimer function removes a timer from a heap. The dodeltimer0
function removes the first timer from a heap; in the old timer code
this common special case was inlined in the timerproc function.
Updates #27707
Change-Id: I1b7c0af46866abb4bffa8aa4d8e7143f9ae8f402
Reviewed-on: https://go-review.googlesource.com/c/go/+/171834
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This adds a new field to P, adjustTimers, that tells the P that one of
its existing timers was modified to be earlier, and that it therefore
needs to resort them.
Updates #27707
Change-Id: I4c5f5b51ed116f1d898d3f87cdddfa1b552337f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/171832
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
When we add a timer, make sure that the network poller is initialized,
since we will use it if we have to wait for the timer to be ready.
Updates #27707
Change-Id: I0637fe646bade2cc5ce50b745712292aa9c445b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/171830
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This reduces the required nosplit stack size, which permits building
on Solaris with -gcflags=all=-N -l.
Fixes#35046
Change-Id: Icb3a421bb791c73e2f670ecfadbe32daea79789f
Reviewed-on: https://go-review.googlesource.com/c/go/+/202446
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Fixes#35037
Change-Id: I0b9bcd001556cd409994d83dabcdd6e32b001d28
Reviewed-on: https://go-review.googlesource.com/c/go/+/202441
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Fixes#35053
Change-Id: I31853d434610880044c169e0c1e9732f97ff1bdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/202444
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
Caught by go test -a -short -gcflags=all=-d=checkptr runtime
TestMincoreErrorSign intentionally uses uintptr(1) to get -EINVAL,
but it violates unsafe pointer rules 2. So use another misaligned
pointer add(new(int32), 1), but do not violate unsafe pointer rules.
TestEpollctlErrorSign passes an unsafe.Pointer of &struct{}{} to
Epollctl, which is then casted to epollevent, causes mis-alignment.
Fixing it by exporting epollevent on runtime_test package, so it can be
passed to Epollctl.
Updates #34972
Change-Id: I78ebfbeaf706fd1d372272af0bbc4e2cabca4631
Reviewed-on: https://go-review.googlesource.com/c/go/+/202157
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add new fields to runtime.timer, and adjust the various timer
functions in preparation for adding timers to P's. This continues to
use the old timer code.
Updates #6239
Updates #27707
Change-Id: I9adb3814f657e083ec5e22736c4b5b52b77b6a3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/171829
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
A common idiom for turning an unsafe.Pointer into a slice is to write:
s := (*[Big]T)(ptr)[:n:m]
This technically violates Go's unsafe pointer rules (rule #1 says T2
can't be bigger than T1), but it's fairly common and not too difficult
to recognize, so might as well allow it for now so we can make
progress on #34972.
This should be revisited if #19367 is accepted.
Updates #22218.
Updates #34972.
Change-Id: Id824e2461904e770910b6e728b4234041d2cc8bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/201839
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Since the previous CL, we will not reference static symbols by
name. Therefore no need to put them into the name lookup table.
On Linux/ARM, in runtime/internal/atomic/sys_linux_arm.s, the
kernelcas function has a definition and a reference written in
two different forms, one with package prefix, one without. This
way, the assembler cannot know they are the same symbol, only the
linker knows. This is quite unusual, unify the names to so the
assembler can resolve it to index.
Change-Id: Ie7223097be6a3b65f3fa43ed4575da9972ef5b69
Reviewed-on: https://go-review.googlesource.com/c/go/+/201998
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
When we put timers on P's, the wasm code will not be able to rely on
the timer goroutine. Use the beforeIdle hook to schedule a wakeup.
Updates #6239
Updates #27707
Change-Id: Idf6309944778b8c3d7178f5d09431940843ea233
Reviewed-on: https://go-review.googlesource.com/c/go/+/171827
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This change introduces an arm intrinsic that generates the FMULAD
instruction for the fused-multiply-add operation on systems that
support it. System support is detected via cpu.ARM.HasVFPv4. A rewrite
rule translates the generic intrinsic to FMULAD.
Updates #25819.
Change-Id: I8459e5dd1cdbdca35f88a78dbeb7d387f1e20efa
Reviewed-on: https://go-review.googlesource.com/c/go/+/142117
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Add support to the main scheduler loop for handling timers on P's.
This is not used yet, as timers are not yet put on P's.
Updates #6239
Updates #27707
Change-Id: I6a359df408629f333a9232142ce19e8be8496dae
Reviewed-on: https://go-review.googlesource.com/c/go/+/171826
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
As a small step toward speeding up timers, restrict modification
of the timer.when field to the timer code itself. Other code that
wants to change the when field of an existing timer must now call
resettimer rather than changing the when field and calling addtimer.
The new resettimer function also works for a new timer.
This is just a refactoring in preparation for later code.
Updates #27707
Change-Id: Iccd5dcad415ffbeac4c2a3cf015e91f82692acf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/171825
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
To permit ssa-level optimization, this change introduces an amd64 intrinsic
that generates the VFMADD231SD instruction for the fused-multiply-add
operation on systems that support it. System support is detected via
cpu.X86.HasFMA. A rewrite rule can then translate the generic ssa intrinsic
("Fma") to VFMADD231SD.
The benchmark compares the software implementation (old) with the intrinsic
(new).
name old time/op new time/op delta
Fma-4 27.2ns ± 1% 1.0ns ± 9% -96.48% (p=0.008 n=5+5)
Updates #25819.
Change-Id: I966655e5f96817a5d06dff5942418a3915b09584
Reviewed-on: https://go-review.googlesource.com/c/go/+/137156
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The new netpollBreak function can be used to interrupt a blocking netpoll.
This function is not currently used; it will be used by later CLs.
Updates #27707
Change-Id: I5cb936609ba13c3c127ea1368a49194fc58c9f4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/171824
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The internal read and write functions used to return -1 on error;
change them to return a negative errno value instead.
This will be used by later CLs in this series.
For most targets this is a simplification, although for ones that call
into libc it is a complication.
Updates #27707
Change-Id: Id02bf9487f03e7e88e4f2b85e899e986738697ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/171823
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
On MIPS, pipe returns two values rather than taking a pointer.
On MIPS64, call pipe2 rather than pipe.
Also, use the correct system call number for fcntl on mips64.
Change-Id: Ie72acdefeb593f44cb98735fc40eac99cf73509e
Reviewed-on: https://go-review.googlesource.com/c/go/+/202417
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This requires defining pipe, pipe2, and setNonblock for various platforms.
The new function is currently only used on AIX. It will be used by
later CLs in this series.
Updates #27707
Change-Id: Id2f987b66b4c66a3ef40c22484ff1d14f58e9b31
Reviewed-on: https://go-review.googlesource.com/c/go/+/171822
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
On Wasm, at program start, we set the SP to
wasmStack+sizeof(wasmStack), and start to write on it. This write
is actually past the end of wasmStack. This may scribble to some
other variable next to it in the data segment. Or if wasmStack
happens to be the last object in the data segment, we'll scribble
to unreserved memory and cause the next sysReserve return
non-zero memory. Either way, this is bad. Fix this by subtracting
16 before writing.
Found while debugging the new page allocator (CL 190622 and the
stack) with Michael. We found that on Wasm, the first sysReserve
may return memory with the first a few words being non-zero.
Change-Id: I2d76dd3fee85bddb2ff6a902b5876dea3f2969a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/202086
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Add test to make sure we get the right traceback when mid-stack inlining.
Update #33309
Change-Id: I23979cbe6b12fad105dbd26698243648aa86a354
Reviewed-on: https://go-review.googlesource.com/c/go/+/195984
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Pulls in a new snapshot of the race detector, containing
a fix that lets it handle mid-stack inlining correctly.
Fixes#33309
Change-Id: I7551912a491f0615e77d069f198c1b8a6eead280
Reviewed-on: https://go-review.googlesource.com/c/go/+/201898
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The race detector C code expects the g register (aka R28) to be
preserved per the C calling convention. Make sure we save/restore it.
Once this is in we can revert the O3 -> O1 change to racebuild.
Change-Id: Ia785b2717c136f565d45bed283e87b744e35c62d
Reviewed-on: https://go-review.googlesource.com/c/go/+/201744
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL extends the runtime instrumentation for (*T)(ptr) to also
check that the first and last bytes of *(*T)(ptr) are part of the same
heap object.
Updates #22218.
Updates #34959.
Change-Id: I2c8063fe1b7fe6e6145e41c5654cb64dd1c9dd41
Reviewed-on: https://go-review.googlesource.com/c/go/+/201778
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL adds -d=checkptr as a compile-time option for adding
instrumentation to check that Go code is following unsafe.Pointer
safety rules dynamically. In particular, it currently checks two
things:
1. When converting unsafe.Pointer to *T, make sure the resulting
pointer is aligned appropriately for T.
2. When performing pointer arithmetic, if the result points to a Go
heap object, make sure we can find an unsafe.Pointer-typed operand
that pointed into the same object.
These checks are currently disabled for the runtime, and can also be
disabled through a new //go:nocheckptr annotation. The latter is
necessary for functions like strings.noescape, which intentionally
violate safety rules to workaround escape analysis limitations.
Fixes#22218.
Change-Id: If5a51273881d93048f74bcff10a3275c9c91da6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/162237
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Generate inline code at defer time to save the args of defer calls to unique
(autotmp) stack slots, and generate inline code at exit time to check which defer
calls were made and make the associated function/method/interface calls. We
remember that a particular defer statement was reached by storing in the deferBits
variable (always stored on the stack). At exit time, we check the bits of the
deferBits variable to determine which defer function calls to make (in reverse
order). These low-cost defers are only used for functions where no defers
appear in loops. In addition, we don't do these low-cost defers if there are too
many defer statements or too many exits in a function (to limit code increase).
When a function uses open-coded defers, we produce extra
FUNCDATA_OpenCodedDeferInfo information that specifies the number of defers, and
for each defer, the stack slots where the closure and associated args have been
stored. The funcdata also includes the location of the deferBits variable.
Therefore, for panics, we can use this funcdata to determine exactly which defers
are active, and call the appropriate functions/methods/closures with the correct
arguments for each active defer.
In order to unwind the stack correctly after a recover(), we need to add an extra
code segment to functions with open-coded defers that simply calls deferreturn()
and returns. This segment is not reachable by the normal function, but is returned
to by the runtime during recovery. We set the liveness information of this
deferreturn() to be the same as the liveness at the first function call during the
last defer exit code (so all return values and all stack slots needed by the defer
calls will be live).
I needed to increase the stackguard constant from 880 to 896, because of a small
amount of new code in deferreturn().
The -N flag disables open-coded defers. '-d defer' prints out the kind of defer
being used at each defer statement (heap-allocated, stack-allocated, or
open-coded).
Cost of defer statement [ go test -run NONE -bench BenchmarkDefer$ runtime ]
With normal (stack-allocated) defers only: 35.4 ns/op
With open-coded defers: 5.6 ns/op
Cost of function call alone (remove defer keyword): 4.4 ns/op
Text size increase (including funcdata) for go cmd without/with open-coded defers: 0.09%
The average size increase (including funcdata) for only the functions that use
open-coded defers is 1.1%.
The cost of a panic followed by a recover got noticeably slower, since panic
processing now requires a scan of the stack for open-coded defer frames. This scan
is required, even if no frames are using open-coded defers:
Cost of panic and recover [ go test -run NONE -bench BenchmarkPanicRecover runtime ]
Without open-coded defers: 62.0 ns/op
With open-coded defers: 255 ns/op
A CGO Go-to-C-to-Go benchmark got noticeably faster because of open-coded defers:
CGO Go-to-C-to-Go benchmark [cd misc/cgo/test; go test -run NONE -bench BenchmarkCGoCallback ]
Without open-coded defers: 443 ns/op
With open-coded defers: 347 ns/op
Updates #14939 (defer performance)
Updates #34481 (design doc)
Change-Id: I51a389860b9676cfa1b84722f5fb84d3c4ee9e28
Reviewed-on: https://go-review.googlesource.com/c/go/+/190098
Reviewed-by: Austin Clements <austin@google.com>
Switch the default to new object files.
Internal linking cgo is disabled for now, as it does not work yet
in newobj mode.
Shared libraries are also broken.
Disable some tests that are known broken for now.
Change-Id: I8ca74793423861d607a2aa7b0d89a4f4d4ca7671
Reviewed-on: https://go-review.googlesource.com/c/go/+/200161
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
This new facility will be used by future CLs in this series.
Change the only blocking call to netpoll to do the right thing when
netpoll returns an empty list.
Updates #6239
Updates #27707
Change-Id: I58b3c2903eda61a3698b1a4729ed0e81382bb1ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/171821
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This changes fixes an oversight in wakeScavenger which would cause ready
to be called off of the system stack. This change makes it so that
wakeScavenger calls goready, which switches to the system stack before
calling ready.
Fixes#34773.
Change-Id: Icb13f180b4d8fdd47c921eac1b896e3dd49e43b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/200999
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Make it a bit more relaxed on the expected fairness, as fastrand()
isn't a truly perfect random number generator.
Fixes#34808
Change-Id: Ib55b2bbe3c1bf63fb4f446fd1291eb1236efc33b
Reviewed-on: https://go-review.googlesource.com/c/go/+/200857
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
nanotime() currently uses the REALTIME clock to get the elapsed
time in Solaris. This commit changes it to use the MONOTONIC clock
instead, similar to how it's done in Linux and other OSs. Also changed
nanotime() and walltime() to call clock_gettime() library function
directly from Go code rather than from assembly.
Fixes#33674
Change-Id: Ie4a687b17d2140998ecd97af6ce048c86cf5fc02
Reviewed-on: https://go-review.googlesource.com/c/go/+/199502
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
If the runtime disables the SIGPROF handler, because this is Go code
that is linked into a non-Go program, then don't go back to the
default handling of SIGPROF; just start ignoring SIGPROF.
Otherwise the program can get killed by a stray SIGPROF that is
delivered, presumably to a different thread, after profiling is disabled.
Fixes#19320
Change-Id: Ifebae477d726699c8c82c867604b73110c1cf262
Reviewed-on: https://go-review.googlesource.com/c/go/+/200740
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
An extra goroutine is necessary to handle asynchronous events on wasm.
However, we do not want this goroutine to exist all the time.
This change makes it short-lived, so it ends after the asynchronous
event was handled.
Fixes#34768
Change-Id: I24626ff0af9d803a01ebe33fbb584d04d2059a44
Reviewed-on: https://go-review.googlesource.com/c/go/+/200497
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
We couldn't do this before because sighandler was compiled for nacl.
Updates #30439
Change-Id: Ieec9938b6a1796c48d251cd8b1db1a42c25f3943
Reviewed-on: https://go-review.googlesource.com/c/go/+/200739
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Part 1: CL 199499 (GOOS nacl)
Part 2: CL 200077 (amd64p32 files, toolchain)
Part 3: stuff that arguably should've been part of Part 2, but I forgot
one of my grep patterns when splitting the original CL up into
two parts.
This one might also have interesting stuff to resurrect for any future
x32 ABI support.
Updates #30439
Change-Id: I2b4143374a253a003666f3c69e776b7e456bdb9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/200318
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is part two if the nacl removal. Part 1 was CL 199499.
This CL removes amd64p32 support, which might be useful in the future
if we implement the x32 ABI. It also removes the nacl bits in the
toolchain, and some remaining nacl bits.
Updates #30439
Change-Id: I2475d5bb066d1b474e00e40d95b520e7c2e286e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/200077
Reviewed-by: Ian Lance Taylor <iant@golang.org>
You were a useful port and you've served your purpose.
Thanks for all the play.
A subsequent CL will remove amd64p32 (including assembly files and
toolchain bits) and remaining bits. The amd64p32 removal will be
separated into its own CL in case we want to support the Linux x32 ABI
in the future and want our old amd64p32 support as a starting point.
Updates #30439
Change-Id: Ia3a0c7d49804adc87bf52a4dea7e3d3007f2b1cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/199499
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
On wasm there is a special goroutine that handles asynchronous events.
Blocking this goroutine often causes a deadlock. However, the stack
trace of this goroutine was omitted when printing the deadlock error.
This change adds an exception so the goroutine is not considered as
an internal system goroutine and the stack trace gets printed, which
helps with debugging the deadlock.
Updates #32764
Change-Id: Icc8f5ba3ca5a485d557b7bdd76bf2f1ffb92eb3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/199537
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
It's pointless to reach all ms via allgs, and doing so introduces a
race, since the m member of a g can change underneath it. Instead
iterate directly through the allm linked list.
Updates: #31528
Updates: #34130
Change-Id: I34b88402b44339b0a5b4cd76eafd0ce6e43e2be1
Reviewed-on: https://go-review.googlesource.com/c/go/+/198417
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently cgoCheckPointer is only used with one optional argument.
Using a slice for the optional arguments is quite expensive, hence
replace it with a single interface{}. This results in ~30% improvement.
When checking struct fields, they quite often end up being without
pointers. Check this before calling cgoCheckPointer, which results in
additional ~20% improvement.
Inline some p == nil checks from cgoIsGoPointer which gives
additional ~15% improvement.
All of this translates to:
name old time/op new time/op delta
CgoCall/add-int-32 46.9ns ± 1% 46.6ns ± 1% -0.75% (p=0.000 n=18+20)
CgoCall/one-pointer-32 143ns ± 1% 87ns ± 1% -38.96% (p=0.000 n=20+20)
CgoCall/eight-pointers-32 767ns ± 0% 327ns ± 1% -57.30% (p=0.000 n=18+16)
CgoCall/eight-pointers-nil-32 110ns ± 1% 89ns ± 2% -19.10% (p=0.000 n=19+19)
CgoCall/eight-pointers-array-32 5.09µs ± 1% 3.56µs ± 2% -30.09% (p=0.000 n=19+19)
CgoCall/eight-pointers-slice-32 3.92µs ± 0% 2.57µs ± 2% -34.48% (p=0.000 n=20+20)
Change-Id: I2aa9f5ae8962a9a41a7fb1db0c300893109d0d75
Reviewed-on: https://go-review.googlesource.com/c/go/+/198081
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
By lazily starting the signal watch loop only on Notify,
we are able to have deadlock detection even when
"os/signal" is imported.
Thanks to Ian Lance Taylor for the solution and discussion.
With this change in, fix a runtime gorountine count test that
assumed that os/signal.init would unconditionally start the
signal watching goroutine, but alas no more.
Fixes#21576.
Change-Id: I6eecf82a887f59f2ec8897f1bcd67ca311ca42ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/101036
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
While understanding why syscall.Read is 2x slower on darwin/amd64, I found
out that, contrary to popular belief, the slowdown is not due to the migration
to use libSystem.dylib instead of direct SYSCALLs, i.e., CL 141639 (and #17490),
but due to a subtle change introduced in CL 141639.
Previously, syscall.Read used syscall.Syscall(SYS_READ), whose preamble called
runtime.entersyscall, but after CL 141639, syscall.Read changes to call
runtime.syscall_syscall instead, which in turn calls runtime.entersyscallblock
instead of runtime.entersyscall. And the entire 2x slow down can be attributed
to this change.
I think this is unnecessary as even though syscalls like Read might block, it
does not always block, so there is no need to handoff P proactively for each
Read. Additionally, we have been fine with not handing off P for each Read
prior to Go 1.12, so we probably don't need to change it. This changes restores
the pre-Go 1.12 behavior, where syscall preamble uses runtime.entersyscall,
and we rely on sysmon to take P back from g blocked in syscalls.
Change-Id: If76e97b5a7040cf1c10380a567c4f5baec3121ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/197938
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There are currently two edges in the lock cycle graph caused by
scavenge.lock: with sched.lock and mheap_.lock. These edges appear
because of the call to ready() and stack growths respectively.
Furthermore, there's already an invariant in the code wherein
mheap_.lock must be acquired before scavenge.lock, hence the cycle.
The fix to this is to bring scavenge.lock higher in the lock cycle
graph, such that sched.lock and mheap_.lock are only acquired once
scavenge.lock is already held.
To faciliate this change, we move scavenger waking outside of
gcSetTriggerRatio such that it doesn't have to happen with the heap
locked. Furthermore, we check scavenge generation numbers with the heap
locked by using gopark instead of goparkunlock, and specify a function
which aborts the park should there be any skew in generation count.
Fixes#34047.
Change-Id: I3519119214bac66375e2b1262b36ce376c820d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/191977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Change the comment to make more conformable to the function implementation.
Change-Id: I8461e2f09824c50e16223a27d0f61070f04bd21b
GitHub-Last-Rev: c25a8493d3
GitHub-Pull-Request: golang/go#27404
Reviewed-on: https://go-review.googlesource.com/c/go/+/132477
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Add a bunch of extra tests and benchmarks for defer, in preparation for new
low-cost (open-coded) implementation of defers (see #34481),
- New file defer_test.go that tests a bunch more unusual defer scenarios,
including things that might have problems for open-coded defers.
- Additions to callers_test.go actually verifying what the stack trace looks like
for various panic or panic-recover scenarios.
- Additions to crash_test.go testing several more crash scenarios involving
recursive panics.
- New benchmark in runtime_test.go measuring speed of panic-recover
- New CGo benchmark in cgo_test.go calling from Go to C back to Go that
shows defer overhead
Updates #34481
Change-Id: I423523f3e05fc0229d4277dd00073289a5526188
Reviewed-on: https://go-review.googlesource.com/c/go/+/197017
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Inline scavenging causes significant performance regressions in tail
latency for k8s and has relatively little benefit for RSS footprint.
We disabled inline scavenging in Go 1.12.5 (CL 174102) as well, but
we thought other changes in Go 1.13 had mitigated the issues with
inline scavenging. Apparently we were wrong.
This CL switches back to only doing foreground scavenging on heap
growth, rather than doing it when allocation tries to allocate from
scavenged space.
Fixes#32828.
Change-Id: I1f5df44046091f0b4f89fec73c2cde98bf9448cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/183857
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently, we map and grow the heap a whole arena (64MB) at a time.
Unfortunately, in order to fix#32828, we need to switch from
scavenging inline with allocation back to scavenging on heap growth,
but heap-growth scavenging happens in large jumps because we grow the
heap in large jumps.
In order to prepare for better heap-growth scavenging, this CL
separates mapping more space for the heap from actually "growing" it
(tracking the new space with spans). Instead, growing the heap keeps
track of the "current arena" it's growing into. It track that with new
spans as needed, and only maps more arena space when the current arena
is inadequate. The effect to the user is the same, but this will let
us scavenge on much smaller increments of heap growth.
There are two slightly subtleties to this change:
1. If an allocation requires mapping a new arena and that new arena
isn't contiguous with the current arena, we don't want to lose the
unused space in the current arena, so we have to immediately track
that with a span.
2. The mapped space must be accounted as released and idle, even
though it isn't actually tracked in a span.
For #32828, since this makes heap-growth scavenging far more
effective, especially at small heap sizes. For example, this change is
necessary for TestPhysicalMemoryUtilization to pass once we remove
inline scavenging.
Change-Id: I300e74a0534062467e4ce91cdc3508e5ef9aa73a
Reviewed-on: https://go-review.googlesource.com/c/go/+/189957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
This change makes it so that the scavenge goal is defined primarily in
terms of heap_inuse at the end of the last GC rather than next_gc. The
reason behind this change is that next_gc doesn't take into account
fragmentation, and we can fall into situation where the scavenger thinks
it should have work to do but there's no free and unscavenged memory
available.
In order to ensure the scavenge goal still tracks next_gc, we multiply
heap_inuse by the ratio between the current heap goal and the last heap
goal, which describes whether the heap is growing or shrinking, and by
how much.
Finally, this change updates the documentation for scavenging and
elaborates on why the scavenge goal is defined the way it is.
Fixes#34048.
Updates #32828.
Change-Id: I8deaf87620b5dc12a40ab8a90bf27932868610da
Reviewed-on: https://go-review.googlesource.com/c/go/+/193040
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This code is not currently compiling, the asm vet checks fail. When running race.bash on ppc64le, I get:
runtime/race_ppc64le.s:104:1: [ppc64le] RaceReadRange: wrong argument size 24; expected $...-16
runtime/race_ppc64le.s:514:1: [ppc64le] racecallbackthunk: unknown variable cmd; offset 0 is arg+0(FP)
runtime/race_ppc64le.s:515:1: [ppc64le] racecallbackthunk: unknown variable ctx
I'm also not sure why it ever worked; it looks like it is writing
the arguments to racecallback in the wrong place (the race detector
itself probably still works, it would just have trouble symbolizing
any resulting race report).
At a meta-level, we should really add a ppc64le/race builder.
Otherwise this code will rot, as evidenced by the rot this CL fixes :)
Update #33309
Change-Id: I3b49c2442aa78538fbb631a143a757389a1368fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/197337
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 192937 introduced some changes which weren't properly gofmt'ed. Do so
now.
Change-Id: I2d2d57ea8a79fb41bc4ca59fa23f12198d615fd8
Reviewed-on: https://go-review.googlesource.com/c/go/+/196812
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
`cmd/compile/internal/gc/reflect.go:/^func.dumptypestructs` was modified many times, now is `cmd/compile/internal/gc/reflect.go:/^func.dumptabs`
Change-Id: Ie949a5bee7878c998591468a04f67a8a70c61da7
GitHub-Last-Rev: 9ecc26985e
GitHub-Pull-Request: golang/go#34489
Reviewed-on: https://go-review.googlesource.com/c/go/+/197037
Reviewed-by: Keith Randall <khr@golang.org>
This fixes a regression introduced with CL 192937. That change
was intended to fix a problem in arm and arm64 but also added
code to change the behavior in ppc64 and ppc64le even though the
error never occurred there. The change to function sigFetchG
assumes that the register holding 'g' could be clobbered by
vdso code when in fact 'g' is in R30 and that is nonvolatile
in the 64-bit PowerPC ELF ABI so would not be clobbered in vdso code.
So if this happens somehow the path it takes is incorrect,
falling through to a call to badsignal which doesn't seem right.
This regression caused intermittent hangs on the builder dashboard
for ppc64, and can be reproduced consistently when running os/signal
TestStress on some ppc64 systems.
I mentioned this problem is issue #34391 because I thought it was
related to another problem described there.
Change-Id: I2ee3606de302bafe509d300077ce3b44b88571a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/196658
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Upgrade the thread sanitizer to handle mid-stack inlining correctly.
We can now return multiple stack frames for each pc that the thread sanitizer
gives us to symbolize.
To fix#33309, we still need to modify the tsan library with its portion
of this fix, rebuild the .syso files on all supported archs, and check
them into runtime/race.
Update #33309
Change-Id: I340013631ffc8428043ab7efe3a41b6bf5638eaf
Reviewed-on: https://go-review.googlesource.com/c/go/+/195781
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
All spelling in source code is "fieldAlign", except this place, so change
"fieldalign" to use mixedCaps.
Change-Id: Icbd9b9d23d9b4f756174e9a3cc4b25776fd90def
GitHub-Last-Rev: 44a4fe140a
GitHub-Pull-Request: golang/go#34441
Reviewed-on: https://go-review.googlesource.com/c/go/+/196757
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
In a position independent executable the data or BSS may be located
close to the end of memory. If it is placed closer than
rootBlockBytes, then the calculations in markrootBlock would overflow,
and the test that ensures that n is not larger than n0 would fail.
This would then cause scanblock to scan data that it shouldn't,
using an effectively random ptrmask, leading to program crashes.
No test because the only way to test it is to build a PIE and convince
the kernel to put the data section near the end of memory, and I don't
know how to do that. Or perhaps we could use a linker script, but that
is painful.
The new code is algebraically identical to the original code, but
avoids the potential overflow of b+rootBlockBytes.
Change-Id: Ieb4e5465174bb762b063d2491caeaa745017345e
Reviewed-on: https://go-review.googlesource.com/c/go/+/195717
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The current msanwrite() segfaults during libpreinit
when built with -msan on arm64. The cause is msancall()
in runtime/msan_arm64.s called by msanwrite() assumes
that it is always called with a valid g, leading to a
segfult.
This CL adds a check for nil g in msancall().
Fixes#34338
Change-Id: If4ad7e37556cd1d99346c1a7b4852651d1e4e4aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/196157
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Now that mid-stack inlining reports backtraces correctly, we no
longer need to protect against inlining in a few critical areas.
Update #19348
Update #28640
Update #34276
Change-Id: Ie68487e6482c3a9509ecf7ecbbd40fe43cee8381
Reviewed-on: https://go-review.googlesource.com/c/go/+/195818
Reviewed-by: David Chase <drchase@google.com>
As discussed in #32912, a crash occurs when go runtime calls a VDSO function (say
__vdso_clock_gettime) and a signal arrives to that thread.
Since VDSO functions temporarily destroy the G register (R10),
Go functions asynchronously executed in that thread (i.e. Go's signal
handler) can try to load data from the destroyed G, which causes
segmentation fault.
To fix the issue a guard is inserted in front of sigtrampgo, so that the control escapes from
signal handlers without touching G in case the signal occurred in the VDSO context.
The test case included in the patch is take from discussion in a relevant thread on github:
https://github.com/golang/go/issues/32912#issuecomment-517874531.
This patch not only fixes the issue on AArch64 but also that on 32bit ARM.
Fixes#32912
Change-Id: I657472e54b7aa3c617fabc5019ce63aa4105624a
GitHub-Last-Rev: 28ce42c4a0
GitHub-Pull-Request: golang/go#34030
Reviewed-on: https://go-review.googlesource.com/c/go/+/192937
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Use the following (suboptimal) script to obtain a list of possible
typos:
#!/usr/bin/env sh
set -x
git ls-files |\
grep -e '\.\(c\|cc\|go\)$' |\
xargs -n 1\
awk\
'/\/\// { gsub(/.*\/\//, ""); print; } /\/\*/, /\*\// { gsub(/.*\/\*/, ""); gsub(/\*\/.*/, ""); }' |\
hunspell -d en_US -l |\
grep '^[[:upper:]]\{0,1\}[[:lower:]]\{1,\}$' |\
grep -v -e '^.\{1,4\}$' -e '^.\{16,\}$' |\
sort -f |\
uniq -c |\
awk '$1 == 1 { print $2; }'
Then, go through the results manually and fix the most obvious typos in
the non-vendored code.
Change-Id: I3cb5830a176850e1a0584b8a40b47bde7b260eae
Reviewed-on: https://go-review.googlesource.com/c/go/+/193848
Reviewed-by: Robert Griesemer <gri@golang.org>
This change makes it so that if we're already finding ourselves in a
situation where we've done more scan work than expected in the
steady-state (that is, 50% of heap_scan for GOGC=100), then we fall back
on the hard heap goal instead of continuing to assume the expected case.
In some cases its possible that we're already doing more scan work than
expected, and if GC assists come in just at that window where we notice
it, they might accumulate way too much assist credit, causing undue heap
growths if GOMAXPROCS=1 (since the fractional background worker isn't
guaranteed to fire). This case seems awfully specific, and that's
because it's exactly the case for TestGcSys, which has been flaky for
some time as a result.
Fixes#28574, #27636, and #27156.
Change-Id: I771f42bed34739dbb1b84ad82cfe247f70836031
Reviewed-on: https://go-review.googlesource.com/c/go/+/184097
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This adds a platform-independent implementation of nacl's faketime
support. It can be enabled by setting the faketime build tag.
Updates #30439.
Change-Id: Iee097004d56d796e6d2bfdd303a092c067ade87e
Reviewed-on: https://go-review.googlesource.com/c/go/+/192740
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In preparation for general faketime support, this renames the existing
nanotime, walltime, and write functions to nanotime1, walltime1, and
write1 and wraps them with trivial Go functions. This will let us
inject different implementations on all platforms when faketime is
enabled.
Updates #30439.
Change-Id: Ice5ccc513a32a6d89ea051638676d3ee05b00418
Reviewed-on: https://go-review.googlesource.com/c/go/+/192738
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change makes it so that worldsema isn't held across the mark phase.
This means that various operations like ReadMemStats may now stop the
world during the mark phase, reducing latency on such operations.
Only three such operations are still no longer allowed to occur during
marking: GOMAXPROCS, StartTrace, and StopTrace.
For the former it's because any change to GOMAXPROCS impacts GC mark
background worker scheduling and the details there are tricky.
For the latter two it's because tracing needs to observe consistent GC
start and GC end events, and if StartTrace or StopTrace may stop the
world during marking, then it's possible for it to see a GC end event
without a start or GC start event without an end, respectively.
To ensure that GOMAXPROCS and StartTrace/StopTrace cannot proceed until
marking is complete, the runtime now holds a new semaphore, gcsema,
across the mark phase just like it used to with worldsema.
Fixes#19812.
Change-Id: I15d43ed184f711b3d104e8f267fb86e335f86bf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/182657
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add a space before build tag comments so it corresponds to the format
documented at https://golang.org/pkg/go/build/.
Change-Id: I8349d0343597e304b97fb5479847231ed8945b1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/193237
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Right now we generate hash functions for all types, just in case they
are used as map keys. That's a lot of wasted effort and binary size
for types which will never be used as a map key. Instead, generate
hash functions only for types that we know are map keys.
Just doing that is a bit too simple, since maps with an interface type
as a key might have to hash any concrete key type that implements that
interface. So for that case, implement hashing of such types at
runtime (instead of with generated code). It will be slower, but only
for maps with interface types as keys, and maybe only a bit slower as
the aeshash time probably dominates the dispatch time.
Reorg where we keep the equals and hash functions. Move the hash function
from the key type to the map type, saving a field in every non-map type.
That leaves only one function in the alg structure, so get rid of that and
just keep the equal function in the type descriptor itself.
cmd/go now has 10 generated hash functions, instead of 504. Makes
cmd/go 1.0% smaller. Update #6853.
Speed on non-interface keys is unchanged. Speed on interface keys
is ~20% slower:
name old time/op new time/op delta
MapInterfaceString-8 23.0ns ±21% 27.6ns ±14% +20.01% (p=0.002 n=10+10)
MapInterfacePtr-8 19.4ns ±16% 23.7ns ± 7% +22.48% (p=0.000 n=10+8)
Change-Id: I7c2e42292a46b5d4e288aaec4029bdbb01089263
Reviewed-on: https://go-review.googlesource.com/c/go/+/191198
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
This reverts CL 191759.
Reason for revert: broke most Go programs using the time package on Wine,
including on 4.15, the latest stable version. Only wine-staging (with
experimental patches) contains an upstream fix we could rely on.
Change-Id: Ic8ba126022e54f412174042fbb9abed82d5eb318
Reviewed-on: https://go-review.googlesource.com/c/go/+/192622
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
This CL removes an outdated comment regarding converting a pointer to `uintptr`.
The comment was introduced in Go 1.4 and runtime GC was under the consideration of major revisions. According to the current situation, Go runtime memory allocator has no fragmentation issue. Therefore compact GC won't be implemented in the near future.
Change-Id: I5c336d81d810cf57b76797f05428421bb39a5b9f
GitHub-Last-Rev: 2ab4be3885
GitHub-Pull-Request: golang/go#33685
Reviewed-on: https://go-review.googlesource.com/c/go/+/190520
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
macOS and iOS deliver SIGPIPE signals to the main thread and not
the thread that raised it by writing to a closed socket or pipe.
SIGPIPE signals can be suppressed for sockets with the SO_NOSIGPIPE
option, but there is no similar option for pipes. We have no other
choice but to never forward SIGPIPE on macOS.
This is a fixup of reverted CL 188297.
Fixes#33384
Change-Id: I09b258b078857ad3b22025bc2902d1b12d2afd92
Reviewed-on: https://go-review.googlesource.com/c/go/+/191785
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
If Go DLL is used by external C program, and lastcontinuehandler
is reached, lastcontinuehandler will crash the process it is
running in.
But it should not be up to Go runtime to decide if process to be
crashed or not - it should be up to C runtime. This CL adjusts
lastcontinuehandler to not to crash when running as DLL.
Fixes#32648.
Change-Id: Ia455e69b8dde2a6f42f06b90e8af4aa322ca269a
GitHub-Last-Rev: dbdffcb432
GitHub-Pull-Request: golang/go#32574
Reviewed-on: https://go-review.googlesource.com/c/go/+/181839
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
runtime/asm.s contains two variable declarations that don't seem
needed. The variables are defined in Go and not referenced in
assembly. They were added in 2014 during the C to Go transition.
Maybe they were useful at that time, but not now. Remove them.
Change-Id: Id00d724813d18db47126c2f2b8cacfc9d77ffd4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/192378
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Starting in Windows 8, the wait functions don't take into account
suspend time, even though the monotonic counters do. This results in
timer buckets stalling on resume. Therefore, this commit makes it so
that on resume, we return from the wait functions and recalculate the
amount of time left to wait.
Fixes: #31528
Change-Id: I0db02cc72188cb620954e87a0180e0a3c83f4a56
Reviewed-on: https://go-review.googlesource.com/c/go/+/191957
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Currently the standard hasher is memhash, which checks whether aes
instructions are available, and if so redirects to aeshash.
With this CL, we call aeshash directly, which then redirects to the
fallback hash if aes instructions are not available.
This reduces the overhead for the hash function in the common case,
as it requires just one call instead of two. On architectures which
have no assembly hasher, it's a single jump slower.
Thanks to Martin for this idea.
name old time/op new time/op delta
BigKeyMap-4 22.6ns ± 1% 21.1ns ± 2% -6.55% (p=0.000 n=9+10)
Change-Id: Ib7ca77b63d28222eb0189bc3d7130531949d853c
Reviewed-on: https://go-review.googlesource.com/c/go/+/190998
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
This reverts CL 188297.
Reason for revert: broke multiple of the darwin builders.
Fixes#33943.
Change-Id: Iacff98d1450edc70402dc7a220d16fcd73337c9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/191784
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
A few years ago, Wine-specific detection was added as an ugly hack to
work around shortcomings in the emulation layer. Probably it's best to
not special case this emulator versus that emulator versus the real
deal, but there were two arguments presented in the hack's favor:
1. Wine is useful and developers will appreciate being able to debug
stuff with it.
2. The existing KUSER_SHARED_DATA technique for gathering time is
undocumented, and we shouldn't be relying on it anyway, since
Microsoft might remove it without notice.
As it turns out, neither one of these are, at the time of writing, true.
(1) has been handled for some time by Wine with the introduction of the
commit entitled "ntdll: Create thread to update user_shared_data time
values when necessary". And (2) is in fact documented:
https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ntddk/ns-ntddk-kuser_shared_datahttps://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/-kuser
It's in use so widely by both third-party software (such as games, a
massive market segment) and by Microsoft binaries that removing it from
the operating system will basically never happen.
So with both issues taken care of, this commit simply gets rid of the
old hack.
Change-Id: I80093f50e0d10d53648128d0f9dd76b1b92a119e
Reviewed-on: https://go-review.googlesource.com/c/go/+/191759
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
macOS and iOS deliver SIGPIPE signals to the main thread and not
the thread that raised it by writing to a closed socket or pipe.
SIGPIPE signals can be suppressed for sockets with the SO_NOSIGPIPE
option, but there is no similar option for pipes. We have no other
choice but to never forward SIGPIPE on macOS.
Fixes#33384
Change-Id: Ice3de75b121f00006ee11c26d560e619536460be
Reviewed-on: https://go-review.googlesource.com/c/go/+/188297
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This matches the existing behavior of treating CTRL_C_EVENT, CTRL_BREAK_EVENT as a synthesized SIGINT event.
See https://docs.microsoft.com/en-us/windows/console/handlerroutine for a good documentation source upstream to confirm these values.
As for the usage of these events, the "Timeouts" section of that upstream documentation is important to note, especially the limited window in which to do any cleanup before the program will be forcibly killed (defaults typically 5s, but as low as 500ms, and in many cases configurable system-wide).
These events are especially relevant for Windows containers, where these events (particularly `CTRL_SHUTDOWN_EVENT`) are one of the only ways containers can "gracefully" shut down (https://github.com/moby/moby/issues/25982#issuecomment-466804071).
This was verified by making a simple `main()` which implements the same code as in `ExampleNotify_allSignals` but in a `for` loop, building a `main.exe`, running that in a container, then doing `docker kill -sTERM` on said container. The program prints `Got signal: SIGTERM`, then exits after the aforementioned timeout, as expected. Behavior before this patch is that the program gets no notification (and thus no output) but still exits after the timeout.
Fixes#7479
Change-Id: I2af79421cd484a0fbb9467bb7ddb5f0e8bc3610e
GitHub-Last-Rev: 9e05d631b5
GitHub-Pull-Request: golang/go#33311
Reviewed-on: https://go-review.googlesource.com/c/go/+/187739
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
The only place set releasetime to negative is in runtime.selectgo
(when blockprofilerate greater than zero), so we are safe in compiler
code.
But scasetype must keep in sync with runtime/select.go scase struct, so
releasetime must be int64.
Change-Id: I39ea944f5f2872452d3ffd57f7604d51e0d2590a
Reviewed-on: https://go-review.googlesource.com/c/go/+/179799
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The existing condition is long and repetitive. Using select/case with
multiple values in the expression list is more concise and clearer.
Change-Id: I43f8abcf958e433468728f1d89ff1436332b29da
Reviewed-on: https://go-review.googlesource.com/c/go/+/188519
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Use efaceOf to safely convert from *interface{} to *_eface, and to
make it clearer what the pointer arithmetic is computing.
Incidentally, remove a spurious unsafe.Pointer->*uint8->unsafe.Pointer
round trip conversion in newproc.
No behavior change.
Change-Id: I2ad9d791d35d8bd008ef43b03dad1589713c5fd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/190457
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Document goroutine label inheritance. Goroutine labels are copied upon
goroutine creation and there is a test enforcing this, but it was not
mentioned in the docstrings for `Do` or `SetGoroutineLabels`.
Add notes to both of those functions' docstrings so it's clear that one
does not need to set labels as soon as a new goroutine is spawned if
they want to propagate tags.
Updates #32223
Updates #23458
Change-Id: Idfa33031af0104b884b03ca855ac82b98500c8b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/189317
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently when we coalesce memory we make a sysHugePage call
(MADV_HUGEPAGE) to ensure freed and coalesced huge pages are treated as
such so the scavenger's assumptions about performance are more in line
with reality.
Unfortunately we do it way too often because we do it if there was any
change to the huge page count for the span we're coalescing into, not
taking into account that it could coalesce with its neighbors and not
actually create a new huge page.
This change makes it so that it only calls sysHugePage if the original
huge page counts between the span to be coalesced into and its neighbors
do not add up (i.e. a new huge page was created due to alignment). Calls
to sysHugePage will now happen much less frequently, as intended.
Updates #32828.
Change-Id: Ia175919cb79b730a658250425f97189e27d7fda3
Reviewed-on: https://go-review.googlesource.com/c/go/+/186926
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds physHugePageShift which is defined such that
1 << physHugePageShift == physHugePageSize. The purpose of this variable
is to avoid doing expensive divisions in key functions, such as
(*mspan).hugePages.
This change also does a sweep of any place we might do a division or mod
operation with physHugePageSize and turns it into bit shifts and other
bitwise operations.
Finally, this change adds a check to mallocinit which ensures that
physHugePageSize is always a power of two. osinit might choose to ignore
non-powers-of-two for the value and replace it with zero, but mallocinit
will fail if it's not a power of two (or zero). It also derives
physHugePageShift from physHugePageSize.
This change helps improve the performance of most applications because
of how often (*mspan).hugePages is called.
Updates #32828.
Change-Id: I1a6db113d52d563f59ae8fd4f0e130858859e68f
Reviewed-on: https://go-review.googlesource.com/c/go/+/186598
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Overflow of the comparison caused very large (>=1<<32) allocations to
sometimes not get sampled at all. Use uintptr so the comparison will
never overflow.
Fixes#33342
Tested on the example in 33342. I don't want to check a test in that
needs that much memory, however.
Change-Id: I51fe77a9117affed8094da93c0bc5f445ac2d3d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/188017
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently, GODEBUG=sbrk=1 mode aligns allocations by their type's
alignment. You would think this would be the right thing to do, but
because 64-bit fields are only 4-byte aligned right now (see #599),
this can cause a 64-bit field of an allocated object to be 4-byte
aligned, but not 8-byte aligned. If there is an atomic access to that
unaligned 64-bit field, it will crash.
This doesn't happen in normal allocation mode because the
size-segregated allocation and the current size classes will cause any
types larger than 8 bytes to be 8 byte aligned.
We fix this by making sbrk=1 mode use alignment based on the type's
size rather than its declared alignment. This matches how the tiny
allocator aligns allocations.
This was tested with
GOARCH=386 GODEBUG=sbrk=1 go test sync/atomic
This crashes with an unaligned access before this change, and passes
with this change.
This should be reverted when/if we fix#599.
Fixes#33159.
Change-Id: Ifc52c72c6b99c5d370476685271baa43ad907565
Reviewed-on: https://go-review.googlesource.com/c/go/+/186919
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
With gccgo, if we generate getg inlined, the backend may cache
the address of the TLS variable, which will become invalid after
a thread switch.
Currently there is no known bug for this. But if we didn't
implement this carefully, we may get subtle bugs. This CL adds a
test that will fail loudly if this is wrong. (See also
https://go.googlesource.com/gofrontend/+/refs/heads/master/libgo/runtime/proc.c#333
and an incorrect attempt CL 185337.)
Note: at least on Linux/AMD64, even with an incorrect
implementation, this only fails if the test is compiled with
-fPIC, which is not the default setting for gccgo test suite. So
some manual work is needed. Maybe we could extend the test suite
to run the runtime test with more settings (e.g. PIC and static).
Change-Id: I459a3b4c31f09b9785c0eca19b7756f80e8ef54c
Reviewed-on: https://go-review.googlesource.com/c/go/+/186357
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This adds comments explaining why it's important that some panics are
allowed in the runtime (even though this isn't ideal).
Change-Id: I04c6fc4f792f3793f951619ccaea6bfef2f1763c
Reviewed-on: https://go-review.googlesource.com/c/go/+/181737
Reviewed-by: Keith Randall <khr@golang.org>
The implementation of semaphores, and therefore notes, used on Darwin
is not async-signal-safe. The runtime has one case where a note needs
to be woken up from a signal handler: the call to notewakeup in sigsend.
That notewakeup call is only called on a single note, and it doesn't
need the full functionality of notes: nothing ever does a timed wait on it.
So change that one note to use a different implementation on Darwin,
based on a pipe. This lets the wakeup code use the write call, which is
async-signal-safe.
Fixes#31264
Change-Id: If705072d7a961dd908ea9d639c8d12b222c64806
Reviewed-on: https://go-review.googlesource.com/c/go/+/184169
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
CL 42652 changed the profile handler for mips/mipsle to
avoid recording a profile when in atomic functions, for fear
of interrupting the 32-bit simulation of a 64-bit atomic with
a lock. The profile logger itself uses 64-bit atomics and might
deadlock (#20146).
The change was to accumulate a count of dropped profile events
and then send the count when the next ordinary event was sent:
if prof.hz != 0 {
+ if (GOARCH == "mips" || GOARCH == "mipsle") && lostAtomic64Count > 0 {
+ cpuprof.addLostAtomic64(lostAtomic64Count)
+ lostAtomic64Count = 0
+ }
cpuprof.add(gp, stk[:n])
}
CL 117057 extended this behavior to include GOARCH == "arm".
Unfortunately, the inserted cpuprof.addLostAtomic64 differs from
the original cpuprof.add in that it neglects to acquire the lock
protecting the profile buffer.
This has caused a steady stream of flakes on the arm builders
for the past 12 months, ever since CL 117057 landed.
This CL moves the lostAtomic count into the profile buffer and
then lets the existing addExtra calls take care of it, instead of
duplicating the locking logic.
Fixes#24991.
Change-Id: Ia386c40034fcf46b31f080ce18f2420df4bb8004
Reviewed-on: https://go-review.googlesource.com/c/go/+/184164
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Currently we use R16 and R17 for ARM64's Duff's devices.
According to ARM64 ABI, R16 and R17 can be used by the (external)
linker as scratch registers in trampolines. So don't use these
registers to pass information across functions.
It seems unlikely that calling Duff's devices would need a
trampoline in normal cases. But it could happen if the call
target is out of the 128 MB direct jump limit.
The choice of R20 and R21 is kind of arbitrary. The register
allocator allocates from low-numbered registers. High numbered
registers are chosen so it is unlikely to hold a live value and
forces a spill.
Fixes#32773.
Change-Id: Id22d555b5afeadd4efcf62797d1580d641c39218
Reviewed-on: https://go-review.googlesource.com/c/go/+/183842
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
In sigtramp of sys_darwin_arm.s, the callee-save register R4 is
saved to the stack, but later R2 is also saved to the save position.
That CL fixes the unexpected lost of the value in R4.
fixes#32744
Change-Id: Ifaeb99f11e4abf0c79bec9da67e0db97c358010c
Reviewed-on: https://go-review.googlesource.com/c/go/+/183517
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts https://golang.org/cl/182258.
The new code caused unpredictable crashes that are not understood. The old code was occasionally flaky but still better than this approach.
Fixes#32655
Updates #31264
Change-Id: I2e9d27d6052e84bf75106d8b844549ba4f571695
Reviewed-on: https://go-review.googlesource.com/c/go/+/182880
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Changes Darwin semaphore support from using pthread mutexes and
condition variables to using dispatch semaphores. Signaling a dispatch
semaphore is async-signal-safe.
Fixes#31264
Change-Id: If0ce47623501db13e3804b14ace5f4d8eaef461e
Reviewed-on: https://go-review.googlesource.com/c/go/+/182258
Reviewed-by: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Keith Randall <khr@golang.org>
This reverts CL 180761
Reason for revert: Reinstate the stack-allocated defer CL.
There was nothing wrong with the CL proper, but stack allocation of defers exposed two other issues.
Issue #32477: Fix has been submitted as CL 181258.
Issue #32498: Possible fix is CL 181377 (not submitted yet).
Change-Id: I32b3365d5026600069291b068bbba6cb15295eb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/181378
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 179862 introduced go:linkname directives to create ABI wrappers for
Store and Store64 on s390x, but a concurrent change (CL 180439)
replaced the Go definitions of these functions with assembly
definitions. This resulted in conflicting definitions for the ABI0
symbols, which led to a bootstrap linking failure.
Fix this by removing the now-incorrect go:linkname directives for
Store and Store64. This should fix the linux-s390x builders.
Updates #31230.
Change-Id: I8de8c03c23412fc217d428c0018cc56eb2f9996f
Reviewed-on: https://go-review.googlesource.com/c/go/+/181078
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Somehow I missed these two functions in CL 179863. This should fix the
linux-arm builders.
Updates #31230.
Change-Id: I3f8bef3fac331b505a55c0850b0fbc799b7c06c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/181077
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The built-in Go resolver works significantly better.
In particular, the use of res_search does not support
CNAME or PTR queries and may not even be thread-safe.
This CL is essentially a revert of CL 166297 plus fixes,
including CL 180842.
See CL 180842 for additional notes about problems
with this approach.
Fixes#31705.
Change-Id: I0a30a0de2fbd04f6c461520fd34378c84aadf66c
Reviewed-on: https://go-review.googlesource.com/c/go/+/180843
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
It matters whether we are calling a function that would
return a 32-bit or 64-bit -1 on error. A few sites were wrong
and this key detail was omitted from syscall/syscallX docs.
Change-Id: I48a421b6cc4d2d2b5e58f790cc947e3cb2f98940
Reviewed-on: https://go-review.googlesource.com/c/go/+/180841
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This marks all Go symbols called from assembly in other packages with
"go:linkname" directives to ensure they get ABI wrappers.
Now that we have this go:linkname convention, this also removes the
abi0Syms definition in the runtime, which was used to give morestackc
an ABI0 wrapper. Instead, we now just mark morestackc with a
go:linkname directive.
This was tested with buildall.bash in the default configuration, with
-race, and with -gcflags=all=-d=ssa/intrinsics/off. Since I couldn't
test cgo on non-Linux configurations, I manually grepped for runtime
symbols in runtime/cgo.
Updates #31230.
Change-Id: I6c8aa56be2ca6802dfa2bf159e49c411b9071bf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/179862
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The z/Architecture does not guarantee that a load following a store
will not be reordered with that store, unless they access the same
address. Therefore if we want to ensure the sequential consistency
of atomic loads and stores we need to perform serialization
operations after atomic stores.
We do not need to serialize in the runtime when using StoreRel[ease]
and LoadAcq[uire]. The z/Architecture already provides sufficient
ordering guarantees for these operations.
name old time/op new time/op delta
AtomicLoad64-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal)
AtomicStore64-16 0.51ns ± 0% 0.60ns ± 9% +16.47% (p=0.000 n=17+20)
AtomicLoad-16 0.51ns ± 0% 0.51ns ± 0% ~ (all equal)
AtomicStore-16 0.51ns ± 0% 0.60ns ± 9% +16.50% (p=0.000 n=18+20)
Fixes#32428.
Change-Id: I88d19a4010c46070e4fff4b41587efe4c628d4d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/180439
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
At least one libc call we make
(res_search, which calls _mdns_query and then mdns_item_call)
pushes a 64 kB stack frame onto the stack.
Then it faults on the guard page.
Use the default system stack size, under the assumption
that the C code being called is compatible with that stack size.
For #31705.
Change-Id: I1b0bfc2e54043c49f0709255988ef920ce30ee82
Reviewed-on: https://go-review.googlesource.com/c/go/+/180779
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts commit fff4f599fe.
Reason for revert: Seems to still have issues around GC.
Fixes#32452
Change-Id: Ibe7af629f9ad6a3d5312acd7b066123f484da7f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/180761
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
When a defer is executed at most once in a function body,
we can allocate the defer record for it on the stack instead
of on the heap.
This should make defers like this (which are very common) faster.
This optimization applies to 363 out of the 370 static defer sites
in the cmd/go binary.
name old time/op new time/op delta
Defer-4 52.2ns ± 5% 36.2ns ± 3% -30.70% (p=0.000 n=10+10)
Fixes#6980
Update #14939
Change-Id: I697109dd7aeef9e97a9eeba2ef65ff53d3ee1004
Reviewed-on: https://go-review.googlesource.com/c/go/+/171758
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
ARM64's R19-R29 and F8-F15 are callee saved registers, which
should be saved in the beginning of sigtramp, and restored at
the end.
fixes#31827
Change-Id: I622e03f1a13fec969d3a11b6a303a8a492e02bcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/177045
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Normally, reflect.makeFuncStub records the context value at a known
point in the stack frame, so that the runtime can get the argument map
for reflect.makeFuncStub from that known location.
This doesn't work for defers or goroutines that haven't started yet,
because they haven't allocated a frame or run an instruction yet. The
argument map must be extracted from the context value. We already do
this for defers (the non-nil ctxt arg to getArgInfo), we just need to
do it for unstarted goroutines as well.
When we traceback a goroutine, remember the context value from
g.sched. Use it for the first frame we find.
(We never need it for deeper frames, because we normally don't stop at
the start of reflect.makeFuncStub, as it is nosplit. With this CL we
could allow makeFuncStub to no longer be nosplit.)
Fixes#25897
Change-Id: I427abf332a741a80728cdc0b8412aa8f37e7c418
Reviewed-on: https://go-review.googlesource.com/c/go/+/180258
Reviewed-by: Cherry Zhang <cherryyz@google.com>
These tests assume that it is OK to switch between time implementations,
but the clock_gettime call uses CLOCK_MONOTONIC and the fallback call,
gettimeofday, uses CLOCK_REALTIME. Disabling the clock_gettime call means
that calls to nanotime will start returning very different values.
This breaks the new timer code, which assumes that nanotime will return
a consistently increasing value.
This test is not very useful in any case as it doesn't check the results.
Removing this file also removes BenchmarkTimeNow, which is a duplicate
of BenchmarkNow in the time package.
Updates #27707Fixes#32109
Change-Id: I6a884af07f75822d724193c5eed94742f524f07d
Reviewed-on: https://go-review.googlesource.com/c/go/+/174679
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently there's an invariant in the runtime wherein the heap lock
can only be acquired on the system stack, otherwise a self-deadlock
could occur if the stack grows while the lock is held.
This invariant is upheld and documented in a number of situations (e.g.
allocManual, freeManual) but there are other places where the invariant
is either not maintained at all which risks self-deadlock (e.g.
setGCPercent, gcResetMarkState, allocmcache) or is maintained but
undocumented (e.g. gcSweep, readGCStats_m).
This change adds go:systemstack to any function that acquires the heap
lock or adds a systemstack(func() { ... }) around the critical section,
where appropriate. It also documents the invariant on (*mheap).lock
directly and updates repetitive documentation to refer to that comment.
Fixes#32105.
Change-Id: I702b1290709c118b837389c78efde25c51a2cafb
Reviewed-on: https://go-review.googlesource.com/c/go/+/177857
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Shorten some of the longest tests that run during all.bash.
Removes 7r 50u 21s from all.bash.
After this change, all.bash is under 5 minutes again on my laptop.
For #26473.
Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79
Reviewed-on: https://go-review.googlesource.com/c/go/+/177559
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
First, remove the randomization of initialization order.
Then, revert to source code order instead of sorted package path order.
This restores the behavior that was in 1.12.
A larger change which will implement the suggestion in #31636 will
wait for 1.14. It's too complicated for 1.13 at this point (it has
tricky interactions with plugins).
Fixes#31636
Change-Id: I35b48e8cc21cf9f93c0973edd9193d2eac197628
Reviewed-on: https://go-review.googlesource.com/c/go/+/178297
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
That way we will never have to look up the file/line for the frame
that's next to be returned when the user stops calling Next.
For the benchmark from #32093:
name old time/op new time/op delta
Helper-4 948ns ± 1% 836ns ± 3% -11.89% (p=0.000 n=9+9)
(#32093 was fixed with a more specific, and better, fix, but this
fix is much more general.)
Change-Id: I89e796f80c9706706d8d8b30eb14be3a8a442846
Reviewed-on: https://go-review.googlesource.com/c/go/+/178077
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, this test allocates many objects and relies on heap-growth
scavenging to happen unconditionally on heap-growth. However with the
new pacing system for the scavenging, this is no longer true and the
test is flaky.
So, this change overhauls TestPhysicalMemoryUtilization to check the
same aspect of the runtime, but in a much more robust way.
Firstly, it sets up a much more constrained scenario: only 5 objects are
allocated total with a maximum worst-case (i.e. the test fails) memory
footprint of about 16 MiB. The test is now aware that scavenging will
only happen if the heap growth causes us to push way past our scavenge
goal, which is based on the heap goal. So, it makes the holes in the
test much bigger and the actual retained allocations much smaller to
keep the heap goal at the heap's minimum size. It does this twice to
create exactly two unscavenged holes. Because the ratio between the size
of the "saved" objects and the "condemned" object is so small, two holes
are sufficient to create a consistent test.
Then, the test allocates one enormous object (the size of the 4 other
objects allocated, combined) with the intent that heap-growth scavenging
should kick in and scavenge the holes. The heap goal will rise after
this object is allocated, so it's very important we do all the
scavenging in a single allocation that exceeds the heap goal because
otherwise the rising heap goal could foil our test.
Finally, we check memory use relative to HeapAlloc as before. Since the
runtime should scavenge the entirety of the remaining holes,
theoretically there should be no more free and unscavenged memory.
However due to other allocations that may happen during the test we may
still see unscavenged memory, so we need to have some threshold. We keep
the current 10% threshold which, while arbitrary, is very conservative
and should easily account for any other allocations the test makes.
Before, we also had to ensure the allocations we were making looked
large relative to the size of a heap arena since newly-mapped memory was
considered unscavenged, and so that could significantly skew the test.
However, thanks to the fix for #32012 we were able to reduce memory use
to 16 MiB in the worst case.
Fixes#32010.
Change-Id: Ia38130481e292f581da7fa3289c98c99dc5394ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/177237
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
A timespec on netbsd/amd64 is int64/int64, not int64/int32.
This bug appears to have been introduced in 7777bac6e4.
Spotted by Cherry Zhang while reviewing https://go-review.googlesource.com/c/go/+/177120.
Change-Id: I163c55d926965defd981bdbfd2511de7d9d4c542
Reviewed-on: https://go-review.googlesource.com/c/go/+/177637
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
On most platforms newly-mapped memory is untouched, meaning the pages
backing the region haven't been faulted in yet. However, we mark this
memory as unscavenged which means the background scavenger
aggressively "returns" this memory to the OS if the heap is small.
The only platform where newly-mapped memory is actually unscavenged (and
counts toward the application's RSS) is on Windows, since
(*mheap).sysAlloc commits the reservation. Instead of making a special
case for Windows, I change the requirements a bit for a sysReserve'd
region. It must now be both sysMap'd and sysUsed'd, with sysMap being a
no-op on Windows. Comments about memory allocation have been updated to
include a more up-to-date mental model of which states a region of memory
may be in (at a very low level) and how to transition between these
states.
Now this means we can correctly mark newly-mapped heap memory as
scavenged on every platform, reducing the load on the background
scavenger early on in the application for small heaps. As a result,
heap-growth scavenging is no longer necessary, since any actual RSS
growth will be accounted for on the allocation codepath.
Finally, this change also cleans up grow a little bit to avoid
pretending that it's freeing an in-use span and just does the necessary
operations directly.
Fixes#32012.
Fixes#31966.
Updates #26473.
Change-Id: Ie06061eb638162e0560cdeb0b8993d94cfb4d290
Reviewed-on: https://go-review.googlesource.com/c/go/+/177097
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This updates the Go function declarations to match race_amd64.s.
Change-Id: I2b541a6b335ce732f4c31652aa615240ce7bb1c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/177397
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The memory might not be synchronized in a thread being woken up after a
semasleep. Using atomic instructions in noteclear function will force
this synchronisation.
Fixes#30189
Change-Id: If7432f29b2a1a56288231822db52f3f8d1d6dbfe
Reviewed-on: https://go-review.googlesource.com/c/go/+/163624
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This was left over from the C->Go transition.
Change-Id: I52494af3d49a388dc45b57210ba68292ae01cf84
Reviewed-on: https://go-review.googlesource.com/c/go/+/176897
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
I forgot that in Go assembly, x+16(SP) is not the same as 16(SP).
The former is the virtual stack pointer (one word below FP on x86)
while the latter is the actual stack pointer.
Change-Id: Ibb7012bb97261949f5e1a0dc70869d9a6f50aa99
Reviewed-on: https://go-review.googlesource.com/c/go/+/176557
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The new prototypes of duffzero and duffcopy must be
accompanied by functions. Otherwise buildmode=shared
(in particular, misc/cgo/testshared) has missing symbols.
The right fix, of course, is to implement these on s390x.
For #31916.
Change-Id: I3efff5e3011956341e1b26223a4847a8a91a0453
Reviewed-on: https://go-review.googlesource.com/c/go/+/176397
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for nacl/*,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I6adb4a7b0c2b03d901fba37f9c05e74e5b7b6691
Reviewed-on: https://go-review.googlesource.com/c/go/+/176107
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for windows/*,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: If37ab2b3f6fca4696b8a6afb2ef11ba6c4fb42e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/176106
Reviewed-by: Austin Clements <austin@google.com>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOSes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I63c4805bdd44b301072da66c77086940e2a2765e
Reviewed-on: https://go-review.googlesource.com/c/go/+/176105
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: Ic64f7f4034695dd4c32c9b7f258960faf3742a83
Reviewed-on: https://go-review.googlesource.com/c/go/+/176103
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
except for an unresolved complaint on mips/mipsle that is a bug in vet,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I6ef7e982a2fdbbfbc22cee876ca37ac54d8109e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176102
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "GOOS=linux GOARCH=arm go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: Ifae75b832320b5356ac8773cf85055bfb2bd7214
Reviewed-on: https://go-review.googlesource.com/c/go/+/176101
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "GOOS=linux GOARCH=amd64 go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I4ca1acb02f4666b102d25fcc55fac96b8f80379a
Reviewed-on: https://go-review.googlesource.com/c/go/+/176100
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
This CL makes "GOOS=linux GOARCH=386 go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.
For #31916.
Change-Id: I3e5586a7ff6e359357350d0602c2259493280ded
Reviewed-on: https://go-review.googlesource.com/c/go/+/176099
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Trybots started failing on js/wasm after golang.org/cl/175797 landed,
but it seemed completely unrelated. It would fail very consistently on
the heapsampling.go test.
Digging deeper it was very difficult to ascertain what was going wrong,
but clearly m.locks for some m was non-zero when calling into the
scheduler.
The failure comes from the fact that lock calls into gosched, but it's
unclear how exactly we got there in the first place; there should be no
preemption in this single-threaded context.
Furthermore, lock shouldn't be calling gosched_m at all because in a
single-threaded context, the thread shouldn't be preempted until it
actually unlocks.
But, digging further it turns out the implementation in lock_js.go never
incremented or decremented m.locks. This is definitely wrong because
many parts of the runtime depend on that value being set correctly.
So, this change removes the loop which calls into gosched_m (which
should be unnecessary) and increments and decrements m.locks
appropriately. This appears to fix the aforementioned failure.
Change-Id: Id214c0762c3fb2b405ff55543d7e2a78c17443c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/176297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change makes it so that during scavenging we split spans when the
span we have next for scavenging is larger than the amount of work we
have left to do.
The purpose of this change is to improve the worst-case behavior of the
scavenger: currently, if the scavenger only has a little bit of work to
do but sees a very large free span, it will scavenge the whole thing,
spending a lot of time to get way ahead of the scavenge pacing for no
reason.
With this change the scavenger should follow the pacing more closely,
but may still over-scavenge by up to a physical huge page since the
splitting behavior avoids breaking up huge pages in free spans.
This change is also the culmination of the scavenging improvements, so
we also include benchmark results for this series (starting from
"runtime: merge all treaps into one implementation" until this patch).
This patch stack results in average and peak RSS reductions (up to 11%
and 7% respectively) for some benchmarks, with mostly minimal
performance degredation (3-4% for some benchmarks, ~0% geomean). Each of
these benchmarks was executed with GODEBUG=madvdontneed=1 on Linux; the
performance degredation is even smaller when MADV_FREE may be used, but
the impact on RSS is much harder to measure. Applications that generally
maintain a steady heap size for the most part show no change in
application performance.
These benchmarks are taken from an experimental benchmarking suite
representing a variety of open-source Go packages, the raw results may
be found here:
https://perf.golang.org/search?q=upload:20190509.1
For #30333.
Change-Id: I618a48534d2d6ce5f656bb66825e3c383ab1ffba
Reviewed-on: https://go-review.googlesource.com/c/go/+/175797
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a background scavenging goroutine whose pacing is
determined when the heap goal changes. The scavenger is paced to use
at most 1% of the mutator's time for most systems. Furthermore, the
scavenger's pacing is computed based on the estimated number of
scavengable huge pages to take advantage of optimizations provided by
the OS.
The purpose of this scavenger is to deal with a shrinking heap: if the
heap goal is falling over time, the scavenger should kick in and start
returning free pages from the heap to the OS.
Also, now that we have a pacing system, the credit system used by
scavengeLocked has become redundant. Replace it with a mechanism which
only scavenges on the allocation path if it makes sense to do so with
respect to the new pacing system.
Fixes#30333.
Change-Id: I6203f8dc84affb26c3ab04528889dd9663530edc
Reviewed-on: https://go-review.googlesource.com/c/go/+/142960
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change removes the periodic scavenger which goes over every span
in the heap and scavenges it if it hasn't been used for 5 minutes. It
should no longer be necessary if we have background scavenging
(follow-up).
For #30333.
Change-Id: Ic3a1a4e85409dc25719ba4593a3b60273a4c71e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/143157
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The vetall builder runs vet straight out of golang.org/x/tools,
so submiting CL 176097 in that repo will break the builder
by making all these whitelist entries stale.
Submiting this CL will fix it, by removing them.
The addition of the gcWriteBarrier declaration in runtime/stubs.go
is necessary because the diagnostic is no longer emitted on arm,
so it must be removed from all.txt. Adding it to runtime is better
than adding it to every-other-goarch.txt.
For #31916.
Change-Id: I432f6049cd3ee5a467add5066c440be8616d9d54
Reviewed-on: https://go-review.googlesource.com/c/go/+/176177
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds a new sysHugePage function to provide the equivalent of
Linux's madvise(MADV_HUGEPAGE) support to the runtime. It then uses
sysHugePage to mark a newly-coalesced free span as backable by huge
pages to make the freeHugePages approximation a bit more accurate.
The problem being solved here is that if a large free span is composed
of many small spans which were coalesced together, then there's a chance
that they have had madvise(MADV_NOHUGEPAGE) called on them at some point,
which makes freeHugePages less accurate.
For #30333.
Change-Id: Idd4b02567619fc8d45647d9abd18da42f96f0522
Reviewed-on: https://go-review.googlesource.com/c/go/+/173338
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Now that the treap is first-fit, we can make a nice optimization.
Mainly, since we know that span splitting doesn't modify the relative
position of a span in a treap, we can actually modify a span in-place
on the treap. The only caveat is that we need to update the relevant
metadata.
To enable this optimization, this change introduces a mutate method on
the iterator which takes a callback that is passed the iterator's span.
The method records some properties of the span before it calls into the
callback and then uses those records to see what changed and update
treap metadata appropriately.
Change-Id: I74f7d2ee172800828434ba0194d3d78d3942acf2
Reviewed-on: https://go-review.googlesource.com/c/go/+/174879
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This change tracks the number of potential free and unscavenged huge
pages which will be used to inform the rate at which scavenging should
occur.
For #30333.
Change-Id: I47663e5ffb64cac44ffa10db158486783f707479
Reviewed-on: https://go-review.googlesource.com/c/go/+/170860
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds two new treap iteration types: one for large
unscavenged spans (contain at least one huge page) and one for small
unscavenged spans. This allows us to scavenge the huge spans first by
first iterating over the large ones, then the small ones.
Also, since we now depend on physHugePageSize being a power of two,
ensure that that's the case when it's retrieved from the OS.
For #30333.
Change-Id: I51662740205ad5e4905404a0856f5f2b2d2a5680
Reviewed-on: https://go-review.googlesource.com/c/go/+/174399
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change introduces a treapIterFilter type which represents the
power set of states described by a treapIterType.
This change then adds a treapIterFilter field to each treap node
indicating the types of spans that live in that subtree. The field is
maintained via the same mechanism used to maintain maxPages. This allows
pred, succ, start, and end to be judicious about which subtrees it will
visit, ensuring that iteration avoids traversing irrelevant territory.
Without this change, repeated scavenging attempts can end up being N^2
as the scavenger walks over what it already scavenged before finding new
spans available for scavenging.
Finally, this change also only scavenges a span once it is removed from
the treap. There was always an invariant that spans owned by the treap
may not be mutated in-place, but with this change violating that
invariant can cause issues with scavenging.
For #30333.
Change-Id: I8040b997e21c94a8d3d9c8c6accfe23cebe0c3d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174878
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change modifies the treap implementation to support holding all
spans in a single treap, instead of keeping them all in separate treaps.
This improves ergonomics for nearly all treap-related callsites.
With that said, iteration is now more expensive, but it never occurs on
the fast path, only on scavenging-related paths.
This change opens up the opportunity for further optimizations, such as
splitting spans without treap removal (taking treap removal off the span
allocator's critical path) as well as improvements to treap iteration
(building linked lists for each iteration type and managing them on
insert/removal, since those operations should be less frequent).
For #30333.
Change-Id: I3dac97afd3682a37fda09ae8656a770e1369d0a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174398
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Those print statements are not a good debug helpers
and only clutter the code.
Change-Id: Ifbf450a04e6fa538af68e6352c016728edb4119a
Reviewed-on: https://go-review.googlesource.com/c/go/+/160537
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
sys.HugePageSize was superceded in the last commit by physHugePageSize
which is determined dynamically by querying the operating system.
For #30333.
Change-Id: I827bfca8bdb347e989cead31564a8fffe56c66ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/173757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds the global physHugePageSize which is initialized in
osinit(). physHugePageSize contains the system's transparent huge page
(or superpage) size in bytes.
For #30333.
Change-Id: I2f0198c40729dbbe6e6f2676cef1d57dd107562c
Reviewed-on: https://go-review.googlesource.com/c/go/+/170858
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This repairs one of the several causes of pauses uncovered
by a GC microbenchmark. A pause can occur when a goroutine's
quantum expires "at the same time" a GC is needed. The
current M switches to running a GC worker, which means that
the amount of available work has expanded by one. The GC
worker, however, does not call ready, and does not itself
conditionally wake a P (a "normal" thread would do this).
This is also true if M switches to a traceReader.
This is problem 4 in this list:
https://github.com/golang/go/issues/27732#issuecomment-423301252
Updates #27732.
Change-Id: I6905365cac8504cde6faab2420f4421536551f0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/146817
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change modifies the treap implementation to be address-ordered
instead of size-ordered, and further augments it so it may be used for
allocation. It then modifies the find method to implement a first-fit
allocation policy.
This change to the treap implementation consequently makes it so that
spans are scavenged in highest-address-first order without any
additional changes to the scavenging code. Because the treap itself is
now address ordered, and the scavenging code iterates over it in
reverse, the highest address is now chosen instead of the largest span.
This change also renames the now wrongly-named "scavengeLargest" method
on mheap to just "scavengeLocked" and also fixes up logic in that method
which made assumptions about size.
For #30333.
Change-Id: I94b6f3209211cc1bfdc8cdaea04152a232cfbbb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/164101
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The spec carefully and consistently uses "key" and "element"
as map terminology. The implementation, not so much.
This change attempts to make the implementation consistently
hew to the spec's terminology. Beyond consistency, this has
the advantage of avoid some confusion and naming collisions,
since v and value are very generic and commonly used terms.
I believe that I found all everything, but there are a lot of
non-obvious places for these to hide, and grepping for them is hard.
Hopefully this change changes enough of them that we will start using
elem going forward. Any remaining hidden cases can be removed ad hoc
as they are discovered.
The only externally-facing part of this change is in package reflect,
where there is a minor doc change and a function parameter name change.
Updates #27167
Change-Id: I2f2d78f16c360dc39007b9966d5c2046a29d3701
Reviewed-on: https://go-review.googlesource.com/c/go/+/174523
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The return values are 32 bit, not 64 bit.
I don't think this would be the cause of any problems, but
it can't hurt to fix it.
Change-Id: Icdd50606360ab9d74070271f9d1721d5fe640bc7
Reviewed-on: https://go-review.googlesource.com/c/go/+/174518
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Like GOOS=android which implies the "linux" build tag, GOOS=illumos
implies the "solaris" build tag. This lets the existing ecosystem of
packages still work on illumos, but still permits packages to start
differentiating between solaris and illumos.
Fixes#20603
Change-Id: I8f4eabf1a66060538dca15d7658c1fbc6c826622
Reviewed-on: https://go-review.googlesource.com/c/go/+/174457
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
They were not needed when Go only produced binaries with cgo suppport.
Now that Go is about to run self-hosted on iOS we do need these.
Updates #31722
Change-Id: If233aa2b31edc7b1c2dcac68974f9fba0604f9a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174300
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The address space of WebAssembly's linear memory is contiguous, so
requesting specific addresses is not supported. Do not use heap arena
hints so we do not have unused memory ranges.
This fixes go1 benchmarks on wasm which ran out of memory since
https://golang.org/cl/170950.
Change-Id: I70115b18dbe43abe16dd5f57996343d97bf94760
Reviewed-on: https://go-review.googlesource.com/c/go/+/174203
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Updates #31656.
Change-Id: Ide6f829282fcdf20c67998b766a201a6a92c3035
Reviewed-on: https://go-review.googlesource.com/c/go/+/174132
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When a callback runs on a different thread in Windows, as in the
runtime package test TestCallbackInAnotherThread, it will use the
extra M. That can cause the test in checkdead to fail incorrectly.
Check whether there actually is an extra M before expecting it.
I think this is a general problem unrelated to timers. I think the test
was passing previously because the timer goroutine was using an M.
But I haven't proved that. This change seems correct, and it avoids
the test failure when using the new timers on Windows.
Updates #27707
Change-Id: Ieb31c04ff0354d6fae7e173b59bcfadb8b0464cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174037
Reviewed-by: Keith Randall <khr@golang.org>
Most of the platforms are returning 0 instead of -1 when mmap syscalls
is failing. This patch corrects it for AIX in order to fix
TestMmapErrorSign and to improve AIX compatibility.
Change-Id: I1dad88d0e69163ad55c504b2b4a997892fd876cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174297
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When running Go programs on Corellium virtual iPhones, the Info.plist
files might not exist. Ignore the error.
Updates #31722
Change-Id: Id2e315c09346b69dda9e10cf29fb5dba6743aac4
Reviewed-on: https://go-review.googlesource.com/c/go/+/174202
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
For example, can use `goroutine all bt` to dump all goroutines'
information.
Change-Id: I51b547c2b837913e4bdabf0f45b28f09250a3e34
GitHub-Last-Rev: d04dcd4f58
GitHub-Pull-Request: golang/go#26283
Reviewed-on: https://go-review.googlesource.com/c/go/+/122589
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
OpenBSD does not provide auxv, however we still need to initialise cpu.HWCap.
For now initialise it to the bare minimum, until some form of CPU capability
detection is implemented or becomes available - see issue #31746.
Updates #31656
Change-Id: I68c3c069319fe60dc873f46def2a67c9f3d937d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/174129
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Whitelists functions debugCall32 through debugCall65536 in
runtime.debugCallCheck so that any instruction inside those functions
is considered a safe point.
This is useful for implementing nested function calls.
For example when evaluating:
f(g(x))
The debugger should:
1. initiate the call to 'f' until the entry point of 'f',
2. complete the call to 'g(x)'
3. copy the return value of 'g(x)' in the arguments of 'f'
4. complete the call to 'f'
Similarly for:
f().amethod()
The debugger should initiate the call to '.amethod()', then initiate
and complete the call to f(), copy the return value to the arguments
of '.amethod()' and finish its call.
However in this example, unlike the other example, it may be
impossible to determine the entry point of '.amethod()' until after
'f()' is evaluated, which means that the call to 'f()' needs to be
initiated while stopped inside a debugCall... function.
Change-Id: I575c23542709cedb1a171d63576f7e11069c7674
Reviewed-on: https://go-review.googlesource.com/c/go/+/161137
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
The kqueue system call takes no arguments, hence there should be no need
to zero the registers used for the first syscall arguments.
Change-Id: Ia79b2d4f4d568bb6795cb885e1464cf1fc2bf7c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/174128
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
The notdead argument to sys___threxit() is a pointer, hence requires a 64-bit
move rather than a 32-bit one.
Updates #31656
Change-Id: I52ad31ed5afaf43ccc3d934025288216e8052528
Reviewed-on: https://go-review.googlesource.com/c/go/+/174124
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Testing whether cgo is enabled in go/build is not the same as testing
whether the go tool supports cgo. They differ, for example, when using
GOARCH=386 on an amd64 system, as for a cross-build cgo is disabled by default.
Change-Id: Ib59106c92a3131b73ac6a91c0f7658a1769acf73
Reviewed-on: https://go-review.googlesource.com/c/go/+/174098
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
AIX doesn't allow to mmap an address range which is already mmap.
Therefore, once the region has been allocated, it must munmap before
being able to play with it.
Change-Id: I1547782f0379024f57869f1dda8c1c9bb12d831f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174059
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ps are strictly numbered from 0 to GOMAXPROCS-1, so if procresize
happens to be running on a P that's being destroyed, it moves itself
to P 0.
However, currently procresize destroys the unused Ps *before* moving
itself to P 0. This means it may briefly run on a destroyed P. This is
basically harmless, but has at least one very confusing consequence:
since destroying a P has write barriers, it may enqueue pointers to a
destroyed write barrier buffer. As far as I can tell, there are no
negative consequences of this, but this seems really fragile.
This CL swaps the order of things, so now procresize moves itself to P
0 if necessary before destroying Ps. This ensures it always has a
valid P.
This is part of refactoring for #10958 and #24543, but is a good
cleanup regardless.
Change-Id: I91a23dd6ed27e372f8d6291feec9bc991bcf9812
Reviewed-on: https://go-review.googlesource.com/c/go/+/173941
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
We're about to change some of these rules, so it's about time we wrote
them down!
For #10958, #24543.
Change-Id: I3efce0c44b53bfb6f31ce2d299809b2b4eb329f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/172857
Reviewed-by: Russ Cox <rsc@golang.org>
This change modifies the implementation of mTreap.find to find the
best-fit span with the lowest possible base address.
Fixes#31616.
Change-Id: Ib4bda0f85d7d0590326f939a243a6e4665f37d3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/173479
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This is one small step to force people to not depend on the order of
initialization of packages which are not explicitly ordered by import
directives. Similar to randomizing map iteration order, this makes
sure people aren't depending on the behavior of the current release,
so that we can change the order in future releases without breaking
everyone.
Maybe one day we can randomize always, but for now we do it just in
race mode. (We would need to measure the impact on startup time before
we enabled it always.)
RELNOTE=yes
Change-Id: I99026394796125974c5f2c3660a88becb92c9df3
Reviewed-on: https://go-review.googlesource.com/c/go/+/170318
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
It is easier to ensure that the symbol is always present
if we move it to package runtime. Avoids init-time work.
Also moves it next to buildVersion, the other similar symbol.
Setting up for "go version <binary>".
For #31624.
Change-Id: I943724469ce6992153e701257eb6f12da88c8e4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/173341
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Setting environment variables for go command configuration
is too difficult and system-specific. This CL adds go env -w,
to change the default settings more easily, in a portable way.
It also adds go env -u, to unset those changes.
See https://golang.org/design/30411-env for details.
Fixes#30411.
Change-Id: I36e83f55b666459f8f7f482432a4a6ee015da71d
Reviewed-on: https://go-review.googlesource.com/c/go/+/171137
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
It will use the full names that appear in netbsd's /usr/include/sys/syscall.h names.
This adds some compat-goo (sys_sigprocmask->SYS_sigprocmask14), which might not be pretty, but the information about whether the compat version is used is probably important, as Go will keep using interfaces even after they are considered compatibility, which has caused problems in the past.
also, the same names appear in ktrace (with the numbers).
Change-Id: Idc1bb254ee33757a39ba224d91e8fbb0331e2149
GitHub-Last-Rev: b915e8f8a3
GitHub-Pull-Request: golang/go#31594
Reviewed-on: https://go-review.googlesource.com/c/go/+/173158
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some of the comments were unclear or outdated.
Change-Id: I02e01bf60def0074c1fa760e94aa992e9e4969b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/172987
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
g.m is an muintptr, but we want to print it in hex like a pointer.
Change-Id: Ifc48ed77fb2e93cff7a49d98adc7b9679d26c3b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/172988
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This works well enough to run some code natively on arm64, but not well enough for more complicated code. I've been suggested to start a pull request anyway.
Updates #30824
Change-Id: Ib4f63e0e8a9edfc862cf65b5f1b0fbf9a8a1628e
GitHub-Last-Rev: b01b105e04
GitHub-Pull-Request: golang/go#29398
Reviewed-on: https://go-review.googlesource.com/c/go/+/155739
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Allows us to stop whitelisting this error on many OS/arch combinations
XXX I'm not sure I am running vet correctly, and testing all platforms right.
Change-Id: I29f548bd5f4a63bd13c4d0667d4209c75c886fd9
GitHub-Last-Rev: 52f6ff4a6b
GitHub-Pull-Request: golang/go#31583
Reviewed-on: https://go-review.googlesource.com/c/go/+/173157
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Pass "set print thread-events off" to gdb to suppress thread
event prints, like "[New Thread 0xe7b83b40 (LWP 18609)]". We
don't check them, and the extra output may confuse our other
checks, in particular, checkCleanBacktrace.
Hopefully fixes#31569.
Change-Id: I6549e1280da7afa1d2e38da2b2fa7cc18c2f0373
Reviewed-on: https://go-review.googlesource.com/c/go/+/172980
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The pclntab encoding supports writing only some PCDATA and FUNCDATA values.
However, the encoding is dense: The max index in use determines the space used.
We should thus choose a numbering in which frequently used indices are smaller.
This change re-orders the PCDATA and FUNCDATA indices using that principle,
using a quick and dirty instrumentation to measure index frequency.
It shrinks binaries by about 0.5%.
Updates #6853
file before after Δ %
go 14745044 14671316 -73728 -0.500%
addr2line 4305128 4280552 -24576 -0.571%
api 6095800 6058936 -36864 -0.605%
asm 4930928 4906352 -24576 -0.498%
buildid 2881520 2861040 -20480 -0.711%
cgo 4896584 4867912 -28672 -0.586%
compile 25868408 25770104 -98304 -0.380%
cover 5319656 5286888 -32768 -0.616%
dist 3654528 3634048 -20480 -0.560%
doc 4719672 4691000 -28672 -0.607%
fix 3418312 3393736 -24576 -0.719%
link 6137952 6109280 -28672 -0.467%
nm 4250536 4225960 -24576 -0.578%
objdump 4665192 4636520 -28672 -0.615%
pack 2297488 2285200 -12288 -0.535%
pprof 14735332 14657508 -77824 -0.528%
test2json 2834952 2818568 -16384 -0.578%
trace 11679964 11618524 -61440 -0.526%
vet 8452696 8403544 -49152 -0.581%
Change-Id: I30665dce57ec7a52e7d3c6718560b3aa5b83dd0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/171760
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Fixes#31554
Updates #12032 (also originally about plan9, but later openbsd/arm)
Change-Id: Ib9f35d27a2304f38bf271c38c0b9153d210d8f95
Reviewed-on: https://go-review.googlesource.com/c/go/+/172837
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
As the stackguard was increased on AIX by CL 157117, every syscalls can
now have libcall directly on the stack. This fixes some concurrency bugs
which seems to occur when semasleep is interrupted by a SIGPROF signal.
Change-Id: I905a9618d13ef227dad6f8328b0f958f2f917a5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/172359
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
variable setg_gcc in runtime/cgo/*.c should be static, otherwise it
will be mixed with the function of the same name in runtime/asm_*.s or
tls_*.s, which causes an error when building PIE with internal linking
mode.
Fixes#31485
Change-Id: I79b311ffcaf450984328db65397840ae7d85e65d
Reviewed-on: https://go-review.googlesource.com/c/go/+/172498
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The //go:noescape directive says that arguments don't leak at all,
which is too aggressive of a claim for functions that return pointers
derived from their parameters.
Remove the directive for now. Long term fix will require a new
directive that allows more fine-grained control over escape analysis
information supplied for functions implemented in assembly.
Also, update the BAD comments in the test cases for Loadp: we really
want that *ptr leaks to the result parameter, not that *ptr leaks to
the heap.
Updates #31525.
Change-Id: Ibfa61f2b70daa7ed3223056b57eeee777eef2e31
Reviewed-on: https://go-review.googlesource.com/c/go/+/172578
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Though there is variation in the spelling of canceled,
cancellation is always spelled with a double l.
Reference: https://www.grammarly.com/blog/canceled-vs-cancelled/
Change-Id: I240f1a297776c8e27e74f3eca566d2bc4c856f2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/170060
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Same as CL 170323, but for s390x instead of wasm.
Fixes#31495.
Change-Id: Ie39f649f5e33690375a8bcb1bc3b92d912ca4398
Reviewed-on: https://go-review.googlesource.com/c/go/+/172417
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
sigaction is called as part of library mode initializers
(_rt0_*_lib). Sigaction in turn calls getg, but on Android the TLS
offset for g has not been initialized and getg might return garbage.
Add a check for initialization before calling getg.
Fixes the golang.org/x/mobile/bind/java tests on amd64 and 386.
Fixes#31476
Change-Id: Id2c41fdc983239eca039b49a54b8853c5669d127
Reviewed-on: https://go-review.googlesource.com/c/go/+/172158
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Keith Randall
Gerrit Code Review
Ian Lance Taylor
Michael Anthony Knyszek
Cherry Zhang
Bryan C. Mills
Hana (Hyang-Ah) Kim
Carlo Alberto Ferraris
Austin Clements
Dmitry Vyukov
Dan Scales
Matthew Dempsky
Cuong Manh Le
Wang Xuerui
Joshua M. Clulow
Than McIntosh
Michael Munday
Clément Chigot
Lynn Boger
Tobias Klauser
Ben Shi
Meng Zhuo
smasher164
Giovanni Bajo
Jerrin Shaji George
Richard Musiol
Agniva De Sarker
Brad Fitzpatrick
alan
Ben Schwartz
Jason A. Donenfeld
Egon Elbre
Emmanuel T Odeke
Shenghou Ma
Tardis Xu
Sean Chen
two
fanzha02
Tom Thorogood
Yuichi Nishiwaki
Ainar Garipov
Keith Randall
Changkun Ou
Daniel Martí
Elias Naur
Simon Ferquel
Tianon Gravi
Ayan George
Martin Möhrmann
David Finkel
Russ Cox
Keith Randall
Joel Sing
Joey
Clément Chigot
Tamir Duberstein
Lorenz Nickel
Iskander Sharipov
David Chase
Josh Bleecher Snyder
Joshua M. Clulow
Haosdent Huang
Alessandro Arzilli
Maya Rashish
erifan01