Commit Graph

2393 Commits

Author SHA1 Message Date
Cherry Zhang 3444e5b355 runtime: fix mips assembly
I meant to say ~7, instead of ^7, in the review.

Fix build.

Change-Id: I5060bbcd98b4ab6f00251fdb68b6b35767e5acf1
Reviewed-on: https://go-review.googlesource.com/34411
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-12-15 02:50:23 +00:00
Vladimir Stefanovic b909d01152 runtime: add cgo support for GOARCH=mips{,le}
Change-Id: Ib425ead7b340672837d3cb983bd785488706bd6d
Reviewed-on: https://go-review.googlesource.com/34314
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-12-14 23:52:33 +00:00
David Crawshaw 96414ca39f cmd/link: do not export plugin C symbols
Explicitly filter any C-only cgo functions out of pclntable,
which allows them to be duplicated with the host binary.

Updates #18190.

Change-Id: I50d8706777a6133b3e95f696bc0bc586b84faa9e
Reviewed-on: https://go-review.googlesource.com/34199
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-12-14 19:36:20 +00:00
Euan Kemp afb350811e runtime: correct writebarrier typos
Change-Id: I7d67c3d64be915f0be5932d2c068606d74f93c29
Reviewed-on: https://go-review.googlesource.com/34378
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2016-12-14 18:19:10 +00:00
Ian Lance Taylor fded5dbb2f runtime: don't crash if signal delivered on g0 stack
Also, if we changed the gsignal stack to match the stack we are
executing on, restore it when returning from the signal handler, for
safety.

Fixes #18255.

Change-Id: Ic289b36e4e38a56f8a6d4b5d74f68121c242e81a
Reviewed-on: https://go-review.googlesource.com/34239
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2016-12-12 19:19:59 +00:00
Joel Sing f91ddaabe6 runtime, syscall: update openbsd for changes to syskill
Change the openbsd runtime to use the current sys_kill and sys_thrkill
system calls.

Prior to OpenBSD 5.9 the sys_kill system call could be used with both
processes and threads. In OpenBSD 5.9 this functionality was split into
a sys_kill system call for processes (with a new syscall number) and a
sys_thrkill system call for threads. The original/legacy system call was
retained in OpenBSD 5.9 and OpenBSD 6.0, however has been removed and
will not exist in the upcoming OpenBSD 6.1 release.

Note: This change is needed to make Go work on OpenBSD 6.1 (to be
released in May 2017) and should be included in the Go 1.8 release.
This change also drops support for OpenBSD 5.8, which is already an
unsupported OpenBSD release.

Change-Id: I525ed9b57c66c0c6f438dfa32feb29c7eefc72b0
Reviewed-on: https://go-review.googlesource.com/34093
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-12 01:30:39 +00:00
Raul Silvera c6228ef7e2 runtime/pprof: track locations for goroutine profiles
Must add locations to the profile when generating a profile.proto.
This fixes #18229

Change-Id: I49cd63a30759d3fe8960d7b7c8bd5a554907f8d1
Reviewed-on: https://go-review.googlesource.com/34028
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-12-09 19:14:26 +00:00
Brad Fitzpatrick 4c4201f0e2 all: make spelling consistent
Fixes #17938

Change-Id: Iad12155f4976846bd4a9a53869f89e40e5b3deb3
Reviewed-on: https://go-review.googlesource.com/34147
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2016-12-08 23:22:37 +00:00
Austin Clements 01c6a19e04 runtime: add number of forced GCs to MemStats
This adds a counter for the number of times the application forced a
GC by, e.g., calling runtime.GC(). This is useful for detecting
applications that are overusing/abusing runtime.GC() or
debug.FreeOSMemory().

Fixes #18217.

Change-Id: I990ab7a313c1b3b7a50a3d44535c460d7c54f47d
Reviewed-on: https://go-review.googlesource.com/34067
Reviewed-by: Russ Cox <rsc@golang.org>
2016-12-07 20:59:16 +00:00
Keith Randall 1ea60c136a runtime: on stack copy, adjust BP
When we copy the stack, we need to adjust all BPs.
We correctly adjust the ones on the stack, but we also
need to adjust the one that is in g.sched.bp.

Like CL 33754, no test as only kernel-gathered profiles will notice.
Tests will come (in 1.9) with the implementation of #16638.

The invariant should hold that every frame pointer points to
somewhere within its stack.  After this CL, it is mostly true, but
something about cgo breaks it.  The runtime checks are disabled
until I figure that out.

Update #16638
Fixes #18174

Change-Id: I6023ee64adc80574ee3e76491d4f0fa5ede3dbdb
Reviewed-on: https://go-review.googlesource.com/33895
Reviewed-by: Austin Clements <austin@google.com>
2016-12-07 16:59:25 +00:00
Ian Lance Taylor 80acfe950f runtime/cgo: retry pthread_create on EAGAIN for OpenBSD
For reasons that I do not know, OpenBSD does not call pthread_create
directly, but instead looks it up in libpthread.so. That means that we
can't use the code used on other systems to retry pthread_create on
EAGAIN, since that code simply calls pthread_create.

This patch copies that code to an OpenBSD-specific version.

Also, check for an EAGAIN failure in the test, as that seems to be the
underlying cause of the test failure on several systems including OpenBSD.

Fixes #18146.

Change-Id: I3bceaa1e03a7eaebc2da19c9cc146b25b59243ef
Reviewed-on: https://go-review.googlesource.com/33905
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-05 21:15:05 +00:00
Ian Lance Taylor 7cba779cea runtime/cgo: retry pthread_create on EAGAIN
Update #18146.

Change-Id: Ib447aabae9f203a8b61fb8c984b57d8e2bfe69c2
Reviewed-on: https://go-review.googlesource.com/33894
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-05 18:46:18 +00:00
Ian Lance Taylor 8c31f40a4c runtime: don't skip TestCgoCallbackGC on FreeBSD
Seems to be fixed according to discussion on issue 16396.

Fixes #16396.

Change-Id: Ibac7037a24280204e48cb4d3000af524f65afd36
Reviewed-on: https://go-review.googlesource.com/33903
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-05 18:10:50 +00:00
Russ Cox 7e9fa3c321 runtime: check that Version does not contain \r \n
Change-Id: I8982cfa7337ec457b5235a207ebfda00ef6a2e5a
Reviewed-on: https://go-review.googlesource.com/33917
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-12-05 16:49:53 +00:00
Brad Fitzpatrick 93726522ef runtime: skip GDB tests on mips64
Updates #18173

Change-Id: I4c667c89ba3bf16433a4cef7cb01054f1798667d
Reviewed-on: https://go-review.googlesource.com/33892
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-12-03 17:25:26 +00:00
Austin Clements ea1b90f855 runtime: fix incorrect comment about bitvector layout
Commit 303b69fe packed bitvectors more tightly, but missed a comment
describing their old layout. Update that comment.

Change-Id: I095ccb01f245197054252545f37b40605a550dec
Reviewed-on: https://go-review.googlesource.com/33718
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-12-01 16:42:47 +00:00
Elias Naur 0b2daa5650 Revert "runtime: handle SIGPIPE in c-archive and c-shared programs"
This reverts commit d24b57a6a1.

Reason for revert: Further complications arised (issue 18100). We'll try again in Go 1.9.

Change-Id: I5ca93d2643a4be877dd9c2d8df3359718440f02f
Reviewed-on: https://go-review.googlesource.com/33770
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
2016-12-01 11:23:17 +00:00
Austin Clements f6bff1d587 runtime: fix undead arguments in cgocall
From the garbage collector's perspective, time can move backwards in
cgocall. However, in the midst of this time warp, the pointer
arguments to cgocall can go from dead back to live. If a stack growth
happens while they're dead and then a GC happens when they become live
again, GC can crash with a bad heap pointer.

Specifically, the sequence that leads to a panic is:

1. cgocall calls entersyscall, which saves the PC and SP of its call
site in cgocall. Call this PC/SP "X". At "X" both pointer arguments
are live.

2. cgocall calls asmcgocall. Call the PC/SP of this call "Y". At "Y"
neither pointer argument is live.

3. asmcgocall calls the C code, which eventually calls back into the
Go code.

4. cgocallbackg remembers the saved PC/SP "X" in some local variables,
calls exitsyscall, and then calls cgocallbackg1.

5. The Go code causes a stack growth. This stack unwind sees PC/SP "Y"
in the cgocall frame. Since the arguments are dead at "Y", they are
not adjusted.

6. The Go code returns to cgocallbackg1, which calls reentersyscall
with the recorded saved PC/SP "X", so "X" gets stashed back into
gp.syscallpc/sp.

7. GC scans the stack. It sees there's a saved syscall PC/SP, so it
starts the traceback at PC/SP "X". At "X" the arguments are considered
live, so it scans them, but since they weren't adjusted, the pointers
are bad, so it panics.

This issue started as of commit ca4089ad, when the compiler stopped
marking arguments as live for the whole function.

Since this is a variable liveness issue, fix it by adding KeepAlive
calls that keep the arguments live across this whole time warp.

The existing issue7978 test has all of the infrastructure for testing
this except that it's currently up to chance whether a stack growth
happens in the callback (it currently only happens on the
linux-amd64-noopt builder, for example). Update this test to force a
stack growth, which causes it to fail reliably without this fix.

Fixes #17785.

Change-Id: If706963819ee7814e6705693247bcb97a6f7adb8
Reviewed-on: https://go-review.googlesource.com/33710
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2016-11-30 17:09:17 +00:00
Austin Clements 3f0f24df7b runtime: use standard comment style in cgocall
Change-Id: I9f2c2da4aa512729ae40562b06601da95ba50d6f
Reviewed-on: https://go-review.googlesource.com/33689
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2016-11-30 17:09:06 +00:00
Austin Clements 6f287fa2bb runtime: fall back to /proc/self/auxv in Android libs
Android's libc doesn't provide access to auxv, so currently the Go
runtime synthesizes a fake, minimal auxv when loaded as a library on
Android. This used to be sufficient, but now we depend on auxv to
retrieve the system physical page size and panic if we can't retrieve
it.

Fix this by falling back to reading auxv from /proc/self/auxv if the
loader-provided auxv is empty and removing the synthetic auxv vectors.

Fixes #18041.

Change-Id: Ia2ec2c764a6609331494a5d359032c56cbb83482
Reviewed-on: https://go-review.googlesource.com/33652
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2016-11-29 15:27:53 +00:00
Austin Clements d39b7b5347 runtime: extract Linux auxv handling
This refactoring is in preparation for handling auxv differently in
Android shared libraries.

Updates #18041.

Change-Id: If0458a309f9c804e7abd0a58b5a224d89f8da257
Reviewed-on: https://go-review.googlesource.com/33651
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2016-11-29 15:27:49 +00:00
Ian Lance Taylor f7c351bdf6 internal/pprof: don't discard allocations called by reflect.Call
The pprof code discards all heap allocations made by runtime
routines. This caused it to discard heap allocations made by functions
called by reflect.Call, as the calls are made via the functions
`runtime.call32`, `runtime.call64`, etc. Fix the profiler to retain
these heap allocations.

Fixes #18077.

Change-Id: I8962d552f1d0b70fc7e6f7b2dbae8d5bdefb0735
Reviewed-on: https://go-review.googlesource.com/33635
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-28 21:52:15 +00:00
Michael Munday 06fcc32d14 runtime/cgo: save correct floating point registers on s390x
When transitioning from C code to Go code we must respect the C
calling convention. On s390x this means that r6-r13, r15 and f8-f15
must be saved and restored by functions that use them.

On s390x we were saving the wrong set of floating point registers
(f0, f2, f4 and f6) rather than f8-f15 which means that Go code
could clobber registers that C code expects to be restored. This
CL modifies the crosscall functions on s390x to save/restore the
correct floating point registers.

Fixes #18035.

Change-Id: I5cc6f552c893a4e677669c8891521bf735492e97
Reviewed-on: https://go-review.googlesource.com/33571
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-23 22:06:06 +00:00
Michael Munday 5508561180 runtime/pprof/internal/protopprof: fix test on s390x
Applies the fix from CL 32920 to the new test TestSampledHeapAllocProfile
introduced in CL 33422. The test should be skipped rather than fail if
there is only one executable region of memory.

Updates #17852.

Change-Id: Id8c47b1f17ead14f02a58a024c9a04ebb8ec0429
Reviewed-on: https://go-review.googlesource.com/33453
Run-TryBot: Michael Munday <munday@ca.ibm.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-22 20:57:16 +00:00
Russ Cox f9feaffdf5 runtime: do not print runtime panic frame at top of user stack
The expected default behavior (no explicit GOTRACEBACK setting)
is for the stack trace to start in user code, eliding unnecessary runtime
frames that led up to the actual trace printing code. The idea was that
the first line number printed was the one that crashed.

For #5832 we added code to show 'panic' frames so that if code panics
and then starts running defers and then we trace from there, the panic
frame can help explain why the code seems to have made a call not
present in the code. But that's only needed for panics between two different
call frames, not the panic at the very top of the stack trace.
Fix the fix to again elide the runtime code at the very top of the stack trace.

Simple panic:

	package main

	func main() {
		var x []int
		println(x[1])
	}

Before this CL:

	panic: runtime error: index out of range

	goroutine 1 [running]:
	panic(0x1056980, 0x1091bf0)
		/Users/rsc/go/src/runtime/panic.go:531 +0x1cf
	main.main()
		/tmp/x.go:5 +0x5

After this CL:

	panic: runtime error: index out of range

	goroutine 1 [running]:
	main.main()
		/tmp/x.go:5 +0x5

Panic inside defer triggered by panic:

	package main

	func main() {
		var x []int
		defer func() {
			println(x[1])
		}()
		println(x[2])
	}

Before this CL:

	panic: runtime error: index out of range
		panic: runtime error: index out of range

	goroutine 1 [running]:
	panic(0x1056aa0, 0x1091bf0)
		/Users/rsc/go/src/runtime/panic.go:531 +0x1cf
	main.main.func1(0x0, 0x0, 0x0)
		/tmp/y.go:6 +0x62
	panic(0x1056aa0, 0x1091bf0)
		/Users/rsc/go/src/runtime/panic.go:489 +0x2cf
	main.main()
		/tmp/y.go:8 +0x59

The middle panic is important: it explains why main.main ended up calling main.main.func1 on a line that looks like a call to println. The top panic is noise.

After this CL:

	panic: runtime error: index out of range
		panic: runtime error: index out of range

	goroutine 1 [running]:
	main.main.func1(0x0, 0x0, 0x0)
		/tmp/y.go:6 +0x62
	panic(0x1056ac0, 0x1091bf0)
		/Users/rsc/go/src/runtime/panic.go:489 +0x2cf
	main.main()
		/tmp/y.go:8 +0x59

Fixes #17901.

Change-Id: Id6d7c76373f7a658a537a39ca32b7dc23e1e76aa
Reviewed-on: https://go-review.googlesource.com/33165
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-22 18:33:30 +00:00
Michael Matloob 86ab09eed5 runtime/pprof: generate heap profiles in compressed proto format
When debug is 0, emit the compressed proto format.
The debug>0 format stays the same.

Updates #16093

Change-Id: I45aa1874a22d34cf44dd4aa78bbff9302381cb34
Reviewed-on: https://go-review.googlesource.com/33422
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-22 15:54:58 +00:00
Ian Lance Taylor 75055de84a runtime: sleep a bit to let a bad signal be delivered
When we raise a signal that was delivered to C code, it's possible that
the kernel will not deliver it immediately. This is especially possible
on Darwin where we use send the signal to the entire process rather than
just the current thread. Sleep for a millisecond after sending the
signal to give it a chance to be delivered before we restore the Go
signal handler. In most real cases the program is going to crash at this
point, so sleeping is kind of irrelevant anyhow.

Fixes #14809.

Change-Id: Ib2c0d2c4e240977fb4535dc1dd2bdc50d430eb85
Reviewed-on: https://go-review.googlesource.com/33300
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2016-11-22 04:28:59 +00:00
Cherry Zhang 526b2f85ce runtime/internal/atomic: crash on unaligned 64-bit ops on 32-bit MIPS
This check was originally implemented by Vladimir in
https://go-review.googlesource.com/c/31489/1/src/runtime/internal/atomic/atomic_mipsx.go#30
but removed due to my comment (Sorry!). This CL adds it back.

Fixes #17786.

Change-Id: I7ff4c2539fc9e2afd8199964b587a8ccf093b896
Reviewed-on: https://go-review.googlesource.com/33431
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-22 02:05:07 +00:00
Cherry Zhang 01b4ddb377 runtime/internal/atomic: crash on unaligned 64-bit ops on 386 and ARM
Updates #17786. Will fix mips(32) when the port is fully landed.

Change-Id: I00d4ff666ec14a38cadbcd52569b347bb5bc8b75
Reviewed-on: https://go-review.googlesource.com/33236
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-21 20:26:11 +00:00
Cherry Zhang bbe96f5673 runtime: make work.bytesMarked 8-byte aligned
Make atomic access on 32-bit architectures happy.

Updates #17786.

Change-Id: I42de63ff1381af42124dc51befc887160f71797d
Reviewed-on: https://go-review.googlesource.com/33235
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2016-11-21 20:25:17 +00:00
Michael Matloob ccd69d0582 runtime/pprof: emit count profiles with debug=0 as proto profiles
count profiles with debug=1 retain their previous format.
Also add a test check for the proto profiles since all runtime/pprof
tests only look at the debug=1 profiles.

Change-Id: Ibe805585b597e5d3570807115940a1dc4535c03f
Reviewed-on: https://go-review.googlesource.com/33148
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2016-11-21 17:15:30 +00:00
Austin Clements 0bae74e8c9 runtime: wake idle Ps when enqueuing GC work
If the scheduler has no user work and there's no GC work visible, it
puts the P to sleep (or blocks on the network). However, if we later
enqueue more GC work, there's currently nothing that specifically
wakes up the scheduler to let it start an idle GC worker. As a result,
we can underutilize the CPU during GC if Ps have been put to sleep.

Fix this by making GC wake idle Ps when work buffers are put on the
full list. We already have a hook to do this, since we use this to
preempt a random P if we need more dedicated workers. We expand this
hook to instead wake an idle P if there is one. The logic we use for
this is identical to the logic used to wake an idle P when we ready a
goroutine.

To make this really sound, we also fix the scheduler to re-check the
idle GC worker condition after releasing its P. This closes a race
where 1) the scheduler checks for idle work and finds none, 2) new
work is enqueued but there are no idle Ps so none are woken, and 3)
the scheduler releases its P.

There is one subtlety here. Currently we call enlistWorker directly
from putfull, but the gcWork is in an inconsistent state in the places
that call putfull. This isn't a problem right now because nothing that
enlistWorker does touches the gcWork, but with the added call to
wakep, it's possible to get a recursive call into the gcWork
(specifically, while write barriers are disallowed, this can do an
allocation, which can dispose a gcWork, which can put a workbuf). To
handle this, we lift the enlistWorker calls up a layer and delay them
until the gcWork is in a consistent state.

Fixes #14179.

Change-Id: Ia2467a52e54c9688c3c1752e1fc00f5b37bbfeeb
Reviewed-on: https://go-review.googlesource.com/32434
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-11-20 22:44:22 +00:00
Austin Clements 49ea9207b6 runtime: exit idle worker if there's higher-priority work
Idle GC workers trigger whenever there's a GC running and the
scheduler doesn't find any other work. However, they currently run for
a full scheduler quantum (~10ms) once started.

This is really bad for event-driven applications, where work may come
in on the network hundreds of times during that window. In the
go-gcbench rpc benchmark, this is bad enough to often cause effective
STWs where all Ps are in the idle worker. When this happens, we don't
even poll the network any more (except for the background 10ms poll in
sysmon), so we don't even know there's more work to do.

Fix this by making idle workers check with the scheduler roughly every
100 µs to see if there's any higher-priority work the P should be
doing. This check includes polling the network for incoming work.

Fixes #16528.

Change-Id: I6f62ebf6d36a92368da9891bafbbfd609b9bd003
Reviewed-on: https://go-review.googlesource.com/32433
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2016-11-20 22:44:17 +00:00
Elias Naur d24b57a6a1 runtime: handle SIGPIPE in c-archive and c-shared programs
Before this CL, Go programs in c-archive or c-shared buildmodes
would not handle SIGPIPE. That leads to surprising behaviour where
writes on a closed pipe or socket would raise SIGPIPE and terminate
the program. This CL changes the Go runtime to handle
SIGPIPE regardless of buildmode. In addition, SIGPIPE from non-Go
code is forwarded.

Fixes #17393
Updates #16760

Change-Id: I155e82020a03a5cdc627a147c27da395662c3fe8
Reviewed-on: https://go-review.googlesource.com/32796
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-18 01:19:11 +00:00
Vladimir Stefanovic 5cd6ab5b6d runtime/pprof/internal/protopprof: fix TestTranslateCPUProfileWithSamples test for mips
Change-Id: I01168a7530e18dd1098d467d0c8a330f727ba91f
Reviewed-on: https://go-review.googlesource.com/33281
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-17 19:58:45 +00:00
Austin Clements 05dc6b26ca runtime: improve diagnostics for "scan missed a g"
Currently there are no diagnostics for mark root check during marking.
Fix this by printing out the same diagnostics we print during mark
termination.

Also, drop the allglock before throwing. Holding that across a throw
causes a self-deadlock with tracebackothers.

For #16083.

Change-Id: Ib605f3ae0c17e70704b31d8378274cfaa2307dc2
Reviewed-on: https://go-review.googlesource.com/33339
Reviewed-by: Rick Hudson <rlh@golang.org>
2016-11-17 19:30:14 +00:00
Brad Fitzpatrick afb0ae67b7 runtime/pprof: fix typo in test
Not sure what I was thinking.

Change-Id: I143cdf7c5ef8e7b2394afeca6b30c46bb2c19a55
Reviewed-on: https://go-review.googlesource.com/33340
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-17 16:54:41 +00:00
Lynn Boger b2d34fa51b runtime: handle bad ftab index in symtab.go
If a program has had its text section split into multiple
sections then the ftab that is built is based on addresses
prior to splitting.  That means all the function addresses
are there and correct because of relocation but the
but the computed idx won't always match up quite right and
in some cases go beyond the end of the table, causing a panic.

To resolve this, determine if the idx is too large and if it is,
set it to the last index in ftab.  Then search backward to find the
matching function address.

Fixes #17854

Change-Id: I6940e76a5238727b0a9ac23dc80000996db2579a
Reviewed-on: https://go-review.googlesource.com/32972
Reviewed-by: David Chase <drchase@google.com>
2016-11-17 04:36:53 +00:00
Ian Lance Taylor fe057c1478 runtime/cgo: fixes for calling sigaction in C
Zero out the sigaction structs, in case the sa_restorer field is set.

Clear the SA_RESTORER flag; it is part of the kernel interface, not the
libc interface.

Fixes #17947.

Change-Id: I610348ce3c196d3761cf2170f06c24ecc3507cf7
Reviewed-on: https://go-review.googlesource.com/33331
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2016-11-16 23:10:33 +00:00
David Chase 9f5673d930 cmd/compile: ensure necessary types appear in .debug_info
Autotmp filtering was too aggressive and excluded types
necessary to make debuggers work properly.  Restore the
"late filter" in dwarf.go based on names to exclude autotmps,
and remove the "early filter" in pgen.go based on how the
name was introduced.  However, the updated naming scheme
with a dot prefix is retained to prevent accidental clashes
with legal Go identifier names.

Includes test (grouped with runtime gdb tests),
verified to fail without the fix.

Updates #17644.
Fixes #17830.

Change-Id: I7ec3f7230083889660236e5f6bc77ba5fe434e93
Reviewed-on: https://go-review.googlesource.com/33233
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2016-11-16 22:05:19 +00:00
Brad Fitzpatrick cd66c38619 runtime/pprof: skip profiling tests on mips if highres timers not available
Fixes #17936

Change-Id: I20d09712b7d7303257994356904052ba64bc5bf2
Reviewed-on: https://go-review.googlesource.com/33306
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-16 20:07:47 +00:00
Bryan C. Mills 1f605175b0 runtime/cgo: use libc for sigaction syscalls when possible
This ensures that runtime's signal handlers pass through the TSAN and
MSAN libc interceptors and subsequent calls to the intercepted
sigaction function from C will correctly see them.

Fixes #17753.

Change-Id: I9798bb50291a4b8fa20caa39c02a4465ec40bb8d
Reviewed-on: https://go-review.googlesource.com/33142
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-11-16 05:38:38 +00:00
Vladimir Stefanovic 272032d0b2 runtime: add support files for linux/mips{,le} port
Only exe buildmode without cgo supported.

Change-Id: Id104a79a99d3285c04db00fd98b8affa94ea3c37
Reviewed-on: https://go-review.googlesource.com/31487
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2016-11-15 21:49:01 +00:00
David Crawshaw 03da2690c9 cmd/link, runtime, plugin: versioning
In plugins and every program that opens a plugin, include a hash of
every imported package.

There are two versions of each hash: one local and one exported.
As the program starts and plugins are loaded, the first exported
symbol for each package becomes the canonical version.

Any subsequent plugin's local package hash symbol has to match the
canonical version.

Fixes #17832

Change-Id: I4e62c8e1729d322e14b1673bada40fa7a74ea8bc
Reviewed-on: https://go-review.googlesource.com/33161
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-15 16:14:27 +00:00
Ian Lance Taylor a145890059 all: don't call t.Fatal from a goroutine
Fixes #17900.

Change-Id: I42cda6ac9cf48ed739d3a015a90b3cb15edf8ddf
Reviewed-on: https://go-review.googlesource.com/33243
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-15 15:13:48 +00:00
Josh Bleecher Snyder 7c9f910607 all: fix vet nits
Fixes these vet complaints:

net/error_test.go:254: unrecognized printf flag for verb 'T': '#'
os/os_test.go:1067: arg mt for printf verb %d of wrong type: time.Time
runtime/debug/garbage_test.go:83: arg dt for printf verb %d of wrong type: time.Time

Change-Id: I0e986712a4b083b75fb111e687e424d06a85a47b
Reviewed-on: https://go-review.googlesource.com/33167
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-13 21:27:49 +00:00
Brad Fitzpatrick 37020dd510 runtime/internal/atomic: add TestUnaligned64
Add a variant of sync/atomic's TestUnaligned64 to
runtime/internal/atomic.

Skips the test on arm for now where it's currently failing.

Updates #17786

Change-Id: If63f9c1243e9db7b243a95205b2d27f7d1dc1e6e
Reviewed-on: https://go-review.googlesource.com/33159
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-12 06:38:40 +00:00
David du Colombier adb384ad2c net: implement asynchonous cancelable I/O on Plan 9
This change is an experimental implementation of asynchronous
cancelable I/O operations on Plan 9, which are required to
implement deadlines.

There are no asynchronous syscalls on Plan 9. I/O operations
are performed with blocking pread and pwrite syscalls.

Implementing deadlines in Go requires a way to interrupt
I/O operations.

It is possible to interrupt reads and writes on a TCP connection
by forcing the closure of the TCP connection. This approach
has been used successfully in CL 31390.

However, we can't implement deadlines with this method, since
we require to be able to reuse the connection after the timeout.

On Plan 9, I/O operations are interrupted when the process
receives a note. We can rely on this behavior to implement
a more generic approach.

When doing an I/O operation (read or write), we start the I/O in
its own process, then wait for the result asynchronously. The
process is able to handle the "hangup" note. When receiving the
"hangup" note, the currently running I/O operation is canceled
and the process returns.

This way, deadlines can be implemented by sending an "hangup"
note to the process running the blocking I/O operation, after
the expiration of a timer.

Fixes #11932.
Fixes #17498.

Change-Id: I414f72c7a9a4f9b8f9c09ed3b6c269f899d9b430
Reviewed-on: https://go-review.googlesource.com/31521
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-12 05:44:36 +00:00
Rhys Hiltner e0aedfb496 runtime: include pre-panic/throw logs in core dumps
When a Go program crashes with GOTRACEBACK=crash, the OS creates a
core dump. Include the text-formatted output of some of the cause of
that crash in the core dump.

Output printed by the runtime before crashing is maintained in a
circular buffer to allow access to messages that may be printed
immediately before calling runtime.throw.

The stack traces printed by the runtime as it crashes are not stored.
The information required to recreate them should be included in the
core file.

Updates #16893

There are no tests covering the generation of core dumps; this change
has not added any.

This adds (reentrant) locking to runtime.gwrite, which may have an
undesired performance impact.

Change-Id: Ia2463be3c12429354d290bdec5f3c8d565d1a2c3
Reviewed-on: https://go-review.googlesource.com/32013
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2016-11-11 21:29:53 +00:00
Russ Cox e6da64b6c0 runtime: fix Windows profiling crash
I don't have any way to test or reproduce this problem,
but the current code is clearly wrong for Windows.
Make it better.

As I said on #17165:

But the borrowing of M's and the profiling of M's by the CPU profiler
seem not synchronized enough. This code implements the CPU profiler
on Windows:

	func profileloop1(param uintptr) uint32 {
		stdcall2(_SetThreadPriority, currentThread, _THREAD_PRIORITY_HIGHEST)

		for {
			stdcall2(_WaitForSingleObject, profiletimer, _INFINITE)
			first := (*m)(atomic.Loadp(unsafe.Pointer(&allm)))
			for mp := first; mp != nil; mp = mp.alllink {
				thread := atomic.Loaduintptr(&mp.thread)
				// Do not profile threads blocked on Notes,
				// this includes idle worker threads,
				// idle timer thread, idle heap scavenger, etc.
				if thread == 0 || mp.profilehz == 0 || mp.blocked {
					continue
				}
				stdcall1(_SuspendThread, thread)
				if mp.profilehz != 0 && !mp.blocked {
					profilem(mp)
				}
				stdcall1(_ResumeThread, thread)
			}
		}
	}

	func profilem(mp *m) {
		var r *context
		rbuf := make([]byte, unsafe.Sizeof(*r)+15)

		tls := &mp.tls[0]
		gp := *((**g)(unsafe.Pointer(tls)))

		// align Context to 16 bytes
		r = (*context)(unsafe.Pointer((uintptr(unsafe.Pointer(&rbuf[15]))) &^ 15))
		r.contextflags = _CONTEXT_CONTROL
		stdcall2(_GetThreadContext, mp.thread, uintptr(unsafe.Pointer(r)))
		sigprof(r.ip(), r.sp(), 0, gp, mp)
	}

	func sigprof(pc, sp, lr uintptr, gp *g, mp *m) {
		if prof.hz == 0 {
			return
		}

		// Profiling runs concurrently with GC, so it must not allocate.
		mp.mallocing++

		... lots of code ...

		mp.mallocing--
	}

A borrowed M may migrate between threads. Between the
atomic.Loaduintptr(&mp.thread) and the SuspendThread, mp may have
moved to a new thread, so that it's in active use. In particular
it might be calling malloc, as in the crash stack trace. If so, the
mp.mallocing++ in sigprof would provoke the crash.

Those lines are trying to guard against allocation during sigprof.
But on Windows, mp is the thread being traced, not the current
thread. Those lines should really be using getg().m.mallocing, which
is the same on Unix but not on Windows. With that change, it's
possible the race on the actual thread is not a problem: the traceback
would get confused and eventually return an error, but that's fine.
The code expects that possibility.

Fixes #17165.

Change-Id: If6619731910d65ca4b1a6e7de761fa2518ef339e
Reviewed-on: https://go-review.googlesource.com/33132
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-11-11 20:50:08 +00:00