runtime: add GODEBUG=dontfreezetheworld=1

This GODEBUG flag disables the freezetheworld call during fatal panic.
freezetheworld asks the scheduler to stop running goroutines on all Ms.
This is normally useful, as it ensures we can collect a traceback from
every goroutine. However, it can be frustrating when debugging the
scheduler itself, as it significantly changes the scheduler state from
when the panic started.

Setting this flag has some disadvantages. Most notably, running
goroutines will not traceback in the standard output (though they may be
included in the final SIGQUIT loop). Additionally, we may missing
concurrently created goroutines when looping over allgs (CL 270861 made
this safe, but still racy). The final state of all goroutines will also
be further removed from the time of panic, as they continued to run for
a while.

One unfortunate part of this flag is the final SIGQUIT loop in the
runtime leaves every thread in the signal handler at exit. This is a bit
frustrating in gdb, which doesn't understand how to step beyond
sigtramp. The data is still there, but you must manually walk.

Change-Id: Ie6bd3ac521fcababea668196b60cf225a0be1a00
Reviewed-on: https://go-review.googlesource.com/c/go/+/478975
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>

src/runtime/extern.go

@@ -55,6 +55,15 @@ It is a comma-separated list of name=val pairs setting these named variables:
 	cgocheck mode can be enabled using GOEXPERIMENT (which
 	requires a rebuild), see https://pkg.go.dev/internal/goexperiment for details.
 
+	dontfreezetheworld: by default, the start of a fatal panic or throw
+	"freezes the world", stopping all goroutines, which makes it possible
+	to traceback all goroutines (running goroutines cannot be traced), and
+	keeps their state close to the point of panic. Setting
+	dontfreezetheworld=1 disables freeze, allowing goroutines to continue
+	executing during panic processing. This can be useful when debugging
+	the runtime scheduler, as freezetheworld perturbs scheduler state and
+	thus may hide problems.
+
 	efence: setting efence=1 causes the allocator to run in a mode
 	where each object is allocated on a unique page and addresses are
 	never recycled.

src/runtime/panic.go

@@ -1242,6 +1242,9 @@ func startpanic_m() bool {
 		if debug.schedtrace > 0 || debug.scheddetail > 0 {
 			schedtrace(true)
 		}
+		if debug.dontfreezetheworld > 0 {
+			return true
+		}
 		freezetheworld()
 		return true
 	case 1:

src/runtime/runtime1.go

@@ -309,6 +309,7 @@ type dbgVar struct {
 var debug struct {
 	cgocheck           int32
 	clobberfree        int32
+	dontfreezetheworld int32
 	efence             int32
 	gccheckmark        int32
 	gcpacertrace       int32
@@ -340,6 +341,7 @@ var dbgvars = []*dbgVar{
 	{name: "allocfreetrace", value: &debug.allocfreetrace},
 	{name: "clobberfree", value: &debug.clobberfree},
 	{name: "cgocheck", value: &debug.cgocheck},
+	{name: "dontfreezetheworld", value: &debug.dontfreezetheworld},
 	{name: "efence", value: &debug.efence},
 	{name: "gccheckmark", value: &debug.gccheckmark},
 	{name: "gcpacertrace", value: &debug.gcpacertrace},