Commit Graph

50596 Commits

Author SHA1 Message Date
Robert Griesemer 0b3dd75f6c go/types, types2: better error message when missing method is unexported
Change lookupMethod such that "foldCase" means "ignore case
and package" and analyze a lookup result further to determine
if a method name was not exported, and report a better error
message in that case.

Fixes #59831.

Change-Id: Ice6222e1fc00dba13caeda6c48971e8473d12da5
Reviewed-on: https://go-review.googlesource.com/c/go/+/549298
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
2023-12-13 17:54:56 +00:00
Mauri de Souza Meneguzzo f2d243db8f iter: fix race instrumentation for Pull2
Pull2 tests are failing with -race, giving false-positive race conditions
due to bad race instrumentation.

No tests for this as it should be caught by the race builders. The only
reason it was not caught is because it is behind GOEXPERIMENT=rangefunc.

Fixes #64651

Change-Id: I20554da930b0e19594e0e267f01a1e7a9cbc577a
GitHub-Last-Rev: 7c1f19238d
GitHub-Pull-Request: golang/go#64653
Reviewed-on: https://go-review.googlesource.com/c/go/+/548895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-12-13 16:20:09 +00:00
Robert Griesemer 58c28ba286 go/types, types2: avoid confusing error message "have m(T), want m(T)"
This is a partial fix for situations where a method lookup leads to
an error due to non-matching signatures, but where the signatures
print exactly the same. This can happen if both signatures contain
type parameters (after instantiation) and the type parameters have
the same name (such as "T").

For now, rather than printing a confusing error message in this
case, leave away the confusing part of the error message (at the
cost of providing slightly less information).

In the long run, we need to find a better solution for this problem;
but this seems better than what we had before.

For #61685.

Change-Id: I259183f08b9db400ffc8e1cf447967c640a0f444
Reviewed-on: https://go-review.googlesource.com/c/go/+/549296
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
2023-12-13 14:32:38 +00:00
Roland Shoemaker 400e24a8be crypto/x509: properly gate test on macos version
Fixes the gating of TestIssue51759 by shelling out to sw_vers to check
what version of macOS we are on.

Fixes #64677

Change-Id: I5eef4fa39e5449e7b2aa73864625c3abf002aef8
Reviewed-on: https://go-review.googlesource.com/c/go/+/549195
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-13 00:22:17 +00:00
Mauri de Souza Meneguzzo c2079deb24 cmd/cgo/internal/testsanitizers: check for go build and cgo in fuzzer and msan tests
Make sure the platform we are running the tests on can compile programs
and has cgo support in order to run the fuzzer and msan tests. This is the
same approach used by the asan tests, which share the same requirements.

Fixes #64626

Change-Id: I7c0b912dabdd1b7d7d44437e4ade5e5994994796
GitHub-Last-Rev: 9fae6970f0
GitHub-Pull-Request: golang/go#64640
Reviewed-on: https://go-review.googlesource.com/c/go/+/548715
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-12-12 19:42:27 +00:00
Robert Griesemer 0ac1e3b245 go/types, types2: flip message contents for reverse type inference errors
Add a new flag 'reverse' to control the formatting of type inference
error messages.

This change only impacts error messages.

Fixes #60747.

Change-Id: I81e13075e3157252ccc09f358bd29bd676c34499
Reviewed-on: https://go-review.googlesource.com/c/go/+/549055
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-12 18:44:59 +00:00
Robert Griesemer 2643a59172 go/types, types2: better error message for cases of reverse type inference
Introduce a new type "target" to pass around target types together
with a suitable description (typically a variable name) for a better
error message.

As a side effect, using a specific type (target), rather than just Type
avoids accidental confusion with other types.

Use the target type description for a better error message in some
cases.

The error message can be further improved by flipping the order of
the sentence (for another CL to keep this one small and simple).

Also, and unrelated to this fix, remove the first argument to errorf
in infer.go: the argument is always "type" (there's only one call).

For #60747.

Change-Id: I2118d0fe9e2b4aac959371941064e0e9ca7b3b6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/548995
Auto-Submit: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-12 18:44:28 +00:00
Bryan C. Mills e70d343306 cmd/go: explicitly name the default branch “main” in testdata/vcstest/git/issue57650.txt
Updates #47650.

Change-Id: I90c0a51b390326129643d5b90fe935d44e5e5372
Reviewed-on: https://go-review.googlesource.com/c/go/+/549155
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2023-12-12 16:13:43 +00:00
Roland Shoemaker b18b058816 crypto/x509: gate Policies marshaling with GODEBUG
Use a GODEBUG to choose which certificate policy field to use. If
x509usepolicies=1 is set, use the Policies field, otherwise use the
PolicyIdentifiers field.

Fixes #64248

Change-Id: I3f0b56102e0bac4ebe800497717c61c58ef3f092
Reviewed-on: https://go-review.googlesource.com/c/go/+/546916
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-11 23:29:45 +00:00
Bryan C. Mills f4ff0e2505 net: remove an arbitrary timeout in TestAcceptTimeoutMustReturn
This probably fixes the failure mode seen in
https://build.golang.org/log/e73acfd930cbe82302505cac0041d9883e2360c5.

If not, allowing the test to deadlock and dump goroutines
should produce better debugging information than the existing
"didn't return in an expected time" failure message.

For #58901.

Change-Id: Ie0bb1887a4329d2f6b0e7348a4820af71385494e
Reviewed-on: https://go-review.googlesource.com/c/go/+/548881
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-12-11 23:01:52 +00:00
Bryan C. Mills a839ec1e73 cmd/go/internal/modload: classify "invalid pseudo-version" errors in Query
If we encounter an unclassified error in modload.Query, it takes
precedence even if the package is found in some other module.
(That is intentional, so that if a package exists in both a parent
and a nested module the outcome is deterministic, and does not shift
if a temporary error causes one of the modules to be unavailable.)

A pseudo-version is formed from a base version and a commit hash.
Each version tag is specific to the module in a particular directory
of the repo (often the root directory), whereas the commit hash is
the same for all subdirectories. When we go to check a particular
subdirectory for the requested package, we may find that that version
is not valid for that combination of <subdirectory, commit hash>,
but we should keep looking to see whether it is valid for a module
in some other subdirectory.

Fixes #47650.

Change-Id: Id48f590ce906a3d4cf4e82fc66137bf67613277d
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/548475
Reviewed-by: Michael Matloob <matloob@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-12-11 22:29:11 +00:00
Tolya Korniltsev 20a03fc713 runtime/pprof: fix generics function names
profileBuilder is using Frame->Function as key for checking if we already
emitted a function. However for generics functions it has dots there [...],
so sometimes for different functions with different generics types,
the profileBuilder emits wrong functions.

Fixes #64528

Change-Id: I8b39245e0b18f4288ce758c912c6748f87cba39a
Reviewed-on: https://go-review.googlesource.com/c/go/+/546815
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-12-11 17:49:10 +00:00
Keith Randall 052d402985 runtime: clear mspan.largeType more carefully in the case of arenas
The pointer stored in mspan.largeType is an invalid pointer when
the span is an arena. We need to make sure that pointer isn't seen
by the garbage collector, as it might barf on it. Make sure we
zero the pointer using a uintptr write so the old value isn't picked
up by the write barrier.

The mspan.largeType field itself is in a NotInHeap struct, so a heap
scan won't find it. The only way we find it is when writing it, or
when reading it and putting it in a GC-reachable location. I think we
might need to audit the runtime to make sure these pointers aren't
being passed in places where the GC might (non-conservatively) scan a
stack frame it lives in. (It might be ok, many such places are either
systemstack or nosplit.)

Change-Id: Ie059d054e0da4d48a4c4b3be88b8e1e46ffa7d10
Reviewed-on: https://go-review.googlesource.com/c/go/+/548535
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-11 17:15:22 +00:00
Russ Cox ba7c4e47b3 os: limit temp file randomness to uint32
CL 516860 accidentally changed the randomness
used in TempFile from 32 to 64 bits on 64-bit platforms,
meaning from 10 to 20 decimal bytes.
This is enough to cause problems in a few tests
because it makes temporary directory names just
a little bit longer.

Limit back down to 32 bits of randomness, which is fine,
and add a test to avoid repeating the mistake.

Fixes #64605.

Change-Id: I17b8c063d11d5c0a96a68b5e5f83c889a13bca77
Reviewed-on: https://go-review.googlesource.com/c/go/+/548635
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2023-12-11 15:36:54 +00:00
Robert Griesemer e1cba47ee0 go/parser: fix panic in object resolution for invalid type parameter list
This change restores the original logic in parseParameterList to what
it was before CL 538858 (which caused the issue), not in exact wording
but in identical semantic meaning, and thus restores this function to
a state that we know was working fine.

However, the change keeps the improved error reporting introduced by
CL 538858. To keep the code changes somewhat minimal as we are close
to RC1, the improved error handling exists twice for now even though
it could be factored out.

Fixes #64534.

Change-Id: I0b7bbf74d28811e8aae74f838f2d424f78af1f38
Reviewed-on: https://go-review.googlesource.com/c/go/+/548395
Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-08 20:07:50 +00:00
Bryan C. Mills 6cdf2ccae8 cmd/go: relax version regexp from CL 547998
In CL 547998 I relaxed cmd/go's parsing of version lines to allow it
to recognize clang versions with vendor prefixes. To prevent false-positives,
I added a check for a version 3-tuple following the word "version".
However, it appears that some releases of GCC use only a 2-tuple instead.

Updates #64423.
Fixes #64619.

Change-Id: I5f1d0881b6295544a46ab958c6ad4c2155cf51fe
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/548120
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-08 18:34:24 +00:00
eric fang 78b42a5338 cmd/internal/obj/arm64: fix invalid register pair for LDP
ZR register can be used in register pair of LDP, LDPW and LDPSW
instructions, but now it's not allowed. This CL fixes this issue.

Change-Id: I8467502de4664214e0b7dad0295c44f6cff16ee6
Reviewed-on: https://go-review.googlesource.com/c/go/+/547815
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Eric Fang <eric.fang@arm.com>
2023-12-08 03:28:17 +00:00
Raghvender 4bf1ca4b0c cmd/compile: fix error message for mismatch between the number of type params and arguments
Fixes #64276

Change-Id: Ib6651669904e6ea0daf275d85d8bd008b8b21cc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/544018
Reviewed-by: raghvender sundarjee <raghvenders@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-08 00:50:55 +00:00
Keith Randall 788a227759 cmd/compile: fix SCCP propagation into jump tables
We can't delete all the outgoing edges and then add one back in, because
then we've lost the argument of any phi at the target. Instead, move
the important target to the front of the list and delete the rest.

This normally isn't a problem, because there is never normally a phi
at the target of a jump table. But this isn't quite true when in race
build mode, because there is a phi of the result of a bunch of raceread
calls.

The reason this happens is that each case is written like this (where e
is the runtime.eface we're switching on):

if e.type == $type.int32 {
   m = raceread(e.data, m1)
}
m2 = phi(m1, m)
if e.type == $type.int32 {
   .. do case ..
   goto blah
}

so that if e.type is not $type.int32, it falls through to the default
case. This default case will have a memory phi for all the (jumped around
and not actually called) raceread calls.

If we instead did it like

if e.type == $type.int32 {
  raceread(e.data)
  .. do case ..
  goto blah
}

That would paper over this bug, as it is the only way to construct
a jump table whose target is a block with a phi in it. (Yet.)

But we'll fix the underlying bug in this CL. Maybe we can do the
rewrite mentioned above later.  (It is an optimization for -race mode,
which isn't particularly important.)

Fixes #64606

Change-Id: I6f6e3c90eb1e2638112920ee2e5b6581cef04ea4
Reviewed-on: https://go-review.googlesource.com/c/go/+/548356
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-08 00:29:01 +00:00
Russ Cox dca2ef2361 runtime: put runtime.fastrand back temporarily
Callers should be using math/rand/v2.Uint64 instead,
but there are lots of linkname references to runtime.fastrand
in public code. If we break it all now, that will require people
to use //go:build tags to use rand/v2.Uint64 with Go 1.22
and keep using the linkname for earlier versions.
Instead, leave the linkname working and then we can remove
it in Go 1.24, at which point everyone should be able to use
math/rand/v2.Uint64 unconditionally.

Change-Id: I7287ca4f67c270b009562313661cc28a4c2219a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/548235
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2023-12-07 23:44:31 +00:00
Michael Pratt e1c0349a7c internal/profile: fully decode proto even if there are no samples
This is a partial revert of CL 483137.

CL 483137 started checking errors in postDecode, which is good. Now we
can catch more malformed pprof protos. However this made
TestEmptyProfile fail, so an early return was added when the profile was
"empty" (no samples).

Unfortunately, this was problematic. Profiles with no samples can still
be valid, but skipping postDecode meant that the resulting Profile was
missing values from the string table. In particular, net/http/pprof
needs to parse empty profiles in order to pass through the sample and
period types to a final output proto. CL 483137 broke this behavior.

internal/profile.Parse is only used in two places: in cmd/compile to
parse PGO pprof profiles, and in net/http/pprof to parse before/after
pprof profiles for delta profiles. In both cases, the input is never
literally empty (0 bytes). Even a pprof proto with no samples still
contains some header fields, such as sample and period type. Upstream
github.com/google/pprof/profile even has an explicit error on 0 byte
input, so `go tool pprof` will not support such an input.

Thus TestEmptyProfile was misleading; this profile doesn't need to
support empty input at all.

Resolve this by removing TestEmptyProfile and replacing it with an
explicit error on empty input, as upstream
github.com/google/pprof/profile has. For non-empty input, always run
postDecode to ensure the string table is processed.

TestConvertCPUProfileEmpty is reverted back to assert the values from
before CL 483137. Note that in this case "Empty" means no samples, not a
0 byte input.

Continue to allow empty files for PGO in order to minimize the chance of
last minute breakage if some users have empty files.

Fixes #64566.

Change-Id: I83a1f0200ae225ac6da0009d4b2431fe215b283f
Reviewed-on: https://go-review.googlesource.com/c/go/+/547996
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-12-07 19:52:28 +00:00
Bryan C. Mills c71eedf90a cmd/go: accept clang versions with vendor prefixes
To better diagnose bugs like this one in the future, I think
we should also refuse to use a C compiler if we can't identify
a sensible version for it. I did not do that in this CL because
I want it to be small and low-risk for possible backporting.

Fixes #64423.

Change-Id: I21e44fc55f6fcf76633e4fecf6400c226a742351
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/547998
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2023-12-07 19:13:29 +00:00
Bryan C. Mills 77e76c4387 cmd/go: unset CC when we remove it from PATH in TestScript/cgo_stale_precompiled
Otherwise, if make.bash produced a relative default CC path but the
user has an absolute path to CC set in their environment, the test
will fail spuriously.

For #64423.

Change-Id: I0f3e1d04851585e1b39266badcda9f17489332d9
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/547997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-12-07 19:13:27 +00:00
Ruinan b11d19e03b cmd/asm: print absolute PC for all patterns "off(PC)" in testEndToEnd
Before this CL, testEndToEnd only turns the relative PC to absolute PC
when pattern "off(PC)" is the suffix of an instruction. But there are
some instructions like:

  ADR 10(PC), R10

it's also acceptable for the assembler while the pattern "off(PC)" is
not a suffix, which makes the test fail.

This CL fixes this issue by searching the pattern in the whole string
instead of only in the suffix.

Change-Id: I0cffedeb7b3c63abca7697671088cf993aff71ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/547235
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ruinan Sun <Ruinan.Sun@arm.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2023-12-07 18:42:59 +00:00
Bryan C. Mills bb34112d4d os: document Readlink behavior for relative links
Also provide a runnable example to illustrate that behavior.

This should help users to avoid the common mistake of expecting
os.Readlink to return an absolute path.

Fixes #57766.

Change-Id: I8f60aa111ebda0cae985758615019aaf26d5cb41
Reviewed-on: https://go-review.googlesource.com/c/go/+/546995
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2023-12-07 17:27:54 +00:00
Bryan C. Mills 2e4d9a1801 cmd/go: set global .gitconfig location more robustly in tests
Also confirm that setting the location actually worked before
proceeding with the rest of the test.

This fixes a test failure with git versions older than 2.32.0.

Updates #53955.
Fixes #64603.

Change-Id: I1a954975a3d8300e8b4dca045d3a15438a0407ec
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/548215
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-12-07 16:37:00 +00:00
Robert Griesemer 5e724ccb2b go/types, types2: don't print aliased type (in comments) for Alias types
Fixes #64584.

Change-Id: I756d6026d10f130c1dd5026891b7cc3c640d7f78
Reviewed-on: https://go-review.googlesource.com/c/go/+/548135
Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
2023-12-07 01:05:50 +00:00
Bryan C. Mills 0ff2b33ea0 cmd/go: avoid hard-coding runtime dependencies in TestNewReleaseRebuildsStalePackagesInGOPATH
Fixes #64583.

Change-Id: Idbe50fe77f6030f2402ac436abe6710cb1183730
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/547995
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-06 22:04:54 +00:00
Michael Pratt e914671f5d runtime/pprof: add section headers to Profile doc
Adding explicit section headers makes it cleaner to split the profile
descriptions into multiple paragraphs, as there is now an explicit
transition from discussion of one profile type to the next.

For #14689.

Change-Id: Ifcff918367e91a165ee5f74423be3935b421972b
Reviewed-on: https://go-review.googlesource.com/c/go/+/547955
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
2023-12-06 21:50:36 +00:00
Michael Pratt e9eb2beeb2 runtime/pprof: document block and mutex profiles
Amazingly, we seem to have nearly no in-tree documentation on the
semantics of block and mutex profiles. Add brief summaries, including
the new behavior from CL 506415 and CL 544195.

For #14689.
For #44920.
For #57071.
For #61015.

Change-Id: I1a6edce7c434fcb43f17c83eb362b1f9d1a32df1
Reviewed-on: https://go-review.googlesource.com/c/go/+/547057
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-12-06 21:50:25 +00:00
lotusirous 71fc9d4da5 gover: support Semantic Versioning major versions beyond 1
For #64033

Change-Id: Iab132f86c66aa6115a349d8032e9766a14dad02e
Reviewed-on: https://go-review.googlesource.com/c/go/+/541915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-12-06 21:39:46 +00:00
Russ Cox a9c9cc07ac iter, runtime: add coroutine support
The exported API is only available with GOEXPERIMENT=rangefunc.
This will let Go 1.22 users who want to experiment with rangefuncs
access an efficient implementation of iter.Pull and iter.Pull2.

For #61897.

Change-Id: I6ef5fa8f117567efe4029b7b8b0f4d9b85697fb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/543319
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-06 21:33:59 +00:00
Bryan C. Mills 7b5a3733fc cmd/go/internal/modfetch/codehost: add new git tags before statLocal instead of after
gitRepo.statLocal reports tag and version information.
If we are statting a hash that corresponds to a tag, we need to add that tag
before calling statLocal so that it can be included in that information.

Fixes #53955.
Updates #56881.

Change-Id: I69a71428e6ed9096d4cb8ed1bb79531415ff06c1
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/547155
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2023-12-06 19:29:47 +00:00
Michael Pratt 3f2bf706f7 runtime/metrics: document runtime-internal locks in /sync/mutex/wait/total:seconds
For #57071.

Change-Id: I7ce6c35bed95a6ea3cdc17007f861c5dd82404d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/547056
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-06 17:59:12 +00:00
Michael Pratt 98fd8f5768 runtime: rename GODEBUG=profileruntimelocks to runtimecontentionstacks
profileruntimelocks is new in CL 544195, but the name is deceptive. Even
with profileruntimelocks=0, runtime-internal locks are still profiled.
The actual difference is that call stacks are not collected. Instead all
contention is reported at runtime._LostContendedLock.

Rename this setting to runtimecontentionstacks to make its name more
aligned with its behavior.

In addition, for this release the default is profileruntimelocks=0,
meaning that users are fairly likely to encounter
runtime._LostContendedLock. Rename it to
runtime._LostContendedRuntimeLock in an attempt to make it more
intuitive that these are runtime locks, not locks in application code.

For #57071.

Change-Id: I38aac28b2c0852db643d53b1eab3f3bc42a43393
Reviewed-on: https://go-review.googlesource.com/c/go/+/547055
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
2023-12-06 17:57:59 +00:00
Than McIntosh 6e33a6376e cmd/go: use local go cache for cover_statements script test
Use a test-local directory for GOCACHE in "cover_statements" script
test, as a workaround for issue 64014.

For the portion of this test that verifies that caching works
correctly, the cache should theoretically always behave
reliably/deterministically, however if other tests are concurrently
accessing the cache while this test is running, it can lead to cache
lookup failures, which manifest as a flaky failure. To avoid such
flakes, use a separate isolated GOCACHE for this test.

For #64014.

Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Change-Id: Ia66798215a75b7c41188ed15920c17b73f40152a
Reviewed-on: https://go-review.googlesource.com/c/go/+/545235
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-12-06 17:36:30 +00:00
Roland Shoemaker 5214949d41 crypto/x509: revert Policies marshaling behavior
Don't marshal Policies field.

Updates #64248

Change-Id: I7e6d8b9ff1b3698bb4f585fa82fc4050eff3ae4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/546915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-12-06 17:10:44 +00:00
Matthew Dempsky 34416d7f6f cmd/compile: fix escape analysis of string min/max
When I was plumbing min/max support through the compiler, I was
thinking mostly about numeric argument types. As a result, I forgot
that escape analysis would need to be aware that min/max can operate
on string values, which contain pointers.

Fixes #64565.

Change-Id: I36127ce5a2da942401910fa0f9de922726c9f94d
Reviewed-on: https://go-review.googlesource.com/c/go/+/547715
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 22:06:07 +00:00
Russ Cox c29444ef39 math/rand, math/rand/v2: use ChaCha8 for global rand
Move ChaCha8 code into internal/chacha8rand and use it to implement
runtime.rand, which is used for the unseeded global source for
both math/rand and math/rand/v2. This also affects the calculation of
the start point for iteration over very very large maps (when the
32-bit fastrand is not big enough).

The benefit is that misuse of the global random number generators
in math/rand and math/rand/v2 in contexts where non-predictable
randomness is important for security reasons is no longer a
security problem, removing a common mistake among programmers
who are unaware of the different kinds of randomness.

The cost is an extra 304 bytes per thread stored in the m struct
plus 2-3ns more per random uint64 due to the more sophisticated
algorithm. Using PCG looks like it would cost about the same,
although I haven't benchmarked that.

Before this, the math/rand and math/rand/v2 global generator
was wyrand (https://github.com/wangyi-fudan/wyhash).
For math/rand, using wyrand instead of the Mitchell/Reeds/Thompson
ALFG was justifiable, since the latter was not any better.
But for math/rand/v2, the global generator really should be
at least as good as one of the well-studied, specific algorithms
provided directly by the package, and it's not.

(Wyrand is still reasonable for scheduling and cache decisions.)

Good randomness does have a cost: about twice wyrand.

Also rationalize the various runtime rand references.

goos: linux
goarch: amd64
pkg: math/rand/v2
cpu: AMD Ryzen 9 7950X 16-Core Processor
                        │ bbb48afeb7.amd64 │           5cf807d1ea.amd64           │
                        │      sec/op      │    sec/op     vs base                │
ChaCha8-32                     1.862n ± 2%    1.861n ± 2%        ~ (p=0.825 n=20)
PCG_DXSM-32                    1.471n ± 1%    1.460n ± 2%        ~ (p=0.153 n=20)
SourceUint64-32                1.636n ± 2%    1.582n ± 1%   -3.30% (p=0.000 n=20)
GlobalInt64-32                 2.087n ± 1%    3.663n ± 1%  +75.54% (p=0.000 n=20)
GlobalInt64Parallel-32        0.1042n ± 1%   0.2026n ± 1%  +94.48% (p=0.000 n=20)
GlobalUint64-32                2.263n ± 2%    3.724n ± 1%  +64.57% (p=0.000 n=20)
GlobalUint64Parallel-32       0.1019n ± 1%   0.1973n ± 1%  +93.67% (p=0.000 n=20)
Int64-32                       1.771n ± 1%    1.774n ± 1%        ~ (p=0.449 n=20)
Uint64-32                      1.863n ± 2%    1.866n ± 1%        ~ (p=0.364 n=20)
GlobalIntN1000-32              3.134n ± 3%    4.730n ± 2%  +50.95% (p=0.000 n=20)
IntN1000-32                    2.489n ± 1%    2.489n ± 1%        ~ (p=0.683 n=20)
Int64N1000-32                  2.521n ± 1%    2.516n ± 1%        ~ (p=0.394 n=20)
Int64N1e8-32                   2.479n ± 1%    2.478n ± 2%        ~ (p=0.743 n=20)
Int64N1e9-32                   2.530n ± 2%    2.514n ± 2%        ~ (p=0.193 n=20)
Int64N2e9-32                   2.501n ± 1%    2.494n ± 1%        ~ (p=0.616 n=20)
Int64N1e18-32                  3.227n ± 1%    3.205n ± 1%        ~ (p=0.101 n=20)
Int64N2e18-32                  3.647n ± 1%    3.599n ± 1%        ~ (p=0.019 n=20)
Int64N4e18-32                  5.135n ± 1%    5.069n ± 2%        ~ (p=0.034 n=20)
Int32N1000-32                  2.657n ± 1%    2.637n ± 1%        ~ (p=0.180 n=20)
Int32N1e8-32                   2.636n ± 1%    2.636n ± 1%        ~ (p=0.763 n=20)
Int32N1e9-32                   2.660n ± 2%    2.638n ± 1%        ~ (p=0.358 n=20)
Int32N2e9-32                   2.662n ± 2%    2.618n ± 2%        ~ (p=0.064 n=20)
Float32-32                     2.272n ± 2%    2.239n ± 2%        ~ (p=0.194 n=20)
Float64-32                     2.272n ± 1%    2.286n ± 2%        ~ (p=0.763 n=20)
ExpFloat64-32                  3.762n ± 1%    3.744n ± 1%        ~ (p=0.171 n=20)
NormFloat64-32                 3.706n ± 1%    3.655n ± 2%        ~ (p=0.066 n=20)
Perm3-32                       32.93n ± 3%    34.62n ± 1%   +5.13% (p=0.000 n=20)
Perm30-32                      202.9n ± 1%    204.0n ± 1%        ~ (p=0.482 n=20)
Perm30ViaShuffle-32            115.0n ± 1%    114.9n ± 1%        ~ (p=0.358 n=20)
ShuffleOverhead-32             112.8n ± 1%    112.7n ± 1%        ~ (p=0.692 n=20)
Concurrent-32                  2.107n ± 0%    3.725n ± 1%  +76.75% (p=0.000 n=20)

goos: darwin
goarch: arm64
pkg: math/rand/v2
                       │ bbb48afeb7.arm64 │           5cf807d1ea.arm64            │
                       │      sec/op      │    sec/op     vs base                 │
ChaCha8-8                     2.480n ± 0%    2.429n ± 0%    -2.04% (p=0.000 n=20)
PCG_DXSM-8                    2.531n ± 0%    2.530n ± 0%         ~ (p=0.877 n=20)
SourceUint64-8                2.534n ± 0%    2.533n ± 0%         ~ (p=0.732 n=20)
GlobalInt64-8                 2.172n ± 1%    4.794n ± 0%  +120.67% (p=0.000 n=20)
GlobalInt64Parallel-8        0.4320n ± 0%   0.9605n ± 0%  +122.32% (p=0.000 n=20)
GlobalUint64-8                2.182n ± 0%    4.770n ± 0%  +118.58% (p=0.000 n=20)
GlobalUint64Parallel-8       0.4307n ± 0%   0.9583n ± 0%  +122.51% (p=0.000 n=20)
Int64-8                       4.107n ± 0%    4.104n ± 0%         ~ (p=0.416 n=20)
Uint64-8                      4.080n ± 0%    4.080n ± 0%         ~ (p=0.052 n=20)
GlobalIntN1000-8              2.814n ± 2%    5.643n ± 0%  +100.50% (p=0.000 n=20)
IntN1000-8                    4.141n ± 0%    4.139n ± 0%         ~ (p=0.140 n=20)
Int64N1000-8                  4.140n ± 0%    4.140n ± 0%         ~ (p=0.313 n=20)
Int64N1e8-8                   4.140n ± 0%    4.139n ± 0%         ~ (p=0.103 n=20)
Int64N1e9-8                   4.139n ± 0%    4.140n ± 0%         ~ (p=0.761 n=20)
Int64N2e9-8                   4.140n ± 0%    4.140n ± 0%         ~ (p=0.636 n=20)
Int64N1e18-8                  5.266n ± 0%    5.326n ± 1%    +1.14% (p=0.001 n=20)
Int64N2e18-8                  6.052n ± 0%    6.167n ± 0%    +1.90% (p=0.000 n=20)
Int64N4e18-8                  8.826n ± 0%    9.051n ± 0%    +2.55% (p=0.000 n=20)
Int32N1000-8                  4.127n ± 0%    4.132n ± 0%    +0.12% (p=0.000 n=20)
Int32N1e8-8                   4.126n ± 0%    4.131n ± 0%    +0.12% (p=0.000 n=20)
Int32N1e9-8                   4.127n ± 0%    4.132n ± 0%    +0.12% (p=0.000 n=20)
Int32N2e9-8                   4.132n ± 0%    4.131n ± 0%         ~ (p=0.017 n=20)
Float32-8                     4.109n ± 0%    4.105n ± 0%         ~ (p=0.379 n=20)
Float64-8                     4.107n ± 0%    4.106n ± 0%         ~ (p=0.867 n=20)
ExpFloat64-8                  5.339n ± 0%    5.383n ± 0%    +0.82% (p=0.000 n=20)
NormFloat64-8                 5.735n ± 0%    5.737n ± 1%         ~ (p=0.856 n=20)
Perm3-8                       26.65n ± 0%    26.80n ± 1%    +0.58% (p=0.000 n=20)
Perm30-8                      194.8n ± 1%    197.0n ± 0%    +1.18% (p=0.000 n=20)
Perm30ViaShuffle-8            156.6n ± 0%    157.6n ± 1%    +0.61% (p=0.000 n=20)
ShuffleOverhead-8             124.9n ± 0%    125.5n ± 0%    +0.52% (p=0.000 n=20)
Concurrent-8                  2.434n ± 3%    5.066n ± 0%  +108.09% (p=0.000 n=20)

goos: linux
goarch: 386
pkg: math/rand/v2
cpu: AMD Ryzen 9 7950X 16-Core Processor
                        │ bbb48afeb7.386 │            5cf807d1ea.386             │
                        │     sec/op     │    sec/op     vs base                 │
ChaCha8-32                  11.295n ± 1%    4.748n ± 2%   -57.96% (p=0.000 n=20)
PCG_DXSM-32                  7.693n ± 1%    7.738n ± 2%         ~ (p=0.542 n=20)
SourceUint64-32              7.658n ± 2%    7.622n ± 2%         ~ (p=0.344 n=20)
GlobalInt64-32               3.473n ± 2%    7.526n ± 2%  +116.73% (p=0.000 n=20)
GlobalInt64Parallel-32      0.3198n ± 0%   0.5444n ± 0%   +70.22% (p=0.000 n=20)
GlobalUint64-32              3.612n ± 0%    7.575n ± 1%  +109.69% (p=0.000 n=20)
GlobalUint64Parallel-32     0.3168n ± 0%   0.5403n ± 0%   +70.51% (p=0.000 n=20)
Int64-32                     7.673n ± 2%    7.789n ± 1%         ~ (p=0.122 n=20)
Uint64-32                    7.773n ± 1%    7.827n ± 2%         ~ (p=0.920 n=20)
GlobalIntN1000-32            6.268n ± 1%    9.581n ± 1%   +52.87% (p=0.000 n=20)
IntN1000-32                  10.33n ± 2%    10.45n ± 1%         ~ (p=0.233 n=20)
Int64N1000-32                10.98n ± 2%    11.01n ± 1%         ~ (p=0.401 n=20)
Int64N1e8-32                 11.19n ± 2%    10.97n ± 1%         ~ (p=0.033 n=20)
Int64N1e9-32                 11.06n ± 1%    11.08n ± 1%         ~ (p=0.498 n=20)
Int64N2e9-32                 11.10n ± 1%    11.01n ± 2%         ~ (p=0.995 n=20)
Int64N1e18-32                15.23n ± 2%    15.04n ± 1%         ~ (p=0.973 n=20)
Int64N2e18-32                15.89n ± 1%    15.85n ± 1%         ~ (p=0.409 n=20)
Int64N4e18-32                18.96n ± 2%    19.34n ± 2%         ~ (p=0.048 n=20)
Int32N1000-32                10.46n ± 2%    10.44n ± 2%         ~ (p=0.480 n=20)
Int32N1e8-32                 10.46n ± 2%    10.49n ± 2%         ~ (p=0.951 n=20)
Int32N1e9-32                 10.28n ± 2%    10.26n ± 1%         ~ (p=0.431 n=20)
Int32N2e9-32                 10.50n ± 2%    10.44n ± 2%         ~ (p=0.249 n=20)
Float32-32                   13.80n ± 2%    13.80n ± 2%         ~ (p=0.751 n=20)
Float64-32                   23.55n ± 2%    23.87n ± 0%         ~ (p=0.408 n=20)
ExpFloat64-32                15.36n ± 1%    15.29n ± 2%         ~ (p=0.316 n=20)
NormFloat64-32               13.57n ± 1%    13.79n ± 1%    +1.66% (p=0.005 n=20)
Perm3-32                     45.70n ± 2%    46.99n ± 2%    +2.81% (p=0.001 n=20)
Perm30-32                    399.0n ± 1%    403.8n ± 1%    +1.19% (p=0.006 n=20)
Perm30ViaShuffle-32          349.0n ± 1%    350.4n ± 1%         ~ (p=0.909 n=20)
ShuffleOverhead-32           322.3n ± 1%    323.8n ± 1%         ~ (p=0.410 n=20)
Concurrent-32                3.331n ± 1%    7.312n ± 1%  +119.50% (p=0.000 n=20)

For #61716.

Change-Id: Ibdddeed85c34d9ae397289dc899e04d4845f9ed2
Reviewed-on: https://go-review.googlesource.com/c/go/+/516860
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 20:34:30 +00:00
Russ Cox d92434935f math/rand/v2: add ChaCha8
This is a replay of CL 516859, after its rollback in CL 543895,
with big-endian systems fixed and the tests disabled on RISC-V
since the compiler is broken there (#64285).

ChaCha8 provides a cryptographically strong generator
alongside PCG, so that people who want stronger randomness
have access to that. On systems with 128-bit vector math
assembly (amd64 and arm64), ChaCha8 runs at about the same
speed as PCG (25% slower on amd64, 2% faster on arm64).

Fixes #64284.

Change-Id: I6290bb8ace28e1aff9a61f805dbe380ccdf25b94
Reviewed-on: https://go-review.googlesource.com/c/go/+/546020
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 20:32:54 +00:00
Michael Anthony Knyszek 6d7b3c8cd1 internal/trace/v2: handle thread ID reuse correctly
To begin with, CL 545515 made the trace parser tolerant of
GoCreateSyscall having a P, but that was wrong. Because dropm trashes
the M's syscalltick, that case should never be possible. So the first
thing this change does is it rewrites the test that CL introduced to
expect a failure instead of a success.

What I'd misinterpreted as a case that should be allowed was actually
the same as the other issues causing #64060, which is that the parser
doesn't correctly implement what happens to Ps when a thread calls back
into Go on non-pthread platforms, and what happens when a thread dies
on pthread platorms (or more succinctly, what the runtime does when it
calls dropm).

Specifically, the GoDestroySyscall event implies that if any P is still
running on that M when it's called, that the P stops running. This is
what is intended by the runtime trashing the M's syscalltick; when it
calls back into Go, the tracer models that thread as obtaining a new P
from scratch.

Handling this incorrectly manifests in one of two ways.

On pthread platforms, GoDestroySyscall is only emitted when a C thread
that previously called into Go is destroyed. However, that thread ID can
be reused. Because we have no thread events, whether it's the same
thread or not is totally ambiguous to the tracer. Therefore, the tracer
may observe a thread that previously died try to start running with a
new P under the same identity. The association to the old P is still
intact because the ID is the same, and the tracer gets confused -- it
appears as if two Ps are running on the same M!

On non-pthread platforms, GoDestroySyscall is emitted on every return to
C from Go code. In this case, the same thread with the same identity is
naturally going to keep calling back into Go. But again, since the
runtime trashes syscalltick in dropm, it's always going to acquire a P
from the tracer's perspective. But if this is a different P than before,
just like the pthread case, the parser is going to get confused, since
it looks like two Ps are running on the same M!

The case that CL 545515 actually handled was actually the non-pthread
case, specifically where the same P is reacquired by an M calling back
into Go. In this case, if we tolerate having a P, then what we'll
observe is the M stealing its own P from itself, then running with it.

Now that we know what the problem is, how do we fix it? This change
addresses the problem by emitting an extra event when encountering a
GoDestroySyscall with an active P in its context. In this case, it emits
an additional ProcSteal event to steal from itself, indicating that the
P stopped running. This removes any association between that M and that
P, resolving any ambiguity in the tracer.

There's one other minor detail that needs to be worked out, and that's
what happens to any *real* ProcSteal event that stole the P we're now
emitting an extra ProcSteal event for. Since, this event is going to
look for an M that may have moved on already and the P at this point is
already idle. Luckily, we have *exactly* the right fix for this. The
handler for GoDestroySyscall now moves any active P it has to the
ProcSyscallAbandoned state, indicating that we've lost information about
the P and that it should be treated as already idle. Conceptually this
all makes sense: this is a P in _Psyscall that has been abandoned by the
M it was previously bound to.

It's unfortunate how complicated this has all ended up being, but we can
take a closer look at that in the future.

Fixes #64060.

Change-Id: Ie9e6eb9cf738607617446e3487392643656069a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/546096
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-12-05 20:17:44 +00:00
Michael Anthony Knyszek c915215af6 runtime: add the disablethp GODEBUG setting
Go 1.21.1 and Go 1.22 have ceased working around an issue with Linux
kernel defaults for transparent huge pages that can result in excessive
memory overheads. (https://bugzilla.kernel.org/show_bug.cgi?id=93111)

Many Linux distributions disable huge pages altogether these days, so
this problem isn't quite as far-reaching as it used to be. Also, the
problem only affects Go programs with very particular memory usage
patterns.

That being said, because the runtime used to actively deal with this
problem (but with some unpredictable behavior), it's preventing users
that don't have a lot of control over their execution environment from
upgrading to Go beyond Go 1.20.

This change adds a GODEBUG to smooth over the transition. The GODEBUG
setting disables transparent huge pages for all heap memory on Linux,
which is much more predictable than restoring the old behavior.

Fixes #64332.

Change-Id: I73b1894337f0f0b1a5a17b90da1221e118e0b145
Reviewed-on: https://go-review.googlesource.com/c/go/+/547475
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 20:14:52 +00:00
Jes Cok af5d544b6d log/slog: correct doc examples for Handler.WithGroup
Fixes #64519

Change-Id: Iaea5de74402b77eb5a42979bd3baa1276b13d585
GitHub-Last-Rev: 5080a257b2
GitHub-Pull-Request: golang/go#64551
Reviewed-on: https://go-review.googlesource.com/c/go/+/547215
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Run-TryBot: Jes Cok <xigua67damn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-12-05 18:18:13 +00:00
Damien Neil 285ef16b38 net/http: limit chunked data overhead
The chunked transfer encoding adds some overhead to
the content transferred. When writing one byte per
chunk, for example, there are five bytes of overhead
per byte of data transferred: "1\r\nX\r\n" to send "X".

Chunks may include "chunk extensions",
which we skip over and do not use.
For example: "1;chunk extension here\r\nX\r\n".

A malicious sender can use chunk extensions to add
about 4k of overhead per byte of data.
(The maximum chunk header line size we will accept.)

Track the amount of overhead read in chunked data,
and produce an error if it seems excessive.

Fixes #64433
Fixes CVE-2023-39326

Change-Id: I40f8d70eb6f9575fb43f506eb19132ccedafcf39
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2076135
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/547335
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 17:17:59 +00:00
Keith Randall 16d3040a84 maps: fix aliasing problems with Clone
Make sure to alloc+copy large keys and values instead of aliasing them,
when they might be updated by a future assignment.

Fixes #64474

Change-Id: Ie2226a81cf3897e4e2ee24472f2966d397ace53f
Reviewed-on: https://go-review.googlesource.com/c/go/+/546515
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Mauri de Souza Meneguzzo <mauri870@gmail.com>
2023-12-05 17:05:56 +00:00
Michael Pratt 098f059d07 runtime: require cgo for cgo gdb test
Fixes #64542.

Change-Id: Ic0dc37dfefac83fd3aab6dacfec58b8ee9105fbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/547135
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Commit-Queue: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-12-04 23:03:04 +00:00
Cherry Mui 1926a0b573 cmd/compile: update PGO profile
Since the default.pgo profile is collected, there has been a lot
of development and refactoring in the compiler. It's time to
refresh the compiler's PGO profile. As we are in the freeze, there
will probably not be many major changes to the compiler, so this
should be good for Go 1.22.

Profile collected by running the cmd/compile/profile.sh script on
the linux-amd64-perf gomote.

Benchmark results on Linux/AMD64:

         │  nopgo.txt   │               old.txt               │              new.txt               │
         │    sec/op    │    sec/op     vs base               │   sec/op     vs base               │
Template    127.4m ± 0%    125.3m ± 0%  -1.65% (p=0.000 n=20)   123.5m ± 0%  -3.07% (p=0.000 n=20)
Unicode    105.94m ± 0%   100.77m ± 0%  -4.87% (p=0.000 n=20)   99.26m ± 0%  -6.30% (p=0.000 n=20)
GoTypes     798.2m ± 1%    784.4m ± 0%  -1.73% (p=0.000 n=20)   769.4m ± 1%  -3.61% (p=0.000 n=20)
Compiler    85.10m ± 0%    82.03m ± 1%  -3.61% (p=0.000 n=20)   79.77m ± 1%  -6.26% (p=0.000 n=20)
SSA          6.054 ± 0%     5.945 ± 0%  -1.81% (p=0.000 n=20)    5.811 ± 0%  -4.03% (p=0.000 n=20)
Flate       75.37m ± 1%    74.11m ± 0%  -1.66% (p=0.000 n=20)   73.02m ± 1%  -3.12% (p=0.000 n=20)
GoParser    144.6m ± 1%    141.7m ± 1%  -1.95% (p=0.000 n=20)   140.5m ± 1%  -2.83% (p=0.000 n=20)
Reflect     312.9m ± 1%    304.9m ± 1%  -2.56% (p=0.000 n=20)   300.4m ± 0%  -4.00% (p=0.000 n=20)
Tar         115.8m ± 0%    113.6m ± 0%  -1.88% (p=0.000 n=20)   111.9m ± 1%  -3.33% (p=0.000 n=20)
XML         145.9m ± 1%    143.6m ± 0%  -1.58% (p=0.000 n=20)   141.3m ± 1%  -3.15% (p=0.000 n=20)
geomean     224.1m         218.9m       -2.34%                  215.2m       -3.98%

This also shows that a stale profile (the old one) still brings a
speedup, but smaller, as expected.

As before, the profile is transferrable. Benchmark results on
Darwin/ARM64:

         │  nopgo.txt   │              old.txt               │               new.txt               │
         │    sec/op    │   sec/op     vs base               │    sec/op     vs base               │
Template   71.31m ± 34%   68.97m ± 1%  -3.28% (p=0.000 n=20)   68.66m ± 38%  -3.71% (p=0.030 n=20)
Unicode    58.70m ±  1%   55.64m ± 1%  -5.20% (p=0.000 n=20)   53.05m ±  1%  -9.61% (p=0.000 n=20)
GoTypes    422.9m ±  0%   412.7m ± 0%  -2.43% (p=0.000 n=20)   406.0m ±  1%  -4.01% (p=0.000 n=20)
Compiler   45.89m ±  1%   42.67m ± 2%  -7.00% (p=0.000 n=20)   42.96m ±  1%  -6.38% (p=0.000 n=20)
SSA         3.190 ±  0%    3.090 ± 0%  -3.15% (p=0.000 n=20)    3.007 ±  1%  -5.74% (p=0.000 n=20)
Flate      42.02m ±  1%   41.09m ± 1%  -2.20% (p=0.000 n=20)   40.58m ±  1%  -3.43% (p=0.000 n=20)
GoParser   75.75m ±  1%   73.84m ± 1%  -2.52% (p=0.000 n=20)   72.66m ±  1%  -4.08% (p=0.000 n=20)
Reflect    163.7m ±  1%   158.1m ± 1%  -3.39% (p=0.000 n=20)   155.3m ±  1%  -5.10% (p=0.000 n=20)
Tar        62.03m ±  1%   60.15m ± 0%  -3.02% (p=0.000 n=20)   59.74m ±  1%  -3.69% (p=0.000 n=20)
XML        79.38m ±  1%   77.32m ± 1%  -2.59% (p=0.000 n=20)   75.05m ±  2%  -5.45% (p=0.000 n=20)
geomean    120.9m         116.6m       -3.49%                  114.6m        -5.14%

For #60234.

Change-Id: I4ead6158f835c2e69c0f51ea24d044b82a7320cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/546337
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-04 18:40:12 +00:00
zzkcode de5b418bea runtime: let the fault thread to crash the process
Let the fault thread to crash the program to make sure while gdb coredump file could see the correct backtrace in the number one thread in gdb.

Fixes #63277.

Change-Id: Ie4473f76f0feba596091433918bcd35a4ff7e11b
GitHub-Last-Rev: f4615c23f6
GitHub-Pull-Request: golang/go#63666
Reviewed-on: https://go-review.googlesource.com/c/go/+/536895
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-02 03:15:03 +00:00
Bryan C. Mills 58bfef82fc net/http: avoid leaking io.Copy goroutines (and hijacked connections) in TestTransportNoReuseAfterEarlyResponse
Fixes #64252 (maybe).

Change-Id: Iba2a403a9347be4206f14acb11591dc2eb7f9fb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/546616
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-12-01 22:07:19 +00:00
Michael Anthony Knyszek cf3d253c1b runtime: check if mp.curg is nil before readgstatus in writeProcStatusForP
CL 546025 failed to check if mp.curg is nil, causing all sorts of
failures. It can very well be nil in this context.

For #64318.
Fixes #64507.

Change-Id: I4a95c3fa16d5e1dee8041394c4bdb9c6ba04f032
Reviewed-on: https://go-review.googlesource.com/c/go/+/546636
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-01 21:36:00 +00:00