Commit Graph

41034 Commits

Author SHA1 Message Date
Roland Shoemaker 542e8c74e7 [dev.fuzz] internal/fuzz: stablize mutator benchmark and add additional benchmarks
Adds a few new benchmarks, and attempts to reduce the variability of the
existing BenchmarkMutatorBytes benchmark. These should help provide some
insight when we're working on performance issues.

Change-Id: I45b68ae36da99ec2eb4a610b7a3fc6fbf3d9494a
Reviewed-on: https://go-review.googlesource.com/c/go/+/324969
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-06-04 15:59:32 +00:00
Roland Shoemaker 5542c10fbf [dev.fuzz] internal/fuzz: don't use temporary buffers
We're already allocating a huge amount of space at the end of each byte
slice, so just use that for scratch space instead of allocating a new
temporary buffer.

Change-Id: Ie075a199d4ac5dd5789949af2a19815c3025b809
Reviewed-on: https://go-review.googlesource.com/c/go/+/314272
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-06-04 03:07:33 +00:00
Katie Hockman 106851ad73 [dev.fuzz] all: merge master (dd7ba3b) into dev.fuzz
Merge List:

+ 2021-06-02 dd7ba3ba2c net: don't rely on system hosts in TestCVE202133195
+ 2021-06-02 4f572d7076 io/fs: minor corrections to Sub docs
+ 2021-06-02 e11d14225c doc/go1.17: remove runtime section
+ 2021-06-02 6e189afd3e doc/go1.17: mention SYS_WAIT6/WEXITED on NetBSD
+ 2021-06-02 ff9f5fb859 cmd/link: recognize clang linker error in testCGOLTO
+ 2021-06-02 1c6a2ea2ea doc/go1.17: document time changes for Go1.17
+ 2021-06-02 d743e67e06 doc/go1.17: document flag changes for Go 1.17
+ 2021-06-02 dc8f87b749 runtime/internal/sys: generate //go:build lines in gengoos.go
+ 2021-06-02 84c0e5d47f cmd/link: move issue 43830 tests out of TestScript
+ 2021-06-02 cae68700cc runtime: fix formatting
+ 2021-06-01 567ee865f6 cmd/go: add declaration to cgo_lto_issue43830 test
+ 2021-06-01 24e9707cbf cmd/link, cmd/cgo: support -flto in CFLAGS
+ 2021-06-01 272552275f A+C: update name
+ 2021-06-01 2bec019fb5 doc/go1.17: add release notes for register ABI
+ 2021-06-01 2e59cc5fb4 cmd/go: add [-src] to documentation
+ 2021-06-01 0b80cf1136 cmd/go: make 'go get' save sums for incidentally updated modules
+ 2021-05-30 3b770f2ccb go/types: don't declare 'comparable' when typeparams are disabled
+ 2021-05-30 1607c28172 go/types: unexport the GoVersion configuration option for Go 1.17
+ 2021-05-29 79bda65041 doc/go1.17: mention time.Layout
+ 2021-05-29 f6cc392d1d doc/go1.17: document text/template/parse.SkipFuncCheck
+ 2021-05-28 1419ca7cea doc/go1.17: mention new definitions of MSG_CMSG_CLOEXEC
+ 2021-05-28 6624771c83 doc/go1.17: mention testing.[TB].Setenv methods
+ 2021-05-28 bbda923592 doc/go1.17: mention new Windows SysProcAttr fields
+ 2021-05-28 6f58088bd8 doc/go1.17: document new go/build/BuildContext.ToolTags field
+ 2021-05-28 c295107708 doc/go1.17: mention new encoding/csv/Reader.FieldPos method
+ 2021-05-28 ccd9784edf doc/go1.17: document new debug/elf constant
+ 2021-05-28 3de3440fb9 go/ast: remove FuncDecl.IsMethod for Go 1.17
+ 2021-05-27 639acdc833 doc/go1.17: clarify that compress/lzw Reader and Writer types are new
+ 2021-05-27 193d514131 net/http: correct Client.Do doc about context cancelation
+ 2021-05-27 ab2ef4aaa7 doc/go1.17: document reflect changes
+ 2021-05-27 0ece95a0fe cmd/go: don't let 'go mod download' save sums for inconsistent requirements
+ 2021-05-27 cdcd02842d net: verify results from Lookup* are valid domain names
+ 2021-05-27 8bf5bf5173 cmd/compile: improve debug locations for partially live in-params
+ 2021-05-27 56af34f875 cmd/compile: place reg spills after OpArg{Int,Float}Reg ops
+ 2021-05-27 db66e9e15d cmd/link: accept Windows line-ending in TestTrampolineCgo
+ 2021-05-27 6b8c94b6c5 go/types: guard against check==nil in newNamed
+ 2021-05-27 fca7b8f3e6 Revert "net: verify results from Lookup* are valid domain names"
+ 2021-05-27 950fa11c4c net/http/httputil: always remove hop-by-hop headers
+ 2021-05-27 9bc52686da cmd/go,cmd/link: do not check for staleness in most tests
+ 2021-05-27 6ff0ae2aa4 crypto/elliptic: fix typo in p521Point type name
+ 2021-05-26 3075ffc93e os: deflake TestFdReadRace
+ 2021-05-26 a62c08734f src/os: revert accidentally submitted change
+ 2021-05-26 1d5298d46a doc/go1.17: document net/... changes
+ 2021-05-26 0fbecece98 doc/go1.17: document syscall changes
+ 2021-05-26 02beecb397 mime: document use of the Shared MIME-Info Database
+ 2021-05-26 a92460fd2f doc/go1.17: add release notes for runtime/metrics package
+ 2021-05-26 55aefbb268 doc/go1.17: mention enabling frame pointer on all ARM64
+ 2021-05-26 39da9ae513 go/types: ensure that Named.check is nilled out once it is expanded
+ 2021-05-26 bfd7798a6c runtime,cmd/link/internal/ld: fix typos
+ 2021-05-26 e4615ad74d math/big: move division into natdiv.go
+ 2021-05-26 d050238bb6 doc/go1.17: fix formatting for time changes
+ 2021-05-25 74242baa41 archive/zip: only preallocate File slice if reasonably sized
+ 2021-05-25 f22ec51deb doc: add Go 1.17 release note about inlining functions with closures
+ 2021-05-25 8b462d7567 cmd/go: add a -compat flag to 'go mod tidy'
+ 2021-05-24 c89f1224a5 net: verify results from Lookup* are valid domain names
+ 2021-05-24 08a8fa9c47 misc/wasm: ensure correct stack pointer in catch clauses
+ 2021-05-24 32b73ae180 cmd/go: align checks of module path during initialization.
+ 2021-05-24 15d9d4a009 cmd/go: add tests illustrating what happens when Go 1.16 is used in a Go 1.17 main module
+ 2021-05-24 873401df5b cmd/compile: ensure equal functions don't do unaligned loads
+ 2021-05-24 b83610699a cmd/compile: record regabi status in DW_AT_producer
+ 2021-05-24 a22e317220 cmd/compile: always include underlying type for map types
+ 2021-05-24 4356e7e85f runtime: account for spill slots in Windows callback compilation
+ 2021-05-24 52d7033ff6 cmd/go/internal/modload: set the default GoVersion in a single location
+ 2021-05-24 05819bc104 cmd/go/internal/modcmd: factor out a type for flags whose arguments are Go versions
+ 2021-05-22 cca23a7373 cmd/compile: revert CL/316890
+ 2021-05-21 f87194cbd7 doc/go1.17: document changes to net/http package
+ 2021-05-21 217f5dd496 doc: document additional atomic.Value methods
+ 2021-05-21 3c656445f1 cmd/go: in TestScript/mod_replace, download an explicit module path
+ 2021-05-21 76b2d6afed os: document that StartProcess puts files into blocking mode
+ 2021-05-21 e4d7525c3e cmd/dist: display first class port status in json output
+ 2021-05-21 4fb10b2118 cmd/go: in 'go mod download' without args, don't save module zip sums
+ 2021-05-21 4fda54ce3f doc/go1.17: document database/sql changes for Go 1.17
+ 2021-05-21 8876b9bd6a doc/go1.17: document io/fs changes for Go 1.17
+ 2021-05-21 5fee772c87 doc/go1.17: document archive/zip changes for Go 1.17
+ 2021-05-21 3148694f60 cmd/go: remove warning from module deprecation notice printing
+ 2021-05-21 7e63c8b765 runtime: wait for Go runtime to initialize in Windows signal test
+ 2021-05-21 831573cd21 io/fs: added an example for io/fs.WalkDir
+ 2021-05-20 baa934d26d cmd: go get golang.org/x/tools/analysis@49064d23 && go mod vendor
+ 2021-05-20 7c692cc7ea doc/go1.17: document changes to os package
+ 2021-05-20 ce9a3b79d5 crypto/x509: add new FreeBSD 12.2+ trusted certificate folder
+ 2021-05-20 f8be906d74 test: re-enable test on riscv64 now that it supports external linking
+ 2021-05-20 def5360541 doc/go1.17: add release notes for OpenBSD ports
+ 2021-05-20 ef1f52cc38 doc/go1.17: add release note for windows/arm64 port
+ 2021-05-20 bb7495a46d doc/go1.17: document new math constants
+ 2021-05-20 f07e4dae3c syscall: document NewCallback and NewCallbackCDecl limitations
+ 2021-05-20 a8d85918b6 misc/cgo/testplugin: skip TestIssue25756pie on darwin/arm64 builder
+ 2021-05-19 6c1c055d1e cmd/internal/moddeps: use filepath.SkipDir only on directories
+ 2021-05-19 658b5e66ec net: return nil UDPAddr from ReadFromUDP
+ 2021-05-19 15a374d5c1 test: check portable error message on issue46234.go
+ 2021-05-18 eeadce2d87 go/build/constraint: fix parsing of "// +build" (with no args)
+ 2021-05-18 6d2ef2ef2a cmd/compile: don't emit inltree for closure within body of inlined func
+ 2021-05-18 048cb4ceee crypto/x509: remove duplicate import
+ 2021-05-18 690a8c3fb1 make.bash: fix misuse of continue
+ 2021-05-18 8b0901fd32 doc/go1.17: fix typo "avoding" -> "avoiding"
+ 2021-05-18 5e191f8f48 time: rewrite the documentation for layout strings
+ 2021-05-17 bfe3573d58 go/token: correct the interval notation used in some panic messages
+ 2021-05-17 a2c07a9a1a all: update golang.org/x/net to latest
+ 2021-05-17 b9b2bed893 syscall: some containers may fail syscall.TestSetuidEtc
+ 2021-05-17 b1aff42900 cmd/go: don't print 'go get' deprecation notices in the main module
+ 2021-05-17 bade680867 runtime/cgo: fix crosscall2 on ppc64x
+ 2021-05-15 ce92a2023c cmd/go: error out of 'go mod tidy' if the go version is newer than supported
+ 2021-05-14 02699f810a runtime: mark osyield nosplit on OpenBSD
+ 2021-05-14 3d324f127d net/http: prevent infinite wait during TestMissingStatusNoPanic
+ 2021-05-14 0eb38f2b16 cmd/go/internal/load: override Package.Root in module mode
+ 2021-05-14 a938e52986 cmd/go: fix a portability issue in the cd script command

Change-Id: I72e621368c4435396eb727d40287d1e318505308
2021-06-02 22:39:15 +00:00
Roland Shoemaker dd7ba3ba2c net: don't rely on system hosts in TestCVE202133195
Also don't unnecessarily deref the error return.

Fixes #46504

Change-Id: I22d14ac76776f8988fa0774bdcb5fcd801ce0185
Reviewed-on: https://go-review.googlesource.com/c/go/+/324190
Trust: David Chase <drchase@google.com>
Trust: Damien Neil <dneil@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2021-06-02 21:39:28 +00:00
Jay Conrod bc1fbe2044 [dev.fuzz] internal/fuzz: notify coordinator for minimization
When a worker process finds a crasher, it now sends that result
directly to the coordinator without attempting to minimize it
first. The coordinator stops sending new inputs and sends the
unminimized crasher back to a worker (any worker) for minimization.

This prevents wasted work during minimization and will help us
implement -keepfuzzing later on. We may also be able to minimize
interesting inputs with this approach later.

Since panics are recoverable errors (they don't terminate worker
processes), we no longer attempt to minimize non-recoverable errors.
This didn't work too well before: we lost too much state.

Change-Id: Id142c7e91a33f64584170b0d42d22cb1f22a92d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/321835
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-06-02 20:55:09 +00:00
Jay Conrod a3b5084204 [dev.fuzz] cmd/go: split test_fuzz_minimize from test_fuzz_mutator
This CL moves test cases for minimization into a new script. The tests
are changed to be more deterministic. Any non-zero byte now triggers a
failure.

Change-Id: Id45a721071dbfc3aec46be369966b0a67a745c55
Reviewed-on: https://go-review.googlesource.com/c/go/+/323851
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-06-02 20:51:28 +00:00
Ian Lance Taylor 4f572d7076 io/fs: minor corrections to Sub docs
Fixes #44376

Change-Id: I9cd21adb9d4d434c3d8b8eb8af3042b70c763ea1
Reviewed-on: https://go-review.googlesource.com/c/go/+/296389
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2021-06-02 18:52:58 +00:00
Ian Lance Taylor ff9f5fb859 cmd/link: recognize clang linker error in testCGOLTO
Also recognize a case in which GCC does not run (from
https://build.golang.org/log/7f6d8b35c905b9829f05906beccca44f208aa569).

Fixes #46517

Change-Id: I4fe4164a5df92b2dec08fd767f65a4d5479f3f36
Reviewed-on: https://go-review.googlesource.com/c/go/+/324169
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2021-06-02 15:45:31 +00:00
Tobias Klauser dc8f87b749 runtime/internal/sys: generate //go:build lines in gengoos.go
For #41184

Change-Id: If7a1c3980f47bc28d0a13fe497eaba6178c65c91
Reviewed-on: https://go-review.googlesource.com/c/go/+/323750
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2021-06-02 08:17:59 +00:00
Ian Lance Taylor 84c0e5d47f cmd/link: move issue 43830 tests out of TestScript
These tests pass or fail depending on the exact compiler version,
which the TestScript tests don't support. Rewrite into Go.

For #43830
For #46295

Change-Id: I91b61dfe329d518e461ee56f186f0e9b42858e77
Reviewed-on: https://go-review.googlesource.com/c/go/+/324049
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-06-02 00:36:24 +00:00
Keith Randall cae68700cc runtime: fix formatting
Fix up a gofmt complaint from CL 310591.

Change-Id: I73534ef064a4cfc53539e5e65a8653e2cd684c64
Reviewed-on: https://go-review.googlesource.com/c/go/+/324090
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-06-02 00:20:16 +00:00
Ian Lance Taylor 567ee865f6 cmd/go: add declaration to cgo_lto_issue43830 test
This permits the test to work in C99 mode.

For #43830

Change-Id: Ide54bd62239cfe602e2664300f04e472df5daf43
Reviewed-on: https://go-review.googlesource.com/c/go/+/324009
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-06-01 21:14:02 +00:00
Ian Lance Taylor 24e9707cbf cmd/link, cmd/cgo: support -flto in CFLAGS
The linker now accepts unrecognized object files in external linking mode.
These objects will simply be passed to the external linker.
This permits using -flto which can generate pure byte code objects,
whose symbol table the linker does not know how to read.

The cgo tool now passes -fno-lto when generating objects whose symbols
it needs to read. The cgo tool now emits matching types in different
objects, so that the lto linker does not report a mismatch.

This is based on https://golang.org/cl/293290 by Derek Parker.

For #43505
Fixes #43830
Fixes #46295

Change-Id: I6787de213417466784ddef5af8899e453b4ae1ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/322614
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
2021-06-01 19:59:18 +00:00
Vitaly Zdanevich 2e59cc5fb4 cmd/go: add [-src] to documentation
Change-Id: I554b5021386575af6ff44571a95bb31b38a0547f
GitHub-Last-Rev: 20aaec3aa0
GitHub-Pull-Request: golang/go#45956
Reviewed-on: https://go-review.googlesource.com/c/go/+/317109
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Trust: Bryan C. Mills <bcmills@google.com>
2021-06-01 13:23:26 +00:00
Jay Conrod 0b80cf1136 cmd/go: make 'go get' save sums for incidentally updated modules
When 'go get' updates a module, it may update another module in the
build list that provides a package in 'all' that wasn't loaded as part
of the 'go get' command. If 'go get' doesn't add a sum for that
module, builds may fail later.

With this change, 'go get' will fetch a sum for the content of an
updated module if we had a sum for the version before the update.

'go get' won't load the complete package graph, so there are still
cases where the build may be broken, like when an updated (but not
loaded) package imports a package from a new module.

Fixes #44129

Change-Id: I62eba3df4137a3e84e2ca8d549c36eec3670f08c
Reviewed-on: https://go-review.googlesource.com/c/go/+/322832
Trust: Jay Conrod <jayconrod@google.com>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-06-01 12:20:06 +00:00
Rob Findley 3b770f2ccb go/types: don't declare 'comparable' when typeparams are disabled
Fixes #46453

Change-Id: I92b9b1e43ec5182162b2eeeb667f1f548ea373a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/323609
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-05-30 17:47:50 +00:00
Rob Findley 1607c28172 go/types: unexport the GoVersion configuration option for Go 1.17
The GoVersion field was added to types.Config as part of the work on
type parameters. Specifically, it was added to be consistent with
cmd/compile/internal/types2, which requires such an option.

This configuration option is useful, but is also non-trivial and did not
go through the proposal process. Unexport it for Go 1.17; we can create
a proposal to export it for Go 1.18.

Fixes #46296

Change-Id: Id82d8a7096887dcfc404c4d6d8da9c761b316609
Reviewed-on: https://go-review.googlesource.com/c/go/+/323430
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-05-30 02:37:38 +00:00
Rob Findley 3de3440fb9 go/ast: remove FuncDecl.IsMethod for Go 1.17
The IsMethod method was added to FuncDecl in the process of working on
support for type parameters, but is now only used in one place. It also
didn't go through the proposal process. Remove it for 1.17.

Also clean up a doc comment that mentioned type parameters.

Fixes #46297

Change-Id: I432bdd626324f613baf059540b7c5436985b2b16
Reviewed-on: https://go-review.googlesource.com/c/go/+/323369
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-05-28 03:34:02 +00:00
Manlio Perillo 193d514131 net/http: correct Client.Do doc about context cancelation
The documentation of the Client.Do method and Get function incorrectly
stated that, in case of context cancelation, the returned url.Error
Timeout method returns true.

Update the documentation to correctly match the implementation.
See also CL 200798 that, due to an oversight, corrected only the
documentation of the Client.Get method.

Remove a TODO note added in CL 125575 (net/http: document that Client
methods always return *url.Error), since it is no longer applicable
after CL 200798 (net/http: fix and lock-in Client.Do docs on request
cancelation).

Fixes #46402

Change-Id: Ied2ee971ba22b61777762dbb19f16e08686634ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/323089
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
2021-05-27 21:41:58 +00:00
Jay Conrod 0ece95a0fe cmd/go: don't let 'go mod download' save sums for inconsistent requirements
'go mod download' calls modload.LoadModFile early to find the main
module path in order to validate arguments. LoadModFile may write
go.mod and go.sum to fix formatting and add a go directive. This calls
keepSums, which, in eager mode, loaded the complete module graph in
order to find out what sums are needed to load the complete module
graph. If go.mod requires a lower version of a module than will be
selected later, keepSums causes the sum for that version's go.mod to
be retained, even though it isn't needed later after a consistent
go.mod is written.

This CL fixes keepSums not to load the graph if it hasn't already been
loaded (whether eager or lazy), addressing comments from CL 318629.

For #45332

Change-Id: I20d4404004e4ad335450fd0fd753e7bc0060f702
Reviewed-on: https://go-review.googlesource.com/c/go/+/322369
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-27 20:38:20 +00:00
Roland Shoemaker cdcd02842d net: verify results from Lookup* are valid domain names
For the methods LookupCNAME, LookupSRV, LookupMX, LookupNS, and
LookupAddr check that the returned domain names are in fact valid DNS
names using the existing isDomainName function.

Thanks to Philipp Jeitner and Haya Shulman from Fraunhofer SIT for
reporting this issue.

Fixes #46241
Fixes CVE-2021-33195

Change-Id: I47a4f58c031cb752f732e88bbdae7f819f0af4f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/323131
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-05-27 19:55:46 +00:00
Katie Hockman 2fed6926a1 [dev.fuzz] cmd/go/internal: instrument imports of the test
Previously, the packages that were imported by the
test were not instrumented for coverage. This meant
that a fuzz target in a stand-alone test file would
not be able to perform coverage-guided fuzzing.

The fix uses all of the imports, including those
from the test files, when determining which packages
to instrument. However, certain package should
be ignored when walking the import graph. Otherwise,
packages like internal/fuzz, and its imports, may be
instrumented, which could lead to false positives for
"interesting" corpus values.

There was an additional bug which needed to be fixed
in order for this to work. The bug was in the fact that
the GcFlags which held `-d=libfuzzer` were being
overwritten in some cases. The fix updates the way these
flags are set in order to prevent this behavior.

Change-Id: I21d336c29a33db1181bbae0fd23678d127fe52a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/321960
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-27 19:16:13 +00:00
Than McIntosh 8bf5bf5173 cmd/compile: improve debug locations for partially live in-params
During DWARF debug location generation, as a preamble to the main data
flow analysis, examine the function entry block to look for in-params
arriving in registers that are partially or completely dead, and
insert new OpArg{Int,Float}Reg values for the dead or partially-dead
pieces. In addition, add entries to the f.NamedValues table for
incoming live register-resident params that don't already have
entries. This helps create better/saner DWARF location expressions for
params. Example:

  func foo(s string, used int, notused int) int {
      return len(s) + used
  }

When optimization is complete for this function, the parameter
"notused" is completely dead, meaning that there is no entry for it in
the f.NamedValues table (which then means we don't emit a DWARF
variable location expression for it in the function enty block). In
addition, since only the length field of "s" is used, there is no
DWARF location expression for the other component of "s", leading to
degraded DWARF.

There are still problems/issues with DWARF location generation, but
this does improve things with respect to being able to print the
values of incoming parameters when stopped in the debugger at the
entry point of a function (when optimization is enabled).

Updates #40724.

Change-Id: I5bb5253648942f9fd33b081fe1a5a36208e75785
Reviewed-on: https://go-review.googlesource.com/c/go/+/322631
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-05-27 18:49:23 +00:00
Than McIntosh 56af34f875 cmd/compile: place reg spills after OpArg{Int,Float}Reg ops
Tweak the register allocator to maintain the invariant that
OpArg{Int,Float}Reg values are placed together at the start of the
entry block, before any other non-pseudo-op values. Without this
change, when the register allocator adds spills we can wind up with an
interleaving of OpArg*Reg and stores, which complicates debug location
analysis.

Updates #40724.

Change-Id: Icf30dd814a9e25263ecbea2e48feb840a6e7f2bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/322630
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-05-27 18:47:37 +00:00
Cherry Mui db66e9e15d cmd/link: accept Windows line-ending in TestTrampolineCgo
Apparently C printf emits "\r\n" on Windows. Accept that.

Change-Id: If87ba41435e3147d3892cfc3fe3a105b066ff0aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/322973
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-05-27 18:01:11 +00:00
Rob Findley 6b8c94b6c5 go/types: guard against check==nil in newNamed
When importing generic named types, it is possible for Checker.newNamed
to be called during type instantiation when the Checker is nil.

In this case we should be able to safely skip this delayed expansion.

Updates #45580

Change-Id: I75422100464d57eba24642c93e06e8b47d904fc3
Reviewed-on: https://go-review.googlesource.com/c/go/+/322974
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-05-27 17:28:26 +00:00
Roland Shoemaker fca7b8f3e6 Revert "net: verify results from Lookup* are valid domain names"
This reverts commit c89f1224a5.

Reason for revert: reverting so we can apply follow-up fixes and do a single cherry pick.

Change-Id: I16c6283a0bcab056216f330fb98fa3b5f2b0780c
Reviewed-on: https://go-review.googlesource.com/c/go/+/323129
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-27 15:33:46 +00:00
Filippo Valsorda 950fa11c4c net/http/httputil: always remove hop-by-hop headers
Previously, we'd fail to remove the Connection header from a request
like this:

    Connection:
    Connection: x-header

Fixes #46313
Fixes CVE-2021-33197

Change-Id: Ie3009e926ceecfa86dfa6bcc6fe14ff01086be7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/321929
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-27 15:00:58 +00:00
Bryan C. Mills 9bc52686da cmd/go,cmd/link: do not check for staleness in most tests
Instead, check that stale packages in the standard library
are not rebuilt when already present in the build cache,
and are not installed implicitly when rebuilt.

We retain the staleness checks for the runtime package in tests
involving '-i', because those are guaranteed to fail anyway if the
package is stale and the "stale" failure message is arguably clearer.
They can be removed if/when we remove the '-i' flag, but the runtime
package is less likely to become stale because it does not have cgo
dependencies.

Fixes #46347
Updates #33598
Updates #35459
Updates #41696

Change-Id: I7b0a808addd930f9f4911ff53ded62272af75a40
Reviewed-on: https://go-review.googlesource.com/c/go/+/322629
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-27 14:03:15 +00:00
Filippo Valsorda 6ff0ae2aa4 crypto/elliptic: fix typo in p521Point type name
Change-Id: I6cab3624c875d9a70441a560e84f91c9b2df17b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/320070
Trust: Filippo Valsorda <filippo@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-05-27 11:17:57 +00:00
Roland Shoemaker a573cbfa6c [dev.fuzz] internal/fuzz: support minimization of strings, integers, and floats
Adds support for minimizing strings using the same logic as byte slices
as well as minimizing both signed and unsigned integers and floats using
extremely basic logic. A more complex approach is probably warranted in
the future, but for now this should be _good enough_.

Change-Id: Ibc6c3d6ae82685998f571aa2c1ecea2f85c2708b
Reviewed-on: https://go-review.googlesource.com/c/go/+/320669
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-27 02:15:49 +00:00
Roland Shoemaker 5fcd18bc9b [dev.fuzz] internal/fuzz,testing: treat panics as recoverable
And only log the last panic, not all of them, during minimization.
This change makes the worker processes quiet, so now the only
process that logs anything is the coordinator. This hides all of
the panics caused during minimization of an input which causes
a panic.

This change also alters the usage of tRunner such that we now
recover from recoverable panics instead of terminating the
process. This results in larger stack traces, since we include
a bit more of the trace within testing. There is a TODO to see
if it's possible to slice the stack up so that it is somewhat
more informative.

Change-Id: Ic85eabd2e70b078412fbb88adf424a8da25af876
Reviewed-on: https://go-review.googlesource.com/c/go/+/321230
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-05-27 02:14:18 +00:00
Ian Lance Taylor 3075ffc93e os: deflake TestFdReadRace
The test would hang if the call to Fd set the pipe to be non-blocking
before the Read entered the first read system call. Avoid that problem
by writing data to the pipe to wake up the read.

For #24481
Fixes #44818

Change-Id: I0b798874c7b81e7308a38ebbf657efc4392ffacd
Reviewed-on: https://go-review.googlesource.com/c/go/+/322893
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2021-05-26 23:33:38 +00:00
Damien Neil a62c08734f src/os: revert accidentally submitted change
Change-Id: Ib34984a6bd0abc76266e8aac96f9f8ad8ae21d17
Reviewed-on: https://go-review.googlesource.com/c/go/+/322894
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-05-26 22:54:09 +00:00
Damien Neil 1d5298d46a doc/go1.17: document net/... changes
For #44513.
Fixes #46014.
Fixes #46015.
Fixes #46016.
Fixes #46017.

Change-Id: I356483d68d07159281dfe2ea1e49430ddf200973
Reviewed-on: https://go-review.googlesource.com/c/go/+/322891
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-05-26 22:43:54 +00:00
Damien Neil 02beecb397 mime: document use of the Shared MIME-Info Database
For #44513.
Fixes #46013.

Change-Id: I382603208aa94b66d5220cf0f418b8528a4e4148
Reviewed-on: https://go-review.googlesource.com/c/go/+/322892
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-05-26 22:41:35 +00:00
Rob Findley 39da9ae513 go/types: ensure that Named.check is nilled out once it is expanded
To support lazy expansion of defined types, *Named holds on to a
*Checker field, which can pin the *Checker in memory. This can have
meaningful memory implications for applications that keep type
information around.

Ensure that the Checker field is nilled out for any Named types that are
instantiated during the type checking pass, by deferring a clean up to
'later' boundaries.

In testing this almost exactly offset the ~6% memory footprint increase
I observed with 1.17.

Fixes #45580

Change-Id: I8aa5bb777573a924afe36e79fa65f8729336bceb
Reviewed-on: https://go-review.googlesource.com/c/go/+/318849
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-05-26 18:24:48 +00:00
tyltr bfd7798a6c runtime,cmd/link/internal/ld: fix typos
Change-Id: I558590cef7e2311aadbdcb4088033e350d3aae32
GitHub-Last-Rev: 513944a623
GitHub-Pull-Request: golang/go#46389
Reviewed-on: https://go-review.googlesource.com/c/go/+/322809
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-05-26 16:11:00 +00:00
Katie Hockman ff0164cf73 Revert "[dev.fuzz] internal/fuzz: include coverage in logged stats"
This reverts commit 54f067812d.

Reason for revert: While this is helpful for the engineering team when we're debugging, it might lead to users feeling like the fuzzer is stuck and that there are a lot of edges that are still yet to be reached. In reality, it's very likely that the compiler will instrument more lines of code than are actually reachable by the fuzz target, so showing the ratio between number of edges hit vs. all edges can be misleading. In the future, we may want to consider making this information viewable by a debug flag or something similar.

Change-Id: Ied696f8bf644445bad22c872b64daa7add605ac6
Reviewed-on: https://go-review.googlesource.com/c/go/+/322632
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-26 15:00:45 +00:00
Russ Cox e4615ad74d math/big: move division into natdiv.go
Code moved and functions reordered to be in a consistent
top-down dependency order, but otherwise unchanged.

First step toward commenting division algorithms.

Change-Id: Ib5e604fb5b2867edff3a228ba4e57b5cb32c4137
Reviewed-on: https://go-review.googlesource.com/c/go/+/321077
Trust: Russ Cox <rsc@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-05-26 13:25:43 +00:00
Roland Shoemaker 74242baa41 archive/zip: only preallocate File slice if reasonably sized
Since the number of files in the EOCD record isn't validated, it isn't
safe to preallocate Reader.Files using that field. A malformed archive
can indicate it contains up to 1 << 128 - 1 files. We can still safely
preallocate the slice by checking if the specified number of files in
the archive is reasonable, given the size of the archive.

Thanks to the OSS-Fuzz project for discovering this issue and to
Emmanuel Odeke for reporting it.

Fixes #46242
Fixes CVE-2021-33196

Change-Id: I3c76d8eec178468b380d87fdb4a3f2cb06f0ee76
Reviewed-on: https://go-review.googlesource.com/c/go/+/318909
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-05-25 23:41:42 +00:00
Bryan C. Mills 8b462d7567 cmd/go: add a -compat flag to 'go mod tidy'
Fixes #46141

Change-Id: I9d4032e75252ade9eaa937389ea97ef3fb287499
Reviewed-on: https://go-review.googlesource.com/c/go/+/321071
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-25 13:18:26 +00:00
Roland Shoemaker c89f1224a5 net: verify results from Lookup* are valid domain names
For the methods LookupCNAME, LookupSRV, LookupMX, LookupNS, and
LookupAddr check that the returned domain names are in fact valid DNS
names using the existing isDomainName function.

Thanks to Philipp Jeitner and Haya Shulman from Fraunhofer SIT for
reporting this issue.

Fixes #46241
Fixes CVE-2021-33195

Change-Id: Icf231acd93178a3b6aec3f178cff7e693f74ef8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/320949
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-05-24 22:27:31 +00:00
Constantin Konstantinidis 32b73ae180 cmd/go: align checks of module path during initialization.
Fixes #45025.

Change-Id: I70c2b745f764484e4b3a2824adc470f168fb2c50
Reviewed-on: https://go-review.googlesource.com/c/go/+/320310
Trust: Jay Conrod <jayconrod@google.com>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-24 20:57:01 +00:00
Bryan C. Mills 15d9d4a009 cmd/go: add tests illustrating what happens when Go 1.16 is used in a Go 1.17 main module
For #46141
Updates #46160

Change-Id: Ib22435b8051aaf3fa74d43d3b7f2d091e67f05e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/320172
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-24 20:15:56 +00:00
Keith Randall 873401df5b cmd/compile: ensure equal functions don't do unaligned loads
On architectures which don't support unaligned loads, make sure we
don't generate code that requires them.

Generated hash functions also matter in this respect, but they all look ok.

Update #37716
Fixes #46283

Change-Id: I6197fdfe04da4428092c99bd871d93738789e16b
Reviewed-on: https://go-review.googlesource.com/c/go/+/322151
Trust: Keith Randall <khr@golang.org>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: eric fang <eric.fang@arm.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-24 19:06:05 +00:00
Alessandro Arzilli b83610699a cmd/compile: record regabi status in DW_AT_producer
Records if regabi was enabled during compilation in the DW_AT_producer
attribute of each compile unit.
This is useful to debuggers that support the debugCall protocol.

Change-Id: I5ad2c48ebf126aeb8bfb459b53a1a5304550036a
Reviewed-on: https://go-review.googlesource.com/c/go/+/318050
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-24 18:10:39 +00:00
Keith Randall a22e317220 cmd/compile: always include underlying type for map types
This is a different fix for #37716.

Should help make the fix for #46283 easier, since we will no longer
need to keep compiler-generated hash functions and the runtime
hash function in sync.

Change-Id: I84cb93144e425dcd03afc552b5fbd0f2d2cc6d39
Reviewed-on: https://go-review.googlesource.com/c/go/+/322150
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-05-24 17:43:50 +00:00
Michael Anthony Knyszek 4356e7e85f runtime: account for spill slots in Windows callback compilation
The Go ABI, as it stands, requires spill space to be reserved for
register arguments. syscall.NewCallback (because of compileCallback)
does not actually reserve this space, leading to issues if the Go code
it invokes actually makes use of it.

Fixes #46301.

Change-Id: Idbc3578accaaaa29e4ba32291ef08d464da0b7b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/322029
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Egon Elbre <egonelbre@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-05-24 16:47:51 +00:00
Bryan C. Mills 52d7033ff6 cmd/go/internal/modload: set the default GoVersion in a single location
For #46141
Updates #36460

Change-Id: Ie4c13c73a451650d1e8abb8e5cebfc30d0a71a70
Reviewed-on: https://go-review.googlesource.com/c/go/+/321070
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-24 15:03:18 +00:00