- TODO: https://golang.org/cl/216424: allow conversion from slice to array ptr +
+ Go 1.17 includes three small enhancements to the language.
-- TODO: https://golang.org/cl/312212: add unsafe.Add and unsafe.Slice +
s of
+ type []T may now be converted to array pointer type
+ *[N]T. If a is the result of such a
+ conversion, then corresponding indices that are in range refer to
+ the same underlying elements: &a[i] == &s[i]
+ for 0 <= i < N. The conversion panics if
+ len(s) is less than N.
+ unsafe.Add:
+ unsafe.Add(ptr, len) adds len
+ to ptr and returns the updated pointer
+ unsafe.Pointer(uintptr(ptr) + uintptr(len)).
+ unsafe.Slice:
+ For expression ptr of type *T,
+ unsafe.Slice(ptr, len) returns a slice of
+ type []T whose underlying array starts
+ at ptr and whose length and capacity
+ are len.
+
+ The package unsafe enhancements were added to simplify writing code that conforms
+ to unsafe.Pointer's safety
+ rules, but the rules remain unchanged. In particular, existing
+ programs that correctly use unsafe.Pointer remain
+ valid, and new programs must still follow the rules when
+ using unsafe.Add or unsafe.Slice.
+
+ Note that the new conversion from slice to array pointer is the + first case in which a type conversion can panic at run time. + Analysis tools that assume type conversions can never panic + should be updated to consider this possibility.
openbsd/amd64 and openbsd/arm64
ports) system calls are made through libc, instead
- of directly using the machine instructions. In Go 1.17, this is
- also done on the 32-bit x86 and 32-bit ARM architectures on OpenBSD
+ of directly using machine instructions. In Go 1.17, this is also
+ done on the 32-bit x86 and 32-bit ARM architectures on OpenBSD
(the openbsd/386 and openbsd/arm ports).
- This ensures forward-compatibility with future versions of
- OpenBSD.
+ This ensures compatibility with OpenBSD 6.9 onwards, which require
+ system calls to be made through libc for non-static
+ Go binaries.
- TODO: complete the Ports section -
-- TODO: complete the Tools section -
- To facilitate the upgrade to lazy loading,
- the go mod tidy subcommand now supports
+
+ Because the number of additional explicit requirements in the go.mod file may
+ be substantial, in a Go 1.17 module the newly-added requirements
+ on indirect dependencies are maintained in a
+ separate require block from the block containing direct
+ dependencies.
+
+ To facilitate the upgrade to lazy loading, the
+ go mod tidy subcommand now supports
a -go flag to set or change the go version in
the go.mod file. To enable lazy loading for an existing module
without changing the selected versions of its dependencies, run:
@@ -115,8 +159,39 @@ Do not send CLs removing the interior tags from such phrases.
- TODO: Describe the -compat flag
- for go mod tidy.
+ By default, go mod tidy verifies that
+ the selected versions of dependencies relevant to the main module are the same
+ versions that would be used by the prior Go release (Go 1.16 for a module that
+ specifies go 1.17), and preserves
+ the go.sum entries needed by that release even for dependencies
+ that are not normally needed by other commands.
+
+ The -compat flag allows that version to be overridden to support
+ older (or only newer) versions, up to the version specified by
+ the go directive in the go.mod file. To tidy
+ a go 1.17 module for Go 1.17 only, without saving
+ checksums for (or checking for consistency with) Go 1.16:
+
+ go mod tidy -compat=1.17 ++ +
+ Note that even if the main module is tidied with -compat=1.17,
+ users who require the module from a
+ go 1.16 or earlier module will still be able to
+ use it, provided that the packages use only compatible language and library
+ features.
+
+ The go mod graph subcommand also
+ supports the -go flag, which causes it to report the graph as
+ seen by the indicated Go version, showing dependencies that may otherwise be
+ pruned out by lazy loading.
environment for details.
+
+ go get prints a deprecation warning when installing
+ commands outside the main module (without the -d flag).
+ go install cmd@version should be used
+ instead to install a command at a specific version, using a suffix like
+ @latest or @v1.2.3. In Go 1.18, the -d
+ flag will always be enabled, and go get will only
+ be used to change dependencies in go.mod.
+
go.mod files missing go directives
@@ -210,18 +295,106 @@ Do not send CLs removing the interior tags from such phrases.
mod download all.
- TODO: https://golang.org/cl/249759: cmd/cover: replace code using optimized golang.org/x/tools/cover +
//go:build lines
+ The go command now understands //go:build lines
+ and prefers them over // +build lines. The new syntax uses
+ boolean expressions, just like Go, and should be less error-prone.
+ As of this release, the new syntax is fully supported, and all Go files
+ should be updated to have both forms with the same meaning. To aid in
+ migration, gofmt now automatically
+ synchronizes the two forms. For more details on the syntax and migration plan,
+ see
+ https://golang.org/design/draft-gobuild.
+
go run
+ go run now accepts arguments with version suffixes
+ (for example, go run
+ example.com/cmd@v1.0.0). This causes go
+ run to build and run packages in module-aware mode, ignoring the
+ go.mod file in the current directory or any parent directory, if
+ there is one. This is useful for running executables without installing them or
+ without changing dependencies of the current module.
+
+ gofmt (and go fmt) now synchronizes
+ //go:build lines with // +build lines. If a file
+ only has // +build lines, they will be moved to the appropriate
+ location in the file, and matching //go:build lines will be
+ added. Otherwise, // +build lines will be overwritten based on
+ any existing //go:build lines. For more information, see
+ https://golang.org/design/draft-gobuild.
- TODO: https://golang.org/cl/299532: cmd/vet: bring in sigchanyzer to report unbuffered channels to signal.Notify +
//go:build and // +build lines
+ The vet tool now verifies that //go:build and
+ // +build lines are in the correct part of the file and
+ synchronized with each other. If they aren't,
+ gofmt can be used to fix them. For more
+ information, see
+ https://golang.org/design/draft-gobuild.
signal.Notify on unbuffered channels
+ The vet tool now warns about calls to signal.Notify
+ with incoming signals being sent to an unbuffered channel. Using an unbuffered channel
+ risks missing signals sent on them as signal.Notify does not block when
+ sending to a channel. For example:
+
+c := make(chan os.Signal) +// signals are sent on c before the channel is read from. +// This signal may be dropped as c is unbuffered. +signal.Notify(c, os.Interrupt) ++
- TODO: complete the Vet section
+ Users of signal.Notify should use channels with sufficient buffer space to keep up with the
+ expected signal rate.
+
+ The vet tool now warns about methods named As, Is or Unwrap
+ on types implementing the error interface that have a different signature than the
+ one expected by the errors package. The errors.{As,Is,Unwrap} functions
+ expect such methods to implement either Is(error) bool,
+ As(interface{}) bool, or Unwrap() error
+ respectively. The functions errors.{As,Is,Unwrap} will ignore methods with the same
+ names but a different signature. For example:
+
+type MyError struct { hint string }
+func (m MyError) Error() string { ... } // MyError implements error.
+func (MyError) Is(target interface{}) bool { ... } // target is interface{} instead of error.
+func Foo() bool {
+ x, y := MyError{"A"}, MyError{"B"}
+ return errors.Is(x, y) // returns false as x != y and MyError does not have an `Is(error) bool` function.
+}
+
+
+
+ The cover tool now uses an optimized parser
+ from golang.org/x/tools/cover, which may be noticeably faster
+ when parsing large coverage profiles.
linux/amd64,
darwin/amd64, windows/amd64 ports). For a
- representative set of Go packages and programs, benchmarking has shown
+ representative set of Go packages and programs, benchmarking has shown
performance improvements of about 5%, and a typical reduction in binary size
of about 2%.
@@ -262,7 +435,8 @@ Do not send CLs removing the interior tags from such phrases.
by commas. Aggregate-typed (struct, array, string, slice, interface, and complex)
arguments are delimited by curly braces. A caveat is that the value of an
argument that only lives in a register and is not stored to memory may be
- inaccurate. Results (which were usually inaccurate) are no longer printed.
+ inaccurate. Function return values (which were usually inaccurate) are no longer
+ printed.
@@ -275,34 +449,6 @@ Do not send CLs removing the interior tags from such phrases.
- TODO: complete the Core library section -
- -- (*Conn).HandshakeContext was added to - allow the user to control cancellation of an in-progress TLS Handshake. - The context provided is propagated into the - ClientHelloInfo - and CertificateRequestInfo - structs and accessible through the new - (*ClientHelloInfo).Context - and - - (*CertificateRequestInfo).Context - methods respectively. Canceling the context after the handshake has finished - has no effect. -
- -
- When Config.NextProtos is set, servers now
- enforce that there is an overlap between the configured protocols and the protocols
- advertised by the client, if any. If there is no overlap the connection is closed
- with the no_application_protocol alert, as required by RFC 7301.
-
@@ -312,6 +458,67 @@ Do not send CLs removing the interior tags from such phrases. runtime/cgo.Handle for more information.
+
+ The net/url and net/http packages used to accept
+ ";" (semicolon) as a setting separator in URL queries, in
+ addition to "&" (ampersand). Now, settings with non-percent-encoded
+ semicolons are rejected and net/http servers will log a warning to
+ Server.ErrorLog
+ when encountering one in a request URL.
+
+ For example, before Go 1.17 the Query
+ method of the URL example?a=1;b=2&c=3 would have returned
+ map[a:[1] b:[2] c:[3]], while now it returns map[c:[3]].
+
+ When encountering such a query string,
+ URL.Query
+ and
+ Request.FormValue
+ ignore any settings that contain a semicolon,
+ ParseQuery
+ returns the remaining settings and an error, and
+ Request.ParseForm
+ and
+ Request.ParseMultipartForm
+ return an error but still set Request fields based on the
+ remaining settings.
+
+ net/http users can restore the original behavior by using the new
+ AllowQuerySemicolons
+ handler wrapper. This will also suppress the ErrorLog warning.
+ Note that accepting semicolons as query separators can lead to security issues
+ if different systems interpret cache keys differently.
+ See issue 25192 for more information.
+
+ When Config.NextProtos
+ is set, servers now enforce that there is an overlap between the configured
+ protocols and the ALPN protocols advertised by the client, if any. If there is
+ no mutually supported protocol, the connection is closed with the
+ no_application_protocol alert, as required by RFC 7301. This
+ helps mitigate the ALPACA cross-protocol attack.
+
+ As an exception, when the value "h2" is included in the server's
+ Config.NextProtos, HTTP/1.1 clients will be allowed to connect as
+ if they didn't support ALPN.
+ See issue 46310 for more information.
+
@@ -365,13 +572,126 @@ Do not send CLs removing the interior tags from such phrases. -
- TODO: https://golang.org/cl/302230: fix salt length calculation with PSSSaltLengthAuto +
+ The crypto/ed25519 package has been rewritten, and all
+ operations are now approximately twice as fast on amd64 and arm64.
+ The observable behavior has not otherwise changed.
+ CurveParams
+ methods now automatically invoke faster and safer dedicated
+ implementations for known curves (P-224, P-256, and P-521) when
+ available. Note that this is a best-effort approach and applications
+ should avoid using the generic, not constant-time CurveParams
+ methods and instead use dedicated
+ Curve implementations
+ such as P256.
+
+ The P521 curve
+ implementation has been rewritten using code generated by the
+ fiat-crypto project,
+ which is based on a formally-verified model of the arithmetic
+ operations. It is now constant-time and three times faster on amd64 and
+ arm64. The observable behavior has not otherwise changed.
+
+ The crypto/rand package now uses the getentropy
+ syscall on macOS and the getrandom syscall on Solaris,
+ Illumos, and DragonFlyBSD.
+
+ The new Conn.HandshakeContext
+ method allows the user to control cancellation of an in-progress TLS
+ handshake. The provided context is accessible from various callbacks through the new
+ ClientHelloInfo.Context and
+ CertificateRequestInfo.Context
+ methods. Canceling the context after the handshake has finished has no effect.
+
+ Cipher suite ordering is now handled entirely by the
+ crypto/tls package. Currently, cipher suites are sorted based
+ on their security, performance, and hardware support taking into account
+ both the local and peer's hardware. The order of the
+ Config.CipherSuites
+ field is now ignored, as well as the
+ Config.PreferServerCipherSuites
+ field. Note that Config.CipherSuites still allows
+ applications to choose what TLS 1.0–1.2 cipher suites to enable.
+
+ The 3DES cipher suites have been moved to
+ InsecureCipherSuites
+ due to fundamental block size-related
+ weakness. They are still enabled by default but only as a last resort,
+ thanks to the cipher suite ordering change above.
+
+ Beginning in the next release, Go 1.18, the
+ Config.MinVersion
+ for crypto/tls clients will default to TLS 1.2, disabling TLS 1.0
+ and TLS 1.1 by default. Applications will be able to override the change by
+ explicitly setting Config.MinVersion.
+ This will not affect crypto/tls servers.
+
+ CreateCertificate
+ now returns an error if the provided private key doesn't match the
+ parent's public key, if any. The resulting certificate would have failed
+ to verify.
+
+ The temporary GODEBUG=x509ignoreCN=0 flag has been removed.
+
+ ParseCertificate
+ has been rewritten, and now consumes ~70% fewer resources. The observable
+ behavior has not otherwise changed, except for error messages.
+
+ On BSD systems, /etc/ssl/certs is now searched for trusted
+ roots. This adds support for the new system trusted certificate store in
+ FreeBSD 12.2+.
+
+ Beginning in the next release, Go 1.18, crypto/x509 will
+ reject certificates signed with the SHA-1 hash function. This doesn't
+ apply to self-signed root certificates. Practical attacks against SHA-1
+ have been demonstrated in 2017 and publicly
+ trusted Certificate Authorities have not issued SHA-1 certificates since 2015.
+
+ When a comment appears within a
+ Directive, it is now replaced
+ with a single space instead of being completely elided.
+
+ Invalid element or attribute names with leading, trailing, or multiple
+ colons are now stored unmodified into the
+ Name.Local field.
+
@@ -444,6 +780,45 @@ Do not send CLs removing the interior tags from such phrases.
+ The Source and
+ Node functions now
+ synchronize //go:build lines with // +build
+ lines. If a file only has // +build lines, they will be
+ moved to the appropriate location in the file, and matching
+ //go:build lines will be added. Otherwise,
+ // +build lines will be overwritten based on any existing
+ //go:build lines. For more information, see
+ https://golang.org/design/draft-gobuild.
+
+ The new SkipObjectResolution
+ Mode value instructs the parser not to resolve identifiers to
+ their declaration. This may improve parsing speed.
+
+ The concrete image types (RGBA, Gray16 and so on)
+ now implement a new RGBA64Image
+ interface. Those concrete types, other than the chroma-subsampling
+ related YCbCr and NYCbCrA, also now implement
+ draw.RGBA64Image, a
+ new interface in the image/draw package.
+
@@ -472,6 +847,20 @@ Do not send CLs removing the interior tags from such phrases.
+ Part.FileName
+ now applies
+ filepath.Base to the
+ return value. This mitigates potential path traversal vulnerabilities in
+ applications that accept multipart messages, such as net/http
+ servers that call
+ Request.FormFile.
+
@@ -490,6 +879,16 @@ Do not send CLs removing the interior tags from such phrases.
ParseError error type now implement
the net.Error interface.
+ The ParseIP and ParseCIDR
+ functions now reject IPv4 addresses which contain decimal components with leading zeros.
+
+ These components were always interpreted as decimal, but some operating systems treat them as octal.
+ This mismatch could hypothetically lead to security issues if a Go application was used to validate IP addresses
+ which were then used in their original form with non-Go applications which interpreted components as octal. Generally,
+ it is advisable to always re-encode values after validation, which avoids this class of parser misalignment issues.
+
ReadRequest function
now returns an error when the request has multiple Host headers.
+
+
+ When producing a redirect to the cleaned version of a URL,
+ ServeMux now always
+ uses relative URLs in the Location header. Previously it
+ would echo the full URL of the request, which could lead to unintended
+ redirects if the client could be made to send an absolute request URL.
+
+ When interpreting certain HTTP headers handled by net/http,
+ non-ASCII characters are now ignored or rejected.
+
+ If
+ Request.ParseForm
+ returns an error when called by
+ Request.ParseMultipartForm,
+ the latter now continues populating
+ Request.MultipartForm
+ before returning it.
+
- ResponseRecorder.WriteHeader>
+ ResponseRecorder.WriteHeader
now panics when the provided code is not a valid three-digit HTTP status code.
- This matches the behavior of ResponseWriter>
+ This matches the behavior of ResponseWriter
implementations in the net/http package.
The File.WriteString method
- has been optimized to no longer make a copy of the input string.
+ has been optimized to not make a copy of the input string.
ArrayOf function now panics when
called with a negative length.
+
+
+ Checking the Type.ConvertibleTo method
+ is no longer sufficient to guarantee that a call to
+ Value.Convert will not panic.
+ It may panic when converting `[]T` to `*[N]T` if the slice's length is less than N.
+ See the language changes section above.
+
+ Block profiles are no longer biased to favor infrequent long events over + frequent short events. +
+- TODO: https://golang.org/cl/170079: implement Ryū-like algorithm for fixed precision ftoa -
- -- TODO: https://golang.org/cl/170080: Implement Ryū algorithm for ftoa shortest mode +
+ The strconv package now uses Ulf Adams's Ryū algorithm for formatting floating-point numbers.
+ This algorithm improves performance on most inputs and is more than 99% faster on worst-case inputs.
@@ -659,9 +1095,8 @@ Do not send CLs removing the interior tags from such phrases.
- TODO: https://golang.org/cl/310033: add -shuffle=off|on|N to alter the execution order of tests and benchmarks
+ Added a new testing flag -shuffle which controls the execution order of tests and benchmarks.
The new
T.Setenv
diff --git a/doc/go_spec.html b/doc/go_spec.html
index e59b3554f2..b59b37fd55 100644
--- a/doc/go_spec.html
+++ b/doc/go_spec.html
@@ -1,6 +1,6 @@
@@ -4670,7 +4670,7 @@ The following built-in functions are not permitted in statement context:
append cap complex imag len make new real -unsafe.Alignof unsafe.Offsetof unsafe.Sizeof +unsafe.Add unsafe.Alignof unsafe.Offsetof unsafe.Sizeof unsafe.Slice
@@ -4909,7 +4909,7 @@ if x := f(); x < y {
"Switch" statements provide multi-way execution.
-An expression or type specifier is compared to the "cases"
+An expression or type is compared to the "cases"
inside the "switch" to determine which branch
to execute.
@@ -5020,7 +5020,7 @@ floating point, or string constants in case expressions.
A type switch compares types rather than values. It is otherwise similar
to an expression switch. It is marked by a special switch expression that
has the form of a type assertion
-using the reserved word type rather than an actual type:
+using the keyword type rather than an actual type:
diff --git a/misc/cgo/errors/errors_test.go b/misc/cgo/errors/errors_test.go
index a077b59478..68a30a44fe 100644
--- a/misc/cgo/errors/errors_test.go
+++ b/misc/cgo/errors/errors_test.go
@@ -40,7 +40,8 @@ func check(t *testing.T, file string) {
if len(frags) == 1 {
continue
}
- re, err := regexp.Compile(string(frags[1]))
+ frag := fmt.Sprintf(":%d:.*%s", i+1, frags[1])
+ re, err := regexp.Compile(frag)
if err != nil {
t.Errorf("Invalid regexp after `ERROR HERE: `: %#q", frags[1])
continue
diff --git a/misc/cgo/errors/testdata/err2.go b/misc/cgo/errors/testdata/err2.go
index 1d22401aee..a90598fe35 100644
--- a/misc/cgo/errors/testdata/err2.go
+++ b/misc/cgo/errors/testdata/err2.go
@@ -40,15 +40,15 @@ func main() {
C.foop = x // ERROR HERE
// issue 13129: used to output error about C.unsignedshort with CC=clang
- var x C.ushort
- x = int(0) // ERROR HERE: C\.ushort
+ var x1 C.ushort
+ x1 = int(0) // ERROR HERE: C\.ushort
// issue 13423
_ = C.fopen() // ERROR HERE
// issue 13467
- var x rune = '✈'
- var _ rune = C.transform(x) // ERROR HERE: C\.int
+ var x2 rune = '✈'
+ var _ rune = C.transform(x2) // ERROR HERE: C\.int
// issue 13635: used to output error about C.unsignedchar.
// This test tests all such types.
@@ -91,10 +91,10 @@ func main() {
// issue 26745
_ = func(i int) int {
- return C.i + 1 // ERROR HERE: :13
+ return C.i + 1 // ERROR HERE: 14
}
_ = func(i int) {
- C.fi(i) // ERROR HERE: :6
+ C.fi(i) // ERROR HERE: 7
}
C.fi = C.fi // ERROR HERE
diff --git a/misc/cgo/testcshared/cshared_test.go b/misc/cgo/testcshared/cshared_test.go
index 90d8c365e6..19ad8c76a8 100644
--- a/misc/cgo/testcshared/cshared_test.go
+++ b/misc/cgo/testcshared/cshared_test.go
@@ -292,11 +292,60 @@ func createHeaders() error {
"-installsuffix", "testcshared",
"-o", libgoname,
filepath.Join(".", "libgo", "libgo.go")}
+ if GOOS == "windows" && strings.HasSuffix(args[6], ".a") {
+ args[6] = strings.TrimSuffix(args[6], ".a") + ".dll"
+ }
cmd = exec.Command(args[0], args[1:]...)
out, err = cmd.CombinedOutput()
if err != nil {
return fmt.Errorf("command failed: %v\n%v\n%s\n", args, err, out)
}
+ if GOOS == "windows" {
+ // We can't simply pass -Wl,--out-implib, because this relies on having imports from multiple packages,
+ // which results in the linkers output implib getting overwritten at each step. So instead build the
+ // import library the traditional way, using a def file.
+ err = os.WriteFile("libgo.def",
+ []byte("LIBRARY libgo.dll\nEXPORTS\n\tDidInitRun\n\tDidMainRun\n\tDivu\n\tFromPkg\n\t_cgo_dummy_export\n"),
+ 0644)
+ if err != nil {
+ return fmt.Errorf("unable to write def file: %v", err)
+ }
+ out, err = exec.Command(cc[0], append(cc[1:], "-print-prog-name=dlltool")...).CombinedOutput()
+ if err != nil {
+ return fmt.Errorf("unable to find dlltool path: %v\n%s\n", err, out)
+ }
+ args := []string{strings.TrimSpace(string(out)), "-D", args[6], "-l", libgoname, "-d", "libgo.def"}
+
+ // This is an unfortunate workaround for https://github.com/mstorsjo/llvm-mingw/issues/205 in which
+ // we basically reimplement the contents of the dlltool.sh wrapper: https://git.io/JZFlU
+ dlltoolContents, err := os.ReadFile(args[0])
+ if err != nil {
+ return fmt.Errorf("unable to read dlltool: %v\n", err)
+ }
+ if bytes.HasPrefix(dlltoolContents, []byte("#!/bin/sh")) && bytes.Contains(dlltoolContents, []byte("llvm-dlltool")) {
+ base, name := filepath.Split(args[0])
+ args[0] = filepath.Join(base, "llvm-dlltool")
+ var machine string
+ switch strings.SplitN(name, "-", 2)[0] {
+ case "i686":
+ machine = "i386"
+ case "x86_64":
+ machine = "i386:x86-64"
+ case "armv7":
+ machine = "arm"
+ case "aarch64":
+ machine = "arm64"
+ }
+ if len(machine) > 0 {
+ args = append(args, "-m", machine)
+ }
+ }
+
+ out, err = exec.Command(args[0], args[1:]...).CombinedOutput()
+ if err != nil {
+ return fmt.Errorf("unable to run dlltool to create import library: %v\n%s\n", err, out)
+ }
+ }
if runtime.GOOS != GOOS && GOOS == "android" {
args = append(adbCmd(), "push", libgoname, fmt.Sprintf("%s/%s", androiddir, libgoname))
@@ -400,7 +449,7 @@ func main() {
defer f.Close()
section := f.Section(".edata")
if section == nil {
- t.Fatalf(".edata section is not present")
+ t.Skip(".edata section is not present")
}
// TODO: deduplicate this struct from cmd/link/internal/ld/pe.go
@@ -749,7 +798,12 @@ func TestGo2C2Go(t *testing.T) {
defer os.RemoveAll(tmpdir)
lib := filepath.Join(tmpdir, "libtestgo2c2go."+libSuffix)
- run(t, nil, "go", "build", "-buildmode=c-shared", "-o", lib, "./go2c2go/go")
+ var env []string
+ if GOOS == "windows" && strings.HasSuffix(lib, ".a") {
+ env = append(env, "CGO_LDFLAGS=-Wl,--out-implib,"+lib, "CGO_LDFLAGS_ALLOW=.*")
+ lib = strings.TrimSuffix(lib, ".a") + ".dll"
+ }
+ run(t, env, "go", "build", "-buildmode=c-shared", "-o", lib, "./go2c2go/go")
cgoCflags := os.Getenv("CGO_CFLAGS")
if cgoCflags != "" {
diff --git a/src/cmd/asm/internal/asm/parse.go b/src/cmd/asm/internal/asm/parse.go
index ab48632a44..4cddcf48a4 100644
--- a/src/cmd/asm/internal/asm/parse.go
+++ b/src/cmd/asm/internal/asm/parse.go
@@ -1003,7 +1003,8 @@ func (p *Parser) registerIndirect(a *obj.Addr, prefix rune) {
p.errorf("unimplemented two-register form")
}
a.Index = r1
- if scale != 0 && p.arch.Family == sys.ARM64 {
+ if scale != 0 && scale != 1 && p.arch.Family == sys.ARM64 {
+ // Support (R1)(R2) (no scaling) and (R1)(R2*1).
p.errorf("arm64 doesn't support scaled register format")
} else {
a.Scale = int16(scale)
diff --git a/src/cmd/asm/internal/asm/testdata/arm64.s b/src/cmd/asm/internal/asm/testdata/arm64.s
index 1146c1a789..5f1e68545b 100644
--- a/src/cmd/asm/internal/asm/testdata/arm64.s
+++ b/src/cmd/asm/internal/asm/testdata/arm64.s
@@ -547,6 +547,7 @@ TEXT foo(SB), DUPOK|NOSPLIT, $-8
// shifted or extended register offset.
MOVD (R2)(R6.SXTW), R4 // 44c866f8
MOVD (R3)(R6), R5 // 656866f8
+ MOVD (R3)(R6*1), R5 // 656866f8
MOVD (R2)(R6), R4 // 446866f8
MOVWU (R19)(R20<<2), R20 // 747a74b8
MOVD (R2)(R6<<3), R4 // 447866f8
@@ -579,6 +580,7 @@ TEXT foo(SB), DUPOK|NOSPLIT, $-8
MOVB R4, (R2)(R6.SXTX) // 44e82638
MOVB R8, (R3)(R9.UXTW) // 68482938
MOVB R10, (R5)(R8) // aa682838
+ MOVB R10, (R5)(R8*1) // aa682838
MOVH R11, (R2)(R7.SXTW<<1) // 4bd82778
MOVH R5, (R1)(R2<<1) // 25782278
MOVH R7, (R2)(R5.SXTX<<1) // 47f82578
diff --git a/src/cmd/compile/internal/abi/abiutils.go b/src/cmd/compile/internal/abi/abiutils.go
index cb8e9d7b0f..b8ea1955d1 100644
--- a/src/cmd/compile/internal/abi/abiutils.go
+++ b/src/cmd/compile/internal/abi/abiutils.go
@@ -449,7 +449,7 @@ func (config *ABIConfig) ABIAnalyze(t *types.Type, setNname bool) *ABIParamResul
// parameterUpdateMu protects the Offset field of function/method parameters (a subset of structure Fields)
var parameterUpdateMu sync.Mutex
-// FieldOffsetOf returns a concurency-safe version of f.Offset
+// FieldOffsetOf returns a concurrency-safe version of f.Offset
func FieldOffsetOf(f *types.Field) int64 {
parameterUpdateMu.Lock()
defer parameterUpdateMu.Unlock()
diff --git a/src/cmd/compile/internal/gc/obj.go b/src/cmd/compile/internal/gc/obj.go
index 0b10cb8a9e..55a0ab7da7 100644
--- a/src/cmd/compile/internal/gc/obj.go
+++ b/src/cmd/compile/internal/gc/obj.go
@@ -148,6 +148,7 @@ func dumpdata() {
if reflectdata.ZeroSize > 0 {
zero := base.PkgLinksym("go.map", "zero", obj.ABI0)
objw.Global(zero, int32(reflectdata.ZeroSize), obj.DUPOK|obj.RODATA)
+ zero.Set(obj.AttrContentAddressable, true)
}
staticdata.WriteFuncSyms()
diff --git a/src/cmd/compile/internal/noder/noder.go b/src/cmd/compile/internal/noder/noder.go
index 4c7c9fc322..5fcad096c2 100644
--- a/src/cmd/compile/internal/noder/noder.go
+++ b/src/cmd/compile/internal/noder/noder.go
@@ -882,9 +882,6 @@ func (p *noder) typeExpr(typ syntax.Expr) ir.Ntype {
if n == nil {
return nil
}
- if _, ok := n.(ir.Ntype); !ok {
- ir.Dump("NOT NTYPE", n)
- }
return n.(ir.Ntype)
}
diff --git a/src/cmd/compile/internal/reflectdata/reflect.go b/src/cmd/compile/internal/reflectdata/reflect.go
index b3688fca67..e07294be0f 100644
--- a/src/cmd/compile/internal/reflectdata/reflect.go
+++ b/src/cmd/compile/internal/reflectdata/reflect.go
@@ -1475,8 +1475,8 @@ func (a typesByString) Less(i, j int) bool {
// will be equal for the above checks, but different in DWARF output.
// Sort by source position to ensure deterministic order.
// See issues 27013 and 30202.
- if a[i].t.Kind() == types.TINTER && a[i].t.Methods().Len() > 0 {
- return a[i].t.Methods().Index(0).Pos.Before(a[j].t.Methods().Index(0).Pos)
+ if a[i].t.Kind() == types.TINTER && a[i].t.AllMethods().Len() > 0 {
+ return a[i].t.AllMethods().Index(0).Pos.Before(a[j].t.AllMethods().Index(0).Pos)
}
return false
}
diff --git a/src/cmd/compile/internal/ssagen/ssa.go b/src/cmd/compile/internal/ssagen/ssa.go
index 004e084f72..f1dc56e729 100644
--- a/src/cmd/compile/internal/ssagen/ssa.go
+++ b/src/cmd/compile/internal/ssagen/ssa.go
@@ -3174,7 +3174,7 @@ func (s *state) expr(n ir.Node) *ssa.Value {
arrlen := s.constInt(types.Types[types.TINT], n.Type().Elem().NumElem())
cap := s.newValue1(ssa.OpSliceLen, types.Types[types.TINT], v)
s.boundsCheck(arrlen, cap, ssa.BoundsConvert, false)
- return s.newValue1(ssa.OpSlicePtrUnchecked, types.Types[types.TINT], v)
+ return s.newValue1(ssa.OpSlicePtrUnchecked, n.Type(), v)
case ir.OCALLFUNC:
n := n.(*ir.CallExpr)
diff --git a/src/cmd/compile/internal/typecheck/const.go b/src/cmd/compile/internal/typecheck/const.go
index 5a35eeade9..761b043794 100644
--- a/src/cmd/compile/internal/typecheck/const.go
+++ b/src/cmd/compile/internal/typecheck/const.go
@@ -633,6 +633,17 @@ func defaultlit2(l ir.Node, r ir.Node, force bool) (ir.Node, ir.Node) {
if l.Type() == nil || r.Type() == nil {
return l, r
}
+
+ if !l.Type().IsInterface() && !r.Type().IsInterface() {
+ // Can't mix bool with non-bool, string with non-string.
+ if l.Type().IsBoolean() != r.Type().IsBoolean() {
+ return l, r
+ }
+ if l.Type().IsString() != r.Type().IsString() {
+ return l, r
+ }
+ }
+
if !l.Type().IsUntyped() {
r = convlit(r, l.Type())
return l, r
@@ -647,17 +658,10 @@ func defaultlit2(l ir.Node, r ir.Node, force bool) (ir.Node, ir.Node) {
return l, r
}
- // Can't mix bool with non-bool, string with non-string, or nil with anything (untyped).
- if l.Type().IsBoolean() != r.Type().IsBoolean() {
- return l, r
- }
- if l.Type().IsString() != r.Type().IsString() {
- return l, r
- }
+ // Can't mix nil with anything untyped.
if ir.IsNil(l) || ir.IsNil(r) {
return l, r
}
-
t := defaultType(mixUntyped(l.Type(), r.Type()))
l = convlit(l, t)
r = convlit(r, t)
diff --git a/src/cmd/compile/internal/typecheck/func.go b/src/cmd/compile/internal/typecheck/func.go
index f381e1dbdc..a6dfbbf569 100644
--- a/src/cmd/compile/internal/typecheck/func.go
+++ b/src/cmd/compile/internal/typecheck/func.go
@@ -981,6 +981,12 @@ func tcRecover(n *ir.CallExpr) ir.Node {
// tcUnsafeAdd typechecks an OUNSAFEADD node.
func tcUnsafeAdd(n *ir.BinaryExpr) *ir.BinaryExpr {
+ if !types.AllowsGoVersion(curpkg(), 1, 17) {
+ base.ErrorfVers("go1.17", "unsafe.Add")
+ n.SetType(nil)
+ return n
+ }
+
n.X = AssignConv(Expr(n.X), types.Types[types.TUNSAFEPTR], "argument to unsafe.Add")
n.Y = DefaultLit(Expr(n.Y), types.Types[types.TINT])
if n.X.Type() == nil || n.Y.Type() == nil {
@@ -997,6 +1003,12 @@ func tcUnsafeAdd(n *ir.BinaryExpr) *ir.BinaryExpr {
// tcUnsafeSlice typechecks an OUNSAFESLICE node.
func tcUnsafeSlice(n *ir.BinaryExpr) *ir.BinaryExpr {
+ if !types.AllowsGoVersion(curpkg(), 1, 17) {
+ base.ErrorfVers("go1.17", "unsafe.Slice")
+ n.SetType(nil)
+ return n
+ }
+
n.X = Expr(n.X)
n.Y = Expr(n.Y)
if n.X.Type() == nil || n.Y.Type() == nil {
diff --git a/src/cmd/compile/internal/typecheck/stmt.go b/src/cmd/compile/internal/typecheck/stmt.go
index 175216f279..922a01bfbe 100644
--- a/src/cmd/compile/internal/typecheck/stmt.go
+++ b/src/cmd/compile/internal/typecheck/stmt.go
@@ -204,8 +204,20 @@ assignOK:
r.Use = ir.CallUseList
rtyp := r.Type()
+ mismatched := false
+ failed := false
for i := range lhs {
- assignType(i, rtyp.Field(i).Type)
+ result := rtyp.Field(i).Type
+ assignType(i, result)
+
+ if lhs[i].Type() == nil || result == nil {
+ failed = true
+ } else if lhs[i] != ir.BlankNode && !types.Identical(lhs[i].Type(), result) {
+ mismatched = true
+ }
+ }
+ if mismatched && !failed {
+ rewriteMultiValueCall(stmt, r)
}
return
}
diff --git a/src/cmd/compile/internal/typecheck/typecheck.go b/src/cmd/compile/internal/typecheck/typecheck.go
index 95f7b50259..bf52941b2c 100644
--- a/src/cmd/compile/internal/typecheck/typecheck.go
+++ b/src/cmd/compile/internal/typecheck/typecheck.go
@@ -945,16 +945,18 @@ func typecheckargs(n ir.InitNode) {
return
}
- // Rewrite f(g()) into t1, t2, ... = g(); f(t1, t2, ...).
-
// Save n as n.Orig for fmt.go.
if ir.Orig(n) == n {
n.(ir.OrigNode).SetOrig(ir.SepCopy(n))
}
- as := ir.NewAssignListStmt(base.Pos, ir.OAS2, nil, nil)
- as.Rhs.Append(list...)
+ // Rewrite f(g()) into t1, t2, ... = g(); f(t1, t2, ...).
+ rewriteMultiValueCall(n, list[0])
+}
+// rewriteMultiValueCall rewrites multi-valued f() to use temporaries,
+// so the backend wouldn't need to worry about tuple-valued expressions.
+func rewriteMultiValueCall(n ir.InitNode, call ir.Node) {
// If we're outside of function context, then this call will
// be executed during the generated init function. However,
// init.go hasn't yet created it. Instead, associate the
@@ -964,25 +966,40 @@ func typecheckargs(n ir.InitNode) {
if static {
ir.CurFunc = InitTodoFunc
}
- list = nil
- for _, f := range t.FieldSlice() {
- t := Temp(f.Type)
- as.PtrInit().Append(ir.NewDecl(base.Pos, ir.ODCL, t))
- as.Lhs.Append(t)
- list = append(list, t)
+
+ as := ir.NewAssignListStmt(base.Pos, ir.OAS2, nil, []ir.Node{call})
+ results := call.Type().FieldSlice()
+ list := make([]ir.Node, len(results))
+ for i, result := range results {
+ tmp := Temp(result.Type)
+ as.PtrInit().Append(ir.NewDecl(base.Pos, ir.ODCL, tmp))
+ as.Lhs.Append(tmp)
+ list[i] = tmp
}
if static {
ir.CurFunc = nil
}
+ n.PtrInit().Append(Stmt(as))
+
switch n := n.(type) {
+ default:
+ base.Fatalf("rewriteMultiValueCall %+v", n.Op())
case *ir.CallExpr:
n.Args = list
case *ir.ReturnStmt:
n.Results = list
+ case *ir.AssignListStmt:
+ if n.Op() != ir.OAS2FUNC {
+ base.Fatalf("rewriteMultiValueCall: invalid op %v", n.Op())
+ }
+ as.SetOp(ir.OAS2FUNC)
+ n.SetOp(ir.OAS2)
+ n.Rhs = make([]ir.Node, len(list))
+ for i, tmp := range list {
+ n.Rhs[i] = AssignConv(tmp, n.Lhs[i].Type(), "assignment")
+ }
}
-
- n.PtrInit().Append(Stmt(as))
}
func checksliceindex(l ir.Node, r ir.Node, tp *types.Type) bool {
diff --git a/src/cmd/compile/internal/types2/builtins.go b/src/cmd/compile/internal/types2/builtins.go
index b9e178dd57..f90e06f226 100644
--- a/src/cmd/compile/internal/types2/builtins.go
+++ b/src/cmd/compile/internal/types2/builtins.go
@@ -579,6 +579,11 @@ func (check *Checker) builtin(x *operand, call *syntax.CallExpr, id builtinId) (
case _Add:
// unsafe.Add(ptr unsafe.Pointer, len IntegerType) unsafe.Pointer
+ if !check.allowVersion(check.pkg, 1, 17) {
+ check.error(call.Fun, "unsafe.Add requires go1.17 or later")
+ return
+ }
+
check.assignment(x, Typ[UnsafePointer], "argument to unsafe.Add")
if x.mode == invalid {
return
@@ -675,6 +680,11 @@ func (check *Checker) builtin(x *operand, call *syntax.CallExpr, id builtinId) (
case _Slice:
// unsafe.Slice(ptr *T, len IntegerType) []T
+ if !check.allowVersion(check.pkg, 1, 17) {
+ check.error(call.Fun, "unsafe.Slice requires go1.17 or later")
+ return
+ }
+
typ := asPointer(x.typ)
if typ == nil {
check.errorf(x, invalidArg+"%s is not a pointer", x)
diff --git a/src/cmd/compile/internal/walk/walk.go b/src/cmd/compile/internal/walk/walk.go
index fe2c62cd4f..26da6e3145 100644
--- a/src/cmd/compile/internal/walk/walk.go
+++ b/src/cmd/compile/internal/walk/walk.go
@@ -313,7 +313,7 @@ func mayCall(n ir.Node) bool {
return true
case ir.OINDEX, ir.OSLICE, ir.OSLICEARR, ir.OSLICE3, ir.OSLICE3ARR, ir.OSLICESTR,
- ir.ODEREF, ir.ODOTPTR, ir.ODOTTYPE, ir.ODIV, ir.OMOD:
+ ir.ODEREF, ir.ODOTPTR, ir.ODOTTYPE, ir.ODIV, ir.OMOD, ir.OSLICE2ARRPTR:
// These ops might panic, make sure they are done
// before we start marshaling args for a call. See issue 16760.
return true
diff --git a/src/cmd/dist/test.go b/src/cmd/dist/test.go
index bc49c6d804..f2c4cf0b43 100644
--- a/src/cmd/dist/test.go
+++ b/src/cmd/dist/test.go
@@ -737,9 +737,9 @@ func (t *tester) registerTests() {
fn: func(dt *distTest) error {
cmd := t.addCmd(dt, "misc/swig/callback", t.goTest())
cmd.Env = append(os.Environ(),
- "CGO_CFLAGS=-flto",
- "CGO_CXXFLAGS=-flto",
- "CGO_LDFLAGS=-flto",
+ "CGO_CFLAGS=-flto -Wno-lto-type-mismatch",
+ "CGO_CXXFLAGS=-flto -Wno-lto-type-mismatch",
+ "CGO_LDFLAGS=-flto -Wno-lto-type-mismatch",
)
return nil
},
@@ -1057,7 +1057,7 @@ func (t *tester) supportedBuildmode(mode string) bool {
"darwin-amd64", "darwin-arm64",
"freebsd-amd64",
"android-arm", "android-arm64", "android-386",
- "windows-amd64", "windows-386":
+ "windows-amd64", "windows-386", "windows-arm64":
return true
}
return false
diff --git a/src/cmd/go.mod b/src/cmd/go.mod
index 1aa0320d07..cd03968eed 100644
--- a/src/cmd/go.mod
+++ b/src/cmd/go.mod
@@ -7,7 +7,7 @@ require (
github.com/ianlancetaylor/demangle v0.0.0-20200824232613-28f6c0f3b639 // indirect
golang.org/x/arch v0.0.0-20210502124803-cbf565b21d1e
golang.org/x/crypto v0.0.0-20210503195802-e9a32991a82e // indirect
- golang.org/x/mod v0.4.3-0.20210512182355-6088ed88cecd
+ golang.org/x/mod v0.4.3-0.20210608190319-0f08993efd8a
golang.org/x/sys v0.0.0-20210511113859-b0526f3d8744 // indirect
golang.org/x/term v0.0.0-20210503060354-a79de5458b56
golang.org/x/tools v0.1.2-0.20210519160823-49064d2332f9
diff --git a/src/cmd/go.sum b/src/cmd/go.sum
index eeb625fcf8..d728acaec9 100644
--- a/src/cmd/go.sum
+++ b/src/cmd/go.sum
@@ -13,8 +13,8 @@ golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8U
golang.org/x/crypto v0.0.0-20210503195802-e9a32991a82e h1:8foAy0aoO5GkqCvAEJ4VC4P3zksTg4X4aJCDpZzmgQI=
golang.org/x/crypto v0.0.0-20210503195802-e9a32991a82e/go.mod h1:P+XmwS30IXTQdn5tA2iutPOUgjI07+tq3H3K9MVA1s8=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
-golang.org/x/mod v0.4.3-0.20210512182355-6088ed88cecd h1:CuRnpyMrCCBulv0d/y0CswR4K0vGydgE3DZ2wYPIOo8=
-golang.org/x/mod v0.4.3-0.20210512182355-6088ed88cecd/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
+golang.org/x/mod v0.4.3-0.20210608190319-0f08993efd8a h1:e8qnjKz4EE6OjRki9wTadWSIogINvq10sMcuBRORxMY=
+golang.org/x/mod v0.4.3-0.20210608190319-0f08993efd8a/go.mod h1:5OXOZSfqPIIbmVBIIKWRFfZjPR0E5r58TLhUjH0a2Ro=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
diff --git a/src/cmd/go/alldocs.go b/src/cmd/go/alldocs.go
index 0655153969..3ff0ba1068 100644
--- a/src/cmd/go/alldocs.go
+++ b/src/cmd/go/alldocs.go
@@ -1189,13 +1189,17 @@
//
// Usage:
//
-// go mod graph
+// go mod graph [-go=version]
//
// Graph prints the module requirement graph (with replacements applied)
// in text form. Each line in the output has two space-separated fields: a module
// and one of its requirements. Each module is identified as a string of the form
// path@version, except for the main module, which has no @version suffix.
//
+// The -go flag causes graph to report the module graph as loaded by by the
+// given Go version, instead of the version indicated by the 'go' directive
+// in the go.mod file.
+//
// See https://golang.org/ref/mod#go-mod-graph for more about 'go mod graph'.
//
//
@@ -1573,7 +1577,7 @@
//
// A build constraint, also known as a build tag, is a line comment that begins
//
-// // +build
+// //go:build
//
// that lists the conditions under which a file should be included in the package.
// Constraints may appear in any kind of source file (not just Go), but
@@ -1581,30 +1585,20 @@
// only by blank lines and other line comments. These rules mean that in Go
// files a build constraint must appear before the package clause.
//
-// To distinguish build constraints from package documentation, a series of
-// build constraints must be followed by a blank line.
+// To distinguish build constraints from package documentation,
+// a build constraint should be followed by a blank line.
//
-// A build constraint is evaluated as the OR of space-separated options.
-// Each option evaluates as the AND of its comma-separated terms.
-// Each term consists of letters, digits, underscores, and dots.
-// A term may be negated with a preceding !.
-// For example, the build constraint:
+// A build constraint is evaluated as an expression containing options
+// combined by ||, &&, and ! operators and parentheses. Operators have
+// the same meaning as in Go.
//
-// // +build linux,386 darwin,!cgo
+// For example, the following build constraint constrains a file to
+// build when the "linux" and "386" constraints are satisfied, or when
+// "darwin" is satisfied and "cgo" is not:
//
-// corresponds to the boolean formula:
+// //go:build (linux && 386) || (darwin && !cgo)
//
-// (linux AND 386) OR (darwin AND (NOT cgo))
-//
-// A file may have multiple build constraints. The overall constraint is the AND
-// of the individual constraints. That is, the build constraints:
-//
-// // +build linux darwin
-// // +build amd64
-//
-// corresponds to the boolean formula:
-//
-// (linux OR darwin) AND amd64
+// It is an error for a file to have more than one //go:build line.
//
// During a particular build, the following words are satisfied:
//
@@ -1642,24 +1636,28 @@
//
// To keep a file from being considered for the build:
//
-// // +build ignore
+// //go:build ignore
//
// (any other unsatisfied word will work as well, but "ignore" is conventional.)
//
// To build a file only when using cgo, and only on Linux and OS X:
//
-// // +build linux,cgo darwin,cgo
+// //go:build cgo && (linux || darwin)
//
// Such a file is usually paired with another file implementing the
// default functionality for other systems, which in this case would
// carry the constraint:
//
-// // +build !linux,!darwin !cgo
+// //go:build !(cgo && (linux || darwin))
//
// Naming a file dns_windows.go will cause it to be included only when
// building the package for Windows; similarly, math_386.s will be included
// only when building the package for 32-bit x86.
//
+// Go versions 1.16 and earlier used a different syntax for build constraints,
+// with a "// +build" prefix. The gofmt command will add an equivalent //go:build
+// constraint when encountering the older syntax.
+//
//
// Build modes
//
@@ -1898,6 +1896,9 @@
// GOMIPS64
// For GOARCH=mips64{,le}, whether to use floating point instructions.
// Valid values are hardfloat (default), softfloat.
+// GOPPC64
+// For GOARCH=ppc64{,le}, the target ISA (Instruction Set Architecture).
+// Valid values are power8 (default), power9.
// GOWASM
// For GOARCH=wasm, comma-separated list of experimental WebAssembly features to use.
// Valid values are satconv, signext.
diff --git a/src/cmd/go/go_test.go b/src/cmd/go/go_test.go
index a059a6dd90..c0c86ab9f5 100644
--- a/src/cmd/go/go_test.go
+++ b/src/cmd/go/go_test.go
@@ -72,7 +72,6 @@ func tooSlow(t *testing.T) {
// (temp) directory.
var testGOROOT string
-var testCC string
var testGOCACHE string
var testGo string
@@ -179,13 +178,6 @@ func TestMain(m *testing.M) {
os.Exit(2)
}
- out, err = exec.Command(gotool, "env", "CC").CombinedOutput()
- if err != nil {
- fmt.Fprintf(os.Stderr, "could not find testing CC: %v\n%s", err, out)
- os.Exit(2)
- }
- testCC = strings.TrimSpace(string(out))
-
cmd := exec.Command(testGo, "env", "CGO_ENABLED")
cmd.Stderr = new(strings.Builder)
if out, err := cmd.Output(); err != nil {
@@ -2185,7 +2177,7 @@ func testBuildmodePIE(t *testing.T, useCgo, setBuildmodeToPIE bool) {
// See https://sourceware.org/bugzilla/show_bug.cgi?id=19011
section := f.Section(".edata")
if section == nil {
- t.Fatalf(".edata section is not present")
+ t.Skip(".edata section is not present")
}
// TODO: deduplicate this struct from cmd/link/internal/ld/pe.go
type IMAGE_EXPORT_DIRECTORY struct {
diff --git a/src/cmd/go/internal/cfg/cfg.go b/src/cmd/go/internal/cfg/cfg.go
index 21a56d6df6..36b0658b26 100644
--- a/src/cmd/go/internal/cfg/cfg.go
+++ b/src/cmd/go/internal/cfg/cfg.go
@@ -81,6 +81,14 @@ func defaultContext() build.Context {
ctxt.GOOS = envOr("GOOS", ctxt.GOOS)
ctxt.GOARCH = envOr("GOARCH", ctxt.GOARCH)
+ // The experiments flags are based on GOARCH, so they may
+ // need to change. TODO: This should be cleaned up.
+ buildcfg.UpdateExperiments(ctxt.GOOS, ctxt.GOARCH, envOr("GOEXPERIMENT", buildcfg.DefaultGOEXPERIMENT))
+ ctxt.ToolTags = nil
+ for _, exp := range buildcfg.EnabledExperiments() {
+ ctxt.ToolTags = append(ctxt.ToolTags, "goexperiment."+exp)
+ }
+
// The go/build rule for whether cgo is enabled is:
// 1. If $CGO_ENABLED is set, respect it.
// 2. Otherwise, if this is a cross-compile, disable cgo.
diff --git a/src/cmd/go/internal/help/helpdoc.go b/src/cmd/go/internal/help/helpdoc.go
index 2f86e4195d..b552777e3e 100644
--- a/src/cmd/go/internal/help/helpdoc.go
+++ b/src/cmd/go/internal/help/helpdoc.go
@@ -598,6 +598,9 @@ Architecture-specific environment variables:
GOMIPS64
For GOARCH=mips64{,le}, whether to use floating point instructions.
Valid values are hardfloat (default), softfloat.
+ GOPPC64
+ For GOARCH=ppc64{,le}, the target ISA (Instruction Set Architecture).
+ Valid values are power8 (default), power9.
GOWASM
For GOARCH=wasm, comma-separated list of experimental WebAssembly features to use.
Valid values are satconv, signext.
@@ -784,7 +787,7 @@ var HelpBuildConstraint = &base.Command{
Long: `
A build constraint, also known as a build tag, is a line comment that begins
- // +build
+ //go:build
that lists the conditions under which a file should be included in the package.
Constraints may appear in any kind of source file (not just Go), but
@@ -792,30 +795,20 @@ they must appear near the top of the file, preceded
only by blank lines and other line comments. These rules mean that in Go
files a build constraint must appear before the package clause.
-To distinguish build constraints from package documentation, a series of
-build constraints must be followed by a blank line.
+To distinguish build constraints from package documentation,
+a build constraint should be followed by a blank line.
-A build constraint is evaluated as the OR of space-separated options.
-Each option evaluates as the AND of its comma-separated terms.
-Each term consists of letters, digits, underscores, and dots.
-A term may be negated with a preceding !.
-For example, the build constraint:
+A build constraint is evaluated as an expression containing options
+combined by ||, &&, and ! operators and parentheses. Operators have
+the same meaning as in Go.
- // +build linux,386 darwin,!cgo
+For example, the following build constraint constrains a file to
+build when the "linux" and "386" constraints are satisfied, or when
+"darwin" is satisfied and "cgo" is not:
-corresponds to the boolean formula:
+ //go:build (linux && 386) || (darwin && !cgo)
- (linux AND 386) OR (darwin AND (NOT cgo))
-
-A file may have multiple build constraints. The overall constraint is the AND
-of the individual constraints. That is, the build constraints:
-
- // +build linux darwin
- // +build amd64
-
-corresponds to the boolean formula:
-
- (linux OR darwin) AND amd64
+It is an error for a file to have more than one //go:build line.
During a particular build, the following words are satisfied:
@@ -853,22 +846,26 @@ in addition to ios tags and files.
To keep a file from being considered for the build:
- // +build ignore
+ //go:build ignore
(any other unsatisfied word will work as well, but "ignore" is conventional.)
To build a file only when using cgo, and only on Linux and OS X:
- // +build linux,cgo darwin,cgo
+ //go:build cgo && (linux || darwin)
Such a file is usually paired with another file implementing the
default functionality for other systems, which in this case would
carry the constraint:
- // +build !linux,!darwin !cgo
+ //go:build !(cgo && (linux || darwin))
Naming a file dns_windows.go will cause it to be included only when
building the package for Windows; similarly, math_386.s will be included
only when building the package for 32-bit x86.
+
+Go versions 1.16 and earlier used a different syntax for build constraints,
+with a "// +build" prefix. The gofmt command will add an equivalent //go:build
+constraint when encountering the older syntax.
`,
}
diff --git a/src/cmd/go/internal/imports/read.go b/src/cmd/go/internal/imports/read.go
index 5e270781d7..70d5190450 100644
--- a/src/cmd/go/internal/imports/read.go
+++ b/src/cmd/go/internal/imports/read.go
@@ -8,6 +8,7 @@ package imports
import (
"bufio"
+ "bytes"
"errors"
"io"
"unicode/utf8"
@@ -22,6 +23,19 @@ type importReader struct {
nerr int
}
+var bom = []byte{0xef, 0xbb, 0xbf}
+
+func newImportReader(b *bufio.Reader) *importReader {
+ // Remove leading UTF-8 BOM.
+ // Per https://golang.org/ref/spec#Source_code_representation:
+ // a compiler may ignore a UTF-8-encoded byte order mark (U+FEFF)
+ // if it is the first Unicode code point in the source text.
+ if leadingBytes, err := b.Peek(3); err == nil && bytes.Equal(leadingBytes, bom) {
+ b.Discard(3)
+ }
+ return &importReader{b: b}
+}
+
func isIdent(c byte) bool {
return 'A' <= c && c <= 'Z' || 'a' <= c && c <= 'z' || '0' <= c && c <= '9' || c == '_' || c >= utf8.RuneSelf
}
@@ -201,7 +215,7 @@ func (r *importReader) readImport(imports *[]string) {
// ReadComments is like io.ReadAll, except that it only reads the leading
// block of comments in the file.
func ReadComments(f io.Reader) ([]byte, error) {
- r := &importReader{b: bufio.NewReader(f)}
+ r := newImportReader(bufio.NewReader(f))
r.peekByte(true)
if r.err == nil && !r.eof {
// Didn't reach EOF, so must have found a non-space byte. Remove it.
@@ -213,7 +227,7 @@ func ReadComments(f io.Reader) ([]byte, error) {
// ReadImports is like io.ReadAll, except that it expects a Go file as input
// and stops reading the input once the imports have completed.
func ReadImports(f io.Reader, reportSyntaxError bool, imports *[]string) ([]byte, error) {
- r := &importReader{b: bufio.NewReader(f)}
+ r := newImportReader(bufio.NewReader(f))
r.readKeyword("package")
r.readIdent()
diff --git a/src/cmd/go/internal/imports/read_test.go b/src/cmd/go/internal/imports/read_test.go
index 6ea356f1ff..6a1a6524a1 100644
--- a/src/cmd/go/internal/imports/read_test.go
+++ b/src/cmd/go/internal/imports/read_test.go
@@ -66,6 +66,10 @@ var readImportsTests = []readTest{
`,
"",
},
+ {
+ "\ufeff𝔻" + `package p; import "x";ℙvar x = 1`,
+ "",
+ },
}
var readCommentsTests = []readTest{
@@ -81,6 +85,10 @@ var readCommentsTests = []readTest{
`ℙpackage p; import . "x"`,
"",
},
+ {
+ "\ufeff𝔻" + `ℙpackage p; import . "x"`,
+ "",
+ },
{
`// foo
@@ -90,6 +98,19 @@ var readCommentsTests = []readTest{
/*/ zot */
+ // asdf
+ ℙHello, world`,
+ "",
+ },
+ {
+ "\ufeff𝔻" + `// foo
+
+ /* bar */
+
+ /* quux */ // baz
+
+ /*/ zot */
+
// asdf
ℙHello, world`,
"",
@@ -107,6 +128,11 @@ func testRead(t *testing.T, tests []readTest, read func(io.Reader) ([]byte, erro
in = tt.in[:j] + tt.in[j+len("ℙ"):]
testOut = tt.in[:j]
}
+ d := strings.Index(tt.in, "𝔻")
+ if d >= 0 {
+ in = in[:d] + in[d+len("𝔻"):]
+ testOut = testOut[d+len("𝔻"):]
+ }
r := strings.NewReader(in)
buf, err := read(r)
if err != nil {
diff --git a/src/cmd/go/internal/list/list.go b/src/cmd/go/internal/list/list.go
index 53aaf311ec..7cb9ec6d94 100644
--- a/src/cmd/go/internal/list/list.go
+++ b/src/cmd/go/internal/list/list.go
@@ -724,8 +724,18 @@ func runList(ctx context.Context, cmd *base.Command, args []string) {
// Record non-identity import mappings in p.ImportMap.
for _, p := range pkgs {
- for i, srcPath := range p.Internal.RawImports {
- path := p.Imports[i]
+ nRaw := len(p.Internal.RawImports)
+ for i, path := range p.Imports {
+ var srcPath string
+ if i < nRaw {
+ srcPath = p.Internal.RawImports[i]
+ } else {
+ // This path is not within the raw imports, so it must be an import
+ // found only within CompiledGoFiles. Those paths are found in
+ // CompiledImports.
+ srcPath = p.Internal.CompiledImports[i-nRaw]
+ }
+
if path != srcPath {
if p.ImportMap == nil {
p.ImportMap = make(map[string]string)
diff --git a/src/cmd/go/internal/load/pkg.go b/src/cmd/go/internal/load/pkg.go
index 26115ff6a4..5126c46bbc 100644
--- a/src/cmd/go/internal/load/pkg.go
+++ b/src/cmd/go/internal/load/pkg.go
@@ -194,8 +194,8 @@ type PackageInternal struct {
// Unexported fields are not part of the public API.
Build *build.Package
Imports []*Package // this package's direct imports
- CompiledImports []string // additional Imports necessary when using CompiledGoFiles (all from standard library)
- RawImports []string // this package's original imports as they appear in the text of the program
+ CompiledImports []string // additional Imports necessary when using CompiledGoFiles (all from standard library); 1:1 with the end of PackagePublic.Imports
+ RawImports []string // this package's original imports as they appear in the text of the program; 1:1 with the end of PackagePublic.Imports
ForceLibrary bool // this package is a library (even if named "main")
CmdlineFiles bool // package built from files listed on command line
CmdlinePkg bool // package listed on command line
diff --git a/src/cmd/go/internal/load/test.go b/src/cmd/go/internal/load/test.go
index a84e3e6a6f..52e72c2774 100644
--- a/src/cmd/go/internal/load/test.go
+++ b/src/cmd/go/internal/load/test.go
@@ -116,7 +116,7 @@ func TestPackagesAndErrors(ctx context.Context, opts PackageOpts, p *Package, co
// Can't change that code, because that code is only for loading the
// non-test copy of a package.
ptestErr = &PackageError{
- ImportStack: testImportStack(stk[0], p1, p.ImportPath),
+ ImportStack: importCycleStack(p1, p.ImportPath),
Err: errors.New("import cycle not allowed in test"),
IsImportCycle: true,
}
@@ -375,22 +375,44 @@ func TestPackagesAndErrors(ctx context.Context, opts PackageOpts, p *Package, co
return pmain, ptest, pxtest
}
-func testImportStack(top string, p *Package, target string) []string {
- stk := []string{top, p.ImportPath}
-Search:
- for p.ImportPath != target {
- for _, p1 := range p.Internal.Imports {
- if p1.ImportPath == target || str.Contains(p1.Deps, target) {
- stk = append(stk, p1.ImportPath)
- p = p1
- continue Search
+// importCycleStack returns an import stack from p to the package whose import
+// path is target.
+func importCycleStack(p *Package, target string) []string {
+ // importerOf maps each import path to its importer nearest to p.
+ importerOf := map[string]string{p.ImportPath: ""}
+
+ // q is a breadth-first queue of packages to search for target.
+ // Every package added to q has a corresponding entry in pathTo.
+ //
+ // We search breadth-first for two reasons:
+ //
+ // 1. We want to report the shortest cycle.
+ //
+ // 2. If p contains multiple cycles, the first cycle we encounter might not
+ // contain target. To ensure termination, we have to break all cycles
+ // other than the first.
+ q := []*Package{p}
+
+ for len(q) > 0 {
+ p := q[0]
+ q = q[1:]
+ if path := p.ImportPath; path == target {
+ var stk []string
+ for path != "" {
+ stk = append(stk, path)
+ path = importerOf[path]
+ }
+ return stk
+ }
+ for _, dep := range p.Internal.Imports {
+ if _, ok := importerOf[dep.ImportPath]; !ok {
+ importerOf[dep.ImportPath] = p.ImportPath
+ q = append(q, dep)
}
}
- // Can't happen, but in case it does...
- stk = append(stk, "")
- break
}
- return stk
+
+ panic("lost path to cycle")
}
// recompileForTest copies and replaces certain packages in pmain's dependency
diff --git a/src/cmd/go/internal/modcmd/graph.go b/src/cmd/go/internal/modcmd/graph.go
index 77853304e9..903bd9970f 100644
--- a/src/cmd/go/internal/modcmd/graph.go
+++ b/src/cmd/go/internal/modcmd/graph.go
@@ -18,7 +18,7 @@ import (
)
var cmdGraph = &base.Command{
- UsageLine: "go mod graph",
+ UsageLine: "go mod graph [-go=version]",
Short: "print module requirement graph",
Long: `
Graph prints the module requirement graph (with replacements applied)
@@ -26,12 +26,21 @@ in text form. Each line in the output has two space-separated fields: a module
and one of its requirements. Each module is identified as a string of the form
path@version, except for the main module, which has no @version suffix.
+The -go flag causes graph to report the module graph as loaded by by the
+given Go version, instead of the version indicated by the 'go' directive
+in the go.mod file.
+
See https://golang.org/ref/mod#go-mod-graph for more about 'go mod graph'.
`,
Run: runGraph,
}
+var (
+ graphGo goVersionFlag
+)
+
func init() {
+ cmdGraph.Flag.Var(&graphGo, "go", "")
base.AddModCommonFlags(&cmdGraph.Flag)
}
@@ -41,7 +50,7 @@ func runGraph(ctx context.Context, cmd *base.Command, args []string) {
}
modload.ForceUseModules = true
modload.RootMode = modload.NeedRoot
- mg := modload.LoadModGraph(ctx)
+ mg := modload.LoadModGraph(ctx, graphGo.String())
w := bufio.NewWriter(os.Stdout)
defer w.Flush()
diff --git a/src/cmd/go/internal/modcmd/verify.go b/src/cmd/go/internal/modcmd/verify.go
index 5c321c783a..5a6eca32cf 100644
--- a/src/cmd/go/internal/modcmd/verify.go
+++ b/src/cmd/go/internal/modcmd/verify.go
@@ -54,7 +54,8 @@ func runVerify(ctx context.Context, cmd *base.Command, args []string) {
sem := make(chan token, runtime.GOMAXPROCS(0))
// Use a slice of result channels, so that the output is deterministic.
- mods := modload.LoadModGraph(ctx).BuildList()[1:]
+ const defaultGoVersion = ""
+ mods := modload.LoadModGraph(ctx, defaultGoVersion).BuildList()[1:]
errsChans := make([]<-chan []error, len(mods))
for i, mod := range mods {
diff --git a/src/cmd/go/internal/modfetch/cache.go b/src/cmd/go/internal/modfetch/cache.go
index f3b58a172a..b01b467413 100644
--- a/src/cmd/go/internal/modfetch/cache.go
+++ b/src/cmd/go/internal/modfetch/cache.go
@@ -152,7 +152,7 @@ func lockVersion(mod module.Version) (unlock func(), err error) {
// If err is nil, the caller MUST eventually call the unlock function.
func SideLock() (unlock func(), err error) {
if err := checkCacheDir(); err != nil {
- base.Fatalf("go: %v", err)
+ return nil, err
}
path := filepath.Join(cfg.GOMODCACHE, "cache", "lock")
diff --git a/src/cmd/go/internal/modget/get.go b/src/cmd/go/internal/modget/get.go
index 8eee723f89..9672e5598e 100644
--- a/src/cmd/go/internal/modget/get.go
+++ b/src/cmd/go/internal/modget/get.go
@@ -506,7 +506,8 @@ type versionReason struct {
func newResolver(ctx context.Context, queries []*query) *resolver {
// LoadModGraph also sets modload.Target, which is needed by various resolver
// methods.
- mg := modload.LoadModGraph(ctx)
+ const defaultGoVersion = ""
+ mg := modload.LoadModGraph(ctx, defaultGoVersion)
buildList := mg.BuildList()
initialVersion := make(map[string]string, len(buildList))
@@ -1153,6 +1154,7 @@ func (r *resolver) loadPackages(ctx context.Context, patterns []string, findPack
Tags: imports.AnyTags(),
VendorModulesInGOROOTSrc: true,
LoadTests: *getT,
+ AssumeRootsImported: true, // After 'go get foo', imports of foo should build.
SilencePackageErrors: true, // May be fixed by subsequent upgrades or downgrades.
}
@@ -1802,7 +1804,8 @@ func (r *resolver) updateBuildList(ctx context.Context, additions []module.Versi
return false
}
- r.buildList = modload.LoadModGraph(ctx).BuildList()
+ const defaultGoVersion = ""
+ r.buildList = modload.LoadModGraph(ctx, defaultGoVersion).BuildList()
r.buildListVersion = make(map[string]string, len(r.buildList))
for _, m := range r.buildList {
r.buildListVersion[m.Path] = m.Version
diff --git a/src/cmd/go/internal/modload/buildlist.go b/src/cmd/go/internal/modload/buildlist.go
index e5db41c748..604a57b437 100644
--- a/src/cmd/go/internal/modload/buildlist.go
+++ b/src/cmd/go/internal/modload/buildlist.go
@@ -403,11 +403,33 @@ func (mg *ModuleGraph) allRootsSelected() bool {
// LoadModGraph loads and returns the graph of module dependencies of the main module,
// without loading any packages.
//
+// If the goVersion string is non-empty, the returned graph is the graph
+// as interpreted by the given Go version (instead of the version indicated
+// in the go.mod file).
+//
// Modules are loaded automatically (and lazily) in LoadPackages:
// LoadModGraph need only be called if LoadPackages is not,
// typically in commands that care about modules but no particular package.
-func LoadModGraph(ctx context.Context) *ModuleGraph {
- rs, mg, err := expandGraph(ctx, LoadModFile(ctx))
+func LoadModGraph(ctx context.Context, goVersion string) *ModuleGraph {
+ rs := LoadModFile(ctx)
+
+ if goVersion != "" {
+ depth := modDepthFromGoVersion(goVersion)
+ if depth == eager && rs.depth != eager {
+ // Use newRequirements instead of convertDepth because convertDepth
+ // also updates roots; here, we want to report the unmodified roots
+ // even though they may seem inconsistent.
+ rs = newRequirements(eager, rs.rootModules, rs.direct)
+ }
+
+ mg, err := rs.Graph(ctx)
+ if err != nil {
+ base.Fatalf("go: %v", err)
+ }
+ return mg
+ }
+
+ rs, mg, err := expandGraph(ctx, rs)
if err != nil {
base.Fatalf("go: %v", err)
}
@@ -443,7 +465,7 @@ func expandGraph(ctx context.Context, rs *Requirements) (*Requirements, *ModuleG
// roots — but in a lazy module it may pull in previously-irrelevant
// transitive dependencies.
- newRS, rsErr := updateRoots(ctx, rs.direct, rs, nil, nil)
+ newRS, rsErr := updateRoots(ctx, rs.direct, rs, nil, nil, false)
if rsErr != nil {
// Failed to update roots, perhaps because of an error in a transitive
// dependency needed for the update. Return the original Requirements
@@ -517,11 +539,11 @@ func tidyRoots(ctx context.Context, rs *Requirements, pkgs []*loadPkg) (*Require
return tidyLazyRoots(ctx, rs.direct, pkgs)
}
-func updateRoots(ctx context.Context, direct map[string]bool, rs *Requirements, pkgs []*loadPkg, add []module.Version) (*Requirements, error) {
+func updateRoots(ctx context.Context, direct map[string]bool, rs *Requirements, pkgs []*loadPkg, add []module.Version, rootsImported bool) (*Requirements, error) {
if rs.depth == eager {
return updateEagerRoots(ctx, direct, rs, add)
}
- return updateLazyRoots(ctx, direct, rs, pkgs, add)
+ return updateLazyRoots(ctx, direct, rs, pkgs, add, rootsImported)
}
// tidyLazyRoots returns a minimal set of root requirements that maintains the
@@ -661,7 +683,7 @@ func tidyLazyRoots(ctx context.Context, direct map[string]bool, pkgs []*loadPkg)
//
// (See https://golang.org/design/36460-lazy-module-loading#invariants for more
// detail.)
-func updateLazyRoots(ctx context.Context, direct map[string]bool, rs *Requirements, pkgs []*loadPkg, add []module.Version) (*Requirements, error) {
+func updateLazyRoots(ctx context.Context, direct map[string]bool, rs *Requirements, pkgs []*loadPkg, add []module.Version, rootsImported bool) (*Requirements, error) {
roots := rs.rootModules
rootsUpgraded := false
@@ -688,6 +710,10 @@ func updateLazyRoots(ctx context.Context, direct map[string]bool, rs *Requiremen
//
// (This is the “import invariant” that makes lazy loading possible.)
+ case rootsImported && pkg.flags.has(pkgFromRoot):
+ // pkg is a transitive dependency of some root, and we are treating the
+ // roots as if they are imported by the main module (as in 'go get').
+
case pkg.flags.has(pkgIsRoot):
// pkg is a root of the package-import graph. (Generally this means that
// it matches a command-line argument.) We want future invocations of the
diff --git a/src/cmd/go/internal/modload/import.go b/src/cmd/go/internal/modload/import.go
index f76befcfe3..d2bbe5cbe0 100644
--- a/src/cmd/go/internal/modload/import.go
+++ b/src/cmd/go/internal/modload/import.go
@@ -178,11 +178,13 @@ func (e *ImportMissingSumError) Error() string {
// Importing package is unknown, or the missing package was named on the
// command line. Recommend 'go mod download' for the modules that could
// provide the package, since that shouldn't change go.mod.
- args := make([]string, len(e.mods))
- for i, mod := range e.mods {
- args[i] = mod.Path
+ if len(e.mods) > 0 {
+ args := make([]string, len(e.mods))
+ for i, mod := range e.mods {
+ args[i] = mod.Path
+ }
+ hint = fmt.Sprintf("; to add:\n\tgo mod download %s", strings.Join(args, " "))
}
- hint = fmt.Sprintf("; to add:\n\tgo mod download %s", strings.Join(args, " "))
} else {
// Importing package is known (common case). Recommend 'go get' on the
// current version of the importing package.
@@ -426,6 +428,15 @@ func queryImport(ctx context.Context, path string, rs *Requirements) (module.Ver
mv = module.ZeroPseudoVersion("v0")
}
}
+ mg, err := rs.Graph(ctx)
+ if err != nil {
+ return module.Version{}, err
+ }
+ if cmpVersion(mg.Selected(mp), mv) >= 0 {
+ // We can't resolve the import by adding mp@mv to the module graph,
+ // because the selected version of mp is already at least mv.
+ continue
+ }
mods = append(mods, module.Version{Path: mp, Version: mv})
}
diff --git a/src/cmd/go/internal/modload/init.go b/src/cmd/go/internal/modload/init.go
index ea404b9f78..cbc7289afa 100644
--- a/src/cmd/go/internal/modload/init.go
+++ b/src/cmd/go/internal/modload/init.go
@@ -661,7 +661,7 @@ func requirementsFromModFile(ctx context.Context) *Requirements {
for _, n := range mPathCount {
if n > 1 {
var err error
- rs, err = updateRoots(ctx, rs.direct, rs, nil, nil)
+ rs, err = updateRoots(ctx, rs.direct, rs, nil, nil, false)
if err != nil {
base.Fatalf("go: %v", err)
}
@@ -999,10 +999,14 @@ func commitRequirements(ctx context.Context, goVersion string, rs *Requirements)
Indirect: !rs.direct[m.Path],
})
}
- modFile.SetRequire(list)
if goVersion != "" {
modFile.AddGoStmt(goVersion)
}
+ if semver.Compare("v"+modFileGoVersion(), separateIndirectVersionV) < 0 {
+ modFile.SetRequire(list)
+ } else {
+ modFile.SetRequireSeparateIndirect(list)
+ }
modFile.Cleanup()
dirty := index.modFileIsDirty(modFile)
diff --git a/src/cmd/go/internal/modload/load.go b/src/cmd/go/internal/modload/load.go
index a9d1777125..a3a8021c04 100644
--- a/src/cmd/go/internal/modload/load.go
+++ b/src/cmd/go/internal/modload/load.go
@@ -171,6 +171,11 @@ type PackageOpts struct {
// if the flag is set to "readonly" (the default) or "vendor".
ResolveMissingImports bool
+ // AssumeRootsImported indicates that the transitive dependencies of the root
+ // packages should be treated as if those roots will be imported by the main
+ // module.
+ AssumeRootsImported bool
+
// AllowPackage, if non-nil, is called after identifying the module providing
// each package. If AllowPackage returns a non-nil error, that error is set
// for the package, and the imports and test of that package will not be
@@ -875,6 +880,11 @@ const (
// are also roots (and must be marked pkgIsRoot).
pkgIsRoot
+ // pkgFromRoot indicates that the package is in the transitive closure of
+ // imports starting at the roots. (Note that every package marked as pkgIsRoot
+ // is also trivially marked pkgFromRoot.)
+ pkgFromRoot
+
// pkgImportsLoaded indicates that the imports and testImports fields of a
// loadPkg have been populated.
pkgImportsLoaded
@@ -1068,7 +1078,7 @@ func loadFromRoots(ctx context.Context, params loaderParams) *loader {
// iteration so we don't need to also update it here. (That would waste time
// computing a "direct" map that we'll have to recompute later anyway.)
direct := ld.requirements.direct
- rs, err := updateRoots(ctx, direct, ld.requirements, noPkgs, toAdd)
+ rs, err := updateRoots(ctx, direct, ld.requirements, noPkgs, toAdd, ld.AssumeRootsImported)
if err != nil {
// If an error was found in a newly added module, report the package
// import stack instead of the module requirement stack. Packages
@@ -1274,7 +1284,7 @@ func (ld *loader) updateRequirements(ctx context.Context) (changed bool, err err
addRoots = tidy.rootModules
}
- rs, err = updateRoots(ctx, direct, rs, ld.pkgs, addRoots)
+ rs, err = updateRoots(ctx, direct, rs, ld.pkgs, addRoots, ld.AssumeRootsImported)
if err != nil {
// We don't actually know what even the root requirements are supposed to be,
// so we can't proceed with loading. Return the error to the caller
@@ -1433,6 +1443,9 @@ func (ld *loader) applyPkgFlags(ctx context.Context, pkg *loadPkg, flags loadPkg
// This package matches a root pattern by virtue of being in "all".
flags |= pkgIsRoot
}
+ if flags.has(pkgIsRoot) {
+ flags |= pkgFromRoot
+ }
old := pkg.flags.update(flags)
new := old | flags
@@ -1487,6 +1500,12 @@ func (ld *loader) applyPkgFlags(ctx context.Context, pkg *loadPkg, flags loadPkg
ld.applyPkgFlags(ctx, dep, pkgInAll)
}
}
+
+ if new.has(pkgFromRoot) && !old.has(pkgFromRoot|pkgImportsLoaded) {
+ for _, dep := range pkg.imports {
+ ld.applyPkgFlags(ctx, dep, pkgFromRoot)
+ }
+ }
}
// preloadRootModules loads the module requirements needed to identify the
@@ -1549,7 +1568,7 @@ func (ld *loader) preloadRootModules(ctx context.Context, rootPkgs []string) (ch
}
module.Sort(toAdd)
- rs, err := updateRoots(ctx, ld.requirements.direct, ld.requirements, nil, toAdd)
+ rs, err := updateRoots(ctx, ld.requirements.direct, ld.requirements, nil, toAdd, ld.AssumeRootsImported)
if err != nil {
// We are missing some root dependency, and for some reason we can't load
// enough of the module dependency graph to add the missing root. Package
diff --git a/src/cmd/go/internal/modload/modfile.go b/src/cmd/go/internal/modload/modfile.go
index a9c3a91d35..1145ac4ba5 100644
--- a/src/cmd/go/internal/modload/modfile.go
+++ b/src/cmd/go/internal/modload/modfile.go
@@ -35,6 +35,11 @@ const (
// module's go.mod file is expected to list explicit requirements on every
// module that provides any package transitively imported by that module.
lazyLoadingVersionV = "v1.17"
+
+ // separateIndirectVersionV is the Go version (plus leading "v") at which
+ // "// indirect" dependencies are added in a block separate from the direct
+ // ones. See https://golang.org/issue/45965.
+ separateIndirectVersionV = "v1.17"
)
const (
diff --git a/src/cmd/go/testdata/script/build_ignore_leading_bom.txt b/src/cmd/go/testdata/script/build_ignore_leading_bom.txt
new file mode 100644
index 0000000000..37141f3466
--- /dev/null
+++ b/src/cmd/go/testdata/script/build_ignore_leading_bom.txt
@@ -0,0 +1,27 @@
+# Per https://golang.org/ref/spec#Source_code_representation:
+# a compiler may ignore a UTF-8-encoded byte order mark (U+FEFF)
+# if it is the first Unicode code point in the source text.
+
+go list -f 'Imports: {{.Imports}} EmbedFiles: {{.EmbedFiles}}' .
+stdout '^Imports: \[embed m/hello\] EmbedFiles: \[.*file\]$'
+
+-- go.mod --
+module m
+
+go 1.16
+-- m.go --
+package main
+
+import (
+ _ "embed"
+
+ "m/hello"
+)
+
+//go:embed file
+var s string
+
+-- hello/hello.go --
+package hello
+
+-- file --
diff --git a/src/cmd/go/testdata/script/env_cross_build.txt b/src/cmd/go/testdata/script/env_cross_build.txt
new file mode 100644
index 0000000000..3feeba6b14
--- /dev/null
+++ b/src/cmd/go/testdata/script/env_cross_build.txt
@@ -0,0 +1,29 @@
+# Test that the corect default GOEXPERIMENT is used when cross
+# building with GOENV (#46815).
+
+# Unset variables set by the TestScript harness. Users typically won't
+# explicitly configure these, and #46815 doesn't repro if they are.
+env GOOS=
+env GOARCH=
+env GOEXPERIMENT=
+
+env GOENV=windows-amd64
+go build internal/abi
+
+env GOENV=ios-arm64
+go build internal/abi
+
+env GOENV=linux-mips
+go build internal/abi
+
+-- windows-amd64 --
+GOOS=windows
+GOARCH=amd64
+
+-- ios-arm64 --
+GOOS=ios
+GOARCH=arm64
+
+-- linux-mips --
+GOOS=linux
+GOARCH=mips
diff --git a/src/cmd/go/testdata/script/list_cgo_compiled_importmap.txt b/src/cmd/go/testdata/script/list_cgo_compiled_importmap.txt
new file mode 100644
index 0000000000..3d68ef3055
--- /dev/null
+++ b/src/cmd/go/testdata/script/list_cgo_compiled_importmap.txt
@@ -0,0 +1,38 @@
+# Regression test for https://golang.org/issue/46462.
+#
+# The "runtime/cgo" import found in synthesized .go files (reported in
+# the CompiledGoFiles field) should have a corresponding entry in the
+# ImportMap field when a runtime/cgo variant (such as a test variant)
+# will be used.
+
+[short] skip # -compiled can be slow (because it compiles things)
+[!cgo] skip
+
+env CGO_ENABLED=1
+env GOFLAGS=-tags=netcgo # Force net to use cgo even on Windows.
+
+
+# "runtime/cgo [runtime.test]" appears in the the test dependencies of "runtime",
+# because "runtime/cgo" itself depends on "runtime"
+
+go list -deps -test -compiled -f '{{if eq .ImportPath "net [runtime.test]"}}{{printf "%q" .Imports}}{{end}}' runtime
+
+ # Control case: the explicitly-imported package "sync" is a test variant,
+ # because "sync" depends on "runtime".
+stdout '"sync \[runtime\.test\]"'
+! stdout '"sync"'
+
+ # Experiment: the implicitly-imported package "runtime/cgo" is also a test variant,
+ # because "runtime/cgo" also depends on "runtime".
+stdout '"runtime/cgo \[runtime\.test\]"'
+! stdout '"runtime/cgo"'
+
+
+# Because the import of "runtime/cgo" in the cgo-generated file actually refers
+# to "runtime/cgo [runtime.test]", the latter should be listed in the ImportMap.
+# BUG(#46462): Today, it is not.
+
+go list -deps -test -compiled -f '{{if eq .ImportPath "net [runtime.test]"}}{{printf "%q" .ImportMap}}{{end}}' runtime
+
+stdout '"sync":"sync \[runtime\.test\]"' # control
+stdout '"runtime/cgo":"runtime/cgo \[runtime\.test\]"' # experiment
diff --git a/src/cmd/go/testdata/script/mod_edit_no_modcache.txt b/src/cmd/go/testdata/script/mod_edit_no_modcache.txt
new file mode 100644
index 0000000000..ced15bb301
--- /dev/null
+++ b/src/cmd/go/testdata/script/mod_edit_no_modcache.txt
@@ -0,0 +1,15 @@
+# 'go mod edit' opportunistically locks the side-lock file in the module cache,
+# for compatibility with older versions of the 'go' command.
+# It does not otherwise depend on the module cache, so it should not
+# fail if the module cache directory cannot be created.
+
+[root] skip
+
+mkdir $WORK/readonly
+chmod 0555 $WORK/readonly
+env GOPATH=$WORK/readonly/nonexist
+
+go mod edit -go=1.17
+
+-- go.mod --
+module example.com/m
diff --git a/src/cmd/go/testdata/script/mod_get_lazy_indirect.txt b/src/cmd/go/testdata/script/mod_get_lazy_indirect.txt
new file mode 100644
index 0000000000..1cef9d1c0c
--- /dev/null
+++ b/src/cmd/go/testdata/script/mod_get_lazy_indirect.txt
@@ -0,0 +1,44 @@
+# https://golang.org/issue/45979: after 'go get' on a package,
+# that package should be importable without error.
+
+
+# We start out with an unresolved dependency.
+# 'go list' suggests that we run 'go get' on that dependency.
+
+! go list -deps .
+stderr '^m.go:3:8: no required module provides package rsc\.io/quote; to add it:\n\tgo get rsc.io/quote$'
+
+
+# When we run the suggested 'go get' command, the new dependency can be used
+# immediately.
+#
+# 'go get' marks the new dependency as 'indirect', because it doesn't scan
+# enough source code to know whether it is direct, and it is easier and less
+# invasive to remove an incorrect indirect mark (e.g. using 'go get') than to
+# add one that is missing ('go mod tidy' or 'go mod vendor').
+
+go get rsc.io/quote
+grep 'rsc.io/quote v\d+\.\d+\.\d+ // indirect$' go.mod
+! grep 'rsc.io/quote v\d+\.\d+\.\d+$' go.mod
+
+go list -deps .
+! stderr .
+[!short] go build .
+[!short] ! stderr .
+
+
+# 'go get .' (or 'go mod tidy') removes the indirect mark.
+
+go get .
+grep 'rsc.io/quote v\d+\.\d+\.\d+$' go.mod
+! grep 'rsc.io/quote v\d+\.\d+\.\d+ // indirect$' go.mod
+
+
+-- go.mod --
+module example.com/m
+
+go 1.17
+-- m.go --
+package m
+
+import _ "rsc.io/quote"
diff --git a/src/cmd/go/testdata/script/mod_go_version_missing.txt b/src/cmd/go/testdata/script/mod_go_version_missing.txt
index aca36a0450..d704816729 100644
--- a/src/cmd/go/testdata/script/mod_go_version_missing.txt
+++ b/src/cmd/go/testdata/script/mod_go_version_missing.txt
@@ -73,10 +73,9 @@ module example.com/m
go $goversion
-require (
- example.com/dep v0.1.0
- example.com/testdep v0.1.0 // indirect
-)
+require example.com/dep v0.1.0
+
+require example.com/testdep v0.1.0 // indirect
replace (
example.com/dep v0.1.0 => ./dep
diff --git a/src/cmd/go/testdata/script/mod_graph_version.txt b/src/cmd/go/testdata/script/mod_graph_version.txt
new file mode 100644
index 0000000000..f9a73f4617
--- /dev/null
+++ b/src/cmd/go/testdata/script/mod_graph_version.txt
@@ -0,0 +1,101 @@
+# For this module, Go 1.17 prunes out a (transitive and otherwise-irrelevant)
+# requirement on a retracted higher version of a dependency.
+# However, when Go 1.16 reads the same requirements from the go.mod file,
+# it does not prune out that requirement, and selects the retracted version.
+#
+# The Go 1.16 module graph looks like:
+#
+# m ---- lazy v0.1.0 ---- requireincompatible v0.1.0 ---- incompatible v2.0.0+incompatible
+# | |
+# + -------+------------- incompatible v1.0.0
+#
+# The Go 1.17 module graph is the same except that the dependencies of
+# requireincompatible are pruned out (because the module that requires
+# it — lazy v0.1.0 — specifies 'go 1.17', and it is not otherwise relevant to
+# the main module).
+
+cp go.mod go.mod.orig
+
+go mod graph
+cp stdout graph-1.17.txt
+stdout '^example\.com/m example\.com/retract/incompatible@v1\.0\.0$'
+stdout '^example\.net/lazy@v0\.1\.0 example\.com/retract/incompatible@v1\.0\.0$'
+! stdout 'example\.com/retract/incompatible@v2\.0\.0\+incompatible'
+
+go mod graph -go=1.17
+cmp stdout graph-1.17.txt
+
+cmp go.mod go.mod.orig
+
+
+# Setting -go=1.16 should report the graph as viewed by Go 1.16,
+# but should not edit the go.mod file.
+
+go mod graph -go=1.16
+cp stdout graph-1.16.txt
+stdout '^example\.com/m example\.com/retract/incompatible@v1\.0\.0$'
+stdout '^example\.net/lazy@v0\.1\.0 example.com/retract/incompatible@v1\.0\.0$'
+stdout '^example.net/requireincompatible@v0.1.0 example.com/retract/incompatible@v2\.0\.0\+incompatible$'
+
+cmp go.mod go.mod.orig
+
+
+# If we actually update the go.mod file to the requested go version,
+# we should get the same selected versions, but the roots of the graph
+# may be updated.
+#
+# TODO(#45551): The roots should not be updated.
+
+go mod edit -go=1.16
+go mod graph
+! stdout '^example\.com/m example\.com/retract/incompatible@v1\.0\.0$'
+stdout '^example\.net/lazy@v0.1.0 example.com/retract/incompatible@v1\.0\.0$'
+stdout '^example.net/requireincompatible@v0.1.0 example.com/retract/incompatible@v2\.0\.0\+incompatible$'
+ # TODO(#45551): cmp stdout graph-1.16.txt
+
+
+# Unsupported go versions should be rejected, since we don't know
+# what versions they would report.
+! go mod graph -go=1.99999999999
+stderr '^invalid value "1\.99999999999" for flag -go: maximum supported Go version is '$goversion'\nusage: go mod graph \[-go=version\]\nRun ''go help mod graph'' for details.$'
+
+
+-- go.mod --
+// Module m indirectly imports a package from
+// example.com/retract/incompatible. Its selected version of
+// that module is lower under Go 1.17 semantics than under Go 1.16.
+module example.com/m
+
+go 1.17
+
+replace (
+ example.net/lazy v0.1.0 => ./lazy
+ example.net/requireincompatible v0.1.0 => ./requireincompatible
+)
+
+require (
+ example.com/retract/incompatible v1.0.0 // indirect
+ example.net/lazy v0.1.0
+)
+-- lazy/go.mod --
+// Module lazy requires example.com/retract/incompatible v1.0.0.
+//
+// When viewed from the outside it also has a transitive dependency
+// on v2.0.0+incompatible, but in lazy mode that transitive dependency
+// is pruned out.
+module example.net/lazy
+
+go 1.17
+
+exclude example.com/retract/incompatible v2.0.0+incompatible
+
+require (
+ example.com/retract/incompatible v1.0.0
+ example.net/requireincompatible v0.1.0
+)
+-- requireincompatible/go.mod --
+module example.net/requireincompatible
+
+go 1.15
+
+require example.com/retract/incompatible v2.0.0+incompatible
diff --git a/src/cmd/go/testdata/script/mod_install_hint.txt b/src/cmd/go/testdata/script/mod_install_hint.txt
new file mode 100644
index 0000000000..ab02840eb8
--- /dev/null
+++ b/src/cmd/go/testdata/script/mod_install_hint.txt
@@ -0,0 +1,5 @@
+# Module is replaced but not required. No hint appears as no module is suggested.
+go mod init m
+go mod edit -replace=github.com/notrequired@v0.5.0=github.com/doesnotexist@v0.5.0
+! go install github.com/notrequired
+! stderr 'to add it:'
\ No newline at end of file
diff --git a/src/cmd/go/testdata/script/mod_invalid_version.txt b/src/cmd/go/testdata/script/mod_invalid_version.txt
index 34d9c47674..6846a792a5 100644
--- a/src/cmd/go/testdata/script/mod_invalid_version.txt
+++ b/src/cmd/go/testdata/script/mod_invalid_version.txt
@@ -19,7 +19,7 @@ cp go.mod.orig go.mod
go mod edit -require golang.org/x/text@14c0d48ead0c
cd outside
! go list -m golang.org/x/text
-stderr 'go list -m: example.com@v0.0.0 \(replaced by \./\..\): parsing ../go.mod: '$WORK'/gopath/src/go.mod:5: require golang.org/x/text: version "14c0d48ead0c" invalid: must be of the form v1.2.3'
+stderr 'go list -m: example.com@v0.0.0 \(replaced by \./\.\.\): parsing ..[/\\]go.mod: '$WORK'[/\\]gopath[/\\]src[/\\]go.mod:5: require golang.org/x/text: version "14c0d48ead0c" invalid: must be of the form v1.2.3'
cd ..
go list -m golang.org/x/text
stdout 'golang.org/x/text v0.1.1-0.20170915032832-14c0d48ead0c'
@@ -47,10 +47,10 @@ cp go.mod.orig go.mod
go mod edit -require golang.org/x/text@v2.1.1-0.20170915032832-14c0d48ead0c
cd outside
! go list -m golang.org/x/text
-stderr 'go list -m: example.com@v0.0.0 \(replaced by \./\.\.\): parsing ../go.mod: '$WORK'/gopath/src/go.mod:5: require golang.org/x/text: version "v2.1.1-0.20170915032832-14c0d48ead0c" invalid: should be v0 or v1, not v2'
+stderr 'go list -m: example.com@v0.0.0 \(replaced by \./\.\.\): parsing ..[/\\]go.mod: '$WORK'[/\\]gopath[/\\]src[/\\]go.mod:5: require golang.org/x/text: version "v2.1.1-0.20170915032832-14c0d48ead0c" invalid: should be v0 or v1, not v2'
cd ..
! go list -m golang.org/x/text
-stderr $WORK'/gopath/src/go.mod:5: require golang.org/x/text: version "v2.1.1-0.20170915032832-14c0d48ead0c" invalid: should be v0 or v1, not v2'
+stderr $WORK'[/\\]gopath[/\\]src[/\\]go.mod:5: require golang.org/x/text: version "v2.1.1-0.20170915032832-14c0d48ead0c" invalid: should be v0 or v1, not v2'
# A pseudo-version with fewer than 12 digits of SHA-1 prefix is invalid.
cp go.mod.orig go.mod
diff --git a/src/cmd/go/testdata/script/mod_lazy_import_allmod.txt b/src/cmd/go/testdata/script/mod_lazy_import_allmod.txt
index 3dc1515df2..97718c4513 100644
--- a/src/cmd/go/testdata/script/mod_lazy_import_allmod.txt
+++ b/src/cmd/go/testdata/script/mod_lazy_import_allmod.txt
@@ -139,9 +139,10 @@ go 1.17
require (
a v0.1.0
b v0.1.0
- c v0.1.0 // indirect
)
+require c v0.1.0 // indirect
+
replace (
a v0.1.0 => ./a1
b v0.1.0 => ./b1
diff --git a/src/cmd/go/testdata/script/mod_lazy_new_import.txt b/src/cmd/go/testdata/script/mod_lazy_new_import.txt
index 86b14b64b6..4272a52de1 100644
--- a/src/cmd/go/testdata/script/mod_lazy_new_import.txt
+++ b/src/cmd/go/testdata/script/mod_lazy_new_import.txt
@@ -78,10 +78,9 @@ module example.com/lazy
go 1.17
-require (
- example.com/a v0.1.0
- example.com/b v0.1.0 // indirect
-)
+require example.com/a v0.1.0
+
+require example.com/b v0.1.0 // indirect
replace (
example.com/a v0.1.0 => ./a
@@ -94,8 +93,9 @@ module example.com/lazy
go 1.17
+require example.com/a v0.1.0
+
require (
- example.com/a v0.1.0
example.com/b v0.1.0 // indirect
example.com/c v0.1.0 // indirect
)
diff --git a/src/cmd/go/testdata/script/mod_lazy_test_of_test_dep.txt b/src/cmd/go/testdata/script/mod_lazy_test_of_test_dep.txt
index 722712d1f2..68a5b6dca2 100644
--- a/src/cmd/go/testdata/script/mod_lazy_test_of_test_dep.txt
+++ b/src/cmd/go/testdata/script/mod_lazy_test_of_test_dep.txt
@@ -148,10 +148,9 @@ module example.com/lazy
go 1.17
-require (
- example.com/a v0.1.0
- example.com/b v0.1.0 // indirect
-)
+require example.com/a v0.1.0
+
+require example.com/b v0.1.0 // indirect
replace (
example.com/a v0.1.0 => ./a
diff --git a/src/cmd/go/testdata/script/mod_list_test_cycle.txt b/src/cmd/go/testdata/script/mod_list_test_cycle.txt
new file mode 100644
index 0000000000..755e50b076
--- /dev/null
+++ b/src/cmd/go/testdata/script/mod_list_test_cycle.txt
@@ -0,0 +1,23 @@
+# https://golang.org/issue/45863: a typo in a test package leading to an
+# import cycle should be diagnosed, instead of causing an infinite loop.
+# The failure mode of this test prior to the fix was a timeout or OOM crash.
+
+go list -e -test -deps ./datastore/sql
+
+-- go.mod --
+module golang.org/issue45863
+
+go 1.17
+-- datastore/datastore_health.go --
+package datastore
+
+import (
+ "golang.org/issue45863/datastore"
+ "golang.org/issue45863/datastore/sql"
+)
+-- datastore/sql/sql.go --
+package sql
+-- datastore/sql/sql_test.go --
+package sql
+
+import _ "golang.org/issue45863/datastore"
diff --git a/src/cmd/go/testdata/script/mod_retention.txt b/src/cmd/go/testdata/script/mod_retention.txt
index 0e639db551..7a371b1806 100644
--- a/src/cmd/go/testdata/script/mod_retention.txt
+++ b/src/cmd/go/testdata/script/mod_retention.txt
@@ -140,8 +140,9 @@ module m
go $goversion
require (
- golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c // indirect
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0 // indirect
rsc.io/testonly v1.0.0 // indirect
)
+
+require golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c // indirect
diff --git a/src/cmd/go/testdata/script/mod_sumdb_golang.txt b/src/cmd/go/testdata/script/mod_sumdb_golang.txt
index cc0b0da474..becd88b52e 100644
--- a/src/cmd/go/testdata/script/mod_sumdb_golang.txt
+++ b/src/cmd/go/testdata/script/mod_sumdb_golang.txt
@@ -10,45 +10,73 @@ go env GOSUMDB
stdout '^sum.golang.org$'
# Download direct from github.
+
[!net] skip
[!exec:git] skip
env GOSUMDB=sum.golang.org
env GOPROXY=direct
+
go get -d rsc.io/quote@v1.5.2
cp go.sum saved.sum
+
# Download from proxy.golang.org with go.sum entry already.
# Use 'go list' instead of 'go get' since the latter may download extra go.mod
# files not listed in go.sum.
+
go clean -modcache
env GOSUMDB=
env GOPROXY=
-go list -x -deps rsc.io/quote
+
+go list -x -m all # Download go.mod files.
! stderr github
stderr proxy.golang.org/rsc.io/quote
! stderr sum.golang.org/tile
! stderr sum.golang.org/lookup/rsc.io/quote
+
+go list -x -deps rsc.io/quote # Download module source.
+! stderr github
+stderr proxy.golang.org/rsc.io/quote
+! stderr sum.golang.org/tile
+! stderr sum.golang.org/lookup/rsc.io/quote
+
cmp go.sum saved.sum
+
# Download again.
# Should use the checksum database to validate new go.sum lines,
# but not need to fetch any new data from the proxy.
+
rm go.sum
-go list -mod=mod -x rsc.io/quote
+
+go list -mod=mod -x -m all # Add checksums for go.mod files.
+stderr sum.golang.org/tile
! stderr github
! stderr proxy.golang.org/rsc.io/quote
-stderr sum.golang.org/tile
stderr sum.golang.org/lookup/rsc.io/quote
+
+go list -mod=mod -x rsc.io/quote # Add checksums for module source.
+! stderr . # Adds checksums, but for entities already in the module cache.
+
cmp go.sum saved.sum
+
# test fallback to direct
+
env TESTGOPROXY404=1
go clean -modcache
rm go.sum
-go list -mod=mod -x rsc.io/quote
+
+go list -mod=mod -x -m all # Download go.mod files
stderr 'proxy.golang.org.*404 testing'
stderr github.com/rsc
+
+go list -mod=mod -x rsc.io/quote # Download module source.
+stderr 'proxy.golang.org.*404 testing'
+stderr github.com/rsc
+
cmp go.sum saved.sum
+
-- go.mod --
module m
diff --git a/src/cmd/go/testdata/script/mod_tidy_convergence.txt b/src/cmd/go/testdata/script/mod_tidy_convergence.txt
index 22c8fc66c5..09c46f764b 100644
--- a/src/cmd/go/testdata/script/mod_tidy_convergence.txt
+++ b/src/cmd/go/testdata/script/mod_tidy_convergence.txt
@@ -90,7 +90,6 @@ cmp go.mod go.mod.postget
cp go.mod.orig go.mod
go mod edit -go=1.17 go.mod
go mod edit -go=1.17 go.mod.tidye
-go mod edit -go=1.17 go.mod.postget
go mod tidy -e
cmp go.mod go.mod.tidye
@@ -99,7 +98,7 @@ stderr '^example\.net/m imports\n\texample\.net/x: package example\.net/x provid
go get -d example.net/x@v0.1.0 example.net/y@v0.1.0
go mod tidy
-cmp go.mod go.mod.postget
+cmp go.mod go.mod.postget-117
-- go.mod --
@@ -144,6 +143,21 @@ require (
example.net/x v0.1.0
example.net/y v0.1.0 // indirect
)
+-- go.mod.postget-117 --
+module example.net/m
+
+go 1.17
+
+replace (
+ example.net/x v0.1.0 => ./x1
+ example.net/x v0.2.0-pre => ./x2-pre
+ example.net/y v0.1.0 => ./y1
+ example.net/y v0.2.0 => ./y2
+)
+
+require example.net/x v0.1.0
+
+require example.net/y v0.1.0 // indirect
-- m.go --
package m
diff --git a/src/cmd/go/testdata/script/mod_tidy_replace_old.txt b/src/cmd/go/testdata/script/mod_tidy_replace_old.txt
new file mode 100644
index 0000000000..cfd3792600
--- /dev/null
+++ b/src/cmd/go/testdata/script/mod_tidy_replace_old.txt
@@ -0,0 +1,34 @@
+# Regression test for https://golang.org/issue/46659.
+#
+# If a 'replace' directive specifies an older-than-selected version of a module,
+# 'go mod tidy' shouldn't try to add that version to the build list to resolve a
+# missing package: it won't be selected, and would cause the module loader to
+# loop indefinitely trying to resolve the package.
+
+cp go.mod go.mod.orig
+
+! go mod tidy
+! stderr panic
+stderr '^golang\.org/issue46659 imports\n\texample\.com/missingpkg/deprecated: package example\.com/missingpkg/deprecated provided by example\.com/missingpkg at latest version v1\.0\.0 but not at required version v1\.0\.1-beta$'
+
+go mod tidy -e
+
+cmp go.mod go.mod.orig
+
+-- go.mod --
+module golang.org/issue46659
+
+go 1.17
+
+replace example.com/missingpkg v1.0.1-alpha => example.com/missingpkg v1.0.0
+
+require example.com/usemissingpre v1.0.0
+
+require example.com/missingpkg v1.0.1-beta // indirect
+-- m.go --
+package m
+
+import (
+ _ "example.com/missingpkg/deprecated"
+ _ "example.com/usemissingpre"
+)
diff --git a/src/cmd/go/testdata/script/mod_tidy_version.txt b/src/cmd/go/testdata/script/mod_tidy_version.txt
index eaa6ee7b0d..3bc97bcb1e 100644
--- a/src/cmd/go/testdata/script/mod_tidy_version.txt
+++ b/src/cmd/go/testdata/script/mod_tidy_version.txt
@@ -92,8 +92,9 @@ cmpenv go.mod go.mod.latest
-- go.mod --
module example.com/m
+require example.net/a v0.1.0
+
require (
- example.net/a v0.1.0
example.net/c v0.1.0 // indirect
example.net/d v0.1.0 // indirect
)
@@ -118,8 +119,9 @@ module example.com/m
go 1.15
+require example.net/a v0.1.0
+
require (
- example.net/a v0.1.0
example.net/c v0.1.0 // indirect
example.net/d v0.1.0 // indirect
)
@@ -139,8 +141,9 @@ module example.com/m
go 1.15
+require example.net/a v0.1.0
+
require (
- example.net/a v0.1.0
example.net/c v0.1.0 // indirect
example.net/d v0.2.0 // indirect
)
@@ -160,10 +163,9 @@ module example.com/m
go 1.16
-require (
- example.net/a v0.1.0
- example.net/c v0.1.0 // indirect
-)
+require example.net/a v0.1.0
+
+require example.net/c v0.1.0 // indirect
replace (
example.net/a v0.1.0 => ./a
@@ -180,8 +182,9 @@ module example.com/m
go 1.17
+require example.net/a v0.1.0
+
require (
- example.net/a v0.1.0
example.net/b v0.1.0 // indirect
example.net/c v0.1.0 // indirect
)
@@ -201,8 +204,9 @@ module example.com/m
go $goversion
+require example.net/a v0.1.0
+
require (
- example.net/a v0.1.0
example.net/b v0.1.0 // indirect
example.net/c v0.1.0 // indirect
)
diff --git a/src/cmd/gofmt/doc.go b/src/cmd/gofmt/doc.go
index 68476e7d44..e340665594 100644
--- a/src/cmd/gofmt/doc.go
+++ b/src/cmd/gofmt/doc.go
@@ -26,9 +26,6 @@ The flags are:
Do not print reformatted sources to standard output.
If a file's formatting is different from gofmt's, print its name
to standard output.
- -G
- Allow generic code, using type parameters.
- See golang.org/issues/43651 for more information.
-r rule
Apply the rewrite rule to the source before reformatting.
-s
diff --git a/src/cmd/internal/moddeps/moddeps_test.go b/src/cmd/internal/moddeps/moddeps_test.go
index 7723250468..56c3b2585c 100644
--- a/src/cmd/internal/moddeps/moddeps_test.go
+++ b/src/cmd/internal/moddeps/moddeps_test.go
@@ -5,6 +5,7 @@
package moddeps_test
import (
+ "bytes"
"encoding/json"
"fmt"
"internal/testenv"
@@ -68,7 +69,7 @@ func TestAllDependencies(t *testing.T) {
// There is no vendor directory, so the module must have no dependencies.
// Check that the list of active modules contains only the main module.
- cmd := exec.Command(goBin, "list", "-mod=mod", "-m", "all")
+ cmd := exec.Command(goBin, "list", "-mod=readonly", "-m", "all")
cmd.Env = append(os.Environ(), "GO111MODULE=on")
cmd.Dir = m.Dir
cmd.Stderr = new(strings.Builder)
@@ -123,10 +124,38 @@ func TestAllDependencies(t *testing.T) {
t.Skip("skipping because a diff command with support for --recursive and --unified flags is unavailable")
}
+ // We're going to check the standard modules for tidiness, so we need a usable
+ // GOMODCACHE. If the default directory doesn't exist, use a temporary
+ // directory instead. (That can occur, for example, when running under
+ // run.bash with GO_TEST_SHORT=0: run.bash sets GOPATH=/nonexist-gopath, and
+ // GO_TEST_SHORT=0 causes it to run this portion of the test.)
+ var modcacheEnv []string
+ {
+ out, err := exec.Command(goBin, "env", "GOMODCACHE").Output()
+ if err != nil {
+ t.Fatalf("%s env GOMODCACHE: %v", goBin, err)
+ }
+ modcacheOk := false
+ if gomodcache := string(bytes.TrimSpace(out)); gomodcache != "" {
+ if _, err := os.Stat(gomodcache); err == nil {
+ modcacheOk = true
+ }
+ }
+ if !modcacheOk {
+ modcacheEnv = []string{
+ "GOMODCACHE=" + t.TempDir(),
+ "GOFLAGS=" + os.Getenv("GOFLAGS") + " -modcacherw", // Allow t.TempDir() to clean up subdirectories.
+ }
+ }
+ }
+
// Build the bundle binary at the golang.org/x/tools
// module version specified in GOROOT/src/cmd/go.mod.
bundleDir := t.TempDir()
- r := runner{Dir: filepath.Join(runtime.GOROOT(), "src/cmd")}
+ r := runner{
+ Dir: filepath.Join(runtime.GOROOT(), "src/cmd"),
+ Env: append(os.Environ(), modcacheEnv...),
+ }
r.run(t, goBin, "build", "-mod=readonly", "-o", bundleDir, "golang.org/x/tools/cmd/bundle")
var gorootCopyDir string
@@ -160,7 +189,7 @@ func TestAllDependencies(t *testing.T) {
}
r := runner{
Dir: filepath.Join(gorootCopyDir, rel),
- Env: append(os.Environ(),
+ Env: append(append(os.Environ(), modcacheEnv...),
// Set GOROOT.
"GOROOT="+gorootCopyDir,
// Explicitly override PWD and clear GOROOT_FINAL so that GOROOT=gorootCopyDir is definitely used.
diff --git a/src/cmd/internal/sys/supported.go b/src/cmd/internal/sys/supported.go
index fa477b837f..0d2bad9612 100644
--- a/src/cmd/internal/sys/supported.go
+++ b/src/cmd/internal/sys/supported.go
@@ -74,7 +74,7 @@ func BuildModeSupported(compiler, buildmode, goos, goarch string) bool {
"android/amd64", "android/arm", "android/arm64", "android/386",
"freebsd/amd64",
"darwin/amd64", "darwin/arm64",
- "windows/amd64", "windows/386":
+ "windows/amd64", "windows/386", "windows/arm64":
return true
}
return false
diff --git a/src/cmd/link/internal/ld/elf.go b/src/cmd/link/internal/ld/elf.go
index 6f81e74da2..81011638bc 100644
--- a/src/cmd/link/internal/ld/elf.go
+++ b/src/cmd/link/internal/ld/elf.go
@@ -950,6 +950,11 @@ func elfdynhash(ctxt *Link) {
}
s = ldr.CreateSymForUpdate(".dynamic", 0)
+ if ctxt.BuildMode == BuildModePIE {
+ // https://github.com/bminor/glibc/blob/895ef79e04a953cac1493863bcae29ad85657ee1/elf/elf.h#L986
+ const DTFLAGS_1_PIE = 0x08000000
+ Elfwritedynent(ctxt.Arch, s, elf.DT_FLAGS_1, uint64(DTFLAGS_1_PIE))
+ }
elfverneed = nfile
if elfverneed != 0 {
elfWriteDynEntSym(ctxt, s, elf.DT_VERNEED, gnuVersionR.Sym())
diff --git a/src/cmd/link/internal/ld/pe.go b/src/cmd/link/internal/ld/pe.go
index 3540c07da1..8eb4231c3a 100644
--- a/src/cmd/link/internal/ld/pe.go
+++ b/src/cmd/link/internal/ld/pe.go
@@ -475,7 +475,7 @@ func (f *peFile) addDWARFSection(name string, size int) *peSection {
off := f.stringTable.add(name)
h := f.addSection(name, size, size)
h.shortName = fmt.Sprintf("/%d", off)
- h.characteristics = IMAGE_SCN_MEM_READ | IMAGE_SCN_MEM_DISCARDABLE | IMAGE_SCN_CNT_INITIALIZED_DATA
+ h.characteristics = IMAGE_SCN_ALIGN_1BYTES | IMAGE_SCN_MEM_READ | IMAGE_SCN_MEM_DISCARDABLE | IMAGE_SCN_CNT_INITIALIZED_DATA
return h
}
diff --git a/src/cmd/link/internal/loader/loader.go b/src/cmd/link/internal/loader/loader.go
index 1b71a66c6f..efca824d98 100644
--- a/src/cmd/link/internal/loader/loader.go
+++ b/src/cmd/link/internal/loader/loader.go
@@ -699,12 +699,18 @@ func (l *Loader) checkdup(name string, r *oReader, li uint32, dup Sym) {
p := r.Data(li)
rdup, ldup := l.toLocal(dup)
pdup := rdup.Data(ldup)
- if bytes.Equal(p, pdup) {
- return
- }
reason := "same length but different contents"
if len(p) != len(pdup) {
reason = fmt.Sprintf("new length %d != old length %d", len(p), len(pdup))
+ } else if bytes.Equal(p, pdup) {
+ // For BSS symbols, we need to check size as well, see issue 46653.
+ szdup := l.SymSize(dup)
+ sz := int64(r.Sym(li).Siz())
+ if szdup == sz {
+ return
+ }
+ reason = fmt.Sprintf("different sizes: new size %d != old size %d",
+ sz, szdup)
}
fmt.Fprintf(os.Stderr, "cmd/link: while reading object for '%v': duplicate symbol '%s', previous def at '%v', with mismatched payload: %s\n", r.unit.Lib, name, rdup.unit.Lib, reason)
diff --git a/src/cmd/link/link_test.go b/src/cmd/link/link_test.go
index 4d6bc76aca..7230054bed 100644
--- a/src/cmd/link/link_test.go
+++ b/src/cmd/link/link_test.go
@@ -470,10 +470,30 @@ TEXT ·f(SB), NOSPLIT|DUPOK, $0-0
JMP 0(PC)
`
+const testStrictDupAsmSrc3 = `
+#include "textflag.h"
+GLOBL ·rcon(SB), RODATA|DUPOK, $64
+`
+
+const testStrictDupAsmSrc4 = `
+#include "textflag.h"
+GLOBL ·rcon(SB), RODATA|DUPOK, $32
+`
+
func TestStrictDup(t *testing.T) {
// Check that -strictdups flag works.
testenv.MustHaveGoBuild(t)
+ asmfiles := []struct {
+ fname string
+ payload string
+ }{
+ {"a", testStrictDupAsmSrc1},
+ {"b", testStrictDupAsmSrc2},
+ {"c", testStrictDupAsmSrc3},
+ {"d", testStrictDupAsmSrc4},
+ }
+
t.Parallel()
tmpdir := t.TempDir()
@@ -483,15 +503,12 @@ func TestStrictDup(t *testing.T) {
if err != nil {
t.Fatal(err)
}
- src = filepath.Join(tmpdir, "a.s")
- err = ioutil.WriteFile(src, []byte(testStrictDupAsmSrc1), 0666)
- if err != nil {
- t.Fatal(err)
- }
- src = filepath.Join(tmpdir, "b.s")
- err = ioutil.WriteFile(src, []byte(testStrictDupAsmSrc2), 0666)
- if err != nil {
- t.Fatal(err)
+ for _, af := range asmfiles {
+ src = filepath.Join(tmpdir, af.fname+".s")
+ err = ioutil.WriteFile(src, []byte(af.payload), 0666)
+ if err != nil {
+ t.Fatal(err)
+ }
}
src = filepath.Join(tmpdir, "go.mod")
err = ioutil.WriteFile(src, []byte("module teststrictdup\n"), 0666)
@@ -503,7 +520,7 @@ func TestStrictDup(t *testing.T) {
cmd.Dir = tmpdir
out, err := cmd.CombinedOutput()
if err != nil {
- t.Errorf("linking with -strictdups=1 failed: %v", err)
+ t.Errorf("linking with -strictdups=1 failed: %v\n%s", err, string(out))
}
if !bytes.Contains(out, []byte("mismatched payload")) {
t.Errorf("unexpected output:\n%s", out)
@@ -515,7 +532,11 @@ func TestStrictDup(t *testing.T) {
if err == nil {
t.Errorf("linking with -strictdups=2 did not fail")
}
- if !bytes.Contains(out, []byte("mismatched payload")) {
+ // NB: on amd64 we get the 'new length' error, on arm64 the 'different
+ // contents' error.
+ if !(bytes.Contains(out, []byte("mismatched payload: new length")) ||
+ bytes.Contains(out, []byte("mismatched payload: same length but different contents"))) ||
+ !bytes.Contains(out, []byte("mismatched payload: different sizes")) {
t.Errorf("unexpected output:\n%s", out)
}
}
diff --git a/src/cmd/pprof/pprof.go b/src/cmd/pprof/pprof.go
index 1d10a7b41f..e72c765adc 100644
--- a/src/cmd/pprof/pprof.go
+++ b/src/cmd/pprof/pprof.go
@@ -233,8 +233,8 @@ func (f *file) Name() string {
}
func (f *file) ObjAddr(addr uint64) (uint64, error) {
- // No support for shared libraries.
- return 0, nil
+ // No support for shared libraries, so translation is a no-op.
+ return addr, nil
}
func (f *file) BuildID() string {
diff --git a/src/cmd/pprof/pprof_test.go b/src/cmd/pprof/pprof_test.go
new file mode 100644
index 0000000000..11e251bfde
--- /dev/null
+++ b/src/cmd/pprof/pprof_test.go
@@ -0,0 +1,127 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import (
+ "fmt"
+ "internal/testenv"
+ "os"
+ "os/exec"
+ "path/filepath"
+ "runtime"
+ "strings"
+ "testing"
+)
+
+var tmp, pprofExe string // populated by buildPprof
+
+func TestMain(m *testing.M) {
+ if !testenv.HasGoBuild() {
+ return
+ }
+
+ var exitcode int
+ if err := buildPprof(); err == nil {
+ exitcode = m.Run()
+ } else {
+ fmt.Println(err)
+ exitcode = 1
+ }
+ os.RemoveAll(tmp)
+ os.Exit(exitcode)
+}
+
+func buildPprof() error {
+ var err error
+ tmp, err = os.MkdirTemp("", "TestPprof")
+ if err != nil {
+ return fmt.Errorf("TempDir failed: %v", err)
+ }
+
+ pprofExe = filepath.Join(tmp, "testpprof.exe")
+ gotool, err := testenv.GoTool()
+ if err != nil {
+ return err
+ }
+ out, err := exec.Command(gotool, "build", "-o", pprofExe, "cmd/pprof").CombinedOutput()
+ if err != nil {
+ os.RemoveAll(tmp)
+ return fmt.Errorf("go build -o %v cmd/pprof: %v\n%s", pprofExe, err, string(out))
+ }
+
+ return nil
+}
+
+// See also runtime/pprof.cpuProfilingBroken.
+func mustHaveCPUProfiling(t *testing.T) {
+ switch runtime.GOOS {
+ case "plan9":
+ t.Skipf("skipping on %s, unimplemented", runtime.GOOS)
+ case "aix":
+ t.Skipf("skipping on %s, issue 45170", runtime.GOOS)
+ case "ios", "dragonfly", "netbsd", "illumos", "solaris":
+ t.Skipf("skipping on %s, issue 13841", runtime.GOOS)
+ case "openbsd":
+ if runtime.GOARCH == "arm" || runtime.GOARCH == "arm64" {
+ t.Skipf("skipping on %s/%s, issue 13841", runtime.GOOS, runtime.GOARCH)
+ }
+ }
+}
+
+func mustHaveDisasm(t *testing.T) {
+ switch runtime.GOARCH {
+ case "mips", "mipsle", "mips64", "mips64le":
+ t.Skipf("skipping on %s, issue 12559", runtime.GOARCH)
+ case "riscv64":
+ t.Skipf("skipping on %s, issue 36738", runtime.GOARCH)
+ case "s390x":
+ t.Skipf("skipping on %s, issue 15255", runtime.GOARCH)
+ }
+
+ // Skip PIE platforms, pprof can't disassemble PIE.
+ if runtime.GOOS == "windows" {
+ t.Skipf("skipping on %s, issue 46639", runtime.GOOS)
+ }
+ if runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" {
+ t.Skipf("skipping on %s/%s, issue 46639", runtime.GOOS, runtime.GOARCH)
+ }
+}
+
+// TestDisasm verifies that cmd/pprof can successfully disassemble functions.
+//
+// This is a regression test for issue 46636.
+func TestDisasm(t *testing.T) {
+ mustHaveCPUProfiling(t)
+ mustHaveDisasm(t)
+ testenv.MustHaveGoBuild(t)
+
+ tmpdir := t.TempDir()
+ cpuExe := filepath.Join(tmpdir, "cpu.exe")
+ cmd := exec.Command(testenv.GoToolPath(t), "build", "-o", cpuExe, "cpu.go")
+ cmd.Dir = "testdata/"
+ out, err := cmd.CombinedOutput()
+ if err != nil {
+ t.Fatalf("build failed: %v\n%s", err, out)
+ }
+
+ profile := filepath.Join(tmpdir, "cpu.pprof")
+ cmd = exec.Command(cpuExe, "-output", profile)
+ out, err = cmd.CombinedOutput()
+ if err != nil {
+ t.Fatalf("cpu failed: %v\n%s", err, out)
+ }
+
+ cmd = exec.Command(pprofExe, "-disasm", "main.main", cpuExe, profile)
+ out, err = cmd.CombinedOutput()
+ if err != nil {
+ t.Fatalf("pprof failed: %v\n%s", err, out)
+ }
+
+ sout := string(out)
+ want := "ROUTINE ======================== main.main"
+ if !strings.Contains(sout, want) {
+ t.Errorf("pprof disasm got %s want contains %q", sout, want)
+ }
+}
diff --git a/src/cmd/pprof/testdata/cpu.go b/src/cmd/pprof/testdata/cpu.go
new file mode 100644
index 0000000000..5b682870db
--- /dev/null
+++ b/src/cmd/pprof/testdata/cpu.go
@@ -0,0 +1,41 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import (
+ "flag"
+ "fmt"
+ "os"
+ "runtime/pprof"
+ "time"
+)
+
+var output = flag.String("output", "", "pprof profile output file")
+
+func main() {
+ flag.Parse()
+ if *output == "" {
+ fmt.Fprintf(os.Stderr, "usage: %s -output file.pprof\n", os.Args[0])
+ os.Exit(2)
+ }
+
+ f, err := os.Create(*output)
+ if err != nil {
+ fmt.Fprintln(os.Stderr, err)
+ os.Exit(2)
+ }
+ defer f.Close()
+
+ if err := pprof.StartCPUProfile(f); err != nil {
+ fmt.Fprintln(os.Stderr, err)
+ os.Exit(2)
+ }
+ defer pprof.StopCPUProfile()
+
+ // Spin for long enough to collect some samples.
+ start := time.Now()
+ for time.Since(start) < time.Second {
+ }
+}
diff --git a/src/cmd/vendor/golang.org/x/mod/modfile/read.go b/src/cmd/vendor/golang.org/x/mod/modfile/read.go
index 2a961ca81c..956f30cbb3 100644
--- a/src/cmd/vendor/golang.org/x/mod/modfile/read.go
+++ b/src/cmd/vendor/golang.org/x/mod/modfile/read.go
@@ -194,12 +194,15 @@ func (x *FileSyntax) updateLine(line *Line, tokens ...string) {
line.Token = tokens
}
-func (x *FileSyntax) removeLine(line *Line) {
+// markRemoved modifies line so that it (and its end-of-line comment, if any)
+// will be dropped by (*FileSyntax).Cleanup.
+func (line *Line) markRemoved() {
line.Token = nil
+ line.Comments.Suffix = nil
}
// Cleanup cleans up the file syntax x after any edit operations.
-// To avoid quadratic behavior, removeLine marks the line as dead
+// To avoid quadratic behavior, (*Line).markRemoved marks the line as dead
// by setting line.Token = nil but does not remove it from the slice
// in which it appears. After edits have all been indicated,
// calling Cleanup cleans out the dead lines.
diff --git a/src/cmd/vendor/golang.org/x/mod/modfile/rule.go b/src/cmd/vendor/golang.org/x/mod/modfile/rule.go
index 7299e15500..78f83fa714 100644
--- a/src/cmd/vendor/golang.org/x/mod/modfile/rule.go
+++ b/src/cmd/vendor/golang.org/x/mod/modfile/rule.go
@@ -58,13 +58,6 @@ type Go struct {
Syntax *Line
}
-// A Require is a single require statement.
-type Require struct {
- Mod module.Version
- Indirect bool // has "// indirect" comment
- Syntax *Line
-}
-
// An Exclude is a single exclude statement.
type Exclude struct {
Mod module.Version
@@ -93,6 +86,93 @@ type VersionInterval struct {
Low, High string
}
+// A Require is a single require statement.
+type Require struct {
+ Mod module.Version
+ Indirect bool // has "// indirect" comment
+ Syntax *Line
+}
+
+func (r *Require) markRemoved() {
+ r.Syntax.markRemoved()
+ *r = Require{}
+}
+
+func (r *Require) setVersion(v string) {
+ r.Mod.Version = v
+
+ if line := r.Syntax; len(line.Token) > 0 {
+ if line.InBlock {
+ // If the line is preceded by an empty line, remove it; see
+ // https://golang.org/issue/33779.
+ if len(line.Comments.Before) == 1 && len(line.Comments.Before[0].Token) == 0 {
+ line.Comments.Before = line.Comments.Before[:0]
+ }
+ if len(line.Token) >= 2 { // example.com v1.2.3
+ line.Token[1] = v
+ }
+ } else {
+ if len(line.Token) >= 3 { // require example.com v1.2.3
+ line.Token[2] = v
+ }
+ }
+ }
+}
+
+// setIndirect sets line to have (or not have) a "// indirect" comment.
+func (r *Require) setIndirect(indirect bool) {
+ r.Indirect = indirect
+ line := r.Syntax
+ if isIndirect(line) == indirect {
+ return
+ }
+ if indirect {
+ // Adding comment.
+ if len(line.Suffix) == 0 {
+ // New comment.
+ line.Suffix = []Comment{{Token: "// indirect", Suffix: true}}
+ return
+ }
+
+ com := &line.Suffix[0]
+ text := strings.TrimSpace(strings.TrimPrefix(com.Token, string(slashSlash)))
+ if text == "" {
+ // Empty comment.
+ com.Token = "// indirect"
+ return
+ }
+
+ // Insert at beginning of existing comment.
+ com.Token = "// indirect; " + text
+ return
+ }
+
+ // Removing comment.
+ f := strings.TrimSpace(strings.TrimPrefix(line.Suffix[0].Token, string(slashSlash)))
+ if f == "indirect" {
+ // Remove whole comment.
+ line.Suffix = nil
+ return
+ }
+
+ // Remove comment prefix.
+ com := &line.Suffix[0]
+ i := strings.Index(com.Token, "indirect;")
+ com.Token = "//" + com.Token[i+len("indirect;"):]
+}
+
+// isIndirect reports whether line has a "// indirect" comment,
+// meaning it is in go.mod only for its effect on indirect dependencies,
+// so that it can be dropped entirely once the effective version of the
+// indirect dependency reaches the given minimum version.
+func isIndirect(line *Line) bool {
+ if len(line.Suffix) == 0 {
+ return false
+ }
+ f := strings.Fields(strings.TrimPrefix(line.Suffix[0].Token, string(slashSlash)))
+ return (len(f) == 1 && f[0] == "indirect" || len(f) > 1 && f[0] == "indirect;")
+}
+
func (f *File) AddModuleStmt(path string) error {
if f.Syntax == nil {
f.Syntax = new(FileSyntax)
@@ -476,58 +556,6 @@ func (f *File) fixRetract(fix VersionFixer, errs *ErrorList) {
}
}
-// isIndirect reports whether line has a "// indirect" comment,
-// meaning it is in go.mod only for its effect on indirect dependencies,
-// so that it can be dropped entirely once the effective version of the
-// indirect dependency reaches the given minimum version.
-func isIndirect(line *Line) bool {
- if len(line.Suffix) == 0 {
- return false
- }
- f := strings.Fields(strings.TrimPrefix(line.Suffix[0].Token, string(slashSlash)))
- return (len(f) == 1 && f[0] == "indirect" || len(f) > 1 && f[0] == "indirect;")
-}
-
-// setIndirect sets line to have (or not have) a "// indirect" comment.
-func setIndirect(line *Line, indirect bool) {
- if isIndirect(line) == indirect {
- return
- }
- if indirect {
- // Adding comment.
- if len(line.Suffix) == 0 {
- // New comment.
- line.Suffix = []Comment{{Token: "// indirect", Suffix: true}}
- return
- }
-
- com := &line.Suffix[0]
- text := strings.TrimSpace(strings.TrimPrefix(com.Token, string(slashSlash)))
- if text == "" {
- // Empty comment.
- com.Token = "// indirect"
- return
- }
-
- // Insert at beginning of existing comment.
- com.Token = "// indirect; " + text
- return
- }
-
- // Removing comment.
- f := strings.TrimSpace(strings.TrimPrefix(line.Suffix[0].Token, string(slashSlash)))
- if f == "indirect" {
- // Remove whole comment.
- line.Suffix = nil
- return
- }
-
- // Remove comment prefix.
- com := &line.Suffix[0]
- i := strings.Index(com.Token, "indirect;")
- com.Token = "//" + com.Token[i+len("indirect;"):]
-}
-
// IsDirectoryPath reports whether the given path should be interpreted
// as a directory path. Just like on the go command line, relative paths
// and rooted paths are directory paths; the rest are module paths.
@@ -835,6 +863,12 @@ func (f *File) AddGoStmt(version string) error {
return nil
}
+// AddRequire sets the first require line for path to version vers,
+// preserving any existing comments for that line and removing all
+// other lines for path.
+//
+// If no line currently exists for path, AddRequire adds a new line
+// at the end of the last require block.
func (f *File) AddRequire(path, vers string) error {
need := true
for _, r := range f.Require {
@@ -844,7 +878,7 @@ func (f *File) AddRequire(path, vers string) error {
f.Syntax.updateLine(r.Syntax, "require", AutoQuote(path), vers)
need = false
} else {
- f.Syntax.removeLine(r.Syntax)
+ r.Syntax.markRemoved()
*r = Require{}
}
}
@@ -856,69 +890,235 @@ func (f *File) AddRequire(path, vers string) error {
return nil
}
+// AddNewRequire adds a new require line for path at version vers at the end of
+// the last require block, regardless of any existing require lines for path.
func (f *File) AddNewRequire(path, vers string, indirect bool) {
line := f.Syntax.addLine(nil, "require", AutoQuote(path), vers)
- setIndirect(line, indirect)
- f.Require = append(f.Require, &Require{module.Version{Path: path, Version: vers}, indirect, line})
+ r := &Require{
+ Mod: module.Version{Path: path, Version: vers},
+ Syntax: line,
+ }
+ r.setIndirect(indirect)
+ f.Require = append(f.Require, r)
}
+// SetRequire updates the requirements of f to contain exactly req, preserving
+// the existing block structure and line comment contents (except for 'indirect'
+// markings) for the first requirement on each named module path.
+//
+// The Syntax field is ignored for the requirements in req.
+//
+// Any requirements not already present in the file are added to the block
+// containing the last require line.
+//
+// The requirements in req must specify at most one distinct version for each
+// module path.
+//
+// If any existing requirements may be removed, the caller should call Cleanup
+// after all edits are complete.
func (f *File) SetRequire(req []*Require) {
- need := make(map[string]string)
- indirect := make(map[string]bool)
+ type elem struct {
+ version string
+ indirect bool
+ }
+ need := make(map[string]elem)
for _, r := range req {
- need[r.Mod.Path] = r.Mod.Version
- indirect[r.Mod.Path] = r.Indirect
- }
-
- for _, r := range f.Require {
- if v, ok := need[r.Mod.Path]; ok {
- r.Mod.Version = v
- r.Indirect = indirect[r.Mod.Path]
- } else {
- *r = Require{}
+ if prev, dup := need[r.Mod.Path]; dup && prev.version != r.Mod.Version {
+ panic(fmt.Errorf("SetRequire called with conflicting versions for path %s (%s and %s)", r.Mod.Path, prev.version, r.Mod.Version))
}
+ need[r.Mod.Path] = elem{r.Mod.Version, r.Indirect}
}
- var newStmts []Expr
+ // Update or delete the existing Require entries to preserve
+ // only the first for each module path in req.
+ for _, r := range f.Require {
+ e, ok := need[r.Mod.Path]
+ if ok {
+ r.setVersion(e.version)
+ r.setIndirect(e.indirect)
+ } else {
+ r.markRemoved()
+ }
+ delete(need, r.Mod.Path)
+ }
+
+ // Add new entries in the last block of the file for any paths that weren't
+ // already present.
+ //
+ // This step is nondeterministic, but the final result will be deterministic
+ // because we will sort the block.
+ for path, e := range need {
+ f.AddNewRequire(path, e.version, e.indirect)
+ }
+
+ f.SortBlocks()
+}
+
+// SetRequireSeparateIndirect updates the requirements of f to contain the given
+// requirements. Comment contents (except for 'indirect' markings) are retained
+// from the first existing requirement for each module path, and block structure
+// is maintained as long as the indirect markings match.
+//
+// Any requirements on paths not already present in the file are added. Direct
+// requirements are added to the last block containing *any* other direct
+// requirement. Indirect requirements are added to the last block containing
+// *only* other indirect requirements. If no suitable block exists, a new one is
+// added, with the last block containing a direct dependency (if any)
+// immediately before the first block containing only indirect dependencies.
+//
+// The Syntax field is ignored for requirements in the given blocks.
+func (f *File) SetRequireSeparateIndirect(req []*Require) {
+ type modKey struct {
+ path string
+ indirect bool
+ }
+ need := make(map[modKey]string)
+ for _, r := range req {
+ need[modKey{r.Mod.Path, r.Indirect}] = r.Mod.Version
+ }
+
+ comments := make(map[string]Comments)
+ for _, r := range f.Require {
+ v, ok := need[modKey{r.Mod.Path, r.Indirect}]
+ if !ok {
+ if _, ok := need[modKey{r.Mod.Path, !r.Indirect}]; ok {
+ if _, dup := comments[r.Mod.Path]; !dup {
+ comments[r.Mod.Path] = r.Syntax.Comments
+ }
+ }
+ r.markRemoved()
+ continue
+ }
+ r.setVersion(v)
+ delete(need, modKey{r.Mod.Path, r.Indirect})
+ }
+
+ var (
+ lastDirectOrMixedBlock Expr
+ firstIndirectOnlyBlock Expr
+ lastIndirectOnlyBlock Expr
+ )
for _, stmt := range f.Syntax.Stmt {
switch stmt := stmt.(type) {
- case *LineBlock:
- if len(stmt.Token) > 0 && stmt.Token[0] == "require" {
- var newLines []*Line
- for _, line := range stmt.Line {
- if p, err := parseString(&line.Token[0]); err == nil && need[p] != "" {
- if len(line.Comments.Before) == 1 && len(line.Comments.Before[0].Token) == 0 {
- line.Comments.Before = line.Comments.Before[:0]
- }
- line.Token[1] = need[p]
- delete(need, p)
- setIndirect(line, indirect[p])
- newLines = append(newLines, line)
- }
- }
- if len(newLines) == 0 {
- continue // drop stmt
- }
- stmt.Line = newLines
- }
-
case *Line:
- if len(stmt.Token) > 0 && stmt.Token[0] == "require" {
- if p, err := parseString(&stmt.Token[1]); err == nil && need[p] != "" {
- stmt.Token[2] = need[p]
- delete(need, p)
- setIndirect(stmt, indirect[p])
- } else {
- continue // drop stmt
+ if len(stmt.Token) == 0 || stmt.Token[0] != "require" {
+ continue
+ }
+ if isIndirect(stmt) {
+ lastIndirectOnlyBlock = stmt
+ } else {
+ lastDirectOrMixedBlock = stmt
+ }
+ case *LineBlock:
+ if len(stmt.Token) == 0 || stmt.Token[0] != "require" {
+ continue
+ }
+ indirectOnly := true
+ for _, line := range stmt.Line {
+ if len(line.Token) == 0 {
+ continue
+ }
+ if !isIndirect(line) {
+ indirectOnly = false
+ break
+ }
+ }
+ if indirectOnly {
+ lastIndirectOnlyBlock = stmt
+ if firstIndirectOnlyBlock == nil {
+ firstIndirectOnlyBlock = stmt
+ }
+ } else {
+ lastDirectOrMixedBlock = stmt
+ }
+ }
+ }
+
+ isOrContainsStmt := func(stmt Expr, target Expr) bool {
+ if stmt == target {
+ return true
+ }
+ if stmt, ok := stmt.(*LineBlock); ok {
+ if target, ok := target.(*Line); ok {
+ for _, line := range stmt.Line {
+ if line == target {
+ return true
+ }
}
}
}
- newStmts = append(newStmts, stmt)
+ return false
}
- f.Syntax.Stmt = newStmts
- for path, vers := range need {
- f.AddNewRequire(path, vers, indirect[path])
+ addRequire := func(path, vers string, indirect bool, comments Comments) {
+ var line *Line
+ if indirect {
+ if lastIndirectOnlyBlock != nil {
+ line = f.Syntax.addLine(lastIndirectOnlyBlock, "require", path, vers)
+ } else {
+ // Add a new require block after the last direct-only or mixed "require"
+ // block (if any).
+ //
+ // (f.Syntax.addLine would add the line to an existing "require" block if
+ // present, but here the existing "require" blocks are all direct-only, so
+ // we know we need to add a new block instead.)
+ line = &Line{Token: []string{"require", path, vers}}
+ lastIndirectOnlyBlock = line
+ firstIndirectOnlyBlock = line // only block implies first block
+ if lastDirectOrMixedBlock == nil {
+ f.Syntax.Stmt = append(f.Syntax.Stmt, line)
+ } else {
+ for i, stmt := range f.Syntax.Stmt {
+ if isOrContainsStmt(stmt, lastDirectOrMixedBlock) {
+ f.Syntax.Stmt = append(f.Syntax.Stmt, nil) // increase size
+ copy(f.Syntax.Stmt[i+2:], f.Syntax.Stmt[i+1:]) // shuffle elements up
+ f.Syntax.Stmt[i+1] = line
+ break
+ }
+ }
+ }
+ }
+ } else {
+ if lastDirectOrMixedBlock != nil {
+ line = f.Syntax.addLine(lastDirectOrMixedBlock, "require", path, vers)
+ } else {
+ // Add a new require block before the first indirect block (if any).
+ //
+ // That way if the file initially contains only indirect lines,
+ // the direct lines still appear before it: we preserve existing
+ // structure, but only to the extent that that structure already
+ // reflects the direct/indirect split.
+ line = &Line{Token: []string{"require", path, vers}}
+ lastDirectOrMixedBlock = line
+ if firstIndirectOnlyBlock == nil {
+ f.Syntax.Stmt = append(f.Syntax.Stmt, line)
+ } else {
+ for i, stmt := range f.Syntax.Stmt {
+ if isOrContainsStmt(stmt, firstIndirectOnlyBlock) {
+ f.Syntax.Stmt = append(f.Syntax.Stmt, nil) // increase size
+ copy(f.Syntax.Stmt[i+1:], f.Syntax.Stmt[i:]) // shuffle elements up
+ f.Syntax.Stmt[i] = line
+ break
+ }
+ }
+ }
+ }
+ }
+
+ line.Comments.Before = commentsAdd(line.Comments.Before, comments.Before)
+ line.Comments.Suffix = commentsAdd(line.Comments.Suffix, comments.Suffix)
+
+ r := &Require{
+ Mod: module.Version{Path: path, Version: vers},
+ Indirect: indirect,
+ Syntax: line,
+ }
+ r.setIndirect(indirect)
+ f.Require = append(f.Require, r)
+ }
+
+ for k, vers := range need {
+ addRequire(k.path, vers, k.indirect, comments[k.path])
}
f.SortBlocks()
}
@@ -926,7 +1126,7 @@ func (f *File) SetRequire(req []*Require) {
func (f *File) DropRequire(path string) error {
for _, r := range f.Require {
if r.Mod.Path == path {
- f.Syntax.removeLine(r.Syntax)
+ r.Syntax.markRemoved()
*r = Require{}
}
}
@@ -957,7 +1157,7 @@ func (f *File) AddExclude(path, vers string) error {
func (f *File) DropExclude(path, vers string) error {
for _, x := range f.Exclude {
if x.Mod.Path == path && x.Mod.Version == vers {
- f.Syntax.removeLine(x.Syntax)
+ x.Syntax.markRemoved()
*x = Exclude{}
}
}
@@ -988,7 +1188,7 @@ func (f *File) AddReplace(oldPath, oldVers, newPath, newVers string) error {
continue
}
// Already added; delete other replacements for same.
- f.Syntax.removeLine(r.Syntax)
+ r.Syntax.markRemoved()
*r = Replace{}
}
if r.Old.Path == oldPath {
@@ -1004,7 +1204,7 @@ func (f *File) AddReplace(oldPath, oldVers, newPath, newVers string) error {
func (f *File) DropReplace(oldPath, oldVers string) error {
for _, r := range f.Replace {
if r.Old.Path == oldPath && r.Old.Version == oldVers {
- f.Syntax.removeLine(r.Syntax)
+ r.Syntax.markRemoved()
*r = Replace{}
}
}
@@ -1045,7 +1245,7 @@ func (f *File) AddRetract(vi VersionInterval, rationale string) error {
func (f *File) DropRetract(vi VersionInterval) error {
for _, r := range f.Retract {
if r.VersionInterval == vi {
- f.Syntax.removeLine(r.Syntax)
+ r.Syntax.markRemoved()
*r = Retract{}
}
}
diff --git a/src/cmd/vendor/modules.txt b/src/cmd/vendor/modules.txt
index 9a1723d32c..34dbdaf5dd 100644
--- a/src/cmd/vendor/modules.txt
+++ b/src/cmd/vendor/modules.txt
@@ -28,7 +28,7 @@ golang.org/x/arch/x86/x86asm
## explicit; go 1.17
golang.org/x/crypto/ed25519
golang.org/x/crypto/ed25519/internal/edwards25519
-# golang.org/x/mod v0.4.3-0.20210512182355-6088ed88cecd
+# golang.org/x/mod v0.4.3-0.20210608190319-0f08993efd8a
## explicit; go 1.17
golang.org/x/mod/internal/lazyregexp
golang.org/x/mod/modfile
diff --git a/src/crypto/elliptic/elliptic.go b/src/crypto/elliptic/elliptic.go
index b8e5a3097d..f072960bfe 100644
--- a/src/crypto/elliptic/elliptic.go
+++ b/src/crypto/elliptic/elliptic.go
@@ -455,7 +455,7 @@ func initP384() {
// Multiple invocations of this function will return the same value, so it can
// be used for equality checks and switch statements.
//
-// The cryptographic operations are implemented using constant-time algorithms.
+// ScalarMult and ScalarBaseMult are implemented using constant-time algorithms.
func P256() Curve {
initonce.Do(initAll)
return p256
@@ -479,7 +479,7 @@ func P384() Curve {
// Multiple invocations of this function will return the same value, so it can
// be used for equality checks and switch statements.
//
-// The cryptographic operations do not use constant-time algorithms.
+// The cryptographic operations are implemented using constant-time algorithms.
func P521() Curve {
initonce.Do(initAll)
return p521
diff --git a/src/crypto/tls/common.go b/src/crypto/tls/common.go
index 77957ef82b..d561e61707 100644
--- a/src/crypto/tls/common.go
+++ b/src/crypto/tls/common.go
@@ -619,7 +619,7 @@ type Config struct {
// protocol will be one from this list, and the connection will fail
// if there is no mutually supported protocol. If NextProtos is empty
// or the peer doesn't support ALPN, the connection will succeed and
- // ConnectionState.NegotiatedProtocol will be empty."
+ // ConnectionState.NegotiatedProtocol will be empty.
NextProtos []string
// ServerName is used to verify the hostname on the returned
diff --git a/src/crypto/tls/handshake_client.go b/src/crypto/tls/handshake_client.go
index 13a7f3442c..4af3d998a3 100644
--- a/src/crypto/tls/handshake_client.go
+++ b/src/crypto/tls/handshake_client.go
@@ -711,17 +711,11 @@ func (hs *clientHandshakeState) processServerHello() (bool, error) {
}
}
- if hs.serverHello.alpnProtocol != "" {
- if len(hs.hello.alpnProtocols) == 0 {
- c.sendAlert(alertUnsupportedExtension)
- return false, errors.New("tls: server advertised unrequested ALPN extension")
- }
- if mutualProtocol([]string{hs.serverHello.alpnProtocol}, hs.hello.alpnProtocols) == "" {
- c.sendAlert(alertUnsupportedExtension)
- return false, errors.New("tls: server selected unadvertised ALPN protocol")
- }
- c.clientProtocol = hs.serverHello.alpnProtocol
+ if err := checkALPN(hs.hello.alpnProtocols, hs.serverHello.alpnProtocol); err != nil {
+ c.sendAlert(alertUnsupportedExtension)
+ return false, err
}
+ c.clientProtocol = hs.serverHello.alpnProtocol
c.scts = hs.serverHello.scts
@@ -753,6 +747,23 @@ func (hs *clientHandshakeState) processServerHello() (bool, error) {
return true, nil
}
+// checkALPN ensure that the server's choice of ALPN protocol is compatible with
+// the protocols that we advertised in the Client Hello.
+func checkALPN(clientProtos []string, serverProto string) error {
+ if serverProto == "" {
+ return nil
+ }
+ if len(clientProtos) == 0 {
+ return errors.New("tls: server advertised unrequested ALPN extension")
+ }
+ for _, proto := range clientProtos {
+ if proto == serverProto {
+ return nil
+ }
+ }
+ return errors.New("tls: server selected unadvertised ALPN protocol")
+}
+
func (hs *clientHandshakeState) readFinished(out []byte) error {
c := hs.c
@@ -979,19 +990,6 @@ func clientSessionCacheKey(serverAddr net.Addr, config *Config) string {
return serverAddr.String()
}
-// mutualProtocol finds the mutual ALPN protocol given list of possible
-// protocols and a list of the preference order.
-func mutualProtocol(protos, preferenceProtos []string) string {
- for _, s := range preferenceProtos {
- for _, c := range protos {
- if s == c {
- return s
- }
- }
- }
- return ""
-}
-
// hostnameInSNI converts name into an appropriate hostname for SNI.
// Literal IP addresses and absolute FQDNs are not permitted as SNI values.
// See RFC 6066, Section 3.
diff --git a/src/crypto/tls/handshake_client_tls13.go b/src/crypto/tls/handshake_client_tls13.go
index be37c681c6..eb59ac90d1 100644
--- a/src/crypto/tls/handshake_client_tls13.go
+++ b/src/crypto/tls/handshake_client_tls13.go
@@ -396,17 +396,11 @@ func (hs *clientHandshakeStateTLS13) readServerParameters() error {
}
hs.transcript.Write(encryptedExtensions.marshal())
- if encryptedExtensions.alpnProtocol != "" {
- if len(hs.hello.alpnProtocols) == 0 {
- c.sendAlert(alertUnsupportedExtension)
- return errors.New("tls: server advertised unrequested ALPN extension")
- }
- if mutualProtocol([]string{encryptedExtensions.alpnProtocol}, hs.hello.alpnProtocols) == "" {
- c.sendAlert(alertUnsupportedExtension)
- return errors.New("tls: server selected unadvertised ALPN protocol")
- }
- c.clientProtocol = encryptedExtensions.alpnProtocol
+ if err := checkALPN(hs.hello.alpnProtocols, encryptedExtensions.alpnProtocol); err != nil {
+ c.sendAlert(alertUnsupportedExtension)
+ return err
}
+ c.clientProtocol = encryptedExtensions.alpnProtocol
return nil
}
diff --git a/src/crypto/tls/handshake_server.go b/src/crypto/tls/handshake_server.go
index b231981e09..43f30e2fef 100644
--- a/src/crypto/tls/handshake_server.go
+++ b/src/crypto/tls/handshake_server.go
@@ -217,15 +217,13 @@ func (hs *serverHandshakeState) processClientHello() error {
c.serverName = hs.clientHello.serverName
}
- if len(c.config.NextProtos) > 0 && len(hs.clientHello.alpnProtocols) > 0 {
- selectedProto := mutualProtocol(hs.clientHello.alpnProtocols, c.config.NextProtos)
- if selectedProto == "" {
- c.sendAlert(alertNoApplicationProtocol)
- return fmt.Errorf("tls: client requested unsupported application protocols (%s)", hs.clientHello.alpnProtocols)
- }
- hs.hello.alpnProtocol = selectedProto
- c.clientProtocol = selectedProto
+ selectedProto, err := negotiateALPN(c.config.NextProtos, hs.clientHello.alpnProtocols)
+ if err != nil {
+ c.sendAlert(alertNoApplicationProtocol)
+ return err
}
+ hs.hello.alpnProtocol = selectedProto
+ c.clientProtocol = selectedProto
hs.cert, err = c.config.getCertificate(clientHelloInfo(hs.ctx, c, hs.clientHello))
if err != nil {
@@ -277,6 +275,34 @@ func (hs *serverHandshakeState) processClientHello() error {
return nil
}
+// negotiateALPN picks a shared ALPN protocol that both sides support in server
+// preference order. If ALPN is not configured or the peer doesn't support it,
+// it returns "" and no error.
+func negotiateALPN(serverProtos, clientProtos []string) (string, error) {
+ if len(serverProtos) == 0 || len(clientProtos) == 0 {
+ return "", nil
+ }
+ var http11fallback bool
+ for _, s := range serverProtos {
+ for _, c := range clientProtos {
+ if s == c {
+ return s, nil
+ }
+ if s == "h2" && c == "http/1.1" {
+ http11fallback = true
+ }
+ }
+ }
+ // As a special case, let http/1.1 clients connect to h2 servers as if they
+ // didn't support ALPN. We used not to enforce protocol overlap, so over
+ // time a number of HTTP servers were configured with only "h2", but
+ // expected to accept connections from "http/1.1" clients. See Issue 46310.
+ if http11fallback {
+ return "", nil
+ }
+ return "", fmt.Errorf("tls: client requested unsupported application protocols (%s)", clientProtos)
+}
+
// supportsECDHE returns whether ECDHE key exchanges can be used with this
// pre-TLS 1.3 client.
func supportsECDHE(c *Config, supportedCurves []CurveID, supportedPoints []uint8) bool {
diff --git a/src/crypto/tls/handshake_server_test.go b/src/crypto/tls/handshake_server_test.go
index 4483838045..f61b4c88ef 100644
--- a/src/crypto/tls/handshake_server_test.go
+++ b/src/crypto/tls/handshake_server_test.go
@@ -949,6 +949,27 @@ func TestHandshakeServerALPNNotConfigured(t *testing.T) {
runServerTestTLS13(t, test)
}
+func TestHandshakeServerALPNFallback(t *testing.T) {
+ config := testConfig.Clone()
+ config.NextProtos = []string{"proto1", "h2", "proto2"}
+
+ test := &serverTest{
+ name: "ALPN-Fallback",
+ // Note that this needs OpenSSL 1.0.2 because that is the first
+ // version that supports the -alpn flag.
+ command: []string{"openssl", "s_client", "-alpn", "proto3,http/1.1,proto4", "-cipher", "ECDHE-RSA-CHACHA20-POLY1305", "-ciphersuites", "TLS_CHACHA20_POLY1305_SHA256"},
+ config: config,
+ validate: func(state ConnectionState) error {
+ if state.NegotiatedProtocol != "" {
+ return fmt.Errorf("Got protocol %q, wanted nothing", state.NegotiatedProtocol)
+ }
+ return nil
+ },
+ }
+ runServerTestTLS12(t, test)
+ runServerTestTLS13(t, test)
+}
+
// TestHandshakeServerSNI involves a client sending an SNI extension of
// "snitest.com", which happens to match the CN of testSNICertificate. The test
// verifies that the server correctly selects that certificate.
diff --git a/src/crypto/tls/handshake_server_tls13.go b/src/crypto/tls/handshake_server_tls13.go
index c375ec4246..08251b84de 100644
--- a/src/crypto/tls/handshake_server_tls13.go
+++ b/src/crypto/tls/handshake_server_tls13.go
@@ -11,7 +11,6 @@ import (
"crypto/hmac"
"crypto/rsa"
"errors"
- "fmt"
"hash"
"io"
"sync/atomic"
@@ -551,15 +550,13 @@ func (hs *serverHandshakeStateTLS13) sendServerParameters() error {
encryptedExtensions := new(encryptedExtensionsMsg)
- if len(c.config.NextProtos) > 0 && len(hs.clientHello.alpnProtocols) > 0 {
- selectedProto := mutualProtocol(hs.clientHello.alpnProtocols, c.config.NextProtos)
- if selectedProto == "" {
- c.sendAlert(alertNoApplicationProtocol)
- return fmt.Errorf("tls: client requested unsupported application protocols (%s)", hs.clientHello.alpnProtocols)
- }
- encryptedExtensions.alpnProtocol = selectedProto
- c.clientProtocol = selectedProto
+ selectedProto, err := negotiateALPN(c.config.NextProtos, hs.clientHello.alpnProtocols)
+ if err != nil {
+ c.sendAlert(alertNoApplicationProtocol)
+ return err
}
+ encryptedExtensions.alpnProtocol = selectedProto
+ c.clientProtocol = selectedProto
hs.transcript.Write(encryptedExtensions.marshal())
if _, err := c.writeRecord(recordTypeHandshake, encryptedExtensions.marshal()); err != nil {
diff --git a/src/crypto/tls/testdata/Server-TLSv12-ALPN-Fallback b/src/crypto/tls/testdata/Server-TLSv12-ALPN-Fallback
new file mode 100644
index 0000000000..4fadf39062
--- /dev/null
+++ b/src/crypto/tls/testdata/Server-TLSv12-ALPN-Fallback
@@ -0,0 +1,91 @@
+>>> Flow 1 (client to server)
+00000000 16 03 01 00 a6 01 00 00 a2 03 03 b5 c9 ab 32 7f |..............2.|
+00000010 e1 af 3f f2 ac 2a 11 dd 33 f9 b5 21 88 0d e4 29 |..?..*..3..!...)|
+00000020 e2 47 49 dc c7 31 a8 a5 25 81 0c 00 00 04 cc a8 |.GI..1..%.......|
+00000030 00 ff 01 00 00 75 00 0b 00 04 03 00 01 02 00 0a |.....u..........|
+00000040 00 0c 00 0a 00 1d 00 17 00 1e 00 19 00 18 00 23 |...............#|
+00000050 00 00 00 10 00 19 00 17 06 70 72 6f 74 6f 33 08 |.........proto3.|
+00000060 68 74 74 70 2f 31 2e 31 06 70 72 6f 74 6f 34 00 |http/1.1.proto4.|
+00000070 16 00 00 00 17 00 00 00 0d 00 30 00 2e 04 03 05 |..........0.....|
+00000080 03 06 03 08 07 08 08 08 09 08 0a 08 0b 08 04 08 |................|
+00000090 05 08 06 04 01 05 01 06 01 03 03 02 03 03 01 02 |................|
+000000a0 01 03 02 02 02 04 02 05 02 06 02 |...........|
+>>> Flow 2 (server to client)
+00000000 16 03 03 00 3b 02 00 00 37 03 03 00 00 00 00 00 |....;...7.......|
+00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
+00000020 00 00 00 44 4f 57 4e 47 52 44 01 00 cc a8 00 00 |...DOWNGRD......|
+00000030 0f 00 23 00 00 ff 01 00 01 00 00 0b 00 02 01 00 |..#.............|
+00000040 16 03 03 02 59 0b 00 02 55 00 02 52 00 02 4f 30 |....Y...U..R..O0|
+00000050 82 02 4b 30 82 01 b4 a0 03 02 01 02 02 09 00 e8 |..K0............|
+00000060 f0 9d 3f e2 5b ea a6 30 0d 06 09 2a 86 48 86 f7 |..?.[..0...*.H..|
+00000070 0d 01 01 0b 05 00 30 1f 31 0b 30 09 06 03 55 04 |......0.1.0...U.|
+00000080 0a 13 02 47 6f 31 10 30 0e 06 03 55 04 03 13 07 |...Go1.0...U....|
+00000090 47 6f 20 52 6f 6f 74 30 1e 17 0d 31 36 30 31 30 |Go Root0...16010|
+000000a0 31 30 30 30 30 30 30 5a 17 0d 32 35 30 31 30 31 |1000000Z..250101|
+000000b0 30 30 30 30 30 30 5a 30 1a 31 0b 30 09 06 03 55 |000000Z0.1.0...U|
+000000c0 04 0a 13 02 47 6f 31 0b 30 09 06 03 55 04 03 13 |....Go1.0...U...|
+000000d0 02 47 6f 30 81 9f 30 0d 06 09 2a 86 48 86 f7 0d |.Go0..0...*.H...|
+000000e0 01 01 01 05 00 03 81 8d 00 30 81 89 02 81 81 00 |.........0......|
+000000f0 db 46 7d 93 2e 12 27 06 48 bc 06 28 21 ab 7e c4 |.F}...'.H..(!.~.|
+00000100 b6 a2 5d fe 1e 52 45 88 7a 36 47 a5 08 0d 92 42 |..]..RE.z6G....B|
+00000110 5b c2 81 c0 be 97 79 98 40 fb 4f 6d 14 fd 2b 13 |[.....y.@.Om..+.|
+00000120 8b c2 a5 2e 67 d8 d4 09 9e d6 22 38 b7 4a 0b 74 |....g....."8.J.t|
+00000130 73 2b c2 34 f1 d1 93 e5 96 d9 74 7b f3 58 9f 6c |s+.4......t{.X.l|
+00000140 61 3c c0 b0 41 d4 d9 2b 2b 24 23 77 5b 1c 3b bd |a<..A..++$#w[.;.|
+00000150 75 5d ce 20 54 cf a1 63 87 1d 1e 24 c4 f3 1d 1a |u]. T..c...$....|
+00000160 50 8b aa b6 14 43 ed 97 a7 75 62 f4 14 c8 52 d7 |P....C...ub...R.|
+00000170 02 03 01 00 01 a3 81 93 30 81 90 30 0e 06 03 55 |........0..0...U|
+00000180 1d 0f 01 01 ff 04 04 03 02 05 a0 30 1d 06 03 55 |...........0...U|
+00000190 1d 25 04 16 30 14 06 08 2b 06 01 05 05 07 03 01 |.%..0...+.......|
+000001a0 06 08 2b 06 01 05 05 07 03 02 30 0c 06 03 55 1d |..+.......0...U.|
+000001b0 13 01 01 ff 04 02 30 00 30 19 06 03 55 1d 0e 04 |......0.0...U...|
+000001c0 12 04 10 9f 91 16 1f 43 43 3e 49 a6 de 6d b6 80 |.......CC>I..m..|
+000001d0 d7 9f 60 30 1b 06 03 55 1d 23 04 14 30 12 80 10 |..`0...U.#..0...|
+000001e0 48 13 49 4d 13 7e 16 31 bb a3 01 d5 ac ab 6e 7b |H.IM.~.1......n{|
+000001f0 30 19 06 03 55 1d 11 04 12 30 10 82 0e 65 78 61 |0...U....0...exa|
+00000200 6d 70 6c 65 2e 67 6f 6c 61 6e 67 30 0d 06 09 2a |mple.golang0...*|
+00000210 86 48 86 f7 0d 01 01 0b 05 00 03 81 81 00 9d 30 |.H.............0|
+00000220 cc 40 2b 5b 50 a0 61 cb ba e5 53 58 e1 ed 83 28 |.@+[P.a...SX...(|
+00000230 a9 58 1a a9 38 a4 95 a1 ac 31 5a 1a 84 66 3d 43 |.X..8....1Z..f=C|
+00000240 d3 2d d9 0b f2 97 df d3 20 64 38 92 24 3a 00 bc |.-...... d8.$:..|
+00000250 cf 9c 7d b7 40 20 01 5f aa d3 16 61 09 a2 76 fd |..}.@ ._...a..v.|
+00000260 13 c3 cc e1 0c 5c ee b1 87 82 f1 6c 04 ed 73 bb |.....\.....l..s.|
+00000270 b3 43 77 8d 0c 1c f1 0f a1 d8 40 83 61 c9 4c 72 |.Cw.......@.a.Lr|
+00000280 2b 9d ae db 46 06 06 4d f4 c1 b3 3e c0 d1 bd 42 |+...F..M...>...B|
+00000290 d4 db fe 3d 13 60 84 5c 21 d3 3b e9 fa e7 16 03 |...=.`.\!.;.....|
+000002a0 03 00 ac 0c 00 00 a8 03 00 1d 20 2f e5 7d a3 47 |.......... /.}.G|
+000002b0 cd 62 43 15 28 da ac 5f bb 29 07 30 ff f6 84 af |.bC.(.._.).0....|
+000002c0 c4 cf c2 ed 90 99 5f 58 cb 3b 74 08 04 00 80 5f |......_X.;t...._|
+000002d0 37 27 84 58 1e ea 1e 40 1b de a9 8f 04 d4 94 64 |7'.X...@.......d|
+000002e0 4e 27 c7 f1 b3 30 d0 53 f5 3d 57 50 d2 17 97 c8 |N'...0.S.=WP....|
+000002f0 3d 61 af a6 21 ab 1c 34 47 70 f8 b1 3b 9c 06 86 |=a..!..4Gp..;...|
+00000300 87 00 e2 13 50 83 91 ad bc 84 bd b4 7b f3 4b ed |....P.......{.K.|
+00000310 ca 81 0c 94 37 a8 ec 67 ca 9c f3 00 f6 af c2 92 |....7..g........|
+00000320 c4 8c 78 07 18 0e 43 24 1b 98 16 50 5c 2b 75 0e |..x...C$...P\+u.|
+00000330 40 66 dc 40 cd 10 1a 51 25 f3 96 25 1a 3e 70 af |@f.@...Q%..%.>p.|
+00000340 16 24 d0 1c 0e 33 f9 c1 74 cf b7 e2 28 ac 60 16 |.$...3..t...(.`.|
+00000350 03 03 00 04 0e 00 00 00 |........|
+>>> Flow 3 (client to server)
+00000000 16 03 03 00 25 10 00 00 21 20 30 f2 bb f7 a7 ac |....%...! 0.....|
+00000010 23 20 22 ee 73 0d 49 9c b3 7b c1 9a db 2c 85 f3 |# ".s.I..{...,..|
+00000020 c0 82 31 60 bd 8b 14 4e 73 43 14 03 03 00 01 01 |..1`...NsC......|
+00000030 16 03 03 00 20 09 8d c7 86 ee cc f4 c7 36 a3 49 |.... ........6.I|
+00000040 d3 f7 a1 4a 68 a2 1e b4 fc cc a2 15 cb 01 92 d8 |...Jh...........|
+00000050 72 b0 d1 6f eb |r..o.|
+>>> Flow 4 (server to client)
+00000000 16 03 03 00 8b 04 00 00 87 00 00 00 00 00 81 50 |...............P|
+00000010 46 ad c1 db a8 38 86 7b 2b bb fd d0 c3 42 3e 00 |F....8.{+....B>.|
+00000020 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 94 |................|
+00000030 6f e0 18 83 51 ed 14 ef 68 ca 42 c5 4c a2 ac 05 |o...Q...h.B.L...|
+00000040 9c 69 69 99 08 9f de a4 d4 e7 37 ab 14 38 4c 47 |.ii.......7..8LG|
+00000050 70 f0 97 1d db 2d 0a 14 c2 1e f0 16 9f 6d 37 02 |p....-.......m7.|
+00000060 4b f1 16 be 98 3f df 74 83 7c 19 85 61 49 38 16 |K....?.t.|..aI8.|
+00000070 ee 35 7a e2 3f 74 fe 8d e3 07 93 a1 5e fa f2 02 |.5z.?t......^...|
+00000080 e5 c8 60 3f 11 83 8b 0e 32 52 f1 aa 52 b7 0a 89 |..`?....2R..R...|
+00000090 14 03 03 00 01 01 16 03 03 00 20 9e 65 15 cf 45 |.......... .e..E|
+000000a0 a5 03 69 c9 b1 d8 9e 92 a3 a2 b0 df 2e 62 b1 3a |..i..........b.:|
+000000b0 17 78 cd e5 1d f3 51 42 7e 4e 25 17 03 03 00 1d |.x....QB~N%.....|
+000000c0 d9 ae d0 fa b7 90 a9 2f 28 8d 1d 6f 54 1f c0 1e |......./(..oT...|
+000000d0 4d ae b6 91 f0 e8 84 cf 86 11 22 25 ea 15 03 03 |M........."%....|
+000000e0 00 12 0e 71 f2 11 9e 9f 58 ad c0 d8 fc fa 34 bc |...q....X.....4.|
+000000f0 02 5a 60 00 |.Z`.|
diff --git a/src/crypto/tls/testdata/Server-TLSv13-ALPN-Fallback b/src/crypto/tls/testdata/Server-TLSv13-ALPN-Fallback
new file mode 100644
index 0000000000..6203e6877c
--- /dev/null
+++ b/src/crypto/tls/testdata/Server-TLSv13-ALPN-Fallback
@@ -0,0 +1,100 @@
+>>> Flow 1 (client to server)
+00000000 16 03 01 00 eb 01 00 00 e7 03 03 1c d3 8e 3b d9 |..............;.|
+00000010 fe 7d e7 f9 9f fa c6 51 c3 8c 1b dd dc 87 95 f4 |.}.....Q........|
+00000020 39 23 67 e4 d6 bd 94 93 fc 88 4e 20 c3 c0 e2 c1 |9#g.......N ....|
+00000030 3d 12 ec 4c 0a 3f 40 51 13 24 61 11 c0 5d 09 f9 |=..L.?@Q.$a..]..|
+00000040 08 d6 3e cd e7 b3 51 c3 06 8f b4 42 00 04 13 03 |..>...Q....B....|
+00000050 00 ff 01 00 00 9a 00 0b 00 04 03 00 01 02 00 0a |................|
+00000060 00 0c 00 0a 00 1d 00 17 00 1e 00 19 00 18 00 23 |...............#|
+00000070 00 00 00 10 00 19 00 17 06 70 72 6f 74 6f 33 08 |.........proto3.|
+00000080 68 74 74 70 2f 31 2e 31 06 70 72 6f 74 6f 34 00 |http/1.1.proto4.|
+00000090 16 00 00 00 17 00 00 00 0d 00 1e 00 1c 04 03 05 |................|
+000000a0 03 06 03 08 07 08 08 08 09 08 0a 08 0b 08 04 08 |................|
+000000b0 05 08 06 04 01 05 01 06 01 00 2b 00 03 02 03 04 |..........+.....|
+000000c0 00 2d 00 02 01 01 00 33 00 26 00 24 00 1d 00 20 |.-.....3.&.$... |
+000000d0 f4 05 eb 4a 7a 73 20 18 74 aa 14 2a 0c 35 63 29 |...Jzs .t..*.5c)|
+000000e0 cb f2 ad d1 a2 3d bd 9d 02 b4 62 00 bc eb 10 58 |.....=....b....X|
+>>> Flow 2 (server to client)
+00000000 16 03 03 00 7a 02 00 00 76 03 03 00 00 00 00 00 |....z...v.......|
+00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
+00000020 00 00 00 00 00 00 00 00 00 00 00 20 c3 c0 e2 c1 |........... ....|
+00000030 3d 12 ec 4c 0a 3f 40 51 13 24 61 11 c0 5d 09 f9 |=..L.?@Q.$a..]..|
+00000040 08 d6 3e cd e7 b3 51 c3 06 8f b4 42 13 03 00 00 |..>...Q....B....|
+00000050 2e 00 2b 00 02 03 04 00 33 00 24 00 1d 00 20 2f |..+.....3.$... /|
+00000060 e5 7d a3 47 cd 62 43 15 28 da ac 5f bb 29 07 30 |.}.G.bC.(.._.).0|
+00000070 ff f6 84 af c4 cf c2 ed 90 99 5f 58 cb 3b 74 14 |.........._X.;t.|
+00000080 03 03 00 01 01 17 03 03 00 17 fb 75 d8 5c 50 35 |...........u.\P5|
+00000090 55 82 ba 65 1e 63 73 b8 c1 e9 d7 f5 28 68 3c c1 |U..e.cs.....(h<.|
+000000a0 5d 17 03 03 02 6d 56 c9 a9 09 73 6a bc fd 1a 3c |]....mV...sj...<|
+000000b0 6a f8 3e 32 99 83 e8 f6 01 9e 5e 30 e8 53 7f 72 |j.>2......^0.S.r|
+000000c0 fd 86 72 a8 9e 47 25 67 c1 f1 9a 03 c0 9d 6f 9d |..r..G%g......o.|
+000000d0 bd ed 29 30 8f 3c 01 ce 49 bb 5f dd 58 9a ae 80 |..)0.<..I._.X...|
+000000e0 5c 2d 81 fc ea 7b 03 03 3d 5d bb 92 23 73 67 89 |\-...{..=]..#sg.|
+000000f0 2e f0 ec 08 20 8a 36 eb 43 a6 a1 68 d0 39 95 37 |.... .6.C..h.9.7|
+00000100 6b 15 a9 0e 46 20 92 51 9c 04 bf 3b 07 97 84 cb |k...F .Q...;....|
+00000110 1f 30 38 37 2e ff e7 0f f5 14 93 5a 84 f1 f7 10 |.087.......Z....|
+00000120 c2 a5 0d bb 97 96 ef 4a e0 13 c0 63 72 2b 60 f3 |.......J...cr+`.|
+00000130 59 b5 57 aa 5f d1 da a9 0e dd 9c dd c2 cb 61 fe |Y.W._.........a.|
+00000140 e2 69 8e db 5d 70 6c 3a 33 e0 9e db 9a 31 26 6a |.i..]pl:3....1&j|
+00000150 2b 9e 19 8e bb 5d 06 48 ea c0 a1 c6 11 24 fb c4 |+....].H.....$..|
+00000160 ce ae 48 54 64 81 d1 84 38 a6 e0 7a 7b 74 2b bc |..HTd...8..z{t+.|
+00000170 ce 07 8b b6 04 1f 5b 4c 36 29 68 0c 8c c7 32 15 |......[L6)h...2.|
+00000180 93 e0 10 52 c2 27 23 96 c5 0c 9c e9 e2 a9 08 7d |...R.'#........}|
+00000190 25 68 65 f5 4e 44 eb a9 85 78 13 e1 0d 86 5e dc |%he.ND...x....^.|
+000001a0 fd e5 c6 dd 65 46 8e 2f 32 82 83 0b dd 67 f8 42 |....eF./2....g.B|
+000001b0 65 87 3b 08 fe b1 f5 12 e9 74 21 04 12 6d 75 35 |e.;......t!..mu5|
+000001c0 b2 eb 93 95 72 10 fa 56 96 77 c3 0c 17 8c 9e f6 |....r..V.w......|
+000001d0 77 19 28 37 96 3e 73 98 f4 d2 91 4f 40 db 76 56 |w.(7.>s....O@.vV|
+000001e0 ce b5 a8 7a b8 86 d0 9a ba b5 8b 40 c2 63 e1 cf |...z.......@.c..|
+000001f0 49 29 2c 5d 1a 9b 8b 56 cb 93 ca 2c c0 d0 15 b7 |I),]...V...,....|
+00000200 8a f1 6a d5 0a a8 81 57 b1 6e 10 cd a5 ff b1 4d |..j....W.n.....M|
+00000210 47 c6 9b 35 f1 5f 83 91 22 f6 88 68 65 b3 b9 c9 |G..5._.."..he...|
+00000220 02 dc 4b f7 13 39 06 e6 3a ec 94 ef 51 15 05 72 |..K..9..:...Q..r|
+00000230 1d f4 9d 3b da ca 8d 2c 64 be 9b 45 99 2c 63 cc |...;...,d..E.,c.|
+00000240 22 b3 8b 93 ad f6 2c f0 d2 d9 11 3f 5b c0 40 fa |".....,....?[.@.|
+00000250 90 6e a0 76 b2 43 b9 4c 72 c4 24 28 a2 bf 56 d6 |.n.v.C.Lr.$(..V.|
+00000260 d2 a7 2a d1 8c 5e 1d eb f8 be d0 43 da 7a c7 88 |..*..^.....C.z..|
+00000270 61 67 a2 69 85 23 43 3e d4 88 f2 33 c3 5b 38 0a |ag.i.#C>...3.[8.|
+00000280 1e de 28 3b 3b 19 de 95 2f 84 c0 37 88 80 59 2f |..(;;.../..7..Y/|
+00000290 a6 ee 93 1a 69 08 c3 df 7c cf da c3 9b 96 70 d9 |....i...|.....p.|
+000002a0 60 c5 e9 0f 42 f6 1a f2 58 5e f2 32 61 6a b2 a3 |`...B...X^.2aj..|
+000002b0 1f 97 fa 08 6c 3f 4b 83 1f 04 66 80 8a 26 3a 7f |....l?K...f..&:.|
+000002c0 24 30 ec 10 ae 7d 19 ff 39 91 ca 97 4e ed 0a d7 |$0...}..9...N...|
+000002d0 64 3b 6b 50 29 33 0d b2 10 bc 83 63 3c fb 9a 82 |d;kP)3.....c<...|
+000002e0 3b 7f bc 04 40 f1 33 64 4a 80 cd 01 f9 f4 c6 89 |;...@.3dJ.......|
+000002f0 65 27 25 f9 cf 4f 7e c8 6e d9 0e ec 47 4a 51 29 |e'%..O~.n...GJQ)|
+00000300 2f be 34 50 bd 9b d2 d8 b7 ea bb 0b a1 e0 20 1b |/.4P.......... .|
+00000310 02 9c f2 17 03 03 00 99 61 dc 0b 3a 30 de 39 f6 |........a..:0.9.|
+00000320 f3 db f8 6c 3b fa 4e 1e 7e 62 a5 ae 73 ba e1 41 |...l;.N.~b..s..A|
+00000330 58 77 2a c1 7a 0c 50 bb 0c 57 b4 c4 25 bf 2f 9f |Xw*.z.P..W..%./.|
+00000340 38 91 e2 65 22 9d ca ac 18 58 7e 81 2d fd 74 24 |8..e"....X~.-.t$|
+00000350 28 69 76 11 df 9d 23 b8 be ae 8b e0 93 8e 5d df |(iv...#.......].|
+00000360 0a 64 d0 b7 02 68 aa 86 01 0d 55 11 3b 76 70 c6 |.d...h....U.;vp.|
+00000370 83 0c 5e 0a e3 37 a5 8b ad 25 50 b9 e8 5c 6b 04 |..^..7...%P..\k.|
+00000380 b4 51 ec 9c d3 fa c6 b7 9c f0 46 aa 73 da 3c 0d |.Q........F.s.<.|
+00000390 d3 bd 32 81 d4 d2 f1 1a b0 92 f3 73 3e 54 2b 05 |..2........s>T+.|
+000003a0 92 24 34 75 df d6 18 a0 6a 82 95 4c 9b fc 7e b6 |.$4u....j..L..~.|
+000003b0 8e 17 03 03 00 35 8f 34 0e 3b 91 d8 e7 74 24 71 |.....5.4.;...t$q|
+000003c0 0e 7b f3 12 bb 76 2f 31 12 17 b8 9e 24 ce f9 2f |.{...v/1....$../|
+000003d0 3f 5d f2 13 4b 2e 9b 1e c4 78 03 a6 c8 07 11 a3 |?]..K....x......|
+000003e0 98 79 61 6e 4f 44 6e 18 ee c4 9b 17 03 03 00 93 |.yanODn.........|
+000003f0 64 dd 52 a9 d9 51 63 6a a0 a3 c2 75 6b 5d 1d 54 |d.R..Qcj...uk].T|
+00000400 ce d4 53 7e 14 8e d9 26 93 28 78 65 16 1b 95 77 |..S~...&.(xe...w|
+00000410 68 0a 46 f1 82 36 bb 8a fa 0d df 54 8c 3d 83 e0 |h.F..6.....T.=..|
+00000420 d7 de 2d 96 e9 c4 d7 22 d3 97 8e ae 90 f8 fc e6 |..-...."........|
+00000430 a6 4b 78 98 4c c5 28 87 91 46 fa f4 1c 8d 0e ec |.Kx.L.(..F......|
+00000440 0d 71 40 9a 04 49 b4 e8 5b 62 6f cd 16 c1 d5 fb |.q@..I..[bo.....|
+00000450 73 2a 96 8f e5 a2 f4 11 1e df 2d 40 45 6b d5 a9 |s*........-@Ek..|
+00000460 e4 e3 f7 93 fc fa d7 20 af d5 f7 b4 0e 09 ad d5 |....... ........|
+00000470 26 87 b8 6c e2 20 95 fb c0 70 3e 38 be b7 b1 9f |&..l. ...p>8....|
+00000480 70 da c1 |p..|
+>>> Flow 3 (client to server)
+00000000 14 03 03 00 01 01 17 03 03 00 35 29 d2 b9 bb 9b |..........5)....|
+00000010 de 6c 5d 22 23 c1 fe 99 4c c5 33 bf fd 70 36 6b |.l]"#...L.3..p6k|
+00000020 f1 a5 92 e8 bf 7c 3d 6e ef 6a 44 73 bc cb 27 1c |.....|=n.jDs..'.|
+00000030 09 5d bf 99 4c 19 24 c3 3b 30 91 b5 e3 b6 63 45 |.]..L.$.;0....cE|
+>>> Flow 4 (server to client)
+00000000 17 03 03 00 1e 52 55 85 7c b8 87 dd c7 b2 d9 5b |.....RU.|......[|
+00000010 18 1d bb ac bf b6 ab 76 82 be 64 0e b2 7b 2c 0f |.......v..d..{,.|
+00000020 aa 17 92 17 03 03 00 13 79 0a 60 b1 46 20 33 74 |........y.`.F 3t|
+00000030 ed 12 a0 23 de 68 88 fc 6f dd 8e |...#.h..o..|
diff --git a/src/database/sql/sql_test.go b/src/database/sql/sql_test.go
index 80f63e877d..f771dee4a9 100644
--- a/src/database/sql/sql_test.go
+++ b/src/database/sql/sql_test.go
@@ -2838,7 +2838,7 @@ func TestTxStmtDeadlock(t *testing.T) {
db := newTestDB(t, "people")
defer closeDB(t, db)
- ctx, cancel := context.WithTimeout(context.Background(), 5*time.Millisecond)
+ ctx, cancel := context.WithCancel(context.Background())
defer cancel()
tx, err := db.BeginTx(ctx, nil)
if err != nil {
@@ -2849,6 +2849,7 @@ func TestTxStmtDeadlock(t *testing.T) {
if err != nil {
t.Fatal(err)
}
+ cancel()
// Run number of stmt queries to reproduce deadlock from context cancel
for i := 0; i < 1e3; i++ {
// Encounter any close related errors (e.g. ErrTxDone, stmt is closed)
diff --git a/src/debug/elf/file.go b/src/debug/elf/file.go
index cd5bf8fab0..b25d8209e3 100644
--- a/src/debug/elf/file.go
+++ b/src/debug/elf/file.go
@@ -1164,6 +1164,13 @@ func (f *File) DWARF() (*dwarf.Data, error) {
b = dbuf
}
+ if f.Type == ET_EXEC {
+ // Do not apply relocations to DWARF sections for ET_EXEC binaries.
+ // Relocations should already be applied, and .rela sections may
+ // contain incorrect data.
+ return b, nil
+ }
+
for _, r := range f.Sections {
if r.Type != SHT_RELA && r.Type != SHT_REL {
continue
diff --git a/src/fmt/doc.go b/src/fmt/doc.go
index d05ee519c3..c584cc9465 100644
--- a/src/fmt/doc.go
+++ b/src/fmt/doc.go
@@ -189,7 +189,7 @@
When printing a struct, fmt cannot and therefore does not invoke
formatting methods such as Error or String on unexported fields.
- Explicit argument indexes:
+ Explicit argument indexes
In Printf, Sprintf, and Fprintf, the default behavior is for each
formatting verb to format successive arguments passed in the call.
@@ -211,7 +211,7 @@
fmt.Sprintf("%d %d %#[1]x %#x", 16, 17)
will yield "16 17 0x10 0x11".
- Format errors:
+ Format errors
If an invalid argument is given for a verb, such as providing
a string to %d, the generated string will contain a
diff --git a/src/go/build/deps_test.go b/src/go/build/deps_test.go
index 67989d2e38..cb58416191 100644
--- a/src/go/build/deps_test.go
+++ b/src/go/build/deps_test.go
@@ -440,7 +440,8 @@ var depsRules = `
# HTTP, King of Dependencies.
FMT
- < golang.org/x/net/http2/hpack, net/http/internal, net/http/internal/ascii;
+ < golang.org/x/net/http2/hpack
+ < net/http/internal, net/http/internal/ascii, net/http/internal/testcert;
FMT, NET, container/list, encoding/binary, log
< golang.org/x/text/transform
@@ -459,6 +460,7 @@ var depsRules = `
golang.org/x/net/http2/hpack,
net/http/internal,
net/http/internal/ascii,
+ net/http/internal/testcert,
net/http/httptrace,
mime/multipart,
log
diff --git a/src/go/build/doc.go b/src/go/build/doc.go
index 2c6f0a83be..778b4f40f7 100644
--- a/src/go/build/doc.go
+++ b/src/go/build/doc.go
@@ -59,7 +59,7 @@
//
// A build constraint, also known as a build tag, is a line comment that begins
//
-// // +build
+// //go:build
//
// that lists the conditions under which a file should be included in the
// package. Build constraints may also be part of a file's name
diff --git a/src/go/build/read.go b/src/go/build/read.go
index aa7c6ee59e..b98c7938a8 100644
--- a/src/go/build/read.go
+++ b/src/go/build/read.go
@@ -6,6 +6,7 @@ package build
import (
"bufio"
+ "bytes"
"errors"
"fmt"
"go/ast"
@@ -28,9 +29,19 @@ type importReader struct {
pos token.Position
}
+var bom = []byte{0xef, 0xbb, 0xbf}
+
func newImportReader(name string, r io.Reader) *importReader {
+ b := bufio.NewReader(r)
+ // Remove leading UTF-8 BOM.
+ // Per https://golang.org/ref/spec#Source_code_representation:
+ // a compiler may ignore a UTF-8-encoded byte order mark (U+FEFF)
+ // if it is the first Unicode code point in the source text.
+ if leadingBytes, err := b.Peek(3); err == nil && bytes.Equal(leadingBytes, bom) {
+ b.Discard(3)
+ }
return &importReader{
- b: bufio.NewReader(r),
+ b: b,
pos: token.Position{
Filename: name,
Line: 1,
diff --git a/src/go/build/read_test.go b/src/go/build/read_test.go
index 32e6bae008..1e5e1c2de2 100644
--- a/src/go/build/read_test.go
+++ b/src/go/build/read_test.go
@@ -66,6 +66,10 @@ var readGoInfoTests = []readTest{
`,
"",
},
+ {
+ "\ufeff𝔻" + `package p; import "x";ℙvar x = 1`,
+ "",
+ },
}
var readCommentsTests = []readTest{
@@ -81,6 +85,10 @@ var readCommentsTests = []readTest{
`ℙpackage p; import . "x"`,
"",
},
+ {
+ "\ufeff𝔻" + `ℙpackage p; import . "x"`,
+ "",
+ },
{
`// foo
@@ -90,6 +98,19 @@ var readCommentsTests = []readTest{
/*/ zot */
+ // asdf
+ ℙHello, world`,
+ "",
+ },
+ {
+ "\ufeff𝔻" + `// foo
+
+ /* bar */
+
+ /* quux */ // baz
+
+ /*/ zot */
+
// asdf
ℙHello, world`,
"",
@@ -107,6 +128,11 @@ func testRead(t *testing.T, tests []readTest, read func(io.Reader) ([]byte, erro
in = tt.in[:j] + tt.in[j+len("ℙ"):]
testOut = tt.in[:j]
}
+ d := strings.Index(tt.in, "𝔻")
+ if d >= 0 {
+ in = in[:d] + in[d+len("𝔻"):]
+ testOut = testOut[d+len("𝔻"):]
+ }
r := strings.NewReader(in)
buf, err := read(r)
if err != nil {
@@ -264,6 +290,12 @@ var readEmbedTests = []struct {
test:3:14:y
test:3:16:z`,
},
+ {
+ "\ufeffpackage p\nimport \"embed\"\n//go:embed x y z\nvar files embed.FS",
+ `test:3:12:x
+ test:3:14:y
+ test:3:16:z`,
+ },
{
"package p\nimport \"embed\"\nvar s = \"/*\"\n//go:embed x\nvar files embed.FS",
`test:4:12:x`,
@@ -292,6 +324,10 @@ var readEmbedTests = []struct {
"package p\n//go:embed x y z\nvar files embed.FS", // no import, no scan
"",
},
+ {
+ "\ufeffpackage p\n//go:embed x y z\nvar files embed.FS", // no import, no scan
+ "",
+ },
}
func TestReadEmbed(t *testing.T) {
diff --git a/src/go/internal/gcimporter/gcimporter.go b/src/go/internal/gcimporter/gcimporter.go
index b74daca246..73cf6334fd 100644
--- a/src/go/internal/gcimporter/gcimporter.go
+++ b/src/go/internal/gcimporter/gcimporter.go
@@ -145,17 +145,14 @@ func Import(fset *token.FileSet, packages map[string]*types.Package, path, srcDi
err = fmt.Errorf("import %q: old textual export format no longer supported (recompile library)", path)
case "$$B\n":
- var data []byte
- data, err = io.ReadAll(buf)
- if err != nil {
- break
- }
+ var exportFormat byte
+ exportFormat, err = buf.ReadByte()
// The indexed export format starts with an 'i'; the older
// binary export format starts with a 'c', 'd', or 'v'
// (from "version"). Select appropriate importer.
- if len(data) > 0 && data[0] == 'i' {
- _, pkg, err = iImportData(fset, packages, data[1:], id)
+ if err == nil && exportFormat == 'i' {
+ pkg, err = iImportData(fset, packages, buf, id)
} else {
err = fmt.Errorf("import %q: old binary export format no longer supported (recompile library)", path)
}
diff --git a/src/go/internal/gcimporter/iimport.go b/src/go/internal/gcimporter/iimport.go
index a3184e7641..76d47d08f1 100644
--- a/src/go/internal/gcimporter/iimport.go
+++ b/src/go/internal/gcimporter/iimport.go
@@ -8,6 +8,7 @@
package gcimporter
import (
+ "bufio"
"bytes"
"encoding/binary"
"fmt"
@@ -20,7 +21,7 @@ import (
)
type intReader struct {
- *bytes.Reader
+ *bufio.Reader
path string
}
@@ -61,7 +62,7 @@ const (
// and returns the number of bytes consumed and a reference to the package.
// If the export data version is not recognized or the format is otherwise
// compromised, an error is returned.
-func iImportData(fset *token.FileSet, imports map[string]*types.Package, data []byte, path string) (_ int, pkg *types.Package, err error) {
+func iImportData(fset *token.FileSet, imports map[string]*types.Package, dataReader *bufio.Reader, path string) (pkg *types.Package, err error) {
const currentVersion = 1
version := int64(-1)
defer func() {
@@ -74,7 +75,7 @@ func iImportData(fset *token.FileSet, imports map[string]*types.Package, data []
}
}()
- r := &intReader{bytes.NewReader(data), path}
+ r := &intReader{dataReader, path}
version = int64(r.uint64())
switch version {
@@ -86,10 +87,12 @@ func iImportData(fset *token.FileSet, imports map[string]*types.Package, data []
sLen := int64(r.uint64())
dLen := int64(r.uint64())
- whence, _ := r.Seek(0, io.SeekCurrent)
- stringData := data[whence : whence+sLen]
- declData := data[whence+sLen : whence+sLen+dLen]
- r.Seek(sLen+dLen, io.SeekCurrent)
+ data := make([]byte, sLen+dLen)
+ if _, err := io.ReadFull(r, data); err != nil {
+ errorf("cannot read %d bytes of stringData and declData: %s", len(data), err)
+ }
+ stringData := data[:sLen]
+ declData := data[sLen:]
p := iimporter{
ipath: path,
@@ -165,9 +168,7 @@ func iImportData(fset *token.FileSet, imports map[string]*types.Package, data []
// package was imported completely and without errors
localpkg.MarkComplete()
-
- consumed, _ := r.Seek(0, io.SeekCurrent)
- return int(consumed), localpkg, nil
+ return localpkg, nil
}
type iimporter struct {
diff --git a/src/go/parser/parser.go b/src/go/parser/parser.go
index 3965641713..f10c8650af 100644
--- a/src/go/parser/parser.go
+++ b/src/go/parser/parser.go
@@ -1302,7 +1302,12 @@ func (p *parser) parseIndexOrSliceOrInstance(x ast.Expr) ast.Expr {
p.errorExpected(p.pos, "operand")
rbrack := p.pos
p.next()
- return &ast.BadExpr{From: x.Pos(), To: rbrack}
+ return &ast.IndexExpr{
+ X: x,
+ Lbrack: lbrack,
+ Index: &ast.BadExpr{From: rbrack, To: rbrack},
+ Rbrack: rbrack,
+ }
}
p.exprLev++
diff --git a/src/go/scanner/scanner.go b/src/go/scanner/scanner.go
index 29cbf39721..f08e28cdd6 100644
--- a/src/go/scanner/scanner.go
+++ b/src/go/scanner/scanner.go
@@ -373,7 +373,7 @@ func (s *Scanner) scanIdentifier() string {
continue
}
s.rdOffset += rdOffset
- if b < utf8.RuneSelf {
+ if 0 < b && b < utf8.RuneSelf {
// Optimization: we've encountered an ASCII character that's not a letter
// or number. Avoid the call into s.next() and corresponding set up.
//
diff --git a/src/go/scanner/scanner_test.go b/src/go/scanner/scanner_test.go
index ac8d257716..db123c32e0 100644
--- a/src/go/scanner/scanner_test.go
+++ b/src/go/scanner/scanner_test.go
@@ -812,6 +812,8 @@ var errors = []struct {
{"//\ufeff", token.COMMENT, 2, "//\ufeff", "illegal byte order mark"}, // only first BOM is ignored
{"'\ufeff" + `'`, token.CHAR, 1, "'\ufeff" + `'`, "illegal byte order mark"}, // only first BOM is ignored
{`"` + "abc\ufeffdef" + `"`, token.STRING, 4, `"` + "abc\ufeffdef" + `"`, "illegal byte order mark"}, // only first BOM is ignored
+ {"abc\x00def", token.IDENT, 3, "abc", "illegal character NUL"},
+ {"abc\x00", token.IDENT, 3, "abc", "illegal character NUL"},
}
func TestScanErrors(t *testing.T) {
diff --git a/src/go/types/builtins.go b/src/go/types/builtins.go
index 739051cc61..2a2d54da88 100644
--- a/src/go/types/builtins.go
+++ b/src/go/types/builtins.go
@@ -588,6 +588,11 @@ func (check *Checker) builtin(x *operand, call *ast.CallExpr, id builtinId) (_ b
case _Add:
// unsafe.Add(ptr unsafe.Pointer, len IntegerType) unsafe.Pointer
+ if !check.allowVersion(check.pkg, 1, 17) {
+ check.errorf(call.Fun, _InvalidUnsafeAdd, "unsafe.Add requires go1.17 or later")
+ return
+ }
+
check.assignment(x, Typ[UnsafePointer], "argument to unsafe.Add")
if x.mode == invalid {
return
@@ -684,6 +689,11 @@ func (check *Checker) builtin(x *operand, call *ast.CallExpr, id builtinId) (_ b
case _Slice:
// unsafe.Slice(ptr *T, len IntegerType) []T
+ if !check.allowVersion(check.pkg, 1, 17) {
+ check.errorf(call.Fun, _InvalidUnsafeSlice, "unsafe.Slice requires go1.17 or later")
+ return
+ }
+
typ := asPointer(x.typ)
if typ == nil {
check.invalidArg(x, _InvalidUnsafeSlice, "%s is not a pointer", x)
diff --git a/src/go/types/check_test.go b/src/go/types/check_test.go
index 6c3b630a1b..c85a8e46fb 100644
--- a/src/go/types/check_test.go
+++ b/src/go/types/check_test.go
@@ -202,17 +202,20 @@ func asGoVersion(s string) string {
return ""
}
-func checkFiles(t *testing.T, sizes Sizes, goVersion string, filenames []string, srcs [][]byte, manual bool) {
+func checkFiles(t *testing.T, sizes Sizes, goVersion string, filenames []string, srcs [][]byte, manual bool, imp Importer) {
if len(filenames) == 0 {
t.Fatal("no source files")
}
+ if strings.HasSuffix(filenames[0], ".go2") && !typeparams.Enabled {
+ t.Skip("type params are not enabled")
+ }
+ if strings.HasSuffix(filenames[0], ".go1") && typeparams.Enabled {
+ t.Skip("type params are enabled")
+ }
+
mode := parser.AllErrors
- if strings.HasSuffix(filenames[0], ".go2") {
- if !typeparams.Enabled {
- t.Skip("type params are not enabled")
- }
- } else {
+ if !strings.HasSuffix(filenames[0], ".go2") {
mode |= typeparams.DisallowParsing
}
@@ -249,7 +252,10 @@ func checkFiles(t *testing.T, sizes Sizes, goVersion string, filenames []string,
}
}
- conf.Importer = importer.Default()
+ conf.Importer = imp
+ if imp == nil {
+ conf.Importer = importer.Default()
+ }
conf.Error = func(err error) {
if *haltOnError {
defer panic(err)
@@ -319,7 +325,7 @@ func TestManual(t *testing.T) {
func TestLongConstants(t *testing.T) {
format := "package longconst\n\nconst _ = %s\nconst _ = %s // ERROR excessively long constant"
src := fmt.Sprintf(format, strings.Repeat("1", 9999), strings.Repeat("1", 10001))
- checkFiles(t, nil, "", []string{"longconst.go"}, [][]byte{[]byte(src)}, false)
+ checkFiles(t, nil, "", []string{"longconst.go"}, [][]byte{[]byte(src)}, false, nil)
}
// TestIndexRepresentability tests that constant index operands must
@@ -327,7 +333,7 @@ func TestLongConstants(t *testing.T) {
// represent larger values.
func TestIndexRepresentability(t *testing.T) {
const src = "package index\n\nvar s []byte\nvar _ = s[int64 /* ERROR \"int64\\(1\\) << 40 \\(.*\\) overflows int\" */ (1) << 40]"
- checkFiles(t, &StdSizes{4, 4}, "", []string{"index.go"}, [][]byte{[]byte(src)}, false)
+ checkFiles(t, &StdSizes{4, 4}, "", []string{"index.go"}, [][]byte{[]byte(src)}, false, nil)
}
func TestIssue46453(t *testing.T) {
@@ -335,7 +341,7 @@ func TestIssue46453(t *testing.T) {
t.Skip("type params are enabled")
}
const src = "package p\ntype _ comparable // ERROR \"undeclared name: comparable\""
- checkFiles(t, nil, "", []string{"issue46453.go"}, [][]byte{[]byte(src)}, false)
+ checkFiles(t, nil, "", []string{"issue46453.go"}, [][]byte{[]byte(src)}, false, nil)
}
func TestCheck(t *testing.T) { DefPredeclaredTestFuncs(); testDir(t, "check") }
@@ -385,5 +391,5 @@ func testPkg(t *testing.T, filenames []string, goVersion string, manual bool) {
}
srcs[i] = src
}
- checkFiles(t, nil, goVersion, filenames, srcs, manual)
+ checkFiles(t, nil, goVersion, filenames, srcs, manual, nil)
}
diff --git a/src/go/types/errors.go b/src/go/types/errors.go
index 19e9ae8d44..2263106417 100644
--- a/src/go/types/errors.go
+++ b/src/go/types/errors.go
@@ -31,7 +31,7 @@ func (check *Checker) qualifier(pkg *Package) string {
if check.pkgPathMap == nil {
check.pkgPathMap = make(map[string]map[string]bool)
check.seenPkgMap = make(map[*Package]bool)
- check.markImports(pkg)
+ check.markImports(check.pkg)
}
// If the same package name was used by multiple packages, display the full path.
if len(check.pkgPathMap[pkg.name]) > 1 {
diff --git a/src/go/types/issues_test.go b/src/go/types/issues_test.go
index 44926919ef..519e199536 100644
--- a/src/go/types/issues_test.go
+++ b/src/go/types/issues_test.go
@@ -577,42 +577,64 @@ func TestIssue44515(t *testing.T) {
}
func TestIssue43124(t *testing.T) {
- // TODO(rFindley) enhance the testdata tests to be able to express this type
- // of setup.
+ // TODO(rFindley) move this to testdata by enhancing support for importing.
// All involved packages have the same name (template). Error messages should
// disambiguate between text/template and html/template by printing the full
// path.
const (
asrc = `package a; import "text/template"; func F(template.Template) {}; func G(int) {}`
- bsrc = `package b; import ("a"; "html/template"); func _() { a.F(template.Template{}) }`
- csrc = `package c; import ("a"; "html/template"); func _() { a.G(template.Template{}) }`
+ bsrc = `
+package b
+
+import (
+ "a"
+ "html/template"
+)
+
+func _() {
+ // Packages should be fully qualified when there is ambiguity within the
+ // error string itself.
+ a.F(template /* ERROR cannot use.*html/template.* as .*text/template */ .Template{})
+}
+`
+ csrc = `
+package c
+
+import (
+ "a"
+ "fmt"
+ "html/template"
+)
+
+// Issue #46905: make sure template is not the first package qualified.
+var _ fmt.Stringer = 1 // ERROR cannot use 1.*as fmt\.Stringer
+
+// Packages should be fully qualified when there is ambiguity in reachable
+// packages. In this case both a (and for that matter html/template) import
+// text/template.
+func _() { a.G(template /* ERROR cannot use .*html/template.*Template */ .Template{}) }
+`
+
+ tsrc = `
+package template
+
+import "text/template"
+
+type T int
+
+// Verify that the current package name also causes disambiguation.
+var _ T = template /* ERROR cannot use.*text/template.* as T value */.Template{}
+`
)
a, err := pkgFor("a", asrc, nil)
if err != nil {
t.Fatalf("package a failed to typecheck: %v", err)
}
- conf := Config{Importer: importHelper{pkg: a, fallback: importer.Default()}}
+ imp := importHelper{pkg: a, fallback: importer.Default()}
- // Packages should be fully qualified when there is ambiguity within the
- // error string itself.
- bast := mustParse(t, bsrc)
- _, err = conf.Check(bast.Name.Name, fset, []*ast.File{bast}, nil)
- if err == nil {
- t.Fatal("package b had no errors")
- }
- if !strings.Contains(err.Error(), "text/template") || !strings.Contains(err.Error(), "html/template") {
- t.Errorf("type checking error for b does not disambiguate package template: %q", err)
- }
-
- // ...and also when there is any ambiguity in reachable packages.
- cast := mustParse(t, csrc)
- _, err = conf.Check(cast.Name.Name, fset, []*ast.File{cast}, nil)
- if err == nil {
- t.Fatal("package c had no errors")
- }
- if !strings.Contains(err.Error(), "html/template") {
- t.Errorf("type checking error for c does not disambiguate package template: %q", err)
- }
+ checkFiles(t, nil, "", []string{"b.go"}, [][]byte{[]byte(bsrc)}, false, imp)
+ checkFiles(t, nil, "", []string{"c.go"}, [][]byte{[]byte(csrc)}, false, imp)
+ checkFiles(t, nil, "", []string{"t.go"}, [][]byte{[]byte(tsrc)}, false, imp)
}
diff --git a/src/go/types/stdlib_test.go b/src/go/types/stdlib_test.go
index 503d0a6f44..d86a77a110 100644
--- a/src/go/types/stdlib_test.go
+++ b/src/go/types/stdlib_test.go
@@ -26,9 +26,15 @@ import (
. "go/types"
)
+// The cmd/*/internal packages may have been deleted as part of a binary
+// release. Import from source instead.
+//
+// (See https://golang.org/issue/43232 and
+// https://github.com/golang/build/blob/df58bbac082bc87c4a3cdfe336d1ffe60bbaa916/cmd/release/release.go#L533-L545.)
+//
// Use the same importer for all std lib tests to
// avoid repeated importing of the same packages.
-var stdLibImporter = importer.Default()
+var stdLibImporter = importer.ForCompiler(token.NewFileSet(), "source", nil)
func TestStdlib(t *testing.T) {
testenv.MustHaveGoBuild(t)
diff --git a/src/go/types/testdata/check/issues.src b/src/go/types/testdata/check/issues.src
index e2ac06759b..55fe220337 100644
--- a/src/go/types/testdata/check/issues.src
+++ b/src/go/types/testdata/check/issues.src
@@ -6,7 +6,7 @@ package issues
import (
"fmt"
- syn "cmd/compile/internal/syntax"
+ syn "regexp/syntax"
t1 "text/template"
t2 "html/template"
)
@@ -329,10 +329,10 @@ func (... /* ERROR can only use ... with final parameter */ TT) f()
func issue28281g() (... /* ERROR can only use ... with final parameter */ TT)
// Issue #26234: Make various field/method lookup errors easier to read by matching cmd/compile's output
-func issue26234a(f *syn.File) {
+func issue26234a(f *syn.Prog) {
// The error message below should refer to the actual package name (syntax)
// not the local package name (syn).
- f.foo /* ERROR f.foo undefined \(type \*syntax.File has no field or method foo\) */
+ f.foo /* ERROR f\.foo undefined \(type \*syntax\.Prog has no field or method foo\) */
}
type T struct {
@@ -357,11 +357,11 @@ func issue35895() {
var _ T = 0 // ERROR cannot use 0 \(untyped int constant\) as T
// There is only one package with name syntax imported, only use the (global) package name in error messages.
- var _ *syn.File = 0 // ERROR cannot use 0 \(untyped int constant\) as \*syntax.File
+ var _ *syn.Prog = 0 // ERROR cannot use 0 \(untyped int constant\) as \*syntax.Prog
// Because both t1 and t2 have the same global package name (template),
// qualify packages with full path name in this case.
- var _ t1.Template = t2 /* ERROR cannot use .* \(value of type "html/template".Template\) as "text/template".Template */ .Template{}
+ var _ t1.Template = t2 /* ERROR cannot use .* \(value of type .html/template.\.Template\) as .text/template.\.Template */ .Template{}
}
func issue42989(s uint) {
diff --git a/src/go/types/testdata/examples/functions.go2 b/src/go/types/testdata/examples/functions.go2
index fb74ae7ae2..a053471202 100644
--- a/src/go/types/testdata/examples/functions.go2
+++ b/src/go/types/testdata/examples/functions.go2
@@ -210,5 +210,5 @@ func _() {
func h[] /* ERROR empty type parameter list */ ()
func _() {
- h[] /* ERROR operand */ ()
+ h /* ERROR cannot index */ [] /* ERROR operand */ ()
}
diff --git a/src/go/types/testdata/fixedbugs/issue46403.src b/src/go/types/testdata/fixedbugs/issue46403.src
new file mode 100644
index 0000000000..9d475222ad
--- /dev/null
+++ b/src/go/types/testdata/fixedbugs/issue46403.src
@@ -0,0 +1,11 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package issue46403
+
+func _() {
+ // a should be used, despite the parser error below.
+ var a []int
+ var _ = a[] // ERROR expected operand
+}
diff --git a/src/go/types/testdata/fixedbugs/issue46404.go1 b/src/go/types/testdata/fixedbugs/issue46404.go1
new file mode 100644
index 0000000000..db604bc1ac
--- /dev/null
+++ b/src/go/types/testdata/fixedbugs/issue46404.go1
@@ -0,0 +1,8 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package issue46404
+
+// Check that we don't type check t[_] as an instantiation.
+type t [t /* ERROR not a type */ [_]]_ // ERROR cannot use
diff --git a/src/go/types/typexpr.go b/src/go/types/typexpr.go
index 5185c33fcb..1738d864a6 100644
--- a/src/go/types/typexpr.go
+++ b/src/go/types/typexpr.go
@@ -463,8 +463,12 @@ func (check *Checker) typInternal(e0 ast.Expr, def *Named) (T Type) {
}
case *ast.IndexExpr:
- exprs := typeparams.UnpackExpr(e.Index)
- return check.instantiatedType(e.X, exprs, def)
+ if typeparams.Enabled {
+ exprs := typeparams.UnpackExpr(e.Index)
+ return check.instantiatedType(e.X, exprs, def)
+ }
+ check.errorf(e0, _NotAType, "%s is not a type", e0)
+ check.use(e.X)
case *ast.ParenExpr:
// Generic types must be instantiated before they can be used in any form.
diff --git a/src/image/draw/draw.go b/src/image/draw/draw.go
index 8f96aa2d18..13f6668293 100644
--- a/src/image/draw/draw.go
+++ b/src/image/draw/draw.go
@@ -23,6 +23,16 @@ type Image interface {
Set(x, y int, c color.Color)
}
+// RGBA64Image extends both the Image and image.RGBA64Image interfaces with a
+// SetRGBA64 method to change a single pixel. SetRGBA64 is equivalent to
+// calling Set, but it can avoid allocations from converting concrete color
+// types to the color.Color interface type.
+type RGBA64Image interface {
+ image.RGBA64Image
+ Set(x, y int, c color.Color)
+ SetRGBA64(x, y int, c color.RGBA64)
+}
+
// Quantizer produces a palette for an image.
type Quantizer interface {
// Quantize appends up to cap(p) - len(p) colors to p and returns the
diff --git a/src/image/image.go b/src/image/image.go
index 8adba96ab6..930d9ac6c7 100644
--- a/src/image/image.go
+++ b/src/image/image.go
@@ -45,6 +45,17 @@ type Image interface {
At(x, y int) color.Color
}
+// RGBA64Image is an Image whose pixels can be converted directly to a
+// color.RGBA64.
+type RGBA64Image interface {
+ // RGBA64At returns the RGBA64 color of the pixel at (x, y). It is
+ // equivalent to calling At(x, y).RGBA() and converting the resulting
+ // 32-bit return values to a color.RGBA64, but it can avoid allocations
+ // from converting concrete color types to the color.Color interface type.
+ RGBA64At(x, y int) color.RGBA64
+ Image
+}
+
// PalettedImage is an image whose colors may come from a limited palette.
// If m is a PalettedImage and m.ColorModel() returns a color.Palette p,
// then m.At(x, y) should be equivalent to p[m.ColorIndexAt(x, y)]. If m's
@@ -90,6 +101,24 @@ func (p *RGBA) At(x, y int) color.Color {
return p.RGBAAt(x, y)
}
+func (p *RGBA) RGBA64At(x, y int) color.RGBA64 {
+ if !(Point{x, y}.In(p.Rect)) {
+ return color.RGBA64{}
+ }
+ i := p.PixOffset(x, y)
+ s := p.Pix[i : i+4 : i+4] // Small cap improves performance, see https://golang.org/issue/27857
+ r := uint16(s[0])
+ g := uint16(s[1])
+ b := uint16(s[2])
+ a := uint16(s[3])
+ return color.RGBA64{
+ (r << 8) | r,
+ (g << 8) | g,
+ (b << 8) | b,
+ (a << 8) | a,
+ }
+}
+
func (p *RGBA) RGBAAt(x, y int) color.RGBA {
if !(Point{x, y}.In(p.Rect)) {
return color.RGBA{}
@@ -118,6 +147,18 @@ func (p *RGBA) Set(x, y int, c color.Color) {
s[3] = c1.A
}
+func (p *RGBA) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ i := p.PixOffset(x, y)
+ s := p.Pix[i : i+4 : i+4] // Small cap improves performance, see https://golang.org/issue/27857
+ s[0] = uint8(c.R >> 8)
+ s[1] = uint8(c.G >> 8)
+ s[2] = uint8(c.B >> 8)
+ s[3] = uint8(c.A >> 8)
+}
+
func (p *RGBA) SetRGBA(x, y int, c color.RGBA) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -311,6 +352,11 @@ func (p *NRGBA) At(x, y int) color.Color {
return p.NRGBAAt(x, y)
}
+func (p *NRGBA) RGBA64At(x, y int) color.RGBA64 {
+ r, g, b, a := p.NRGBAAt(x, y).RGBA()
+ return color.RGBA64{uint16(r), uint16(g), uint16(b), uint16(a)}
+}
+
func (p *NRGBA) NRGBAAt(x, y int) color.NRGBA {
if !(Point{x, y}.In(p.Rect)) {
return color.NRGBA{}
@@ -339,6 +385,24 @@ func (p *NRGBA) Set(x, y int, c color.Color) {
s[3] = c1.A
}
+func (p *NRGBA) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ r, g, b, a := uint32(c.R), uint32(c.G), uint32(c.B), uint32(c.A)
+ if (a != 0) && (a != 0xffff) {
+ r = (r * 0xffff) / a
+ g = (g * 0xffff) / a
+ b = (b * 0xffff) / a
+ }
+ i := p.PixOffset(x, y)
+ s := p.Pix[i : i+4 : i+4] // Small cap improves performance, see https://golang.org/issue/27857
+ s[0] = uint8(r >> 8)
+ s[1] = uint8(g >> 8)
+ s[2] = uint8(b >> 8)
+ s[3] = uint8(a >> 8)
+}
+
func (p *NRGBA) SetNRGBA(x, y int, c color.NRGBA) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -415,6 +479,11 @@ func (p *NRGBA64) At(x, y int) color.Color {
return p.NRGBA64At(x, y)
}
+func (p *NRGBA64) RGBA64At(x, y int) color.RGBA64 {
+ r, g, b, a := p.NRGBA64At(x, y).RGBA()
+ return color.RGBA64{uint16(r), uint16(g), uint16(b), uint16(a)}
+}
+
func (p *NRGBA64) NRGBA64At(x, y int) color.NRGBA64 {
if !(Point{x, y}.In(p.Rect)) {
return color.NRGBA64{}
@@ -452,6 +521,28 @@ func (p *NRGBA64) Set(x, y int, c color.Color) {
s[7] = uint8(c1.A)
}
+func (p *NRGBA64) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ r, g, b, a := uint32(c.R), uint32(c.G), uint32(c.B), uint32(c.A)
+ if (a != 0) && (a != 0xffff) {
+ r = (r * 0xffff) / a
+ g = (g * 0xffff) / a
+ b = (b * 0xffff) / a
+ }
+ i := p.PixOffset(x, y)
+ s := p.Pix[i : i+8 : i+8] // Small cap improves performance, see https://golang.org/issue/27857
+ s[0] = uint8(r >> 8)
+ s[1] = uint8(r)
+ s[2] = uint8(g >> 8)
+ s[3] = uint8(g)
+ s[4] = uint8(b >> 8)
+ s[5] = uint8(b)
+ s[6] = uint8(a >> 8)
+ s[7] = uint8(a)
+}
+
func (p *NRGBA64) SetNRGBA64(x, y int, c color.NRGBA64) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -532,6 +623,12 @@ func (p *Alpha) At(x, y int) color.Color {
return p.AlphaAt(x, y)
}
+func (p *Alpha) RGBA64At(x, y int) color.RGBA64 {
+ a := uint16(p.AlphaAt(x, y).A)
+ a |= a << 8
+ return color.RGBA64{a, a, a, a}
+}
+
func (p *Alpha) AlphaAt(x, y int) color.Alpha {
if !(Point{x, y}.In(p.Rect)) {
return color.Alpha{}
@@ -554,6 +651,14 @@ func (p *Alpha) Set(x, y int, c color.Color) {
p.Pix[i] = color.AlphaModel.Convert(c).(color.Alpha).A
}
+func (p *Alpha) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ i := p.PixOffset(x, y)
+ p.Pix[i] = uint8(c.A >> 8)
+}
+
func (p *Alpha) SetAlpha(x, y int, c color.Alpha) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -626,6 +731,11 @@ func (p *Alpha16) At(x, y int) color.Color {
return p.Alpha16At(x, y)
}
+func (p *Alpha16) RGBA64At(x, y int) color.RGBA64 {
+ a := p.Alpha16At(x, y).A
+ return color.RGBA64{a, a, a, a}
+}
+
func (p *Alpha16) Alpha16At(x, y int) color.Alpha16 {
if !(Point{x, y}.In(p.Rect)) {
return color.Alpha16{}
@@ -650,6 +760,15 @@ func (p *Alpha16) Set(x, y int, c color.Color) {
p.Pix[i+1] = uint8(c1.A)
}
+func (p *Alpha16) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ i := p.PixOffset(x, y)
+ p.Pix[i+0] = uint8(c.A >> 8)
+ p.Pix[i+1] = uint8(c.A)
+}
+
func (p *Alpha16) SetAlpha16(x, y int, c color.Alpha16) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -723,6 +842,12 @@ func (p *Gray) At(x, y int) color.Color {
return p.GrayAt(x, y)
}
+func (p *Gray) RGBA64At(x, y int) color.RGBA64 {
+ gray := uint16(p.GrayAt(x, y).Y)
+ gray |= gray << 8
+ return color.RGBA64{gray, gray, gray, 0xffff}
+}
+
func (p *Gray) GrayAt(x, y int) color.Gray {
if !(Point{x, y}.In(p.Rect)) {
return color.Gray{}
@@ -745,6 +870,16 @@ func (p *Gray) Set(x, y int, c color.Color) {
p.Pix[i] = color.GrayModel.Convert(c).(color.Gray).Y
}
+func (p *Gray) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ // This formula is the same as in color.grayModel.
+ gray := (19595*uint32(c.R) + 38470*uint32(c.G) + 7471*uint32(c.B) + 1<<15) >> 24
+ i := p.PixOffset(x, y)
+ p.Pix[i] = uint8(gray)
+}
+
func (p *Gray) SetGray(x, y int, c color.Gray) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -804,6 +939,11 @@ func (p *Gray16) At(x, y int) color.Color {
return p.Gray16At(x, y)
}
+func (p *Gray16) RGBA64At(x, y int) color.RGBA64 {
+ gray := p.Gray16At(x, y).Y
+ return color.RGBA64{gray, gray, gray, 0xffff}
+}
+
func (p *Gray16) Gray16At(x, y int) color.Gray16 {
if !(Point{x, y}.In(p.Rect)) {
return color.Gray16{}
@@ -828,6 +968,17 @@ func (p *Gray16) Set(x, y int, c color.Color) {
p.Pix[i+1] = uint8(c1.Y)
}
+func (p *Gray16) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ // This formula is the same as in color.gray16Model.
+ gray := (19595*uint32(c.R) + 38470*uint32(c.G) + 7471*uint32(c.B) + 1<<15) >> 16
+ i := p.PixOffset(x, y)
+ p.Pix[i+0] = uint8(gray >> 8)
+ p.Pix[i+1] = uint8(gray)
+}
+
func (p *Gray16) SetGray16(x, y int, c color.Gray16) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -888,6 +1039,11 @@ func (p *CMYK) At(x, y int) color.Color {
return p.CMYKAt(x, y)
}
+func (p *CMYK) RGBA64At(x, y int) color.RGBA64 {
+ r, g, b, a := p.CMYKAt(x, y).RGBA()
+ return color.RGBA64{uint16(r), uint16(g), uint16(b), uint16(a)}
+}
+
func (p *CMYK) CMYKAt(x, y int) color.CMYK {
if !(Point{x, y}.In(p.Rect)) {
return color.CMYK{}
@@ -916,6 +1072,19 @@ func (p *CMYK) Set(x, y int, c color.Color) {
s[3] = c1.K
}
+func (p *CMYK) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ cc, mm, yy, kk := color.RGBToCMYK(uint8(c.R>>8), uint8(c.G>>8), uint8(c.B>>8))
+ i := p.PixOffset(x, y)
+ s := p.Pix[i : i+4 : i+4] // Small cap improves performance, see https://golang.org/issue/27857
+ s[0] = cc
+ s[1] = mm
+ s[2] = yy
+ s[3] = kk
+}
+
func (p *CMYK) SetCMYK(x, y int, c color.CMYK) {
if !(Point{x, y}.In(p.Rect)) {
return
@@ -988,6 +1157,26 @@ func (p *Paletted) At(x, y int) color.Color {
return p.Palette[p.Pix[i]]
}
+func (p *Paletted) RGBA64At(x, y int) color.RGBA64 {
+ if len(p.Palette) == 0 {
+ return color.RGBA64{}
+ }
+ c := color.Color(nil)
+ if !(Point{x, y}.In(p.Rect)) {
+ c = p.Palette[0]
+ } else {
+ i := p.PixOffset(x, y)
+ c = p.Palette[p.Pix[i]]
+ }
+ r, g, b, a := c.RGBA()
+ return color.RGBA64{
+ uint16(r),
+ uint16(g),
+ uint16(b),
+ uint16(a),
+ }
+}
+
// PixOffset returns the index of the first element of Pix that corresponds to
// the pixel at (x, y).
func (p *Paletted) PixOffset(x, y int) int {
@@ -1002,6 +1191,14 @@ func (p *Paletted) Set(x, y int, c color.Color) {
p.Pix[i] = uint8(p.Palette.Index(c))
}
+func (p *Paletted) SetRGBA64(x, y int, c color.RGBA64) {
+ if !(Point{x, y}.In(p.Rect)) {
+ return
+ }
+ i := p.PixOffset(x, y)
+ p.Pix[i] = uint8(p.Palette.Index(c))
+}
+
func (p *Paletted) ColorIndexAt(x, y int) uint8 {
if !(Point{x, y}.In(p.Rect)) {
return 0
diff --git a/src/image/image_test.go b/src/image/image_test.go
index b9b9bfaa28..c64b6107b7 100644
--- a/src/image/image_test.go
+++ b/src/image/image_test.go
@@ -6,6 +6,7 @@ package image
import (
"image/color"
+ "image/color/palette"
"testing"
)
@@ -191,6 +192,80 @@ func Test16BitsPerColorChannel(t *testing.T) {
}
}
+func TestRGBA64Image(t *testing.T) {
+ // memset sets every element of s to v.
+ memset := func(s []byte, v byte) {
+ for i := range s {
+ s[i] = v
+ }
+ }
+
+ r := Rect(0, 0, 3, 2)
+ testCases := []Image{
+ NewAlpha(r),
+ NewAlpha16(r),
+ NewCMYK(r),
+ NewGray(r),
+ NewGray16(r),
+ NewNRGBA(r),
+ NewNRGBA64(r),
+ NewNYCbCrA(r, YCbCrSubsampleRatio444),
+ NewPaletted(r, palette.Plan9),
+ NewRGBA(r),
+ NewRGBA64(r),
+ NewYCbCr(r, YCbCrSubsampleRatio444),
+ }
+ for _, tc := range testCases {
+ switch tc := tc.(type) {
+ // Most of the concrete image types in the testCases implement the
+ // draw.RGBA64Image interface: they have a SetRGBA64 method. We use an
+ // interface literal here, instead of importing "image/draw", to avoid
+ // an import cycle.
+ //
+ // The YCbCr and NYCbCrA types are special-cased. Chroma subsampling
+ // means that setting one pixel can modify neighboring pixels. They
+ // don't have Set or SetRGBA64 methods because that side effect could
+ // be surprising. Here, we just memset the channel buffers instead.
+ case interface {
+ SetRGBA64(x, y int, c color.RGBA64)
+ }:
+ tc.SetRGBA64(1, 1, color.RGBA64{0x7FFF, 0x3FFF, 0x0000, 0x7FFF})
+
+ case *NYCbCrA:
+ memset(tc.YCbCr.Y, 0x77)
+ memset(tc.YCbCr.Cb, 0x88)
+ memset(tc.YCbCr.Cr, 0x99)
+ memset(tc.A, 0xAA)
+
+ case *YCbCr:
+ memset(tc.Y, 0x77)
+ memset(tc.Cb, 0x88)
+ memset(tc.Cr, 0x99)
+
+ default:
+ t.Errorf("could not initialize pixels for %T", tc)
+ continue
+ }
+
+ // Check that RGBA64At(x, y) is equivalent to At(x, y).RGBA().
+ rgba64Image, ok := tc.(RGBA64Image)
+ if !ok {
+ t.Errorf("%T is not an RGBA64Image", tc)
+ continue
+ }
+ got := rgba64Image.RGBA64At(1, 1)
+ wantR, wantG, wantB, wantA := tc.At(1, 1).RGBA()
+ if (uint32(got.R) != wantR) || (uint32(got.G) != wantG) ||
+ (uint32(got.B) != wantB) || (uint32(got.A) != wantA) {
+ t.Errorf("%T:\ngot (0x%04X, 0x%04X, 0x%04X, 0x%04X)\n"+
+ "want (0x%04X, 0x%04X, 0x%04X, 0x%04X)", tc,
+ got.R, got.G, got.B, got.A,
+ wantR, wantG, wantB, wantA)
+ continue
+ }
+ }
+}
+
func BenchmarkAt(b *testing.B) {
for _, tc := range testImages {
b.Run(tc.name, func(b *testing.B) {
diff --git a/src/image/ycbcr.go b/src/image/ycbcr.go
index fbdffe1bd1..328b90d152 100644
--- a/src/image/ycbcr.go
+++ b/src/image/ycbcr.go
@@ -71,6 +71,11 @@ func (p *YCbCr) At(x, y int) color.Color {
return p.YCbCrAt(x, y)
}
+func (p *YCbCr) RGBA64At(x, y int) color.RGBA64 {
+ r, g, b, a := p.YCbCrAt(x, y).RGBA()
+ return color.RGBA64{uint16(r), uint16(g), uint16(b), uint16(a)}
+}
+
func (p *YCbCr) YCbCrAt(x, y int) color.YCbCr {
if !(Point{x, y}.In(p.Rect)) {
return color.YCbCr{}
@@ -210,6 +215,11 @@ func (p *NYCbCrA) At(x, y int) color.Color {
return p.NYCbCrAAt(x, y)
}
+func (p *NYCbCrA) RGBA64At(x, y int) color.RGBA64 {
+ r, g, b, a := p.NYCbCrAAt(x, y).RGBA()
+ return color.RGBA64{uint16(r), uint16(g), uint16(b), uint16(a)}
+}
+
func (p *NYCbCrA) NYCbCrAAt(x, y int) color.NYCbCrA {
if !(Point{X: x, Y: y}.In(p.Rect)) {
return color.NYCbCrA{}
diff --git a/src/internal/buildcfg/exp.go b/src/internal/buildcfg/exp.go
index 2435a79dce..9a60253aab 100644
--- a/src/internal/buildcfg/exp.go
+++ b/src/internal/buildcfg/exp.go
@@ -6,7 +6,6 @@ package buildcfg
import (
"fmt"
- "os"
"reflect"
"strings"
@@ -18,20 +17,19 @@ import (
//
// (This is not necessarily the set of experiments the compiler itself
// was built with.)
-var Experiment goexperiment.Flags = parseExperiments()
-
-var regabiSupported = GOARCH == "amd64" && (GOOS == "android" || GOOS == "linux" || GOOS == "darwin" || GOOS == "windows")
-
+//
// experimentBaseline specifies the experiment flags that are enabled by
// default in the current toolchain. This is, in effect, the "control"
// configuration and any variation from this is an experiment.
-var experimentBaseline = goexperiment.Flags{
- RegabiWrappers: regabiSupported,
- RegabiG: regabiSupported,
- RegabiReflect: regabiSupported,
- RegabiDefer: regabiSupported,
- RegabiArgs: regabiSupported,
-}
+var Experiment, experimentBaseline = func() (goexperiment.Flags, goexperiment.Flags) {
+ flags, baseline, err := ParseGOEXPERIMENT(GOOS, GOARCH, envOr("GOEXPERIMENT", defaultGOEXPERIMENT))
+ if err != nil {
+ Error = err
+ }
+ return flags, baseline
+}()
+
+const DefaultGOEXPERIMENT = defaultGOEXPERIMENT
// FramePointerEnabled enables the use of platform conventions for
// saving frame pointers.
@@ -42,16 +40,29 @@ var experimentBaseline = goexperiment.Flags{
// Note: must agree with runtime.framepointer_enabled.
var FramePointerEnabled = GOARCH == "amd64" || GOARCH == "arm64"
-func parseExperiments() goexperiment.Flags {
+// ParseGOEXPERIMENT parses a (GOOS, GOARCH, GOEXPERIMENT)
+// configuration tuple and returns the enabled and baseline experiment
+// flag sets.
+//
+// TODO(mdempsky): Move to internal/goexperiment.
+func ParseGOEXPERIMENT(goos, goarch, goexp string) (flags, baseline goexperiment.Flags, err error) {
+ regabiSupported := goarch == "amd64" && (goos == "android" || goos == "linux" || goos == "darwin" || goos == "windows")
+
+ baseline = goexperiment.Flags{
+ RegabiWrappers: regabiSupported,
+ RegabiG: regabiSupported,
+ RegabiReflect: regabiSupported,
+ RegabiDefer: regabiSupported,
+ RegabiArgs: regabiSupported,
+ }
+
// Start with the statically enabled set of experiments.
- flags := experimentBaseline
+ flags = baseline
// Pick up any changes to the baseline configuration from the
// GOEXPERIMENT environment. This can be set at make.bash time
// and overridden at build time.
- env := envOr("GOEXPERIMENT", defaultGOEXPERIMENT)
-
- if env != "" {
+ if goexp != "" {
// Create a map of known experiment names.
names := make(map[string]func(bool))
rv := reflect.ValueOf(&flags).Elem()
@@ -74,7 +85,7 @@ func parseExperiments() goexperiment.Flags {
}
// Parse names.
- for _, f := range strings.Split(env, ",") {
+ for _, f := range strings.Split(goexp, ",") {
if f == "" {
continue
}
@@ -91,15 +102,15 @@ func parseExperiments() goexperiment.Flags {
}
set, ok := names[f]
if !ok {
- fmt.Printf("unknown experiment %s\n", f)
- os.Exit(2)
+ err = fmt.Errorf("unknown GOEXPERIMENT %s", f)
+ return
}
set(val)
}
}
// regabi is only supported on amd64.
- if GOARCH != "amd64" {
+ if goarch != "amd64" {
flags.RegabiWrappers = false
flags.RegabiG = false
flags.RegabiReflect = false
@@ -108,12 +119,12 @@ func parseExperiments() goexperiment.Flags {
}
// Check regabi dependencies.
if flags.RegabiG && !flags.RegabiWrappers {
- Error = fmt.Errorf("GOEXPERIMENT regabig requires regabiwrappers")
+ err = fmt.Errorf("GOEXPERIMENT regabig requires regabiwrappers")
}
if flags.RegabiArgs && !(flags.RegabiWrappers && flags.RegabiG && flags.RegabiReflect && flags.RegabiDefer) {
- Error = fmt.Errorf("GOEXPERIMENT regabiargs requires regabiwrappers,regabig,regabireflect,regabidefer")
+ err = fmt.Errorf("GOEXPERIMENT regabiargs requires regabiwrappers,regabig,regabireflect,regabidefer")
}
- return flags
+ return
}
// expList returns the list of lower-cased experiment names for
@@ -165,3 +176,14 @@ func EnabledExperiments() []string {
func AllExperiments() []string {
return expList(&Experiment, nil, true)
}
+
+// UpdateExperiments updates the Experiment global based on a new GOARCH value.
+// This is only required for cmd/go, which can change GOARCH after
+// program startup due to use of "go env -w".
+func UpdateExperiments(goos, goarch, goexperiment string) {
+ var err error
+ Experiment, experimentBaseline, err = ParseGOEXPERIMENT(goos, goarch, goexperiment)
+ if err != nil {
+ Error = err
+ }
+}
diff --git a/src/internal/bytealg/index_generic.go b/src/internal/bytealg/index_generic.go
index 287bdba4c6..0a6eb90d2d 100644
--- a/src/internal/bytealg/index_generic.go
+++ b/src/internal/bytealg/index_generic.go
@@ -2,7 +2,6 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
-// go:build !amd64 && !arm64 && !s390x && !ppc64le && !ppc64
//go:build !amd64 && !arm64 && !s390x && !ppc64le && !ppc64
// +build !amd64,!arm64,!s390x,!ppc64le,!ppc64
diff --git a/src/internal/bytealg/index_native.go b/src/internal/bytealg/index_native.go
index 75aff4b3cb..9547a5d8e2 100644
--- a/src/internal/bytealg/index_native.go
+++ b/src/internal/bytealg/index_native.go
@@ -2,7 +2,6 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
-// go:build amd64 || arm64 || s390x || ppc64le || ppc64
//go:build amd64 || arm64 || s390x || ppc64le || ppc64
// +build amd64 arm64 s390x ppc64le ppc64
diff --git a/src/math/big/natdiv.go b/src/math/big/natdiv.go
index 1330990c2c..882bb6d3ba 100644
--- a/src/math/big/natdiv.go
+++ b/src/math/big/natdiv.go
@@ -2,10 +2,506 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
+/*
+
+Multi-precision division. Here be dragons.
+
+Given u and v, where u is n+m digits, and v is n digits (with no leading zeros),
+the goal is to return quo, rem such that u = quo*v + rem, where 0 ≤ rem < v.
+That is, quo = ⌊u/v⌋ where ⌊x⌋ denotes the floor (truncation to integer) of x,
+and rem = u - quo·v.
+
+
+Long Division
+
+Division in a computer proceeds the same as long division in elementary school,
+but computers are not as good as schoolchildren at following vague directions,
+so we have to be much more precise about the actual steps and what can happen.
+
+We work from most to least significant digit of the quotient, doing:
+
+ • Guess a digit q, the number of v to subtract from the current
+ section of u to zero out the topmost digit.
+ • Check the guess by multiplying q·v and comparing it against
+ the current section of u, adjusting the guess as needed.
+ • Subtract q·v from the current section of u.
+ • Add q to the corresponding section of the result quo.
+
+When all digits have been processed, the final remainder is left in u
+and returned as rem.
+
+For example, here is a sketch of dividing 5 digits by 3 digits (n=3, m=2).
+
+ q₂ q₁ q₀
+ _________________
+ v₂ v₁ v₀ ) u₄ u₃ u₂ u₁ u₀
+ ↓ ↓ ↓ | |
+ [u₄ u₃ u₂]| |
+ - [ q₂·v ]| |
+ ----------- ↓ |
+ [ rem | u₁]|
+ - [ q₁·v ]|
+ ----------- ↓
+ [ rem | u₀]
+ - [ q₀·v ]
+ ------------
+ [ rem ]
+
+Instead of creating new storage for the remainders and copying digits from u
+as indicated by the arrows, we use u's storage directly as both the source
+and destination of the subtractions, so that the remainders overwrite
+successive overlapping sections of u as the division proceeds, using a slice
+of u to identify the current section. This avoids all the copying as well as
+shifting of remainders.
+
+Division of u with n+m digits by v with n digits (in base B) can in general
+produce at most m+1 digits, because:
+
+ • u < B^(n+m) [B^(n+m) has n+m+1 digits]
+ • v ≥ B^(n-1) [B^(n-1) is the smallest n-digit number]
+ • u/v < B^(n+m) / B^(n-1) [divide bounds for u, v]
+ • u/v < B^(m+1) [simplify]
+
+The first step is special: it takes the top n digits of u and divides them by
+the n digits of v, producing the first quotient digit and an n-digit remainder.
+In the example, q₂ = ⌊u₄u₃u₂ / v⌋.
+
+The first step divides n digits by n digits to ensure that it produces only a
+single digit.
+
+Each subsequent step appends the next digit from u to the remainder and divides
+those n+1 digits by the n digits of v, producing another quotient digit and a
+new n-digit remainder.
+
+Subsequent steps divide n+1 digits by n digits, an operation that in general
+might produce two digits. However, as used in the algorithm, that division is
+guaranteed to produce only a single digit. The dividend is of the form
+rem·B + d, where rem is a remainder from the previous step and d is a single
+digit, so:
+
+ • rem ≤ v - 1 [rem is a remainder from dividing by v]
+ • rem·B ≤ v·B - B [multiply by B]
+ • d ≤ B - 1 [d is a single digit]
+ • rem·B + d ≤ v·B - 1 [add]
+ • rem·B + d < v·B [change ≤ to <]
+ • (rem·B + d)/v < B [divide by v]
+
+
+Guess and Check
+
+At each step we need to divide n+1 digits by n digits, but this is for the
+implementation of division by n digits, so we can't just invoke a division
+routine: we _are_ the division routine. Instead, we guess at the answer and
+then check it using multiplication. If the guess is wrong, we correct it.
+
+How can this guessing possibly be efficient? It turns out that the following
+statement (let's call it the Good Guess Guarantee) is true.
+
+If
+
+ • q = ⌊u/v⌋ where u is n+1 digits and v is n digits,
+ • q < B, and
+ • the topmost digit of v = vₙ₋₁ ≥ B/2,
+
+then q̂ = ⌊uₙuₙ₋₁ / vₙ₋₁⌋ satisfies q ≤ q̂ ≤ q+2. (Proof below.)
+
+That is, if we know the answer has only a single digit and we guess an answer
+by ignoring the bottom n-1 digits of u and v, using a 2-by-1-digit division,
+then that guess is at least as large as the correct answer. It is also not
+too much larger: it is off by at most two from the correct answer.
+
+Note that in the first step of the overall division, which is an n-by-n-digit
+division, the 2-by-1 guess uses an implicit uₙ = 0.
+
+Note that using a 2-by-1-digit division here does not mean calling ourselves
+recursively. Instead, we use an efficient direct hardware implementation of
+that operation.
+
+Note that because q is u/v rounded down, q·v must not exceed u: u ≥ q·v.
+If a guess q̂ is too big, it will not satisfy this test. Viewed a different way,
+the remainder r̂ for a given q̂ is u - q̂·v, which must be positive. If it is
+negative, then the guess q̂ is too big.
+
+This gives us a way to compute q. First compute q̂ with 2-by-1-digit division.
+Then, while u < q̂·v, decrement q̂; this loop executes at most twice, because
+q̂ ≤ q+2.
+
+
+Scaling Inputs
+
+The Good Guess Guarantee requires that the top digit of v (vₙ₋₁) be at least B/2.
+For example in base 10, ⌊172/19⌋ = 9, but ⌊18/1⌋ = 18: the guess is wildly off
+because the first digit 1 is smaller than B/2 = 5.
+
+We can ensure that v has a large top digit by multiplying both u and v by the
+right amount. Continuing the example, if we multiply both 172 and 19 by 3, we
+now have ⌊516/57⌋, the leading digit of v is now ≥ 5, and sure enough
+⌊51/5⌋ = 10 is much closer to the correct answer 9. It would be easier here
+to multiply by 4, because that can be done with a shift. Specifically, we can
+always count the number of leading zeros i in the first digit of v and then
+shift both u and v left by i bits.
+
+Having scaled u and v, the value ⌊u/v⌋ is unchanged, but the remainder will
+be scaled: 172 mod 19 is 1, but 516 mod 57 is 3. We have to divide the remainder
+by the scaling factor (shifting right i bits) when we finish.
+
+Note that these shifts happen before and after the entire division algorithm,
+not at each step in the per-digit iteration.
+
+Note the effect of scaling inputs on the size of the possible quotient.
+In the scaled u/v, u can gain a digit from scaling; v never does, because we
+pick the scaling factor to make v's top digit larger but without overflowing.
+If u and v have n+m and n digits after scaling, then:
+
+ • u < B^(n+m) [B^(n+m) has n+m+1 digits]
+ • v ≥ B^n / 2 [vₙ₋₁ ≥ B/2, so vₙ₋₁·B^(n-1) ≥ B^n/2]
+ • u/v < B^(n+m) / (B^n / 2) [divide bounds for u, v]
+ • u/v < 2 B^m [simplify]
+
+The quotient can still have m+1 significant digits, but if so the top digit
+must be a 1. This provides a different way to handle the first digit of the
+result: compare the top n digits of u against v and fill in either a 0 or a 1.
+
+
+Refining Guesses
+
+Before we check whether u < q̂·v, we can adjust our guess to change it from
+q̂ = ⌊uₙuₙ₋₁ / vₙ₋₁⌋ into the refined guess ⌊uₙuₙ₋₁uₙ₋₂ / vₙ₋₁vₙ₋₂⌋.
+Although not mentioned above, the Good Guess Guarantee also promises that this
+3-by-2-digit division guess is more precise and at most one away from the real
+answer q. The improvement from the 2-by-1 to the 3-by-2 guess can also be done
+without n-digit math.
+
+If we have a guess q̂ = ⌊uₙuₙ₋₁ / vₙ₋₁⌋ and we want to see if it also equal to
+⌊uₙuₙ₋₁uₙ₋₂ / vₙ₋₁vₙ₋₂⌋, we can use the same check we would for the full division:
+if uₙuₙ₋₁uₙ₋₂ < q̂·vₙ₋₁vₙ₋₂, then the guess is too large and should be reduced.
+
+Checking uₙuₙ₋₁uₙ₋₂ < q̂·vₙ₋₁vₙ₋₂ is the same as uₙuₙ₋₁uₙ₋₂ - q̂·vₙ₋₁vₙ₋₂ < 0,
+and
+
+ uₙuₙ₋₁uₙ₋₂ - q̂·vₙ₋₁vₙ₋₂ = (uₙuₙ₋₁·B + uₙ₋₂) - q̂·(vₙ₋₁·B + vₙ₋₂)
+ [splitting off the bottom digit]
+ = (uₙuₙ₋₁ - q̂·vₙ₋₁)·B + uₙ₋₂ - q̂·vₙ₋₂
+ [regrouping]
+
+The expression (uₙuₙ₋₁ - q̂·vₙ₋₁) is the remainder of uₙuₙ₋₁ / vₙ₋₁.
+If the initial guess returns both q̂ and its remainder r̂, then checking
+whether uₙuₙ₋₁uₙ₋₂ < q̂·vₙ₋₁vₙ₋₂ is the same as checking r̂·B + uₙ₋₂ < q̂·vₙ₋₂.
+
+If we find that r̂·B + uₙ₋₂ < q̂·vₙ₋₂, then we can adjust the guess by
+decrementing q̂ and adding vₙ₋₁ to r̂. We repeat until r̂·B + uₙ₋₂ ≥ q̂·vₙ₋₂.
+(As before, this fixup is only needed at most twice.)
+
+Now that q̂ = ⌊uₙuₙ₋₁uₙ₋₂ / vₙ₋₁vₙ₋₂⌋, as mentioned above it is at most one
+away from the correct q, and we've avoided doing any n-digit math.
+(If we need the new remainder, it can be computed as r̂·B + uₙ₋₂ - q̂·vₙ₋₂.)
+
+The final check u < q̂·v and the possible fixup must be done at full precision.
+For random inputs, a fixup at this step is exceedingly rare: the 3-by-2 guess
+is not often wrong at all. But still we must do the check. Note that since the
+3-by-2 guess is off by at most 1, it can be convenient to perform the final
+u < q̂·v as part of the computation of the remainder r = u - q̂·v. If the
+subtraction underflows, decremeting q̂ and adding one v back to r is enough to
+arrive at the final q, r.
+
+That's the entirety of long division: scale the inputs, and then loop over
+each output position, guessing, checking, and correcting the next output digit.
+
+For a 2n-digit number divided by an n-digit number (the worst size-n case for
+division complexity), this algorithm uses n+1 iterations, each of which must do
+at least the 1-by-n-digit multiplication q̂·v. That's O(n) iterations of
+O(n) time each, so O(n²) time overall.
+
+
+Recursive Division
+
+For very large inputs, it is possible to improve on the O(n²) algorithm.
+Let's call a group of n/2 real digits a (very) “wide digit”. We can run the
+standard long division algorithm explained above over the wide digits instead of
+the actual digits. This will result in many fewer steps, but the math involved in
+each step is more work.
+
+Where basic long division uses a 2-by-1-digit division to guess the initial q̂,
+the new algorithm must use a 2-by-1-wide-digit division, which is of course
+really an n-by-n/2-digit division. That's OK: if we implement n-digit division
+in terms of n/2-digit division, the recursion will terminate when the divisor
+becomes small enough to handle with standard long division or even with the
+2-by-1 hardware instruction.
+
+For example, here is a sketch of dividing 10 digits by 4, proceeding with
+wide digits corresponding to two regular digits. The first step, still special,
+must leave off a (regular) digit, dividing 5 by 4 and producing a 4-digit
+remainder less than v. The middle steps divide 6 digits by 4, guaranteed to
+produce two output digits each (one wide digit) with 4-digit remainders.
+The final step must use what it has: the 4-digit remainder plus one more,
+5 digits to divide by 4.
+
+ q₆ q₅ q₄ q₃ q₂ q₁ q₀
+ _______________________________
+ v₃ v₂ v₁ v₀ ) u₉ u₈ u₇ u₆ u₅ u₄ u₃ u₂ u₁ u₀
+ ↓ ↓ ↓ ↓ ↓ | | | | |
+ [u₉ u₈ u₇ u₆ u₅]| | | | |
+ - [ q₆q₅·v ]| | | | |
+ ----------------- ↓ ↓ | | |
+ [ rem |u₄ u₃]| | |
+ - [ q₄q₃·v ]| | |
+ -------------------- ↓ ↓ |
+ [ rem |u₂ u₁]|
+ - [ q₂q₁·v ]|
+ -------------------- ↓
+ [ rem |u₀]
+ - [ q₀·v ]
+ ------------------
+ [ rem ]
+
+An alternative would be to look ahead to how well n/2 divides into n+m and
+adjust the first step to use fewer digits as needed, making the first step
+more special to make the last step not special at all. For example, using the
+same input, we could choose to use only 4 digits in the first step, leaving
+a full wide digit for the last step:
+
+ q₆ q₅ q₄ q₃ q₂ q₁ q₀
+ _______________________________
+ v₃ v₂ v₁ v₀ ) u₉ u₈ u₇ u₆ u₅ u₄ u₃ u₂ u₁ u₀
+ ↓ ↓ ↓ ↓ | | | | | |
+ [u₉ u₈ u₇ u₆]| | | | | |
+ - [ q₆·v ]| | | | | |
+ -------------- ↓ ↓ | | | |
+ [ rem |u₅ u₄]| | | |
+ - [ q₅q₄·v ]| | | |
+ -------------------- ↓ ↓ | |
+ [ rem |u₃ u₂]| |
+ - [ q₃q₂·v ]| |
+ -------------------- ↓ ↓
+ [ rem |u₁ u₀]
+ - [ q₁q₀·v ]
+ ---------------------
+ [ rem ]
+
+Today, the code in divRecursiveStep works like the first example. Perhaps in
+the future we will make it work like the alternative, to avoid a special case
+in the final iteration.
+
+Either way, each step is a 3-by-2-wide-digit division approximated first by
+a 2-by-1-wide-digit division, just as we did for regular digits in long division.
+Because the actual answer we want is a 3-by-2-wide-digit division, instead of
+multiplying q̂·v directly during the fixup, we can use the quick refinement
+from long division (an n/2-by-n/2 multiply) to correct q to its actual value
+and also compute the remainder (as mentioned above), and then stop after that,
+never doing a full n-by-n multiply.
+
+Instead of using an n-by-n/2-digit division to produce n/2 digits, we can add
+(not discard) one more real digit, doing an (n+1)-by-(n/2+1)-digit division that
+produces n/2+1 digits. That single extra digit tightens the Good Guess Guarantee
+to q ≤ q̂ ≤ q+1 and lets us drop long division's special treatment of the first
+digit. These benefits are discussed more after the Good Guess Guarantee proof
+below.
+
+
+How Fast is Recursive Division?
+
+For a 2n-by-n-digit division, this algorithm runs a 4-by-2 long division over
+wide digits, producing two wide digits plus a possible leading regular digit 1,
+which can be handled without a recursive call. That is, the algorithm uses two
+full iterations, each using an n-by-n/2-digit division and an n/2-by-n/2-digit
+multiplication, along with a few n-digit additions and subtractions. The standard
+n-by-n-digit multiplication algorithm requires O(n²) time, making the overall
+algorithm require time T(n) where
+
+ T(n) = 2T(n/2) + O(n) + O(n²)
+
+which, by the Bentley-Haken-Saxe theorem, ends up reducing to T(n) = O(n²).
+This is not an improvement over regular long division.
+
+When the number of digits n becomes large enough, Karatsuba's algorithm for
+multiplication can be used instead, which takes O(n^log₂3) = O(n^1.6) time.
+(Karatsuba multiplication is implemented in func karatsuba in nat.go.)
+That makes the overall recursive division algorithm take O(n^1.6) time as well,
+which is an improvement, but again only for large enough numbers.
+
+It is not critical to make sure that every recursion does only two recursive
+calls. While in general the number of recursive calls can change the time
+analysis, in this case doing three calls does not change the analysis:
+
+ T(n) = 3T(n/2) + O(n) + O(n^log₂3)
+
+ends up being T(n) = O(n^log₂3). Because the Karatsuba multiplication taking
+time O(n^log₂3) is itself doing 3 half-sized recursions, doing three for the
+division does not hurt the asymptotic performance. Of course, it is likely
+still faster in practice to do two.
+
+
+Proof of the Good Guess Guarantee
+
+Given numbers x, y, let us break them into the quotients and remainders when
+divided by some scaling factor S, with the added constraints that the quotient
+x/y and the high part of y are both less than some limit T, and that the high
+part of y is at least half as big as T.
+
+ x₁ = ⌊x/S⌋ y₁ = ⌊y/S⌋
+ x₀ = x mod S y₀ = y mod S
+
+ x = x₁·S + x₀ 0 ≤ x₀ < S x/y < T
+ y = y₁·S + y₀ 0 ≤ y₀ < S T/2 ≤ y₁ < T
+
+And consider the two truncated quotients:
+
+ q = ⌊x/y⌋
+ q̂ = ⌊x₁/y₁⌋
+
+We will prove that q ≤ q̂ ≤ q+2.
+
+The guarantee makes no real demands on the scaling factor S: it is simply the
+magnitude of the digits cut from both x and y to produce x₁ and y₁.
+The guarantee makes only limited demands on T: it must be large enough to hold
+the quotient x/y, and y₁ must have roughly the same size.
+
+To apply to the earlier discussion of 2-by-1 guesses in long division,
+we would choose:
+
+ S = Bⁿ⁻¹
+ T = B
+ x = u
+ x₁ = uₙuₙ₋₁
+ x₀ = uₙ₋₂...u₀
+ y = v
+ y₁ = vₙ₋₁
+ y₀ = vₙ₋₂...u₀
+
+These simpler variables avoid repeating those longer expressions in the proof.
+
+Note also that, by definition, truncating division ⌊x/y⌋ satisfies
+
+ x/y - 1 < ⌊x/y⌋ ≤ x/y.
+
+This fact will be used a few times in the proofs.
+
+Proof that q ≤ q̂:
+
+ q̂·y₁ = ⌊x₁/y₁⌋·y₁ [by definition, q̂ = ⌊x₁/y₁⌋]
+ > (x₁/y₁ - 1)·y₁ [x₁/y₁ - 1 < ⌊x₁/y₁⌋]
+ = x₁ - y₁ [distribute y₁]
+
+ So q̂·y₁ > x₁ - y₁.
+ Since q̂·y₁ is an integer, q̂·y₁ ≥ x₁ - y₁ + 1.
+
+ q̂ - q = q̂ - ⌊x/y⌋ [by definition, q = ⌊x/y⌋]
+ ≥ q̂ - x/y [⌊x/y⌋ < x/y]
+ = (1/y)·(q̂·y - x) [factor out 1/y]
+ ≥ (1/y)·(q̂·y₁·S - x) [y = y₁·S + y₀ ≥ y₁·S]
+ ≥ (1/y)·((x₁ - y₁ + 1)·S - x) [above: q̂·y₁ ≥ x₁ - y₁ + 1]
+ = (1/y)·(x₁·S - y₁·S + S - x) [distribute S]
+ = (1/y)·(S - x₀ - y₁·S) [-x = -x₁·S - x₀]
+ > -y₁·S / y [x₀ < S, so S - x₀ < 0; drop it]
+ ≥ -1 [y₁·S ≤ y]
+
+ So q̂ - q > -1.
+ Since q̂ - q is an integer, q̂ - q ≥ 0, or equivalently q ≤ q̂.
+
+Proof that q̂ ≤ q+2:
+
+ x₁/y₁ - x/y = x₁·S/y₁·S - x/y [multiply left term by S/S]
+ ≤ x/y₁·S - x/y [x₁S ≤ x]
+ = (x/y)·(y/y₁·S - 1) [factor out x/y]
+ = (x/y)·((y - y₁·S)/y₁·S) [move -1 into y/y₁·S fraction]
+ = (x/y)·(y₀/y₁·S) [y - y₁·S = y₀]
+ = (x/y)·(1/y₁)·(y₀/S) [factor out 1/y₁]
+ < (x/y)·(1/y₁) [y₀ < S, so y₀/S < 1]
+ ≤ (x/y)·(2/T) [y₁ ≥ T/2, so 1/y₁ ≤ 2/T]
+ < T·(2/T) [x/y < T]
+ = 2 [T·(2/T) = 2]
+
+ So x₁/y₁ - x/y < 2.
+
+ q̂ - q = ⌊x₁/y₁⌋ - q [by definition, q̂ = ⌊x₁/y₁⌋]
+ = ⌊x₁/y₁⌋ - ⌊x/y⌋ [by definition, q = ⌊x/y⌋]
+ ≤ x₁/y₁ - ⌊x/y⌋ [⌊x₁/y₁⌋ ≤ x₁/y₁]
+ < x₁/y₁ - (x/y - 1) [⌊x/y⌋ > x/y - 1]
+ = (x₁/y₁ - x/y) + 1 [regrouping]
+ < 2 + 1 [above: x₁/y₁ - x/y < 2]
+ = 3
+
+ So q̂ - q < 3.
+ Since q̂ - q is an integer, q̂ - q ≤ 2.
+
+Note that when x/y < T/2, the bounds tighten to x₁/y₁ - x/y < 1 and therefore
+q̂ - q ≤ 1.
+
+Note also that in the general case 2n-by-n division where we don't know that
+x/y < T, we do know that x/y < 2T, yielding the bound q̂ - q ≤ 4. So we could
+remove the special case first step of long division as long as we allow the
+first fixup loop to run up to four times. (Using a simple comparison to decide
+whether the first digit is 0 or 1 is still more efficient, though.)
+
+Finally, note that when dividing three leading base-B digits by two (scaled),
+we have T = B² and x/y < B = T/B, a much tighter bound than x/y < T.
+This in turn yields the much tighter bound x₁/y₁ - x/y < 2/B. This means that
+⌊x₁/y₁⌋ and ⌊x/y⌋ can only differ when x/y is less than 2/B greater than an
+integer. For random x and y, the chance of this is 2/B, or, for large B,
+approximately zero. This means that after we produce the 3-by-2 guess in the
+long division algorithm, the fixup loop essentially never runs.
+
+In the recursive algorithm, the extra digit in (2·⌊n/2⌋+1)-by-(⌊n/2⌋+1)-digit
+division has exactly the same effect: the probability of needing a fixup is the
+same 2/B. Even better, we can allow the general case x/y < 2T and the fixup
+probability only grows to 4/B, still essentially zero.
+
+
+References
+
+There are no great references for implementing long division; thus this comment.
+Here are some notes about what to expect from the obvious references.
+
+Knuth Volume 2 (Seminumerical Algorithms) section 4.3.1 is the usual canonical
+reference for long division, but that entire series is highly compressed, never
+repeating a necessary fact and leaving important insights to the exercises.
+For example, no rationale whatsoever is given for the calculation that extends
+q̂ from a 2-by-1 to a 3-by-2 guess, nor why it reduces the error bound.
+The proof that the calculation even has the desired effect is left to exercises.
+The solutions to those exercises provided at the back of the book are entirely
+calculations, still with no explanation as to what is going on or how you would
+arrive at the idea of doing those exact calculations. Nowhere is it mentioned
+that this test extends the 2-by-1 guess into a 3-by-2 guess. The proof of the
+Good Guess Guarantee is only for the 2-by-1 guess and argues by contradiction,
+making it difficult to understand how modifications like adding another digit
+or adjusting the quotient range affects the overall bound.
+
+All that said, Knuth remains the canonical reference. It is dense but packed
+full of information and references, and the proofs are simpler than many other
+presentations. The proofs above are reworkings of Knuth's to remove the
+arguments by contradiction and add explanations or steps that Knuth omitted.
+But beware of errors in older printings. Take the published errata with you.
+
+Brinch Hansen's “Multiple-length Division Revisited: a Tour of the Minefield”
+starts with a blunt critique of Knuth's presentation (among others) and then
+presents a more detailed and easier to follow treatment of long division,
+including an implementation in Pascal. But the algorithm and implementation
+work entirely in terms of 3-by-2 division, which is much less useful on modern
+hardware than an algorithm using 2-by-1 division. The proofs are a bit too
+focused on digit counting and seem needlessly complex, especially compared to
+the ones given above.
+
+Burnikel and Ziegler's “Fast Recursive Division” introduced the key insight of
+implementing division by an n-digit divisor using recursive calls to division
+by an n/2-digit divisor, relying on Karatsuba multiplication to yield a
+sub-quadratic run time. However, the presentation decisions are made almost
+entirely for the purpose of simplifying the run-time analysis, rather than
+simplifying the presentation. Instead of a single algorithm that loops over
+quotient digits, the paper presents two mutually-recursive algorithms, for
+2n-by-n and 3n-by-2n. The paper also does not present any general (n+m)-by-n
+algorithm.
+
+The proofs in the paper are remarkably complex, especially considering that
+the algorithm is at its core just long division on wide digits, so that the
+usual long division proofs apply essentially unaltered.
+*/
+
package big
import "math/bits"
+// div returns q, r such that q = ⌊u/v⌋ and r = u%v = u - q·v.
+// It uses z and z2 as the storage for q and r.
func (z nat) div(z2, u, v nat) (q, r nat) {
if len(v) == 0 {
panic("division by zero")
@@ -18,6 +514,8 @@ func (z nat) div(z2, u, v nat) (q, r nat) {
}
if len(v) == 1 {
+ // Short division: long optimized for a single-word divisor.
+ // In that case, the 2-by-1 guess is all we need at each step.
var r2 Word
q, r2 = z.divW(u, v[0])
r = z2.setWord(r2)
@@ -28,7 +526,9 @@ func (z nat) div(z2, u, v nat) (q, r nat) {
return
}
-// q = (x-r)/y, with 0 <= r < y
+// divW returns q, r such that q = ⌊x/y⌋ and r = x%y = x - q·y.
+// It uses z as the storage for q.
+// Note that y is a single digit (Word), not a big number.
func (z nat) divW(x nat, y Word) (q nat, r Word) {
m := len(x)
switch {
@@ -56,6 +556,8 @@ func (x nat) modW(d Word) (r Word) {
return divWVW(q, 0, x, d)
}
+// divWVW overwrites z with ⌊x/y⌋, returning the remainder r.
+// The caller must ensure that len(z) = len(x).
func divWVW(z []Word, xn Word, x []Word, y Word) (r Word) {
r = xn
if len(x) == 1 {
@@ -70,34 +572,33 @@ func divWVW(z []Word, xn Word, x []Word, y Word) (r Word) {
return r
}
-// q = (uIn-r)/vIn, with 0 <= r < vIn
-// Uses z as storage for q, and u as storage for r if possible.
-// See Knuth, Volume 2, section 4.3.1, Algorithm D.
-// Preconditions:
-// len(vIn) >= 2
-// len(uIn) >= len(vIn)
-// u must not alias z
+// div returns q, r such that q = ⌊uIn/vIn⌋ and r = uIn%vIn = uIn - q·vIn.
+// It uses z and u as the storage for q and r.
+// The caller must ensure that len(vIn) ≥ 2 (use divW otherwise)
+// and that len(uIn) ≥ len(vIn) (the answer is 0, uIn otherwise).
func (z nat) divLarge(u, uIn, vIn nat) (q, r nat) {
n := len(vIn)
m := len(uIn) - n
- // D1.
+ // Scale the inputs so vIn's top bit is 1 (see “Scaling Inputs” above).
+ // vIn is treated as a read-only input (it may be in use by another
+ // goroutine), so we must make a copy.
+ // uIn is copied to u.
shift := nlz(vIn[n-1])
- // do not modify vIn, it may be used by another goroutine simultaneously
vp := getNat(n)
v := *vp
shlVU(v, vIn, shift)
-
- // u may safely alias uIn or vIn, the value of uIn is used to set u and vIn was already used
u = u.make(len(uIn) + 1)
u[len(uIn)] = shlVU(u[0:len(uIn)], uIn, shift)
- // z may safely alias uIn or vIn, both values were used already
+ // The caller should not pass aliased z and u, since those are
+ // the two different outputs, but correct just in case.
if alias(z, u) {
- z = nil // z is an alias for u - cannot reuse
+ z = nil
}
q = z.make(m + 1)
+ // Use basic or recursive long division depending on size.
if n < divRecursiveThreshold {
q.divBasic(u, v)
} else {
@@ -106,19 +607,17 @@ func (z nat) divLarge(u, uIn, vIn nat) (q, r nat) {
putNat(vp)
q = q.norm()
+
+ // Undo scaling of remainder.
shrVU(u, u, shift)
r = u.norm()
return q, r
}
-// divBasic performs word-by-word division of u by v.
-// The quotient is written in pre-allocated q.
-// The remainder overwrites input u.
-//
-// Precondition:
-// - q is large enough to hold the quotient u / v
-// which has a maximum length of len(u)-len(v)+1.
+// divBasic implements long division as described above.
+// It overwrites q with ⌊u/v⌋ and overwrites u with the remainder r.
+// q must be large enough to hold ⌊u/v⌋.
func (q nat) divBasic(u, v nat) {
n := len(v)
m := len(u) - n
@@ -126,45 +625,56 @@ func (q nat) divBasic(u, v nat) {
qhatvp := getNat(n + 1)
qhatv := *qhatvp
- // D2.
+ // Set up for divWW below, precomputing reciprocal argument.
vn1 := v[n-1]
rec := reciprocalWord(vn1)
+
+ // Compute each digit of quotient.
for j := m; j >= 0; j-- {
- // D3.
+ // Compute the 2-by-1 guess q̂.
+ // The first iteration must invent a leading 0 for u.
qhat := Word(_M)
var ujn Word
if j+n < len(u) {
ujn = u[j+n]
}
+
+ // ujn ≤ vn1, or else q̂ would be more than one digit.
+ // For ujn == vn1, we set q̂ to the max digit M above.
+ // Otherwise, we compute the 2-by-1 guess.
if ujn != vn1 {
var rhat Word
qhat, rhat = divWW(ujn, u[j+n-1], vn1, rec)
- // x1 | x2 = q̂v_{n-2}
+ // Refine q̂ to a 3-by-2 guess. See “Refining Guesses” above.
vn2 := v[n-2]
x1, x2 := mulWW(qhat, vn2)
- // test if q̂v_{n-2} > br̂ + u_{j+n-2}
ujn2 := u[j+n-2]
- for greaterThan(x1, x2, rhat, ujn2) {
+ for greaterThan(x1, x2, rhat, ujn2) { // x1x2 > r̂ u[j+n-2]
qhat--
prevRhat := rhat
rhat += vn1
- // v[n-1] >= 0, so this tests for overflow.
+ // If r̂ overflows, then
+ // r̂ u[j+n-2]v[n-1] is now definitely > x1 x2.
if rhat < prevRhat {
break
}
+ // TODO(rsc): No need for a full mulWW.
+ // x2 += vn2; if x2 overflows, x1++
x1, x2 = mulWW(qhat, vn2)
}
}
- // D4.
- // Compute the remainder u - (q̂*v) << (_W*j).
- // The subtraction may overflow if q̂ estimate was off by one.
+ // Compute q̂·v.
qhatv[n] = mulAddVWW(qhatv[0:n], v, qhat, 0)
qhl := len(qhatv)
if j+qhl > len(u) && qhatv[n] == 0 {
qhl--
}
+
+ // Subtract q̂·v from the current section of u.
+ // If it underflows, q̂·v > u, which we fix up
+ // by decrementing q̂ and adding v back.
c := subVV(u[j:j+qhl], u[j:], qhatv)
if c != 0 {
c := addVV(u[j:j+n], u[j:], v)
@@ -176,6 +686,8 @@ func (q nat) divBasic(u, v nat) {
qhat--
}
+ // Save quotient digit.
+ // Caller may know the top digit is zero and not leave room for it.
if j == m && m == len(q) && qhat == 0 {
continue
}
@@ -185,30 +697,34 @@ func (q nat) divBasic(u, v nat) {
putNat(qhatvp)
}
-// greaterThan reports whether (x1<<_W + x2) > (y1<<_W + y2)
+// greaterThan reports whether the two digit numbers x1 x2 > y1 y2.
+// TODO(rsc): In contradiction to most of this file, x1 is the high
+// digit and x2 is the low digit. This should be fixed.
func greaterThan(x1, x2, y1, y2 Word) bool {
return x1 > y1 || x1 == y1 && x2 > y2
}
+// divRecursiveThreshold is the number of divisor digits
+// at which point divRecursive is faster than divBasic.
const divRecursiveThreshold = 100
-// divRecursive performs word-by-word division of u by v.
-// The quotient is written in pre-allocated z.
-// The remainder overwrites input u.
-//
-// Precondition:
-// - len(z) >= len(u)-len(v)
-//
-// See Burnikel, Ziegler, "Fast Recursive Division", Algorithm 1 and 2.
+// divRecursive implements recursive division as described above.
+// It overwrites z with ⌊u/v⌋ and overwrites u with the remainder r.
+// z must be large enough to hold ⌊u/v⌋.
+// This function is just for allocating and freeing temporaries
+// around divRecursiveStep, the real implementation.
func (z nat) divRecursive(u, v nat) {
- // Recursion depth is less than 2 log2(len(v))
- // Allocate a slice of temporaries to be reused across recursion.
+ // Recursion depth is (much) less than 2 log₂(len(v)).
+ // Allocate a slice of temporaries to be reused across recursion,
+ // plus one extra temporary not live across the recursion.
recDepth := 2 * bits.Len(uint(len(v)))
- // large enough to perform Karatsuba on operands as large as v
tmp := getNat(3 * len(v))
temps := make([]*nat, recDepth)
+
z.clear()
z.divRecursiveStep(u, v, 0, tmp, temps)
+
+ // Free temporaries.
for _, n := range temps {
if n != nil {
putNat(n)
@@ -217,72 +733,92 @@ func (z nat) divRecursive(u, v nat) {
putNat(tmp)
}
-// divRecursiveStep computes the division of u by v.
-// - z must be large enough to hold the quotient
-// - the quotient will overwrite z
-// - the remainder will overwrite u
+// divRecursiveStep is the actual implementation of recursive division.
+// It adds ⌊u/v⌋ to z and overwrites u with the remainder r.
+// z must be large enough to hold ⌊u/v⌋.
+// It uses temps[depth] (allocating if needed) as a temporary live across
+// the recursive call. It also uses tmp, but not live across the recursion.
func (z nat) divRecursiveStep(u, v nat, depth int, tmp *nat, temps []*nat) {
+ // u is a subsection of the original and may have leading zeros.
+ // TODO(rsc): The v = v.norm() is useless and should be removed.
+ // We know (and require) that v's top digit is ≥ B/2.
u = u.norm()
v = v.norm()
-
if len(u) == 0 {
z.clear()
return
}
+
+ // Fall back to basic division if the problem is now small enough.
n := len(v)
if n < divRecursiveThreshold {
z.divBasic(u, v)
return
}
+
+ // Nothing to do if u is shorter than v (implies u < v).
m := len(u) - n
if m < 0 {
return
}
- // Produce the quotient by blocks of B words.
- // Division by v (length n) is done using a length n/2 division
- // and a length n/2 multiplication for each block. The final
- // complexity is driven by multiplication complexity.
+ // We consider B digits in a row as a single wide digit.
+ // (See “Recursive Division” above.)
+ //
+ // TODO(rsc): rename B to Wide, to avoid confusion with _B,
+ // which is something entirely different.
+ // TODO(rsc): Look into whether using ⌈n/2⌉ is better than ⌊n/2⌋.
B := n / 2
// Allocate a nat for qhat below.
if temps[depth] == nil {
- temps[depth] = getNat(n)
+ temps[depth] = getNat(n) // TODO(rsc): Can be just B+1.
} else {
*temps[depth] = temps[depth].make(B + 1)
}
+ // Compute each wide digit of the quotient.
+ //
+ // TODO(rsc): Change the loop to be
+ // for j := (m+B-1)/B*B; j > 0; j -= B {
+ // which will make the final step a regular step, letting us
+ // delete what amounts to an extra copy of the loop body below.
j := m
for j > B {
- // Divide u[j-B:j+n] by vIn. Keep remainder in u
- // for next block.
+ // Divide u[j-B:j+n] (3 wide digits) by v (2 wide digits).
+ // First make the 2-by-1-wide-digit guess using a recursive call.
+ // Then extend the guess to the full 3-by-2 (see “Refining Guesses”).
//
- // The following property will be used (Lemma 2):
- // if u = u1 << s + u0
- // v = v1 << s + v0
- // then floor(u1/v1) >= floor(u/v)
- //
- // Moreover, the difference is at most 2 if len(v1) >= len(u/v)
- // We choose s = B-1 since len(v)-s >= B+1 >= len(u/v)
+ // For the 2-by-1-wide-digit guess, instead of doing 2B-by-B-digit,
+ // we use a (2B+1)-by-(B+1) digit, which handles the possibility that
+ // the result has an extra leading 1 digit as well as guaranteeing
+ // that the computed q̂ will be off by at most 1 instead of 2.
+
+ // s is the number of digits to drop from the 3B- and 2B-digit chunks.
+ // We drop B-1 to be left with 2B+1 and B+1.
s := (B - 1)
- // Except for the first step, the top bits are always
- // a division remainder, so the quotient length is <= n.
+
+ // uu is the up-to-3B-digit section of u we are working on.
uu := u[j-B:]
+ // Compute the 2-by-1 guess q̂, leaving r̂ in uu[s:B+n].
qhat := *temps[depth]
qhat.clear()
qhat.divRecursiveStep(uu[s:B+n], v[s:], depth+1, tmp, temps)
qhat = qhat.norm()
- // Adjust the quotient:
- // u = u_h << s + u_l
- // v = v_h << s + v_l
- // u_h = q̂ v_h + rh
- // u = q̂ (v - v_l) + rh << s + u_l
- // After the above step, u contains a remainder:
- // u = rh << s + u_l
- // and we need to subtract q̂ v_l
- //
- // But it may be a bit too large, in which case q̂ needs to be smaller.
+
+ // Extend to a 3-by-2 quotient and remainder.
+ // Because divRecursiveStep overwrote the top part of uu with
+ // the remainder r̂, the full uu already contains the equivalent
+ // of r̂·B + uₙ₋₂ from the “Refining Guesses” discussion.
+ // Subtracting q̂·vₙ₋₂ from it will compute the full-length remainder.
+ // If that subtraction underflows, q̂·v > u, which we fix up
+ // by decrementing q̂ and adding v back, same as in long division.
+
+ // TODO(rsc): Instead of subtract and fix-up, this code is computing
+ // q̂·vₙ₋₂ and decrementing q̂ until that product is ≤ u.
+ // But we can do the subtraction directly, as in the comment above
+ // and in long division, because we know that q̂ is wrong by at most one.
qhatv := tmp.make(3 * n)
qhatv.clear()
qhatv = qhatv.mul(qhat, v[:s])
@@ -309,6 +845,8 @@ func (z nat) divRecursiveStep(u, v nat, depth int, tmp *nat, temps []*nat) {
j -= B
}
+ // TODO(rsc): Rewrite loop as described above and delete all this code.
+
// Now u < (v<> 1) // clear sign bit if int == int32
}
-// Int63n returns, as an int64, a non-negative pseudo-random number in [0,n).
+// Int63n returns, as an int64, a non-negative pseudo-random number in the half-open interval [0,n).
// It panics if n <= 0.
func (r *Rand) Int63n(n int64) int64 {
if n <= 0 {
@@ -123,7 +120,7 @@ func (r *Rand) Int63n(n int64) int64 {
return v % n
}
-// Int31n returns, as an int32, a non-negative pseudo-random number in [0,n).
+// Int31n returns, as an int32, a non-negative pseudo-random number in the half-open interval [0,n).
// It panics if n <= 0.
func (r *Rand) Int31n(n int32) int32 {
if n <= 0 {
@@ -140,7 +137,7 @@ func (r *Rand) Int31n(n int32) int32 {
return v % n
}
-// int31n returns, as an int32, a non-negative pseudo-random number in [0,n).
+// int31n returns, as an int32, a non-negative pseudo-random number in the half-open interval [0,n).
// n must be > 0, but int31n does not check this; the caller must ensure it.
// int31n exists because Int31n is inefficient, but Go 1 compatibility
// requires that the stream of values produced by math/rand remain unchanged.
@@ -164,7 +161,7 @@ func (r *Rand) int31n(n int32) int32 {
return int32(prod >> 32)
}
-// Intn returns, as an int, a non-negative pseudo-random number in [0,n).
+// Intn returns, as an int, a non-negative pseudo-random number in the half-open interval [0,n).
// It panics if n <= 0.
func (r *Rand) Intn(n int) int {
if n <= 0 {
@@ -176,7 +173,7 @@ func (r *Rand) Intn(n int) int {
return int(r.Int63n(int64(n)))
}
-// Float64 returns, as a float64, a pseudo-random number in [0.0,1.0).
+// Float64 returns, as a float64, a pseudo-random number in the half-open interval [0.0,1.0).
func (r *Rand) Float64() float64 {
// A clearer, simpler implementation would be:
// return float64(r.Int63n(1<<53)) / (1<<53)
@@ -202,7 +199,7 @@ again:
return f
}
-// Float32 returns, as a float32, a pseudo-random number in [0.0,1.0).
+// Float32 returns, as a float32, a pseudo-random number in the half-open interval [0.0,1.0).
func (r *Rand) Float32() float32 {
// Same rationale as in Float64: we want to preserve the Go 1 value
// stream except we want to fix it not to return 1.0
@@ -215,7 +212,8 @@ again:
return f
}
-// Perm returns, as a slice of n ints, a pseudo-random permutation of the integers [0,n).
+// Perm returns, as a slice of n ints, a pseudo-random permutation of the integers
+// in the half-open interval [0,n).
func (r *Rand) Perm(n int) []int {
m := make([]int, n)
// In the following loop, the iteration when i=0 always swaps m[0] with m[0].
@@ -323,31 +321,31 @@ func Int31() int32 { return globalRand.Int31() }
// Int returns a non-negative pseudo-random int from the default Source.
func Int() int { return globalRand.Int() }
-// Int63n returns, as an int64, a non-negative pseudo-random number in [0,n)
+// Int63n returns, as an int64, a non-negative pseudo-random number in the half-open interval [0,n)
// from the default Source.
// It panics if n <= 0.
func Int63n(n int64) int64 { return globalRand.Int63n(n) }
-// Int31n returns, as an int32, a non-negative pseudo-random number in [0,n)
+// Int31n returns, as an int32, a non-negative pseudo-random number in the half-open interval [0,n)
// from the default Source.
// It panics if n <= 0.
func Int31n(n int32) int32 { return globalRand.Int31n(n) }
-// Intn returns, as an int, a non-negative pseudo-random number in [0,n)
+// Intn returns, as an int, a non-negative pseudo-random number in the half-open interval [0,n)
// from the default Source.
// It panics if n <= 0.
func Intn(n int) int { return globalRand.Intn(n) }
-// Float64 returns, as a float64, a pseudo-random number in [0.0,1.0)
+// Float64 returns, as a float64, a pseudo-random number in the half-open interval [0.0,1.0)
// from the default Source.
func Float64() float64 { return globalRand.Float64() }
-// Float32 returns, as a float32, a pseudo-random number in [0.0,1.0)
+// Float32 returns, as a float32, a pseudo-random number in the half-open interval [0.0,1.0)
// from the default Source.
func Float32() float32 { return globalRand.Float32() }
-// Perm returns, as a slice of n ints, a pseudo-random permutation of the integers [0,n)
-// from the default Source.
+// Perm returns, as a slice of n ints, a pseudo-random permutation of the integers
+// in the half-open interval [0,n) from the default Source.
func Perm(n int) []int { return globalRand.Perm(n) }
// Shuffle pseudo-randomizes the order of elements using the default Source.
diff --git a/src/net/dial_test.go b/src/net/dial_test.go
index f899da10cf..723038c7a2 100644
--- a/src/net/dial_test.go
+++ b/src/net/dial_test.go
@@ -155,40 +155,27 @@ func slowDialTCP(ctx context.Context, network string, laddr, raddr *TCPAddr) (*T
return c, err
}
-func dialClosedPort(t *testing.T) (actual, expected time.Duration) {
- // Estimate the expected time for this platform.
- // On Windows, dialing a closed port takes roughly 1 second,
- // but other platforms should be instantaneous.
- if runtime.GOOS == "windows" {
- expected = 1500 * time.Millisecond
- } else if runtime.GOOS == "darwin" || runtime.GOOS == "ios" {
- expected = 150 * time.Millisecond
- } else {
- expected = 95 * time.Millisecond
- }
+func dialClosedPort(t *testing.T) (dialLatency time.Duration) {
+ // On most platforms, dialing a closed port should be nearly instantaneous —
+ // less than a few hundred milliseconds. However, on some platforms it may be
+ // much slower: on Windows and OpenBSD, it has been observed to take up to a
+ // few seconds.
l, err := Listen("tcp", "127.0.0.1:0")
if err != nil {
- t.Logf("dialClosedPort: Listen failed: %v", err)
- return 999 * time.Hour, expected
+ t.Fatalf("dialClosedPort: Listen failed: %v", err)
}
addr := l.Addr().String()
l.Close()
- // On OpenBSD, interference from TestTCPSelfConnect is mysteriously
- // causing the first attempt to hang for a few seconds, so we throw
- // away the first result and keep the second.
- for i := 1; ; i++ {
- startTime := time.Now()
- c, err := Dial("tcp", addr)
- if err == nil {
- c.Close()
- }
- elapsed := time.Now().Sub(startTime)
- if i == 2 {
- t.Logf("dialClosedPort: measured delay %v", elapsed)
- return elapsed, expected
- }
+
+ startTime := time.Now()
+ c, err := Dial("tcp", addr)
+ if err == nil {
+ c.Close()
}
+ elapsed := time.Now().Sub(startTime)
+ t.Logf("dialClosedPort: measured delay %v", elapsed)
+ return elapsed
}
func TestDialParallel(t *testing.T) {
@@ -198,10 +185,7 @@ func TestDialParallel(t *testing.T) {
t.Skip("both IPv4 and IPv6 are required")
}
- closedPortDelay, expectClosedPortDelay := dialClosedPort(t)
- if closedPortDelay > expectClosedPortDelay {
- t.Errorf("got %v; want <= %v", closedPortDelay, expectClosedPortDelay)
- }
+ closedPortDelay := dialClosedPort(t)
const instant time.Duration = 0
const fallbackDelay = 200 * time.Millisecond
@@ -675,10 +659,7 @@ func TestDialerDualStack(t *testing.T) {
t.Skip("both IPv4 and IPv6 are required")
}
- closedPortDelay, expectClosedPortDelay := dialClosedPort(t)
- if closedPortDelay > expectClosedPortDelay {
- t.Errorf("got %v; want <= %v", closedPortDelay, expectClosedPortDelay)
- }
+ closedPortDelay := dialClosedPort(t)
origTestHookLookupIP := testHookLookupIP
defer func() { testHookLookupIP = origTestHookLookupIP }()
diff --git a/src/net/hosts_test.go b/src/net/hosts_test.go
index f850e2fccf..19c43999f9 100644
--- a/src/net/hosts_test.go
+++ b/src/net/hosts_test.go
@@ -36,7 +36,7 @@ var lookupStaticHostTests = []struct {
},
},
{
- "testdata/ipv4-hosts", // see golang.org/issue/8996
+ "testdata/ipv4-hosts",
[]staticHostEntry{
{"localhost", []string{"127.0.0.1", "127.0.0.2", "127.0.0.3"}},
{"localhost.localdomain", []string{"127.0.0.3"}},
@@ -102,7 +102,7 @@ var lookupStaticAddrTests = []struct {
},
},
{
- "testdata/ipv4-hosts", // see golang.org/issue/8996
+ "testdata/ipv4-hosts",
[]staticHostEntry{
{"127.0.0.1", []string{"localhost"}},
{"127.0.0.2", []string{"localhost"}},
diff --git a/src/net/http/httptest/server.go b/src/net/http/httptest/server.go
index a02a6d64c3..4f85ff55d8 100644
--- a/src/net/http/httptest/server.go
+++ b/src/net/http/httptest/server.go
@@ -14,7 +14,7 @@ import (
"log"
"net"
"net/http"
- "net/http/internal"
+ "net/http/internal/testcert"
"os"
"strings"
"sync"
@@ -144,7 +144,7 @@ func (s *Server) StartTLS() {
if s.client == nil {
s.client = &http.Client{Transport: &http.Transport{}}
}
- cert, err := tls.X509KeyPair(internal.LocalhostCert, internal.LocalhostKey)
+ cert, err := tls.X509KeyPair(testcert.LocalhostCert, testcert.LocalhostKey)
if err != nil {
panic(fmt.Sprintf("httptest: NewTLSServer: %v", err))
}
diff --git a/src/net/http/internal/testcert.go b/src/net/http/internal/testcert/testcert.go
similarity index 94%
rename from src/net/http/internal/testcert.go
rename to src/net/http/internal/testcert/testcert.go
index 2284a836fb..5f94704ef5 100644
--- a/src/net/http/internal/testcert.go
+++ b/src/net/http/internal/testcert/testcert.go
@@ -2,7 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
-package internal
+// Package testcert contains a test-only localhost certificate.
+package testcert
import "strings"
@@ -25,7 +26,7 @@ h1fIw3cSS2OolhloGw/XM6RWPWtPAlGykKLciQrBru5NAPvCMsb/I1DAceTiotQM
fblo6RBxUQ==
-----END CERTIFICATE-----`)
-// LocalhostKey is the private key for localhostCert.
+// LocalhostKey is the private key for LocalhostCert.
var LocalhostKey = []byte(testingKey(`-----BEGIN RSA TESTING KEY-----
MIICXgIBAAKBgQDuLnQAI3mDgey3VBzWnB2L39JUU4txjeVE6myuDqkM/uGlfjb9
SjY1bIw4iA5sBBZzHi3z0h1YV8QPuxEbi4nW91IJm2gsvvZhIrCHS3l6afab4pZB
diff --git a/src/net/http/request.go b/src/net/http/request.go
index 7895417af5..09cb0c7f56 100644
--- a/src/net/http/request.go
+++ b/src/net/http/request.go
@@ -1293,16 +1293,18 @@ func (r *Request) ParseForm() error {
// its file parts are stored in memory, with the remainder stored on
// disk in temporary files.
// ParseMultipartForm calls ParseForm if necessary.
+// If ParseForm returns an error, ParseMultipartForm returns it but also
+// continues parsing the request body.
// After one call to ParseMultipartForm, subsequent calls have no effect.
func (r *Request) ParseMultipartForm(maxMemory int64) error {
if r.MultipartForm == multipartByReader {
return errors.New("http: multipart handled by MultipartReader")
}
+ var parseFormErr error
if r.Form == nil {
- err := r.ParseForm()
- if err != nil {
- return err
- }
+ // Let errors in ParseForm fall through, and just
+ // return it at the end.
+ parseFormErr = r.ParseForm()
}
if r.MultipartForm != nil {
return nil
@@ -1329,7 +1331,7 @@ func (r *Request) ParseMultipartForm(maxMemory int64) error {
r.MultipartForm = f
- return nil
+ return parseFormErr
}
// FormValue returns the first value for the named component of the query.
diff --git a/src/net/http/request_test.go b/src/net/http/request_test.go
index 952828b395..4e0c4ba207 100644
--- a/src/net/http/request_test.go
+++ b/src/net/http/request_test.go
@@ -32,9 +32,26 @@ func TestQuery(t *testing.T) {
}
}
+// Issue #25192: Test that ParseForm fails but still parses the form when an URL
+// containing a semicolon is provided.
+func TestParseFormSemicolonSeparator(t *testing.T) {
+ for _, method := range []string{"POST", "PATCH", "PUT", "GET"} {
+ req, _ := NewRequest(method, "http://www.google.com/search?q=foo;q=bar&a=1",
+ strings.NewReader("q"))
+ err := req.ParseForm()
+ if err == nil {
+ t.Fatalf(`for method %s, ParseForm expected an error, got success`, method)
+ }
+ wantForm := url.Values{"a": []string{"1"}}
+ if !reflect.DeepEqual(req.Form, wantForm) {
+ t.Fatalf("for method %s, ParseForm expected req.Form = %v, want %v", method, req.Form, wantForm)
+ }
+ }
+}
+
func TestParseFormQuery(t *testing.T) {
req, _ := NewRequest("POST", "http://www.google.com/search?q=foo&q=bar&both=x&prio=1&orphan=nope&empty=not",
- strings.NewReader("z=post&both=y&prio=2&=nokey&orphan;empty=&"))
+ strings.NewReader("z=post&both=y&prio=2&=nokey&orphan&empty=&"))
req.Header.Set("Content-Type", "application/x-www-form-urlencoded; param=value")
if q := req.FormValue("q"); q != "foo" {
@@ -365,6 +382,18 @@ func TestMultipartRequest(t *testing.T) {
validateTestMultipartContents(t, req, false)
}
+// Issue #25192: Test that ParseMultipartForm fails but still parses the
+// multi-part form when an URL containing a semicolon is provided.
+func TestParseMultipartFormSemicolonSeparator(t *testing.T) {
+ req := newTestMultipartRequest(t)
+ req.URL = &url.URL{RawQuery: "q=foo;q=bar"}
+ if err := req.ParseMultipartForm(25); err == nil {
+ t.Fatal("ParseMultipartForm expected error due to invalid semicolon, got nil")
+ }
+ defer req.MultipartForm.RemoveAll()
+ validateTestMultipartContents(t, req, false)
+}
+
func TestMultipartRequestAuto(t *testing.T) {
// Test that FormValue and FormFile automatically invoke
// ParseMultipartForm and return the right values.
diff --git a/src/net/http/serve_test.go b/src/net/http/serve_test.go
index a9714682c7..6394da3bb7 100644
--- a/src/net/http/serve_test.go
+++ b/src/net/http/serve_test.go
@@ -25,6 +25,7 @@ import (
"net/http/httptest"
"net/http/httputil"
"net/http/internal"
+ "net/http/internal/testcert"
"net/url"
"os"
"os/exec"
@@ -1475,7 +1476,7 @@ func TestServeTLS(t *testing.T) {
defer afterTest(t)
defer SetTestHookServerServe(nil)
- cert, err := tls.X509KeyPair(internal.LocalhostCert, internal.LocalhostKey)
+ cert, err := tls.X509KeyPair(testcert.LocalhostCert, testcert.LocalhostKey)
if err != nil {
t.Fatal(err)
}
@@ -1599,7 +1600,7 @@ func TestAutomaticHTTP2_Serve_WithTLSConfig(t *testing.T) {
}
func TestAutomaticHTTP2_ListenAndServe(t *testing.T) {
- cert, err := tls.X509KeyPair(internal.LocalhostCert, internal.LocalhostKey)
+ cert, err := tls.X509KeyPair(testcert.LocalhostCert, testcert.LocalhostKey)
if err != nil {
t.Fatal(err)
}
@@ -1609,7 +1610,7 @@ func TestAutomaticHTTP2_ListenAndServe(t *testing.T) {
}
func TestAutomaticHTTP2_ListenAndServe_GetCertificate(t *testing.T) {
- cert, err := tls.X509KeyPair(internal.LocalhostCert, internal.LocalhostKey)
+ cert, err := tls.X509KeyPair(testcert.LocalhostCert, testcert.LocalhostKey)
if err != nil {
t.Fatal(err)
}
@@ -6524,3 +6525,87 @@ func TestMuxRedirectRelative(t *testing.T) {
t.Errorf("Expected response code %d; got %d", want, got)
}
}
+
+// TestQuerySemicolon tests the behavior of semicolons in queries. See Issue 25192.
+func TestQuerySemicolon(t *testing.T) {
+ t.Cleanup(func() { afterTest(t) })
+
+ tests := []struct {
+ query string
+ xNoSemicolons string
+ xWithSemicolons string
+ warning bool
+ }{
+ {"?a=1;x=bad&x=good", "good", "bad", true},
+ {"?a=1;b=bad&x=good", "good", "good", true},
+ {"?a=1%3Bx=bad&x=good%3B", "good;", "good;", false},
+ {"?a=1;x=good;x=bad", "", "good", true},
+ }
+
+ for _, tt := range tests {
+ t.Run(tt.query+"/allow=false", func(t *testing.T) {
+ allowSemicolons := false
+ testQuerySemicolon(t, tt.query, tt.xNoSemicolons, allowSemicolons, tt.warning)
+ })
+ t.Run(tt.query+"/allow=true", func(t *testing.T) {
+ allowSemicolons, expectWarning := true, false
+ testQuerySemicolon(t, tt.query, tt.xWithSemicolons, allowSemicolons, expectWarning)
+ })
+ }
+}
+
+func testQuerySemicolon(t *testing.T, query string, wantX string, allowSemicolons, expectWarning bool) {
+ setParallel(t)
+
+ writeBackX := func(w ResponseWriter, r *Request) {
+ x := r.URL.Query().Get("x")
+ if expectWarning {
+ if err := r.ParseForm(); err == nil || !strings.Contains(err.Error(), "semicolon") {
+ t.Errorf("expected error mentioning semicolons from ParseForm, got %v", err)
+ }
+ } else {
+ if err := r.ParseForm(); err != nil {
+ t.Errorf("expected no error from ParseForm, got %v", err)
+ }
+ }
+ if got := r.FormValue("x"); x != got {
+ t.Errorf("got %q from FormValue, want %q", got, x)
+ }
+ fmt.Fprintf(w, "%s", x)
+ }
+
+ h := Handler(HandlerFunc(writeBackX))
+ if allowSemicolons {
+ h = AllowQuerySemicolons(h)
+ }
+
+ ts := httptest.NewUnstartedServer(h)
+ logBuf := &bytes.Buffer{}
+ ts.Config.ErrorLog = log.New(logBuf, "", 0)
+ ts.Start()
+ defer ts.Close()
+
+ req, _ := NewRequest("GET", ts.URL+query, nil)
+ res, err := ts.Client().Do(req)
+ if err != nil {
+ t.Fatal(err)
+ }
+ slurp, _ := io.ReadAll(res.Body)
+ res.Body.Close()
+ if got, want := res.StatusCode, 200; got != want {
+ t.Errorf("Status = %d; want = %d", got, want)
+ }
+ if got, want := string(slurp), wantX; got != want {
+ t.Errorf("Body = %q; want = %q", got, want)
+ }
+
+ if expectWarning {
+ if !strings.Contains(logBuf.String(), "semicolon") {
+ t.Errorf("got %q from ErrorLog, expected a mention of semicolons", logBuf.String())
+ }
+ } else {
+ if strings.Contains(logBuf.String(), "semicolon") {
+ t.Errorf("got %q from ErrorLog, expected no mention of semicolons", logBuf.String())
+ }
+ }
+}
diff --git a/src/net/http/server.go b/src/net/http/server.go
index 4e73508973..5b113cff97 100644
--- a/src/net/http/server.go
+++ b/src/net/http/server.go
@@ -333,7 +333,7 @@ func (c *conn) hijackLocked() (rwc net.Conn, buf *bufio.ReadWriter, err error) {
const bufferBeforeChunkingSize = 2048
// chunkWriter writes to a response's conn buffer, and is the writer
-// wrapped by the response.bufw buffered writer.
+// wrapped by the response.w buffered writer.
//
// chunkWriter also is responsible for finalizing the Header, including
// conditionally setting the Content-Type and setting a Content-Length
@@ -577,37 +577,17 @@ func (w *response) ReadFrom(src io.Reader) (n int64, err error) {
return io.CopyBuffer(writerOnly{w}, src, buf)
}
- // sendfile path:
-
- // Do not start actually writing response until src is readable.
- // If body length is <= sniffLen, sendfile/splice path will do
- // little anyway. This small read also satisfies sniffing the
- // body in case Content-Type is missing.
- nr, er := src.Read(buf[:sniffLen])
- atEOF := errors.Is(er, io.EOF)
- n += int64(nr)
-
- if nr > 0 {
- // Write the small amount read normally.
- nw, ew := w.Write(buf[:nr])
- if ew != nil {
- err = ew
- } else if nr != nw {
- err = io.ErrShortWrite
+ // Copy the first sniffLen bytes before switching to ReadFrom.
+ // This ensures we don't start writing the response before the
+ // source is available (see golang.org/issue/5660) and provides
+ // enough bytes to perform Content-Type sniffing when required.
+ if !w.cw.wroteHeader {
+ n0, err := io.CopyBuffer(writerOnly{w}, io.LimitReader(src, sniffLen), buf)
+ n += n0
+ if err != nil || n0 < sniffLen {
+ return n, err
}
}
- if err == nil && er != nil && !atEOF {
- err = er
- }
-
- // Do not send StatusOK in the error case where nothing has been written.
- if err == nil && !w.wroteHeader {
- w.WriteHeader(StatusOK) // nr == 0, no error (or EOF)
- }
-
- if err != nil || atEOF {
- return n, err
- }
w.w.Flush() // get rid of any previous writes
w.cw.flush() // make sure Header is written; flush data to rwc
@@ -620,7 +600,7 @@ func (w *response) ReadFrom(src io.Reader) (n int64, err error) {
return n, err
}
- n0, err := io.Copy(writerOnly{w}, src)
+ n0, err := io.CopyBuffer(writerOnly{w}, src, buf)
n += n0
return n, err
}
@@ -1549,12 +1529,12 @@ func (w *response) bodyAllowed() bool {
// The Writers are wired together like:
//
// 1. *response (the ResponseWriter) ->
-// 2. (*response).w, a *bufio.Writer of bufferBeforeChunkingSize bytes
+// 2. (*response).w, a *bufio.Writer of bufferBeforeChunkingSize bytes ->
// 3. chunkWriter.Writer (whose writeHeader finalizes Content-Length/Type)
-// and which writes the chunk headers, if needed.
-// 4. conn.buf, a bufio.Writer of default (4kB) bytes, writing to ->
+// and which writes the chunk headers, if needed ->
+// 4. conn.bufw, a *bufio.Writer of default (4kB) bytes, writing to ->
// 5. checkConnErrorWriter{c}, which notes any non-nil error on Write
-// and populates c.werr with it if so. but otherwise writes to:
+// and populates c.werr with it if so, but otherwise writes to ->
// 6. the rwc, the net.Conn.
//
// TODO(bradfitz): short-circuit some of the buffering when the
@@ -2882,9 +2862,51 @@ func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) {
if req.RequestURI == "*" && req.Method == "OPTIONS" {
handler = globalOptionsHandler{}
}
+
+ if req.URL != nil && strings.Contains(req.URL.RawQuery, ";") {
+ var allowQuerySemicolonsInUse int32
+ req = req.WithContext(context.WithValue(req.Context(), silenceSemWarnContextKey, func() {
+ atomic.StoreInt32(&allowQuerySemicolonsInUse, 1)
+ }))
+ defer func() {
+ if atomic.LoadInt32(&allowQuerySemicolonsInUse) == 0 {
+ sh.srv.logf("http: URL query contains semicolon, which is no longer a supported separator; parts of the query may be stripped when parsed; see golang.org/issue/25192")
+ }
+ }()
+ }
+
handler.ServeHTTP(rw, req)
}
+var silenceSemWarnContextKey = &contextKey{"silence-semicolons"}
+
+// AllowQuerySemicolons returns a handler that serves requests by converting any
+// unescaped semicolons in the URL query to ampersands, and invoking the handler h.
+//
+// This restores the pre-Go 1.17 behavior of splitting query parameters on both
+// semicolons and ampersands. (See golang.org/issue/25192). Note that this
+// behavior doesn't match that of many proxies, and the mismatch can lead to
+// security issues.
+//
+// AllowQuerySemicolons should be invoked before Request.ParseForm is called.
+func AllowQuerySemicolons(h Handler) Handler {
+ return HandlerFunc(func(w ResponseWriter, r *Request) {
+ if silenceSemicolonsWarning, ok := r.Context().Value(silenceSemWarnContextKey).(func()); ok {
+ silenceSemicolonsWarning()
+ }
+ if strings.Contains(r.URL.RawQuery, ";") {
+ r2 := new(Request)
+ *r2 = *r
+ r2.URL = new(url.URL)
+ *r2.URL = *r.URL
+ r2.URL.RawQuery = strings.ReplaceAll(r.URL.RawQuery, ";", "&")
+ h.ServeHTTP(w, r2)
+ } else {
+ h.ServeHTTP(w, r)
+ }
+ })
+}
+
// ListenAndServe listens on the TCP network address srv.Addr and then
// calls Serve to handle requests on incoming connections.
// Accepted connections are configured to enable TCP keep-alives.
diff --git a/src/net/http/sniff_test.go b/src/net/http/sniff_test.go
index 8d5350374d..e91335729a 100644
--- a/src/net/http/sniff_test.go
+++ b/src/net/http/sniff_test.go
@@ -157,9 +157,25 @@ func testServerIssue5953(t *testing.T, h2 bool) {
resp.Body.Close()
}
-func TestContentTypeWithCopy_h1(t *testing.T) { testContentTypeWithCopy(t, h1Mode) }
-func TestContentTypeWithCopy_h2(t *testing.T) { testContentTypeWithCopy(t, h2Mode) }
-func testContentTypeWithCopy(t *testing.T, h2 bool) {
+type byteAtATimeReader struct {
+ buf []byte
+}
+
+func (b *byteAtATimeReader) Read(p []byte) (n int, err error) {
+ if len(p) < 1 {
+ return 0, nil
+ }
+ if len(b.buf) == 0 {
+ return 0, io.EOF
+ }
+ p[0] = b.buf[0]
+ b.buf = b.buf[1:]
+ return 1, nil
+}
+
+func TestContentTypeWithVariousSources_h1(t *testing.T) { testContentTypeWithVariousSources(t, h1Mode) }
+func TestContentTypeWithVariousSources_h2(t *testing.T) { testContentTypeWithVariousSources(t, h2Mode) }
+func testContentTypeWithVariousSources(t *testing.T, h2 bool) {
defer afterTest(t)
const (
@@ -167,30 +183,86 @@ func testContentTypeWithCopy(t *testing.T, h2 bool) {
expected = "text/html; charset=utf-8"
)
- cst := newClientServerTest(t, h2, HandlerFunc(func(w ResponseWriter, r *Request) {
- // Use io.Copy from a bytes.Buffer to trigger ReadFrom.
- buf := bytes.NewBuffer([]byte(input))
- n, err := io.Copy(w, buf)
- if int(n) != len(input) || err != nil {
- t.Errorf("io.Copy(w, %q) = %v, %v want %d, nil", input, n, err, len(input))
- }
- }))
- defer cst.close()
+ for _, test := range []struct {
+ name string
+ handler func(ResponseWriter, *Request)
+ }{{
+ name: "write",
+ handler: func(w ResponseWriter, r *Request) {
+ // Write the whole input at once.
+ n, err := w.Write([]byte(input))
+ if int(n) != len(input) || err != nil {
+ t.Errorf("w.Write(%q) = %v, %v want %d, nil", input, n, err, len(input))
+ }
+ },
+ }, {
+ name: "write one byte at a time",
+ handler: func(w ResponseWriter, r *Request) {
+ // Write the input one byte at a time.
+ buf := []byte(input)
+ for i := range buf {
+ n, err := w.Write(buf[i : i+1])
+ if n != 1 || err != nil {
+ t.Errorf("w.Write(%q) = %v, %v want 1, nil", input, n, err)
+ }
+ }
+ },
+ }, {
+ name: "copy from Reader",
+ handler: func(w ResponseWriter, r *Request) {
+ // Use io.Copy from a plain Reader.
+ type readerOnly struct{ io.Reader }
+ buf := bytes.NewBuffer([]byte(input))
+ n, err := io.Copy(w, readerOnly{buf})
+ if int(n) != len(input) || err != nil {
+ t.Errorf("io.Copy(w, %q) = %v, %v want %d, nil", input, n, err, len(input))
+ }
+ },
+ }, {
+ name: "copy from bytes.Buffer",
+ handler: func(w ResponseWriter, r *Request) {
+ // Use io.Copy from a bytes.Buffer to trigger ReadFrom.
+ buf := bytes.NewBuffer([]byte(input))
+ n, err := io.Copy(w, buf)
+ if int(n) != len(input) || err != nil {
+ t.Errorf("io.Copy(w, %q) = %v, %v want %d, nil", input, n, err, len(input))
+ }
+ },
+ }, {
+ name: "copy one byte at a time",
+ handler: func(w ResponseWriter, r *Request) {
+ // Use io.Copy from a Reader that returns one byte at a time.
+ n, err := io.Copy(w, &byteAtATimeReader{[]byte(input)})
+ if int(n) != len(input) || err != nil {
+ t.Errorf("io.Copy(w, %q) = %v, %v want %d, nil", input, n, err, len(input))
+ }
+ },
+ }} {
+ t.Run(test.name, func(t *testing.T) {
+ cst := newClientServerTest(t, h2, HandlerFunc(test.handler))
+ defer cst.close()
+
+ resp, err := cst.c.Get(cst.ts.URL)
+ if err != nil {
+ t.Fatalf("Get: %v", err)
+ }
+ if ct := resp.Header.Get("Content-Type"); ct != expected {
+ t.Errorf("Content-Type = %q, want %q", ct, expected)
+ }
+ if want, got := resp.Header.Get("Content-Length"), fmt.Sprint(len(input)); want != got {
+ t.Errorf("Content-Length = %q, want %q", want, got)
+ }
+ data, err := io.ReadAll(resp.Body)
+ if err != nil {
+ t.Errorf("reading body: %v", err)
+ } else if !bytes.Equal(data, []byte(input)) {
+ t.Errorf("data is %q, want %q", data, input)
+ }
+ resp.Body.Close()
+
+ })
- resp, err := cst.c.Get(cst.ts.URL)
- if err != nil {
- t.Fatalf("Get: %v", err)
}
- if ct := resp.Header.Get("Content-Type"); ct != expected {
- t.Errorf("Content-Type = %q, want %q", ct, expected)
- }
- data, err := io.ReadAll(resp.Body)
- if err != nil {
- t.Errorf("reading body: %v", err)
- } else if !bytes.Equal(data, []byte(input)) {
- t.Errorf("data is %q, want %q", data, input)
- }
- resp.Body.Close()
}
func TestSniffWriteSize_h1(t *testing.T) { testSniffWriteSize(t, h1Mode) }
diff --git a/src/net/http/transport_internal_test.go b/src/net/http/transport_internal_test.go
index 1097ffd173..1cce27235d 100644
--- a/src/net/http/transport_internal_test.go
+++ b/src/net/http/transport_internal_test.go
@@ -12,7 +12,7 @@ import (
"errors"
"io"
"net"
- "net/http/internal"
+ "net/http/internal/testcert"
"strings"
"testing"
)
@@ -191,7 +191,7 @@ func (f roundTripFunc) RoundTrip(r *Request) (*Response, error) {
// Issue 25009
func TestTransportBodyAltRewind(t *testing.T) {
- cert, err := tls.X509KeyPair(internal.LocalhostCert, internal.LocalhostKey)
+ cert, err := tls.X509KeyPair(testcert.LocalhostCert, testcert.LocalhostKey)
if err != nil {
t.Fatal(err)
}
diff --git a/src/net/http/transport_test.go b/src/net/http/transport_test.go
index dcaacece61..690e0c299d 100644
--- a/src/net/http/transport_test.go
+++ b/src/net/http/transport_test.go
@@ -30,7 +30,7 @@ import (
"net/http/httptest"
"net/http/httptrace"
"net/http/httputil"
- "net/http/internal"
+ "net/http/internal/testcert"
"net/textproto"
"net/url"
"os"
@@ -4299,7 +4299,7 @@ func TestTransportReuseConnEmptyResponseBody(t *testing.T) {
// Issue 13839
func TestNoCrashReturningTransportAltConn(t *testing.T) {
- cert, err := tls.X509KeyPair(internal.LocalhostCert, internal.LocalhostKey)
+ cert, err := tls.X509KeyPair(testcert.LocalhostCert, testcert.LocalhostKey)
if err != nil {
t.Fatal(err)
}
diff --git a/src/net/ip.go b/src/net/ip.go
index 0477269761..38e1aa2247 100644
--- a/src/net/ip.go
+++ b/src/net/ip.go
@@ -574,6 +574,10 @@ func parseIPv4(s string) IP {
if !ok || n > 0xFF {
return nil
}
+ if c > 1 && s[0] == '0' {
+ // Reject non-zero components with leading zeroes.
+ return nil
+ }
s = s[c:]
p[i] = byte(n)
}
diff --git a/src/net/ip_test.go b/src/net/ip_test.go
index 3af5e41ceb..5bbda6024d 100644
--- a/src/net/ip_test.go
+++ b/src/net/ip_test.go
@@ -21,9 +21,7 @@ var parseIPTests = []struct {
}{
{"127.0.1.2", IPv4(127, 0, 1, 2)},
{"127.0.0.1", IPv4(127, 0, 0, 1)},
- {"127.001.002.003", IPv4(127, 1, 2, 3)},
{"::ffff:127.1.2.3", IPv4(127, 1, 2, 3)},
- {"::ffff:127.001.002.003", IPv4(127, 1, 2, 3)},
{"::ffff:7f01:0203", IPv4(127, 1, 2, 3)},
{"0:0:0:0:0000:ffff:127.1.2.3", IPv4(127, 1, 2, 3)},
{"0:0:0:0:000000:ffff:127.1.2.3", IPv4(127, 1, 2, 3)},
@@ -43,6 +41,11 @@ var parseIPTests = []struct {
{"fe80::1%911", nil},
{"", nil},
{"a1:a2:a3:a4::b1:b2:b3:b4", nil}, // Issue 6628
+ {"127.001.002.003", nil},
+ {"::ffff:127.001.002.003", nil},
+ {"123.000.000.000", nil},
+ {"1.2..4", nil},
+ {"0123.0.0.1", nil},
}
func TestParseIP(t *testing.T) {
@@ -358,6 +361,7 @@ var parseCIDRTests = []struct {
{"0.0.-2.0/32", nil, nil, &ParseError{Type: "CIDR address", Text: "0.0.-2.0/32"}},
{"0.0.0.-3/32", nil, nil, &ParseError{Type: "CIDR address", Text: "0.0.0.-3/32"}},
{"0.0.0.0/-0", nil, nil, &ParseError{Type: "CIDR address", Text: "0.0.0.0/-0"}},
+ {"127.000.000.001/32", nil, nil, &ParseError{Type: "CIDR address", Text: "127.000.000.001/32"}},
{"", nil, nil, &ParseError{Type: "CIDR address", Text: ""}},
}
diff --git a/src/net/lookup_windows_test.go b/src/net/lookup_windows_test.go
index 62b61ed6c2..aa95501d02 100644
--- a/src/net/lookup_windows_test.go
+++ b/src/net/lookup_windows_test.go
@@ -299,7 +299,7 @@ func lookupPTR(name string) (ptr []string, err error) {
ptr = make([]string, 0, 10)
rx := regexp.MustCompile(`(?m)^Pinging\s+([a-zA-Z0-9.\-]+)\s+\[.*$`)
for _, ans := range rx.FindAllStringSubmatch(r, -1) {
- ptr = append(ptr, ans[1]+".")
+ ptr = append(ptr, absDomainName([]byte(ans[1])))
}
return
}
diff --git a/src/net/testdata/ipv4-hosts b/src/net/testdata/ipv4-hosts
index 5208bb44ac..6b99675dfc 100644
--- a/src/net/testdata/ipv4-hosts
+++ b/src/net/testdata/ipv4-hosts
@@ -1,12 +1,8 @@
# See https://tools.ietf.org/html/rfc1123.
-#
-# The literal IPv4 address parser in the net package is a relaxed
-# one. It may accept a literal IPv4 address in dotted-decimal notation
-# with leading zeros such as "001.2.003.4".
# internet address and host name
127.0.0.1 localhost # inline comment separated by tab
-127.000.000.002 localhost # inline comment separated by space
+127.0.0.2 localhost # inline comment separated by space
# internet address, host name and aliases
-127.000.000.003 localhost localhost.localdomain
+127.0.0.3 localhost localhost.localdomain
diff --git a/src/net/url/example_test.go b/src/net/url/example_test.go
index cb9e8922a2..476132a1c9 100644
--- a/src/net/url/example_test.go
+++ b/src/net/url/example_test.go
@@ -72,13 +72,13 @@ func ExampleURL_ResolveReference() {
}
func ExampleParseQuery() {
- m, err := url.ParseQuery(`x=1&y=2&y=3;z`)
+ m, err := url.ParseQuery(`x=1&y=2&y=3`)
if err != nil {
log.Fatal(err)
}
fmt.Println(toJSON(m))
// Output:
- // {"x":["1"], "y":["2", "3"], "z":[""]}
+ // {"x":["1"], "y":["2", "3"]}
}
func ExampleURL_EscapedPath() {
diff --git a/src/net/url/url.go b/src/net/url/url.go
index 73bef22e45..20de0f6f51 100644
--- a/src/net/url/url.go
+++ b/src/net/url/url.go
@@ -921,9 +921,10 @@ func (v Values) Has(key string) bool {
// valid query parameters found; err describes the first decoding error
// encountered, if any.
//
-// Query is expected to be a list of key=value settings separated by
-// ampersands or semicolons. A setting without an equals sign is
-// interpreted as a key set to an empty value.
+// Query is expected to be a list of key=value settings separated by ampersands.
+// A setting without an equals sign is interpreted as a key set to an empty
+// value.
+// Settings containing a non-URL-encoded semicolon are considered invalid.
func ParseQuery(query string) (Values, error) {
m := make(Values)
err := parseQuery(m, query)
@@ -933,11 +934,15 @@ func ParseQuery(query string) (Values, error) {
func parseQuery(m Values, query string) (err error) {
for query != "" {
key := query
- if i := strings.IndexAny(key, "&;"); i >= 0 {
+ if i := strings.IndexAny(key, "&"); i >= 0 {
key, query = key[:i], key[i+1:]
} else {
query = ""
}
+ if strings.Contains(key, ";") {
+ err = fmt.Errorf("invalid semicolon separator in query")
+ continue
+ }
if key == "" {
continue
}
diff --git a/src/net/url/url_test.go b/src/net/url/url_test.go
index 55348c4a7d..63c8e695af 100644
--- a/src/net/url/url_test.go
+++ b/src/net/url/url_test.go
@@ -1334,57 +1334,125 @@ func TestQueryValues(t *testing.T) {
type parseTest struct {
query string
out Values
+ ok bool
}
var parseTests = []parseTest{
+ {
+ query: "a=1",
+ out: Values{"a": []string{"1"}},
+ ok: true,
+ },
{
query: "a=1&b=2",
out: Values{"a": []string{"1"}, "b": []string{"2"}},
+ ok: true,
},
{
query: "a=1&a=2&a=banana",
out: Values{"a": []string{"1", "2", "banana"}},
+ ok: true,
},
{
query: "ascii=%3Ckey%3A+0x90%3E",
out: Values{"ascii": []string{""}},
+ ok: true,
+ }, {
+ query: "a=1;b=2",
+ out: Values{},
+ ok: false,
+ }, {
+ query: "a;b=1",
+ out: Values{},
+ ok: false,
+ }, {
+ query: "a=%3B", // hex encoding for semicolon
+ out: Values{"a": []string{";"}},
+ ok: true,
},
{
- query: "a=1;b=2",
- out: Values{"a": []string{"1"}, "b": []string{"2"}},
+ query: "a%3Bb=1",
+ out: Values{"a;b": []string{"1"}},
+ ok: true,
},
{
query: "a=1&a=2;a=banana",
- out: Values{"a": []string{"1", "2", "banana"}},
+ out: Values{"a": []string{"1"}},
+ ok: false,
+ },
+ {
+ query: "a;b&c=1",
+ out: Values{"c": []string{"1"}},
+ ok: false,
+ },
+ {
+ query: "a=1&b=2;a=3&c=4",
+ out: Values{"a": []string{"1"}, "c": []string{"4"}},
+ ok: false,
+ },
+ {
+ query: "a=1&b=2;c=3",
+ out: Values{"a": []string{"1"}},
+ ok: false,
+ },
+ {
+ query: ";",
+ out: Values{},
+ ok: false,
+ },
+ {
+ query: "a=1;",
+ out: Values{},
+ ok: false,
+ },
+ {
+ query: "a=1&;",
+ out: Values{"a": []string{"1"}},
+ ok: false,
+ },
+ {
+ query: ";a=1&b=2",
+ out: Values{"b": []string{"2"}},
+ ok: false,
+ },
+ {
+ query: "a=1&b=2;",
+ out: Values{"a": []string{"1"}},
+ ok: false,
},
}
func TestParseQuery(t *testing.T) {
- for i, test := range parseTests {
- form, err := ParseQuery(test.query)
- if err != nil {
- t.Errorf("test %d: Unexpected error: %v", i, err)
- continue
- }
- if len(form) != len(test.out) {
- t.Errorf("test %d: len(form) = %d, want %d", i, len(form), len(test.out))
- }
- for k, evs := range test.out {
- vs, ok := form[k]
- if !ok {
- t.Errorf("test %d: Missing key %q", i, k)
- continue
+ for _, test := range parseTests {
+ t.Run(test.query, func(t *testing.T) {
+ form, err := ParseQuery(test.query)
+ if test.ok != (err == nil) {
+ want := ""
+ if test.ok {
+ want = ""
+ }
+ t.Errorf("Unexpected error: %v, want %v", err, want)
}
- if len(vs) != len(evs) {
- t.Errorf("test %d: len(form[%q]) = %d, want %d", i, k, len(vs), len(evs))
- continue
+ if len(form) != len(test.out) {
+ t.Errorf("len(form) = %d, want %d", len(form), len(test.out))
}
- for j, ev := range evs {
- if v := vs[j]; v != ev {
- t.Errorf("test %d: form[%q][%d] = %q, want %q", i, k, j, v, ev)
+ for k, evs := range test.out {
+ vs, ok := form[k]
+ if !ok {
+ t.Errorf("Missing key %q", k)
+ continue
+ }
+ if len(vs) != len(evs) {
+ t.Errorf("len(form[%q]) = %d, want %d", k, len(vs), len(evs))
+ continue
+ }
+ for j, ev := range evs {
+ if v := vs[j]; v != ev {
+ t.Errorf("form[%q][%d] = %q, want %q", k, j, v, ev)
+ }
}
}
- }
+ })
}
}
diff --git a/src/os/exec_windows.go b/src/os/exec_windows.go
index 5710401acd..239bed198f 100644
--- a/src/os/exec_windows.go
+++ b/src/os/exec_windows.go
@@ -45,16 +45,6 @@ func (p *Process) wait() (ps *ProcessState, err error) {
return &ProcessState{p.Pid, syscall.WaitStatus{ExitCode: ec}, &u}, nil
}
-func terminateProcess(pid, exitcode int) error {
- h, e := syscall.OpenProcess(syscall.PROCESS_TERMINATE, false, uint32(pid))
- if e != nil {
- return NewSyscallError("OpenProcess", e)
- }
- defer syscall.CloseHandle(h)
- e = syscall.TerminateProcess(h, uint32(exitcode))
- return NewSyscallError("TerminateProcess", e)
-}
-
func (p *Process) signal(sig Signal) error {
handle := atomic.LoadUintptr(&p.handle)
if handle == uintptr(syscall.InvalidHandle) {
@@ -64,16 +54,22 @@ func (p *Process) signal(sig Signal) error {
return ErrProcessDone
}
if sig == Kill {
- err := terminateProcess(p.Pid, 1)
+ var terminationHandle syscall.Handle
+ e := syscall.DuplicateHandle(^syscall.Handle(0), syscall.Handle(handle), ^syscall.Handle(0), &terminationHandle, syscall.PROCESS_TERMINATE, false, 0)
+ if e != nil {
+ return NewSyscallError("DuplicateHandle", e)
+ }
runtime.KeepAlive(p)
- return err
+ defer syscall.CloseHandle(terminationHandle)
+ e = syscall.TerminateProcess(syscall.Handle(terminationHandle), 1)
+ return NewSyscallError("TerminateProcess", e)
}
// TODO(rsc): Handle Interrupt too?
return syscall.Errno(syscall.EWINDOWS)
}
func (p *Process) release() error {
- handle := atomic.LoadUintptr(&p.handle)
+ handle := atomic.SwapUintptr(&p.handle, uintptr(syscall.InvalidHandle))
if handle == uintptr(syscall.InvalidHandle) {
return syscall.EINVAL
}
@@ -81,7 +77,6 @@ func (p *Process) release() error {
if e != nil {
return NewSyscallError("CloseHandle", e)
}
- atomic.StoreUintptr(&p.handle, uintptr(syscall.InvalidHandle))
// no need for a finalizer anymore
runtime.SetFinalizer(p, nil)
return nil
diff --git a/src/os/fifo_test.go b/src/os/fifo_test.go
index 9b262f8205..007ed29129 100644
--- a/src/os/fifo_test.go
+++ b/src/os/fifo_test.go
@@ -26,9 +26,6 @@ func TestFifoEOF(t *testing.T) {
switch runtime.GOOS {
case "android":
t.Skip("skipping on Android; mkfifo syscall not available")
- case "openbsd":
- // On OpenBSD 6.2 this test just hangs for some reason.
- t.Skip("skipping on OpenBSD; issue 25877")
}
dir := t.TempDir()
diff --git a/src/os/signal/signal_test.go b/src/os/signal/signal_test.go
index cea68742d2..649854b746 100644
--- a/src/os/signal/signal_test.go
+++ b/src/os/signal/signal_test.go
@@ -32,6 +32,11 @@ import (
// The current value is set based on flakes observed in the Go builders.
var settleTime = 100 * time.Millisecond
+// fatalWaitingTime is an absurdly long time to wait for signals to be
+// delivered but, using it, we (hopefully) eliminate test flakes on the
+// build servers. See #46736 for discussion.
+var fatalWaitingTime = 30 * time.Second
+
func init() {
if testenv.Builder() == "solaris-amd64-oraclerel" {
// The solaris-amd64-oraclerel builder has been observed to time out in
@@ -84,7 +89,7 @@ func waitSig1(t *testing.T, c <-chan os.Signal, sig os.Signal, all bool) {
// General user code should filter out all unexpected signals instead of just
// SIGURG, but since os/signal is tightly coupled to the runtime it seems
// appropriate to be stricter here.
- for time.Since(start) < settleTime {
+ for time.Since(start) < fatalWaitingTime {
select {
case s := <-c:
if s == sig {
@@ -97,7 +102,7 @@ func waitSig1(t *testing.T, c <-chan os.Signal, sig os.Signal, all bool) {
timer.Reset(settleTime / 10)
}
}
- t.Fatalf("timeout after %v waiting for %v", settleTime, sig)
+ t.Fatalf("timeout after %v waiting for %v", fatalWaitingTime, sig)
}
// quiesce waits until we can be reasonably confident that all pending signals
diff --git a/src/reflect/abi_test.go b/src/reflect/abi_test.go
index 1a2a48b5ed..5a0130f7b4 100644
--- a/src/reflect/abi_test.go
+++ b/src/reflect/abi_test.go
@@ -79,7 +79,34 @@ func TestMethodValueCallABI(t *testing.T) {
t.Errorf("bad method value call: got %#v, want %#v", r2, a2)
}
if s.Value != 3 {
- t.Errorf("bad method value call: failed to set s.Value: got %d, want %d", s.Value, 1)
+ t.Errorf("bad method value call: failed to set s.Value: got %d, want %d", s.Value, 3)
+ }
+
+ s, i = makeMethodValue("ValueRegMethodSpillInt")
+ f3 := i.(func(StructFillRegs, int, MagicLastTypeNameForTestingRegisterABI) (StructFillRegs, int))
+ r3a, r3b := f3(a2, 42, MagicLastTypeNameForTestingRegisterABI{})
+ if r3a != a2 {
+ t.Errorf("bad method value call: got %#v, want %#v", r3a, a2)
+ }
+ if r3b != 42 {
+ t.Errorf("bad method value call: got %#v, want %#v", r3b, 42)
+ }
+ if s.Value != 4 {
+ t.Errorf("bad method value call: failed to set s.Value: got %d, want %d", s.Value, 4)
+ }
+
+ s, i = makeMethodValue("ValueRegMethodSpillPtr")
+ f4 := i.(func(StructFillRegs, *byte, MagicLastTypeNameForTestingRegisterABI) (StructFillRegs, *byte))
+ vb := byte(10)
+ r4a, r4b := f4(a2, &vb, MagicLastTypeNameForTestingRegisterABI{})
+ if r4a != a2 {
+ t.Errorf("bad method value call: got %#v, want %#v", r4a, a2)
+ }
+ if r4b != &vb {
+ t.Errorf("bad method value call: got %#v, want %#v", r4b, &vb)
+ }
+ if s.Value != 5 {
+ t.Errorf("bad method value call: failed to set s.Value: got %d, want %d", s.Value, 5)
}
}
@@ -112,6 +139,20 @@ func (m *StructWithMethods) SpillStructCall(s StructFillRegs, _ MagicLastTypeNam
return s
}
+// When called as a method value, i is passed on the stack.
+// When called as a method, i is passed in a register.
+func (m *StructWithMethods) ValueRegMethodSpillInt(s StructFillRegs, i int, _ MagicLastTypeNameForTestingRegisterABI) (StructFillRegs, int) {
+ m.Value = 4
+ return s, i
+}
+
+// When called as a method value, i is passed on the stack.
+// When called as a method, i is passed in a register.
+func (m *StructWithMethods) ValueRegMethodSpillPtr(s StructFillRegs, i *byte, _ MagicLastTypeNameForTestingRegisterABI) (StructFillRegs, *byte) {
+ m.Value = 5
+ return s, i
+}
+
func TestReflectCallABI(t *testing.T) {
// Enable register-based reflect.Call and ensure we don't
// use potentially incorrect cached versions by clearing
diff --git a/src/reflect/all_test.go b/src/reflect/all_test.go
index 17104ad4fa..0db5e13217 100644
--- a/src/reflect/all_test.go
+++ b/src/reflect/all_test.go
@@ -4371,7 +4371,7 @@ func TestConvertPanic(t *testing.T) {
if !v.Type().ConvertibleTo(pt) {
t.Errorf("[]byte should be convertible to *[8]byte")
}
- shouldPanic("reflect: cannot convert slice with length 4 to array pointer with length 8", func() {
+ shouldPanic("reflect: cannot convert slice with length 4 to pointer to array with length 8", func() {
_ = v.Convert(pt)
})
}
diff --git a/src/reflect/type.go b/src/reflect/type.go
index 39414fc2a6..df863ae106 100644
--- a/src/reflect/type.go
+++ b/src/reflect/type.go
@@ -107,10 +107,14 @@ type Type interface {
// ConvertibleTo reports whether a value of the type is convertible to type u.
// Even if ConvertibleTo returns true, the conversion may still panic.
+ // For example, a slice of type []T is convertible to *[N]T,
+ // but the conversion will panic if its length is less than N.
ConvertibleTo(u Type) bool
// Comparable reports whether values of this type are comparable.
// Even if Comparable returns true, the comparison may still panic.
+ // For example, values of interface type are comparable,
+ // but the comparison will panic if their dynamic type is not comparable.
Comparable() bool
// Methods applicable only to some types, depending on Kind.
diff --git a/src/reflect/value.go b/src/reflect/value.go
index 418dff781f..9dce251ac5 100644
--- a/src/reflect/value.go
+++ b/src/reflect/value.go
@@ -952,25 +952,47 @@ func callMethod(ctxt *methodValue, frame unsafe.Pointer, retValid *bool, regs *a
continue
}
- // There are three cases to handle in translating each
+ // There are four cases to handle in translating each
// argument:
// 1. Stack -> stack translation.
- // 2. Registers -> stack translation.
- // 3. Registers -> registers translation.
- // The fourth cases can't happen, because a method value
- // call uses strictly fewer registers than a method call.
+ // 2. Stack -> registers translation.
+ // 3. Registers -> stack translation.
+ // 4. Registers -> registers translation.
+ // TODO(mknyszek): Cases 2 and 3 below only work on little endian
+ // architectures. This is OK for now, but this needs to be fixed
+ // before supporting the register ABI on big endian architectures.
// If the value ABI passes the value on the stack,
// then the method ABI does too, because it has strictly
// fewer arguments. Simply copy between the two.
if vStep := valueSteps[0]; vStep.kind == abiStepStack {
mStep := methodSteps[0]
- if mStep.kind != abiStepStack || vStep.size != mStep.size {
- panic("method ABI and value ABI do not align")
+ // Handle stack -> stack translation.
+ if mStep.kind == abiStepStack {
+ if vStep.size != mStep.size {
+ panic("method ABI and value ABI do not align")
+ }
+ typedmemmove(t,
+ add(methodFrame, mStep.stkOff, "precomputed stack offset"),
+ add(valueFrame, vStep.stkOff, "precomputed stack offset"))
+ continue
+ }
+ // Handle stack -> register translation.
+ for _, mStep := range methodSteps {
+ from := add(valueFrame, vStep.stkOff+mStep.offset, "precomputed stack offset")
+ switch mStep.kind {
+ case abiStepPointer:
+ // Do the pointer copy directly so we get a write barrier.
+ methodRegs.Ptrs[mStep.ireg] = *(*unsafe.Pointer)(from)
+ fallthrough // We need to make sure this ends up in Ints, too.
+ case abiStepIntReg:
+ memmove(unsafe.Pointer(&methodRegs.Ints[mStep.ireg]), from, mStep.size)
+ case abiStepFloatReg:
+ memmove(unsafe.Pointer(&methodRegs.Floats[mStep.freg]), from, mStep.size)
+ default:
+ panic("unexpected method step")
+ }
}
- typedmemmove(t,
- add(methodFrame, mStep.stkOff, "precomputed stack offset"),
- add(valueFrame, vStep.stkOff, "precomputed stack offset"))
continue
}
// Handle register -> stack translation.
@@ -1359,10 +1381,16 @@ func valueInterface(v Value, safe bool) interface{} {
return packEface(v)
}
-// InterfaceData returns the interface v's value as a uintptr pair.
+// InterfaceData returns a pair of unspecified uintptr values.
// It panics if v's Kind is not Interface.
+//
+// In earlier versions of Go, this function returned the interface's
+// value as a uintptr pair. As of Go 1.4, the implementation of
+// interface values precludes any defined use of InterfaceData.
+//
+// Deprecated: The memory representation of interface values is not
+// compatible with InterfaceData.
func (v Value) InterfaceData() [2]uintptr {
- // TODO: deprecate this
v.mustBe(Interface)
// We treat this as a read operation, so we allow
// it even for unexported data, because the caller
@@ -3045,7 +3073,7 @@ func cvtSliceArrayPtr(v Value, t Type) Value {
n := t.Elem().Len()
h := (*unsafeheader.Slice)(v.ptr)
if n > h.Len {
- panic("reflect: cannot convert slice with length " + itoa.Itoa(h.Len) + " to array pointer with length " + itoa.Itoa(n))
+ panic("reflect: cannot convert slice with length " + itoa.Itoa(h.Len) + " to pointer to array with length " + itoa.Itoa(n))
}
return Value{t.common(), h.Data, v.flag&^(flagIndir|flagAddr|flagKindMask) | flag(Ptr)}
}
diff --git a/src/runtime/cgocall.go b/src/runtime/cgocall.go
index 0e287d0b8e..8ffb48a888 100644
--- a/src/runtime/cgocall.go
+++ b/src/runtime/cgocall.go
@@ -110,6 +110,8 @@ func syscall_cgocaller(fn unsafe.Pointer, args ...uintptr) uintptr {
return as.retval
}
+var ncgocall uint64 // number of cgo calls in total for dead m
+
// Call from Go to C.
//
// This must be nosplit because it's used for syscalls on some
diff --git a/src/runtime/debug.go b/src/runtime/debug.go
index f411b22676..82deefa200 100644
--- a/src/runtime/debug.go
+++ b/src/runtime/debug.go
@@ -45,7 +45,7 @@ func NumCPU() int {
// NumCgoCall returns the number of cgo calls made by the current process.
func NumCgoCall() int64 {
- var n int64
+ var n = int64(atomic.Load64(&ncgocall))
for mp := (*m)(atomic.Loadp(unsafe.Pointer(&allm))); mp != nil; mp = mp.alllink {
n += int64(mp.ncgocall)
}
diff --git a/src/runtime/example_test.go b/src/runtime/example_test.go
index e4912a5158..dcb8f7798e 100644
--- a/src/runtime/example_test.go
+++ b/src/runtime/example_test.go
@@ -12,12 +12,15 @@ import (
func ExampleFrames() {
c := func() {
- // Ask runtime.Callers for up to 10 pcs, including runtime.Callers itself.
+ // Ask runtime.Callers for up to 10 PCs, including runtime.Callers itself.
pc := make([]uintptr, 10)
n := runtime.Callers(0, pc)
if n == 0 {
- // No pcs available. Stop now.
- // This can happen if the first argument to runtime.Callers is large.
+ // No PCs available. This can happen if the first argument to
+ // runtime.Callers is large.
+ //
+ // Return now to avoid processing the zero Frame that would
+ // otherwise be returned by frames.Next below.
return
}
@@ -25,9 +28,12 @@ func ExampleFrames() {
frames := runtime.CallersFrames(pc)
// Loop to get frames.
- // A fixed number of pcs can expand to an indefinite number of Frames.
+ // A fixed number of PCs can expand to an indefinite number of Frames.
for {
frame, more := frames.Next()
+
+ // Process this frame.
+ //
// To keep this example's output stable
// even if there are changes in the testing package,
// stop unwinding when we leave package runtime.
@@ -35,6 +41,8 @@ func ExampleFrames() {
break
}
fmt.Printf("- more:%v | %s\n", more, frame.Function)
+
+ // Check whether there are more frames to process after this one.
if !more {
break
}
diff --git a/src/runtime/internal/atomic/atomic_386.s b/src/runtime/internal/atomic/atomic_386.s
index 37318e0ad7..724d515231 100644
--- a/src/runtime/internal/atomic/atomic_386.s
+++ b/src/runtime/internal/atomic/atomic_386.s
@@ -65,7 +65,7 @@ TEXT ·Xaddint64(SB), NOSPLIT, $0-20
// bool ·Cas64(uint64 *val, uint64 old, uint64 new)
// Atomically:
-// if(*val == *old){
+// if(*val == old){
// *val = new;
// return 1;
// } else {
diff --git a/src/runtime/os_darwin.go b/src/runtime/os_darwin.go
index 00139351ab..079be107d7 100644
--- a/src/runtime/os_darwin.go
+++ b/src/runtime/os_darwin.go
@@ -118,10 +118,15 @@ func sigNoteWakeup(*note) {
// sigNoteSleep waits for a note created by sigNoteSetup to be woken.
func sigNoteSleep(*note) {
- entersyscallblock()
- var b byte
- read(sigNoteRead, unsafe.Pointer(&b), 1)
- exitsyscall()
+ for {
+ var b byte
+ entersyscallblock()
+ n := read(sigNoteRead, unsafe.Pointer(&b), 1)
+ exitsyscall()
+ if n != -_EINTR {
+ return
+ }
+ }
}
// BSD interface for threading.
diff --git a/src/runtime/pprof/pprof_test.go b/src/runtime/pprof/pprof_test.go
index 7cbb4fc7ae..7c71d8263b 100644
--- a/src/runtime/pprof/pprof_test.go
+++ b/src/runtime/pprof/pprof_test.go
@@ -260,6 +260,27 @@ func parseProfile(t *testing.T, valBytes []byte, f func(uintptr, []*profile.Loca
return p
}
+func cpuProfilingBroken() bool {
+ switch runtime.GOOS {
+ case "plan9":
+ // Profiling unimplemented.
+ return true
+ case "aix":
+ // See https://golang.org/issue/45170.
+ return true
+ case "ios", "dragonfly", "netbsd", "illumos", "solaris":
+ // See https://golang.org/issue/13841.
+ return true
+ case "openbsd":
+ if runtime.GOARCH == "arm" || runtime.GOARCH == "arm64" {
+ // See https://golang.org/issue/13841.
+ return true
+ }
+ }
+
+ return false
+}
+
// testCPUProfile runs f under the CPU profiler, checking for some conditions specified by need,
// as interpreted by matches, and returns the parsed profile.
func testCPUProfile(t *testing.T, matches matchFunc, need []string, avoid []string, f func(dur time.Duration)) *profile.Profile {
@@ -275,16 +296,7 @@ func testCPUProfile(t *testing.T, matches matchFunc, need []string, avoid []stri
t.Skip("skipping on plan9")
}
- broken := false
- switch runtime.GOOS {
- // See https://golang.org/issue/45170 for AIX.
- case "ios", "dragonfly", "netbsd", "illumos", "solaris", "aix":
- broken = true
- case "openbsd":
- if runtime.GOARCH == "arm" || runtime.GOARCH == "arm64" {
- broken = true
- }
- }
+ broken := cpuProfilingBroken()
maxDuration := 5 * time.Second
if testing.Short() && broken {
@@ -611,7 +623,7 @@ func growstack1() {
//go:noinline
func growstack(n int) {
- var buf [8 << 16]byte
+ var buf [8 << 18]byte
use(buf)
if n > 0 {
growstack(n - 1)
@@ -619,7 +631,7 @@ func growstack(n int) {
}
//go:noinline
-func use(x [8 << 16]byte) {}
+func use(x [8 << 18]byte) {}
func TestBlockProfile(t *testing.T) {
type TestCase struct {
diff --git a/src/runtime/proc.go b/src/runtime/proc.go
index ded406cc28..4c92588a66 100644
--- a/src/runtime/proc.go
+++ b/src/runtime/proc.go
@@ -529,8 +529,8 @@ var (
allglock mutex
allgs []*g
- // allglen and allgptr are atomic variables that contain len(allg) and
- // &allg[0] respectively. Proper ordering depends on totally-ordered
+ // allglen and allgptr are atomic variables that contain len(allgs) and
+ // &allgs[0] respectively. Proper ordering depends on totally-ordered
// loads and stores. Writes are protected by allglock.
//
// allgptr is updated before allglen. Readers should read allglen
@@ -1522,6 +1522,8 @@ found:
}
unlock(&sched.lock)
+ atomic.Xadd64(&ncgocall, int64(m.ncgocall))
+
// Release the P.
handoffp(releasep())
// After this point we must not have write barriers.
@@ -4083,8 +4085,16 @@ func exitsyscall0(gp *g) {
if schedEnabled(gp) {
_p_ = pidleget()
}
+ var locked bool
if _p_ == nil {
globrunqput(gp)
+
+ // Below, we stoplockedm if gp is locked. globrunqput releases
+ // ownership of gp, so we must check if gp is locked prior to
+ // committing the release by unlocking sched.lock, otherwise we
+ // could race with another M transitioning gp from unlocked to
+ // locked.
+ locked = gp.lockedm != 0
} else if atomic.Load(&sched.sysmonwait) != 0 {
atomic.Store(&sched.sysmonwait, 0)
notewakeup(&sched.sysmonnote)
@@ -4094,7 +4104,7 @@ func exitsyscall0(gp *g) {
acquirep(_p_)
execute(gp, false) // Never returns.
}
- if gp.lockedm != 0 {
+ if locked {
// Wait until another thread schedules gp and so m again.
//
// N.B. lockedm must be this M, as this g was running on this M
diff --git a/src/runtime/race/syso_test.go b/src/runtime/race/syso_test.go
index cbce5a8f18..f5095737a4 100644
--- a/src/runtime/race/syso_test.go
+++ b/src/runtime/race/syso_test.go
@@ -2,14 +2,8 @@
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
-//go:build !android && !js && !ppc64le
-// +build !android,!js,!ppc64le
-
-// Note: we don't run on Android or ppc64 because if there is any non-race test
-// file in this package, the OS tries to link the .syso file into the
-// test (even when we're not in race mode), which fails. I'm not sure
-// why, but easiest to just punt - as long as a single builder runs
-// this test, we're good.
+//go:build race
+// +build race
package race
diff --git a/src/runtime/signal_unix.go b/src/runtime/signal_unix.go
index f2e526973d..6096760b50 100644
--- a/src/runtime/signal_unix.go
+++ b/src/runtime/signal_unix.go
@@ -280,6 +280,8 @@ func setProcessCPUProfiler(hz int32) {
it.it_value = it.it_interval
setitimer(_ITIMER_PROF, &it, nil)
} else {
+ setitimer(_ITIMER_PROF, &itimerval{}, nil)
+
// If the Go signal handler should be disabled by default,
// switch back to the signal handler that was installed
// when we enabled profiling. We don't try to handle the case
@@ -303,8 +305,6 @@ func setProcessCPUProfiler(hz int32) {
setsig(_SIGPROF, h)
}
}
-
- setitimer(_ITIMER_PROF, &itimerval{}, nil)
}
}
@@ -382,7 +382,7 @@ func preemptM(mp *m) {
//go:nosplit
func sigFetchG(c *sigctxt) *g {
switch GOARCH {
- case "arm", "arm64":
+ case "arm", "arm64", "ppc64", "ppc64le":
if !iscgo && inVDSOPage(c.sigpc()) {
// When using cgo, we save the g on TLS and load it from there
// in sigtramp. Just use that.
diff --git a/src/runtime/symtab.go b/src/runtime/symtab.go
index 6b535dfcbf..999300a58e 100644
--- a/src/runtime/symtab.go
+++ b/src/runtime/symtab.go
@@ -68,8 +68,15 @@ func CallersFrames(callers []uintptr) *Frames {
return f
}
-// Next returns frame information for the next caller.
-// If more is false, there are no more callers (the Frame value is valid).
+// Next returns a Frame representing the next call frame in the slice
+// of PC values. If it has already returned all call frames, Next
+// returns a zero Frame.
+//
+// The more result indicates whether the next call to Next will return
+// a valid Frame. It does not necessarily indicate whether this call
+// returned one.
+//
+// See the Frames example for idiomatic usage.
func (ci *Frames) Next() (frame Frame, more bool) {
for len(ci.frames) < 2 {
// Find the next frame.
diff --git a/src/runtime/sys_linux_ppc64x.s b/src/runtime/sys_linux_ppc64x.s
index 05b5916db4..005fa4d2b4 100644
--- a/src/runtime/sys_linux_ppc64x.s
+++ b/src/runtime/sys_linux_ppc64x.s
@@ -216,15 +216,45 @@ TEXT runtime·walltime(SB),NOSPLIT,$16-12
MOVD (g_sched+gobuf_sp)(R7), R1 // Set SP to g0 stack
noswitch:
- SUB $16, R1 // Space for results
- RLDICR $0, R1, $59, R1 // Align for C code
+ SUB $16, R1 // Space for results
+ RLDICR $0, R1, $59, R1 // Align for C code
MOVD R12, CTR
MOVD R1, R4
- BL (CTR) // Call from VDSO
- MOVD $0, R0 // Restore R0
- MOVD 0(R1), R3 // sec
- MOVD 8(R1), R5 // nsec
- MOVD R15, R1 // Restore SP
+
+ // Store g on gsignal's stack, so if we receive a signal
+ // during VDSO code we can find the g.
+ // If we don't have a signal stack, we won't receive signal,
+ // so don't bother saving g.
+ // When using cgo, we already saved g on TLS, also don't save
+ // g here.
+ // Also don't save g if we are already on the signal stack.
+ // We won't get a nested signal.
+ MOVBZ runtime·iscgo(SB), R22
+ CMP R22, $0
+ BNE nosaveg
+ MOVD m_gsignal(R21), R22 // g.m.gsignal
+ CMP R22, $0
+ BEQ nosaveg
+
+ CMP g, R22
+ BEQ nosaveg
+ MOVD (g_stack+stack_lo)(R22), R22 // g.m.gsignal.stack.lo
+ MOVD g, (R22)
+
+ BL (CTR) // Call from VDSO
+
+ MOVD $0, (R22) // clear g slot, R22 is unchanged by C code
+
+ JMP finish
+
+nosaveg:
+ BL (CTR) // Call from VDSO
+
+finish:
+ MOVD $0, R0 // Restore R0
+ MOVD 0(R1), R3 // sec
+ MOVD 8(R1), R5 // nsec
+ MOVD R15, R1 // Restore SP
// Restore vdsoPC, vdsoSP
// We don't worry about being signaled between the two stores.
@@ -236,7 +266,7 @@ noswitch:
MOVD 32(R1), R6
MOVD R6, m_vdsoPC(R21)
-finish:
+return:
MOVD R3, sec+0(FP)
MOVW R5, nsec+8(FP)
RET
@@ -247,7 +277,7 @@ fallback:
SYSCALL $SYS_clock_gettime
MOVD 32(R1), R3
MOVD 40(R1), R5
- JMP finish
+ JMP return
TEXT runtime·nanotime1(SB),NOSPLIT,$16-8
MOVD $1, R3 // CLOCK_MONOTONIC
@@ -283,7 +313,37 @@ noswitch:
RLDICR $0, R1, $59, R1 // Align for C code
MOVD R12, CTR
MOVD R1, R4
- BL (CTR) // Call from VDSO
+
+ // Store g on gsignal's stack, so if we receive a signal
+ // during VDSO code we can find the g.
+ // If we don't have a signal stack, we won't receive signal,
+ // so don't bother saving g.
+ // When using cgo, we already saved g on TLS, also don't save
+ // g here.
+ // Also don't save g if we are already on the signal stack.
+ // We won't get a nested signal.
+ MOVBZ runtime·iscgo(SB), R22
+ CMP R22, $0
+ BNE nosaveg
+ MOVD m_gsignal(R21), R22 // g.m.gsignal
+ CMP R22, $0
+ BEQ nosaveg
+
+ CMP g, R22
+ BEQ nosaveg
+ MOVD (g_stack+stack_lo)(R22), R22 // g.m.gsignal.stack.lo
+ MOVD g, (R22)
+
+ BL (CTR) // Call from VDSO
+
+ MOVD $0, (R22) // clear g slot, R22 is unchanged by C code
+
+ JMP finish
+
+nosaveg:
+ BL (CTR) // Call from VDSO
+
+finish:
MOVD $0, R0 // Restore R0
MOVD 0(R1), R3 // sec
MOVD 8(R1), R5 // nsec
@@ -299,7 +359,7 @@ noswitch:
MOVD 32(R1), R6
MOVD R6, m_vdsoPC(R21)
-finish:
+return:
// sec is in R3, nsec in R5
// return nsec in R3
MOVD $1000000000, R4
@@ -314,7 +374,7 @@ fallback:
SYSCALL $SYS_clock_gettime
MOVD 32(R1), R3
MOVD 40(R1), R5
- JMP finish
+ JMP return
TEXT runtime·rtsigprocmask(SB),NOSPLIT|NOFRAME,$0-28
MOVW how+0(FP), R3
@@ -469,7 +529,7 @@ TEXT sigtramp<>(SB),NOSPLIT|NOFRAME,$0
// this might be called in external code context,
// where g is not set.
MOVBZ runtime·iscgo(SB), R6
- CMP R6, $0
+ CMP R6, $0
BEQ 2(PC)
BL runtime·load_g(SB)
diff --git a/src/runtime/testdata/testprogcgo/aprof.go b/src/runtime/testdata/testprogcgo/aprof.go
index aabca9e1eb..44a15b0865 100644
--- a/src/runtime/testdata/testprogcgo/aprof.go
+++ b/src/runtime/testdata/testprogcgo/aprof.go
@@ -10,7 +10,7 @@ package main
// The test fails when the function is the first C function.
// The exported functions are the first C functions, so we use that.
-// extern void GoNop();
+// extern void CallGoNop();
import "C"
import (
@@ -38,7 +38,7 @@ func CgoCCodeSIGPROF() {
break
}
}
- C.GoNop()
+ C.CallGoNop()
}
c <- true
}()
diff --git a/src/runtime/testdata/testprogcgo/aprof_c.c b/src/runtime/testdata/testprogcgo/aprof_c.c
new file mode 100644
index 0000000000..d588e13045
--- /dev/null
+++ b/src/runtime/testdata/testprogcgo/aprof_c.c
@@ -0,0 +1,9 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+#include "_cgo_export.h"
+
+void CallGoNop() {
+ GoNop();
+}
diff --git a/src/runtime/testdata/testprogcgo/bigstack1_windows.c b/src/runtime/testdata/testprogcgo/bigstack1_windows.c
new file mode 100644
index 0000000000..551fb68309
--- /dev/null
+++ b/src/runtime/testdata/testprogcgo/bigstack1_windows.c
@@ -0,0 +1,12 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// This is not in bigstack_windows.c because it needs to be part of
+// testprogcgo but is not part of the DLL built from bigstack_windows.c.
+
+#include "_cgo_export.h"
+
+void CallGoBigStack1(char* p) {
+ goBigStack1(p);
+}
diff --git a/src/runtime/testdata/testprogcgo/bigstack_windows.go b/src/runtime/testdata/testprogcgo/bigstack_windows.go
index f58fcf993f..135b5fcfe0 100644
--- a/src/runtime/testdata/testprogcgo/bigstack_windows.go
+++ b/src/runtime/testdata/testprogcgo/bigstack_windows.go
@@ -6,7 +6,7 @@ package main
/*
typedef void callback(char*);
-extern void goBigStack1(char*);
+extern void CallGoBigStack1(char*);
extern void bigStack(callback*);
*/
import "C"
@@ -18,7 +18,7 @@ func init() {
func BigStack() {
// Create a large thread stack and call back into Go to test
// if Go correctly determines the stack bounds.
- C.bigStack((*C.callback)(C.goBigStack1))
+ C.bigStack((*C.callback)(C.CallGoBigStack1))
}
//export goBigStack1
diff --git a/src/runtime/traceback.go b/src/runtime/traceback.go
index 89780edc1f..814c323634 100644
--- a/src/runtime/traceback.go
+++ b/src/runtime/traceback.go
@@ -56,8 +56,6 @@ func tracebackdefers(gp *g, callback func(*stkframe, unsafe.Pointer) bool, v uns
}
}
-const sizeofSkipFunction = 256
-
// Generic traceback. Handles runtime stack prints (pcbuf == nil),
// the runtime.Callers function (pcbuf != nil), as well as the garbage
// collector (callback != nil). A little clunky to merge these, but avoids
@@ -65,9 +63,7 @@ const sizeofSkipFunction = 256
//
// The skip argument is only valid with pcbuf != nil and counts the number
// of logical frames to skip rather than physical frames (with inlining, a
-// PC in pcbuf can represent multiple calls). If a PC is partially skipped
-// and max > 1, pcbuf[1] will be runtime.skipPleaseUseCallersFrames+N where
-// N indicates the number of logical frames to skip in pcbuf[0].
+// PC in pcbuf can represent multiple calls).
func gentraceback(pc0, sp0, lr0 uintptr, gp *g, skip int, pcbuf *uintptr, max int, callback func(*stkframe, unsafe.Pointer) bool, v unsafe.Pointer, flags uint) int {
if skip > 0 && callback != nil {
throw("gentraceback callback cannot be used with non-zero skip")
diff --git a/src/strconv/atof.go b/src/strconv/atof.go
index 9010a66ca8..57556c7047 100644
--- a/src/strconv/atof.go
+++ b/src/strconv/atof.go
@@ -689,7 +689,7 @@ func atof64(s string) (f float64, n int, err error) {
// as their respective special floating point values. It ignores case when matching.
func ParseFloat(s string, bitSize int) (float64, error) {
f, n, err := parseFloatPrefix(s, bitSize)
- if err == nil && n != len(s) {
+ if n != len(s) && (err == nil || err.(*NumError).Err != ErrSyntax) {
return 0, syntaxError(fnParseFloat, s)
}
return f, err
diff --git a/src/strconv/atof_test.go b/src/strconv/atof_test.go
index 3c058b9be5..aa587a473c 100644
--- a/src/strconv/atof_test.go
+++ b/src/strconv/atof_test.go
@@ -342,6 +342,9 @@ var atoftests = []atofTest{
{"0x12.345p-_12", "0", ErrSyntax},
{"0x12.345p+1__2", "0", ErrSyntax},
{"0x12.345p+12_", "0", ErrSyntax},
+
+ {"1e100x", "0", ErrSyntax},
+ {"1e1000x", "0", ErrSyntax},
}
var atof32tests = []atofTest{
diff --git a/src/strconv/atoi.go b/src/strconv/atoi.go
index c9ba0383b3..631b487d97 100644
--- a/src/strconv/atoi.go
+++ b/src/strconv/atoi.go
@@ -57,6 +57,8 @@ const IntSize = intSize
const maxUint64 = 1<<64 - 1
// ParseUint is like ParseInt but for unsigned numbers.
+//
+// A sign prefix is not permitted.
func ParseUint(s string, base int, bitSize int) (uint64, error) {
const fnParseUint = "ParseUint"
@@ -159,10 +161,13 @@ func ParseUint(s string, base int, bitSize int) (uint64, error) {
// ParseInt interprets a string s in the given base (0, 2 to 36) and
// bit size (0 to 64) and returns the corresponding value i.
//
+// The string may begin with a leading sign: "+" or "-".
+//
// If the base argument is 0, the true base is implied by the string's
-// prefix: 2 for "0b", 8 for "0" or "0o", 16 for "0x", and 10 otherwise.
-// Also, for argument base 0 only, underscore characters are permitted
-// as defined by the Go syntax for integer literals.
+// prefix following the sign (if present): 2 for "0b", 8 for "0" or "0o",
+// 16 for "0x", and 10 otherwise. Also, for argument base 0 only,
+// underscore characters are permitted as defined by the Go syntax for
+// integer literals.
//
// The bitSize argument specifies the integer type
// that the result must fit into. Bit sizes 0, 8, 16, 32, and 64
diff --git a/src/strconv/atoi_test.go b/src/strconv/atoi_test.go
index 178fb01ea7..867fa66a14 100644
--- a/src/strconv/atoi_test.go
+++ b/src/strconv/atoi_test.go
@@ -33,6 +33,9 @@ var parseUint64Tests = []parseUint64Test{
{"_12345", 0, ErrSyntax},
{"1__2345", 0, ErrSyntax},
{"12345_", 0, ErrSyntax},
+ {"-0", 0, ErrSyntax},
+ {"-1", 0, ErrSyntax},
+ {"+1", 0, ErrSyntax},
}
type parseUint64BaseTest struct {
@@ -140,8 +143,10 @@ var parseInt64Tests = []parseInt64Test{
{"", 0, ErrSyntax},
{"0", 0, nil},
{"-0", 0, nil},
+ {"+0", 0, nil},
{"1", 1, nil},
{"-1", -1, nil},
+ {"+1", 1, nil},
{"12345", 12345, nil},
{"-12345", -12345, nil},
{"012345", 12345, nil},
@@ -236,6 +241,11 @@ var parseInt64BaseTests = []parseInt64BaseTest{
{"0__12345", 0, 0, ErrSyntax},
{"01234__5", 0, 0, ErrSyntax},
{"012345_", 0, 0, ErrSyntax},
+
+ {"+0xf", 0, 0xf, nil},
+ {"-0xf", 0, -0xf, nil},
+ {"0x+f", 0, 0, ErrSyntax},
+ {"0x-f", 0, 0, ErrSyntax},
}
type parseUint32Test struct {
diff --git a/src/syscall/exec_linux_test.go b/src/syscall/exec_linux_test.go
index 7d89eaae63..85b59ad00d 100644
--- a/src/syscall/exec_linux_test.go
+++ b/src/syscall/exec_linux_test.go
@@ -318,6 +318,7 @@ func TestGroupCleanupUserNamespace(t *testing.T) {
"uid=0(root) gid=0(root) groups=0(root),65534",
"uid=0(root) gid=0(root) groups=0(root),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody),65534(nobody)", // Alpine; see https://golang.org/issue/19938
"uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023", // CentOS with SELinux context, see https://golang.org/issue/34547
+ "uid=0(root) gid=0(root) groups=0(root),65534(nobody) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023", // Fedora with SElinux context, see https://golang.org/issue/46752
}
for _, e := range expected {
if strOut == e {
diff --git a/src/syscall/exec_windows.go b/src/syscall/exec_windows.go
index 253e9e8c1f..18d15028c3 100644
--- a/src/syscall/exec_windows.go
+++ b/src/syscall/exec_windows.go
@@ -313,6 +313,17 @@ func StartProcess(argv0 string, argv []string, attr *ProcAttr) (pid int, handle
}
}
+ var maj, min, build uint32
+ rtlGetNtVersionNumbers(&maj, &min, &build)
+ isWin7 := maj < 6 || (maj == 6 && min <= 1)
+ // NT kernel handles are divisible by 4, with the bottom 3 bits left as
+ // a tag. The fully set tag correlates with the types of handles we're
+ // concerned about here. Except, the kernel will interpret some
+ // special handle values, like -1, -2, and so forth, so kernelbase.dll
+ // checks to see that those bottom three bits are checked, but that top
+ // bit is not checked.
+ isLegacyWin7ConsoleHandle := func(handle Handle) bool { return isWin7 && handle&0x10000003 == 3 }
+
p, _ := GetCurrentProcess()
parentProcess := p
if sys.ParentProcess != 0 {
@@ -321,7 +332,15 @@ func StartProcess(argv0 string, argv []string, attr *ProcAttr) (pid int, handle
fd := make([]Handle, len(attr.Files))
for i := range attr.Files {
if attr.Files[i] > 0 {
- err := DuplicateHandle(p, Handle(attr.Files[i]), parentProcess, &fd[i], 0, true, DUPLICATE_SAME_ACCESS)
+ destinationProcessHandle := parentProcess
+
+ // On Windows 7, console handles aren't real handles, and can only be duplicated
+ // into the current process, not a parent one, which amounts to the same thing.
+ if parentProcess != p && isLegacyWin7ConsoleHandle(Handle(attr.Files[i])) {
+ destinationProcessHandle = p
+ }
+
+ err := DuplicateHandle(p, Handle(attr.Files[i]), destinationProcessHandle, &fd[i], 0, true, DUPLICATE_SAME_ACCESS)
if err != nil {
return 0, 0, err
}
@@ -351,19 +370,40 @@ func StartProcess(argv0 string, argv []string, attr *ProcAttr) (pid int, handle
si.StdErr = fd[2]
fd = append(fd, sys.AdditionalInheritedHandles...)
+
+ // On Windows 7, console handles aren't real handles, so don't pass them
+ // through to PROC_THREAD_ATTRIBUTE_HANDLE_LIST.
+ for i := range fd {
+ if isLegacyWin7ConsoleHandle(fd[i]) {
+ fd[i] = 0
+ }
+ }
+
+ // The presence of a NULL handle in the list is enough to cause PROC_THREAD_ATTRIBUTE_HANDLE_LIST
+ // to treat the entire list as empty, so remove NULL handles.
+ j := 0
+ for i := range fd {
+ if fd[i] != 0 {
+ fd[j] = fd[i]
+ j++
+ }
+ }
+ fd = fd[:j]
+
// Do not accidentally inherit more than these handles.
- err = updateProcThreadAttribute(si.ProcThreadAttributeList, 0, _PROC_THREAD_ATTRIBUTE_HANDLE_LIST, unsafe.Pointer(&fd[0]), uintptr(len(fd))*unsafe.Sizeof(fd[0]), nil, nil)
- if err != nil {
- return 0, 0, err
+ if len(fd) > 0 {
+ err = updateProcThreadAttribute(si.ProcThreadAttributeList, 0, _PROC_THREAD_ATTRIBUTE_HANDLE_LIST, unsafe.Pointer(&fd[0]), uintptr(len(fd))*unsafe.Sizeof(fd[0]), nil, nil)
+ if err != nil {
+ return 0, 0, err
+ }
}
pi := new(ProcessInformation)
-
flags := sys.CreationFlags | CREATE_UNICODE_ENVIRONMENT | _EXTENDED_STARTUPINFO_PRESENT
if sys.Token != 0 {
- err = CreateProcessAsUser(sys.Token, argv0p, argvp, sys.ProcessAttributes, sys.ThreadAttributes, !sys.NoInheritHandles, flags, createEnvBlock(attr.Env), dirp, &si.StartupInfo, pi)
+ err = CreateProcessAsUser(sys.Token, argv0p, argvp, sys.ProcessAttributes, sys.ThreadAttributes, len(fd) > 0 && !sys.NoInheritHandles, flags, createEnvBlock(attr.Env), dirp, &si.StartupInfo, pi)
} else {
- err = CreateProcess(argv0p, argvp, sys.ProcessAttributes, sys.ThreadAttributes, !sys.NoInheritHandles, flags, createEnvBlock(attr.Env), dirp, &si.StartupInfo, pi)
+ err = CreateProcess(argv0p, argvp, sys.ProcessAttributes, sys.ThreadAttributes, len(fd) > 0 && !sys.NoInheritHandles, flags, createEnvBlock(attr.Env), dirp, &si.StartupInfo, pi)
}
if err != nil {
return 0, 0, err
diff --git a/src/syscall/syscall_windows.go b/src/syscall/syscall_windows.go
index fc734effbb..660179ae9e 100644
--- a/src/syscall/syscall_windows.go
+++ b/src/syscall/syscall_windows.go
@@ -198,6 +198,7 @@ func NewCallbackCDecl(fn interface{}) uintptr {
//sys FreeLibrary(handle Handle) (err error)
//sys GetProcAddress(module Handle, procname string) (proc uintptr, err error)
//sys GetVersion() (ver uint32, err error)
+//sys rtlGetNtVersionNumbers(majorVersion *uint32, minorVersion *uint32, buildNumber *uint32) = ntdll.RtlGetNtVersionNumbers
//sys formatMessage(flags uint32, msgsrc uintptr, msgid uint32, langid uint32, buf []uint16, args *byte) (n uint32, err error) = FormatMessageW
//sys ExitProcess(exitcode uint32)
//sys CreateFile(name *uint16, access uint32, mode uint32, sa *SecurityAttributes, createmode uint32, attrs uint32, templatefile int32) (handle Handle, err error) [failretval==InvalidHandle] = CreateFileW
diff --git a/src/syscall/syscall_windows_test.go b/src/syscall/syscall_windows_test.go
index ea8fa191dc..194c87805c 100644
--- a/src/syscall/syscall_windows_test.go
+++ b/src/syscall/syscall_windows_test.go
@@ -78,6 +78,7 @@ func TestTOKEN_ALL_ACCESS(t *testing.T) {
func TestStdioAreInheritable(t *testing.T) {
testenv.MustHaveGoBuild(t)
+ testenv.MustHaveCGO(t)
testenv.MustHaveExecPath(t, "gcc")
tmpdir := t.TempDir()
@@ -110,18 +111,28 @@ func main() {}
t.Fatalf("failed to build go library: %s\n%s", err, out)
}
- // run powershell script
- psscript := fmt.Sprintf(`
-hostname;
-$signature = " [DllImport("%q")] public static extern void HelloWorld(); ";
-Add-Type -MemberDefinition $signature -Name World -Namespace Hello;
-[Hello.World]::HelloWorld();
-hostname;
-`, dll)
- psscript = strings.ReplaceAll(psscript, "\n", "")
- out, err = exec.Command("powershell", "-Command", psscript).CombinedOutput()
+ // build c exe
+ const exetext = `
+#include
+#include
+int main(int argc, char *argv[])
+{
+ system("hostname");
+ ((void(*)(void))GetProcAddress(LoadLibraryA(%q), "HelloWorld"))();
+ system("hostname");
+ return 0;
+}
+`
+ exe := filepath.Join(tmpdir, "helloworld.exe")
+ cmd = exec.Command("gcc", "-o", exe, "-xc", "-")
+ cmd.Stdin = strings.NewReader(fmt.Sprintf(exetext, dll))
+ out, err = testenv.CleanCmdEnv(cmd).CombinedOutput()
if err != nil {
- t.Fatalf("Powershell command failed: %v: %v", err, string(out))
+ t.Fatalf("failed to build c executable: %s\n%s", err, out)
+ }
+ out, err = exec.Command(exe).CombinedOutput()
+ if err != nil {
+ t.Fatalf("c program execution failed: %v: %v", err, string(out))
}
hostname, err := os.Hostname()
@@ -133,6 +144,6 @@ hostname;
have = strings.ReplaceAll(have, "\r", "")
want := fmt.Sprintf("%sHello World%s", hostname, hostname)
if have != want {
- t.Fatalf("Powershell command output is wrong: got %q, want %q", have, want)
+ t.Fatalf("c program output is wrong: got %q, want %q", have, want)
}
}
diff --git a/src/syscall/zsyscall_windows.go b/src/syscall/zsyscall_windows.go
index 10d0f54e8c..7bfff16be6 100644
--- a/src/syscall/zsyscall_windows.go
+++ b/src/syscall/zsyscall_windows.go
@@ -43,6 +43,7 @@ var (
modkernel32 = NewLazyDLL(sysdll.Add("kernel32.dll"))
modmswsock = NewLazyDLL(sysdll.Add("mswsock.dll"))
modnetapi32 = NewLazyDLL(sysdll.Add("netapi32.dll"))
+ modntdll = NewLazyDLL(sysdll.Add("ntdll.dll"))
modsecur32 = NewLazyDLL(sysdll.Add("secur32.dll"))
modshell32 = NewLazyDLL(sysdll.Add("shell32.dll"))
moduserenv = NewLazyDLL(sysdll.Add("userenv.dll"))
@@ -167,6 +168,7 @@ var (
procNetApiBufferFree = modnetapi32.NewProc("NetApiBufferFree")
procNetGetJoinInformation = modnetapi32.NewProc("NetGetJoinInformation")
procNetUserGetInfo = modnetapi32.NewProc("NetUserGetInfo")
+ procRtlGetNtVersionNumbers = modntdll.NewProc("RtlGetNtVersionNumbers")
procGetUserNameExW = modsecur32.NewProc("GetUserNameExW")
procTranslateNameW = modsecur32.NewProc("TranslateNameW")
procCommandLineToArgvW = modshell32.NewProc("CommandLineToArgvW")
@@ -1213,6 +1215,11 @@ func NetUserGetInfo(serverName *uint16, userName *uint16, level uint32, buf **by
return
}
+func rtlGetNtVersionNumbers(majorVersion *uint32, minorVersion *uint32, buildNumber *uint32) {
+ Syscall(procRtlGetNtVersionNumbers.Addr(), 3, uintptr(unsafe.Pointer(majorVersion)), uintptr(unsafe.Pointer(minorVersion)), uintptr(unsafe.Pointer(buildNumber)))
+ return
+}
+
func GetUserNameEx(nameFormat uint32, nameBuffre *uint16, nSize *uint32) (err error) {
r1, _, e1 := Syscall(procGetUserNameExW.Addr(), 3, uintptr(nameFormat), uintptr(unsafe.Pointer(nameBuffre)), uintptr(unsafe.Pointer(nSize)))
if r1&0xff == 0 {
diff --git a/src/testing/fuzz.go b/src/testing/fuzz.go
index d62eb55dec..6f5cdcc389 100644
--- a/src/testing/fuzz.go
+++ b/src/testing/fuzz.go
@@ -173,6 +173,11 @@ func (f *F) Helper() {
}
}
+// Setenv is not supported since fuzzing runs in parallel.
+func (f *F) Setenv(key, value string) {
+ panic("testing: f.Setenv is not supported")
+}
+
// Skip is equivalent to Log followed by SkipNow.
func (f *F) Skip(args ...interface{}) {
if f.inFuzzFn {
diff --git a/src/testing/testing.go b/src/testing/testing.go
index 581271e748..4bf5685a07 100644
--- a/src/testing/testing.go
+++ b/src/testing/testing.go
@@ -297,6 +297,8 @@ import (
"sync"
"sync/atomic"
"time"
+ "unicode"
+ "unicode/utf8"
)
var initRan bool
@@ -978,11 +980,6 @@ func (c *common) Cleanup(f func()) {
c.cleanups = append(c.cleanups, fn)
}
-var tempDirReplacer struct {
- sync.Once
- r *strings.Replacer
-}
-
// TempDir returns a temporary directory for the test to use.
// The directory is automatically removed by Cleanup when the test and
// all its subtests complete.
@@ -1006,13 +1003,26 @@ func (c *common) TempDir() string {
if nonExistent {
c.Helper()
- // os.MkdirTemp doesn't like path separators in its pattern,
- // so mangle the name to accommodate subtests.
- tempDirReplacer.Do(func() {
- tempDirReplacer.r = strings.NewReplacer("/", "_", "\\", "_", ":", "_")
- })
- pattern := tempDirReplacer.r.Replace(c.Name())
-
+ // Drop unusual characters (such as path separators or
+ // characters interacting with globs) from the directory name to
+ // avoid surprising os.MkdirTemp behavior.
+ mapper := func(r rune) rune {
+ if r < utf8.RuneSelf {
+ const allowed = "!#$%&()+,-.=@^_{}~ "
+ if '0' <= r && r <= '9' ||
+ 'a' <= r && r <= 'z' ||
+ 'A' <= r && r <= 'Z' {
+ return r
+ }
+ if strings.ContainsRune(allowed, r) {
+ return r
+ }
+ } else if unicode.IsLetter(r) || unicode.IsNumber(r) {
+ return r
+ }
+ return -1
+ }
+ pattern := strings.Map(mapper, c.Name())
c.tempDir, c.tempDirErr = os.MkdirTemp("", pattern)
if c.tempDirErr == nil {
c.Cleanup(func() {
diff --git a/src/testing/testing_test.go b/src/testing/testing_test.go
index 55a4df4739..08ae23991f 100644
--- a/src/testing/testing_test.go
+++ b/src/testing/testing_test.go
@@ -58,6 +58,9 @@ func TestTempDir(t *testing.T) {
t.Run("test:subtest", testTempDir)
t.Run("test/..", testTempDir)
t.Run("../test", testTempDir)
+ t.Run("test[]", testTempDir)
+ t.Run("test*", testTempDir)
+ t.Run("äöüéè", testTempDir)
}
func testTempDir(t *testing.T) {
@@ -74,7 +77,7 @@ func testTempDir(t *testing.T) {
if err != nil {
t.Fatal(err)
}
- t.Errorf("directory %q stil exists: %v, isDir=%v", dir, fi, fi.IsDir())
+ t.Errorf("directory %q still exists: %v, isDir=%v", dir, fi, fi.IsDir())
default:
if !t.Failed() {
t.Fatal("never received dir channel")
@@ -108,6 +111,11 @@ func testTempDir(t *testing.T) {
if len(files) > 0 {
t.Errorf("unexpected %d files in TempDir: %v", len(files), files)
}
+
+ glob := filepath.Join(dir, "*.txt")
+ if _, err := filepath.Glob(glob); err != nil {
+ t.Error(err)
+ }
}
func TestSetenv(t *testing.T) {
diff --git a/src/time/format.go b/src/time/format.go
index 6040ed5aeb..bb173a21c2 100644
--- a/src/time/format.go
+++ b/src/time/format.go
@@ -751,8 +751,11 @@ type ParseError struct {
// These are borrowed from unicode/utf8 and strconv and replicate behavior in
// that package, since we can't take a dependency on either.
-const runeSelf = 0x80
-const lowerhex = "0123456789abcdef"
+const (
+ lowerhex = "0123456789abcdef"
+ runeSelf = 0x80
+ runeError = '\uFFFD'
+)
func quote(s string) string {
buf := make([]byte, 1, len(s)+2) // slice will be at least len(s) + quotes
@@ -765,7 +768,16 @@ func quote(s string) string {
// reproduce strconv.Quote's behavior with full fidelity but
// given how rarely we expect to hit these edge cases, speed and
// conciseness are better.
- for j := 0; j < len(string(c)) && j < len(s); j++ {
+ var width int
+ if c == runeError {
+ width = 1
+ if i+2 < len(s) && s[i:i+3] == string(runeError) {
+ width = 3
+ }
+ } else {
+ width = len(string(c))
+ }
+ for j := 0; j < width; j++ {
buf = append(buf, `\x`...)
buf = append(buf, lowerhex[s[i+j]>>4])
buf = append(buf, lowerhex[s[i+j]&0xF])
diff --git a/src/time/time.go b/src/time/time.go
index cd756bbf5f..1cf1e2bbf6 100644
--- a/src/time/time.go
+++ b/src/time/time.go
@@ -1340,7 +1340,7 @@ func UnixMicro(usec int64) Time {
}
// IsDST reports whether the time in the configured location is in Daylight Savings Time.
-func (t *Time) IsDST() bool {
+func (t Time) IsDST() bool {
_, _, _, _, isDST := t.loc.lookup(t.Unix())
return isDST
}
diff --git a/src/time/time_test.go b/src/time/time_test.go
index f272bbd558..cea5f2d3f5 100644
--- a/src/time/time_test.go
+++ b/src/time/time_test.go
@@ -917,6 +917,11 @@ var parseDurationErrorTests = []struct {
{".s", `".s"`},
{"+.s", `"+.s"`},
{"1d", `"1d"`},
+ {"\x85\x85", `"\x85\x85"`},
+ {"\xffff", `"\xffff"`},
+ {"hello \xffff world", `"hello \xffff world"`},
+ {"\uFFFD", `"\xef\xbf\xbd"`}, // utf8.RuneError
+ {"\uFFFD hello \uFFFD world", `"\xef\xbf\xbd hello \xef\xbf\xbd world"`}, // utf8.RuneError
// overflow
{"9223372036854775810ns", `"9223372036854775810ns"`},
{"9223372036854775808ns", `"9223372036854775808ns"`},
diff --git a/test/declbad.go b/test/declbad.go
index 728eceb7f1..b978652a2b 100644
--- a/test/declbad.go
+++ b/test/declbad.go
@@ -23,13 +23,13 @@ func main() {
{
// change of type for f
i, f, s := f3()
- f, g, t := f3() // ERROR "redeclared|cannot assign|incompatible"
+ f, g, t := f3() // ERROR "redeclared|cannot assign|incompatible|cannot use"
_, _, _, _, _ = i, f, s, g, t
}
{
// change of type for i
i, f, s := f3()
- j, i, t := f3() // ERROR "redeclared|cannot assign|incompatible"
+ j, i, t := f3() // ERROR "redeclared|cannot assign|incompatible|cannot use"
_, _, _, _, _ = i, f, s, j, t
}
{
diff --git a/test/fixedbugs/issue46386.go b/test/fixedbugs/issue46386.go
new file mode 100644
index 0000000000..89dea8abf3
--- /dev/null
+++ b/test/fixedbugs/issue46386.go
@@ -0,0 +1,32 @@
+// compile -p=main
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+type I interface {
+ M() interface{}
+}
+
+type S1 struct{}
+
+func (S1) M() interface{} {
+ return nil
+}
+
+type EI interface{}
+
+type S struct{}
+
+func (S) M(as interface{ I }) {}
+
+func f() interface{ EI } {
+ return &S1{}
+}
+
+func main() {
+ var i interface{ I }
+ (&S{}).M(i)
+}
diff --git a/test/fixedbugs/issue46525.go b/test/fixedbugs/issue46525.go
new file mode 100644
index 0000000000..164e1473ce
--- /dev/null
+++ b/test/fixedbugs/issue46525.go
@@ -0,0 +1,14 @@
+// errorcheck -lang=go1.16
+
+// Copyright 2021 The Go Authors. All rights reserved. Use of this
+// source code is governed by a BSD-style license that can be found in
+// the LICENSE file.
+
+package p
+
+import "unsafe"
+
+func main() {
+ _ = unsafe.Add(unsafe.Pointer(nil), 0) // ERROR "unsafe.Add requires go1.17 or later"
+ _ = unsafe.Slice(new(byte), 1) // ERROR "unsafe.Slice requires go1.17 or later"
+}
diff --git a/test/fixedbugs/issue46653.dir/bad/bad.go b/test/fixedbugs/issue46653.dir/bad/bad.go
new file mode 100644
index 0000000000..c1611b8347
--- /dev/null
+++ b/test/fixedbugs/issue46653.dir/bad/bad.go
@@ -0,0 +1,64 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package a
+
+func Bad() {
+ m := make(map[int64]A)
+ a := m[0]
+ if len(a.B.C1.D2.E2.F1) != 0 ||
+ len(a.B.C1.D2.E2.F2) != 0 ||
+ len(a.B.C1.D2.E2.F3) != 0 ||
+ len(a.B.C1.D2.E2.F4) != 0 ||
+ len(a.B.C1.D2.E2.F5) != 0 ||
+ len(a.B.C1.D2.E2.F6) != 0 ||
+ len(a.B.C1.D2.E2.F7) != 0 ||
+ len(a.B.C1.D2.E2.F8) != 0 ||
+ len(a.B.C1.D2.E2.F9) != 0 ||
+ len(a.B.C1.D2.E2.F10) != 0 ||
+ len(a.B.C1.D2.E2.F11) != 0 ||
+ len(a.B.C1.D2.E2.F16) != 0 {
+ panic("bad")
+ }
+}
+
+type A struct {
+ B
+}
+
+type B struct {
+ C1 C
+ C2 C
+}
+
+type C struct {
+ D1 D
+ D2 D
+}
+
+type D struct {
+ E1 E
+ E2 E
+ E3 E
+ E4 E
+}
+
+type E struct {
+ F1 string
+ F2 string
+ F3 string
+ F4 string
+ F5 string
+ F6 string
+ F7 string
+ F8 string
+ F9 string
+ F10 string
+ F11 string
+ F12 string
+ F13 string
+ F14 string
+ F15 string
+ F16 string
+}
diff --git a/test/fixedbugs/issue46653.dir/main.go b/test/fixedbugs/issue46653.dir/main.go
new file mode 100644
index 0000000000..e2a96e54ec
--- /dev/null
+++ b/test/fixedbugs/issue46653.dir/main.go
@@ -0,0 +1,27 @@
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import (
+ bad "issue46653.dir/bad"
+)
+
+func main() {
+ bad.Bad()
+}
+
+func neverCalled() L {
+ m := make(map[string]L)
+ return m[""]
+}
+
+type L struct {
+ A Data
+ B Data
+}
+
+type Data struct {
+ F1 [22][]string
+}
diff --git a/test/fixedbugs/issue46653.go b/test/fixedbugs/issue46653.go
new file mode 100644
index 0000000000..e6283b1de5
--- /dev/null
+++ b/test/fixedbugs/issue46653.go
@@ -0,0 +1,10 @@
+// runindir
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// Test to verify compiler and linker handling of multiple
+// competing map.zero symbol definitions.
+
+package ignored
diff --git a/test/fixedbugs/issue46720.go b/test/fixedbugs/issue46720.go
new file mode 100644
index 0000000000..3b0151ae84
--- /dev/null
+++ b/test/fixedbugs/issue46720.go
@@ -0,0 +1,15 @@
+// compile
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+func f() {
+ nonce := make([]byte, 24)
+ g((*[24]byte)(nonce))
+}
+
+//go:noinline
+func g(*[24]byte) {}
diff --git a/test/fixedbugs/issue46725.go b/test/fixedbugs/issue46725.go
new file mode 100644
index 0000000000..29799c7d7e
--- /dev/null
+++ b/test/fixedbugs/issue46725.go
@@ -0,0 +1,48 @@
+// run
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import "runtime"
+
+type T [4]int
+
+//go:noinline
+func g(x []*T) ([]*T, []*T) { return x, x }
+
+func main() {
+ const Jenny = 8675309
+ s := [10]*T{{Jenny}}
+
+ done := make(chan struct{})
+ runtime.SetFinalizer(s[0], func(p *T) { close(done) })
+
+ var h, _ interface{} = g(s[:])
+
+ if wait(done) {
+ panic("GC'd early")
+ }
+
+ if h.([]*T)[0][0] != Jenny {
+ panic("lost Jenny's number")
+ }
+
+ if !wait(done) {
+ panic("never GC'd")
+ }
+}
+
+func wait(done <-chan struct{}) bool {
+ for i := 0; i < 10; i++ {
+ runtime.GC()
+ select {
+ case <-done:
+ return true
+ default:
+ }
+ }
+ return false
+}
diff --git a/test/fixedbugs/issue46749.go b/test/fixedbugs/issue46749.go
new file mode 100644
index 0000000000..63ed19795e
--- /dev/null
+++ b/test/fixedbugs/issue46749.go
@@ -0,0 +1,37 @@
+// errorcheck
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+var s string
+var b bool
+var i int
+var iface interface{}
+
+var (
+ _ = "" + b // ERROR "invalid operation.*mismatched types.*untyped string and bool"
+ _ = "" + i // ERROR "invalid operation.*mismatched types.*untyped string and int"
+ _ = "" + nil // ERROR "invalid operation.*mismatched types.*untyped string and nil"
+)
+
+var (
+ _ = s + false // ERROR "invalid operation.*mismatched types.*string and untyped bool"
+ _ = s + 1 // ERROR "invalid operation.*mismatched types.*string and untyped int"
+ _ = s + nil // ERROR "invalid operation.*mismatched types.*string and nil"
+)
+
+var (
+ _ = "" + false // ERROR "invalid operation.*mismatched types.*untyped string and untyped bool"
+ _ = "" + 1 // ERROR "invalid operation.*mismatched types.*untyped string and untyped int"
+)
+
+var (
+ _ = b + 1 // ERROR "invalid operation.*mismatched types.*bool and untyped int"
+ _ = i + false // ERROR "invalid operation.*mismatched types.*int and untyped bool"
+ _ = iface + 1 // ERROR "invalid operation.*mismatched types.*interface {} and int"
+ _ = iface + 1.0 // ERROR "invalid operation.*mismatched types.*interface {} and float64"
+ _ = iface + false // ERROR "invalid operation.*mismatched types.*interface {} and bool"
+)
diff --git a/test/fixedbugs/issue46907.go b/test/fixedbugs/issue46907.go
new file mode 100644
index 0000000000..bd82f4f2b1
--- /dev/null
+++ b/test/fixedbugs/issue46907.go
@@ -0,0 +1,11 @@
+// compile
+
+// Copyright 2021 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+func f(b []byte) []byte {
+ return (*[32]byte)(b[:32])[:]
+}
diff --git a/test/run.go b/test/run.go
index 5e60de7624..d7f5d02391 100644
--- a/test/run.go
+++ b/test/run.go
@@ -2002,4 +2002,5 @@ var excluded = map[string]bool{
"fixedbugs/issue7525c.go": true, // types2 reports init cycle error on different line - ok otherwise
"fixedbugs/issue7525d.go": true, // types2 reports init cycle error on different line - ok otherwise
"fixedbugs/issue7525e.go": true, // types2 reports init cycle error on different line - ok otherwise
+ "fixedbugs/issue46749.go": true, // types2 reports can not convert error instead of type mismatched
}