- TODO: complete this section + +
+ There is only one small change to the language, + a very small correction + to the scope of type parameters in method declarations. + Existing programs are unaffected.
+ +
+ The Go memory model has been revised to align Go with
+ the memory model used by C, C++, Java, JavaScript, Rust, and Swift.
+ Go only provides sequentially consistent atomics, not any of the more relaxed forms found in other languages.
+ Along with the memory model update,
+ Go 1.19 introduces new types in the sync/atomic package
+ that make it easier to use atomic values, such as
+ atomic.Int64
+ and
+ atomic.Pointer[T].
+
- TODO: complete this section, or delete if not needed + +
+ Go 1.19 adds support for the Loongson 64-bit architecture LoongArch
+ on Linux (GOOS=linux, GOARCH=loong64).
+ The riscv64 port now supports passing function arguments
+ and result using registers. Benchmarking shows typical performance
+ improvements of 10% or more on riscv64.
+
- TODO: complete this section, or delete if not needed + +
+Go 1.19 adds support for links, lists, and clearer headings in doc comments.
+As part of this change, gofmt
+now reformats doc comments to make their rendered meaning clearer.
+See “Go Doc Comments”
+for syntax details and descriptions of common mistakes now highlighted by gofmt.
+As another part of this change, the new package go/doc/comment
+provides parsing and reformatting of doc comments
+as well as support for rendering them to HTML, Markdown, and text.
unix build constraint
+ The build constraint unix is now recognized
+ in //go:build lines. The constraint is satisfied
+ if the target operating system, also known as GOOS, is
+ a Unix or Unix-like system. For the 1.19 release it is satisfied
+ if GOOS is one of
+ aix, android, darwin,
+ dragonfly, freebsd, hurd,
+ illumos, ios, linux,
+ netbsd, openbsd, or solaris.
+ In future releases the unix constraint may match
+ additional newly supported operating systems.
+
- TODO: complete this section. -
@@ -63,31 +118,61 @@ Do not send CLs removing the interior tags from such phrases.
and GOGCCFLAGS variables it reports.
+ The go command now caches information necessary to load some modules,
+ which should result in a speed-up of some go list invocations.
+
unix build constraint
- The build constraint unix is now recognized
- in //go:build lines. The constraint is satisfied
- if the target operating system, also known as GOOS, is
- a Unix or Unix-like system. For the 1.19 release it is satisfied
- if GOOS is one of
- aix, android, darwin,
- dragonfly, freebsd, hurd,
- illumos, ios, linux,
- netbsd, openbsd, or solaris.
- In future releases the unix constraint may match
- additional newly supported operating systems.
+
+ The vet checker “errorsas” now reports when
+ errors.As is called
+ with a second argument of type *error,
+ a common mistake.
- TODO: soft memory limit +
+ The runtime now includes support for a soft memory limit. This memory limit
+ includes the Go heap and all other memory managed by the runtime, and
+ excludes external memory sources such as mappings of the binary itself,
+ memory managed in other languages, and memory held by the operating system on
+ behalf of the Go program. This limit may be managed via
+ runtime/debug.SetMemoryLimit
+ or the equivalent
+ GOMEMLIMIT
+ environment variable. The limit works in conjunction with
+ runtime/debug.SetGCPercent
+ / GOGC,
+ and will be respected even if GOGC=off, allowing Go programs to
+ always make maximal use of their memory limit, improving resource efficiency
+ in some cases. See the GC guide for
+ a detailed guide explaining the soft memory limit in more detail, as well as
+ a variety of common use-cases and scenarios. Please note that small memory
+ limits, on the order of tens of megabytes or less, are less likely to be
+ respected due to external latency factors, such as OS scheduling. See
+ issue 52433 for more details. Larger
+ memory limits, on the order of hundreds of megabytes or more, are stable and
+ production-ready.
+
+ In order to limit the effects of GC thrashing when the program's live heap
+ size approaches the soft memory limit, the Go runtime also attempts to limit
+ total GC CPU utilization to 50%, excluding idle time, choosing to use more
+ memory over preventing application progress. In practice, we expect this limit
+ to only play a role in exceptional cases, and the new
+ runtime metric
+ /gc/limiter/last-enabled:gc-cycle reports when this last
+ occurred.
- TODO: idle mark workers + The runtime now schedules many fewer GC worker goroutines on idle operating + system threads when the application is idle enough to force a periodic GC + cycle.
@@ -97,13 +182,18 @@ Do not send CLs removing the interior tags from such phrases. space on below-average goroutines.
-+
On Unix operating systems, Go programs that import package
os now automatically increase the open file limit
- (RLIMIT_NOFILE) to the maximum allowed value. Programs that need
- a lower limit (for compatibility with select, for example) can
- set the limit back as needed, or lower the hard limit prior to starting the
- Go program.
+ (RLIMIT_NOFILE) to the maximum allowed value;
+ that is, they change the soft limit to match the hard limit.
+ This corrects artificially low limits set on some systems for compatibility with very old C programs using the
+ select system call.
+ Go programs are not helped by that limit, and instead even simple programs like gofmt
+ often ran out of file descriptors on such systems when processing many files in parallel.
+ One impact of this change is that Go programs that in turn execute very old C programs in child processes
+ may run those programs with too high a limit.
+ This can be corrected by setting the hard limit before invoking the Go program.
@@ -121,6 +211,11 @@ Do not send CLs removing the interior tags from such phrases. functionality.
++ The address sanitizer support added in Go 1.18 + now handles function arguments and global variables more precisely. +
+
@@ -131,23 +226,34 @@ Do not send CLs removing the interior tags from such phrases.
on the order of 20% faster.
(GOARCH=amd64 and GOARCH=arm64 only)
- TODO: https://go.dev/cl/402374: enable regabi on riscv64 by default +
+ The Go compiler now requires the -p=importpath flag to
+ build a linkable object file. This is already supplied by
+ the go command and by Bazel. Any other build systems
+ that invoke the Go compiler directly will need to make sure they
+ pass this flag as well.
- TODO: https://go.dev/cl/391014: The Go compiler now requires the -p=importpath flag, which is already supplied by the go command and by Bazel. Any other build systems that invoke the Go compiler directly will need to make sure they pass this flag as well in order to use Go 1.19.: cmd/compile: require -p flag -
-- TODO: complete this section, or delete if not needed + +
+ Like the compiler, the assembler now requires the
+ -p=importpath flag to build a linkable object file.
+ This is already supplied by the go command. Any other
+ build systems that invoke the Go assembler directly will need to
+ make sure they pass this flag as well.
- TODO: complete this section, or delete if not needed +
+ On ELF platforms, the linker now emits compressed DWARF sections in
+ the standard gABI format (SHF_COMPRESSED), instead of
+ the legacy .zdebug format.
The sync/atomic package defines new atomic types
Bool,
@@ -163,17 +269,30 @@ Do not send CLs removing the interior tags from such phrases.
the need to convert to
unsafe.Pointer at call sites.
Int64 and
- Uint64 automatically
- receive 64-bit alignment on ARM, 386, and 32-bit MIPS required for 64-bit
- atomics on these systems.
+ Uint64 are
+ automatically aligned to 64-bit boundaries in structs and allocated data,
+ even on 32-bit systems.
- TODO: https://go.dev/issue/51940: all: move dev.boringcrypto into main branch behind GOEXPERIMENT +
+
+ Command and
+ LookPath no longer
+ allow results from a PATH search to be found relative to the current directory.
+ This removes a common source of security problems
+ but may also break existing programs that depend on using, say, exec.Command("prog")
+ to run a binary named prog (or, on Windows, prog.exe) in the current directory.
+ See the os/exec package documentation for
+ information about how best to update such programs.
- TODO: complete this section +
+ On Windows, Command and LookPath now respect the
+ NoDefaultCurrentDirectoryInExePath
+ environment variable, making it possible to disable
+ the default implicit search of “.” in PATH lookups on Windows systems.
- TODO: complete this section + There are also various performance improvements, not enumerated here.
- TODO: https://go.dev/cl/387976: permit zip files to have prefixes
+ Reader
+ now ignores non-ZIP data at the start of a ZIP file, matching most other implementations.
+ This is necessary to read some Java JAR files, among other uses.
- TODO: https://go.dev/cl/370894: batch and buffer calls to getrandom/getentropy +
+ Read no longer buffers
+ random data obtained from the operating system between calls.
- TODO: https://go.dev/cl/375215: use fast key erasure RNG on plan9 instead of ANSI X9.31 -
- -
- TODO: https://go.dev/cl/390038: remove all buffering
+ On Plan 9, Read has been reimplemented, replacing the ANSI
+ X9.31 algorithm with fast key erasure.
+
The tls10default GODEBUG option has been
removed. It is still possible to enable TLS 1.0 client-side by setting
- Config.MinVersion.
+ Config.MinVersion.
+
+ The TLS server and client now reject duplicate extensions in TLS + handshakes, as required by RFC 5246, Section 7.4.1.4 and RFC 8446, Section + 4.2.
- TODO: https://go.dev/cl/285872: disable signing with MD5WithRSA
+ CreateCertificate
+ no longer supports creating certificates with SignatureAlgorithm
+ set to MD5WithRSA.
+
+ CreateCertificate no longer accepts negative serial numbers.
+
+ ParseCertificate
+ and ParseCertificateRequest
+ now reject certificates and CSRs which contain duplicate extensions.
+
+ The new CertPool.Clone
+ and CertPool.Equal
+ methods allow cloning a CertPool and checking the equality of two
+ CertPools respectively.
+
+ The new function ParseRevocationList
+ provides a faster, safer to use CRL parser which returns a
+ RevocationList.
+ To support this addition, RevocationList adds new fields
+ RawIssuer, Signature,
+ AuthorityKeyId, and Extensions.
+
+ The new method RevocationList.CheckSignatureFrom
+ checks that the signature on a CRL is a valid signature from a
+ Certificate.
+
+ With the new CRL functionality, the existing functions
+ ParseCRL and
+ ParseDERCRL are deprecated.
+ Additionally the method Certificate.CheckCRLSignature
+ is deprecated.
+
+ When building paths, Certificate.Verify
+ now considers certificates to be equal when the subjects, public keys, and SANs
+ are all equal. Before, it required byte-for-byte equality.
+ The types CertificateList and
+ TBSCertificateList
+ have been deprecated. The new crypto/x509 CRL functionality
+ should be used instead.
+
+ The new EM_LONGARCH and R_LARCH_* constants
+ support the loong64 port.
+
+ The new File.COFFSymbolReadSectionDefAux
+ method, which returns a COFFSymbolAuxFormat5,
+ provides access to COMDAT information in PE file sections.
+ These are supported by new IMAGE_COMDAT_* and IMAGE_SCN_* constants.
+
- TODO: https://go.dev/cl/386017: add AppendByteOrder +
+ The new interface
+ AppendByteOrder
+ provides efficient methods for appending a uint16, uint32, or uint64
+ to a byte slice.
+ BigEndian and
+ LittleEndian now implement this interface.
+
+ Similarly, the new functions
+ AppendUvarint and
+ AppendVarint
+ are efficient appending versions of
+ PutUvarint and
+ PutVarint.
- TODO: https://go.dev/cl/405675: add Reader.InputOffset method +
+ The new method
+ Reader.InputOffset
+ reports the reader's current input position as a byte offset,
+ analogous to encoding/json's
+ Decoder.InputOffset.
+ The new method
+ Decoder.InputPos
+ reports the reader's current input position as a line and column,
+ analogous to encoding/csv's
+ Decoder.FieldPos.
+
- TODO: https://go.dev/cl/313329: add TextVar function +
+ The new function
+ TextVar
+ defines a flag with a value implementing
+ encoding.TextUnmarshaler,
+ allowing command-line flag variables to have types such as
+ big.Int,
+ netip.Addr, and
+ time.Time.
- TODO: https://go.dev/cl/406177: add Append, Appendln, Appendf +
+ The new functions
+ Append,
+ Appendf, and
+ Appendln
+ append formatted data to byte slices.
- TODO: https://go.dev/cl/403696: parser to accept ~x as unary expression
+ The parser now recognizes ~x as a unary expression with operator
+ token.TILDE,
+ allowing better error recovery when a type constraint such as ~int is used in an incorrect context.
+ The new methods Func.Origin
+ and Var.Origin return the
+ corresponding Object of the
+ generic type for synthetic Func
+ and Var objects created during type
+ instantiation.
+
+ It is no longer possible to produce an infinite number of distinct-but-identical
+ Named type instantiations via
+ recursive calls to
+ Named.Underlying or
+ Named.Method.
+
+ The new functions
+ Bytes
+ and
+ String
+ provide an efficient way hash a single byte slice or string.
+ They are equivalent to using the more general
+ Hash
+ with a single write, but they avoid setup overhead for small inputs.
+
+ The type FuncMap
+ is now an alias for
+ text/template's FuncMap
+ instead of its own named type.
+ This allows writing code that operates on a FuncMap from either setting.
+
- Draw with the Src operator preserves
+ Draw with the
+ Src operator preserves
non-premultiplied-alpha colors when destination and source images are
- both *image.NRGBA (or both *image.NRGBA64).
+ both image.NRGBA
+ or both image.NRGBA64.
This reverts a behavior change accidentally introduced by a Go 1.18
- library optimization, to match the behavior in Go 1.17 and earlier.
+ library optimization; the code now matches the behavior in Go 1.17 and earlier.
- TODO: https://go.dev/cl/396215: add an Err field to LimitedReader +
+ NopCloser's result now implements
+ WriterTo
+ whenever its input does.
- TODO: https://go.dev/cl/400236: NopCloser forward WriterTo implementations if the reader supports it +
+ MultiReader's result now implements
+ WriterTo unconditionally.
+ If any underlying reader does not implement WriterTo,
+ it is simulated appropriately.
text/javascript; charset=utf-8.
Applications that expect text/plain on Windows must
now explicitly call
- AddExtensionType.
+ AddExtensionType.
@@ -322,7 +617,7 @@ Do not send CLs removing the interior tags from such phrases.
issue tracker.
- +
When a net package function or method returns an "I/O timeout"
error, the error will now satisfy errors.Is(err,
context.DeadlineExceeded). When a net package function
@@ -333,10 +628,23 @@ Do not send CLs removing the interior tags from such phrases.
package function or method to return an error, while preserving
backward compatibility for error messages.
- TODO: https://go.dev/cl/400654: permit use of Resolver.PreferGo, netgo on Windows and Plan 9 + +
+ Resolver.PreferGo
+ is now implemented on Windows and Plan 9. It previously only worked on Unix
+ platforms. Combined with
+ Dialer.Resolver and
+ Resolver.Dial, it's now
+ possible to write portable programs and be in control of all DNS name lookups
+ when dialing.
+
+ The net package now has initial support for the netgo
+ build tag on Windows. When used, the package uses the Go DNS client (as used
+ by Resolver.PreferGo) instead of asking Windows for
+ DNS results. The upstream DNS server it discovers from Windows
+ may not yet be correct with complex system network configurations, however.
- TODO: https://go.dev/cl/269997: allow sending 1xx responses
+ ResponseWriter.WriteHeader
+ now supports sending user-defined 1xx informational headers.
+
+ The io.ReadCloser returned by
+ MaxBytesReader
+ will now return the defined error type
+ MaxBytesError
+ when its read limit is exceeded.
+
+ The HTTP client will handle a 3xx response without a
+ Location header by returning it to the caller,
+ rather than treating it as an error.
- TODO: https://go.dev/cl/374654: add JoinPath, URL.JoinPath
+ The new
+ JoinPath
+ function and
+ URL.JoinPath
+ method create a new URL by joining a list of path
+ elements.
+ The URL type now distinguishes between URLs with no
+ authority and URLs with an empty authority. For example,
+ http:///path has an empty authority (host),
+ while http:/path has none.
+
+ The new URL field
+ OmitHost is set to true when a
+ URL has an empty authority.
+
- TODO: https://go.dev/cl/392415: raise open file rlimit at startup -
-
- An exec.Cmd with a non-empty Dir and a
- nil Env now implicitly sets the PWD environment
+
+ A Cmd with a non-empty Dir field
+ and nil Env now implicitly sets the PWD environment
variable for the subprocess to match Dir.
- The new method (*exec.Cmd).Environ reports the
+ The new method Cmd.Environ reports the
environment that would be used to run the command, including the
- aforementioned PWD variable.
+ implicitly set PWD variable.
- The method Value.Bytes now accepts addressable arrays in addition to slices.
+
+ The method Value.Bytes
+ now accepts addressable arrays in addition to slices.
- The methods Value.Len and Value.Cap now successfully operate on a pointer to an array and return the length of that array, to match what the builtin len and cap functions do.
+ The methods Value.Len
+ and Value.Cap
+ now successfully operate on a pointer to an array and return the length of that array,
+ to match what the builtin
+ len and cap functions do.
+ Go 1.18 release candidate 1, Go 1.17.8, and Go 1.16.15 included a security fix
+ to the regular expression parser, making it reject very deeply nested expressions.
+ Because Go patch releases do not introduce new API,
+ the parser returned syntax.ErrInternalError in this case.
+ Go 1.19 adds a more specific error, syntax.ErrNestingDepth,
+ which the parser now returns instead.
+
- The GOROOT function now returns the empty string
+ The GOROOT function now returns the empty string
(instead of "go") when the binary was built with
the -trimpath flag set and the GOROOT
variable is not set in the process environment.
@@ -406,20 +755,27 @@ Do not send CLs removing the interior tags from such phrases.
- The new /sched/gomaxprocs:threads metric reports the current
- runtime.GOMAXPROCS value.
+ The new /sched/gomaxprocs:threads
+ metric reports
+ the current
+ runtime.GOMAXPROCS
+ value.
- The new /cgo/go-to-c-calls:calls metric reports the total
- number of calls made from Go to C. This metric is identical to the runtime.NumCgoCall
+ The new /cgo/go-to-c-calls:calls
+ metric
+ reports the total number of calls made from Go to C. This metric is
+ identical to the
+ runtime.NumCgoCall
function.
- The new /gc/limiter/last-enabled:gc-cycle metric reports the
- last GC cycle when the GC CPU limiter was enabled.
+ The new /gc/limiter/last-enabled:gc-cycle
+ metric
+ reports the last GC cycle when the GC CPU limiter was enabled. See the
+ runtime notes for details about the GC CPU limiter.
The race detector has been upgraded to use thread sanitizer - version v3. -
windows/amd64
+ and openbsd/amd64, which remain on v2.
+ Compared to v2, it is now typically 1.5x to 2x faster, uses half
+ as much memory, and it supports an unlimited number of
+ goroutines.
- The race detector is now supported on S390.
+ The race detector is now supported on GOARCH=s390x.
+
+ Race detector support for openbsd/amd64 has been
+ removed from thread sanitizer upstream, so it is unlikely to
+ ever be updated from v2.
- When used together with the - CPU profiler, the - execution trace includes CPU profile samples. + When tracing and the + CPU profiler are + enabled simultaneously, the execution trace includes CPU profile + samples as instantaneous events.
+ The new function + Find + is like + Search + but often easier to use: it returns an additional boolean reporting whether an equal value was found. +
+ @@ -489,8 +854,9 @@ Do not send CLs removing the interior tags from such phrases.
Quote
- and related functions now quote the rune 007F as \x7f,
- not \u007f.
+ and related functions now quote the rune U+007F as \x7f,
+ not \u007f,
+ for consistency with other ASCII values.
- TODO: https://go.dev/cl/393515: add Duration.Abs +
+ The new method
+ Duration.Abs
+ provides a convenient and safe way to take the absolute value of a duration,
+ converting −2⁶³ to 2⁶³−1.
+ (This boundary case can happen as the result of subtracting a recent time from the zero time.)
+
+ The new method
+ Time.ZoneBounds
+ returns the start and end times of the time zone in effect at a given time.
+ It can be used in a loop to enumerate all the known time zone transitions at a given location.
The Go memory model specifies the conditions under which @@ -22,7 +19,7 @@ observe values produced by writes to the same variable in a different goroutine.
-Programs that modify data being simultaneously accessed by multiple goroutines @@ -44,90 +41,237 @@ you are being too clever. Don't be clever.
-
-Within a single goroutine, reads and writes must behave
-as if they executed in the order specified by the program.
-That is, compilers and processors may reorder the reads and writes
-executed within a single goroutine only when the reordering
-does not change the behavior within that goroutine
-as defined by the language specification.
-Because of this reordering, the execution order observed
-by one goroutine may differ from the order perceived
-by another. For example, if one goroutine
-executes a = 1; b = 2;, another might observe
-the updated value of b before the updated value of a.
+Go approaches its memory model in much the same way as the rest of the language,
+aiming to keep the semantics simple, understandable, and useful.
+This section gives a general overview of the approach and should suffice for most programmers.
+The memory model is specified more formally in the next section.
-To specify the requirements of reads and writes, we define -happens before, a partial order on the execution -of memory operations in a Go program. If event e1 happens -before event e2, then we say that e2 happens after e1. -Also, if e1 does not happen before e2 and does not happen -after e2, then we say that e1 and e2 happen concurrently. -
- -
-Within a single goroutine, the happens-before order is the
-order expressed by the program.
+A data race is defined as
+a write to a memory location happening concurrently with another read or write to that same location,
+unless all the accesses involved are atomic data accesses as provided by the sync/atomic package.
+As noted already, programmers are strongly encouraged to use appropriate synchronization
+to avoid data races.
+In the absence of data races, Go programs behave as if all the goroutines
+were multiplexed onto a single processor.
+This property is sometimes referred to as DRF-SC: data-race-free programs
+execute in a sequentially consistent manner.
-A read r of a variable v is allowed to observe a write w to v
-if both of the following hold:
+While programmers should write Go programs without data races,
+there are limitations to what a Go implementation can do in response to a data race.
+An implementation may always react to a data race by reporting the race and terminating the program.
+Otherwise, each read of a single-word-sized or sub-word-sized memory location
+must observe a value actually written to that location (perhaps by a concurrent executing goroutine)
+and not yet overwritten.
+These implementation constraints make Go more like Java or JavaScript,
+in that most races have a limited number of outcomes,
+and less like C and C++, where the meaning of any program with a race
+is entirely undefined, and the compiler may do anything at all.
+Go's approach aims to make errant programs more reliable and easier to debug,
+while still insisting that races are errors and that tools can diagnose and report them.
+The following formal definition of Go's memory model closely follows +the approach presented by Hans-J. Boehm and Sarita V. Adve in +“Foundations of the C++ Concurrency Memory Model”, +published in PLDI 2008. +The definition of data-race-free programs and the guarantee of sequential consistency +for race-free programs are equivalent to the ones in that work. +
+ ++The memory model describes the requirements on program executions, +which are made up of goroutine executions, +which in turn are made up of memory operations. +
+ ++A memory operation is modeled by four details: +
++Some memory operations are read-like, including read, atomic read, mutex lock, and channel receive. +Other memory operations are write-like, including write, atomic write, mutex unlock, channel send, and channel close. +Some, such as atomic compare-and-swap, are both read-like and write-like. +
+ ++A goroutine execution is modeled as a set of memory operations executed by a single goroutine. +
+ ++Requirement 1: +The memory operations in each goroutine must correspond to a correct sequential execution of that goroutine, +given the values read from and written to memory. +That execution must be consistent with the sequenced before relation, +defined as the partial order requirements set out by the Go language specification +for Go's control flow constructs as well as the order of evaluation for expressions. +
+ ++A Go program execution is modeled as a set of goroutine executions, +together with a mapping W that specifies the write-like operation that each read-like operation reads from. +(Multiple executions of the same program can have different program executions.) +
+ ++Requirement 2: +For a given program execution, the mapping W, when limited to synchronizing operations, +must be explainable by some implicit total order of the synchronizing operations +that is consistent with sequencing and the values read and written by those operations. +
+ ++The synchronized before relation is a partial order on synchronizing memory operations, +derived from W. +If a synchronizing read-like memory operation r +observes a synchronizing write-like memory operation w +(that is, if W(r) = w), +then w is synchronized before r. +Informally, the synchronized before relation is a subset of the implied total order +mentioned in the previous paragraph, +limited to the information that W directly observes. +
+ ++The happens before relation is defined as the transitive closure of the +union of the sequenced before and synchronized before relations. +
+ ++Requirement 3: +For an ordinary (non-synchronizing) data read r on a memory location x, +W(r) must be a write w that is visible to r, +where visible means that both of the following hold: +
v that happens
- after w but before r.
-To guarantee that a read r of a variable v observes a
-particular write w to v, ensure that w is the only
-write r is allowed to observe.
-That is, r is guaranteed to observe w if both of the following hold:
-
v
-either happens before w or after r.-This pair of conditions is stronger than the first pair; -it requires that there are no other writes happening -concurrently with w or r. +A read-write data race on memory location x +consists of a read-like memory operation r on x +and a write-like memory operation w on x, +at least one of which is non-synchronizing, +which are unordered by happens before +(that is, neither r happens before w +nor w happens before r).
-Within a single goroutine,
-there is no concurrency, so the two definitions are equivalent:
-a read r observes the value written by the most recent write w to v.
-When multiple goroutines access a shared variable v,
-they must use synchronization events to establish
-happens-before conditions that ensure reads observe the
-desired writes.
+A write-write data race on memory location x
+consists of two write-like memory operations w and w' on x,
+at least one of which is non-synchronizing,
+which are unordered by happens before.
-The initialization of variable v with the zero value
-for v's type behaves as a write in the memory model.
+Note that if there are no read-write or write-write data races on memory location x,
+then any read r on x has only one possible W(r):
+the single w that immediately precedes it in the happens before order.
-Reads and writes of values larger than a single machine word -behave as multiple machine-word-sized operations in an -unspecified order. +More generally, it can be shown that any Go program that is data-race-free, +meaning it has no program executions with read-write or write-write data races, +can only have outcomes explained by some sequentially consistent interleaving +of the goroutine executions. +(The proof is the same as Section 7 of Boehm and Adve's paper cited above.) +This property is called DRF-SC.
-+The intent of the formal definition is to match +the DRF-SC guarantee provided to race-free programs +by other languages, including C, C++, Java, JavaScript, Rust, and Swift. +
-+Certain Go language operations such as goroutine creation and memory allocation +act as synchronization operations. +The effect of these operations on the synchronized-before partial order +is documented in the “Synchronization” section below. +Individual packages are responsible for providing similar documentation +for their own operations. +
+ ++The preceding section gave a formal definition of data-race-free program execution. +This section informally describes the semantics that implementations must provide +for programs that do contain races. +
+ +
+First, any implementation can, upon detecting a data race,
+report the race and halt execution of the program.
+Implementations using ThreadSanitizer
+(accessed with “go build -race”)
+do exactly this.
+
+Otherwise, a read r of a memory location x +that is not larger than a machine word must observe +some write w such that r does not happen before w +and there is no write w' such that w happens before w' +and w' happens before r. +That is, each read must observe a value written by a preceding or concurrent write. +
+ ++Additionally, observation of acausal and “out of thin air” writes is disallowed. +
+ ++Reads of memory locations larger than a single machine word +are encouraged but not required to meet the same semantics +as word-sized memory locations, +observing a single allowed write w. +For performance reasons, +implementations may instead treat larger operations +as a set of individual machine-word-sized operations +in an unspecified order. +This means that races on multiword data structures +can lead to inconsistent values not corresponding to a single write. +When the values depend on the consistency +of internal (pointer, length) or (pointer, type) pairs, +as can be the case for interface values, maps, +slices, and strings in most Go implementations, +such races can in turn lead to arbitrary memory corruption. +
+ ++Examples of incorrect synchronization are given in the +“Incorrect synchronization” section below. +
+ ++Examples of the limitations on implementations are given in the +“Incorrect compilation” section below. +
+ +
Program initialization runs in a single goroutine,
@@ -141,15 +285,15 @@ If a package p imports package q, the completion of
-The start of the function main.main happens after
-all init functions have finished.
+The completion of all init functions is synchronized before
+the start of the function main.main.
The go statement that starts a new goroutine
-happens before the goroutine's execution begins.
+is synchronized before the start of the goroutine's execution.
@@ -174,11 +318,12 @@ calling hello will print "hello, world"
at some point in the future (perhaps after hello has returned).
-The exit of a goroutine is not guaranteed to happen before -any event in the program. For example, in this program: +The exit of a goroutine is not guaranteed to be synchronized before +any event in the program. +For example, in this program:
@@ -203,7 +348,7 @@ use a synchronization mechanism such as a lock or channel communication to establish a relative ordering. --Channel communication
+Channel communication
Channel communication is the main method of synchronization @@ -213,8 +358,8 @@ usually in a different goroutine.
-A send on a channel happens before the corresponding -receive from that channel completes. +A send on a channel is synchronized before the completion of the +corresponding receive from that channel.
@@ -239,13 +384,13 @@ func main() {
is guaranteed to print
"hello, world". The write toa-happens before the send onc, which happens before -the corresponding receive onccompletes, which happens before +is sequenced before the send onc, which is synchronized before +the corresponding receive onccompletes, which is sequenced before the-The closing of a channel happens before a receive that returns a zero value +The closing of a channel is synchronized before a receive that returns a zero value because the channel is closed.
@@ -256,8 +401,8 @@ yields a program with the same guaranteed behavior.-A receive from an unbuffered channel happens before -the send on that channel completes. +A receive from an unbuffered channel is synchronized before the completion of +the corresponding send on that channel.
@@ -283,8 +428,8 @@ func main() {
is also guaranteed to print
@@ -296,7 +441,7 @@ crash, or do something else.)"hello, world". The write toa-happens before the receive onc, which happens before -the corresponding send onccompletes, which happens +is sequenced before the receive onc, which is synchronized before +the corresponding send onccompletes, which is sequenced before the-The kth receive on a channel with capacity C happens before the k+Cth send from that channel completes. +The kth receive on a channel with capacity C is synchronized before the completion of the k+Cth send from that channel completes.
@@ -330,7 +475,7 @@ func main() { }
The sync package implements two lock data types,
@@ -339,7 +484,7 @@ The sync package implements two lock data types,
For any sync.Mutex or sync.RWMutex variable l and n < m,
-call n of l.Unlock() happens before call m of l.Lock() returns.
+call n of l.Unlock() is synchronized before call m of l.Lock() returns.
@@ -365,19 +510,29 @@ func main() {
is guaranteed to print "hello, world".
-The first call to l.Unlock() (in f) happens
+The first call to l.Unlock() (in f) is synchronized
before the second call to l.Lock() (in main) returns,
-which happens before the print.
+which is sequenced before the print.
For any call to l.RLock on a sync.RWMutex variable l,
-there is an n such that the l.RLock happens (returns) after call n to
-l.Unlock and the matching l.RUnlock happens
-before call n+1 to l.Lock.
+there is an n such that the nth call to l.Unlock
+is synchronized before the return from l.RLock,
+and the matching call to l.RUnlock is synchronized before the return from call n+1 to l.Lock.
+A successful call to l.TryLock (or l.TryRLock)
+is equivalent to a call to l.Lock (or l.RLock).
+An unsuccessful call has no synchronizing effect at all.
+As far as the memory model is concerned,
+l.TryLock (or l.TryRLock)
+may be considered to be able to return false
+even when the mutex l is unlocked.
+
The sync package provides a safe mechanism for
@@ -389,7 +544,8 @@ until f() has returned.
-A single call of f() from once.Do(f) happens (returns) before any call of once.Do(f) returns.
+The completion of a single call of f() from once.Do(f)
+is synchronized before the return of any call of once.Do(f).
@@ -424,13 +580,60 @@ The result will be that "hello, world" will be printed
twice.
-Note that a read r may observe the value written by a write w
-that happens concurrently with r.
-Even if this occurs, it does not imply that reads happening after r
-will observe writes that happened before w.
+The APIs in the sync/atomic
+package are collectively “atomic operations”
+that can be used to synchronize the execution of different goroutines.
+If the effect of an atomic operation A is observed by atomic operation B,
+then A is synchronized before B.
+All the atomic operations executed in a program behave as though executed
+in some sequentially consistent order.
+
+The preceding definition has the same semantics as C++’s sequentially consistent atomics
+and Java’s volatile variables.
+
+The runtime package provides
+a SetFinalizer function that adds a finalizer to be called when
+a particular object is no longer reachable by the program.
+A call to SetFinalizer(x, f) is synchronized before the finalization call f(x).
+
+The sync package provides additional synchronization abstractions,
+including condition variables,
+lock-free maps,
+allocation pools,
+and
+wait groups.
+The documentation for each of these specifies the guarantees it
+makes concerning synchronization.
+
+Other packages that provide synchronization abstractions +should document the guarantees they make too. +
+ + ++Programs with races are incorrect and +can exhibit non-sequentially consistent executions. +In particular, note that a read r may observe the value written by any write w +that executes concurrently with r. +Even if this occurs, it does not imply that reads happening after r +will observe writes that happened before w.
@@ -566,3 +769,197 @@ value for g.msg.
In all these examples, the solution is the same:
use explicit synchronization.
+The Go memory model restricts compiler optimizations as much as it does Go programs. +Some compiler optimizations that would be valid in single-threaded programs are not valid in all Go programs. +In particular, a compiler must not introduce writes that do not exist in the original program, +it must not allow a single read to observe multiple values, +and it must not allow a single write to write multiple values. +
+ ++All the following examples assume that `*p` and `*q` refer to +memory locations accessible to multiple goroutines. +
+ ++Not introducing data races into race-free programs means not moving +writes out of conditional statements in which they appear. +For example, a compiler must not invert the conditional in this program: +
+ +
+*p = 1
+if cond {
+ *p = 2
+}
+
+
++That is, the compiler must not rewrite the program into this one: +
+ +
+*p = 2
+if !cond {
+ *p = 1
+}
+
+
+
+If cond is false and another goroutine is reading *p,
+then in the original program, the other goroutine can only observe any prior value of *p and 1.
+In the rewritten program, the other goroutine can observe 2, which was previously impossible.
+
+Not introducing data races also means not assuming that loops terminate.
+For example, a compiler must in general not move the accesses to *p or *q
+ahead of the loop in this program:
+
+n := 0
+for e := list; e != nil; e = e.next {
+ n++
+}
+i := *p
+*q = 1
+
+
+
+If list pointed to a cyclic list,
+then the original program would never access *p or *q,
+but the rewritten program would.
+(Moving `*p` ahead would be safe if the compiler can prove `*p` will not panic;
+moving `*q` ahead would also require the compiler proving that no other
+goroutine can access `*q`.)
+
+Not introducing data races also means not assuming that called functions
+always return or are free of synchronization operations.
+For example, a compiler must not move the accesses to *p or *q
+ahead of the function call in this program
+(at least not without direct knowledge of the precise behavior of f):
+
+f() +i := *p +*q = 1 ++ +
+If the call never returned, then once again the original program
+would never access *p or *q, but the rewritten program would.
+And if the call contained synchronizing operations, then the original program
+could establish happens before edges preceding the accesses
+to *p and *q, but the rewritten program would not.
+
+Not allowing a single read to observe multiple values means
+not reloading local variables from shared memory.
+For example, a compiler must not discard i and reload it
+a second time from *p in this program:
+
+i := *p
+if i < 0 || i >= len(funcs) {
+ panic("invalid function index")
+}
+... complex code ...
+// compiler must NOT reload i = *p here
+funcs[i]()
+
+
+
+If the complex code needs many registers, a compiler for single-threaded programs
+could discard i without saving a copy and then reload
+i = *p just before
+funcs[i]().
+A Go compiler must not, because the value of *p may have changed.
+(Instead, the compiler could spill i to the stack.)
+
+Not allowing a single write to write multiple values also means not using
+the memory where a local variable will be written as temporary storage before the write.
+For example, a compiler must not use *p as temporary storage in this program:
+
+*p = i + *p/2 ++ +
+That is, it must not rewrite the program into this one: +
+ ++*p /= 2 +*p += i ++ +
+If i and *p start equal to 2,
+the original code does *p = 3,
+so a racing thread can read only 2 or 3 from *p.
+The rewritten code does *p = 1 and then *p = 3,
+allowing a racing thread to read 1 as well.
+
+Note that all these optimizations are permitted in C/C++ compilers: +a Go compiler sharing a back end with a C/C++ compiler must take care +to disable optimizations that are invalid for Go. +
+ ++Note that the prohibition on introducing data races +does not apply if the compiler can prove that the races +do not affect correct execution on the target platform. +For example, on essentially all CPUs, it is valid to rewrite +
+ +
+n := 0
+for i := 0; i < m; i++ {
+ n += *shared
+}
+
+
+into:
+
+
+n := 0
+local := *shared
+for i := 0; i < m; i++ {
+ n += local
+}
+
+
+
+provided it can be proved that *shared will not fault on access,
+because the potential added read will not affect any existing concurrent reads or writes.
+On the other hand, the rewrite would not be valid in a source-to-source translator.
+
+Go programmers writing data-race-free programs can rely on +sequentially consistent execution of those programs, +just as in essentially all other modern programming languages. +
+ ++When it comes to programs with races, +both programmers and compilers should remember the advice: +don't be clever. +
diff --git a/doc/go_spec.html b/doc/go_spec.html index 37580a83b7..ab172ac40e 100644 --- a/doc/go_spec.html +++ b/doc/go_spec.html @@ -1,6 +1,6 @@ @@ -502,8 +502,9 @@ After a backslash, certain single-character escapes represent special values:-All other sequences starting with a backslash are illegal inside rune literals. +An unrecognized character following a backslash in a rune literal is illegal.
+rune_lit = "'" ( unicode_value | byte_value ) "'" . unicode_value = unicode_char | little_u_value | big_u_value | escaped_char . @@ -530,6 +531,7 @@ escaped_char = `\` ( "a" | "b" | "f" | "n" | "r" | "t" | "v" | `\` | "'" | ` '\U00101234' '\'' // rune literal containing single quote character 'aa' // illegal: too many characters +'\k' // illegal: k is not recognized after a backslash '\xa' // illegal: too few hexadecimal digits '\0' // illegal: too few octal digits '\400' // illegal: octal value over 255 @@ -1025,7 +1027,7 @@ be unique.@@ -5457,8 +5464,12 @@ string([]rune{0x767d, 0x9d6c, 0x7fd4}) // "\u767d\u9d6c\u7fd4" == "白鵬翔" string([]rune{}) // "" string([]rune(nil)) // "" -type MyRunes []rune -string(MyRunes{0x767d, 0x9d6c, 0x7fd4}) // "\u767d\u9d6c\u7fd4" == "白鵬翔" +type runes []rune +string(runes{0x767d, 0x9d6c, 0x7fd4}) // "\u767d\u9d6c\u7fd4" == "白鵬翔" + +type myRune rune +string([]myRune{0x266b, 0x266c}) // "\u266b\u266c" == "♫♬" +myString([]myRune{0x1F30E}) // "\U0001f30e" == "🌎" @@ -5467,10 +5478,13 @@ Converting a value of a string type to a slice of bytes type yields a slice whose successive elements are the bytes of the string.StructType = "struct" "{" { FieldDecl ";" } "}" . FieldDecl = (IdentifierList Type | EmbeddedField) [ Tag ] . -EmbeddedField = [ "*" ] TypeName . +EmbeddedField = [ "*" ] TypeName [ TypeArgs ] . Tag = string_lit .@@ -3029,7 +3031,7 @@ Each element may optionally be preceded by a corresponding key.CompositeLit = LiteralType LiteralValue . LiteralType = StructType | ArrayType | "[" "..." "]" ElementType | - SliceType | MapType | TypeName . + SliceType | MapType | TypeName [ TypeArgs ] . LiteralValue = "{" [ ElementList [ "," ] ] "}" . ElementList = KeyedElement { "," KeyedElement } . KeyedElement = [ Key ":" ] Element . @@ -5245,7 +5247,7 @@ float32(0.49999999) // 0.5 of type float32 float64(-1e-1000) // 0.0 of type float64 string('x') // "x" of type string string(0x266c) // "♬" of type string -MyString("foo" + "bar") // "foobar" of type MyString +myString("foo" + "bar") // "foobar" of type myString string([]byte{'a'}) // not a constant: []byte{'a'} is not a constant (*int)(nil) // not a constant: nil is not a constant, *int is not a boolean, numeric, or string type int(1.2) // illegal: 1.2 cannot be represented as an int @@ -5428,8 +5430,9 @@ the range of valid Unicode code points are converted to@@ -5442,8 +5445,12 @@ string([]byte{'h', 'e', 'l', 'l', '\xc3', '\xb8'}) // "hellø" string([]byte{}) // "" string([]byte(nil)) // "" -type MyBytes []byte -string(MyBytes{'h', 'e', 'l', 'l', '\xc3', '\xb8'}) // "hellø" +type bytes []byte +string(bytes{'h', 'e', 'l', 'l', '\xc3', '\xb8'}) // "hellø" + +type myByte byte +string([]myByte{'w', 'o', 'r', 'l', 'd', '!'}) // "world!" +myString([]myByte{'\xf0', '\x9f', '\x8c', '\x8d'}) // "🌍""\uFFFD". string('a') // "a" string(-1) // "\ufffd" == "\xef\xbf\xbd" string(0xf8) // "\u00f8" == "ø" == "\xc3\xb8" -type MyString string -MyString(0x65e5) // "\u65e5" == "日" == "\xe6\x97\xa5" + +type myString string +myString(0x65e5) // "\u65e5" == "日" == "\xe6\x97\xa5"
-[]byte("hellø") // []byte{'h', 'e', 'l', 'l', '\xc3', '\xb8'}
-[]byte("") // []byte{}
+[]byte("hellø") // []byte{'h', 'e', 'l', 'l', '\xc3', '\xb8'}
+[]byte("") // []byte{}
-MyBytes("hellø") // []byte{'h', 'e', 'l', 'l', '\xc3', '\xb8'}
+bytes("hellø") // []byte{'h', 'e', 'l', 'l', '\xc3', '\xb8'}
+
+[]myByte("world!") // []myByte{'w', 'o', 'r', 'l', 'd', '!'}
+[]myByte(myString("🌏")) // []myByte{'\xf0', '\x9f', '\x8c', '\x8f'}
@@ -5479,10 +5493,13 @@ Converting a value of a string type to a slice of runes type
yields a slice containing the individual Unicode code points of the string.
-[]rune(MyString("白鵬翔")) // []rune{0x767d, 0x9d6c, 0x7fd4}
-[]rune("") // []rune{}
+[]rune(myString("白鵬翔")) // []rune{0x767d, 0x9d6c, 0x7fd4}
+[]rune("") // []rune{}
-MyRunes("白鵬翔") // []rune{0x767d, 0x9d6c, 0x7fd4}
+runes("白鵬翔") // []rune{0x767d, 0x9d6c, 0x7fd4}
+
+[]myRune("♫♬") // []myRune{0x266b, 0x266c}
+[]myRune(myString("🌐")) // []myRune{0x1f310}
@@ -7141,7 +7158,7 @@ A constant size argument must be non-negative and re
by a value of type int; if it is an untyped constant it is given type int.
If both n and m are provided and are constant, then
n must be no larger than m.
-If n is negative or larger than m at run time,
+For slices and channels, if n is negative or larger than m at run time,
a run-time panic occurs.
diff --git a/src/cmd/buildid/buildid.go b/src/cmd/buildid/buildid.go
index 8e02a7ae10..72ad80dbbb 100644
--- a/src/cmd/buildid/buildid.go
+++ b/src/cmd/buildid/buildid.go
@@ -53,6 +53,11 @@ func main() {
log.Fatal(err)
}
+ // <= go 1.7 doesn't embed the contentID or actionID, so no slash is present
+ if !strings.Contains(id, "/") {
+ log.Fatalf("%s: build ID is a legacy format...binary too old for this tool", file)
+ }
+
newID := id[:strings.LastIndex(id, "/")] + "/" + buildid.HashToString(hash)
if len(newID) != len(id) {
log.Fatalf("%s: build ID length mismatch %q vs %q", file, id, newID)
diff --git a/src/cmd/cgo/gcc.go b/src/cmd/cgo/gcc.go
index 4d1a5bd8de..4dff5e2b1c 100644
--- a/src/cmd/cgo/gcc.go
+++ b/src/cmd/cgo/gcc.go
@@ -1831,6 +1831,23 @@ func (p *Package) gccDebug(stdin []byte, nnames int) (d *dwarf.Data, ints []int6
bo := f.ByteOrder
symtab, err := f.Symbols()
if err == nil {
+ // Check for use of -fsanitize=hwaddress (issue 53285).
+ removeTag := func(v uint64) uint64 { return v }
+ if goarch == "arm64" {
+ for i := range symtab {
+ if symtab[i].Name == "__hwasan_init" {
+ // -fsanitize=hwaddress on ARM
+ // uses the upper byte of a
+ // memory address as a hardware
+ // tag. Remove it so that
+ // we can find the associated
+ // data.
+ removeTag = func(v uint64) uint64 { return v &^ (0xff << (64 - 8)) }
+ break
+ }
+ }
+ }
+
for i := range symtab {
s := &symtab[i]
switch {
@@ -1838,9 +1855,10 @@ func (p *Package) gccDebug(stdin []byte, nnames int) (d *dwarf.Data, ints []int6
// Found it. Now find data section.
if i := int(s.Section); 0 <= i && i < len(f.Sections) {
sect := f.Sections[i]
- if sect.Addr <= s.Value && s.Value < sect.Addr+sect.Size {
+ val := removeTag(s.Value)
+ if sect.Addr <= val && val < sect.Addr+sect.Size {
if sdat, err := sect.Data(); err == nil {
- data := sdat[s.Value-sect.Addr:]
+ data := sdat[val-sect.Addr:]
ints = make([]int64, len(data)/8)
for i := range ints {
ints[i] = int64(bo.Uint64(data[i*8:]))
@@ -1852,9 +1870,10 @@ func (p *Package) gccDebug(stdin []byte, nnames int) (d *dwarf.Data, ints []int6
// Found it. Now find data section.
if i := int(s.Section); 0 <= i && i < len(f.Sections) {
sect := f.Sections[i]
- if sect.Addr <= s.Value && s.Value < sect.Addr+sect.Size {
+ val := removeTag(s.Value)
+ if sect.Addr <= val && val < sect.Addr+sect.Size {
if sdat, err := sect.Data(); err == nil {
- data := sdat[s.Value-sect.Addr:]
+ data := sdat[val-sect.Addr:]
floats = make([]float64, len(data)/8)
for i := range floats {
floats[i] = math.Float64frombits(bo.Uint64(data[i*8:]))
@@ -1867,9 +1886,10 @@ func (p *Package) gccDebug(stdin []byte, nnames int) (d *dwarf.Data, ints []int6
// Found it. Now find data section.
if i := int(s.Section); 0 <= i && i < len(f.Sections) {
sect := f.Sections[i]
- if sect.Addr <= s.Value && s.Value < sect.Addr+sect.Size {
+ val := removeTag(s.Value)
+ if sect.Addr <= val && val < sect.Addr+sect.Size {
if sdat, err := sect.Data(); err == nil {
- data := sdat[s.Value-sect.Addr:]
+ data := sdat[val-sect.Addr:]
strdata[n] = string(data)
}
}
@@ -1880,9 +1900,10 @@ func (p *Package) gccDebug(stdin []byte, nnames int) (d *dwarf.Data, ints []int6
// Found it. Now find data section.
if i := int(s.Section); 0 <= i && i < len(f.Sections) {
sect := f.Sections[i]
- if sect.Addr <= s.Value && s.Value < sect.Addr+sect.Size {
+ val := removeTag(s.Value)
+ if sect.Addr <= val && val < sect.Addr+sect.Size {
if sdat, err := sect.Data(); err == nil {
- data := sdat[s.Value-sect.Addr:]
+ data := sdat[val-sect.Addr:]
strlen := bo.Uint64(data[:8])
if strlen > (1<<(uint(p.IntSize*8)-1) - 1) { // greater than MaxInt?
fatalf("string literal too big")
@@ -2242,6 +2263,8 @@ var dwarfToName = map[string]string{
"long long unsigned int": "ulonglong",
"signed char": "schar",
"unsigned char": "uchar",
+ "unsigned long": "ulong", // Used by Clang 14; issue 53013.
+ "unsigned long long": "ulonglong", // Used by Clang 14; issue 53013.
}
const signedDelta = 64
diff --git a/src/cmd/compile/internal/noder/stencil.go b/src/cmd/compile/internal/noder/stencil.go
index 88e4961666..3f12aa3cbd 100644
--- a/src/cmd/compile/internal/noder/stencil.go
+++ b/src/cmd/compile/internal/noder/stencil.go
@@ -1330,10 +1330,10 @@ func (g *genInst) dictPass(info *instInfo) {
m = convertUsingDictionary(info, info.dictParam, m.Pos(), mce.X, m, m.Type())
}
case ir.ODOTTYPE, ir.ODOTTYPE2:
- if !m.Type().HasShape() {
+ dt := m.(*ir.TypeAssertExpr)
+ if !dt.Type().HasShape() && !dt.X.Type().HasShape() {
break
}
- dt := m.(*ir.TypeAssertExpr)
var rtype, itab ir.Node
if dt.Type().IsInterface() || dt.X.Type().IsEmptyInterface() {
// TODO(mdempsky): Investigate executing this block unconditionally.
diff --git a/src/cmd/compile/internal/reflectdata/reflect.go b/src/cmd/compile/internal/reflectdata/reflect.go
index 3ffb7dcefa..21301ab149 100644
--- a/src/cmd/compile/internal/reflectdata/reflect.go
+++ b/src/cmd/compile/internal/reflectdata/reflect.go
@@ -412,7 +412,7 @@ func dimportpath(p *types.Pkg) {
}
s := base.Ctxt.Lookup("type..importpath." + p.Prefix + ".")
- ot := dnameData(s, 0, p.Path, "", nil, false)
+ ot := dnameData(s, 0, p.Path, "", nil, false, false)
objw.Global(s, int32(ot), obj.DUPOK|obj.RODATA)
s.Set(obj.AttrContentAddressable, true)
p.Pathsym = s
@@ -461,12 +461,12 @@ func dnameField(lsym *obj.LSym, ot int, spkg *types.Pkg, ft *types.Field) int {
if !types.IsExported(ft.Sym.Name) && ft.Sym.Pkg != spkg {
base.Fatalf("package mismatch for %v", ft.Sym)
}
- nsym := dname(ft.Sym.Name, ft.Note, nil, types.IsExported(ft.Sym.Name))
+ nsym := dname(ft.Sym.Name, ft.Note, nil, types.IsExported(ft.Sym.Name), ft.Embedded != 0)
return objw.SymPtr(lsym, ot, nsym, 0)
}
// dnameData writes the contents of a reflect.name into s at offset ot.
-func dnameData(s *obj.LSym, ot int, name, tag string, pkg *types.Pkg, exported bool) int {
+func dnameData(s *obj.LSym, ot int, name, tag string, pkg *types.Pkg, exported, embedded bool) int {
if len(name) >= 1<<29 {
base.Fatalf("name too long: %d %s...", len(name), name[:1024])
}
@@ -491,6 +491,9 @@ func dnameData(s *obj.LSym, ot int, name, tag string, pkg *types.Pkg, exported b
if pkg != nil {
bits |= 1 << 2
}
+ if embedded {
+ bits |= 1 << 3
+ }
b := make([]byte, l)
b[0] = bits
copy(b[1:], nameLen[:nameLenLen])
@@ -513,7 +516,7 @@ func dnameData(s *obj.LSym, ot int, name, tag string, pkg *types.Pkg, exported b
var dnameCount int
// dname creates a reflect.name for a struct field or method.
-func dname(name, tag string, pkg *types.Pkg, exported bool) *obj.LSym {
+func dname(name, tag string, pkg *types.Pkg, exported, embedded bool) *obj.LSym {
// Write out data as "type.." to signal two things to the
// linker, first that when dynamically linking, the symbol
// should be moved to a relro section, and second that the
@@ -538,11 +541,14 @@ func dname(name, tag string, pkg *types.Pkg, exported bool) *obj.LSym {
sname = fmt.Sprintf(`%s"".%d`, sname, dnameCount)
dnameCount++
}
+ if embedded {
+ sname += ".embedded"
+ }
s := base.Ctxt.Lookup(sname)
if len(s.P) > 0 {
return s
}
- ot := dnameData(s, 0, name, tag, pkg, exported)
+ ot := dnameData(s, 0, name, tag, pkg, exported, embedded)
objw.Global(s, int32(ot), obj.DUPOK|obj.RODATA)
s.Set(obj.AttrContentAddressable, true)
return s
@@ -610,7 +616,7 @@ func dextratypeData(lsym *obj.LSym, ot int, t *types.Type) int {
if !exported && a.name.Pkg != typePkg(t) {
pkg = a.name.Pkg
}
- nsym := dname(a.name.Name, "", pkg, exported)
+ nsym := dname(a.name.Name, "", pkg, exported, false)
ot = objw.SymPtrOff(lsym, ot, nsym)
ot = dmethodptrOff(lsym, ot, writeType(a.mtype))
@@ -775,7 +781,7 @@ func dcommontype(lsym *obj.LSym, t *types.Type) int {
}
ot = objw.SymPtr(lsym, ot, gcsym, 0) // gcdata
- nsym := dname(p, "", nil, exported)
+ nsym := dname(p, "", nil, exported, false)
ot = objw.SymPtrOff(lsym, ot, nsym) // str
// ptrToThis
if sptr == nil {
@@ -1074,7 +1080,7 @@ func writeType(t *types.Type) *obj.LSym {
if !exported && a.name.Pkg != tpkg {
pkg = a.name.Pkg
}
- nsym := dname(a.name.Name, "", pkg, exported)
+ nsym := dname(a.name.Name, "", pkg, exported, false)
ot = objw.SymPtrOff(lsym, ot, nsym)
ot = objw.SymPtrOff(lsym, ot, writeType(a.type_))
@@ -1180,14 +1186,7 @@ func writeType(t *types.Type) *obj.LSym {
// ../../../../runtime/type.go:/structField
ot = dnameField(lsym, ot, spkg, f)
ot = objw.SymPtr(lsym, ot, writeType(f.Type), 0)
- offsetAnon := uint64(f.Offset) << 1
- if offsetAnon>>1 != uint64(f.Offset) {
- base.Fatalf("%v: bad field offset for %s", t, f.Sym.Name)
- }
- if f.Embedded != 0 {
- offsetAnon |= 1
- }
- ot = objw.Uintptr(lsym, ot, offsetAnon)
+ ot = objw.Uintptr(lsym, ot, uint64(f.Offset))
}
}
@@ -1356,7 +1355,7 @@ func WriteTabs() {
// name nameOff
// typ typeOff // pointer to symbol
// }
- nsym := dname(p.Sym().Name, "", nil, true)
+ nsym := dname(p.Sym().Name, "", nil, true, false)
t := p.Type()
if p.Class != ir.PFUNC {
t = types.NewPtr(t)
diff --git a/src/cmd/compile/internal/ssa/gen/RISCV64.rules b/src/cmd/compile/internal/ssa/gen/RISCV64.rules
index 7aea622c5e..dd20be2aeb 100644
--- a/src/cmd/compile/internal/ssa/gen/RISCV64.rules
+++ b/src/cmd/compile/internal/ssa/gen/RISCV64.rules
@@ -735,9 +735,9 @@
(NEGW (MOVDconst [x])) => (MOVDconst [int64(int32(-x))])
// Shift of a constant.
-(SLLI [x] (MOVDconst [y])) && is32Bit(y << x) => (MOVDconst [y << x])
-(SRLI [x] (MOVDconst [y])) => (MOVDconst [int64(uint64(y) >> x)])
-(SRAI [x] (MOVDconst [y])) => (MOVDconst [int64(y) >> x])
+(SLLI [x] (MOVDconst [y])) && is32Bit(y << uint32(x)) => (MOVDconst [y << uint32(x)])
+(SRLI [x] (MOVDconst [y])) => (MOVDconst [int64(uint64(y) >> uint32(x))])
+(SRAI [x] (MOVDconst [y])) => (MOVDconst [int64(y) >> uint32(x)])
// SLTI/SLTIU with constants.
(SLTI [x] (MOVDconst [y])) => (MOVDconst [b2i(int64(y) < int64(x))])
diff --git a/src/cmd/compile/internal/ssa/rewriteRISCV64.go b/src/cmd/compile/internal/ssa/rewriteRISCV64.go
index 6828d97ff8..2677e99dc0 100644
--- a/src/cmd/compile/internal/ssa/rewriteRISCV64.go
+++ b/src/cmd/compile/internal/ssa/rewriteRISCV64.go
@@ -4843,19 +4843,19 @@ func rewriteValueRISCV64_OpRISCV64SLL(v *Value) bool {
func rewriteValueRISCV64_OpRISCV64SLLI(v *Value) bool {
v_0 := v.Args[0]
// match: (SLLI [x] (MOVDconst [y]))
- // cond: is32Bit(y << x)
- // result: (MOVDconst [y << x])
+ // cond: is32Bit(y << uint32(x))
+ // result: (MOVDconst [y << uint32(x)])
for {
x := auxIntToInt64(v.AuxInt)
if v_0.Op != OpRISCV64MOVDconst {
break
}
y := auxIntToInt64(v_0.AuxInt)
- if !(is32Bit(y << x)) {
+ if !(is32Bit(y << uint32(x))) {
break
}
v.reset(OpRISCV64MOVDconst)
- v.AuxInt = int64ToAuxInt(y << x)
+ v.AuxInt = int64ToAuxInt(y << uint32(x))
return true
}
return false
@@ -4913,7 +4913,7 @@ func rewriteValueRISCV64_OpRISCV64SRA(v *Value) bool {
func rewriteValueRISCV64_OpRISCV64SRAI(v *Value) bool {
v_0 := v.Args[0]
// match: (SRAI [x] (MOVDconst [y]))
- // result: (MOVDconst [int64(y) >> x])
+ // result: (MOVDconst [int64(y) >> uint32(x)])
for {
x := auxIntToInt64(v.AuxInt)
if v_0.Op != OpRISCV64MOVDconst {
@@ -4921,7 +4921,7 @@ func rewriteValueRISCV64_OpRISCV64SRAI(v *Value) bool {
}
y := auxIntToInt64(v_0.AuxInt)
v.reset(OpRISCV64MOVDconst)
- v.AuxInt = int64ToAuxInt(int64(y) >> x)
+ v.AuxInt = int64ToAuxInt(int64(y) >> uint32(x))
return true
}
return false
@@ -4947,7 +4947,7 @@ func rewriteValueRISCV64_OpRISCV64SRL(v *Value) bool {
func rewriteValueRISCV64_OpRISCV64SRLI(v *Value) bool {
v_0 := v.Args[0]
// match: (SRLI [x] (MOVDconst [y]))
- // result: (MOVDconst [int64(uint64(y) >> x)])
+ // result: (MOVDconst [int64(uint64(y) >> uint32(x))])
for {
x := auxIntToInt64(v.AuxInt)
if v_0.Op != OpRISCV64MOVDconst {
@@ -4955,7 +4955,7 @@ func rewriteValueRISCV64_OpRISCV64SRLI(v *Value) bool {
}
y := auxIntToInt64(v_0.AuxInt)
v.reset(OpRISCV64MOVDconst)
- v.AuxInt = int64ToAuxInt(int64(uint64(y) >> x))
+ v.AuxInt = int64ToAuxInt(int64(uint64(y) >> uint32(x)))
return true
}
return false
diff --git a/src/cmd/compile/internal/types2/api_test.go b/src/cmd/compile/internal/types2/api_test.go
index e6de955a6e..f5526bb25a 100644
--- a/src/cmd/compile/internal/types2/api_test.go
+++ b/src/cmd/compile/internal/types2/api_test.go
@@ -38,7 +38,7 @@ func pkgFor(path, source string, info *Info) (*Package, error) {
return conf.Check(f.PkgName.Value, []*syntax.File{f}, info)
}
-func mustTypecheck(t *testing.T, path, source string, info *Info) string {
+func mustTypecheck(t testing.TB, path, source string, info *Info) string {
pkg, err := pkgFor(path, source, info)
if err != nil {
name := path
diff --git a/src/cmd/compile/internal/types2/call.go b/src/cmd/compile/internal/types2/call.go
index 3ade147dfe..b1ea6917fb 100644
--- a/src/cmd/compile/internal/types2/call.go
+++ b/src/cmd/compile/internal/types2/call.go
@@ -72,13 +72,13 @@ func (check *Checker) instantiateSignature(pos syntax.Pos, typ *Signature, targs
}()
}
- inst := check.instance(pos, typ, targs, check.bestContext(nil)).(*Signature)
+ inst := check.instance(pos, typ, targs, nil, check.context()).(*Signature)
assert(len(xlist) <= len(targs))
// verify instantiation lazily (was issue #50450)
check.later(func() {
tparams := typ.TypeParams().list()
- if i, err := check.verify(pos, tparams, targs); err != nil {
+ if i, err := check.verify(pos, tparams, targs, check.context()); err != nil {
// best position for error reporting
pos := pos
if i < len(xlist) {
@@ -395,7 +395,7 @@ func (check *Checker) arguments(call *syntax.CallExpr, sig *Signature, targs []T
// need to compute it from the adjusted list; otherwise we can
// simply use the result signature's parameter list.
if adjusted {
- sigParams = check.subst(call.Pos(), sigParams, makeSubstMap(sig.TypeParams().list(), targs), nil).(*Tuple)
+ sigParams = check.subst(call.Pos(), sigParams, makeSubstMap(sig.TypeParams().list(), targs), nil, check.context()).(*Tuple)
} else {
sigParams = rsig.params
}
diff --git a/src/cmd/compile/internal/types2/check.go b/src/cmd/compile/internal/types2/check.go
index 5cf8454aa4..ff8ae3bc7e 100644
--- a/src/cmd/compile/internal/types2/check.go
+++ b/src/cmd/compile/internal/types2/check.go
@@ -98,7 +98,7 @@ type Checker struct {
nextID uint64 // unique Id for type parameters (first valid Id is 1)
objMap map[Object]*declInfo // maps package-level objects and (non-interface) methods to declaration info
impMap map[importKey]*Package // maps (import path, source directory) to (complete or fake) package
- infoMap map[*Named]typeInfo // maps named types to their associated type info (for cycle detection)
+ valids instanceLookup // valid *Named (incl. instantiated) types per the validType check
// pkgPathMap maps package names to the set of distinct import paths we've
// seen for that name, anywhere in the import graph. It is used for
@@ -241,7 +241,6 @@ func NewChecker(conf *Config, pkg *Package, info *Info) *Checker {
version: version,
objMap: make(map[Object]*declInfo),
impMap: make(map[importKey]*Package),
- infoMap: make(map[*Named]typeInfo),
}
}
diff --git a/src/cmd/compile/internal/types2/decl.go b/src/cmd/compile/internal/types2/decl.go
index b6f81aa8a5..0bc5f9f3e1 100644
--- a/src/cmd/compile/internal/types2/decl.go
+++ b/src/cmd/compile/internal/types2/decl.go
@@ -508,7 +508,7 @@ func (check *Checker) typeDecl(obj *TypeName, tdecl *syntax.TypeDecl, def *Named
}
// type definition or generic type declaration
- named := check.newNamed(obj, nil, nil, nil)
+ named := check.newNamed(obj, nil, nil)
def.setUnderlying(named)
if tdecl.TParamList != nil {
@@ -522,8 +522,8 @@ func (check *Checker) typeDecl(obj *TypeName, tdecl *syntax.TypeDecl, def *Named
assert(rhs != nil)
named.fromRHS = rhs
- // If the underlying was not set while type-checking the right-hand side, it
- // is invalid and an error should have been reported elsewhere.
+ // If the underlying type was not set while type-checking the right-hand
+ // side, it is invalid and an error should have been reported elsewhere.
if named.underlying == nil {
named.underlying = Typ[Invalid]
}
@@ -635,7 +635,7 @@ func (check *Checker) collectMethods(obj *TypeName) {
// and field names must be distinct."
base, _ := obj.typ.(*Named) // shouldn't fail but be conservative
if base != nil {
- assert(base.targs.Len() == 0) // collectMethods should not be called on an instantiated type
+ assert(base.TypeArgs().Len() == 0) // collectMethods should not be called on an instantiated type
// See issue #52529: we must delay the expansion of underlying here, as
// base may not be fully set-up.
@@ -646,8 +646,8 @@ func (check *Checker) collectMethods(obj *TypeName) {
// Checker.Files may be called multiple times; additional package files
// may add methods to already type-checked types. Add pre-existing methods
// so that we can detect redeclarations.
- for i := 0; i < base.methods.Len(); i++ {
- m := base.methods.At(i, nil)
+ for i := 0; i < base.NumMethods(); i++ {
+ m := base.Method(i)
assert(m.name != "_")
assert(mset.insert(m) == nil)
}
@@ -679,8 +679,8 @@ func (check *Checker) collectMethods(obj *TypeName) {
func (check *Checker) checkFieldUniqueness(base *Named) {
if t, _ := base.under().(*Struct); t != nil {
var mset objset
- for i := 0; i < base.methods.Len(); i++ {
- m := base.methods.At(i, nil)
+ for i := 0; i < base.NumMethods(); i++ {
+ m := base.Method(i)
assert(m.name != "_")
assert(mset.insert(m) == nil)
}
diff --git a/src/cmd/compile/internal/types2/infer.go b/src/cmd/compile/internal/types2/infer.go
index 9e77d67a7d..b0c6a4fcea 100644
--- a/src/cmd/compile/internal/types2/infer.go
+++ b/src/cmd/compile/internal/types2/infer.go
@@ -110,11 +110,11 @@ func (check *Checker) infer(pos syntax.Pos, tparams []*TypeParam, targs []Type,
renameMap := makeRenameMap(tparams, tparams2)
for i, tparam := range tparams {
- tparams2[i].bound = check.subst(pos, tparam.bound, renameMap, nil)
+ tparams2[i].bound = check.subst(pos, tparam.bound, renameMap, nil, check.context())
}
tparams = tparams2
- params = check.subst(pos, params, renameMap, nil).(*Tuple)
+ params = check.subst(pos, params, renameMap, nil, check.context()).(*Tuple)
}
}
@@ -188,7 +188,7 @@ func (check *Checker) infer(pos syntax.Pos, tparams []*TypeParam, targs []Type,
// but that doesn't impact the isParameterized check for now).
if params.Len() > 0 {
smap := makeSubstMap(tparams, targs)
- params = check.subst(nopos, params, smap, nil).(*Tuple)
+ params = check.subst(nopos, params, smap, nil, check.context()).(*Tuple)
}
// Unify parameter and argument types for generic parameters with typed arguments
@@ -224,7 +224,7 @@ func (check *Checker) infer(pos syntax.Pos, tparams []*TypeParam, targs []Type,
}
}
smap := makeSubstMap(tparams, targs)
- inferred := check.subst(arg.Pos(), tpar, smap, nil)
+ inferred := check.subst(arg.Pos(), tpar, smap, nil, check.context())
if inferred != tpar {
check.errorf(arg, "%s %s of %s does not match inferred type %s for %s", kind, targ, arg.expr, inferred, tpar)
} else {
@@ -434,7 +434,7 @@ func (w *tpWalker) isParameterized(typ Type) (res bool) {
return w.isParameterized(t.elem)
case *Named:
- return w.isParameterizedTypeList(t.targs.list())
+ return w.isParameterizedTypeList(t.TypeArgs().list())
case *TypeParam:
// t must be one of w.tparams
@@ -626,7 +626,7 @@ func (check *Checker) inferB(pos syntax.Pos, tparams []*TypeParam, targs []Type)
n := 0
for _, index := range dirty {
t0 := types[index]
- if t1 := check.subst(nopos, t0, smap, nil); t1 != t0 {
+ if t1 := check.subst(nopos, t0, smap, nil, check.context()); t1 != t0 {
types[index] = t1
dirty[n] = index
n++
diff --git a/src/cmd/compile/internal/types2/instantiate.go b/src/cmd/compile/internal/types2/instantiate.go
index bb90ab3736..5833f8db7e 100644
--- a/src/cmd/compile/internal/types2/instantiate.go
+++ b/src/cmd/compile/internal/types2/instantiate.go
@@ -40,6 +40,9 @@ import (
// count is incorrect; for *Named types, a panic may occur later inside the
// *Named API.
func Instantiate(ctxt *Context, orig Type, targs []Type, validate bool) (Type, error) {
+ if ctxt == nil {
+ ctxt = NewContext()
+ }
if validate {
var tparams []*TypeParam
switch t := orig.(type) {
@@ -51,40 +54,72 @@ func Instantiate(ctxt *Context, orig Type, targs []Type, validate bool) (Type, e
if len(targs) != len(tparams) {
return nil, fmt.Errorf("got %d type arguments but %s has %d type parameters", len(targs), orig, len(tparams))
}
- if i, err := (*Checker)(nil).verify(nopos, tparams, targs); err != nil {
+ if i, err := (*Checker)(nil).verify(nopos, tparams, targs, ctxt); err != nil {
return nil, &ArgumentError{i, err}
}
}
- inst := (*Checker)(nil).instance(nopos, orig, targs, ctxt)
+ inst := (*Checker)(nil).instance(nopos, orig, targs, nil, ctxt)
return inst, nil
}
-// instance creates a type or function instance using the given original type
-// typ and arguments targs. For Named types the resulting instance will be
-// unexpanded. check may be nil.
-func (check *Checker) instance(pos syntax.Pos, orig Type, targs []Type, ctxt *Context) (res Type) {
- var h string
+// instance instantiates the given original (generic) function or type with the
+// provided type arguments and returns the resulting instance. If an identical
+// instance exists already in the given contexts, it returns that instance,
+// otherwise it creates a new one.
+//
+// If expanding is non-nil, it is the Named instance type currently being
+// expanded. If ctxt is non-nil, it is the context associated with the current
+// type-checking pass or call to Instantiate. At least one of expanding or ctxt
+// must be non-nil.
+//
+// For Named types the resulting instance may be unexpanded.
+func (check *Checker) instance(pos syntax.Pos, orig Type, targs []Type, expanding *Named, ctxt *Context) (res Type) {
+ // The order of the contexts below matters: we always prefer instances in the
+ // expanding instance context in order to preserve reference cycles.
+ //
+ // Invariant: if expanding != nil, the returned instance will be the instance
+ // recorded in expanding.inst.ctxt.
+ var ctxts []*Context
+ if expanding != nil {
+ ctxts = append(ctxts, expanding.inst.ctxt)
+ }
if ctxt != nil {
- h = ctxt.instanceHash(orig, targs)
- // typ may already have been instantiated with identical type arguments. In
- // that case, re-use the existing instance.
- if inst := ctxt.lookup(h, orig, targs); inst != nil {
- return inst
+ ctxts = append(ctxts, ctxt)
+ }
+ assert(len(ctxts) > 0)
+
+ // Compute all hashes; hashes may differ across contexts due to different
+ // unique IDs for Named types within the hasher.
+ hashes := make([]string, len(ctxts))
+ for i, ctxt := range ctxts {
+ hashes[i] = ctxt.instanceHash(orig, targs)
+ }
+
+ // If local is non-nil, updateContexts return the type recorded in
+ // local.
+ updateContexts := func(res Type) Type {
+ for i := len(ctxts) - 1; i >= 0; i-- {
+ res = ctxts[i].update(hashes[i], orig, targs, res)
+ }
+ return res
+ }
+
+ // typ may already have been instantiated with identical type arguments. In
+ // that case, re-use the existing instance.
+ for i, ctxt := range ctxts {
+ if inst := ctxt.lookup(hashes[i], orig, targs); inst != nil {
+ return updateContexts(inst)
}
}
switch orig := orig.(type) {
case *Named:
- tname := NewTypeName(pos, orig.obj.pkg, orig.obj.name, nil)
- named := check.newNamed(tname, orig, nil, nil) // underlying, tparams, and methods are set when named is resolved
- named.targs = newTypeList(targs)
- named.resolver = func(ctxt *Context, n *Named) (*TypeParamList, Type, *methodList) {
- return expandNamed(ctxt, n, pos)
- }
- res = named
+ res = check.newNamedInstance(pos, orig, targs, expanding) // substituted lazily
case *Signature:
+ assert(expanding == nil) // function instances cannot be reached from Named types
+
tparams := orig.TypeParams()
if !check.validateTArgLen(pos, tparams.Len(), len(targs)) {
return Typ[Invalid]
@@ -92,7 +127,7 @@ func (check *Checker) instance(pos syntax.Pos, orig Type, targs []Type, ctxt *Co
if tparams.Len() == 0 {
return orig // nothing to do (minor optimization)
}
- sig := check.subst(pos, orig, makeSubstMap(tparams.list(), targs), ctxt).(*Signature)
+ sig := check.subst(pos, orig, makeSubstMap(tparams.list(), targs), nil, ctxt).(*Signature)
// If the signature doesn't use its type parameters, subst
// will not make a copy. In that case, make a copy now (so
// we can set tparams to nil w/o causing side-effects).
@@ -110,13 +145,8 @@ func (check *Checker) instance(pos syntax.Pos, orig Type, targs []Type, ctxt *Co
panic(fmt.Sprintf("%v: cannot instantiate %v", pos, orig))
}
- if ctxt != nil {
- // It's possible that we've lost a race to add named to the context.
- // In this case, use whichever instance is recorded in the context.
- res = ctxt.update(h, orig, targs, res)
- }
-
- return res
+ // Update all contexts; it's possible that we've lost a race.
+ return updateContexts(res)
}
// validateTArgLen verifies that the length of targs and tparams matches,
@@ -134,7 +164,7 @@ func (check *Checker) validateTArgLen(pos syntax.Pos, ntparams, ntargs int) bool
return true
}
-func (check *Checker) verify(pos syntax.Pos, tparams []*TypeParam, targs []Type) (int, error) {
+func (check *Checker) verify(pos syntax.Pos, tparams []*TypeParam, targs []Type, ctxt *Context) (int, error) {
smap := makeSubstMap(tparams, targs)
for i, tpar := range tparams {
// Ensure that we have a (possibly implicit) interface as type bound (issue #51048).
@@ -143,7 +173,7 @@ func (check *Checker) verify(pos syntax.Pos, tparams []*TypeParam, targs []Type)
// as the instantiated type; before we can use it for bounds checking we
// need to instantiate it with the type arguments with which we instantiated
// the parameterized type.
- bound := check.subst(pos, tpar.bound, smap, nil)
+ bound := check.subst(pos, tpar.bound, smap, nil, ctxt)
if err := check.implements(targs[i], bound); err != nil {
return i, err
}
@@ -248,7 +278,7 @@ func (check *Checker) implements(V, T Type) error {
if alt != nil {
return errorf("%s does not implement %s (possibly missing ~ for %s in constraint %s)", V, T, alt, T)
} else {
- return errorf("%s does not implement %s", V, T)
+ return errorf("%s does not implement %s (%s missing in %s)", V, T, V, Ti.typeSet().terms)
}
}
diff --git a/src/cmd/compile/internal/types2/methodlist.go b/src/cmd/compile/internal/types2/methodlist.go
deleted file mode 100644
index cd6c06c5fb..0000000000
--- a/src/cmd/compile/internal/types2/methodlist.go
+++ /dev/null
@@ -1,79 +0,0 @@
-// Copyright 2022 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package types2
-
-import "sync"
-
-// methodList holds a list of methods that may be lazily resolved by a provided
-// resolution method.
-type methodList struct {
- methods []*Func
-
- // guards synchronizes the instantiation of lazy methods. For lazy method
- // lists, guards is non-nil and of the length passed to newLazyMethodList.
- // For non-lazy method lists, guards is nil.
- guards *[]sync.Once
-}
-
-// newMethodList creates a non-lazy method list holding the given methods.
-func newMethodList(methods []*Func) *methodList {
- return &methodList{methods: methods}
-}
-
-// newLazyMethodList creates a lazy method list of the given length. Methods
-// may be resolved lazily for a given index by providing a resolver function.
-func newLazyMethodList(length int) *methodList {
- guards := make([]sync.Once, length)
- return &methodList{
- methods: make([]*Func, length),
- guards: &guards,
- }
-}
-
-// isLazy reports whether the receiver is a lazy method list.
-func (l *methodList) isLazy() bool {
- return l != nil && l.guards != nil
-}
-
-// Add appends a method to the method list if not not already present. Add
-// panics if the receiver is lazy.
-func (l *methodList) Add(m *Func) {
- assert(!l.isLazy())
- if i, _ := lookupMethod(l.methods, m.pkg, m.name, false); i < 0 {
- l.methods = append(l.methods, m)
- }
-}
-
-// Lookup looks up the method identified by pkg and name in the receiver.
-// Lookup panics if the receiver is lazy. If foldCase is true, method names
-// are considered equal if they are equal with case folding.
-func (l *methodList) Lookup(pkg *Package, name string, foldCase bool) (int, *Func) {
- assert(!l.isLazy())
- if l == nil {
- return -1, nil
- }
- return lookupMethod(l.methods, pkg, name, foldCase)
-}
-
-// Len returns the length of the method list.
-func (l *methodList) Len() int {
- if l == nil {
- return 0
- }
- return len(l.methods)
-}
-
-// At returns the i'th method of the method list. At panics if i is out of
-// bounds, or if the receiver is lazy and resolve is nil.
-func (l *methodList) At(i int, resolve func() *Func) *Func {
- if !l.isLazy() {
- return l.methods[i]
- }
- assert(resolve != nil)
- (*l.guards)[i].Do(func() {
- l.methods[i] = resolve()
- })
- return l.methods[i]
-}
diff --git a/src/cmd/compile/internal/types2/methodlist_test.go b/src/cmd/compile/internal/types2/methodlist_test.go
deleted file mode 100644
index 7a183ac7f9..0000000000
--- a/src/cmd/compile/internal/types2/methodlist_test.go
+++ /dev/null
@@ -1,40 +0,0 @@
-// Copyright 2022 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package types2
-
-import (
- "testing"
-)
-
-func TestLazyMethodList(t *testing.T) {
- l := newLazyMethodList(2)
-
- if got := l.Len(); got != 2 {
- t.Fatalf("Len() = %d, want 2", got)
- }
-
- f0 := NewFunc(nopos, nil, "f0", nil)
- f1 := NewFunc(nopos, nil, "f1", nil)
-
- // Verify that methodList.At is idempotent, by calling it repeatedly with a
- // resolve func that returns different pointer values (f0 or f1).
- steps := []struct {
- index int
- resolve *Func // the *Func returned by the resolver
- want *Func // the actual *Func returned by methodList.At
- }{
- {0, f0, f0},
- {0, f1, f0},
- {1, f1, f1},
- {1, f0, f1},
- }
-
- for i, step := range steps {
- got := l.At(step.index, func() *Func { return step.resolve })
- if got != step.want {
- t.Errorf("step %d: At(%d, ...) = %s, want %s", i, step.index, got.Name(), step.want.Name())
- }
- }
-}
diff --git a/src/cmd/compile/internal/types2/named.go b/src/cmd/compile/internal/types2/named.go
index 849398a6f4..2cf6d3871f 100644
--- a/src/cmd/compile/internal/types2/named.go
+++ b/src/cmd/compile/internal/types2/named.go
@@ -7,30 +7,136 @@ package types2
import (
"cmd/compile/internal/syntax"
"sync"
+ "sync/atomic"
)
+// Type-checking Named types is subtle, because they may be recursively
+// defined, and because their full details may be spread across multiple
+// declarations (via methods). For this reason they are type-checked lazily,
+// to avoid information being accessed before it is complete.
+//
+// Conceptually, it is helpful to think of named types as having two distinct
+// sets of information:
+// - "LHS" information, defining their identity: Obj() and TypeArgs()
+// - "RHS" information, defining their details: TypeParams(), Underlying(),
+// and methods.
+//
+// In this taxonomy, LHS information is available immediately, but RHS
+// information is lazy. Specifically, a named type N may be constructed in any
+// of the following ways:
+// 1. type-checked from the source
+// 2. loaded eagerly from export data
+// 3. loaded lazily from export data (when using unified IR)
+// 4. instantiated from a generic type
+//
+// In cases 1, 3, and 4, it is possible that the underlying type or methods of
+// N may not be immediately available.
+// - During type-checking, we allocate N before type-checking its underlying
+// type or methods, so that we may resolve recursive references.
+// - When loading from export data, we may load its methods and underlying
+// type lazily using a provided load function.
+// - After instantiating, we lazily expand the underlying type and methods
+// (note that instances may be created while still in the process of
+// type-checking the original type declaration).
+//
+// In cases 3 and 4 this lazy construction may also occur concurrently, due to
+// concurrent use of the type checker API (after type checking or importing has
+// finished). It is critical that we keep track of state, so that Named types
+// are constructed exactly once and so that we do not access their details too
+// soon.
+//
+// We achieve this by tracking state with an atomic state variable, and
+// guarding potentially concurrent calculations with a mutex. At any point in
+// time this state variable determines which data on N may be accessed. As
+// state monotonically progresses, any data available at state M may be
+// accessed without acquiring the mutex at state N, provided N >= M.
+//
+// GLOSSARY: Here are a few terms used in this file to describe Named types:
+// - We say that a Named type is "instantiated" if it has been constructed by
+// instantiating a generic named type with type arguments.
+// - We say that a Named type is "declared" if it corresponds to a type
+// declaration in the source. Instantiated named types correspond to a type
+// instantiation in the source, not a declaration. But their Origin type is
+// a declared type.
+// - We say that a Named type is "resolved" if its RHS information has been
+// loaded or fully type-checked. For Named types constructed from export
+// data, this may involve invoking a loader function to extract information
+// from export data. For instantiated named types this involves reading
+// information from their origin.
+// - We say that a Named type is "expanded" if it is an instantiated type and
+// type parameters in its underlying type and methods have been substituted
+// with the type arguments from the instantiation. A type may be partially
+// expanded if some but not all of these details have been substituted.
+// Similarly, we refer to these individual details (underlying type or
+// method) as being "expanded".
+// - When all information is known for a named type, we say it is "complete".
+//
+// Some invariants to keep in mind: each declared Named type has a single
+// corresponding object, and that object's type is the (possibly generic) Named
+// type. Declared Named types are identical if and only if their pointers are
+// identical. On the other hand, multiple instantiated Named types may be
+// identical even though their pointers are not identical. One has to use
+// Identical to compare them. For instantiated named types, their obj is a
+// synthetic placeholder that records their position of the corresponding
+// instantiation in the source (if they were constructed during type checking).
+//
+// To prevent infinite expansion of named instances that are created outside of
+// type-checking, instances share a Context with other instances created during
+// their expansion. Via the pidgeonhole principle, this guarantees that in the
+// presence of a cycle of named types, expansion will eventually find an
+// existing instance in the Context and short-circuit the expansion.
+//
+// Once an instance is complete, we can nil out this shared Context to unpin
+// memory, though this Context may still be held by other incomplete instances
+// in its "lineage".
+
// A Named represents a named (defined) type.
type Named struct {
- check *Checker
- obj *TypeName // corresponding declared object for declared types; placeholder for instantiated types
- orig *Named // original, uninstantiated type
- fromRHS Type // type (on RHS of declaration) this *Named type is derived from (for cycle reporting)
+ check *Checker // non-nil during type-checking; nil otherwise
+ obj *TypeName // corresponding declared object for declared types; see above for instantiated types
+
+ // fromRHS holds the type (on RHS of declaration) this *Named type is derived
+ // from (for cycle reporting). Only used by validType, and therefore does not
+ // require synchronization.
+ fromRHS Type
+
+ // information for instantiated types; nil otherwise
+ inst *instance
+
+ mu sync.Mutex // guards all fields below
+ state_ uint32 // the current state of this type; must only be accessed atomically
underlying Type // possibly a *Named during setup; never a *Named once set up completely
tparams *TypeParamList // type parameters, or nil
- targs *TypeList // type arguments (after instantiation), or nil
- // methods declared for this type (not the method set of this type).
+ // methods declared for this type (not the method set of this type)
// Signatures are type-checked lazily.
// For non-instantiated types, this is a fully populated list of methods. For
- // instantiated types, this is a 'lazy' list, and methods are instantiated
- // when they are first accessed.
- methods *methodList
+ // instantiated types, methods are individually expanded when they are first
+ // accessed.
+ methods []*Func
- // resolver may be provided to lazily resolve type parameters, underlying, and methods.
- resolver func(*Context, *Named) (tparams *TypeParamList, underlying Type, methods *methodList)
- once sync.Once // ensures that tparams, underlying, and methods are resolved before accessing
+ // loader may be provided to lazily load type parameters, underlying type, and methods.
+ loader func(*Named) (tparams []*TypeParam, underlying Type, methods []*Func)
}
+// instance holds information that is only necessary for instantiated named
+// types.
+type instance struct {
+ orig *Named // original, uninstantiated type
+ targs *TypeList // type arguments
+ expandedMethods int // number of expanded methods; expandedMethods <= len(orig.methods)
+ ctxt *Context // local Context; set to nil after full expansion
+}
+
+// namedState represents the possible states that a named type may assume.
+type namedState uint32
+
+const (
+ unresolved namedState = iota // tparams, underlying type and methods might be unavailable
+ resolved // resolve has run; methods might be incomplete (for instances)
+ complete // all data is known
+)
+
// NewNamed returns a new named type for the given type name, underlying type, and associated methods.
// If the given type name obj doesn't have a type yet, its type is set to the returned named type.
// The underlying type must not be a *Named.
@@ -38,39 +144,122 @@ func NewNamed(obj *TypeName, underlying Type, methods []*Func) *Named {
if _, ok := underlying.(*Named); ok {
panic("underlying type must not be *Named")
}
- return (*Checker)(nil).newNamed(obj, nil, underlying, newMethodList(methods))
+ return (*Checker)(nil).newNamed(obj, underlying, methods)
}
-func (t *Named) resolve(ctxt *Context) *Named {
- if t.resolver == nil {
- return t
+// resolve resolves the type parameters, methods, and underlying type of n.
+// This information may be loaded from a provided loader function, or computed
+// from an origin type (in the case of instances).
+//
+// After resolution, the type parameters, methods, and underlying type of n are
+// accessible; but if n is an instantiated type, its methods may still be
+// unexpanded.
+func (n *Named) resolve() *Named {
+ if n.state() >= resolved { // avoid locking below
+ return n
}
- t.once.Do(func() {
- // TODO(mdempsky): Since we're passing t to the resolver anyway
- // (necessary because types2 expects the receiver type for methods
- // on defined interface types to be the Named rather than the
- // underlying Interface), maybe it should just handle calling
- // SetTypeParams, SetUnderlying, and AddMethod instead? Those
- // methods would need to support reentrant calls though. It would
- // also make the API more future-proof towards further extensions
- // (like SetTypeParams).
- t.tparams, t.underlying, t.methods = t.resolver(ctxt, t)
- t.fromRHS = t.underlying // for cycle detection
- })
- return t
+ // TODO(rfindley): if n.check is non-nil we can avoid locking here, since
+ // type-checking is not concurrent. Evaluate if this is worth doing.
+ n.mu.Lock()
+ defer n.mu.Unlock()
+
+ if n.state() >= resolved {
+ return n
+ }
+
+ if n.inst != nil {
+ assert(n.underlying == nil) // n is an unresolved instance
+ assert(n.loader == nil) // instances are created by instantiation, in which case n.loader is nil
+
+ orig := n.inst.orig
+ orig.resolve()
+ underlying := n.expandUnderlying()
+
+ n.tparams = orig.tparams
+ n.underlying = underlying
+ n.fromRHS = orig.fromRHS // for cycle detection
+
+ if len(orig.methods) == 0 {
+ n.setState(complete) // nothing further to do
+ n.inst.ctxt = nil
+ } else {
+ n.setState(resolved)
+ }
+ return n
+ }
+
+ // TODO(mdempsky): Since we're passing n to the loader anyway
+ // (necessary because types2 expects the receiver type for methods
+ // on defined interface types to be the Named rather than the
+ // underlying Interface), maybe it should just handle calling
+ // SetTypeParams, SetUnderlying, and AddMethod instead? Those
+ // methods would need to support reentrant calls though. It would
+ // also make the API more future-proof towards further extensions.
+ if n.loader != nil {
+ assert(n.underlying == nil)
+ assert(n.TypeArgs().Len() == 0) // instances are created by instantiation, in which case n.loader is nil
+
+ tparams, underlying, methods := n.loader(n)
+
+ n.tparams = bindTParams(tparams)
+ n.underlying = underlying
+ n.fromRHS = underlying // for cycle detection
+ n.methods = methods
+ n.loader = nil
+ }
+
+ n.setState(complete)
+ return n
+}
+
+// state atomically accesses the current state of the receiver.
+func (n *Named) state() namedState {
+ return namedState(atomic.LoadUint32(&n.state_))
+}
+
+// setState atomically stores the given state for n.
+// Must only be called while holding n.mu.
+func (n *Named) setState(state namedState) {
+ atomic.StoreUint32(&n.state_, uint32(state))
}
// newNamed is like NewNamed but with a *Checker receiver and additional orig argument.
-func (check *Checker) newNamed(obj *TypeName, orig *Named, underlying Type, methods *methodList) *Named {
- typ := &Named{check: check, obj: obj, orig: orig, fromRHS: underlying, underlying: underlying, methods: methods}
- if typ.orig == nil {
- typ.orig = typ
- }
+func (check *Checker) newNamed(obj *TypeName, underlying Type, methods []*Func) *Named {
+ typ := &Named{check: check, obj: obj, fromRHS: underlying, underlying: underlying, methods: methods}
if obj.typ == nil {
obj.typ = typ
}
- // Ensure that typ is always expanded and sanity-checked.
+ // Ensure that typ is always sanity-checked.
+ if check != nil {
+ check.needsCleanup(typ)
+ }
+ return typ
+}
+
+// newNamedInstance creates a new named instance for the given origin and type
+// arguments, recording pos as the position of its synthetic object (for error
+// reporting).
+//
+// If set, expanding is the named type instance currently being expanded, that
+// led to the creation of this instance.
+func (check *Checker) newNamedInstance(pos syntax.Pos, orig *Named, targs []Type, expanding *Named) *Named {
+ assert(len(targs) > 0)
+
+ obj := NewTypeName(pos, orig.obj.pkg, orig.obj.name, nil)
+ inst := &instance{orig: orig, targs: newTypeList(targs)}
+
+ // Only pass the expanding context to the new instance if their packages
+ // match. Since type reference cycles are only possible within a single
+ // package, this is sufficient for the purposes of short-circuiting cycles.
+ // Avoiding passing the context in other cases prevents unnecessary coupling
+ // of types across packages.
+ if expanding != nil && expanding.Obj().pkg == obj.pkg {
+ inst.ctxt = expanding.inst.ctxt
+ }
+ typ := &Named{check: check, obj: obj, inst: inst}
+ obj.typ = typ
+ // Ensure that typ is always sanity-checked.
if check != nil {
check.needsCleanup(typ)
}
@@ -78,18 +267,18 @@ func (check *Checker) newNamed(obj *TypeName, orig *Named, underlying Type, meth
}
func (t *Named) cleanup() {
- assert(t.orig.orig == t.orig)
+ assert(t.inst == nil || t.inst.orig.inst == nil)
// Ensure that every defined type created in the course of type-checking has
- // either non-*Named underlying, or is unresolved.
+ // either non-*Named underlying type, or is unexpanded.
//
- // This guarantees that we don't leak any types whose underlying is *Named,
- // because any unresolved instances will lazily compute their underlying by
- // substituting in the underlying of their origin. The origin must have
- // either been imported or type-checked and expanded here, and in either case
- // its underlying will be fully expanded.
+ // This guarantees that we don't leak any types whose underlying type is
+ // *Named, because any unexpanded instances will lazily compute their
+ // underlying type by substituting in the underlying type of their origin.
+ // The origin must have either been imported or type-checked and expanded
+ // here, and in either case its underlying type will be fully expanded.
switch t.underlying.(type) {
case nil:
- if t.resolver == nil {
+ if t.TypeArgs().Len() == 0 {
panic("nil underlying")
}
case *Named:
@@ -100,51 +289,91 @@ func (t *Named) cleanup() {
// Obj returns the type name for the declaration defining the named type t. For
// instantiated types, this is same as the type name of the origin type.
-func (t *Named) Obj() *TypeName { return t.orig.obj } // for non-instances this is the same as t.obj
+func (t *Named) Obj() *TypeName {
+ if t.inst == nil {
+ return t.obj
+ }
+ return t.inst.orig.obj
+}
// Origin returns the generic type from which the named type t is
// instantiated. If t is not an instantiated type, the result is t.
-func (t *Named) Origin() *Named { return t.orig }
-
-// TODO(gri) Come up with a better representation and API to distinguish
-// between parameterized instantiated and non-instantiated types.
+func (t *Named) Origin() *Named {
+ if t.inst == nil {
+ return t
+ }
+ return t.inst.orig
+}
// TypeParams returns the type parameters of the named type t, or nil.
// The result is non-nil for an (originally) generic type even if it is instantiated.
-func (t *Named) TypeParams() *TypeParamList { return t.resolve(nil).tparams }
+func (t *Named) TypeParams() *TypeParamList { return t.resolve().tparams }
// SetTypeParams sets the type parameters of the named type t.
// t must not have type arguments.
func (t *Named) SetTypeParams(tparams []*TypeParam) {
- assert(t.targs.Len() == 0)
- t.resolve(nil).tparams = bindTParams(tparams)
+ assert(t.inst == nil)
+ t.resolve().tparams = bindTParams(tparams)
}
// TypeArgs returns the type arguments used to instantiate the named type t.
-func (t *Named) TypeArgs() *TypeList { return t.targs }
-
-// NumMethods returns the number of explicit methods defined for t.
-//
-// For an ordinary or instantiated type t, the receiver base type of these
-// methods will be the named type t. For an uninstantiated generic type t, each
-// method receiver will be instantiated with its receiver type parameters.
-func (t *Named) NumMethods() int { return t.resolve(nil).methods.Len() }
-
-// Method returns the i'th method of named type t for 0 <= i < t.NumMethods().
-func (t *Named) Method(i int) *Func {
- t.resolve(nil)
- return t.methods.At(i, func() *Func {
- return t.instantiateMethod(i)
- })
+func (t *Named) TypeArgs() *TypeList {
+ if t.inst == nil {
+ return nil
+ }
+ return t.inst.targs
}
-// instantiateMethod instantiates the i'th method for an instantiated receiver.
-func (t *Named) instantiateMethod(i int) *Func {
- assert(t.TypeArgs().Len() > 0) // t must be an instance
+// NumMethods returns the number of explicit methods defined for t.
+func (t *Named) NumMethods() int {
+ return len(t.Origin().resolve().methods)
+}
+// Method returns the i'th method of named type t for 0 <= i < t.NumMethods().
+//
+// For an ordinary or instantiated type t, the receiver base type of this
+// method is the named type t. For an uninstantiated generic type t, each
+// method receiver is instantiated with its receiver type parameters.
+func (t *Named) Method(i int) *Func {
+ t.resolve()
+
+ if t.state() >= complete {
+ return t.methods[i]
+ }
+
+ assert(t.inst != nil) // only instances should have incomplete methods
+ orig := t.inst.orig
+
+ t.mu.Lock()
+ defer t.mu.Unlock()
+
+ if len(t.methods) != len(orig.methods) {
+ assert(len(t.methods) == 0)
+ t.methods = make([]*Func, len(orig.methods))
+ }
+
+ if t.methods[i] == nil {
+ assert(t.inst.ctxt != nil) // we should still have a context remaining from the resolution phase
+ t.methods[i] = t.expandMethod(i)
+ t.inst.expandedMethods++
+
+ // Check if we've created all methods at this point. If we have, mark the
+ // type as fully expanded.
+ if t.inst.expandedMethods == len(orig.methods) {
+ t.setState(complete)
+ t.inst.ctxt = nil // no need for a context anymore
+ }
+ }
+
+ return t.methods[i]
+}
+
+// expandMethod substitutes type arguments in the i'th method for an
+// instantiated receiver.
+func (t *Named) expandMethod(i int) *Func {
// t.orig.methods is not lazy. origm is the method instantiated with its
// receiver type parameters (the "origin" method).
- origm := t.orig.Method(i)
+ origm := t.inst.orig.Method(i)
assert(origm != nil)
check := t.check
@@ -171,10 +400,13 @@ func (t *Named) instantiateMethod(i int) *Func {
// We can only substitute if we have a correspondence between type arguments
// and type parameters. This check is necessary in the presence of invalid
// code.
- if origSig.RecvTypeParams().Len() == t.targs.Len() {
- ctxt := check.bestContext(nil)
- smap := makeSubstMap(origSig.RecvTypeParams().list(), t.targs.list())
- sig = check.subst(origm.pos, origSig, smap, ctxt).(*Signature)
+ if origSig.RecvTypeParams().Len() == t.inst.targs.Len() {
+ smap := makeSubstMap(origSig.RecvTypeParams().list(), t.inst.targs.list())
+ var ctxt *Context
+ if check != nil {
+ ctxt = check.context()
+ }
+ sig = check.subst(origm.pos, origSig, smap, t, ctxt).(*Signature)
}
if sig == origSig {
@@ -198,14 +430,14 @@ func (t *Named) instantiateMethod(i int) *Func {
// SetUnderlying sets the underlying type and marks t as complete.
// t must not have type arguments.
func (t *Named) SetUnderlying(underlying Type) {
- assert(t.targs.Len() == 0)
+ assert(t.inst == nil)
if underlying == nil {
panic("underlying type must not be nil")
}
if _, ok := underlying.(*Named); ok {
panic("underlying type must not be *Named")
}
- t.resolve(nil).underlying = underlying
+ t.resolve().underlying = underlying
if t.fromRHS == nil {
t.fromRHS = underlying // for cycle detection
}
@@ -214,19 +446,21 @@ func (t *Named) SetUnderlying(underlying Type) {
// AddMethod adds method m unless it is already in the method list.
// t must not have type arguments.
func (t *Named) AddMethod(m *Func) {
- assert(t.targs.Len() == 0)
- t.resolve(nil)
- if t.methods == nil {
- t.methods = newMethodList(nil)
+ assert(t.inst == nil)
+ t.resolve()
+ if i, _ := lookupMethod(t.methods, m.pkg, m.name, false); i < 0 {
+ t.methods = append(t.methods, m)
}
- t.methods.Add(m)
}
-func (t *Named) Underlying() Type { return t.resolve(nil).underlying }
+func (t *Named) Underlying() Type { return t.resolve().underlying }
func (t *Named) String() string { return TypeString(t, nil) }
// ----------------------------------------------------------------------------
// Implementation
+//
+// TODO(rfindley): reorganize the loading and expansion methods under this
+// heading.
// under returns the expanded underlying type of n0; possibly by following
// forward chains of named types. If an underlying type is found, resolve
@@ -275,7 +509,7 @@ func (n0 *Named) under() Type {
check := n0.check
n := n0
- seen := make(map[*Named]int) // types that need their underlying resolved
+ seen := make(map[*Named]int) // types that need their underlying type resolved
var path []Object // objects encountered, for cycle reporting
loop:
@@ -323,11 +557,11 @@ func (n *Named) setUnderlying(typ Type) {
}
func (n *Named) lookupMethod(pkg *Package, name string, foldCase bool) (int, *Func) {
- n.resolve(nil)
+ n.resolve()
// If n is an instance, we may not have yet instantiated all of its methods.
// Look up the method index in orig, and only instantiate method at the
// matching index (if any).
- i, _ := n.orig.methods.Lookup(pkg, name, foldCase)
+ i, _ := lookupMethod(n.Origin().methods, pkg, name, foldCase)
if i < 0 {
return -1, nil
}
@@ -335,83 +569,83 @@ func (n *Named) lookupMethod(pkg *Package, name string, foldCase bool) (int, *Fu
return i, n.Method(i)
}
-// bestContext returns the best available context. In order of preference:
-// - the given ctxt, if non-nil
-// - check.ctxt, if check is non-nil
-// - a new Context
-func (check *Checker) bestContext(ctxt *Context) *Context {
- if ctxt != nil {
- return ctxt
+// context returns the type-checker context.
+func (check *Checker) context() *Context {
+ if check.ctxt == nil {
+ check.ctxt = NewContext()
}
- if check != nil {
- if check.ctxt == nil {
- check.ctxt = NewContext()
- }
- return check.ctxt
- }
- return NewContext()
+ return check.ctxt
}
-// expandNamed ensures that the underlying type of n is instantiated.
-// The underlying type will be Typ[Invalid] if there was an error.
-func expandNamed(ctxt *Context, n *Named, instPos syntax.Pos) (tparams *TypeParamList, underlying Type, methods *methodList) {
+// expandUnderlying substitutes type arguments in the underlying type n.orig,
+// returning the result. Returns Typ[Invalid] if there was an error.
+func (n *Named) expandUnderlying() Type {
check := n.check
if check != nil && check.conf.Trace {
- check.trace(instPos, "-- expandNamed %s", n)
+ check.trace(n.obj.pos, "-- Named.expandUnderlying %s", n)
check.indent++
defer func() {
check.indent--
- check.trace(instPos, "=> %s (tparams = %s, under = %s)", n, tparams.list(), underlying)
+ check.trace(n.obj.pos, "=> %s (tparams = %s, under = %s)", n, n.tparams.list(), n.underlying)
}()
}
- n.orig.resolve(ctxt)
- assert(n.orig.underlying != nil)
+ assert(n.inst.orig.underlying != nil)
+ if n.inst.ctxt == nil {
+ n.inst.ctxt = NewContext()
+ }
- if _, unexpanded := n.orig.underlying.(*Named); unexpanded {
- // We should only get an unexpanded underlying here during type checking
+ orig := n.inst.orig
+ targs := n.inst.targs
+
+ if _, unexpanded := orig.underlying.(*Named); unexpanded {
+ // We should only get a Named underlying type here during type checking
// (for example, in recursive type declarations).
assert(check != nil)
}
- // Mismatching arg and tparam length may be checked elsewhere.
- if n.orig.tparams.Len() == n.targs.Len() {
- // We must always have a context, to avoid infinite recursion.
- ctxt = check.bestContext(ctxt)
- h := ctxt.instanceHash(n.orig, n.targs.list())
- // ensure that an instance is recorded for h to avoid infinite recursion.
- ctxt.update(h, n.orig, n.TypeArgs().list(), n)
-
- smap := makeSubstMap(n.orig.tparams.list(), n.targs.list())
- underlying = n.check.subst(instPos, n.orig.underlying, smap, ctxt)
- // If the underlying of n is an interface, we need to set the receiver of
- // its methods accurately -- we set the receiver of interface methods on
- // the RHS of a type declaration to the defined type.
- if iface, _ := underlying.(*Interface); iface != nil {
- if methods, copied := replaceRecvType(iface.methods, n.orig, n); copied {
- // If the underlying doesn't actually use type parameters, it's possible
- // that it wasn't substituted. In this case we need to create a new
- // *Interface before modifying receivers.
- if iface == n.orig.underlying {
- old := iface
- iface = check.newInterface()
- iface.embeddeds = old.embeddeds
- iface.complete = old.complete
- iface.implicit = old.implicit // should be false but be conservative
- underlying = iface
- }
- iface.methods = methods
- }
- }
- } else {
- underlying = Typ[Invalid]
+ if orig.tparams.Len() != targs.Len() {
+ // Mismatching arg and tparam length may be checked elsewhere.
+ return Typ[Invalid]
}
- return n.orig.tparams, underlying, newLazyMethodList(n.orig.methods.Len())
+ // Ensure that an instance is recorded before substituting, so that we
+ // resolve n for any recursive references.
+ h := n.inst.ctxt.instanceHash(orig, targs.list())
+ n2 := n.inst.ctxt.update(h, orig, n.TypeArgs().list(), n)
+ assert(n == n2)
+
+ smap := makeSubstMap(orig.tparams.list(), targs.list())
+ var ctxt *Context
+ if check != nil {
+ ctxt = check.context()
+ }
+ underlying := n.check.subst(n.obj.pos, orig.underlying, smap, n, ctxt)
+ // If the underlying type of n is an interface, we need to set the receiver of
+ // its methods accurately -- we set the receiver of interface methods on
+ // the RHS of a type declaration to the defined type.
+ if iface, _ := underlying.(*Interface); iface != nil {
+ if methods, copied := replaceRecvType(iface.methods, orig, n); copied {
+ // If the underlying type doesn't actually use type parameters, it's
+ // possible that it wasn't substituted. In this case we need to create
+ // a new *Interface before modifying receivers.
+ if iface == orig.underlying {
+ old := iface
+ iface = check.newInterface()
+ iface.embeddeds = old.embeddeds
+ iface.complete = old.complete
+ iface.implicit = old.implicit // should be false but be conservative
+ underlying = iface
+ }
+ iface.methods = methods
+ }
+ }
+
+ return underlying
}
-// safeUnderlying returns the underlying of typ without expanding instances, to
-// avoid infinite recursion.
+// safeUnderlying returns the underlying type of typ without expanding
+// instances, to avoid infinite recursion.
//
// TODO(rfindley): eliminate this function or give it a better name.
func safeUnderlying(typ Type) Type {
diff --git a/src/cmd/compile/internal/types2/named_test.go b/src/cmd/compile/internal/types2/named_test.go
new file mode 100644
index 0000000000..e5e8eddb05
--- /dev/null
+++ b/src/cmd/compile/internal/types2/named_test.go
@@ -0,0 +1,120 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package types2_test
+
+import (
+ "testing"
+
+ "cmd/compile/internal/syntax"
+ . "cmd/compile/internal/types2"
+)
+
+func BenchmarkNamed(b *testing.B) {
+ const src = `
+package p
+
+type T struct {
+ P int
+}
+
+func (T) M(int) {}
+func (T) N() (i int) { return }
+
+type G[P any] struct {
+ F P
+}
+
+func (G[P]) M(P) {}
+func (G[P]) N() (p P) { return }
+
+type Inst = G[int]
+ `
+ pkg, err := pkgFor("p", src, nil)
+ if err != nil {
+ b.Fatal(err)
+ }
+
+ var (
+ T = pkg.Scope().Lookup("T").Type()
+ G = pkg.Scope().Lookup("G").Type()
+ SrcInst = pkg.Scope().Lookup("Inst").Type()
+ UserInst = mustInstantiate(b, G, Typ[Int])
+ )
+
+ tests := []struct {
+ name string
+ typ Type
+ }{
+ {"nongeneric", T},
+ {"generic", G},
+ {"src instance", SrcInst},
+ {"user instance", UserInst},
+ }
+
+ b.Run("Underlying", func(b *testing.B) {
+ for _, test := range tests {
+ b.Run(test.name, func(b *testing.B) {
+ // Access underlying once, to trigger any lazy calculation.
+ _ = test.typ.Underlying()
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ _ = test.typ.Underlying()
+ }
+ })
+ }
+ })
+}
+
+func mustInstantiate(tb testing.TB, orig Type, targs ...Type) Type {
+ inst, err := Instantiate(nil, orig, targs, true)
+ if err != nil {
+ tb.Fatal(err)
+ }
+ return inst
+}
+
+// Test that types do not expand infinitely, as in golang/go#52715.
+func TestFiniteTypeExpansion(t *testing.T) {
+ const src = `
+package p
+
+type Tree[T any] struct {
+ *Node[T]
+}
+
+func (*Tree[R]) N(r R) R { return r }
+
+type Node[T any] struct {
+ *Tree[T]
+}
+
+func (Node[Q]) M(Q) {}
+
+type Inst = *Tree[int]
+`
+
+ f, err := parseSrc("foo.go", src)
+ if err != nil {
+ t.Fatal(err)
+ }
+ pkg := NewPackage("p", f.PkgName.Value)
+ if err := NewChecker(nil, pkg, nil).Files([]*syntax.File{f}); err != nil {
+ t.Fatal(err)
+ }
+
+ firstFieldType := func(n *Named) *Named {
+ return n.Underlying().(*Struct).Field(0).Type().(*Pointer).Elem().(*Named)
+ }
+
+ Inst := pkg.Scope().Lookup("Inst").Type().(*Pointer).Elem().(*Named)
+ Node := firstFieldType(Inst)
+ Tree := firstFieldType(Node)
+ if !Identical(Inst, Tree) {
+ t.Fatalf("Not a cycle: got %v, want %v", Tree, Inst)
+ }
+ if Inst != Tree {
+ t.Errorf("Duplicate instances in cycle: %s (%p) -> %s (%p) -> %s (%p)", Inst, Inst, Node, Node, Tree, Tree)
+ }
+}
diff --git a/src/cmd/compile/internal/types2/object.go b/src/cmd/compile/internal/types2/object.go
index 75f7ea5b12..df080f071c 100644
--- a/src/cmd/compile/internal/types2/object.go
+++ b/src/cmd/compile/internal/types2/object.go
@@ -279,19 +279,7 @@ func NewTypeName(pos syntax.Pos, pkg *Package, name string, typ Type) *TypeName
// lazily calls resolve to finish constructing the Named object.
func NewTypeNameLazy(pos syntax.Pos, pkg *Package, name string, load func(named *Named) (tparams []*TypeParam, underlying Type, methods []*Func)) *TypeName {
obj := NewTypeName(pos, pkg, name, nil)
-
- resolve := func(_ *Context, t *Named) (*TypeParamList, Type, *methodList) {
- tparams, underlying, methods := load(t)
-
- switch underlying.(type) {
- case nil, *Named:
- panic(fmt.Sprintf("invalid underlying type %T", t.underlying))
- }
-
- return bindTParams(tparams), underlying, newMethodList(methods)
- }
-
- NewNamed(obj, nil, nil).resolver = resolve
+ NewNamed(obj, nil, nil).loader = load
return obj
}
diff --git a/src/cmd/compile/internal/types2/predicates.go b/src/cmd/compile/internal/types2/predicates.go
index 6bce26137e..f7b5b16204 100644
--- a/src/cmd/compile/internal/types2/predicates.go
+++ b/src/cmd/compile/internal/types2/predicates.go
@@ -102,7 +102,7 @@ func isTypeParam(t Type) bool {
func isGeneric(t Type) bool {
// A parameterized type is only generic if it doesn't have an instantiation already.
named, _ := t.(*Named)
- return named != nil && named.obj != nil && named.targs == nil && named.TypeParams() != nil
+ return named != nil && named.obj != nil && named.inst == nil && named.TypeParams().Len() > 0
}
// Comparable reports whether values of type T are comparable.
@@ -283,18 +283,19 @@ func identical(x, y Type, cmpTags bool, p *ifacePair) bool {
}
smap := makeSubstMap(ytparams, targs)
- var check *Checker // ok to call subst on a nil *Checker
+ var check *Checker // ok to call subst on a nil *Checker
+ ctxt := NewContext() // need a non-nil Context for the substitution below
// Constraints must be pair-wise identical, after substitution.
for i, xtparam := range xtparams {
- ybound := check.subst(nopos, ytparams[i].bound, smap, nil)
+ ybound := check.subst(nopos, ytparams[i].bound, smap, nil, ctxt)
if !identical(xtparam.bound, ybound, cmpTags, p) {
return false
}
}
- yparams = check.subst(nopos, y.params, smap, nil).(*Tuple)
- yresults = check.subst(nopos, y.results, smap, nil).(*Tuple)
+ yparams = check.subst(nopos, y.params, smap, nil, ctxt).(*Tuple)
+ yresults = check.subst(nopos, y.results, smap, nil, ctxt).(*Tuple)
}
return x.variadic == y.variadic &&
@@ -401,7 +402,7 @@ func identical(x, y Type, cmpTags bool, p *ifacePair) bool {
if len(xargs) > 0 {
// Instances are identical if their original type and type arguments
// are identical.
- if !Identical(x.orig, y.orig) {
+ if !Identical(x.Origin(), y.Origin()) {
return false
}
for i, xa := range xargs {
diff --git a/src/cmd/compile/internal/types2/signature.go b/src/cmd/compile/internal/types2/signature.go
index 92d3aadf88..1b61b368d2 100644
--- a/src/cmd/compile/internal/types2/signature.go
+++ b/src/cmd/compile/internal/types2/signature.go
@@ -143,7 +143,7 @@ func (check *Checker) funcType(sig *Signature, recvPar *syntax.Field, tparams []
// recvTPar.bound is (possibly) parameterized in the context of the
// receiver type declaration. Substitute parameters for the current
// context.
- tpar.bound = check.subst(tpar.obj.pos, recvTPar.bound, smap, nil)
+ tpar.bound = check.subst(tpar.obj.pos, recvTPar.bound, smap, nil, check.context())
}
} else if len(tparams) < len(recvTParams) {
// Reporting an error here is a stop-gap measure to avoid crashes in the
diff --git a/src/cmd/compile/internal/types2/sizeof_test.go b/src/cmd/compile/internal/types2/sizeof_test.go
index 7ab7abb317..17876d1f3c 100644
--- a/src/cmd/compile/internal/types2/sizeof_test.go
+++ b/src/cmd/compile/internal/types2/sizeof_test.go
@@ -31,7 +31,7 @@ func TestSizeof(t *testing.T) {
{Interface{}, 40, 80},
{Map{}, 16, 32},
{Chan{}, 12, 24},
- {Named{}, 56, 104},
+ {Named{}, 60, 112},
{TypeParam{}, 28, 48},
{term{}, 12, 24},
diff --git a/src/cmd/compile/internal/types2/subst.go b/src/cmd/compile/internal/types2/subst.go
index 6e41ebdf53..d5a48c6995 100644
--- a/src/cmd/compile/internal/types2/subst.go
+++ b/src/cmd/compile/internal/types2/subst.go
@@ -48,8 +48,11 @@ func (m substMap) lookup(tpar *TypeParam) Type {
// incoming type. If a substitution took place, the result type is different
// from the incoming type.
//
-// If the given context is non-nil, it is used in lieu of check.Config.Context.
-func (check *Checker) subst(pos syntax.Pos, typ Type, smap substMap, ctxt *Context) Type {
+// If expanding is non-nil, it is the instance type currently being expanded.
+// One of expanding or ctxt must be non-nil.
+func (check *Checker) subst(pos syntax.Pos, typ Type, smap substMap, expanding *Named, ctxt *Context) Type {
+ assert(expanding != nil || ctxt != nil)
+
if smap.empty() {
return typ
}
@@ -64,19 +67,21 @@ func (check *Checker) subst(pos syntax.Pos, typ Type, smap substMap, ctxt *Conte
// general case
subst := subster{
- pos: pos,
- smap: smap,
- check: check,
- ctxt: check.bestContext(ctxt),
+ pos: pos,
+ smap: smap,
+ check: check,
+ expanding: expanding,
+ ctxt: ctxt,
}
return subst.typ(typ)
}
type subster struct {
- pos syntax.Pos
- smap substMap
- check *Checker // nil if called via Instantiate
- ctxt *Context
+ pos syntax.Pos
+ smap substMap
+ check *Checker // nil if called via Instantiate
+ expanding *Named // if non-nil, the instance that is being expanded
+ ctxt *Context
}
func (subst *subster) typ(typ Type) Type {
@@ -176,7 +181,7 @@ func (subst *subster) typ(typ Type) Type {
// In this case the interface will not be substituted here, because its
// method signatures do not depend on the type parameter P, but we still
// need to create new interface methods to hold the instantiated
- // receiver. This is handled by expandNamed.
+ // receiver. This is handled by Named.expandUnderlying.
iface.methods, _ = replaceRecvType(methods, t, iface)
return iface
}
@@ -207,19 +212,20 @@ func (subst *subster) typ(typ Type) Type {
}
}
- // subst is called by expandNamed, so in this function we need to be
+ // subst is called during expansion, so in this function we need to be
// careful not to call any methods that would cause t to be expanded: doing
// so would result in deadlock.
//
- // So we call t.orig.TypeParams() rather than t.TypeParams() here and
- // below.
- if t.orig.TypeParams().Len() == 0 {
+ // So we call t.Origin().TypeParams() rather than t.TypeParams().
+ orig := t.Origin()
+ n := orig.TypeParams().Len()
+ if n == 0 {
dump(">>> %s is not parameterized", t)
return t // type is not parameterized
}
var newTArgs []Type
- if t.targs.Len() != t.orig.TypeParams().Len() {
+ if t.TypeArgs().Len() != n {
return Typ[Invalid] // error reported elsewhere
}
@@ -228,14 +234,14 @@ func (subst *subster) typ(typ Type) Type {
// For each (existing) type argument targ, determine if it needs
// to be substituted; i.e., if it is or contains a type parameter
// that has a type argument for it.
- for i, targ := range t.targs.list() {
+ for i, targ := range t.TypeArgs().list() {
dump(">>> %d targ = %s", i, targ)
new_targ := subst.typ(targ)
if new_targ != targ {
dump(">>> substituted %d targ %s => %s", i, targ, new_targ)
if newTArgs == nil {
- newTArgs = make([]Type, t.orig.TypeParams().Len())
- copy(newTArgs, t.targs.list())
+ newTArgs = make([]Type, n)
+ copy(newTArgs, t.TypeArgs().list())
}
newTArgs[i] = new_targ
}
@@ -246,25 +252,11 @@ func (subst *subster) typ(typ Type) Type {
return t // nothing to substitute
}
- // before creating a new named type, check if we have this one already
- h := subst.ctxt.instanceHash(t.orig, newTArgs)
- dump(">>> new type hash: %s", h)
- if named := subst.ctxt.lookup(h, t.orig, newTArgs); named != nil {
- dump(">>> found %s", named)
- return named
- }
-
// Create a new instance and populate the context to avoid endless
// recursion. The position used here is irrelevant because validation only
// occurs on t (we don't call validType on named), but we use subst.pos to
// help with debugging.
- return subst.check.instance(subst.pos, t.orig, newTArgs, subst.ctxt)
-
- // Note that if we were to expose substitution more generally (not just in
- // the context of a declaration), we'd have to substitute in
- // named.underlying as well.
- //
- // But this is unnecessary for now.
+ return subst.check.instance(subst.pos, orig, newTArgs, subst.expanding, subst.ctxt)
case *TypeParam:
return subst.smap.lookup(t)
diff --git a/src/cmd/compile/internal/types2/termlist.go b/src/cmd/compile/internal/types2/termlist.go
index a0108c4638..43e43ce87c 100644
--- a/src/cmd/compile/internal/types2/termlist.go
+++ b/src/cmd/compile/internal/types2/termlist.go
@@ -25,7 +25,7 @@ func (xl termlist) String() string {
var buf bytes.Buffer
for i, x := range xl {
if i > 0 {
- buf.WriteString(" ∪ ")
+ buf.WriteString(" | ")
}
buf.WriteString(x.String())
}
diff --git a/src/cmd/compile/internal/types2/termlist_test.go b/src/cmd/compile/internal/types2/termlist_test.go
index d1e3bdf88e..3005d0edea 100644
--- a/src/cmd/compile/internal/types2/termlist_test.go
+++ b/src/cmd/compile/internal/types2/termlist_test.go
@@ -12,7 +12,7 @@ import (
// maketl makes a term list from a string of the term list.
func maketl(s string) termlist {
s = strings.ReplaceAll(s, " ", "")
- names := strings.Split(s, "∪")
+ names := strings.Split(s, "|")
r := make(termlist, len(names))
for i, n := range names {
r[i] = testTerm(n)
@@ -33,10 +33,10 @@ func TestTermlistString(t *testing.T) {
"int",
"~int",
"myInt",
- "∅ ∪ ∅",
- "𝓤 ∪ 𝓤",
- "∅ ∪ 𝓤 ∪ int",
- "∅ ∪ 𝓤 ∪ int ∪ myInt",
+ "∅ | ∅",
+ "𝓤 | 𝓤",
+ "∅ | 𝓤 | int",
+ "∅ | 𝓤 | int | myInt",
} {
if got := maketl(want).String(); got != want {
t.Errorf("(%v).String() == %v", want, got)
@@ -47,12 +47,12 @@ func TestTermlistString(t *testing.T) {
func TestTermlistIsEmpty(t *testing.T) {
for test, want := range map[string]bool{
"∅": true,
- "∅ ∪ ∅": true,
- "∅ ∪ ∅ ∪ 𝓤": false,
- "∅ ∪ ∅ ∪ myInt": false,
+ "∅ | ∅": true,
+ "∅ | ∅ | 𝓤": false,
+ "∅ | ∅ | myInt": false,
"𝓤": false,
- "𝓤 ∪ int": false,
- "𝓤 ∪ myInt ∪ ∅": false,
+ "𝓤 | int": false,
+ "𝓤 | myInt | ∅": false,
} {
xl := maketl(test)
got := xl.isEmpty()
@@ -65,13 +65,13 @@ func TestTermlistIsEmpty(t *testing.T) {
func TestTermlistIsAll(t *testing.T) {
for test, want := range map[string]bool{
"∅": false,
- "∅ ∪ ∅": false,
- "int ∪ ~string": false,
- "~int ∪ myInt": false,
- "∅ ∪ ∅ ∪ 𝓤": true,
+ "∅ | ∅": false,
+ "int | ~string": false,
+ "~int | myInt": false,
+ "∅ | ∅ | 𝓤": true,
"𝓤": true,
- "𝓤 ∪ int": true,
- "myInt ∪ 𝓤": true,
+ "𝓤 | int": true,
+ "myInt | 𝓤": true,
} {
xl := maketl(test)
got := xl.isAll()
@@ -86,17 +86,17 @@ func TestTermlistNorm(t *testing.T) {
xl, want string
}{
{"∅", "∅"},
- {"∅ ∪ ∅", "∅"},
- {"∅ ∪ int", "int"},
- {"∅ ∪ myInt", "myInt"},
- {"𝓤 ∪ int", "𝓤"},
- {"𝓤 ∪ myInt", "𝓤"},
- {"int ∪ myInt", "int ∪ myInt"},
- {"~int ∪ int", "~int"},
- {"~int ∪ myInt", "~int"},
- {"int ∪ ~string ∪ int", "int ∪ ~string"},
- {"~int ∪ string ∪ 𝓤 ∪ ~string ∪ int", "𝓤"},
- {"~int ∪ string ∪ myInt ∪ ~string ∪ int", "~int ∪ ~string"},
+ {"∅ | ∅", "∅"},
+ {"∅ | int", "int"},
+ {"∅ | myInt", "myInt"},
+ {"𝓤 | int", "𝓤"},
+ {"𝓤 | myInt", "𝓤"},
+ {"int | myInt", "int | myInt"},
+ {"~int | int", "~int"},
+ {"~int | myInt", "~int"},
+ {"int | ~string | int", "int | ~string"},
+ {"~int | string | 𝓤 | ~string | int", "𝓤"},
+ {"~int | string | myInt | ~string | int", "~int | ~string"},
} {
xl := maketl(test.xl)
got := maketl(test.xl).norm()
@@ -116,15 +116,15 @@ func TestTermlistUnion(t *testing.T) {
{"∅", "int", "int"},
{"𝓤", "~int", "𝓤"},
{"int", "~int", "~int"},
- {"int", "string", "int ∪ string"},
- {"int", "myInt", "int ∪ myInt"},
+ {"int", "string", "int | string"},
+ {"int", "myInt", "int | myInt"},
{"~int", "myInt", "~int"},
- {"int ∪ string", "~string", "int ∪ ~string"},
- {"~int ∪ string", "~string ∪ int", "~int ∪ ~string"},
- {"~int ∪ string ∪ ∅", "~string ∪ int", "~int ∪ ~string"},
- {"~int ∪ myInt ∪ ∅", "~string ∪ int", "~int ∪ ~string"},
- {"~int ∪ string ∪ 𝓤", "~string ∪ int", "𝓤"},
- {"~int ∪ string ∪ myInt", "~string ∪ int", "~int ∪ ~string"},
+ {"int | string", "~string", "int | ~string"},
+ {"~int | string", "~string | int", "~int | ~string"},
+ {"~int | string | ∅", "~string | int", "~int | ~string"},
+ {"~int | myInt | ∅", "~string | int", "~int | ~string"},
+ {"~int | string | 𝓤", "~string | int", "𝓤"},
+ {"~int | string | myInt", "~string | int", "~int | ~string"},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
@@ -150,12 +150,12 @@ func TestTermlistIntersect(t *testing.T) {
{"int", "string", "∅"},
{"int", "myInt", "∅"},
{"~int", "myInt", "myInt"},
- {"int ∪ string", "~string", "string"},
- {"~int ∪ string", "~string ∪ int", "int ∪ string"},
- {"~int ∪ string ∪ ∅", "~string ∪ int", "int ∪ string"},
- {"~int ∪ myInt ∪ ∅", "~string ∪ int", "int"},
- {"~int ∪ string ∪ 𝓤", "~string ∪ int", "int ∪ ~string"},
- {"~int ∪ string ∪ myInt", "~string ∪ int", "int ∪ string"},
+ {"int | string", "~string", "string"},
+ {"~int | string", "~string | int", "int | string"},
+ {"~int | string | ∅", "~string | int", "int | string"},
+ {"~int | myInt | ∅", "~string | int", "int"},
+ {"~int | string | 𝓤", "~string | int", "int | ~string"},
+ {"~int | string | myInt", "~string | int", "int | string"},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
@@ -174,12 +174,12 @@ func TestTermlistEqual(t *testing.T) {
{"∅", "∅", true},
{"∅", "𝓤", false},
{"𝓤", "𝓤", true},
- {"𝓤 ∪ int", "𝓤", true},
- {"𝓤 ∪ int", "string ∪ 𝓤", true},
- {"𝓤 ∪ myInt", "string ∪ 𝓤", true},
- {"int ∪ ~string", "string ∪ int", false},
- {"~int ∪ string", "string ∪ myInt", false},
- {"int ∪ ~string ∪ ∅", "string ∪ int ∪ ~string", true},
+ {"𝓤 | int", "𝓤", true},
+ {"𝓤 | int", "string | 𝓤", true},
+ {"𝓤 | myInt", "string | 𝓤", true},
+ {"int | ~string", "string | int", false},
+ {"~int | string", "string | myInt", false},
+ {"int | ~string | ∅", "string | int | ~string", true},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
@@ -201,11 +201,11 @@ func TestTermlistIncludes(t *testing.T) {
{"int", "string", false},
{"~int", "string", false},
{"~int", "myInt", true},
- {"int ∪ string", "string", true},
- {"~int ∪ string", "int", true},
- {"~int ∪ string", "myInt", true},
- {"~int ∪ myInt ∪ ∅", "myInt", true},
- {"myInt ∪ ∅ ∪ 𝓤", "int", true},
+ {"int | string", "string", true},
+ {"~int | string", "int", true},
+ {"~int | string", "myInt", true},
+ {"~int | myInt | ∅", "myInt", true},
+ {"myInt | ∅ | 𝓤", "int", true},
} {
xl := maketl(test.xl)
yl := testTerm(test.typ).typ
@@ -236,12 +236,12 @@ func TestTermlistSupersetOf(t *testing.T) {
{"myInt", "~int", false},
{"int", "string", false},
{"~int", "string", false},
- {"int ∪ string", "string", true},
- {"int ∪ string", "~string", false},
- {"~int ∪ string", "int", true},
- {"~int ∪ string", "myInt", true},
- {"~int ∪ string ∪ ∅", "string", true},
- {"~string ∪ ∅ ∪ 𝓤", "myInt", true},
+ {"int | string", "string", true},
+ {"int | string", "~string", false},
+ {"~int | string", "int", true},
+ {"~int | string", "myInt", true},
+ {"~int | string | ∅", "string", true},
+ {"~string | ∅ | 𝓤", "myInt", true},
} {
xl := maketl(test.xl)
y := testTerm(test.typ)
@@ -261,18 +261,18 @@ func TestTermlistSubsetOf(t *testing.T) {
{"∅", "𝓤", true},
{"𝓤", "∅", false},
{"𝓤", "𝓤", true},
- {"int", "int ∪ string", true},
- {"~int", "int ∪ string", false},
- {"~int", "myInt ∪ string", false},
- {"myInt", "~int ∪ string", true},
- {"~int", "string ∪ string ∪ int ∪ ~int", true},
- {"myInt", "string ∪ string ∪ ~int", true},
- {"int ∪ string", "string", false},
- {"int ∪ string", "string ∪ int", true},
- {"int ∪ ~string", "string ∪ int", false},
- {"myInt ∪ ~string", "string ∪ int ∪ 𝓤", true},
- {"int ∪ ~string", "string ∪ int ∪ ∅ ∪ string", false},
- {"int ∪ myInt", "string ∪ ~int ∪ ∅ ∪ string", true},
+ {"int", "int | string", true},
+ {"~int", "int | string", false},
+ {"~int", "myInt | string", false},
+ {"myInt", "~int | string", true},
+ {"~int", "string | string | int | ~int", true},
+ {"myInt", "string | string | ~int", true},
+ {"int | string", "string", false},
+ {"int | string", "string | int", true},
+ {"int | ~string", "string | int", false},
+ {"myInt | ~string", "string | int | 𝓤", true},
+ {"int | ~string", "string | int | ∅ | string", false},
+ {"int | myInt", "string | ~int | ∅ | string", true},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
diff --git a/src/cmd/compile/internal/types2/testdata/fixedbugs/issue40350.go b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue40350.go
new file mode 100644
index 0000000000..7ffd551c2e
--- /dev/null
+++ b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue40350.go
@@ -0,0 +1,16 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+type number interface {
+ ~float64 | ~int | ~int32
+ float64 | ~int32
+}
+
+func f[T number]() {}
+
+func _() {
+ _ = f[int /* ERROR int does not implement number \(int missing in float64 | ~int32\)*/]
+}
diff --git a/src/cmd/compile/internal/types2/testdata/fixedbugs/issue42881.go b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue42881.go
new file mode 100644
index 0000000000..7122d1c787
--- /dev/null
+++ b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue42881.go
@@ -0,0 +1,16 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+type (
+ T1 interface{ comparable }
+ T2 interface{ int }
+)
+
+var (
+ _ comparable // ERROR cannot use type comparable outside a type constraint: interface is \(or embeds\) comparable
+ _ T1 // ERROR cannot use type T1 outside a type constraint: interface is \(or embeds\) comparable
+ _ T2 // ERROR cannot use type T2 outside a type constraint: interface contains type constraints
+)
diff --git a/src/cmd/compile/internal/types2/testdata/fixedbugs/issue50729.go b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue50729.go
new file mode 100644
index 0000000000..fe19fdfa68
--- /dev/null
+++ b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue50729.go
@@ -0,0 +1,19 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+// version 1
+var x1 T1[B1]
+
+type T1[_ any] struct{}
+type A1 T1[B1]
+type B1 = T1[A1]
+
+// version 2
+type T2[_ any] struct{}
+type A2 T2[B2]
+type B2 = T2[A2]
+
+var x2 T2[B2]
diff --git a/src/cmd/compile/internal/types2/testdata/fixedbugs/issue52698.go b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue52698.go
new file mode 100644
index 0000000000..d1b06a210d
--- /dev/null
+++ b/src/cmd/compile/internal/types2/testdata/fixedbugs/issue52698.go
@@ -0,0 +1,62 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+// correctness check: ensure that cycles through generic instantiations are detected
+type T[P any] struct {
+ _ P
+}
+
+type S /* ERROR illegal cycle */ struct {
+ _ T[S]
+}
+
+// simplified test 1
+
+var _ A1[A1[string]]
+
+type A1[P any] struct {
+ _ B1[P]
+}
+
+type B1[P any] struct {
+ _ P
+}
+
+// simplified test 2
+var _ B2[A2]
+
+type A2 struct {
+ _ B2[string]
+}
+
+type B2[P any] struct {
+ _ C2[P]
+}
+
+type C2[P any] struct {
+ _ P
+}
+
+// test case from issue
+type T23 interface {
+ ~struct {
+ Field0 T13[T15]
+ }
+}
+
+type T1[P1 interface {
+}] struct {
+ Field2 P1
+}
+
+type T13[P2 interface {
+}] struct {
+ Field2 T1[P2]
+}
+
+type T15 struct {
+ Field0 T13[string]
+}
diff --git a/src/cmd/compile/internal/types2/typeset_test.go b/src/cmd/compile/internal/types2/typeset_test.go
index 69eaff741f..40ca28e525 100644
--- a/src/cmd/compile/internal/types2/typeset_test.go
+++ b/src/cmd/compile/internal/types2/typeset_test.go
@@ -21,13 +21,13 @@ func TestTypeSetString(t *testing.T) {
"{}": "𝓤",
"{int}": "{int}",
"{~int}": "{~int}",
- "{int|string}": "{int ∪ string}",
+ "{int|string}": "{int | string}",
"{int; string}": "∅",
"{comparable}": "{comparable}",
"{comparable; int}": "{int}",
"{~int; comparable}": "{~int}",
- "{int|string; comparable}": "{int ∪ string}",
+ "{int|string; comparable}": "{int | string}",
"{comparable; int; string}": "∅",
"{m()}": "{func (p.T).m()}",
@@ -37,7 +37,7 @@ func TestTypeSetString(t *testing.T) {
"{m1(); comparable; m2() int }": "{comparable; func (p.T).m1(); func (p.T).m2() int}",
"{comparable; error}": "{comparable; func (error).Error() string}",
- "{m(); comparable; int|float32|string}": "{func (p.T).m(); int ∪ float32 ∪ string}",
+ "{m(); comparable; int|float32|string}": "{func (p.T).m(); int | float32 | string}",
"{m1(); int; m2(); comparable }": "{func (p.T).m1(); func (p.T).m2(); int}",
"{E}; type E interface{}": "𝓤",
diff --git a/src/cmd/compile/internal/types2/typestring.go b/src/cmd/compile/internal/types2/typestring.go
index e0f36fcec4..c10c2d8973 100644
--- a/src/cmd/compile/internal/types2/typestring.go
+++ b/src/cmd/compile/internal/types2/typestring.go
@@ -284,9 +284,9 @@ func (w *typeWriter) typ(typ Type) {
w.string(strconv.Itoa(w.ctxt.getID(t)))
}
w.typeName(t.obj) // when hashing written for readability of the hash only
- if t.targs != nil {
+ if t.inst != nil {
// instantiated type
- w.typeList(t.targs.list())
+ w.typeList(t.inst.targs.list())
} else if w.ctxt == nil && t.TypeParams().Len() != 0 { // For type hashing, don't need to format the TypeParams
// parameterized type
w.tParamList(t.TypeParams().list())
diff --git a/src/cmd/compile/internal/types2/typexpr.go b/src/cmd/compile/internal/types2/typexpr.go
index 1f8b40dba6..692feb9751 100644
--- a/src/cmd/compile/internal/types2/typexpr.go
+++ b/src/cmd/compile/internal/types2/typexpr.go
@@ -167,9 +167,9 @@ func (check *Checker) validVarType(e syntax.Expr, typ Type) {
tset := computeInterfaceTypeSet(check, pos, t) // TODO(gri) is this the correct position?
if !tset.IsMethodSet() {
if tset.comparable {
- check.softErrorf(pos, "interface is (or embeds) comparable")
+ check.softErrorf(pos, "cannot use type %s outside a type constraint: interface is (or embeds) comparable", typ)
} else {
- check.softErrorf(pos, "interface contains type constraints")
+ check.softErrorf(pos, "cannot use type %s outside a type constraint: interface contains type constraints", typ)
}
}
}
@@ -432,57 +432,19 @@ func (check *Checker) instantiatedType(x syntax.Expr, xlist []syntax.Expr, def *
return Typ[Invalid]
}
- // enableTypeTypeInference controls whether to infer missing type arguments
- // using constraint type inference. See issue #51527.
- const enableTypeTypeInference = false
-
// create the instance
- ctxt := check.bestContext(nil)
- h := ctxt.instanceHash(orig, targs)
- // targs may be incomplete, and require inference. In any case we should de-duplicate.
- inst, _ := ctxt.lookup(h, orig, targs).(*Named)
- // If inst is non-nil, we can't just return here. Inst may have been
- // constructed via recursive substitution, in which case we wouldn't do the
- // validation below. Ensure that the validation (and resulting errors) runs
- // for each instantiated type in the source.
- if inst == nil {
- // x may be a selector for an imported type; use its start pos rather than x.Pos().
- tname := NewTypeName(syntax.StartPos(x), orig.obj.pkg, orig.obj.name, nil)
- inst = check.newNamed(tname, orig, nil, nil) // underlying, methods and tparams are set when named is resolved
- inst.targs = newTypeList(targs)
- inst = ctxt.update(h, orig, targs, inst).(*Named)
- }
+ inst := check.instance(x.Pos(), orig, targs, nil, check.context()).(*Named)
def.setUnderlying(inst)
- inst.resolver = func(ctxt *Context, n *Named) (*TypeParamList, Type, *methodList) {
- tparams := n.orig.TypeParams().list()
-
- targs := n.targs.list()
- if enableTypeTypeInference && len(targs) < len(tparams) {
- // If inference fails, len(inferred) will be 0, and inst.underlying will
- // be set to Typ[Invalid] in expandNamed.
- inferred := check.infer(x.Pos(), tparams, targs, nil, nil)
- if len(inferred) > len(targs) {
- n.targs = newTypeList(inferred)
- }
- }
-
- return expandNamed(ctxt, n, x.Pos())
- }
-
// orig.tparams may not be set up, so we need to do expansion later.
check.later(func() {
// This is an instance from the source, not from recursive substitution,
// and so it must be resolved during type-checking so that we can report
// errors.
- inst.resolve(ctxt)
- // Since check is non-nil, we can still mutate inst. Unpinning the resolver
- // frees some memory.
- inst.resolver = nil
check.recordInstance(x, inst.TypeArgs().list(), inst)
- if check.validateTArgLen(x.Pos(), inst.tparams.Len(), inst.targs.Len()) {
- if i, err := check.verify(x.Pos(), inst.tparams.list(), inst.targs.list()); err != nil {
+ if check.validateTArgLen(x.Pos(), inst.TypeParams().Len(), inst.TypeArgs().Len()) {
+ if i, err := check.verify(x.Pos(), inst.TypeParams().list(), inst.TypeArgs().list(), check.context()); err != nil {
// best position for error reporting
pos := x.Pos()
if i < len(xlist) {
@@ -490,10 +452,13 @@ func (check *Checker) instantiatedType(x syntax.Expr, xlist []syntax.Expr, def *
}
check.softErrorf(pos, "%s", err)
} else {
- check.mono.recordInstance(check.pkg, x.Pos(), inst.tparams.list(), inst.targs.list(), xlist)
+ check.mono.recordInstance(check.pkg, x.Pos(), inst.TypeParams().list(), inst.TypeArgs().list(), xlist)
}
}
+ // TODO(rfindley): remove this call: we don't need to call validType here,
+ // as cycles can only occur for types used inside a Named type declaration,
+ // and so it suffices to call validType from declared types.
check.validType(inst)
}).describef(x, "resolve instance %s", inst)
diff --git a/src/cmd/compile/internal/types2/unify.go b/src/cmd/compile/internal/types2/unify.go
index a7f68a05b1..7063789f3f 100644
--- a/src/cmd/compile/internal/types2/unify.go
+++ b/src/cmd/compile/internal/types2/unify.go
@@ -546,8 +546,8 @@ func (u *unifier) nify(x, y Type, p *ifacePair) (result bool) {
case *Named:
// TODO(gri) This code differs now from the parallel code in Checker.identical. Investigate.
if y, ok := y.(*Named); ok {
- xargs := x.targs.list()
- yargs := y.targs.list()
+ xargs := x.TypeArgs().list()
+ yargs := y.TypeArgs().list()
if len(xargs) != len(yargs) {
return false
diff --git a/src/cmd/compile/internal/types2/validtype.go b/src/cmd/compile/internal/types2/validtype.go
index d495c6788e..99fdebc978 100644
--- a/src/cmd/compile/internal/types2/validtype.go
+++ b/src/cmd/compile/internal/types2/validtype.go
@@ -5,29 +5,24 @@
package types2
// validType verifies that the given type does not "expand" indefinitely
-// producing a cycle in the type graph. Cycles are detected by marking
-// defined types.
+// producing a cycle in the type graph.
// (Cycles involving alias types, as in "type A = [10]A" are detected
// earlier, via the objDecl cycle detection mechanism.)
func (check *Checker) validType(typ *Named) {
check.validType0(typ, nil, nil)
}
-type typeInfo uint
-
// validType0 checks if the given type is valid. If typ is a type parameter
-// its value is looked up in the provided environment. The environment is
-// nil if typ is not part of (the RHS of) an instantiated type, in that case
-// any type parameter encountered must be from an enclosing function and can
-// be ignored. The path is the list of type names that lead to the current typ.
-func (check *Checker) validType0(typ Type, env *tparamEnv, path []Object) typeInfo {
- const (
- unknown typeInfo = iota
- marked
- valid
- invalid
- )
-
+// its value is looked up in the type argument list of the instantiated
+// (enclosing) type, if it exists. Otherwise the type parameter must be from
+// an enclosing function and can be ignored.
+// The nest list describes the stack (the "nest in memory") of types which
+// contain (or embed in the case of interfaces) other types. For instance, a
+// struct named S which contains a field of named type F contains (the memory
+// of) F in S, leading to the nest S->F. If a type appears in its own nest
+// (say S->F->S) we have an invalid recursive type. The path list is the full
+// path of named types in a cycle, it is only needed for error reporting.
+func (check *Checker) validType0(typ Type, nest, path []*Named) bool {
switch t := typ.(type) {
case nil:
// We should never see a nil type but be conservative and panic
@@ -37,112 +32,204 @@ func (check *Checker) validType0(typ Type, env *tparamEnv, path []Object) typeIn
}
case *Array:
- return check.validType0(t.elem, env, path)
+ return check.validType0(t.elem, nest, path)
case *Struct:
for _, f := range t.fields {
- if check.validType0(f.typ, env, path) == invalid {
- return invalid
+ if !check.validType0(f.typ, nest, path) {
+ return false
}
}
case *Union:
for _, t := range t.terms {
- if check.validType0(t.typ, env, path) == invalid {
- return invalid
+ if !check.validType0(t.typ, nest, path) {
+ return false
}
}
case *Interface:
for _, etyp := range t.embeddeds {
- if check.validType0(etyp, env, path) == invalid {
- return invalid
+ if !check.validType0(etyp, nest, path) {
+ return false
}
}
case *Named:
+ // Exit early if we already know t is valid.
+ // This is purely an optimization but it prevents excessive computation
+ // times in pathological cases such as testdata/fixedbugs/issue6977.go.
+ // (Note: The valids map could also be allocated locally, once for each
+ // validType call.)
+ if check.valids.lookup(t) != nil {
+ break
+ }
+
// Don't report a 2nd error if we already know the type is invalid
// (e.g., if a cycle was detected earlier, via under).
// Note: ensure that t.orig is fully resolved by calling Underlying().
if t.Underlying() == Typ[Invalid] {
- check.infoMap[t] = invalid
- return invalid
+ return false
}
- switch check.infoMap[t] {
- case unknown:
- check.infoMap[t] = marked
- check.infoMap[t] = check.validType0(t.orig.fromRHS, env.push(t), append(path, t.obj))
- case marked:
- // We have seen type t before and thus must have a cycle.
- check.infoMap[t] = invalid
- // t cannot be in an imported package otherwise that package
- // would have reported a type cycle and couldn't have been
- // imported in the first place.
- assert(t.obj.pkg == check.pkg)
- t.underlying = Typ[Invalid] // t is in the current package (no race possibility)
- // Find the starting point of the cycle and report it.
- for i, tn := range path {
- if tn == t.obj {
- check.cycleError(path[i:])
- return invalid
+ // If the current type t is also found in nest, (the memory of) t is
+ // embedded in itself, indicating an invalid recursive type.
+ for _, e := range nest {
+ if Identical(e, t) {
+ // t cannot be in an imported package otherwise that package
+ // would have reported a type cycle and couldn't have been
+ // imported in the first place.
+ assert(t.obj.pkg == check.pkg)
+ t.underlying = Typ[Invalid] // t is in the current package (no race possibility)
+ // Find the starting point of the cycle and report it.
+ // Because each type in nest must also appear in path (see invariant below),
+ // type t must be in path since it was found in nest. But not every type in path
+ // is in nest. Specifically t may appear in path with an earlier index than the
+ // index of t in nest. Search again.
+ for start, p := range path {
+ if Identical(p, t) {
+ check.cycleError(makeObjList(path[start:]))
+ return false
+ }
}
+ panic("cycle start not found")
}
- panic("cycle start not found")
}
- return check.infoMap[t]
+
+ // No cycle was found. Check the RHS of t.
+ // Every type added to nest is also added to path; thus every type that is in nest
+ // must also be in path (invariant). But not every type in path is in nest, since
+ // nest may be pruned (see below, *TypeParam case).
+ if !check.validType0(t.Origin().fromRHS, append(nest, t), append(path, t)) {
+ return false
+ }
+
+ check.valids.add(t) // t is valid
case *TypeParam:
// A type parameter stands for the type (argument) it was instantiated with.
- // Check the corresponding type argument for validity if we have one.
- if env != nil {
- if targ := env.tmap[t]; targ != nil {
- // Type arguments found in targ must be looked
- // up in the enclosing environment env.link.
- return check.validType0(targ, env.link, path)
+ // Check the corresponding type argument for validity if we are in an
+ // instantiated type.
+ if len(nest) > 0 {
+ inst := nest[len(nest)-1] // the type instance
+ // Find the corresponding type argument for the type parameter
+ // and proceed with checking that type argument.
+ for i, tparam := range inst.TypeParams().list() {
+ // The type parameter and type argument lists should
+ // match in length but be careful in case of errors.
+ if t == tparam && i < inst.TypeArgs().Len() {
+ targ := inst.TypeArgs().At(i)
+ // The type argument must be valid in the enclosing
+ // type (where inst was instantiated), hence we must
+ // check targ's validity in the type nest excluding
+ // the current (instantiated) type (see the example
+ // at the end of this file).
+ // For error reporting we keep the full path.
+ return check.validType0(targ, nest[:len(nest)-1], path)
+ }
}
}
}
- return valid
+ return true
}
-// A tparamEnv provides the environment for looking up the type arguments
-// with which type parameters for a given instance were instantiated.
-// If we don't have an instance, the corresponding tparamEnv is nil.
-type tparamEnv struct {
- tmap substMap
- link *tparamEnv
+// makeObjList returns the list of type name objects for the given
+// list of named types.
+func makeObjList(tlist []*Named) []Object {
+ olist := make([]Object, len(tlist))
+ for i, t := range tlist {
+ olist[i] = t.obj
+ }
+ return olist
}
-func (env *tparamEnv) push(typ *Named) *tparamEnv {
- // If typ is not an instantiated type there are no typ-specific
- // type parameters to look up and we don't need an environment.
- targs := typ.TypeArgs()
- if targs == nil {
- return nil // no instance => nil environment
- }
-
- // Populate tmap: remember the type argument for each type parameter.
- // We cannot use makeSubstMap because the number of type parameters
- // and arguments may not match due to errors in the source (too many
- // or too few type arguments). Populate tmap "manually".
- tparams := typ.TypeParams()
- n, m := targs.Len(), tparams.Len()
- if n > m {
- n = m // too many targs
- }
- tmap := make(substMap, n)
- for i := 0; i < n; i++ {
- tmap[tparams.At(i)] = targs.At(i)
- }
-
- return &tparamEnv{tmap: tmap, link: env}
-}
-
-// TODO(gri) Alternative implementation:
-// We may not need to build a stack of environments to
-// look up the type arguments for type parameters. The
-// same information should be available via the path:
-// We should be able to just walk the path backwards
-// and find the type arguments in the instance objects.
+// Here is an example illustrating why we need to exclude the
+// instantiated type from nest when evaluating the validity of
+// a type parameter. Given the declarations
+//
+// var _ A[A[string]]
+//
+// type A[P any] struct { _ B[P] }
+// type B[P any] struct { _ P }
+//
+// we want to determine if the type A[A[string]] is valid.
+// We start evaluating A[A[string]] outside any type nest:
+//
+// A[A[string]]
+// nest =
+// path =
+//
+// The RHS of A is now evaluated in the A[A[string]] nest:
+//
+// struct{_ B[P₁]}
+// nest = A[A[string]]
+// path = A[A[string]]
+//
+// The struct has a single field of type B[P₁] with which
+// we continue:
+//
+// B[P₁]
+// nest = A[A[string]]
+// path = A[A[string]]
+//
+// struct{_ P₂}
+// nest = A[A[string]]->B[P]
+// path = A[A[string]]->B[P]
+//
+// Eventutally we reach the type parameter P of type B (P₂):
+//
+// P₂
+// nest = A[A[string]]->B[P]
+// path = A[A[string]]->B[P]
+//
+// The type argument for P of B is the type parameter P of A (P₁).
+// It must be evaluated in the type nest that existed when B was
+// instantiated:
+//
+// P₁
+// nest = A[A[string]] <== type nest at B's instantiation time
+// path = A[A[string]]->B[P]
+//
+// If we'd use the current nest it would correspond to the path
+// which will be wrong as we will see shortly. P's type argument
+// is A[string], which again must be evaluated in the type nest
+// that existed when A was instantiated with A[string]. That type
+// nest is empty:
+//
+// A[string]
+// nest = <== type nest at A's instantiation time
+// path = A[A[string]]->B[P]
+//
+// Evaluation then proceeds as before for A[string]:
+//
+// struct{_ B[P₁]}
+// nest = A[string]
+// path = A[A[string]]->B[P]->A[string]
+//
+// Now we reach B[P] again. If we had not adjusted nest, it would
+// correspond to path, and we would find B[P] in nest, indicating
+// a cycle, which would clearly be wrong since there's no cycle in
+// A[string]:
+//
+// B[P₁]
+// nest = A[string]
+// path = A[A[string]]->B[P]->A[string] <== path contains B[P]!
+//
+// But because we use the correct type nest, evaluation proceeds without
+// errors and we get the evaluation sequence:
+//
+// struct{_ P₂}
+// nest = A[string]->B[P]
+// path = A[A[string]]->B[P]->A[string]->B[P]
+// P₂
+// nest = A[string]->B[P]
+// path = A[A[string]]->B[P]->A[string]->B[P]
+// P₁
+// nest = A[string]
+// path = A[A[string]]->B[P]->A[string]->B[P]
+// string
+// nest =
+// path = A[A[string]]->B[P]->A[string]->B[P]
+//
+// At this point we're done and A[A[string]] and is valid.
diff --git a/src/cmd/dist/test.go b/src/cmd/dist/test.go
index 677be336ac..26d7fe0f73 100644
--- a/src/cmd/dist/test.go
+++ b/src/cmd/dist/test.go
@@ -290,7 +290,7 @@ func (t *tester) maybeLogMetadata() error {
//
// TODO(prattmic): If we split dist bootstrap and dist test then this
// could be simplified to directly use internal/sysinfo here.
- return t.dirCmd(filepath.Join(goroot, "src/cmd/internal/metadata"), "go", []string{"run", "."}).Run()
+ return t.dirCmd(filepath.Join(goroot, "src/cmd/internal/metadata"), "go", []string{"run", "main.go"}).Run()
}
// short returns a -short flag value to use with 'go test'
diff --git a/src/cmd/go/internal/clean/clean.go b/src/cmd/go/internal/clean/clean.go
index 8564411fb6..019d36490f 100644
--- a/src/cmd/go/internal/clean/clean.go
+++ b/src/cmd/go/internal/clean/clean.go
@@ -22,6 +22,7 @@ import (
"cmd/go/internal/lockedfile"
"cmd/go/internal/modfetch"
"cmd/go/internal/modload"
+ "cmd/go/internal/str"
"cmd/go/internal/work"
)
@@ -141,7 +142,7 @@ func runClean(ctx context.Context, cmd *base.Command, args []string) {
// The top cache directory may have been created with special permissions
// and not something that we want to remove. Also, we'd like to preserve
// the access log for future analysis, even if the cache is cleared.
- subdirs, _ := filepath.Glob(filepath.Join(dir, "[0-9a-f][0-9a-f]"))
+ subdirs, _ := filepath.Glob(filepath.Join(str.QuoteGlob(dir), "[0-9a-f][0-9a-f]"))
printedErrors := false
if len(subdirs) > 0 {
if cfg.BuildN || cfg.BuildX {
diff --git a/src/cmd/go/internal/load/pkg.go b/src/cmd/go/internal/load/pkg.go
index 8ceacec326..fe4a82472d 100644
--- a/src/cmd/go/internal/load/pkg.go
+++ b/src/cmd/go/internal/load/pkg.go
@@ -14,7 +14,6 @@ import (
"go/build"
"go/scanner"
"go/token"
- "internal/goroot"
"io/fs"
"os"
"os/exec"
@@ -872,7 +871,7 @@ func loadPackageData(ctx context.Context, path, parentPath, parentDir, parentRoo
if !cfg.ModulesEnabled {
buildMode = build.ImportComment
}
- if modroot := modload.PackageModRoot(ctx, r.dir); modroot != "" {
+ if modroot := modload.PackageModRoot(ctx, r.path); modroot != "" {
if mi, err := modindex.Get(modroot); err == nil {
data.p, data.err = mi.Import(cfg.BuildContext, mi.RelPath(r.dir), buildMode)
goto Happy
@@ -2064,7 +2063,7 @@ func resolveEmbed(pkgdir string, patterns []string) (files []string, pmap map[st
}
// Glob to find matches.
- match, err := fsys.Glob(pkgdir + string(filepath.Separator) + filepath.FromSlash(glob))
+ match, err := fsys.Glob(str.QuoteGlob(pkgdir) + string(filepath.Separator) + filepath.FromSlash(glob))
if err != nil {
return nil, nil, err
}
@@ -3072,7 +3071,7 @@ func PackagesAndErrorsOutsideModule(ctx context.Context, opts PackageOpts, args
return nil, fmt.Errorf("%s: argument must be a package path, not a meta-package", arg)
case path.Clean(p) != p:
return nil, fmt.Errorf("%s: argument must be a clean package path", arg)
- case !strings.Contains(p, "...") && search.IsStandardImportPath(p) && goroot.IsStandardPackage(cfg.GOROOT, cfg.BuildContext.Compiler, p):
+ case !strings.Contains(p, "...") && search.IsStandardImportPath(p) && modindex.IsStandardPackage(cfg.GOROOT, cfg.BuildContext.Compiler, p):
return nil, fmt.Errorf("%s: argument must not be a package in the standard library", arg)
default:
patterns[i] = p
diff --git a/src/cmd/go/internal/modfetch/fetch.go b/src/cmd/go/internal/modfetch/fetch.go
index 21d5f54688..a7c8c2c769 100644
--- a/src/cmd/go/internal/modfetch/fetch.go
+++ b/src/cmd/go/internal/modfetch/fetch.go
@@ -26,6 +26,7 @@ import (
"cmd/go/internal/lockedfile"
"cmd/go/internal/par"
"cmd/go/internal/robustio"
+ "cmd/go/internal/str"
"cmd/go/internal/trace"
"golang.org/x/mod/module"
@@ -102,7 +103,7 @@ func download(ctx context.Context, mod module.Version) (dir string, err error) {
// active.
parentDir := filepath.Dir(dir)
tmpPrefix := filepath.Base(dir) + ".tmp-"
- if old, err := filepath.Glob(filepath.Join(parentDir, tmpPrefix+"*")); err == nil {
+ if old, err := filepath.Glob(filepath.Join(str.QuoteGlob(parentDir), str.QuoteGlob(tmpPrefix)+"*")); err == nil {
for _, path := range old {
RemoveAll(path) // best effort
}
@@ -224,7 +225,7 @@ func downloadZip(ctx context.Context, mod module.Version, zipfile string) (err e
// This is only safe to do because the lock file ensures that their
// writers are no longer active.
tmpPattern := filepath.Base(zipfile) + "*.tmp"
- if old, err := filepath.Glob(filepath.Join(filepath.Dir(zipfile), tmpPattern)); err == nil {
+ if old, err := filepath.Glob(filepath.Join(str.QuoteGlob(filepath.Dir(zipfile)), tmpPattern)); err == nil {
for _, path := range old {
os.Remove(path) // best effort
}
diff --git a/src/cmd/go/internal/modindex/build.go b/src/cmd/go/internal/modindex/build.go
index 78bd12636d..d6d4ea371a 100644
--- a/src/cmd/go/internal/modindex/build.go
+++ b/src/cmd/go/internal/modindex/build.go
@@ -177,6 +177,37 @@ func hasSubdir(root, dir string) (rel string, ok bool) {
return filepath.ToSlash(dir[len(root):]), true
}
+// gopath returns the list of Go path directories.
+func (ctxt *Context) gopath() []string {
+ var all []string
+ for _, p := range ctxt.splitPathList(ctxt.GOPATH) {
+ if p == "" || p == ctxt.GOROOT {
+ // Empty paths are uninteresting.
+ // If the path is the GOROOT, ignore it.
+ // People sometimes set GOPATH=$GOROOT.
+ // Do not get confused by this common mistake.
+ continue
+ }
+ if strings.HasPrefix(p, "~") {
+ // Path segments starting with ~ on Unix are almost always
+ // users who have incorrectly quoted ~ while setting GOPATH,
+ // preventing it from expanding to $HOME.
+ // The situation is made more confusing by the fact that
+ // bash allows quoted ~ in $PATH (most shells do not).
+ // Do not get confused by this, and do not try to use the path.
+ // It does not exist, and printing errors about it confuses
+ // those users even more, because they think "sure ~ exists!".
+ // The go command diagnoses this situation and prints a
+ // useful error.
+ // On Windows, ~ is used in short names, such as c:\progra~1
+ // for c:\program files.
+ continue
+ }
+ all = append(all, p)
+ }
+ return all
+}
+
var defaultToolTags, defaultReleaseTags []string
// A Package describes the Go package found in a directory.
@@ -887,8 +918,8 @@ func (ctxt *Context) eval(x constraint.Expr, allTags map[string]bool) bool {
// $GOARCH
// boringcrypto
// ctxt.Compiler
-// linux (if GOOS = android)
-// solaris (if GOOS = illumos)
+// linux (if GOOS == android)
+// solaris (if GOOS == illumos)
// tag (if tag is listed in ctxt.BuildTags or ctxt.ReleaseTags)
//
// It records all consulted tags in allTags.
diff --git a/src/cmd/go/internal/modindex/read.go b/src/cmd/go/internal/modindex/read.go
index e180ca5450..ffa091df41 100644
--- a/src/cmd/go/internal/modindex/read.go
+++ b/src/cmd/go/internal/modindex/read.go
@@ -1,7 +1,31 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
package modindex
import (
"bytes"
+ "encoding/binary"
+ "errors"
+ "fmt"
+ "go/build"
+ "go/build/constraint"
+ "go/token"
+ "internal/goroot"
+ "internal/unsafeheader"
+ "io/fs"
+ "math"
+ "os"
+ "path"
+ "path/filepath"
+ "runtime"
+ "runtime/debug"
+ "sort"
+ "strings"
+ "sync"
+ "unsafe"
+
"cmd/go/internal/base"
"cmd/go/internal/cache"
"cmd/go/internal/cfg"
@@ -9,30 +33,21 @@ import (
"cmd/go/internal/imports"
"cmd/go/internal/par"
"cmd/go/internal/str"
- "encoding/binary"
- "errors"
- "fmt"
- "go/build"
- "go/build/constraint"
- "go/token"
- "internal/unsafeheader"
- "io/fs"
- "math"
- "os"
- "path/filepath"
- "runtime/debug"
- "sort"
- "strconv"
- "strings"
- "sync"
- "unsafe"
)
// enabled is used to flag off the behavior of the module index on tip.
// It will be removed before the release.
// TODO(matloob): Remove enabled once we have more confidence on the
// module index.
-var enabled, _ = strconv.ParseBool(os.Getenv("GOINDEX"))
+var enabled = func() bool {
+ debug := strings.Split(os.Getenv("GODEBUG"), ",")
+ for _, f := range debug {
+ if f == "goindex=0" {
+ return false
+ }
+ }
+ return true
+}()
// ModuleIndex represents and encoded module index file. It is used to
// do the equivalent of build.Import of packages in the module and answer other
@@ -47,38 +62,56 @@ type ModuleIndex struct {
var fcache par.Cache
func moduleHash(modroot string, ismodcache bool) (cache.ActionID, error) {
- h := cache.NewHash("moduleIndex")
- fmt.Fprintf(h, "module index %s %v", indexVersion, modroot)
- if ismodcache {
- return h.Sum(), nil
+ // We expect modules stored within the module cache to be checksummed and
+ // immutable, and we expect released Go modules to change only infrequently
+ // (when the Go version changes).
+ if !ismodcache || !str.HasFilePathPrefix(modroot, cfg.GOROOT) {
+ return cache.ActionID{}, ErrNotIndexed
}
- // walkdir happens in deterministic order.
- err := fsys.Walk(modroot, func(path string, info fs.FileInfo, err error) error {
- if modroot == path {
- // Check for go.mod in root directory, and return ErrNotIndexed
- // if it doesn't exist. Outside the module cache, it's not a module
- // if it doesn't have a go.mod file.
- }
- if err := moduleWalkErr(modroot, path, info, err); err != nil {
- return err
- }
- if info.IsDir() {
- return nil
- }
- fmt.Fprintf(h, "file %v %v\n", info.Name(), info.ModTime())
- if info.Mode()&fs.ModeSymlink != 0 {
- targ, err := fsys.Stat(path)
- if err != nil {
+ h := cache.NewHash("moduleIndex")
+ fmt.Fprintf(h, "module index %s %s %v\n", runtime.Version(), indexVersion, modroot)
+
+ if strings.HasPrefix(runtime.Version(), "devel ") {
+ // This copy of the standard library is a development version, not a
+ // release. It could be based on a Git commit (like "devel go1.19-2a78e8afc0
+ // Wed Jun 15 00:06:24 2022 +0000") with or without changes on top of that
+ // commit, or it could be completly artificial due to lacking a `git` binary
+ // (like "devel gomote.XXXXX", as synthesized by "gomote push" as of
+ // 2022-06-15). Compute an inexpensive hash of its files using mtimes so
+ // that during development we can continue to exercise the logic for cached
+ // GOROOT indexes.
+ //
+ // mtimes may be granular, imprecise, and loosely updated (see
+ // https://apenwarr.ca/log/20181113), but we don't expect Go contributors to
+ // be mucking around with the import graphs in GOROOT often enough for mtime
+ // collisions to matter essentially ever.
+ //
+ // Note that fsys.Walk walks paths in deterministic order, so this hash
+ // should be completely deterministic if the files are unchanged.
+ err := fsys.Walk(modroot, func(path string, info fs.FileInfo, err error) error {
+ if err := moduleWalkErr(modroot, path, info, err); err != nil {
return err
}
- fmt.Fprintf(h, "target %v %v\n", targ.Name(), targ.ModTime())
+
+ if info.IsDir() {
+ return nil
+ }
+ fmt.Fprintf(h, "file %v %v\n", info.Name(), info.ModTime())
+ if info.Mode()&fs.ModeSymlink != 0 {
+ targ, err := fsys.Stat(path)
+ if err != nil {
+ return err
+ }
+ fmt.Fprintf(h, "target %v %v\n", targ.Name(), targ.ModTime())
+ }
+ return nil
+ })
+ if err != nil {
+ return cache.ActionID{}, err
}
- return nil
- })
- if err != nil {
- return cache.ActionID{}, err
}
+
return h.Sum(), nil
}
@@ -97,10 +130,7 @@ func Get(modroot string) (*ModuleIndex, error) {
if modroot == "" {
panic("modindex.Get called with empty modroot")
}
- if str.HasFilePathPrefix(modroot, cfg.GOROOT) {
- // TODO(matloob): add a case for stdlib here.
- return nil, ErrNotIndexed
- }
+ modroot = filepath.Clean(modroot)
isModCache := str.HasFilePathPrefix(modroot, cfg.GOMODCACHE)
return openIndex(modroot, isModCache)
}
@@ -121,7 +151,7 @@ func openIndex(modroot string, ismodcache bool) (*ModuleIndex, error) {
data, _, err := cache.Default().GetMmap(id)
if err != nil {
// Couldn't read from modindex. Assume we couldn't read from
- // the index because the module has't been indexed yet.
+ // the index because the module hasn't been indexed yet.
data, err = indexModule(modroot)
if err != nil {
return result{nil, err}
@@ -206,7 +236,7 @@ func (mi *ModuleIndex) Packages() []string {
// RelPath returns the path relative to the module's root.
func (mi *ModuleIndex) RelPath(path string) string {
- return filepath.Clean(str.TrimFilePathPrefix(path, mi.modroot))
+ return str.TrimFilePathPrefix(filepath.Clean(path), mi.modroot) // mi.modroot is already clean
}
// ImportPackage is the equivalent of build.Import given the information in ModuleIndex.
@@ -225,9 +255,6 @@ func (mi *ModuleIndex) Import(bctxt build.Context, relpath string, mode build.Im
p.ImportPath = "."
p.Dir = filepath.Join(mi.modroot, rp.dir)
- if rp.error != "" {
- return p, errors.New(rp.error)
- }
var pkgerr error
switch ctxt.Compiler {
@@ -241,6 +268,78 @@ func (mi *ModuleIndex) Import(bctxt build.Context, relpath string, mode build.Im
return p, fmt.Errorf("import %q: import of unknown directory", p.Dir)
}
+ // goroot and gopath
+ inTestdata := func(sub string) bool {
+ return strings.Contains(sub, "/testdata/") || strings.HasSuffix(sub, "/testdata") || str.HasPathPrefix(sub, "testdata")
+ }
+ if !inTestdata(relpath) {
+ // In build.go, p.Root should only be set in the non-local-import case, or in
+ // GOROOT or GOPATH. Since module mode only calls Import with path set to "."
+ // and the module index doesn't apply outside modules, the GOROOT case is
+ // the only case where GOROOT needs to be set.
+ // But: p.Root is actually set in the local-import case outside GOROOT, if
+ // the directory is contained in GOPATH/src
+ // TODO(#37015): fix that behavior in go/build and remove the gopath case
+ // below.
+ if ctxt.GOROOT != "" && str.HasFilePathPrefix(p.Dir, cfg.GOROOTsrc) && p.Dir != cfg.GOROOTsrc {
+ p.Root = ctxt.GOROOT
+ p.Goroot = true
+ modprefix := str.TrimFilePathPrefix(mi.modroot, cfg.GOROOTsrc)
+ p.ImportPath = relpath
+ if modprefix != "" {
+ p.ImportPath = filepath.Join(modprefix, p.ImportPath)
+ }
+ }
+ for _, root := range ctxt.gopath() {
+ // TODO(matloob): do we need to reimplement the conflictdir logic?
+
+ // TODO(matloob): ctxt.hasSubdir evaluates symlinks, so it
+ // can be slower than we'd like. Find out if we can drop this
+ // logic before the release.
+ if sub, ok := ctxt.hasSubdir(filepath.Join(root, "src"), p.Dir); ok {
+ p.ImportPath = sub
+ p.Root = root
+ }
+ }
+ }
+ if p.Root != "" {
+ // Set GOROOT-specific fields (sometimes for modules in a GOPATH directory).
+ // The fields set below (SrcRoot, PkgRoot, BinDir, PkgTargetRoot, and PkgObj)
+ // are only set in build.Import if p.Root != "". As noted in the comment
+ // on setting p.Root above, p.Root should only be set in the GOROOT case for the
+ // set of packages we care about, but is also set for modules in a GOPATH src
+ // directory.
+ var pkgtargetroot string
+ var pkga string
+ suffix := ""
+ if ctxt.InstallSuffix != "" {
+ suffix = "_" + ctxt.InstallSuffix
+ }
+ switch ctxt.Compiler {
+ case "gccgo":
+ pkgtargetroot = "pkg/gccgo_" + ctxt.GOOS + "_" + ctxt.GOARCH + suffix
+ dir, elem := path.Split(p.ImportPath)
+ pkga = pkgtargetroot + "/" + dir + "lib" + elem + ".a"
+ case "gc":
+ pkgtargetroot = "pkg/" + ctxt.GOOS + "_" + ctxt.GOARCH + suffix
+ pkga = pkgtargetroot + "/" + p.ImportPath + ".a"
+ }
+ p.SrcRoot = ctxt.joinPath(p.Root, "src")
+ p.PkgRoot = ctxt.joinPath(p.Root, "pkg")
+ p.BinDir = ctxt.joinPath(p.Root, "bin")
+ if pkga != "" {
+ p.PkgTargetRoot = ctxt.joinPath(p.Root, pkgtargetroot)
+ p.PkgObj = ctxt.joinPath(p.Root, pkga)
+ }
+ }
+
+ if rp.error != nil {
+ if errors.Is(rp.error, errCannotFindPackage) && ctxt.Compiler == "gccgo" && p.Goroot {
+ return p, nil
+ }
+ return p, rp.error
+ }
+
if mode&build.FindOnly != 0 {
return p, pkgerr
}
@@ -444,8 +543,31 @@ func (mi *ModuleIndex) Import(bctxt build.Context, relpath string, mode build.Im
return p, pkgerr
}
-// IsDirWithGoFiles is the equivalent of fsys.IsDirWithGoFiles using the information in the
-// RawPackage.
+// IsStandardPackage reports whether path is a standard package
+// for the goroot and compiler using the module index if possible,
+// and otherwise falling back to internal/goroot.IsStandardPackage
+func IsStandardPackage(goroot_, compiler, path string) bool {
+ if !enabled || compiler != "gc" {
+ return goroot.IsStandardPackage(goroot_, compiler, path)
+ }
+
+ reldir := filepath.FromSlash(path) // relative dir path in module index for package
+ modroot := filepath.Join(goroot_, "src")
+ if str.HasFilePathPrefix(reldir, "cmd") {
+ reldir = str.TrimFilePathPrefix(reldir, "cmd")
+ modroot = filepath.Join(modroot, "cmd")
+ }
+ mod, err := Get(modroot)
+ if err != nil {
+ return goroot.IsStandardPackage(goroot_, compiler, path)
+ }
+
+ pkgs := mod.Packages()
+ i := sort.SearchStrings(pkgs, reldir)
+ return i != len(pkgs) && pkgs[i] == reldir
+}
+
+// IsDirWithGoFiles is the equivalent of fsys.IsDirWithGoFiles using the information in the index.
func (mi *ModuleIndex) IsDirWithGoFiles(relpath string) (_ bool, err error) {
rp := mi.indexPackage(relpath)
@@ -462,7 +584,7 @@ func (mi *ModuleIndex) IsDirWithGoFiles(relpath string) (_ bool, err error) {
return false, nil
}
-// ScanDir implements imports.ScanDir using the information in the RawPackage.
+// ScanDir implements imports.ScanDir using the information in the index.
func (mi *ModuleIndex) ScanDir(path string, tags map[string]bool) (sortedImports []string, sortedTestImports []string, err error) {
rp := mi.indexPackage(path)
@@ -556,13 +678,15 @@ func shouldBuild(sf *sourceFile, tags map[string]bool) bool {
// index package holds the information needed to access information in the
// index about a package.
type indexPackage struct {
- error string
+ error error
dir string // directory of the package relative to the modroot
// Source files
sourceFiles []*sourceFile
}
+var errCannotFindPackage = errors.New("cannot find package")
+
// indexPackage returns an indexPackage constructed using the information in the ModuleIndex.
func (mi *ModuleIndex) indexPackage(path string) *indexPackage {
defer func() {
@@ -572,13 +696,15 @@ func (mi *ModuleIndex) indexPackage(path string) *indexPackage {
}()
offset, ok := mi.packages[path]
if !ok {
- return &indexPackage{error: fmt.Sprintf("cannot find package %q in:\n\t%s", path, filepath.Join(mi.modroot, path))}
+ return &indexPackage{error: fmt.Errorf("%w %q in:\n\t%s", errCannotFindPackage, path, filepath.Join(mi.modroot, path))}
}
// TODO(matloob): do we want to lock on the module index?
d := mi.od.decoderAt(offset)
rp := new(indexPackage)
- rp.error = d.string()
+ if errstr := d.string(); errstr != "" {
+ rp.error = errors.New(errstr)
+ }
rp.dir = d.string()
numSourceFiles := d.uint32()
rp.sourceFiles = make([]*sourceFile, numSourceFiles)
diff --git a/src/cmd/go/internal/modindex/scan.go b/src/cmd/go/internal/modindex/scan.go
index e40d3e0f53..d1f73dbb53 100644
--- a/src/cmd/go/internal/modindex/scan.go
+++ b/src/cmd/go/internal/modindex/scan.go
@@ -4,6 +4,7 @@ import (
"cmd/go/internal/base"
"cmd/go/internal/fsys"
"cmd/go/internal/par"
+ "cmd/go/internal/str"
"encoding/json"
"errors"
"fmt"
@@ -54,10 +55,10 @@ func indexModule(modroot string) ([]byte, error) {
if !info.IsDir() {
return nil
}
- rel, err := filepath.Rel(modroot, path)
- if err != nil {
- panic(err)
+ if !str.HasFilePathPrefix(path, modroot) {
+ panic(fmt.Errorf("path %v in walk doesn't have modroot %v as prefix:", path, modroot))
}
+ rel := str.TrimFilePathPrefix(path, modroot)
packages = append(packages, importRaw(modroot, rel))
return nil
})
diff --git a/src/cmd/go/internal/modload/build.go b/src/cmd/go/internal/modload/build.go
index 7b1bc732fc..0799fec35c 100644
--- a/src/cmd/go/internal/modload/build.go
+++ b/src/cmd/go/internal/modload/build.go
@@ -9,7 +9,6 @@ import (
"encoding/hex"
"errors"
"fmt"
- "internal/goroot"
"io/fs"
"os"
"path/filepath"
@@ -18,6 +17,7 @@ import (
"cmd/go/internal/base"
"cmd/go/internal/cfg"
"cmd/go/internal/modfetch"
+ "cmd/go/internal/modindex"
"cmd/go/internal/modinfo"
"cmd/go/internal/search"
@@ -39,7 +39,7 @@ func findStandardImportPath(path string) string {
panic("findStandardImportPath called with empty path")
}
if search.IsStandardImportPath(path) {
- if goroot.IsStandardPackage(cfg.GOROOT, cfg.BuildContext.Compiler, path) {
+ if modindex.IsStandardPackage(cfg.GOROOT, cfg.BuildContext.Compiler, path) {
return filepath.Join(cfg.GOROOT, "src", path)
}
}
diff --git a/src/cmd/go/internal/modload/import.go b/src/cmd/go/internal/modload/import.go
index 22286e5e2d..f7810ca5c6 100644
--- a/src/cmd/go/internal/modload/import.go
+++ b/src/cmd/go/internal/modload/import.go
@@ -9,7 +9,6 @@ import (
"errors"
"fmt"
"go/build"
- "internal/goroot"
"io/fs"
"os"
pathpkg "path"
@@ -281,7 +280,7 @@ func importFromModules(ctx context.Context, path string, rs *Requirements, mg *M
// Is the package in the standard library?
pathIsStd := search.IsStandardImportPath(path)
- if pathIsStd && goroot.IsStandardPackage(cfg.GOROOT, cfg.BuildContext.Compiler, path) {
+ if pathIsStd && modindex.IsStandardPackage(cfg.GOROOT, cfg.BuildContext.Compiler, path) {
for _, mainModule := range MainModules.Versions() {
if MainModules.InGorootSrc(mainModule) {
if dir, ok, err := dirInModule(path, MainModules.PathPrefix(mainModule), MainModules.ModRoot(mainModule), true); err != nil {
diff --git a/src/cmd/go/internal/modload/search.go b/src/cmd/go/internal/modload/search.go
index cddb9f8067..4b90392d94 100644
--- a/src/cmd/go/internal/modload/search.go
+++ b/src/cmd/go/internal/modload/search.go
@@ -6,15 +6,22 @@ package modload
import (
"context"
+ "errors"
"fmt"
"io/fs"
"os"
+ "path"
"path/filepath"
+ "runtime"
+ "sort"
"strings"
+ "sync"
"cmd/go/internal/cfg"
"cmd/go/internal/fsys"
"cmd/go/internal/imports"
+ "cmd/go/internal/modindex"
+ "cmd/go/internal/par"
"cmd/go/internal/search"
"golang.org/x/mod/module"
@@ -40,9 +47,15 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
treeCanMatch = search.TreeCanMatchPattern(m.Pattern())
}
+ var mu sync.Mutex
have := map[string]bool{
"builtin": true, // ignore pseudo-package that exists only for documentation
}
+ addPkg := func(p string) {
+ mu.Lock()
+ m.Pkgs = append(m.Pkgs, p)
+ mu.Unlock()
+ }
if !cfg.BuildContext.CgoEnabled {
have["runtime/cgo"] = true // ignore during walk
}
@@ -53,6 +66,8 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
pruneGoMod
)
+ q := par.NewQueue(runtime.GOMAXPROCS(0))
+
walkPkgs := func(root, importPathRoot string, prune pruning) {
root = filepath.Clean(root)
err := fsys.Walk(root, func(path string, fi fs.FileInfo, err error) error {
@@ -107,9 +122,11 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
if !have[name] {
have[name] = true
if isMatch(name) {
- if _, _, err := scanDir(root, path, tags); err != imports.ErrNoGo {
- m.Pkgs = append(m.Pkgs, name)
- }
+ q.Add(func() {
+ if _, _, err := scanDir(root, path, tags); err != imports.ErrNoGo {
+ addPkg(name)
+ }
+ })
}
}
@@ -123,6 +140,12 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
}
}
+ // Wait for all in-flight operations to complete before returning.
+ defer func() {
+ <-q.Idle()
+ sort.Strings(m.Pkgs) // sort everything we added for determinism
+ }()
+
if filter == includeStd {
walkPkgs(cfg.GOROOTsrc, "", pruneGoMod)
if treeCanMatch("cmd") {
@@ -165,6 +188,12 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
}
modPrefix = mod.Path
}
+ if mi, err := modindex.Get(root); err == nil {
+ walkFromIndex(mi, modPrefix, isMatch, treeCanMatch, tags, have, addPkg)
+ continue
+ } else if !errors.Is(err, modindex.ErrNotIndexed) {
+ m.AddError(err)
+ }
prune := pruneVendor
if isLocal {
@@ -176,6 +205,54 @@ func matchPackages(ctx context.Context, m *search.Match, tags map[string]bool, f
return
}
+// walkFromIndex matches packages in a module using the module index. modroot
+// is the module's root directory on disk, index is the ModuleIndex for the
+// module, and importPathRoot is the module's path prefix.
+func walkFromIndex(index *modindex.ModuleIndex, importPathRoot string, isMatch, treeCanMatch func(string) bool, tags, have map[string]bool, addPkg func(string)) {
+loopPackages:
+ for _, reldir := range index.Packages() {
+ // Avoid .foo, _foo, and testdata subdirectory trees.
+ p := reldir
+ for {
+ elem, rest, found := strings.Cut(p, string(filepath.Separator))
+ if strings.HasPrefix(elem, ".") || strings.HasPrefix(elem, "_") || elem == "testdata" {
+ continue loopPackages
+ }
+ if found && elem == "vendor" {
+ // Ignore this path if it contains the element "vendor" anywhere
+ // except for the last element (packages named vendor are allowed
+ // for historical reasons). Note that found is true when this
+ // isn't the last path element.
+ continue loopPackages
+ }
+ if !found {
+ // Didn't find the separator, so we're considering the last element.
+ break
+ }
+ p = rest
+ }
+
+ // Don't use GOROOT/src.
+ if reldir == "" && importPathRoot == "" {
+ continue
+ }
+
+ name := path.Join(importPathRoot, filepath.ToSlash(reldir))
+ if !treeCanMatch(name) {
+ continue
+ }
+
+ if !have[name] {
+ have[name] = true
+ if isMatch(name) {
+ if _, _, err := index.ScanDir(reldir, tags); err != imports.ErrNoGo {
+ addPkg(name)
+ }
+ }
+ }
+ }
+}
+
// MatchInModule identifies the packages matching the given pattern within the
// given module version, which does not need to be in the build list or module
// requirement graph.
diff --git a/src/cmd/go/internal/str/path.go b/src/cmd/go/internal/str/path.go
index a69e171f8c..c165b91785 100644
--- a/src/cmd/go/internal/str/path.go
+++ b/src/cmd/go/internal/str/path.go
@@ -66,3 +66,21 @@ func TrimFilePathPrefix(s, prefix string) string {
}
return trimmed[1:]
}
+
+// QuoteGlob returns s with all Glob metacharacters quoted.
+// We don't try to handle backslash here, as that can appear in a
+// file path on Windows.
+func QuoteGlob(s string) string {
+ if !strings.ContainsAny(s, `*?[]`) {
+ return s
+ }
+ var sb strings.Builder
+ for _, c := range s {
+ switch c {
+ case '*', '?', '[', ']':
+ sb.WriteByte('\\')
+ }
+ sb.WriteRune(c)
+ }
+ return sb.String()
+}
diff --git a/src/cmd/go/script_test.go b/src/cmd/go/script_test.go
index d1fe36ec21..3ad0608725 100644
--- a/src/cmd/go/script_test.go
+++ b/src/cmd/go/script_test.go
@@ -170,7 +170,7 @@ func (ts *testScript) setup() {
"GOCACHE=" + testGOCACHE,
"GODEBUG=" + os.Getenv("GODEBUG"),
"GOEXE=" + cfg.ExeSuffix,
- "GOINDEX=true",
+ "GOEXPERIMENT=" + os.Getenv("GOEXPERIMENT"),
"GOOS=" + runtime.GOOS,
"GOPATH=" + filepath.Join(ts.workdir, "gopath"),
"GOPROXY=" + proxyURL,
diff --git a/src/cmd/go/testdata/script/embed_brackets.txt b/src/cmd/go/testdata/script/embed_brackets.txt
new file mode 100644
index 0000000000..7093a8497e
--- /dev/null
+++ b/src/cmd/go/testdata/script/embed_brackets.txt
@@ -0,0 +1,18 @@
+# issue 53314
+[windows] skip
+cd [pkg]
+go build
+
+-- [pkg]/go.mod --
+module m
+
+go 1.19
+-- [pkg]/x.go --
+package p
+
+import _ "embed"
+
+//go:embed t.txt
+var S string
+
+-- [pkg]//t.txt --
diff --git a/src/cmd/go/testdata/script/list_replace_absolute_windows.txt b/src/cmd/go/testdata/script/list_replace_absolute_windows.txt
new file mode 100644
index 0000000000..b3ff2a7c2d
--- /dev/null
+++ b/src/cmd/go/testdata/script/list_replace_absolute_windows.txt
@@ -0,0 +1,38 @@
+# Test a replacement with an absolute path (so the path isn't
+# cleaned by having filepath.Abs called on it). This checks
+# whether the modindex logic cleans the modroot path before using
+# it.
+
+[!windows] skip
+[short] skip
+
+go run print_go_mod.go # use this program to write a go.mod with an absolute path
+cp stdout go.mod
+
+go list -modfile=go.mod all
+-- print_go_mod.go --
+//go:build ignore
+package main
+
+import (
+ "fmt"
+ "os"
+)
+
+func main() {
+ work := os.Getenv("WORK")
+fmt.Printf(`module example.com/mod
+
+require b.com v0.0.0
+
+replace b.com => %s\gopath\src/modb
+`, work)
+}
+-- a.go --
+package a
+
+import _ "b.com/b"
+-- modb/go.mod --
+module b.com
+-- modb/b/b.go --
+package b
diff --git a/src/cmd/internal/metadata/main.go b/src/cmd/internal/metadata/main.go
index 157226e890..7478eec1c9 100644
--- a/src/cmd/internal/metadata/main.go
+++ b/src/cmd/internal/metadata/main.go
@@ -5,6 +5,12 @@
// Metadata prints basic system metadata to include in test logs. This is
// separate from cmd/dist so it does not need to build with the bootstrap
// toolchain.
+
+// This program is only used by cmd/dist. Add an "ignore" build tag so it
+// is not installed. cmd/dist does "go run main.go" directly.
+
+//go:build ignore
+
package main
import (
diff --git a/src/cmd/internal/notsha256/example_test.go b/src/cmd/internal/notsha256/example_test.go
deleted file mode 100644
index 06e9c379c9..0000000000
--- a/src/cmd/internal/notsha256/example_test.go
+++ /dev/null
@@ -1,41 +0,0 @@
-// Copyright 2016 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package notsha256_test
-
-import (
- "crypto/sha256"
- "fmt"
- "io"
- "log"
- "os"
-)
-
-func ExampleSum256() {
- sum := sha256.Sum256([]byte("hello world\n"))
- fmt.Printf("%x", sum)
- // Output: a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447
-}
-
-func ExampleNew() {
- h := sha256.New()
- h.Write([]byte("hello world\n"))
- fmt.Printf("%x", h.Sum(nil))
- // Output: a948904f2f0f479b8f8197694b30184b0d2ed1c1cd2a1ec0fb85d299a192a447
-}
-
-func ExampleNew_file() {
- f, err := os.Open("file.txt")
- if err != nil {
- log.Fatal(err)
- }
- defer f.Close()
-
- h := sha256.New()
- if _, err := io.Copy(h, f); err != nil {
- log.Fatal(err)
- }
-
- fmt.Printf("%x", h.Sum(nil))
-}
diff --git a/src/cmd/link/internal/ld/data.go b/src/cmd/link/internal/ld/data.go
index f12cb78fb8..cb2afeaa9a 100644
--- a/src/cmd/link/internal/ld/data.go
+++ b/src/cmd/link/internal/ld/data.go
@@ -1854,6 +1854,9 @@ func (state *dodataState) allocateDataSections(ctxt *Link) {
}
for _, symn := range sym.ReadOnly {
symnStartValue := state.datsize
+ if len(state.data[symn]) != 0 {
+ symnStartValue = aligndatsize(state, symnStartValue, state.data[symn][0])
+ }
state.assignToSection(sect, symn, sym.SRODATA)
setCarrierSize(symn, state.datsize-symnStartValue)
if ctxt.HeadType == objabi.Haix {
@@ -1935,6 +1938,9 @@ func (state *dodataState) allocateDataSections(ctxt *Link) {
symn := sym.RelROMap[symnro]
symnStartValue := state.datsize
+ if len(state.data[symn]) != 0 {
+ symnStartValue = aligndatsize(state, symnStartValue, state.data[symn][0])
+ }
for _, s := range state.data[symn] {
outer := ldr.OuterSym(s)
diff --git a/src/cmd/link/internal/ld/decodesym.go b/src/cmd/link/internal/ld/decodesym.go
index a6ae202859..b0f4b87563 100644
--- a/src/cmd/link/internal/ld/decodesym.go
+++ b/src/cmd/link/internal/ld/decodesym.go
@@ -132,6 +132,15 @@ func decodetypeName(ldr *loader.Loader, symIdx loader.Sym, relocs *loader.Relocs
return string(data[1+nameLenLen : 1+nameLenLen+int(nameLen)])
}
+func decodetypeNameEmbedded(ldr *loader.Loader, symIdx loader.Sym, relocs *loader.Relocs, off int) bool {
+ r := decodeRelocSym(ldr, symIdx, relocs, int32(off))
+ if r == 0 {
+ return false
+ }
+ data := ldr.Data(r)
+ return data[0]&(1<<3) != 0
+}
+
func decodetypeFuncInType(ldr *loader.Loader, arch *sys.Arch, symIdx loader.Sym, relocs *loader.Relocs, i int) loader.Sym {
uadd := commonsize(arch) + 4
if arch.PtrSize == 8 {
@@ -204,12 +213,18 @@ func decodetypeStructFieldType(ldr *loader.Loader, arch *sys.Arch, symIdx loader
return decodeRelocSym(ldr, symIdx, &relocs, int32(off+arch.PtrSize))
}
-func decodetypeStructFieldOffsAnon(ldr *loader.Loader, arch *sys.Arch, symIdx loader.Sym, i int) int64 {
+func decodetypeStructFieldOffset(ldr *loader.Loader, arch *sys.Arch, symIdx loader.Sym, i int) int64 {
off := decodetypeStructFieldArrayOff(ldr, arch, symIdx, i)
data := ldr.Data(symIdx)
return int64(decodeInuxi(arch, data[off+2*arch.PtrSize:], arch.PtrSize))
}
+func decodetypeStructFieldEmbedded(ldr *loader.Loader, arch *sys.Arch, symIdx loader.Sym, i int) bool {
+ off := decodetypeStructFieldArrayOff(ldr, arch, symIdx, i)
+ relocs := ldr.Relocs(symIdx)
+ return decodetypeNameEmbedded(ldr, symIdx, &relocs, off)
+}
+
// decodetypeStr returns the contents of an rtype's str field (a nameOff).
func decodetypeStr(ldr *loader.Loader, arch *sys.Arch, symIdx loader.Sym) string {
relocs := ldr.Relocs(symIdx)
diff --git a/src/cmd/link/internal/ld/dwarf.go b/src/cmd/link/internal/ld/dwarf.go
index 6ed9697aec..c42511ea3f 100644
--- a/src/cmd/link/internal/ld/dwarf.go
+++ b/src/cmd/link/internal/ld/dwarf.go
@@ -682,9 +682,9 @@ func (d *dwctxt) newtype(gotype loader.Sym) *dwarf.DWDie {
}
fld := d.newdie(die, dwarf.DW_ABRV_STRUCTFIELD, f)
d.newrefattr(fld, dwarf.DW_AT_type, d.defgotype(s))
- offsetAnon := decodetypeStructFieldOffsAnon(d.ldr, d.arch, gotype, i)
- newmemberoffsetattr(fld, int32(offsetAnon>>1))
- if offsetAnon&1 != 0 { // is embedded field
+ offset := decodetypeStructFieldOffset(d.ldr, d.arch, gotype, i)
+ newmemberoffsetattr(fld, int32(offset))
+ if decodetypeStructFieldEmbedded(d.ldr, d.arch, gotype, i) {
newattr(fld, dwarf.DW_AT_go_embedded_field, dwarf.DW_CLS_FLAG, 1, 0)
}
}
diff --git a/src/cmd/link/internal/ld/lib.go b/src/cmd/link/internal/ld/lib.go
index 19678adbd5..9a5d89a6f7 100644
--- a/src/cmd/link/internal/ld/lib.go
+++ b/src/cmd/link/internal/ld/lib.go
@@ -1463,12 +1463,12 @@ func (ctxt *Link) hostlink() {
// We force all symbol resolution to be done at program startup
// because lazy PLT resolution can use large amounts of stack at
// times we cannot allow it to do so.
- argv = append(argv, "-Wl,-znow")
+ argv = append(argv, "-Wl,-z,now")
// Do not let the host linker generate COPY relocations. These
// can move symbols out of sections that rely on stable offsets
// from the beginning of the section (like sym.STYPE).
- argv = append(argv, "-Wl,-znocopyreloc")
+ argv = append(argv, "-Wl,-z,nocopyreloc")
if buildcfg.GOOS == "android" {
// Use lld to avoid errors from default linker (issue #38838)
diff --git a/src/cmd/link/internal/ld/symtab.go b/src/cmd/link/internal/ld/symtab.go
index cc6a2c0e10..ee963bc366 100644
--- a/src/cmd/link/internal/ld/symtab.go
+++ b/src/cmd/link/internal/ld/symtab.go
@@ -475,16 +475,19 @@ func (ctxt *Link) symtab(pcln *pclntab) []sym.SymKind {
s = ldr.CreateSymForUpdate("type.*", 0)
s.SetType(sym.STYPE)
s.SetSize(0)
+ s.SetAlign(int32(ctxt.Arch.PtrSize))
symtype = s.Sym()
s = ldr.CreateSymForUpdate("typerel.*", 0)
s.SetType(sym.STYPERELRO)
s.SetSize(0)
+ s.SetAlign(int32(ctxt.Arch.PtrSize))
symtyperel = s.Sym()
} else {
s = ldr.CreateSymForUpdate("type.*", 0)
s.SetType(sym.STYPE)
s.SetSize(0)
+ s.SetAlign(int32(ctxt.Arch.PtrSize))
symtype = s.Sym()
symtyperel = s.Sym()
}
@@ -496,6 +499,7 @@ func (ctxt *Link) symtab(pcln *pclntab) []sym.SymKind {
s := ldr.CreateSymForUpdate(name, 0)
s.SetType(t)
s.SetSize(0)
+ s.SetAlign(int32(ctxt.Arch.PtrSize))
s.SetLocal(true)
setCarrierSym(t, s.Sym())
return s.Sym()
diff --git a/src/cmd/link/internal/loadpe/ldpe.go b/src/cmd/link/internal/loadpe/ldpe.go
index bfe2e837c9..bc66252cfa 100644
--- a/src/cmd/link/internal/loadpe/ldpe.go
+++ b/src/cmd/link/internal/loadpe/ldpe.go
@@ -551,7 +551,13 @@ func (state *peLoaderState) readpesym(pesym *pe.COFFSymbol) (*loader.SymbolBuild
name = strings.TrimPrefix(name, "__imp_") // __imp_Name => Name
}
}
- if state.arch.Family == sys.I386 && name[0] == '_' {
+ // A note on the "_main" exclusion below: the main routine
+ // defined by the Go runtime is named "_main", not "main", so
+ // when reading references to _main from a host object we want
+ // to avoid rewriting "_main" to "main" in this specific
+ // instance. See #issuecomment-1143698749 on #35006 for more
+ // details on this problem.
+ if state.arch.Family == sys.I386 && name[0] == '_' && name != "_main" {
name = name[1:] // _Name => Name
}
}
diff --git a/src/crypto/aes/cipher.go b/src/crypto/aes/cipher.go
index 29d01796eb..db0ee38b78 100644
--- a/src/crypto/aes/cipher.go
+++ b/src/crypto/aes/cipher.go
@@ -6,12 +6,11 @@ package aes
import (
"crypto/cipher"
+ "crypto/internal/boring"
"crypto/internal/subtle"
"strconv"
)
-import "crypto/internal/boring"
-
// The AES block size in bytes.
const BlockSize = 16
diff --git a/src/crypto/aes/cipher_asm.go b/src/crypto/aes/cipher_asm.go
index b7e59d7edb..1482b22d08 100644
--- a/src/crypto/aes/cipher_asm.go
+++ b/src/crypto/aes/cipher_asm.go
@@ -8,13 +8,12 @@ package aes
import (
"crypto/cipher"
+ "crypto/internal/boring"
"crypto/internal/subtle"
"internal/cpu"
"internal/goarch"
)
-import "crypto/internal/boring"
-
// defined in asm_*.s
//go:noescape
diff --git a/src/crypto/ecdsa/ecdsa.go b/src/crypto/ecdsa/ecdsa.go
index 7ce7542872..d0e52ad864 100644
--- a/src/crypto/ecdsa/ecdsa.go
+++ b/src/crypto/ecdsa/ecdsa.go
@@ -24,6 +24,7 @@ import (
"crypto/aes"
"crypto/cipher"
"crypto/elliptic"
+ "crypto/internal/boring"
"crypto/internal/boring/bbig"
"crypto/internal/randutil"
"crypto/sha512"
@@ -31,8 +32,6 @@ import (
"io"
"math/big"
- "crypto/internal/boring"
-
"golang.org/x/crypto/cryptobyte"
"golang.org/x/crypto/cryptobyte/asn1"
)
diff --git a/src/crypto/hmac/hmac.go b/src/crypto/hmac/hmac.go
index 34805765d5..ed3ebc0602 100644
--- a/src/crypto/hmac/hmac.go
+++ b/src/crypto/hmac/hmac.go
@@ -22,12 +22,11 @@ timing side-channels:
package hmac
import (
+ "crypto/internal/boring"
"crypto/subtle"
"hash"
)
-import "crypto/internal/boring"
-
// FIPS 198-1:
// https://csrc.nist.gov/publications/fips/fips198-1/FIPS-198-1_final.pdf
diff --git a/src/crypto/internal/boring/aes.go b/src/crypto/internal/boring/aes.go
index 515b60bb8a..eaa1adc892 100644
--- a/src/crypto/internal/boring/aes.go
+++ b/src/crypto/internal/boring/aes.go
@@ -72,9 +72,6 @@ type extraModes interface {
NewCBCDecrypter(iv []byte) cipher.BlockMode
NewCTR(iv []byte) cipher.Stream
NewGCM(nonceSize, tagSize int) (cipher.AEAD, error)
-
- // Invented for BoringCrypto.
- NewGCMTLS() (cipher.AEAD, error)
}
var _ extraModes = (*aesCipher)(nil)
@@ -235,8 +232,8 @@ func (c *aesCipher) NewGCM(nonceSize, tagSize int) (cipher.AEAD, error) {
return c.newGCM(false)
}
-func (c *aesCipher) NewGCMTLS() (cipher.AEAD, error) {
- return c.newGCM(true)
+func NewGCMTLS(c cipher.Block) (cipher.AEAD, error) {
+ return c.(*aesCipher).newGCM(true)
}
func (c *aesCipher) newGCM(tls bool) (cipher.AEAD, error) {
diff --git a/src/crypto/internal/boring/notboring.go b/src/crypto/internal/boring/notboring.go
index bb88fb0004..53096a68d1 100644
--- a/src/crypto/internal/boring/notboring.go
+++ b/src/crypto/internal/boring/notboring.go
@@ -50,6 +50,7 @@ func SHA512([]byte) [64]byte { panic("boringcrypto: not available") }
func NewHMAC(h func() hash.Hash, key []byte) hash.Hash { panic("boringcrypto: not available") }
func NewAESCipher(key []byte) (cipher.Block, error) { panic("boringcrypto: not available") }
+func NewGCMTLS(cipher.Block) (cipher.AEAD, error) { panic("boringcrypto: not available") }
type PublicKeyECDSA struct{ _ int }
type PrivateKeyECDSA struct{ _ int }
diff --git a/src/crypto/rand/rand_unix.go b/src/crypto/rand/rand_unix.go
index 830983c74a..746e90cc91 100644
--- a/src/crypto/rand/rand_unix.go
+++ b/src/crypto/rand/rand_unix.go
@@ -10,6 +10,7 @@
package rand
import (
+ "crypto/internal/boring"
"errors"
"io"
"os"
@@ -19,8 +20,6 @@ import (
"time"
)
-import "crypto/internal/boring"
-
const urandomDevice = "/dev/urandom"
func init() {
diff --git a/src/crypto/rsa/pkcs1v15.go b/src/crypto/rsa/pkcs1v15.go
index 8cf3b6e255..ab19229a6c 100644
--- a/src/crypto/rsa/pkcs1v15.go
+++ b/src/crypto/rsa/pkcs1v15.go
@@ -6,16 +6,14 @@ package rsa
import (
"crypto"
+ "crypto/internal/boring"
+ "crypto/internal/randutil"
"crypto/subtle"
"errors"
"io"
"math/big"
-
- "crypto/internal/randutil"
)
-import "crypto/internal/boring"
-
// This file implements encryption and decryption using PKCS #1 v1.5 padding.
// PKCS1v15DecrypterOpts is for passing options to PKCS #1 v1.5 decryption using
@@ -32,7 +30,7 @@ type PKCS1v15DecryptOptions struct {
// scheme from PKCS #1 v1.5. The message must be no longer than the
// length of the public modulus minus 11 bytes.
//
-// The rand parameter is used as a source of entropy to ensure that
+// The random parameter is used as a source of entropy to ensure that
// encrypting the same message twice doesn't result in the same
// ciphertext.
//
@@ -84,14 +82,14 @@ func EncryptPKCS1v15(random io.Reader, pub *PublicKey, msg []byte) ([]byte, erro
}
// DecryptPKCS1v15 decrypts a plaintext using RSA and the padding scheme from PKCS #1 v1.5.
-// If rand != nil, it uses RSA blinding to avoid timing side-channel attacks.
+// If random != nil, it uses RSA blinding to avoid timing side-channel attacks.
//
// Note that whether this function returns an error or not discloses secret
// information. If an attacker can cause this function to run repeatedly and
// learn whether each instance returned an error then they can decrypt and
// forge signatures as if they had the private key. See
// DecryptPKCS1v15SessionKey for a way of solving this problem.
-func DecryptPKCS1v15(rand io.Reader, priv *PrivateKey, ciphertext []byte) ([]byte, error) {
+func DecryptPKCS1v15(random io.Reader, priv *PrivateKey, ciphertext []byte) ([]byte, error) {
if err := checkPub(&priv.PublicKey); err != nil {
return nil, err
}
@@ -108,7 +106,7 @@ func DecryptPKCS1v15(rand io.Reader, priv *PrivateKey, ciphertext []byte) ([]byt
return out, nil
}
- valid, out, index, err := decryptPKCS1v15(rand, priv, ciphertext)
+ valid, out, index, err := decryptPKCS1v15(random, priv, ciphertext)
if err != nil {
return nil, err
}
@@ -119,7 +117,7 @@ func DecryptPKCS1v15(rand io.Reader, priv *PrivateKey, ciphertext []byte) ([]byt
}
// DecryptPKCS1v15SessionKey decrypts a session key using RSA and the padding scheme from PKCS #1 v1.5.
-// If rand != nil, it uses RSA blinding to avoid timing side-channel attacks.
+// If random != nil, it uses RSA blinding to avoid timing side-channel attacks.
// It returns an error if the ciphertext is the wrong length or if the
// ciphertext is greater than the public modulus. Otherwise, no error is
// returned. If the padding is valid, the resulting plaintext message is copied
@@ -137,7 +135,7 @@ func DecryptPKCS1v15(rand io.Reader, priv *PrivateKey, ciphertext []byte) ([]byt
// a random value was used (because it'll be different for the same ciphertext)
// and thus whether the padding was correct. This defeats the point of this
// function. Using at least a 16-byte key will protect against this attack.
-func DecryptPKCS1v15SessionKey(rand io.Reader, priv *PrivateKey, ciphertext []byte, key []byte) error {
+func DecryptPKCS1v15SessionKey(random io.Reader, priv *PrivateKey, ciphertext []byte, key []byte) error {
if err := checkPub(&priv.PublicKey); err != nil {
return err
}
@@ -146,7 +144,7 @@ func DecryptPKCS1v15SessionKey(rand io.Reader, priv *PrivateKey, ciphertext []by
return ErrDecryption
}
- valid, em, index, err := decryptPKCS1v15(rand, priv, ciphertext)
+ valid, em, index, err := decryptPKCS1v15(random, priv, ciphertext)
if err != nil {
return err
}
@@ -163,12 +161,12 @@ func DecryptPKCS1v15SessionKey(rand io.Reader, priv *PrivateKey, ciphertext []by
}
// decryptPKCS1v15 decrypts ciphertext using priv and blinds the operation if
-// rand is not nil. It returns one or zero in valid that indicates whether the
+// random is not nil. It returns one or zero in valid that indicates whether the
// plaintext was correctly structured. In either case, the plaintext is
// returned in em so that it may be read independently of whether it was valid
// in order to maintain constant memory access patterns. If the plaintext was
// valid then index contains the index of the original message in em.
-func decryptPKCS1v15(rand io.Reader, priv *PrivateKey, ciphertext []byte) (valid int, em []byte, index int, err error) {
+func decryptPKCS1v15(random io.Reader, priv *PrivateKey, ciphertext []byte) (valid int, em []byte, index int, err error) {
k := priv.Size()
if k < 11 {
err = ErrDecryption
@@ -188,7 +186,7 @@ func decryptPKCS1v15(rand io.Reader, priv *PrivateKey, ciphertext []byte) (valid
} else {
c := new(big.Int).SetBytes(ciphertext)
var m *big.Int
- m, err = decrypt(rand, priv, c)
+ m, err = decrypt(random, priv, c)
if err != nil {
return
}
@@ -220,15 +218,15 @@ func decryptPKCS1v15(rand io.Reader, priv *PrivateKey, ciphertext []byte) (valid
}
// nonZeroRandomBytes fills the given slice with non-zero random octets.
-func nonZeroRandomBytes(s []byte, rand io.Reader) (err error) {
- _, err = io.ReadFull(rand, s)
+func nonZeroRandomBytes(s []byte, random io.Reader) (err error) {
+ _, err = io.ReadFull(random, s)
if err != nil {
return
}
for i := 0; i < len(s); i++ {
for s[i] == 0 {
- _, err = io.ReadFull(rand, s[i:i+1])
+ _, err = io.ReadFull(random, s[i:i+1])
if err != nil {
return
}
@@ -268,7 +266,7 @@ var hashPrefixes = map[crypto.Hash][]byte{
// function. If hash is zero, hashed is signed directly. This isn't
// advisable except for interoperability.
//
-// If rand is not nil then RSA blinding will be used to avoid timing
+// If random is not nil then RSA blinding will be used to avoid timing
// side-channel attacks.
//
// This function is deterministic. Thus, if the set of possible
diff --git a/src/crypto/rsa/pss.go b/src/crypto/rsa/pss.go
index 16ebc0e6a7..29e79bd342 100644
--- a/src/crypto/rsa/pss.go
+++ b/src/crypto/rsa/pss.go
@@ -9,14 +9,13 @@ package rsa
import (
"bytes"
"crypto"
+ "crypto/internal/boring"
"errors"
"hash"
"io"
"math/big"
)
-import "crypto/internal/boring"
-
// Per RFC 8017, Section 9.1
//
// EM = MGF1 xor DB || H( 8*0x00 || mHash || salt ) || 0xbc
@@ -298,6 +297,7 @@ func SignPSS(rand io.Reader, priv *PrivateKey, hash crypto.Hash, digest []byte,
}
return boring.SignRSAPSS(bkey, hash, digest, saltLength)
}
+ boring.UnreachableExceptTests()
salt := make([]byte, saltLength)
if _, err := io.ReadFull(rand, salt); err != nil {
diff --git a/src/crypto/tls/cipher_suites.go b/src/crypto/tls/cipher_suites.go
index 3004b31698..9a1fa3104b 100644
--- a/src/crypto/tls/cipher_suites.go
+++ b/src/crypto/tls/cipher_suites.go
@@ -4,14 +4,13 @@
package tls
-import "crypto/internal/boring"
-
import (
"crypto"
"crypto/aes"
"crypto/cipher"
"crypto/des"
"crypto/hmac"
+ "crypto/internal/boring"
"crypto/rc4"
"crypto/sha1"
"crypto/sha256"
@@ -517,12 +516,9 @@ func aeadAESGCM(key, noncePrefix []byte) aead {
if err != nil {
panic(err)
}
- type gcmtls interface {
- NewGCMTLS() (cipher.AEAD, error)
- }
var aead cipher.AEAD
- if aesTLS, ok := aes.(gcmtls); ok {
- aead, err = aesTLS.NewGCMTLS()
+ if boring.Enabled {
+ aead, err = boring.NewGCMTLS(aes)
} else {
boring.Unreachable()
aead, err = cipher.NewGCM(aes)
diff --git a/src/database/sql/driver/driver.go b/src/database/sql/driver/driver.go
index 43fa579bda..daf282bf74 100644
--- a/src/database/sql/driver/driver.go
+++ b/src/database/sql/driver/driver.go
@@ -192,9 +192,9 @@ type Execer interface {
// DB.Exec will first prepare a query, execute the statement, and then
// close the statement.
//
-// ExecerContext may return ErrSkip.
+// ExecContext may return ErrSkip.
//
-// ExecerContext must honor the context timeout and return when the context is canceled.
+// ExecContext must honor the context timeout and return when the context is canceled.
type ExecerContext interface {
ExecContext(ctx context.Context, query string, args []NamedValue) (Result, error)
}
@@ -219,9 +219,9 @@ type Queryer interface {
// DB.Query will first prepare a query, execute the statement, and then
// close the statement.
//
-// QueryerContext may return ErrSkip.
+// QueryContext may return ErrSkip.
//
-// QueryerContext must honor the context timeout and return when the context is canceled.
+// QueryContext must honor the context timeout and return when the context is canceled.
type QueryerContext interface {
QueryContext(ctx context.Context, query string, args []NamedValue) (Rows, error)
}
diff --git a/src/expvar/expvar.go b/src/expvar/expvar.go
index 5629f89353..08cd05565d 100644
--- a/src/expvar/expvar.go
+++ b/src/expvar/expvar.go
@@ -118,7 +118,12 @@ func (v *Map) String() string {
if !first {
fmt.Fprintf(&b, ", ")
}
- fmt.Fprintf(&b, "%q: %v", kv.Key, kv.Value)
+ fmt.Fprintf(&b, "%q: ", kv.Key)
+ if kv.Value != nil {
+ fmt.Fprintf(&b, "%v", kv.Value)
+ } else {
+ fmt.Fprint(&b, "null")
+ }
first = false
})
fmt.Fprintf(&b, "}")
@@ -224,7 +229,8 @@ func (v *Map) Do(f func(KeyValue)) {
defer v.keysMu.RUnlock()
for _, k := range v.keys {
i, _ := v.m.Load(k)
- f(KeyValue{k, i.(Var)})
+ val, _ := i.(Var)
+ f(KeyValue{k, val})
}
}
diff --git a/src/expvar/expvar_test.go b/src/expvar/expvar_test.go
index ba95a36066..552bae8c64 100644
--- a/src/expvar/expvar_test.go
+++ b/src/expvar/expvar_test.go
@@ -261,6 +261,29 @@ func TestMapCounter(t *testing.T) {
}
}
+func TestMapNil(t *testing.T) {
+ RemoveAll()
+ const key = "key"
+ m := NewMap("issue527719")
+ m.Set(key, nil)
+ s := m.String()
+ var j any
+ if err := json.Unmarshal([]byte(s), &j); err != nil {
+ t.Fatalf("m.String() == %q isn't valid JSON: %v", s, err)
+ }
+ m2, ok := j.(map[string]any)
+ if !ok {
+ t.Fatalf("m.String() produced %T, wanted a map", j)
+ }
+ v, ok := m2[key]
+ if !ok {
+ t.Fatalf("missing %q in %v", key, m2)
+ }
+ if v != nil {
+ t.Fatalf("m[%q] = %v, want nil", key, v)
+ }
+}
+
func BenchmarkMapSet(b *testing.B) {
m := new(Map).Init()
diff --git a/src/go/build/build.go b/src/go/build/build.go
index 039b422dab..bfe3f444ca 100644
--- a/src/go/build/build.go
+++ b/src/go/build/build.go
@@ -1883,11 +1883,13 @@ func (ctxt *Context) eval(x constraint.Expr, allTags map[string]bool) bool {
// cgo (if cgo is enabled)
// $GOOS
// $GOARCH
-// boringcrypto
// ctxt.Compiler
// linux (if GOOS = android)
// solaris (if GOOS = illumos)
-// tag (if tag is listed in ctxt.BuildTags or ctxt.ReleaseTags)
+// darwin (if GOOS = ios)
+// unix (if this is a Unix GOOS)
+// boringcrypto (if GOEXPERIMENT=boringcrypto is enabled)
+// tag (if tag is listed in ctxt.BuildTags, ctxt.ToolTags, or ctxt.ReleaseTags)
//
// It records all consulted tags in allTags.
func (ctxt *Context) matchTag(name string, allTags map[string]bool) bool {
diff --git a/src/go/build/deps_test.go b/src/go/build/deps_test.go
index 5b971b93e2..1ddf8f69be 100644
--- a/src/go/build/deps_test.go
+++ b/src/go/build/deps_test.go
@@ -543,7 +543,10 @@ var depsRules = `
internal/fuzz, internal/testlog, runtime/pprof, regexp
< testing/internal/testdeps;
- OS, flag, testing, internal/cfg
+ MATH, errors, testing
+ < internal/testmath;
+
+ OS, flag, testing, internal/cfg, internal/testmath
< internal/testenv;
OS, encoding/base64
diff --git a/src/go/build/syslist.go b/src/go/build/syslist.go
index ea67662c3e..35cffce6dc 100644
--- a/src/go/build/syslist.go
+++ b/src/go/build/syslist.go
@@ -4,6 +4,10 @@
package build
+// Note that this file is read by internal/goarch/gengoarch.go and by
+// internal/goos/gengoos.go. If you change this file, look at those
+// files as well.
+
// knownOS is the list of past, present, and future known GOOS values.
// Do not remove from this list, as it is used for filename matching.
// If you add an entry to this list, look at unixOS, below.
diff --git a/src/go/doc/comment/doc.go b/src/go/doc/comment/doc.go
new file mode 100644
index 0000000000..45a476aa9a
--- /dev/null
+++ b/src/go/doc/comment/doc.go
@@ -0,0 +1,36 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+/*
+Package comment implements parsing and reformatting of Go doc comments,
+(documentation comments), which are comments that immediately precede
+a top-level declaration of a package, const, func, type, or var.
+
+Go doc comment syntax is a simplified subset of Markdown that supports
+links, headings, paragraphs, lists (without nesting), and preformatted text blocks.
+The details of the syntax are documented at https://go.dev/doc/comment.
+
+To parse the text associated with a doc comment (after removing comment markers),
+use a [Parser]:
+
+ var p comment.Parser
+ doc := p.Parse(text)
+
+The result is a [*Doc].
+To reformat it as a doc comment, HTML, Markdown, or plain text,
+use a [Printer]:
+
+ var pr comment.Printer
+ os.Stdout.Write(pr.Text(doc))
+
+The [Parser] and [Printer] types are structs whose fields can be
+modified to customize the operations.
+For details, see the documentation for those types.
+
+Use cases that need additional control over reformatting can
+implement their own logic by inspecting the parsed syntax itself.
+See the documentation for [Doc], [Block], [Text] for an overview
+and links to additional types.
+*/
+package comment
diff --git a/src/go/doc/comment/parse.go b/src/go/doc/comment/parse.go
index 83b37c32c5..4de8ce710d 100644
--- a/src/go/doc/comment/parse.go
+++ b/src/go/doc/comment/parse.go
@@ -260,7 +260,8 @@ func (d *parseDoc) lookupPkg(pkg string) (importPath string, ok bool) {
}
func isStdPkg(path string) bool {
- // TODO(rsc): Use sort.Find.
+ // TODO(rsc): Use sort.Find once we don't have to worry about
+ // copying this code into older Go environments.
i := sort.Search(len(stdPkgs), func(i int) bool { return stdPkgs[i] >= path })
return i < len(stdPkgs) && stdPkgs[i] == path
}
@@ -297,44 +298,27 @@ func (p *Parser) Parse(text string) *Doc {
// First pass: break into block structure and collect known links.
// The text is all recorded as Plain for now.
- // TODO: Break into actual block structure.
- didHeading := false
- all := lines
- for len(lines) > 0 {
- line := lines[0]
- n := len(lines)
+ var prev span
+ for _, s := range parseSpans(lines) {
var b Block
-
- switch {
- case line == "":
- // emit nothing
-
- case isList(line):
- prevWasBlank := len(lines) < len(all) && all[len(all)-len(lines)-1] == ""
- b, lines = d.list(lines, prevWasBlank)
-
- case isIndented(line):
- b, lines = d.code(lines)
-
- case (len(lines) == 1 || lines[1] == "") && !didHeading && isOldHeading(line, all, len(all)-n):
- b = d.oldHeading(line)
- didHeading = true
-
- case (len(lines) == 1 || lines[1] == "") && isHeading(line):
- b = d.heading(line)
- didHeading = true
-
+ switch s.kind {
default:
- b, lines = d.paragraph(lines)
- didHeading = false
+ panic("go/doc/comment: internal error: unknown span kind")
+ case spanList:
+ b = d.list(lines[s.start:s.end], prev.end < s.start)
+ case spanCode:
+ b = d.code(lines[s.start:s.end])
+ case spanOldHeading:
+ b = d.oldHeading(lines[s.start])
+ case spanHeading:
+ b = d.heading(lines[s.start])
+ case spanPara:
+ b = d.paragraph(lines[s.start:s.end])
}
-
if b != nil {
d.Content = append(d.Content, b)
}
- if len(lines) == n {
- lines = lines[1:]
- }
+ prev = s
}
// Second pass: interpret all the Plain text now that we know the links.
@@ -348,9 +332,172 @@ func (p *Parser) Parse(text string) *Doc {
return d.Doc
}
+// A span represents a single span of comment lines (lines[start:end])
+// of an identified kind (code, heading, paragraph, and so on).
+type span struct {
+ start int
+ end int
+ kind spanKind
+}
+
+// A spanKind describes the kind of span.
+type spanKind int
+
+const (
+ _ spanKind = iota
+ spanCode
+ spanHeading
+ spanList
+ spanOldHeading
+ spanPara
+)
+
+func parseSpans(lines []string) []span {
+ var spans []span
+
+ // The loop may process a line twice: once as unindented
+ // and again forced indented. So the maximum expected
+ // number of iterations is 2*len(lines). The repeating logic
+ // can be subtle, though, and to protect against introduction
+ // of infinite loops in future changes, we watch to see that
+ // we are not looping too much. A panic is better than a
+ // quiet infinite loop.
+ watchdog := 2 * len(lines)
+
+ i := 0
+ forceIndent := 0
+Spans:
+ for {
+ // Skip blank lines.
+ for i < len(lines) && lines[i] == "" {
+ i++
+ }
+ if i >= len(lines) {
+ break
+ }
+ if watchdog--; watchdog < 0 {
+ panic("go/doc/comment: internal error: not making progress")
+ }
+
+ var kind spanKind
+ start := i
+ end := i
+ if i < forceIndent || indented(lines[i]) {
+ // Indented (or force indented).
+ // Ends before next unindented. (Blank lines are OK.)
+ // If this is an unindented list that we are heuristically treating as indented,
+ // then accept unindented list item lines up to the first blank lines.
+ // The heuristic is disabled at blank lines to contain its effect
+ // to non-gofmt'ed sections of the comment.
+ unindentedListOK := isList(lines[i]) && i < forceIndent
+ i++
+ for i < len(lines) && (lines[i] == "" || i < forceIndent || indented(lines[i]) || (unindentedListOK && isList(lines[i]))) {
+ if lines[i] == "" {
+ unindentedListOK = false
+ }
+ i++
+ }
+
+ // Drop trailing blank lines.
+ end = i
+ for end > start && lines[end-1] == "" {
+ end--
+ }
+
+ // If indented lines are followed (without a blank line)
+ // by an unindented line ending in a brace,
+ // take that one line too. This fixes the common mistake
+ // of pasting in something like
+ //
+ // func main() {
+ // fmt.Println("hello, world")
+ // }
+ //
+ // and forgetting to indent it.
+ // The heuristic will never trigger on a gofmt'ed comment,
+ // because any gofmt'ed code block or list would be
+ // followed by a blank line or end of comment.
+ if end < len(lines) && strings.HasPrefix(lines[end], "}") {
+ end++
+ }
+
+ if isList(lines[start]) {
+ kind = spanList
+ } else {
+ kind = spanCode
+ }
+ } else {
+ // Unindented. Ends at next blank or indented line.
+ i++
+ for i < len(lines) && lines[i] != "" && !indented(lines[i]) {
+ i++
+ }
+ end = i
+
+ // If unindented lines are followed (without a blank line)
+ // by an indented line that would start a code block,
+ // check whether the final unindented lines
+ // should be left for the indented section.
+ // This can happen for the common mistakes of
+ // unindented code or unindented lists.
+ // The heuristic will never trigger on a gofmt'ed comment,
+ // because any gofmt'ed code block would have a blank line
+ // preceding it after the unindented lines.
+ if i < len(lines) && lines[i] != "" && !isList(lines[i]) {
+ switch {
+ case isList(lines[i-1]):
+ // If the final unindented line looks like a list item,
+ // this may be the first indented line wrap of
+ // a mistakenly unindented list.
+ // Leave all the unindented list items.
+ forceIndent = end
+ end--
+ for end > start && isList(lines[end-1]) {
+ end--
+ }
+
+ case strings.HasSuffix(lines[i-1], "{") || strings.HasSuffix(lines[i-1], `\`):
+ // If the final unindented line ended in { or \
+ // it is probably the start of a misindented code block.
+ // Give the user a single line fix.
+ // Often that's enough; if not, the user can fix the others themselves.
+ forceIndent = end
+ end--
+ }
+
+ if start == end && forceIndent > start {
+ i = start
+ continue Spans
+ }
+ }
+
+ // Span is either paragraph or heading.
+ if end-start == 1 && isHeading(lines[start]) {
+ kind = spanHeading
+ } else if end-start == 1 && isOldHeading(lines[start], lines, start) {
+ kind = spanOldHeading
+ } else {
+ kind = spanPara
+ }
+ }
+
+ spans = append(spans, span{start, end, kind})
+ i = end
+ }
+
+ return spans
+}
+
+// indented reports whether line is indented
+// (starts with a leading space or tab).
+func indented(line string) bool {
+ return line != "" && (line[0] == ' ' || line[0] == '\t')
+}
+
// unindent removes any common space/tab prefix
// from each line in lines, returning a copy of lines in which
// those prefixes have been trimmed from each line.
+// It also replaces any lines containing only spaces with blank lines (empty strings).
func unindent(lines []string) []string {
// Trim leading and trailing blank lines.
for len(lines) > 0 && isBlank(lines[0]) {
@@ -480,58 +627,16 @@ func (d *parseDoc) heading(line string) Block {
return &Heading{Text: []Text{Plain(strings.TrimSpace(line[1:]))}}
}
-// code returns a code block built from the indented text
-// at the start of lines, along with the remainder of the lines.
-// If there is no indented text at the start, or if the indented
-// text consists only of empty lines, code returns a nil Block.
-func (d *parseDoc) code(lines []string) (b Block, rest []string) {
- lines, rest = indented(lines)
+// code returns a code block built from the lines.
+func (d *parseDoc) code(lines []string) *Code {
body := unindent(lines)
- if len(body) == 0 {
- return nil, rest
- }
body = append(body, "") // to get final \n from Join
- return &Code{Text: strings.Join(body, "\n")}, rest
+ return &Code{Text: strings.Join(body, "\n")}
}
-// isIndented reports whether the line is indented,
-// meaning it starts with a space or tab.
-func isIndented(line string) bool {
- return line != "" && (line[0] == ' ' || line[0] == '\t')
-}
-
-// indented splits lines into an initial indented section
-// and the remaining lines, returning the two halves.
-func indented(lines []string) (indented, rest []string) {
- // Blank lines mid-run are OK, but not at the end.
- i := 0
- for i < len(lines) && (isIndented(lines[i]) || lines[i] == "") {
- i++
- }
- for i > 0 && lines[i-1] == "" {
- i--
- }
- return lines[:i], lines[i:]
-}
-
-// paragraph returns a paragraph block built from the
-// unindented text at the start of lines, along with the remainder of the lines.
-// If there is no unindented text at the start of lines,
-// then paragraph returns a nil Block.
-func (d *parseDoc) paragraph(lines []string) (b Block, rest []string) {
- // Paragraph is interrupted by any indented line,
- // which is either a list or a code block,
- // and of course by a blank line.
- // It is not interrupted by a # line - headings must stand alone.
- i := 0
- for i < len(lines) && lines[i] != "" && !isIndented(lines[i]) {
- i++
- }
- lines, rest = lines[:i], lines[i:]
- if len(lines) == 0 {
- return nil, rest
- }
-
+// paragraph returns a paragraph block built from the lines.
+// If the lines are link definitions, paragraph adds them to d and returns nil.
+func (d *parseDoc) paragraph(lines []string) Block {
// Is this a block of known links? Handle.
var defs []*LinkDef
for _, line := range lines {
@@ -547,10 +652,10 @@ func (d *parseDoc) paragraph(lines []string) (b Block, rest []string) {
d.links[def.Text] = def
}
}
- return nil, rest
+ return nil
NoDefs:
- return &Paragraph{Text: []Text{Plain(strings.Join(lines, "\n"))}}, rest
+ return &Paragraph{Text: []Text{Plain(strings.Join(lines, "\n"))}}
}
// parseLink parses a single link definition line:
@@ -581,14 +686,9 @@ func parseLink(line string) (*LinkDef, bool) {
return &LinkDef{Text: text, URL: url}, true
}
-// list returns a list built from the indented text at the start of lines,
+// list returns a list built from the indented lines,
// using forceBlankBefore as the value of the List's ForceBlankBefore field.
-// The caller is responsible for ensuring that the first line of lines
-// satisfies isList.
-// list returns the *List as a Block along with the remaining lines.
-func (d *parseDoc) list(lines []string, forceBlankBefore bool) (b Block, rest []string) {
- lines, rest = indented(lines)
-
+func (d *parseDoc) list(lines []string, forceBlankBefore bool) *List {
num, _, _ := listMarker(lines[0])
var (
list *List = &List{ForceBlankBefore: forceBlankBefore}
@@ -597,7 +697,7 @@ func (d *parseDoc) list(lines []string, forceBlankBefore bool) (b Block, rest []
)
flush := func() {
if item != nil {
- if para, _ := d.paragraph(text); para != nil {
+ if para := d.paragraph(text); para != nil {
item.Content = append(item.Content, para)
}
}
@@ -622,17 +722,14 @@ func (d *parseDoc) list(lines []string, forceBlankBefore bool) (b Block, rest []
text = append(text, strings.TrimSpace(line))
}
flush()
- return list, rest
+ return list
}
-// listMarker parses the line as an indented line beginning with a list marker.
+// listMarker parses the line as beginning with a list marker.
// If it can do that, it returns the numeric marker ("" for a bullet list),
// the rest of the line, and ok == true.
// Otherwise, it returns "", "", false.
func listMarker(line string) (num, rest string, ok bool) {
- if !isIndented(line) {
- return "", "", false
- }
line = strings.TrimSpace(line)
if line == "" {
return "", "", false
@@ -654,7 +751,7 @@ func listMarker(line string) (num, rest string, ok bool) {
return "", "", false
}
- if !isIndented(rest) || strings.TrimSpace(rest) == "" {
+ if !indented(rest) || strings.TrimSpace(rest) == "" {
return "", "", false
}
@@ -662,7 +759,8 @@ func listMarker(line string) (num, rest string, ok bool) {
}
// isList reports whether the line is the first line of a list,
-// meaning is indented and starts with a list marker.
+// meaning starts with a list marker after any indentation.
+// (The caller is responsible for checking the line is indented, as appropriate.)
func isList(line string) bool {
_, _, ok := listMarker(line)
return ok
@@ -840,6 +938,14 @@ func (d *parseDoc) parseText(out []Text, s string, autoLink bool) []Text {
}
switch {
case strings.HasPrefix(t, "``"):
+ if len(t) >= 3 && t[2] == '`' {
+ // Do not convert `` inside ```, in case people are mistakenly writing Markdown.
+ i += 3
+ for i < len(t) && t[i] == '`' {
+ i++
+ }
+ break
+ }
writeUntil(i)
w.WriteRune('“')
i += 2
diff --git a/src/go/doc/comment/testdata/code4.txt b/src/go/doc/comment/testdata/code4.txt
new file mode 100644
index 0000000000..f128c9aeff
--- /dev/null
+++ b/src/go/doc/comment/testdata/code4.txt
@@ -0,0 +1,38 @@
+-- input --
+To test, run this command:
+ go test -more
+
+Or, to test specific things, run this command:
+
+go test -more \
+ -pkg first/package \
+ -pkg second/package \
+ -pkg third/package
+
+Happy testing!
+-- gofmt --
+To test, run this command:
+
+ go test -more
+
+Or, to test specific things, run this command:
+
+ go test -more \
+ -pkg first/package \
+ -pkg second/package \
+ -pkg third/package
+
+Happy testing!
+-- markdown --
+To test, run this command:
+
+ go test -more
+
+Or, to test specific things, run this command:
+
+ go test -more \
+ -pkg first/package \
+ -pkg second/package \
+ -pkg third/package
+
+Happy testing!
diff --git a/src/go/doc/comment/testdata/code5.txt b/src/go/doc/comment/testdata/code5.txt
new file mode 100644
index 0000000000..0e340dd129
--- /dev/null
+++ b/src/go/doc/comment/testdata/code5.txt
@@ -0,0 +1,21 @@
+-- input --
+L1
+L2
+L3
+L4
+L5
+- L6 {
+ L7
+}
+L8
+-- gofmt --
+L1
+L2
+L3
+L4
+L5
+ - L6 {
+ L7
+ }
+
+L8
diff --git a/src/go/doc/comment/testdata/code6.txt b/src/go/doc/comment/testdata/code6.txt
new file mode 100644
index 0000000000..d2915d1068
--- /dev/null
+++ b/src/go/doc/comment/testdata/code6.txt
@@ -0,0 +1,24 @@
+-- input --
+Run this program:
+
+func main() {
+ fmt.Println("hello, world")
+}
+
+Or this:
+
+go func() {
+ fmt.Println("hello, world")
+}()
+-- gofmt --
+Run this program:
+
+ func main() {
+ fmt.Println("hello, world")
+ }
+
+Or this:
+
+ go func() {
+ fmt.Println("hello, world")
+ }()
diff --git a/src/go/doc/comment/testdata/list10.txt b/src/go/doc/comment/testdata/list10.txt
new file mode 100644
index 0000000000..9c49083456
--- /dev/null
+++ b/src/go/doc/comment/testdata/list10.txt
@@ -0,0 +1,13 @@
+-- input --
+
+ 1. This list
+ 2. Starts the comment
+ 3. And also has a blank line before it.
+
+All of which is a little weird.
+-- gofmt --
+ 1. This list
+ 2. Starts the comment
+ 3. And also has a blank line before it.
+
+All of which is a little weird.
diff --git a/src/go/doc/comment/testdata/list9.txt b/src/go/doc/comment/testdata/list9.txt
new file mode 100644
index 0000000000..48e4673d54
--- /dev/null
+++ b/src/go/doc/comment/testdata/list9.txt
@@ -0,0 +1,30 @@
+-- input --
+Text.
+
+1. Not a list
+2. because it is
+3. unindented.
+
+4. This one
+ is a list
+ because of the indented text.
+5. More wrapped
+ items.
+6. And unwrapped.
+
+7. The blank line stops the heuristic.
+-- gofmt --
+Text.
+
+1. Not a list
+2. because it is
+3. unindented.
+
+ 4. This one
+ is a list
+ because of the indented text.
+ 5. More wrapped
+ items.
+ 6. And unwrapped.
+
+7. The blank line stops the heuristic.
diff --git a/src/go/doc/comment/testdata/quote.txt b/src/go/doc/comment/testdata/quote.txt
index 799663af80..b64adae0b3 100644
--- a/src/go/doc/comment/testdata/quote.txt
+++ b/src/go/doc/comment/testdata/quote.txt
@@ -1,12 +1,15 @@
-- input --
Doubled single quotes like `` and '' turn into Unicode double quotes,
but single quotes ` and ' do not.
+Misplaced markdown fences ``` do not either.
-- gofmt --
Doubled single quotes like “ and ” turn into Unicode double quotes,
but single quotes ` and ' do not.
+Misplaced markdown fences ``` do not either.
-- text --
Doubled single quotes like “ and ” turn into Unicode double quotes, but single
-quotes ` and ' do not.
+quotes ` and ' do not. Misplaced markdown fences ``` do not either.
-- html --
Doubled single quotes like “ and ” turn into Unicode double quotes,
but single quotes ` and ' do not.
+Misplaced markdown fences ``` do not either.
diff --git a/src/go/doc/comment/text.go b/src/go/doc/comment/text.go
index 86e5eebe9a..6f9c2e201d 100644
--- a/src/go/doc/comment/text.go
+++ b/src/go/doc/comment/text.go
@@ -134,7 +134,6 @@ func (p *textPrinter) block(out *bytes.Buffer, x Block) {
}
// text prints the text sequence x to out.
-// TODO: Wrap lines.
func (p *textPrinter) text(out *bytes.Buffer, indent string, x []Text) {
p.oneLongLine(&p.long, x)
words := strings.Fields(p.long.String())
diff --git a/src/go/parser/parser.go b/src/go/parser/parser.go
index 18041ff808..ca2f24c8b8 100644
--- a/src/go/parser/parser.go
+++ b/src/go/parser/parser.go
@@ -1821,23 +1821,6 @@ func (p *parser) parseBinaryExpr(x ast.Expr, prec1 int, check bool) ast.Expr {
}
}
-// checkBinaryExpr checks binary expressions that were not already checked by
-// parseBinaryExpr, because the latter was called with check=false.
-func (p *parser) checkBinaryExpr(x ast.Expr) {
- bx, ok := x.(*ast.BinaryExpr)
- if !ok {
- return
- }
-
- bx.X = p.checkExpr(bx.X)
- bx.Y = p.checkExpr(bx.Y)
-
- // parseBinaryExpr checks x and y for each binary expr in a tree, so we
- // traverse the tree of binary exprs starting from x.
- p.checkBinaryExpr(bx.X)
- p.checkBinaryExpr(bx.Y)
-}
-
// The result may be a type or even a raw type ([...]int). Callers must
// check the result (using checkExpr or checkExprOrType), depending on
// context.
diff --git a/src/go/types/api_test.go b/src/go/types/api_test.go
index eb17f9280d..db2ace5feb 100644
--- a/src/go/types/api_test.go
+++ b/src/go/types/api_test.go
@@ -43,7 +43,7 @@ func pkgForMode(path, source string, info *Info, mode parser.Mode) (*Package, er
return conf.Check(f.Name.Name, fset, []*ast.File{f}, info)
}
-func mustTypecheck(t *testing.T, path, source string, info *Info) string {
+func mustTypecheck(t testing.TB, path, source string, info *Info) string {
pkg, err := pkgFor(path, source, info)
if err != nil {
name := path
diff --git a/src/go/types/call.go b/src/go/types/call.go
index 3c7c3226f6..c580885a5a 100644
--- a/src/go/types/call.go
+++ b/src/go/types/call.go
@@ -73,13 +73,13 @@ func (check *Checker) instantiateSignature(pos token.Pos, typ *Signature, targs
}()
}
- inst := check.instance(pos, typ, targs, check.bestContext(nil)).(*Signature)
+ inst := check.instance(pos, typ, targs, nil, check.context()).(*Signature)
assert(len(xlist) <= len(targs))
// verify instantiation lazily (was issue #50450)
check.later(func() {
tparams := typ.TypeParams().list()
- if i, err := check.verify(pos, tparams, targs); err != nil {
+ if i, err := check.verify(pos, tparams, targs, check.context()); err != nil {
// best position for error reporting
pos := pos
if i < len(xlist) {
@@ -400,7 +400,7 @@ func (check *Checker) arguments(call *ast.CallExpr, sig *Signature, targs []Type
// need to compute it from the adjusted list; otherwise we can
// simply use the result signature's parameter list.
if adjusted {
- sigParams = check.subst(call.Pos(), sigParams, makeSubstMap(sig.TypeParams().list(), targs), nil).(*Tuple)
+ sigParams = check.subst(call.Pos(), sigParams, makeSubstMap(sig.TypeParams().list(), targs), nil, check.context()).(*Tuple)
} else {
sigParams = rsig.params
}
diff --git a/src/go/types/check.go b/src/go/types/check.go
index d920d9c080..b787c5c38b 100644
--- a/src/go/types/check.go
+++ b/src/go/types/check.go
@@ -105,7 +105,7 @@ type Checker struct {
nextID uint64 // unique Id for type parameters (first valid Id is 1)
objMap map[Object]*declInfo // maps package-level objects and (non-interface) methods to declaration info
impMap map[importKey]*Package // maps (import path, source directory) to (complete or fake) package
- infoMap map[*Named]typeInfo // maps named types to their associated type info (for cycle detection)
+ valids instanceLookup // valid *Named (incl. instantiated) types per the validType check
// pkgPathMap maps package names to the set of distinct import paths we've
// seen for that name, anywhere in the import graph. It is used for
@@ -249,7 +249,6 @@ func NewChecker(conf *Config, fset *token.FileSet, pkg *Package, info *Info) *Ch
version: version,
objMap: make(map[Object]*declInfo),
impMap: make(map[importKey]*Package),
- infoMap: make(map[*Named]typeInfo),
}
}
diff --git a/src/go/types/decl.go b/src/go/types/decl.go
index 123d296791..c176042852 100644
--- a/src/go/types/decl.go
+++ b/src/go/types/decl.go
@@ -565,7 +565,7 @@ func (check *Checker) typeDecl(obj *TypeName, tdecl *ast.TypeSpec, def *Named) {
}
// type definition or generic type declaration
- named := check.newNamed(obj, nil, nil, nil)
+ named := check.newNamed(obj, nil, nil)
def.setUnderlying(named)
if tdecl.TypeParams != nil {
@@ -579,8 +579,8 @@ func (check *Checker) typeDecl(obj *TypeName, tdecl *ast.TypeSpec, def *Named) {
assert(rhs != nil)
named.fromRHS = rhs
- // If the underlying was not set while type-checking the right-hand side, it
- // is invalid and an error should have been reported elsewhere.
+ // If the underlying type was not set while type-checking the right-hand
+ // side, it is invalid and an error should have been reported elsewhere.
if named.underlying == nil {
named.underlying = Typ[Invalid]
}
@@ -711,7 +711,7 @@ func (check *Checker) collectMethods(obj *TypeName) {
// and field names must be distinct."
base, _ := obj.typ.(*Named) // shouldn't fail but be conservative
if base != nil {
- assert(base.targs.Len() == 0) // collectMethods should not be called on an instantiated type
+ assert(base.TypeArgs().Len() == 0) // collectMethods should not be called on an instantiated type
// See issue #52529: we must delay the expansion of underlying here, as
// base may not be fully set-up.
@@ -722,8 +722,8 @@ func (check *Checker) collectMethods(obj *TypeName) {
// Checker.Files may be called multiple times; additional package files
// may add methods to already type-checked types. Add pre-existing methods
// so that we can detect redeclarations.
- for i := 0; i < base.methods.Len(); i++ {
- m := base.methods.At(i, nil)
+ for i := 0; i < base.NumMethods(); i++ {
+ m := base.Method(i)
assert(m.name != "_")
assert(mset.insert(m) == nil)
}
@@ -749,8 +749,8 @@ func (check *Checker) collectMethods(obj *TypeName) {
func (check *Checker) checkFieldUniqueness(base *Named) {
if t, _ := base.under().(*Struct); t != nil {
var mset objset
- for i := 0; i < base.methods.Len(); i++ {
- m := base.methods.At(i, nil)
+ for i := 0; i < base.NumMethods(); i++ {
+ m := base.Method(i)
assert(m.name != "_")
assert(mset.insert(m) == nil)
}
diff --git a/src/go/types/infer.go b/src/go/types/infer.go
index 031850b8da..1aa2612638 100644
--- a/src/go/types/infer.go
+++ b/src/go/types/infer.go
@@ -110,11 +110,11 @@ func (check *Checker) infer(posn positioner, tparams []*TypeParam, targs []Type,
renameMap := makeRenameMap(tparams, tparams2)
for i, tparam := range tparams {
- tparams2[i].bound = check.subst(posn.Pos(), tparam.bound, renameMap, nil)
+ tparams2[i].bound = check.subst(posn.Pos(), tparam.bound, renameMap, nil, check.context())
}
tparams = tparams2
- params = check.subst(posn.Pos(), params, renameMap, nil).(*Tuple)
+ params = check.subst(posn.Pos(), params, renameMap, nil, check.context()).(*Tuple)
}
}
@@ -188,7 +188,7 @@ func (check *Checker) infer(posn positioner, tparams []*TypeParam, targs []Type,
// but that doesn't impact the isParameterized check for now).
if params.Len() > 0 {
smap := makeSubstMap(tparams, targs)
- params = check.subst(token.NoPos, params, smap, nil).(*Tuple)
+ params = check.subst(token.NoPos, params, smap, nil, check.context()).(*Tuple)
}
// Unify parameter and argument types for generic parameters with typed arguments
@@ -225,7 +225,7 @@ func (check *Checker) infer(posn positioner, tparams []*TypeParam, targs []Type,
}
smap := makeSubstMap(tparams, targs)
// TODO(rFindley): pass a positioner here, rather than arg.Pos().
- inferred := check.subst(arg.Pos(), tpar, smap, nil)
+ inferred := check.subst(arg.Pos(), tpar, smap, nil, check.context())
// _CannotInferTypeArgs indicates a failure of inference, though the actual
// error may be better attributed to a user-provided type argument (hence
// _InvalidTypeArg). We can't differentiate these cases, so fall back on
@@ -434,7 +434,7 @@ func (w *tpWalker) isParameterized(typ Type) (res bool) {
return w.isParameterized(t.elem)
case *Named:
- return w.isParameterizedTypeList(t.targs.list())
+ return w.isParameterizedTypeList(t.TypeArgs().list())
case *TypeParam:
// t must be one of w.tparams
@@ -626,7 +626,7 @@ func (check *Checker) inferB(posn positioner, tparams []*TypeParam, targs []Type
n := 0
for _, index := range dirty {
t0 := types[index]
- if t1 := check.subst(token.NoPos, t0, smap, nil); t1 != t0 {
+ if t1 := check.subst(token.NoPos, t0, smap, nil, check.context()); t1 != t0 {
types[index] = t1
dirty[n] = index
n++
diff --git a/src/go/types/instantiate.go b/src/go/types/instantiate.go
index 964a4f907c..f7505854d1 100644
--- a/src/go/types/instantiate.go
+++ b/src/go/types/instantiate.go
@@ -40,6 +40,9 @@ import (
// count is incorrect; for *Named types, a panic may occur later inside the
// *Named API.
func Instantiate(ctxt *Context, orig Type, targs []Type, validate bool) (Type, error) {
+ if ctxt == nil {
+ ctxt = NewContext()
+ }
if validate {
var tparams []*TypeParam
switch t := orig.(type) {
@@ -51,40 +54,72 @@ func Instantiate(ctxt *Context, orig Type, targs []Type, validate bool) (Type, e
if len(targs) != len(tparams) {
return nil, fmt.Errorf("got %d type arguments but %s has %d type parameters", len(targs), orig, len(tparams))
}
- if i, err := (*Checker)(nil).verify(token.NoPos, tparams, targs); err != nil {
+ if i, err := (*Checker)(nil).verify(token.NoPos, tparams, targs, ctxt); err != nil {
return nil, &ArgumentError{i, err}
}
}
- inst := (*Checker)(nil).instance(token.NoPos, orig, targs, ctxt)
+ inst := (*Checker)(nil).instance(token.NoPos, orig, targs, nil, ctxt)
return inst, nil
}
-// instance creates a type or function instance using the given original type
-// typ and arguments targs. For Named types the resulting instance will be
-// unexpanded. check may be nil.
-func (check *Checker) instance(pos token.Pos, orig Type, targs []Type, ctxt *Context) (res Type) {
- var h string
+// instance instantiates the given original (generic) function or type with the
+// provided type arguments and returns the resulting instance. If an identical
+// instance exists already in the given contexts, it returns that instance,
+// otherwise it creates a new one.
+//
+// If expanding is non-nil, it is the Named instance type currently being
+// expanded. If ctxt is non-nil, it is the context associated with the current
+// type-checking pass or call to Instantiate. At least one of expanding or ctxt
+// must be non-nil.
+//
+// For Named types the resulting instance may be unexpanded.
+func (check *Checker) instance(pos token.Pos, orig Type, targs []Type, expanding *Named, ctxt *Context) (res Type) {
+ // The order of the contexts below matters: we always prefer instances in the
+ // expanding instance context in order to preserve reference cycles.
+ //
+ // Invariant: if expanding != nil, the returned instance will be the instance
+ // recorded in expanding.inst.ctxt.
+ var ctxts []*Context
+ if expanding != nil {
+ ctxts = append(ctxts, expanding.inst.ctxt)
+ }
if ctxt != nil {
- h = ctxt.instanceHash(orig, targs)
- // typ may already have been instantiated with identical type arguments. In
- // that case, re-use the existing instance.
- if inst := ctxt.lookup(h, orig, targs); inst != nil {
- return inst
+ ctxts = append(ctxts, ctxt)
+ }
+ assert(len(ctxts) > 0)
+
+ // Compute all hashes; hashes may differ across contexts due to different
+ // unique IDs for Named types within the hasher.
+ hashes := make([]string, len(ctxts))
+ for i, ctxt := range ctxts {
+ hashes[i] = ctxt.instanceHash(orig, targs)
+ }
+
+ // If local is non-nil, updateContexts return the type recorded in
+ // local.
+ updateContexts := func(res Type) Type {
+ for i := len(ctxts) - 1; i >= 0; i-- {
+ res = ctxts[i].update(hashes[i], orig, targs, res)
+ }
+ return res
+ }
+
+ // typ may already have been instantiated with identical type arguments. In
+ // that case, re-use the existing instance.
+ for i, ctxt := range ctxts {
+ if inst := ctxt.lookup(hashes[i], orig, targs); inst != nil {
+ return updateContexts(inst)
}
}
switch orig := orig.(type) {
case *Named:
- tname := NewTypeName(pos, orig.obj.pkg, orig.obj.name, nil)
- named := check.newNamed(tname, orig, nil, nil) // underlying, tparams, and methods are set when named is resolved
- named.targs = newTypeList(targs)
- named.resolver = func(ctxt *Context, n *Named) (*TypeParamList, Type, *methodList) {
- return expandNamed(ctxt, n, pos)
- }
- res = named
+ res = check.newNamedInstance(pos, orig, targs, expanding) // substituted lazily
case *Signature:
+ assert(expanding == nil) // function instances cannot be reached from Named types
+
tparams := orig.TypeParams()
if !check.validateTArgLen(pos, tparams.Len(), len(targs)) {
return Typ[Invalid]
@@ -92,7 +127,7 @@ func (check *Checker) instance(pos token.Pos, orig Type, targs []Type, ctxt *Con
if tparams.Len() == 0 {
return orig // nothing to do (minor optimization)
}
- sig := check.subst(pos, orig, makeSubstMap(tparams.list(), targs), ctxt).(*Signature)
+ sig := check.subst(pos, orig, makeSubstMap(tparams.list(), targs), nil, ctxt).(*Signature)
// If the signature doesn't use its type parameters, subst
// will not make a copy. In that case, make a copy now (so
// we can set tparams to nil w/o causing side-effects).
@@ -110,13 +145,8 @@ func (check *Checker) instance(pos token.Pos, orig Type, targs []Type, ctxt *Con
panic(fmt.Sprintf("%v: cannot instantiate %v", pos, orig))
}
- if ctxt != nil {
- // It's possible that we've lost a race to add named to the context.
- // In this case, use whichever instance is recorded in the context.
- res = ctxt.update(h, orig, targs, res)
- }
-
- return res
+ // Update all contexts; it's possible that we've lost a race.
+ return updateContexts(res)
}
// validateTArgLen verifies that the length of targs and tparams matches,
@@ -134,7 +164,7 @@ func (check *Checker) validateTArgLen(pos token.Pos, ntparams, ntargs int) bool
return true
}
-func (check *Checker) verify(pos token.Pos, tparams []*TypeParam, targs []Type) (int, error) {
+func (check *Checker) verify(pos token.Pos, tparams []*TypeParam, targs []Type, ctxt *Context) (int, error) {
smap := makeSubstMap(tparams, targs)
for i, tpar := range tparams {
// Ensure that we have a (possibly implicit) interface as type bound (issue #51048).
@@ -143,7 +173,7 @@ func (check *Checker) verify(pos token.Pos, tparams []*TypeParam, targs []Type)
// as the instantiated type; before we can use it for bounds checking we
// need to instantiate it with the type arguments with which we instantiated
// the parameterized type.
- bound := check.subst(pos, tpar.bound, smap, nil)
+ bound := check.subst(pos, tpar.bound, smap, nil, ctxt)
if err := check.implements(targs[i], bound); err != nil {
return i, err
}
@@ -248,7 +278,7 @@ func (check *Checker) implements(V, T Type) error {
if alt != nil {
return errorf("%s does not implement %s (possibly missing ~ for %s in constraint %s)", V, T, alt, T)
} else {
- return errorf("%s does not implement %s", V, T)
+ return errorf("%s does not implement %s (%s missing in %s)", V, T, V, Ti.typeSet().terms)
}
}
diff --git a/src/go/types/methodlist.go b/src/go/types/methodlist.go
deleted file mode 100644
index afe919013d..0000000000
--- a/src/go/types/methodlist.go
+++ /dev/null
@@ -1,79 +0,0 @@
-// Copyright 2022 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package types
-
-import "sync"
-
-// methodList holds a list of methods that may be lazily resolved by a provided
-// resolution method.
-type methodList struct {
- methods []*Func
-
- // guards synchronizes the instantiation of lazy methods. For lazy method
- // lists, guards is non-nil and of the length passed to newLazyMethodList.
- // For non-lazy method lists, guards is nil.
- guards *[]sync.Once
-}
-
-// newMethodList creates a non-lazy method list holding the given methods.
-func newMethodList(methods []*Func) *methodList {
- return &methodList{methods: methods}
-}
-
-// newLazyMethodList creates a lazy method list of the given length. Methods
-// may be resolved lazily for a given index by providing a resolver function.
-func newLazyMethodList(length int) *methodList {
- guards := make([]sync.Once, length)
- return &methodList{
- methods: make([]*Func, length),
- guards: &guards,
- }
-}
-
-// isLazy reports whether the receiver is a lazy method list.
-func (l *methodList) isLazy() bool {
- return l != nil && l.guards != nil
-}
-
-// Add appends a method to the method list if not not already present. Add
-// panics if the receiver is lazy.
-func (l *methodList) Add(m *Func) {
- assert(!l.isLazy())
- if i, _ := lookupMethod(l.methods, m.pkg, m.name, false); i < 0 {
- l.methods = append(l.methods, m)
- }
-}
-
-// Lookup looks up the method identified by pkg and name in the receiver.
-// Lookup panics if the receiver is lazy. If foldCase is true, method names
-// are considered equal if they are equal with case folding.
-func (l *methodList) Lookup(pkg *Package, name string, foldCase bool) (int, *Func) {
- assert(!l.isLazy())
- if l == nil {
- return -1, nil
- }
- return lookupMethod(l.methods, pkg, name, foldCase)
-}
-
-// Len returns the length of the method list.
-func (l *methodList) Len() int {
- if l == nil {
- return 0
- }
- return len(l.methods)
-}
-
-// At returns the i'th method of the method list. At panics if i is out of
-// bounds, or if the receiver is lazy and resolve is nil.
-func (l *methodList) At(i int, resolve func() *Func) *Func {
- if !l.isLazy() {
- return l.methods[i]
- }
- assert(resolve != nil)
- (*l.guards)[i].Do(func() {
- l.methods[i] = resolve()
- })
- return l.methods[i]
-}
diff --git a/src/go/types/methodlist_test.go b/src/go/types/methodlist_test.go
deleted file mode 100644
index e628bce767..0000000000
--- a/src/go/types/methodlist_test.go
+++ /dev/null
@@ -1,41 +0,0 @@
-// Copyright 2022 The Go Authors. All rights reserved.
-// Use of this source code is governed by a BSD-style
-// license that can be found in the LICENSE file.
-
-package types
-
-import (
- "go/token"
- "testing"
-)
-
-func TestLazyMethodList(t *testing.T) {
- l := newLazyMethodList(2)
-
- if got := l.Len(); got != 2 {
- t.Fatalf("Len() = %d, want 2", got)
- }
-
- f0 := NewFunc(token.NoPos, nil, "f0", nil)
- f1 := NewFunc(token.NoPos, nil, "f1", nil)
-
- // Verify that methodList.At is idempotent, by calling it repeatedly with a
- // resolve func that returns different pointer values (f0 or f1).
- steps := []struct {
- index int
- resolve *Func // the *Func returned by the resolver
- want *Func // the actual *Func returned by methodList.At
- }{
- {0, f0, f0},
- {0, f1, f0},
- {1, f1, f1},
- {1, f0, f1},
- }
-
- for i, step := range steps {
- got := l.At(step.index, func() *Func { return step.resolve })
- if got != step.want {
- t.Errorf("step %d: At(%d, ...) = %s, want %s", i, step.index, got.Name(), step.want.Name())
- }
- }
-}
diff --git a/src/go/types/named.go b/src/go/types/named.go
index a82679eb10..c08997aa77 100644
--- a/src/go/types/named.go
+++ b/src/go/types/named.go
@@ -7,30 +7,136 @@ package types
import (
"go/token"
"sync"
+ "sync/atomic"
)
+// Type-checking Named types is subtle, because they may be recursively
+// defined, and because their full details may be spread across multiple
+// declarations (via methods). For this reason they are type-checked lazily,
+// to avoid information being accessed before it is complete.
+//
+// Conceptually, it is helpful to think of named types as having two distinct
+// sets of information:
+// - "LHS" information, defining their identity: Obj() and TypeArgs()
+// - "RHS" information, defining their details: TypeParams(), Underlying(),
+// and methods.
+//
+// In this taxonomy, LHS information is available immediately, but RHS
+// information is lazy. Specifically, a named type N may be constructed in any
+// of the following ways:
+// 1. type-checked from the source
+// 2. loaded eagerly from export data
+// 3. loaded lazily from export data (when using unified IR)
+// 4. instantiated from a generic type
+//
+// In cases 1, 3, and 4, it is possible that the underlying type or methods of
+// N may not be immediately available.
+// - During type-checking, we allocate N before type-checking its underlying
+// type or methods, so that we may resolve recursive references.
+// - When loading from export data, we may load its methods and underlying
+// type lazily using a provided load function.
+// - After instantiating, we lazily expand the underlying type and methods
+// (note that instances may be created while still in the process of
+// type-checking the original type declaration).
+//
+// In cases 3 and 4 this lazy construction may also occur concurrently, due to
+// concurrent use of the type checker API (after type checking or importing has
+// finished). It is critical that we keep track of state, so that Named types
+// are constructed exactly once and so that we do not access their details too
+// soon.
+//
+// We achieve this by tracking state with an atomic state variable, and
+// guarding potentially concurrent calculations with a mutex. At any point in
+// time this state variable determines which data on N may be accessed. As
+// state monotonically progresses, any data available at state M may be
+// accessed without acquiring the mutex at state N, provided N >= M.
+//
+// GLOSSARY: Here are a few terms used in this file to describe Named types:
+// - We say that a Named type is "instantiated" if it has been constructed by
+// instantiating a generic named type with type arguments.
+// - We say that a Named type is "declared" if it corresponds to a type
+// declaration in the source. Instantiated named types correspond to a type
+// instantiation in the source, not a declaration. But their Origin type is
+// a declared type.
+// - We say that a Named type is "resolved" if its RHS information has been
+// loaded or fully type-checked. For Named types constructed from export
+// data, this may involve invoking a loader function to extract information
+// from export data. For instantiated named types this involves reading
+// information from their origin.
+// - We say that a Named type is "expanded" if it is an instantiated type and
+// type parameters in its underlying type and methods have been substituted
+// with the type arguments from the instantiation. A type may be partially
+// expanded if some but not all of these details have been substituted.
+// Similarly, we refer to these individual details (underlying type or
+// method) as being "expanded".
+// - When all information is known for a named type, we say it is "complete".
+//
+// Some invariants to keep in mind: each declared Named type has a single
+// corresponding object, and that object's type is the (possibly generic) Named
+// type. Declared Named types are identical if and only if their pointers are
+// identical. On the other hand, multiple instantiated Named types may be
+// identical even though their pointers are not identical. One has to use
+// Identical to compare them. For instantiated named types, their obj is a
+// synthetic placeholder that records their position of the corresponding
+// instantiation in the source (if they were constructed during type checking).
+//
+// To prevent infinite expansion of named instances that are created outside of
+// type-checking, instances share a Context with other instances created during
+// their expansion. Via the pidgeonhole principle, this guarantees that in the
+// presence of a cycle of named types, expansion will eventually find an
+// existing instance in the Context and short-circuit the expansion.
+//
+// Once an instance is complete, we can nil out this shared Context to unpin
+// memory, though this Context may still be held by other incomplete instances
+// in its "lineage".
+
// A Named represents a named (defined) type.
type Named struct {
- check *Checker
- obj *TypeName // corresponding declared object for declared types; placeholder for instantiated types
- orig *Named // original, uninstantiated type
- fromRHS Type // type (on RHS of declaration) this *Named type is derived of (for cycle reporting)
+ check *Checker // non-nil during type-checking; nil otherwise
+ obj *TypeName // corresponding declared object for declared types; see above for instantiated types
+
+ // fromRHS holds the type (on RHS of declaration) this *Named type is derived
+ // from (for cycle reporting). Only used by validType, and therefore does not
+ // require synchronization.
+ fromRHS Type
+
+ // information for instantiated types; nil otherwise
+ inst *instance
+
+ mu sync.Mutex // guards all fields below
+ state_ uint32 // the current state of this type; must only be accessed atomically
underlying Type // possibly a *Named during setup; never a *Named once set up completely
tparams *TypeParamList // type parameters, or nil
- targs *TypeList // type arguments (after instantiation), or nil
- // methods declared for this type (not the method set of this type).
+ // methods declared for this type (not the method set of this type)
// Signatures are type-checked lazily.
// For non-instantiated types, this is a fully populated list of methods. For
- // instantiated types, this is a 'lazy' list, and methods are instantiated
- // when they are first accessed.
- methods *methodList
+ // instantiated types, methods are individually expanded when they are first
+ // accessed.
+ methods []*Func
- // resolver may be provided to lazily resolve type parameters, underlying, and methods.
- resolver func(*Context, *Named) (tparams *TypeParamList, underlying Type, methods *methodList)
- once sync.Once // ensures that tparams, underlying, and methods are resolved before accessing
+ // loader may be provided to lazily load type parameters, underlying type, and methods.
+ loader func(*Named) (tparams []*TypeParam, underlying Type, methods []*Func)
}
+// instance holds information that is only necessary for instantiated named
+// types.
+type instance struct {
+ orig *Named // original, uninstantiated type
+ targs *TypeList // type arguments
+ expandedMethods int // number of expanded methods; expandedMethods <= len(orig.methods)
+ ctxt *Context // local Context; set to nil after full expansion
+}
+
+// namedState represents the possible states that a named type may assume.
+type namedState uint32
+
+const (
+ unresolved namedState = iota // tparams, underlying type and methods might be unavailable
+ resolved // resolve has run; methods might be incomplete (for instances)
+ complete // all data is known
+)
+
// NewNamed returns a new named type for the given type name, underlying type, and associated methods.
// If the given type name obj doesn't have a type yet, its type is set to the returned named type.
// The underlying type must not be a *Named.
@@ -38,39 +144,122 @@ func NewNamed(obj *TypeName, underlying Type, methods []*Func) *Named {
if _, ok := underlying.(*Named); ok {
panic("underlying type must not be *Named")
}
- return (*Checker)(nil).newNamed(obj, nil, underlying, newMethodList(methods))
+ return (*Checker)(nil).newNamed(obj, underlying, methods)
}
-func (t *Named) resolve(ctxt *Context) *Named {
- if t.resolver == nil {
- return t
+// resolve resolves the type parameters, methods, and underlying type of n.
+// This information may be loaded from a provided loader function, or computed
+// from an origin type (in the case of instances).
+//
+// After resolution, the type parameters, methods, and underlying type of n are
+// accessible; but if n is an instantiated type, its methods may still be
+// unexpanded.
+func (n *Named) resolve() *Named {
+ if n.state() >= resolved { // avoid locking below
+ return n
}
- t.once.Do(func() {
- // TODO(mdempsky): Since we're passing t to the resolver anyway
- // (necessary because types2 expects the receiver type for methods
- // on defined interface types to be the Named rather than the
- // underlying Interface), maybe it should just handle calling
- // SetTypeParams, SetUnderlying, and AddMethod instead? Those
- // methods would need to support reentrant calls though. It would
- // also make the API more future-proof towards further extensions
- // (like SetTypeParams).
- t.tparams, t.underlying, t.methods = t.resolver(ctxt, t)
- t.fromRHS = t.underlying // for cycle detection
- })
- return t
+ // TODO(rfindley): if n.check is non-nil we can avoid locking here, since
+ // type-checking is not concurrent. Evaluate if this is worth doing.
+ n.mu.Lock()
+ defer n.mu.Unlock()
+
+ if n.state() >= resolved {
+ return n
+ }
+
+ if n.inst != nil {
+ assert(n.underlying == nil) // n is an unresolved instance
+ assert(n.loader == nil) // instances are created by instantiation, in which case n.loader is nil
+
+ orig := n.inst.orig
+ orig.resolve()
+ underlying := n.expandUnderlying()
+
+ n.tparams = orig.tparams
+ n.underlying = underlying
+ n.fromRHS = orig.fromRHS // for cycle detection
+
+ if len(orig.methods) == 0 {
+ n.setState(complete) // nothing further to do
+ n.inst.ctxt = nil
+ } else {
+ n.setState(resolved)
+ }
+ return n
+ }
+
+ // TODO(mdempsky): Since we're passing n to the loader anyway
+ // (necessary because types2 expects the receiver type for methods
+ // on defined interface types to be the Named rather than the
+ // underlying Interface), maybe it should just handle calling
+ // SetTypeParams, SetUnderlying, and AddMethod instead? Those
+ // methods would need to support reentrant calls though. It would
+ // also make the API more future-proof towards further extensions.
+ if n.loader != nil {
+ assert(n.underlying == nil)
+ assert(n.TypeArgs().Len() == 0) // instances are created by instantiation, in which case n.loader is nil
+
+ tparams, underlying, methods := n.loader(n)
+
+ n.tparams = bindTParams(tparams)
+ n.underlying = underlying
+ n.fromRHS = underlying // for cycle detection
+ n.methods = methods
+ n.loader = nil
+ }
+
+ n.setState(complete)
+ return n
+}
+
+// state atomically accesses the current state of the receiver.
+func (n *Named) state() namedState {
+ return namedState(atomic.LoadUint32(&n.state_))
+}
+
+// setState atomically stores the given state for n.
+// Must only be called while holding n.mu.
+func (n *Named) setState(state namedState) {
+ atomic.StoreUint32(&n.state_, uint32(state))
}
// newNamed is like NewNamed but with a *Checker receiver and additional orig argument.
-func (check *Checker) newNamed(obj *TypeName, orig *Named, underlying Type, methods *methodList) *Named {
- typ := &Named{check: check, obj: obj, orig: orig, fromRHS: underlying, underlying: underlying, methods: methods}
- if typ.orig == nil {
- typ.orig = typ
- }
+func (check *Checker) newNamed(obj *TypeName, underlying Type, methods []*Func) *Named {
+ typ := &Named{check: check, obj: obj, fromRHS: underlying, underlying: underlying, methods: methods}
if obj.typ == nil {
obj.typ = typ
}
- // Ensure that typ is always expanded and sanity-checked.
+ // Ensure that typ is always sanity-checked.
+ if check != nil {
+ check.needsCleanup(typ)
+ }
+ return typ
+}
+
+// newNamedInstance creates a new named instance for the given origin and type
+// arguments, recording pos as the position of its synthetic object (for error
+// reporting).
+//
+// If set, expanding is the named type instance currently being expanded, that
+// led to the creation of this instance.
+func (check *Checker) newNamedInstance(pos token.Pos, orig *Named, targs []Type, expanding *Named) *Named {
+ assert(len(targs) > 0)
+
+ obj := NewTypeName(pos, orig.obj.pkg, orig.obj.name, nil)
+ inst := &instance{orig: orig, targs: newTypeList(targs)}
+
+ // Only pass the expanding context to the new instance if their packages
+ // match. Since type reference cycles are only possible within a single
+ // package, this is sufficient for the purposes of short-circuiting cycles.
+ // Avoiding passing the context in other cases prevents unnecessary coupling
+ // of types across packages.
+ if expanding != nil && expanding.Obj().pkg == obj.pkg {
+ inst.ctxt = expanding.inst.ctxt
+ }
+ typ := &Named{check: check, obj: obj, inst: inst}
+ obj.typ = typ
+ // Ensure that typ is always sanity-checked.
if check != nil {
check.needsCleanup(typ)
}
@@ -78,18 +267,18 @@ func (check *Checker) newNamed(obj *TypeName, orig *Named, underlying Type, meth
}
func (t *Named) cleanup() {
- assert(t.orig.orig == t.orig)
+ assert(t.inst == nil || t.inst.orig.inst == nil)
// Ensure that every defined type created in the course of type-checking has
- // either non-*Named underlying, or is unresolved.
+ // either non-*Named underlying type, or is unexpanded.
//
- // This guarantees that we don't leak any types whose underlying is *Named,
- // because any unresolved instances will lazily compute their underlying by
- // substituting in the underlying of their origin. The origin must have
- // either been imported or type-checked and expanded here, and in either case
- // its underlying will be fully expanded.
+ // This guarantees that we don't leak any types whose underlying type is
+ // *Named, because any unexpanded instances will lazily compute their
+ // underlying type by substituting in the underlying type of their origin.
+ // The origin must have either been imported or type-checked and expanded
+ // here, and in either case its underlying type will be fully expanded.
switch t.underlying.(type) {
case nil:
- if t.resolver == nil {
+ if t.TypeArgs().Len() == 0 {
panic("nil underlying")
}
case *Named:
@@ -101,52 +290,90 @@ func (t *Named) cleanup() {
// Obj returns the type name for the declaration defining the named type t. For
// instantiated types, this is same as the type name of the origin type.
func (t *Named) Obj() *TypeName {
- return t.orig.obj // for non-instances this is the same as t.obj
+ if t.inst == nil {
+ return t.obj
+ }
+ return t.inst.orig.obj
}
// Origin returns the generic type from which the named type t is
// instantiated. If t is not an instantiated type, the result is t.
-func (t *Named) Origin() *Named { return t.orig }
-
-// TODO(gri) Come up with a better representation and API to distinguish
-// between parameterized instantiated and non-instantiated types.
+func (t *Named) Origin() *Named {
+ if t.inst == nil {
+ return t
+ }
+ return t.inst.orig
+}
// TypeParams returns the type parameters of the named type t, or nil.
// The result is non-nil for an (originally) generic type even if it is instantiated.
-func (t *Named) TypeParams() *TypeParamList { return t.resolve(nil).tparams }
+func (t *Named) TypeParams() *TypeParamList { return t.resolve().tparams }
// SetTypeParams sets the type parameters of the named type t.
// t must not have type arguments.
func (t *Named) SetTypeParams(tparams []*TypeParam) {
- assert(t.targs.Len() == 0)
- t.resolve(nil).tparams = bindTParams(tparams)
+ assert(t.inst == nil)
+ t.resolve().tparams = bindTParams(tparams)
}
// TypeArgs returns the type arguments used to instantiate the named type t.
-func (t *Named) TypeArgs() *TypeList { return t.targs }
-
-// NumMethods returns the number of explicit methods defined for t.
-//
-// For an ordinary or instantiated type t, the receiver base type of these
-// methods will be the named type t. For an uninstantiated generic type t, each
-// method receiver will be instantiated with its receiver type parameters.
-func (t *Named) NumMethods() int { return t.resolve(nil).methods.Len() }
-
-// Method returns the i'th method of named type t for 0 <= i < t.NumMethods().
-func (t *Named) Method(i int) *Func {
- t.resolve(nil)
- return t.methods.At(i, func() *Func {
- return t.instantiateMethod(i)
- })
+func (t *Named) TypeArgs() *TypeList {
+ if t.inst == nil {
+ return nil
+ }
+ return t.inst.targs
}
-// instantiateMethod instantiates the i'th method for an instantiated receiver.
-func (t *Named) instantiateMethod(i int) *Func {
- assert(t.TypeArgs().Len() > 0) // t must be an instance
+// NumMethods returns the number of explicit methods defined for t.
+func (t *Named) NumMethods() int {
+ return len(t.Origin().resolve().methods)
+}
+// Method returns the i'th method of named type t for 0 <= i < t.NumMethods().
+//
+// For an ordinary or instantiated type t, the receiver base type of this
+// method is the named type t. For an uninstantiated generic type t, each
+// method receiver is instantiated with its receiver type parameters.
+func (t *Named) Method(i int) *Func {
+ t.resolve()
+
+ if t.state() >= complete {
+ return t.methods[i]
+ }
+
+ assert(t.inst != nil) // only instances should have incomplete methods
+ orig := t.inst.orig
+
+ t.mu.Lock()
+ defer t.mu.Unlock()
+
+ if len(t.methods) != len(orig.methods) {
+ assert(len(t.methods) == 0)
+ t.methods = make([]*Func, len(orig.methods))
+ }
+
+ if t.methods[i] == nil {
+ assert(t.inst.ctxt != nil) // we should still have a context remaining from the resolution phase
+ t.methods[i] = t.expandMethod(i)
+ t.inst.expandedMethods++
+
+ // Check if we've created all methods at this point. If we have, mark the
+ // type as fully expanded.
+ if t.inst.expandedMethods == len(orig.methods) {
+ t.setState(complete)
+ t.inst.ctxt = nil // no need for a context anymore
+ }
+ }
+
+ return t.methods[i]
+}
+
+// expandMethod substitutes type arguments in the i'th method for an
+// instantiated receiver.
+func (t *Named) expandMethod(i int) *Func {
// t.orig.methods is not lazy. origm is the method instantiated with its
// receiver type parameters (the "origin" method).
- origm := t.orig.Method(i)
+ origm := t.inst.orig.Method(i)
assert(origm != nil)
check := t.check
@@ -173,10 +400,13 @@ func (t *Named) instantiateMethod(i int) *Func {
// We can only substitute if we have a correspondence between type arguments
// and type parameters. This check is necessary in the presence of invalid
// code.
- if origSig.RecvTypeParams().Len() == t.targs.Len() {
- ctxt := check.bestContext(nil)
- smap := makeSubstMap(origSig.RecvTypeParams().list(), t.targs.list())
- sig = check.subst(origm.pos, origSig, smap, ctxt).(*Signature)
+ if origSig.RecvTypeParams().Len() == t.inst.targs.Len() {
+ smap := makeSubstMap(origSig.RecvTypeParams().list(), t.inst.targs.list())
+ var ctxt *Context
+ if check != nil {
+ ctxt = check.context()
+ }
+ sig = check.subst(origm.pos, origSig, smap, t, ctxt).(*Signature)
}
if sig == origSig {
@@ -200,14 +430,14 @@ func (t *Named) instantiateMethod(i int) *Func {
// SetUnderlying sets the underlying type and marks t as complete.
// t must not have type arguments.
func (t *Named) SetUnderlying(underlying Type) {
- assert(t.targs.Len() == 0)
+ assert(t.inst == nil)
if underlying == nil {
panic("underlying type must not be nil")
}
if _, ok := underlying.(*Named); ok {
panic("underlying type must not be *Named")
}
- t.resolve(nil).underlying = underlying
+ t.resolve().underlying = underlying
if t.fromRHS == nil {
t.fromRHS = underlying // for cycle detection
}
@@ -216,19 +446,21 @@ func (t *Named) SetUnderlying(underlying Type) {
// AddMethod adds method m unless it is already in the method list.
// t must not have type arguments.
func (t *Named) AddMethod(m *Func) {
- assert(t.targs.Len() == 0)
- t.resolve(nil)
- if t.methods == nil {
- t.methods = newMethodList(nil)
+ assert(t.inst == nil)
+ t.resolve()
+ if i, _ := lookupMethod(t.methods, m.pkg, m.name, false); i < 0 {
+ t.methods = append(t.methods, m)
}
- t.methods.Add(m)
}
-func (t *Named) Underlying() Type { return t.resolve(nil).underlying }
+func (t *Named) Underlying() Type { return t.resolve().underlying }
func (t *Named) String() string { return TypeString(t, nil) }
// ----------------------------------------------------------------------------
// Implementation
+//
+// TODO(rfindley): reorganize the loading and expansion methods under this
+// heading.
// under returns the expanded underlying type of n0; possibly by following
// forward chains of named types. If an underlying type is found, resolve
@@ -277,7 +509,7 @@ func (n0 *Named) under() Type {
check := n0.check
n := n0
- seen := make(map[*Named]int) // types that need their underlying resolved
+ seen := make(map[*Named]int) // types that need their underlying type resolved
var path []Object // objects encountered, for cycle reporting
loop:
@@ -325,11 +557,11 @@ func (n *Named) setUnderlying(typ Type) {
}
func (n *Named) lookupMethod(pkg *Package, name string, foldCase bool) (int, *Func) {
- n.resolve(nil)
+ n.resolve()
// If n is an instance, we may not have yet instantiated all of its methods.
// Look up the method index in orig, and only instantiate method at the
// matching index (if any).
- i, _ := n.orig.methods.Lookup(pkg, name, foldCase)
+ i, _ := lookupMethod(n.Origin().methods, pkg, name, foldCase)
if i < 0 {
return -1, nil
}
@@ -337,83 +569,83 @@ func (n *Named) lookupMethod(pkg *Package, name string, foldCase bool) (int, *Fu
return i, n.Method(i)
}
-// bestContext returns the best available context. In order of preference:
-// - the given ctxt, if non-nil
-// - check.ctxt, if check is non-nil
-// - a new Context
-func (check *Checker) bestContext(ctxt *Context) *Context {
- if ctxt != nil {
- return ctxt
+// context returns the type-checker context.
+func (check *Checker) context() *Context {
+ if check.ctxt == nil {
+ check.ctxt = NewContext()
}
- if check != nil {
- if check.ctxt == nil {
- check.ctxt = NewContext()
- }
- return check.ctxt
- }
- return NewContext()
+ return check.ctxt
}
-// expandNamed ensures that the underlying type of n is instantiated.
-// The underlying type will be Typ[Invalid] if there was an error.
-func expandNamed(ctxt *Context, n *Named, instPos token.Pos) (tparams *TypeParamList, underlying Type, methods *methodList) {
+// expandUnderlying substitutes type arguments in the underlying type n.orig,
+// returning the result. Returns Typ[Invalid] if there was an error.
+func (n *Named) expandUnderlying() Type {
check := n.check
if check != nil && trace {
- check.trace(instPos, "-- expandNamed %s", n)
+ check.trace(n.obj.pos, "-- Named.expandUnderlying %s", n)
check.indent++
defer func() {
check.indent--
- check.trace(instPos, "=> %s (tparams = %s, under = %s)", n, tparams.list(), underlying)
+ check.trace(n.obj.pos, "=> %s (tparams = %s, under = %s)", n, n.tparams.list(), n.underlying)
}()
}
- n.orig.resolve(ctxt)
- assert(n.orig.underlying != nil)
+ assert(n.inst.orig.underlying != nil)
+ if n.inst.ctxt == nil {
+ n.inst.ctxt = NewContext()
+ }
- if _, unexpanded := n.orig.underlying.(*Named); unexpanded {
- // We should only get an unexpanded underlying here during type checking
+ orig := n.inst.orig
+ targs := n.inst.targs
+
+ if _, unexpanded := orig.underlying.(*Named); unexpanded {
+ // We should only get a Named underlying type here during type checking
// (for example, in recursive type declarations).
assert(check != nil)
}
- // Mismatching arg and tparam length may be checked elsewhere.
- if n.orig.tparams.Len() == n.targs.Len() {
- // We must always have a context, to avoid infinite recursion.
- ctxt = check.bestContext(ctxt)
- h := ctxt.instanceHash(n.orig, n.targs.list())
- // ensure that an instance is recorded for h to avoid infinite recursion.
- ctxt.update(h, n.orig, n.TypeArgs().list(), n)
-
- smap := makeSubstMap(n.orig.tparams.list(), n.targs.list())
- underlying = n.check.subst(instPos, n.orig.underlying, smap, ctxt)
- // If the underlying of n is an interface, we need to set the receiver of
- // its methods accurately -- we set the receiver of interface methods on
- // the RHS of a type declaration to the defined type.
- if iface, _ := underlying.(*Interface); iface != nil {
- if methods, copied := replaceRecvType(iface.methods, n.orig, n); copied {
- // If the underlying doesn't actually use type parameters, it's possible
- // that it wasn't substituted. In this case we need to create a new
- // *Interface before modifying receivers.
- if iface == n.orig.underlying {
- old := iface
- iface = check.newInterface()
- iface.embeddeds = old.embeddeds
- iface.complete = old.complete
- iface.implicit = old.implicit // should be false but be conservative
- underlying = iface
- }
- iface.methods = methods
- }
- }
- } else {
- underlying = Typ[Invalid]
+ if orig.tparams.Len() != targs.Len() {
+ // Mismatching arg and tparam length may be checked elsewhere.
+ return Typ[Invalid]
}
- return n.orig.tparams, underlying, newLazyMethodList(n.orig.methods.Len())
+ // Ensure that an instance is recorded before substituting, so that we
+ // resolve n for any recursive references.
+ h := n.inst.ctxt.instanceHash(orig, targs.list())
+ n2 := n.inst.ctxt.update(h, orig, n.TypeArgs().list(), n)
+ assert(n == n2)
+
+ smap := makeSubstMap(orig.tparams.list(), targs.list())
+ var ctxt *Context
+ if check != nil {
+ ctxt = check.context()
+ }
+ underlying := n.check.subst(n.obj.pos, orig.underlying, smap, n, ctxt)
+ // If the underlying type of n is an interface, we need to set the receiver of
+ // its methods accurately -- we set the receiver of interface methods on
+ // the RHS of a type declaration to the defined type.
+ if iface, _ := underlying.(*Interface); iface != nil {
+ if methods, copied := replaceRecvType(iface.methods, orig, n); copied {
+ // If the underlying type doesn't actually use type parameters, it's
+ // possible that it wasn't substituted. In this case we need to create
+ // a new *Interface before modifying receivers.
+ if iface == orig.underlying {
+ old := iface
+ iface = check.newInterface()
+ iface.embeddeds = old.embeddeds
+ iface.complete = old.complete
+ iface.implicit = old.implicit // should be false but be conservative
+ underlying = iface
+ }
+ iface.methods = methods
+ }
+ }
+
+ return underlying
}
-// safeUnderlying returns the underlying of typ without expanding instances, to
-// avoid infinite recursion.
+// safeUnderlying returns the underlying type of typ without expanding
+// instances, to avoid infinite recursion.
//
// TODO(rfindley): eliminate this function or give it a better name.
func safeUnderlying(typ Type) Type {
diff --git a/src/go/types/named_test.go b/src/go/types/named_test.go
new file mode 100644
index 0000000000..0fe17418f4
--- /dev/null
+++ b/src/go/types/named_test.go
@@ -0,0 +1,136 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package types_test
+
+import (
+ "go/ast"
+ "go/parser"
+ "go/token"
+ "testing"
+
+ . "go/types"
+)
+
+func BenchmarkNamed(b *testing.B) {
+ const src = `
+package p
+
+type T struct {
+ P int
+}
+
+func (T) M(int) {}
+func (T) N() (i int) { return }
+
+type G[P any] struct {
+ F P
+}
+
+func (G[P]) M(P) {}
+func (G[P]) N() (p P) { return }
+
+type Inst = G[int]
+ `
+ pkg, err := pkgForMode("p", src, nil, 0)
+ if err != nil {
+ b.Fatal(err)
+ }
+
+ var (
+ T = pkg.Scope().Lookup("T").Type()
+ G = pkg.Scope().Lookup("G").Type()
+ SrcInst = pkg.Scope().Lookup("Inst").Type()
+ UserInst = mustInstantiate(b, G, Typ[Int])
+ )
+
+ tests := []struct {
+ name string
+ typ Type
+ }{
+ {"nongeneric", T},
+ {"generic", G},
+ {"src instance", SrcInst},
+ {"user instance", UserInst},
+ }
+
+ b.Run("Underlying", func(b *testing.B) {
+ for _, test := range tests {
+ b.Run(test.name, func(b *testing.B) {
+ // Access underlying once, to trigger any lazy calculation.
+ _ = test.typ.Underlying()
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ _ = test.typ.Underlying()
+ }
+ })
+ }
+ })
+
+ b.Run("NewMethodSet", func(b *testing.B) {
+ for _, test := range tests {
+ b.Run(test.name, func(b *testing.B) {
+ // Access underlying once, to trigger any lazy calculation.
+ _ = NewMethodSet(test.typ)
+ b.ResetTimer()
+ for i := 0; i < b.N; i++ {
+ _ = NewMethodSet(test.typ)
+ }
+ })
+ }
+ })
+}
+
+func mustInstantiate(tb testing.TB, orig Type, targs ...Type) Type {
+ inst, err := Instantiate(nil, orig, targs, true)
+ if err != nil {
+ tb.Fatal(err)
+ }
+ return inst
+}
+
+// Test that types do not expand infinitely, as in golang/go#52715.
+func TestFiniteTypeExpansion(t *testing.T) {
+ const src = `
+package p
+
+type Tree[T any] struct {
+ *Node[T]
+}
+
+func (*Tree[R]) N(r R) R { return r }
+
+type Node[T any] struct {
+ *Tree[T]
+}
+
+func (Node[Q]) M(Q) {}
+
+type Inst = *Tree[int]
+`
+
+ fset := token.NewFileSet()
+ f, err := parser.ParseFile(fset, "foo.go", src, 0)
+ if err != nil {
+ t.Fatal(err)
+ }
+ pkg := NewPackage("p", f.Name.Name)
+ if err := NewChecker(nil, fset, pkg, nil).Files([]*ast.File{f}); err != nil {
+ t.Fatal(err)
+ }
+
+ firstFieldType := func(n *Named) *Named {
+ return n.Underlying().(*Struct).Field(0).Type().(*Pointer).Elem().(*Named)
+ }
+
+ Inst := pkg.Scope().Lookup("Inst").Type().(*Pointer).Elem().(*Named)
+ Node := firstFieldType(Inst)
+ Tree := firstFieldType(Node)
+ if !Identical(Inst, Tree) {
+ t.Fatalf("Not a cycle: got %v, want %v", Tree, Inst)
+ }
+ if Inst != Tree {
+ t.Errorf("Duplicate instances in cycle: %s (%p) -> %s (%p) -> %s (%p)", Inst, Inst, Node, Node, Tree, Tree)
+ }
+}
diff --git a/src/go/types/object.go b/src/go/types/object.go
index ae138a5879..f203b0113d 100644
--- a/src/go/types/object.go
+++ b/src/go/types/object.go
@@ -233,19 +233,7 @@ func NewTypeName(pos token.Pos, pkg *Package, name string, typ Type) *TypeName {
// lazily calls resolve to finish constructing the Named object.
func _NewTypeNameLazy(pos token.Pos, pkg *Package, name string, load func(named *Named) (tparams []*TypeParam, underlying Type, methods []*Func)) *TypeName {
obj := NewTypeName(pos, pkg, name, nil)
-
- resolve := func(_ *Context, t *Named) (*TypeParamList, Type, *methodList) {
- tparams, underlying, methods := load(t)
-
- switch underlying.(type) {
- case nil, *Named:
- panic(fmt.Sprintf("invalid underlying type %T", t.underlying))
- }
-
- return bindTParams(tparams), underlying, newMethodList(methods)
- }
-
- NewNamed(obj, nil, nil).resolver = resolve
+ NewNamed(obj, nil, nil).loader = load
return obj
}
diff --git a/src/go/types/predicates.go b/src/go/types/predicates.go
index 51d056f355..25db4acf4a 100644
--- a/src/go/types/predicates.go
+++ b/src/go/types/predicates.go
@@ -104,7 +104,7 @@ func isTypeParam(t Type) bool {
func isGeneric(t Type) bool {
// A parameterized type is only generic if it doesn't have an instantiation already.
named, _ := t.(*Named)
- return named != nil && named.obj != nil && named.targs == nil && named.TypeParams() != nil
+ return named != nil && named.obj != nil && named.inst == nil && named.TypeParams().Len() > 0
}
// Comparable reports whether values of type T are comparable.
@@ -285,18 +285,19 @@ func identical(x, y Type, cmpTags bool, p *ifacePair) bool {
}
smap := makeSubstMap(ytparams, targs)
- var check *Checker // ok to call subst on a nil *Checker
+ var check *Checker // ok to call subst on a nil *Checker
+ ctxt := NewContext() // need a non-nil Context for the substitution below
// Constraints must be pair-wise identical, after substitution.
for i, xtparam := range xtparams {
- ybound := check.subst(token.NoPos, ytparams[i].bound, smap, nil)
+ ybound := check.subst(token.NoPos, ytparams[i].bound, smap, nil, ctxt)
if !identical(xtparam.bound, ybound, cmpTags, p) {
return false
}
}
- yparams = check.subst(token.NoPos, y.params, smap, nil).(*Tuple)
- yresults = check.subst(token.NoPos, y.results, smap, nil).(*Tuple)
+ yparams = check.subst(token.NoPos, y.params, smap, nil, ctxt).(*Tuple)
+ yresults = check.subst(token.NoPos, y.results, smap, nil, ctxt).(*Tuple)
}
return x.variadic == y.variadic &&
@@ -403,7 +404,7 @@ func identical(x, y Type, cmpTags bool, p *ifacePair) bool {
if len(xargs) > 0 {
// Instances are identical if their original type and type arguments
// are identical.
- if !Identical(x.orig, y.orig) {
+ if !Identical(x.Origin(), y.Origin()) {
return false
}
for i, xa := range xargs {
diff --git a/src/go/types/signature.go b/src/go/types/signature.go
index 4b63f0e6f0..82177a1c58 100644
--- a/src/go/types/signature.go
+++ b/src/go/types/signature.go
@@ -150,7 +150,7 @@ func (check *Checker) funcType(sig *Signature, recvPar *ast.FieldList, ftyp *ast
// recvTPar.bound is (possibly) parameterized in the context of the
// receiver type declaration. Substitute parameters for the current
// context.
- tpar.bound = check.subst(tpar.obj.pos, recvTPar.bound, smap, nil)
+ tpar.bound = check.subst(tpar.obj.pos, recvTPar.bound, smap, nil, check.context())
}
} else if len(tparams) < len(recvTParams) {
// Reporting an error here is a stop-gap measure to avoid crashes in the
diff --git a/src/go/types/sizeof_test.go b/src/go/types/sizeof_test.go
index 3428eb9191..d4ce0a7fcd 100644
--- a/src/go/types/sizeof_test.go
+++ b/src/go/types/sizeof_test.go
@@ -30,7 +30,7 @@ func TestSizeof(t *testing.T) {
{Interface{}, 40, 80},
{Map{}, 16, 32},
{Chan{}, 12, 24},
- {Named{}, 56, 104},
+ {Named{}, 60, 112},
{TypeParam{}, 28, 48},
{term{}, 12, 24},
diff --git a/src/go/types/subst.go b/src/go/types/subst.go
index 63849b9212..42f3619f88 100644
--- a/src/go/types/subst.go
+++ b/src/go/types/subst.go
@@ -48,8 +48,11 @@ func (m substMap) lookup(tpar *TypeParam) Type {
// that it doesn't modify the incoming type. If a substitution took place, the
// result type is different from the incoming type.
//
-// If the given context is non-nil, it is used in lieu of check.Config.Context
-func (check *Checker) subst(pos token.Pos, typ Type, smap substMap, ctxt *Context) Type {
+// If expanding is non-nil, it is the instance type currently being expanded.
+// One of expanding or ctxt must be non-nil.
+func (check *Checker) subst(pos token.Pos, typ Type, smap substMap, expanding *Named, ctxt *Context) Type {
+ assert(expanding != nil || ctxt != nil)
+
if smap.empty() {
return typ
}
@@ -64,19 +67,21 @@ func (check *Checker) subst(pos token.Pos, typ Type, smap substMap, ctxt *Contex
// general case
subst := subster{
- pos: pos,
- smap: smap,
- check: check,
- ctxt: check.bestContext(ctxt),
+ pos: pos,
+ smap: smap,
+ check: check,
+ expanding: expanding,
+ ctxt: ctxt,
}
return subst.typ(typ)
}
type subster struct {
- pos token.Pos
- smap substMap
- check *Checker // nil if called via Instantiate
- ctxt *Context
+ pos token.Pos
+ smap substMap
+ check *Checker // nil if called via Instantiate
+ expanding *Named // if non-nil, the instance that is being expanded
+ ctxt *Context
}
func (subst *subster) typ(typ Type) Type {
@@ -176,7 +181,7 @@ func (subst *subster) typ(typ Type) Type {
// In this case the interface will not be substituted here, because its
// method signatures do not depend on the type parameter P, but we still
// need to create new interface methods to hold the instantiated
- // receiver. This is handled by expandNamed.
+ // receiver. This is handled by Named.expandUnderlying.
iface.methods, _ = replaceRecvType(methods, t, iface)
return iface
}
@@ -207,19 +212,20 @@ func (subst *subster) typ(typ Type) Type {
}
}
- // subst is called by expandNamed, so in this function we need to be
+ // subst is called during expansion, so in this function we need to be
// careful not to call any methods that would cause t to be expanded: doing
// so would result in deadlock.
//
- // So we call t.orig.TypeParams() rather than t.TypeParams() here and
- // below.
- if t.orig.TypeParams().Len() == 0 {
+ // So we call t.Origin().TypeParams() rather than t.TypeParams().
+ orig := t.Origin()
+ n := orig.TypeParams().Len()
+ if n == 0 {
dump(">>> %s is not parameterized", t)
return t // type is not parameterized
}
var newTArgs []Type
- if t.targs.Len() != t.orig.TypeParams().Len() {
+ if t.TypeArgs().Len() != n {
return Typ[Invalid] // error reported elsewhere
}
@@ -228,14 +234,14 @@ func (subst *subster) typ(typ Type) Type {
// For each (existing) type argument targ, determine if it needs
// to be substituted; i.e., if it is or contains a type parameter
// that has a type argument for it.
- for i, targ := range t.targs.list() {
+ for i, targ := range t.TypeArgs().list() {
dump(">>> %d targ = %s", i, targ)
new_targ := subst.typ(targ)
if new_targ != targ {
dump(">>> substituted %d targ %s => %s", i, targ, new_targ)
if newTArgs == nil {
- newTArgs = make([]Type, t.orig.TypeParams().Len())
- copy(newTArgs, t.targs.list())
+ newTArgs = make([]Type, n)
+ copy(newTArgs, t.TypeArgs().list())
}
newTArgs[i] = new_targ
}
@@ -246,25 +252,11 @@ func (subst *subster) typ(typ Type) Type {
return t // nothing to substitute
}
- // before creating a new named type, check if we have this one already
- h := subst.ctxt.instanceHash(t.orig, newTArgs)
- dump(">>> new type hash: %s", h)
- if named := subst.ctxt.lookup(h, t.orig, newTArgs); named != nil {
- dump(">>> found %s", named)
- return named
- }
-
// Create a new instance and populate the context to avoid endless
// recursion. The position used here is irrelevant because validation only
// occurs on t (we don't call validType on named), but we use subst.pos to
// help with debugging.
- return subst.check.instance(subst.pos, t.orig, newTArgs, subst.ctxt)
-
- // Note that if we were to expose substitution more generally (not just in
- // the context of a declaration), we'd have to substitute in
- // named.underlying as well.
- //
- // But this is unnecessary for now.
+ return subst.check.instance(subst.pos, orig, newTArgs, subst.expanding, subst.ctxt)
case *TypeParam:
return subst.smap.lookup(t)
diff --git a/src/go/types/termlist.go b/src/go/types/termlist.go
index 94e49caee0..6d08ddb397 100644
--- a/src/go/types/termlist.go
+++ b/src/go/types/termlist.go
@@ -25,7 +25,7 @@ func (xl termlist) String() string {
var buf bytes.Buffer
for i, x := range xl {
if i > 0 {
- buf.WriteString(" ∪ ")
+ buf.WriteString(" | ")
}
buf.WriteString(x.String())
}
diff --git a/src/go/types/termlist_test.go b/src/go/types/termlist_test.go
index f0d58ac1bc..0ff687ebda 100644
--- a/src/go/types/termlist_test.go
+++ b/src/go/types/termlist_test.go
@@ -12,7 +12,7 @@ import (
// maketl makes a term list from a string of the term list.
func maketl(s string) termlist {
s = strings.ReplaceAll(s, " ", "")
- names := strings.Split(s, "∪")
+ names := strings.Split(s, "|")
r := make(termlist, len(names))
for i, n := range names {
r[i] = testTerm(n)
@@ -33,10 +33,10 @@ func TestTermlistString(t *testing.T) {
"int",
"~int",
"myInt",
- "∅ ∪ ∅",
- "𝓤 ∪ 𝓤",
- "∅ ∪ 𝓤 ∪ int",
- "∅ ∪ 𝓤 ∪ int ∪ myInt",
+ "∅ | ∅",
+ "𝓤 | 𝓤",
+ "∅ | 𝓤 | int",
+ "∅ | 𝓤 | int | myInt",
} {
if got := maketl(want).String(); got != want {
t.Errorf("(%v).String() == %v", want, got)
@@ -47,12 +47,12 @@ func TestTermlistString(t *testing.T) {
func TestTermlistIsEmpty(t *testing.T) {
for test, want := range map[string]bool{
"∅": true,
- "∅ ∪ ∅": true,
- "∅ ∪ ∅ ∪ 𝓤": false,
- "∅ ∪ ∅ ∪ myInt": false,
+ "∅ | ∅": true,
+ "∅ | ∅ | 𝓤": false,
+ "∅ | ∅ | myInt": false,
"𝓤": false,
- "𝓤 ∪ int": false,
- "𝓤 ∪ myInt ∪ ∅": false,
+ "𝓤 | int": false,
+ "𝓤 | myInt | ∅": false,
} {
xl := maketl(test)
got := xl.isEmpty()
@@ -65,13 +65,13 @@ func TestTermlistIsEmpty(t *testing.T) {
func TestTermlistIsAll(t *testing.T) {
for test, want := range map[string]bool{
"∅": false,
- "∅ ∪ ∅": false,
- "int ∪ ~string": false,
- "~int ∪ myInt": false,
- "∅ ∪ ∅ ∪ 𝓤": true,
+ "∅ | ∅": false,
+ "int | ~string": false,
+ "~int | myInt": false,
+ "∅ | ∅ | 𝓤": true,
"𝓤": true,
- "𝓤 ∪ int": true,
- "myInt ∪ 𝓤": true,
+ "𝓤 | int": true,
+ "myInt | 𝓤": true,
} {
xl := maketl(test)
got := xl.isAll()
@@ -86,17 +86,17 @@ func TestTermlistNorm(t *testing.T) {
xl, want string
}{
{"∅", "∅"},
- {"∅ ∪ ∅", "∅"},
- {"∅ ∪ int", "int"},
- {"∅ ∪ myInt", "myInt"},
- {"𝓤 ∪ int", "𝓤"},
- {"𝓤 ∪ myInt", "𝓤"},
- {"int ∪ myInt", "int ∪ myInt"},
- {"~int ∪ int", "~int"},
- {"~int ∪ myInt", "~int"},
- {"int ∪ ~string ∪ int", "int ∪ ~string"},
- {"~int ∪ string ∪ 𝓤 ∪ ~string ∪ int", "𝓤"},
- {"~int ∪ string ∪ myInt ∪ ~string ∪ int", "~int ∪ ~string"},
+ {"∅ | ∅", "∅"},
+ {"∅ | int", "int"},
+ {"∅ | myInt", "myInt"},
+ {"𝓤 | int", "𝓤"},
+ {"𝓤 | myInt", "𝓤"},
+ {"int | myInt", "int | myInt"},
+ {"~int | int", "~int"},
+ {"~int | myInt", "~int"},
+ {"int | ~string | int", "int | ~string"},
+ {"~int | string | 𝓤 | ~string | int", "𝓤"},
+ {"~int | string | myInt | ~string | int", "~int | ~string"},
} {
xl := maketl(test.xl)
got := maketl(test.xl).norm()
@@ -116,15 +116,15 @@ func TestTermlistUnion(t *testing.T) {
{"∅", "int", "int"},
{"𝓤", "~int", "𝓤"},
{"int", "~int", "~int"},
- {"int", "string", "int ∪ string"},
- {"int", "myInt", "int ∪ myInt"},
+ {"int", "string", "int | string"},
+ {"int", "myInt", "int | myInt"},
{"~int", "myInt", "~int"},
- {"int ∪ string", "~string", "int ∪ ~string"},
- {"~int ∪ string", "~string ∪ int", "~int ∪ ~string"},
- {"~int ∪ string ∪ ∅", "~string ∪ int", "~int ∪ ~string"},
- {"~int ∪ myInt ∪ ∅", "~string ∪ int", "~int ∪ ~string"},
- {"~int ∪ string ∪ 𝓤", "~string ∪ int", "𝓤"},
- {"~int ∪ string ∪ myInt", "~string ∪ int", "~int ∪ ~string"},
+ {"int | string", "~string", "int | ~string"},
+ {"~int | string", "~string | int", "~int | ~string"},
+ {"~int | string | ∅", "~string | int", "~int | ~string"},
+ {"~int | myInt | ∅", "~string | int", "~int | ~string"},
+ {"~int | string | 𝓤", "~string | int", "𝓤"},
+ {"~int | string | myInt", "~string | int", "~int | ~string"},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
@@ -150,12 +150,12 @@ func TestTermlistIntersect(t *testing.T) {
{"int", "string", "∅"},
{"int", "myInt", "∅"},
{"~int", "myInt", "myInt"},
- {"int ∪ string", "~string", "string"},
- {"~int ∪ string", "~string ∪ int", "int ∪ string"},
- {"~int ∪ string ∪ ∅", "~string ∪ int", "int ∪ string"},
- {"~int ∪ myInt ∪ ∅", "~string ∪ int", "int"},
- {"~int ∪ string ∪ 𝓤", "~string ∪ int", "int ∪ ~string"},
- {"~int ∪ string ∪ myInt", "~string ∪ int", "int ∪ string"},
+ {"int | string", "~string", "string"},
+ {"~int | string", "~string | int", "int | string"},
+ {"~int | string | ∅", "~string | int", "int | string"},
+ {"~int | myInt | ∅", "~string | int", "int"},
+ {"~int | string | 𝓤", "~string | int", "int | ~string"},
+ {"~int | string | myInt", "~string | int", "int | string"},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
@@ -174,12 +174,12 @@ func TestTermlistEqual(t *testing.T) {
{"∅", "∅", true},
{"∅", "𝓤", false},
{"𝓤", "𝓤", true},
- {"𝓤 ∪ int", "𝓤", true},
- {"𝓤 ∪ int", "string ∪ 𝓤", true},
- {"𝓤 ∪ myInt", "string ∪ 𝓤", true},
- {"int ∪ ~string", "string ∪ int", false},
- {"~int ∪ string", "string ∪ myInt", false},
- {"int ∪ ~string ∪ ∅", "string ∪ int ∪ ~string", true},
+ {"𝓤 | int", "𝓤", true},
+ {"𝓤 | int", "string | 𝓤", true},
+ {"𝓤 | myInt", "string | 𝓤", true},
+ {"int | ~string", "string | int", false},
+ {"~int | string", "string | myInt", false},
+ {"int | ~string | ∅", "string | int | ~string", true},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
@@ -201,11 +201,11 @@ func TestTermlistIncludes(t *testing.T) {
{"int", "string", false},
{"~int", "string", false},
{"~int", "myInt", true},
- {"int ∪ string", "string", true},
- {"~int ∪ string", "int", true},
- {"~int ∪ string", "myInt", true},
- {"~int ∪ myInt ∪ ∅", "myInt", true},
- {"myInt ∪ ∅ ∪ 𝓤", "int", true},
+ {"int | string", "string", true},
+ {"~int | string", "int", true},
+ {"~int | string", "myInt", true},
+ {"~int | myInt | ∅", "myInt", true},
+ {"myInt | ∅ | 𝓤", "int", true},
} {
xl := maketl(test.xl)
yl := testTerm(test.typ).typ
@@ -236,12 +236,12 @@ func TestTermlistSupersetOf(t *testing.T) {
{"myInt", "~int", false},
{"int", "string", false},
{"~int", "string", false},
- {"int ∪ string", "string", true},
- {"int ∪ string", "~string", false},
- {"~int ∪ string", "int", true},
- {"~int ∪ string", "myInt", true},
- {"~int ∪ string ∪ ∅", "string", true},
- {"~string ∪ ∅ ∪ 𝓤", "myInt", true},
+ {"int | string", "string", true},
+ {"int | string", "~string", false},
+ {"~int | string", "int", true},
+ {"~int | string", "myInt", true},
+ {"~int | string | ∅", "string", true},
+ {"~string | ∅ | 𝓤", "myInt", true},
} {
xl := maketl(test.xl)
y := testTerm(test.typ)
@@ -261,18 +261,18 @@ func TestTermlistSubsetOf(t *testing.T) {
{"∅", "𝓤", true},
{"𝓤", "∅", false},
{"𝓤", "𝓤", true},
- {"int", "int ∪ string", true},
- {"~int", "int ∪ string", false},
- {"~int", "myInt ∪ string", false},
- {"myInt", "~int ∪ string", true},
- {"~int", "string ∪ string ∪ int ∪ ~int", true},
- {"myInt", "string ∪ string ∪ ~int", true},
- {"int ∪ string", "string", false},
- {"int ∪ string", "string ∪ int", true},
- {"int ∪ ~string", "string ∪ int", false},
- {"myInt ∪ ~string", "string ∪ int ∪ 𝓤", true},
- {"int ∪ ~string", "string ∪ int ∪ ∅ ∪ string", false},
- {"int ∪ myInt", "string ∪ ~int ∪ ∅ ∪ string", true},
+ {"int", "int | string", true},
+ {"~int", "int | string", false},
+ {"~int", "myInt | string", false},
+ {"myInt", "~int | string", true},
+ {"~int", "string | string | int | ~int", true},
+ {"myInt", "string | string | ~int", true},
+ {"int | string", "string", false},
+ {"int | string", "string | int", true},
+ {"int | ~string", "string | int", false},
+ {"myInt | ~string", "string | int | 𝓤", true},
+ {"int | ~string", "string | int | ∅ | string", false},
+ {"int | myInt", "string | ~int | ∅ | string", true},
} {
xl := maketl(test.xl)
yl := maketl(test.yl)
diff --git a/src/go/types/testdata/fixedbugs/issue40350.go b/src/go/types/testdata/fixedbugs/issue40350.go
new file mode 100644
index 0000000000..7ffd551c2e
--- /dev/null
+++ b/src/go/types/testdata/fixedbugs/issue40350.go
@@ -0,0 +1,16 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+type number interface {
+ ~float64 | ~int | ~int32
+ float64 | ~int32
+}
+
+func f[T number]() {}
+
+func _() {
+ _ = f[int /* ERROR int does not implement number \(int missing in float64 | ~int32\)*/]
+}
diff --git a/src/go/types/testdata/fixedbugs/issue42881.go b/src/go/types/testdata/fixedbugs/issue42881.go
new file mode 100644
index 0000000000..7122d1c787
--- /dev/null
+++ b/src/go/types/testdata/fixedbugs/issue42881.go
@@ -0,0 +1,16 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+type (
+ T1 interface{ comparable }
+ T2 interface{ int }
+)
+
+var (
+ _ comparable // ERROR cannot use type comparable outside a type constraint: interface is \(or embeds\) comparable
+ _ T1 // ERROR cannot use type T1 outside a type constraint: interface is \(or embeds\) comparable
+ _ T2 // ERROR cannot use type T2 outside a type constraint: interface contains type constraints
+)
diff --git a/src/go/types/testdata/fixedbugs/issue50729.go b/src/go/types/testdata/fixedbugs/issue50729.go
new file mode 100644
index 0000000000..fe19fdfa68
--- /dev/null
+++ b/src/go/types/testdata/fixedbugs/issue50729.go
@@ -0,0 +1,19 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+// version 1
+var x1 T1[B1]
+
+type T1[_ any] struct{}
+type A1 T1[B1]
+type B1 = T1[A1]
+
+// version 2
+type T2[_ any] struct{}
+type A2 T2[B2]
+type B2 = T2[A2]
+
+var x2 T2[B2]
diff --git a/src/go/types/testdata/fixedbugs/issue52698.go b/src/go/types/testdata/fixedbugs/issue52698.go
new file mode 100644
index 0000000000..3babc21d92
--- /dev/null
+++ b/src/go/types/testdata/fixedbugs/issue52698.go
@@ -0,0 +1,50 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package p
+
+// sanity check
+type T[P any] struct {
+ _ P
+}
+
+type S /* ERROR illegal cycle */ struct {
+ _ T[S]
+}
+
+// simplified test
+var _ B[A]
+
+type A struct {
+ _ B[string]
+}
+
+type B[P any] struct {
+ _ C[P]
+}
+
+type C[P any] struct {
+ _ P
+}
+
+// test case from issue
+type T23 interface {
+ ~struct {
+ Field0 T13[T15]
+ }
+}
+
+type T1[P1 interface {
+}] struct {
+ Field2 P1
+}
+
+type T13[P2 interface {
+}] struct {
+ Field2 T1[P2]
+}
+
+type T15 struct {
+ Field0 T13[string]
+}
diff --git a/src/go/types/typeset_test.go b/src/go/types/typeset_test.go
index 2bbe611376..5156092483 100644
--- a/src/go/types/typeset_test.go
+++ b/src/go/types/typeset_test.go
@@ -22,13 +22,13 @@ func TestTypeSetString(t *testing.T) {
"{}": "𝓤",
"{int}": "{int}",
"{~int}": "{~int}",
- "{int|string}": "{int ∪ string}",
+ "{int|string}": "{int | string}",
"{int; string}": "∅",
"{comparable}": "{comparable}",
"{comparable; int}": "{int}",
"{~int; comparable}": "{~int}",
- "{int|string; comparable}": "{int ∪ string}",
+ "{int|string; comparable}": "{int | string}",
"{comparable; int; string}": "∅",
"{m()}": "{func (p.T).m()}",
@@ -38,7 +38,7 @@ func TestTypeSetString(t *testing.T) {
"{m1(); comparable; m2() int }": "{comparable; func (p.T).m1(); func (p.T).m2() int}",
"{comparable; error}": "{comparable; func (error).Error() string}",
- "{m(); comparable; int|float32|string}": "{func (p.T).m(); int ∪ float32 ∪ string}",
+ "{m(); comparable; int|float32|string}": "{func (p.T).m(); int | float32 | string}",
"{m1(); int; m2(); comparable }": "{func (p.T).m1(); func (p.T).m2(); int}",
"{E}; type E interface{}": "𝓤",
diff --git a/src/go/types/typestring.go b/src/go/types/typestring.go
index 0325d4a77f..5a2e2c171a 100644
--- a/src/go/types/typestring.go
+++ b/src/go/types/typestring.go
@@ -285,9 +285,9 @@ func (w *typeWriter) typ(typ Type) {
w.string(strconv.Itoa(w.ctxt.getID(t)))
}
w.typeName(t.obj) // when hashing written for readability of the hash only
- if t.targs != nil {
+ if t.inst != nil {
// instantiated type
- w.typeList(t.targs.list())
+ w.typeList(t.inst.targs.list())
} else if w.ctxt == nil && t.TypeParams().Len() != 0 { // For type hashing, don't need to format the TypeParams
// parameterized type
w.tParamList(t.TypeParams().list())
diff --git a/src/go/types/typexpr.go b/src/go/types/typexpr.go
index 7afc66a925..b02929df22 100644
--- a/src/go/types/typexpr.go
+++ b/src/go/types/typexpr.go
@@ -163,9 +163,9 @@ func (check *Checker) validVarType(e ast.Expr, typ Type) {
tset := computeInterfaceTypeSet(check, e.Pos(), t) // TODO(gri) is this the correct position?
if !tset.IsMethodSet() {
if tset.comparable {
- check.softErrorf(e, _MisplacedConstraintIface, "interface is (or embeds) comparable")
+ check.softErrorf(e, _MisplacedConstraintIface, "cannot use type %s outside a type constraint: interface is (or embeds) comparable", typ)
} else {
- check.softErrorf(e, _MisplacedConstraintIface, "interface contains type constraints")
+ check.softErrorf(e, _MisplacedConstraintIface, "cannot use type %s outside a type constraint: interface contains type constraints", typ)
}
}
}
@@ -385,14 +385,13 @@ func (check *Checker) typInternal(e0 ast.Expr, def *Named) (T Type) {
}
func (check *Checker) instantiatedType(ix *typeparams.IndexExpr, def *Named) (res Type) {
- pos := ix.X.Pos()
if trace {
- check.trace(pos, "-- instantiating type %s with %s", ix.X, ix.Indices)
+ check.trace(ix.Pos(), "-- instantiating type %s with %s", ix.X, ix.Indices)
check.indent++
defer func() {
check.indent--
// Don't format the underlying here. It will always be nil.
- check.trace(pos, "=> %s", res)
+ check.trace(ix.Pos(), "=> %s", res)
}()
}
@@ -417,57 +416,19 @@ func (check *Checker) instantiatedType(ix *typeparams.IndexExpr, def *Named) (re
return Typ[Invalid]
}
- // enableTypeTypeInference controls whether to infer missing type arguments
- // using constraint type inference. See issue #51527.
- const enableTypeTypeInference = false
-
// create the instance
- ctxt := check.bestContext(nil)
- h := ctxt.instanceHash(orig, targs)
- // targs may be incomplete, and require inference. In any case we should de-duplicate.
- inst, _ := ctxt.lookup(h, orig, targs).(*Named)
- // If inst is non-nil, we can't just return here. Inst may have been
- // constructed via recursive substitution, in which case we wouldn't do the
- // validation below. Ensure that the validation (and resulting errors) runs
- // for each instantiated type in the source.
- if inst == nil {
- // x may be a selector for an imported type; use its start pos rather than x.Pos().
- tname := NewTypeName(ix.Pos(), orig.obj.pkg, orig.obj.name, nil)
- inst = check.newNamed(tname, orig, nil, nil) // underlying, methods and tparams are set when named is resolved
- inst.targs = newTypeList(targs)
- inst = ctxt.update(h, orig, targs, inst).(*Named)
- }
+ inst := check.instance(ix.Pos(), orig, targs, nil, check.context()).(*Named)
def.setUnderlying(inst)
- inst.resolver = func(ctxt *Context, n *Named) (*TypeParamList, Type, *methodList) {
- tparams := n.orig.TypeParams().list()
-
- targs := n.targs.list()
- if enableTypeTypeInference && len(targs) < len(tparams) {
- // If inference fails, len(inferred) will be 0, and inst.underlying will
- // be set to Typ[Invalid] in expandNamed.
- inferred := check.infer(ix.Orig, tparams, targs, nil, nil)
- if len(inferred) > len(targs) {
- n.targs = newTypeList(inferred)
- }
- }
-
- return expandNamed(ctxt, n, pos)
- }
-
// orig.tparams may not be set up, so we need to do expansion later.
check.later(func() {
// This is an instance from the source, not from recursive substitution,
// and so it must be resolved during type-checking so that we can report
// errors.
- inst.resolve(ctxt)
- // Since check is non-nil, we can still mutate inst. Unpinning the resolver
- // frees some memory.
- inst.resolver = nil
check.recordInstance(ix.Orig, inst.TypeArgs().list(), inst)
- if check.validateTArgLen(pos, inst.tparams.Len(), inst.targs.Len()) {
- if i, err := check.verify(pos, inst.tparams.list(), inst.targs.list()); err != nil {
+ if check.validateTArgLen(ix.Pos(), inst.TypeParams().Len(), inst.TypeArgs().Len()) {
+ if i, err := check.verify(ix.Pos(), inst.TypeParams().list(), inst.TypeArgs().list(), check.context()); err != nil {
// best position for error reporting
pos := ix.Pos()
if i < len(ix.Indices) {
@@ -475,10 +436,13 @@ func (check *Checker) instantiatedType(ix *typeparams.IndexExpr, def *Named) (re
}
check.softErrorf(atPos(pos), _InvalidTypeArg, err.Error())
} else {
- check.mono.recordInstance(check.pkg, pos, inst.tparams.list(), inst.targs.list(), ix.Indices)
+ check.mono.recordInstance(check.pkg, ix.Pos(), inst.TypeParams().list(), inst.TypeArgs().list(), ix.Indices)
}
}
+ // TODO(rfindley): remove this call: we don't need to call validType here,
+ // as cycles can only occur for types used inside a Named type declaration,
+ // and so it suffices to call validType from declared types.
check.validType(inst)
}).describef(ix, "resolve instance %s", inst)
diff --git a/src/go/types/unify.go b/src/go/types/unify.go
index 0742e40d8b..602e304b4a 100644
--- a/src/go/types/unify.go
+++ b/src/go/types/unify.go
@@ -546,8 +546,8 @@ func (u *unifier) nify(x, y Type, p *ifacePair) (result bool) {
case *Named:
// TODO(gri) This code differs now from the parallel code in Checker.identical. Investigate.
if y, ok := y.(*Named); ok {
- xargs := x.targs.list()
- yargs := y.targs.list()
+ xargs := x.TypeArgs().list()
+ yargs := y.TypeArgs().list()
if len(xargs) != len(yargs) {
return false
diff --git a/src/go/types/validtype.go b/src/go/types/validtype.go
index edb4c02ecd..34c9533a05 100644
--- a/src/go/types/validtype.go
+++ b/src/go/types/validtype.go
@@ -5,29 +5,24 @@
package types
// validType verifies that the given type does not "expand" indefinitely
-// producing a cycle in the type graph. Cycles are detected by marking
-// defined types.
+// producing a cycle in the type graph.
// (Cycles involving alias types, as in "type A = [10]A" are detected
// earlier, via the objDecl cycle detection mechanism.)
func (check *Checker) validType(typ *Named) {
check.validType0(typ, nil, nil)
}
-type typeInfo uint
-
// validType0 checks if the given type is valid. If typ is a type parameter
-// its value is looked up in the provided environment. The environment is
-// nil if typ is not part of (the RHS of) an instantiated type, in that case
-// any type parameter encountered must be from an enclosing function and can
-// be ignored. The path is the list of type names that lead to the current typ.
-func (check *Checker) validType0(typ Type, env *tparamEnv, path []Object) typeInfo {
- const (
- unknown typeInfo = iota
- marked
- valid
- invalid
- )
-
+// its value is looked up in the type argument list of the instantiated
+// (enclosing) type, if it exists. Otherwise the type parameter must be from
+// an enclosing function and can be ignored.
+// The nest list describes the stack (the "nest in memory") of types which
+// contain (or embed in the case of interfaces) other types. For instance, a
+// struct named S which contains a field of named type F contains (the memory
+// of) F in S, leading to the nest S->F. If a type appears in its own nest
+// (say S->F->S) we have an invalid recursive type. The path list is the full
+// path of named types in a cycle, it is only needed for error reporting.
+func (check *Checker) validType0(typ Type, nest, path []*Named) bool {
switch t := typ.(type) {
case nil:
// We should never see a nil type but be conservative and panic
@@ -37,111 +32,204 @@ func (check *Checker) validType0(typ Type, env *tparamEnv, path []Object) typeIn
}
case *Array:
- return check.validType0(t.elem, env, path)
+ return check.validType0(t.elem, nest, path)
case *Struct:
for _, f := range t.fields {
- if check.validType0(f.typ, env, path) == invalid {
- return invalid
+ if !check.validType0(f.typ, nest, path) {
+ return false
}
}
case *Union:
for _, t := range t.terms {
- if check.validType0(t.typ, env, path) == invalid {
- return invalid
+ if !check.validType0(t.typ, nest, path) {
+ return false
}
}
case *Interface:
for _, etyp := range t.embeddeds {
- if check.validType0(etyp, env, path) == invalid {
- return invalid
+ if !check.validType0(etyp, nest, path) {
+ return false
}
}
case *Named:
- // Don't report a 2nd error if we already know the type is invalid
- // Note: ensure that t.orig is fully resolved by calling Underlying().
- if t.Underlying() == Typ[Invalid] {
- check.infoMap[t] = invalid
- return invalid
+ // Exit early if we already know t is valid.
+ // This is purely an optimization but it prevents excessive computation
+ // times in pathological cases such as testdata/fixedbugs/issue6977.go.
+ // (Note: The valids map could also be allocated locally, once for each
+ // validType call.)
+ if check.valids.lookup(t) != nil {
+ break
}
- switch check.infoMap[t] {
- case unknown:
- check.infoMap[t] = marked
- check.infoMap[t] = check.validType0(t.orig.fromRHS, env.push(t), append(path, t.obj))
- case marked:
- // We have seen type t before and thus must have a cycle.
- check.infoMap[t] = invalid
- // t cannot be in an imported package otherwise that package
- // would have reported a type cycle and couldn't have been
- // imported in the first place.
- assert(t.obj.pkg == check.pkg)
- t.underlying = Typ[Invalid] // t is in the current package (no race possibility)
- // Find the starting point of the cycle and report it.
- for i, tn := range path {
- if tn == t.obj {
- check.cycleError(path[i:])
- return invalid
- }
- }
- panic("cycle start not found")
+ // Don't report a 2nd error if we already know the type is invalid
+ // (e.g., if a cycle was detected earlier, via under).
+ // Note: ensure that t.orig is fully resolved by calling Underlying().
+ if t.Underlying() == Typ[Invalid] {
+ return false
}
- return check.infoMap[t]
+
+ // If the current type t is also found in nest, (the memory of) t is
+ // embedded in itself, indicating an invalid recursive type.
+ for _, e := range nest {
+ if Identical(e, t) {
+ // t cannot be in an imported package otherwise that package
+ // would have reported a type cycle and couldn't have been
+ // imported in the first place.
+ assert(t.obj.pkg == check.pkg)
+ t.underlying = Typ[Invalid] // t is in the current package (no race possibility)
+ // Find the starting point of the cycle and report it.
+ // Because each type in nest must also appear in path (see invariant below),
+ // type t must be in path since it was found in nest. But not every type in path
+ // is in nest. Specifically t may appear in path with an earlier index than the
+ // index of t in nest. Search again.
+ for start, p := range path {
+ if Identical(p, t) {
+ check.cycleError(makeObjList(path[start:]))
+ return false
+ }
+ }
+ panic("cycle start not found")
+ }
+ }
+
+ // No cycle was found. Check the RHS of t.
+ // Every type added to nest is also added to path; thus every type that is in nest
+ // must also be in path (invariant). But not every type in path is in nest, since
+ // nest may be pruned (see below, *TypeParam case).
+ if !check.validType0(t.Origin().fromRHS, append(nest, t), append(path, t)) {
+ return false
+ }
+
+ check.valids.add(t) // t is valid
case *TypeParam:
// A type parameter stands for the type (argument) it was instantiated with.
- // Check the corresponding type argument for validity if we have one.
- if env != nil {
- if targ := env.tmap[t]; targ != nil {
- // Type arguments found in targ must be looked
- // up in the enclosing environment env.link.
- return check.validType0(targ, env.link, path)
+ // Check the corresponding type argument for validity if we are in an
+ // instantiated type.
+ if len(nest) > 0 {
+ inst := nest[len(nest)-1] // the type instance
+ // Find the corresponding type argument for the type parameter
+ // and proceed with checking that type argument.
+ for i, tparam := range inst.TypeParams().list() {
+ // The type parameter and type argument lists should
+ // match in length but be careful in case of errors.
+ if t == tparam && i < inst.TypeArgs().Len() {
+ targ := inst.TypeArgs().At(i)
+ // The type argument must be valid in the enclosing
+ // type (where inst was instantiated), hence we must
+ // check targ's validity in the type nest excluding
+ // the current (instantiated) type (see the example
+ // at the end of this file).
+ // For error reporting we keep the full path.
+ return check.validType0(targ, nest[:len(nest)-1], path)
+ }
}
}
}
- return valid
+ return true
}
-// A tparamEnv provides the environment for looking up the type arguments
-// with which type parameters for a given instance were instantiated.
-// If we don't have an instance, the corresponding tparamEnv is nil.
-type tparamEnv struct {
- tmap substMap
- link *tparamEnv
+// makeObjList returns the list of type name objects for the given
+// list of named types.
+func makeObjList(tlist []*Named) []Object {
+ olist := make([]Object, len(tlist))
+ for i, t := range tlist {
+ olist[i] = t.obj
+ }
+ return olist
}
-func (env *tparamEnv) push(typ *Named) *tparamEnv {
- // If typ is not an instantiated type there are no typ-specific
- // type parameters to look up and we don't need an environment.
- targs := typ.TypeArgs()
- if targs == nil {
- return nil // no instance => nil environment
- }
-
- // Populate tmap: remember the type argument for each type parameter.
- // We cannot use makeSubstMap because the number of type parameters
- // and arguments may not match due to errors in the source (too many
- // or too few type arguments). Populate tmap "manually".
- tparams := typ.TypeParams()
- n, m := targs.Len(), tparams.Len()
- if n > m {
- n = m // too many targs
- }
- tmap := make(substMap, n)
- for i := 0; i < n; i++ {
- tmap[tparams.At(i)] = targs.At(i)
- }
-
- return &tparamEnv{tmap: tmap, link: env}
-}
-
-// TODO(gri) Alternative implementation:
-// We may not need to build a stack of environments to
-// look up the type arguments for type parameters. The
-// same information should be available via the path:
-// We should be able to just walk the path backwards
-// and find the type arguments in the instance objects.
+// Here is an example illustrating why we need to exclude the
+// instantiated type from nest when evaluating the validity of
+// a type parameter. Given the declarations
+//
+// var _ A[A[string]]
+//
+// type A[P any] struct { _ B[P] }
+// type B[P any] struct { _ P }
+//
+// we want to determine if the type A[A[string]] is valid.
+// We start evaluating A[A[string]] outside any type nest:
+//
+// A[A[string]]
+// nest =
+// path =
+//
+// The RHS of A is now evaluated in the A[A[string]] nest:
+//
+// struct{_ B[P₁]}
+// nest = A[A[string]]
+// path = A[A[string]]
+//
+// The struct has a single field of type B[P₁] with which
+// we continue:
+//
+// B[P₁]
+// nest = A[A[string]]
+// path = A[A[string]]
+//
+// struct{_ P₂}
+// nest = A[A[string]]->B[P]
+// path = A[A[string]]->B[P]
+//
+// Eventutally we reach the type parameter P of type B (P₂):
+//
+// P₂
+// nest = A[A[string]]->B[P]
+// path = A[A[string]]->B[P]
+//
+// The type argument for P of B is the type parameter P of A (P₁).
+// It must be evaluated in the type nest that existed when B was
+// instantiated:
+//
+// P₁
+// nest = A[A[string]] <== type nest at B's instantiation time
+// path = A[A[string]]->B[P]
+//
+// If we'd use the current nest it would correspond to the path
+// which will be wrong as we will see shortly. P's type argument
+// is A[string], which again must be evaluated in the type nest
+// that existed when A was instantiated with A[string]. That type
+// nest is empty:
+//
+// A[string]
+// nest = <== type nest at A's instantiation time
+// path = A[A[string]]->B[P]
+//
+// Evaluation then proceeds as before for A[string]:
+//
+// struct{_ B[P₁]}
+// nest = A[string]
+// path = A[A[string]]->B[P]->A[string]
+//
+// Now we reach B[P] again. If we had not adjusted nest, it would
+// correspond to path, and we would find B[P] in nest, indicating
+// a cycle, which would clearly be wrong since there's no cycle in
+// A[string]:
+//
+// B[P₁]
+// nest = A[string]
+// path = A[A[string]]->B[P]->A[string] <== path contains B[P]!
+//
+// But because we use the correct type nest, evaluation proceeds without
+// errors and we get the evaluation sequence:
+//
+// struct{_ P₂}
+// nest = A[string]->B[P]
+// path = A[A[string]]->B[P]->A[string]->B[P]
+// P₂
+// nest = A[string]->B[P]
+// path = A[A[string]]->B[P]->A[string]->B[P]
+// P₁
+// nest = A[string]
+// path = A[A[string]]->B[P]->A[string]->B[P]
+// string
+// nest =
+// path = A[A[string]]->B[P]->A[string]->B[P]
+//
+// At this point we're done and A[A[string]] and is valid.
diff --git a/src/internal/cpu/cpu_test.go b/src/internal/cpu/cpu_test.go
index e72d2d639c..c95cd51726 100644
--- a/src/internal/cpu/cpu_test.go
+++ b/src/internal/cpu/cpu_test.go
@@ -19,7 +19,7 @@ func MustHaveDebugOptionsSupport(t *testing.T) {
}
}
-func MustSupportFeatureDectection(t *testing.T) {
+func MustSupportFeatureDetection(t *testing.T) {
// TODO: add platforms that do not have CPU feature detection support.
}
@@ -41,7 +41,7 @@ func runDebugOptionsTest(t *testing.T, test string, options string) {
}
func TestDisableAllCapabilities(t *testing.T) {
- MustSupportFeatureDectection(t)
+ MustSupportFeatureDetection(t)
runDebugOptionsTest(t, "TestAllCapabilitiesDisabled", "cpu.all=off")
}
diff --git a/src/internal/goarch/gengoarch.go b/src/internal/goarch/gengoarch.go
index 3c706e04ad..0b0be5cd15 100644
--- a/src/internal/goarch/gengoarch.go
+++ b/src/internal/goarch/gengoarch.go
@@ -11,7 +11,6 @@ import (
"fmt"
"log"
"os"
- "strconv"
"strings"
)
@@ -22,14 +21,18 @@ func main() {
if err != nil {
log.Fatal(err)
}
- const goarchPrefix = `const goarchList = `
+ const goarchPrefix = `var knownArch = map[string]bool{`
+ inGOARCH := false
for _, line := range strings.Split(string(data), "\n") {
if strings.HasPrefix(line, goarchPrefix) {
- text, err := strconv.Unquote(strings.TrimPrefix(line, goarchPrefix))
- if err != nil {
- log.Fatalf("parsing goarchList: %v", err)
- }
- goarches = strings.Fields(text)
+ inGOARCH = true
+ } else if inGOARCH && strings.HasPrefix(line, "}") {
+ break
+ } else if inGOARCH {
+ goarch := strings.Fields(line)[0]
+ goarch = strings.TrimPrefix(goarch, `"`)
+ goarch = strings.TrimSuffix(goarch, `":`)
+ goarches = append(goarches, goarch)
}
}
@@ -39,7 +42,7 @@ func main() {
}
var buf bytes.Buffer
fmt.Fprintf(&buf, "// Code generated by gengoarch.go using 'go generate'. DO NOT EDIT.\n\n")
- fmt.Fprintf(&buf, "//go:build %s\n", target) // must explicitly include target for bootstrapping purposes
+ fmt.Fprintf(&buf, "//go:build %s\n\n", target) // must explicitly include target for bootstrapping purposes
fmt.Fprintf(&buf, "package goarch\n\n")
fmt.Fprintf(&buf, "const GOARCH = `%s`\n\n", target)
for _, goarch := range goarches {
diff --git a/src/internal/goos/gengoos.go b/src/internal/goos/gengoos.go
index 1b62503fd0..37d9706d1e 100644
--- a/src/internal/goos/gengoos.go
+++ b/src/internal/goos/gengoos.go
@@ -11,7 +11,6 @@ import (
"fmt"
"log"
"os"
- "strconv"
"strings"
)
@@ -22,14 +21,18 @@ func main() {
if err != nil {
log.Fatal(err)
}
- const goosPrefix = `const goosList = `
+ const goosPrefix = `var knownOS = map[string]bool{`
+ inGOOS := false
for _, line := range strings.Split(string(data), "\n") {
if strings.HasPrefix(line, goosPrefix) {
- text, err := strconv.Unquote(strings.TrimPrefix(line, goosPrefix))
- if err != nil {
- log.Fatalf("parsing goosList: %v", err)
- }
- gooses = strings.Fields(text)
+ inGOOS = true
+ } else if inGOOS && strings.HasPrefix(line, "}") {
+ break
+ } else if inGOOS {
+ goos := strings.Fields(line)[0]
+ goos = strings.TrimPrefix(goos, `"`)
+ goos = strings.TrimSuffix(goos, `":`)
+ gooses = append(gooses, goos)
}
}
@@ -50,7 +53,7 @@ func main() {
tags = append(tags, target) // must explicitly include target for bootstrapping purposes
var buf bytes.Buffer
fmt.Fprintf(&buf, "// Code generated by gengoos.go using 'go generate'. DO NOT EDIT.\n\n")
- fmt.Fprintf(&buf, "//go:build %s\n", strings.Join(tags, " && "))
+ fmt.Fprintf(&buf, "//go:build %s\n\n", strings.Join(tags, " && "))
fmt.Fprintf(&buf, "package goos\n\n")
fmt.Fprintf(&buf, "const GOOS = `%s`\n\n", target)
for _, goos := range gooses {
diff --git a/src/internal/reflectlite/export_test.go b/src/internal/reflectlite/export_test.go
index 3e5c258fb1..e9a928bdc6 100644
--- a/src/internal/reflectlite/export_test.go
+++ b/src/internal/reflectlite/export_test.go
@@ -36,7 +36,7 @@ func Field(v Value, i int) Value {
// In the former case, we want v.ptr + offset.
// In the latter case, we must have field.offset = 0,
// so v.ptr + field.offset is still the correct address.
- ptr := add(v.ptr, field.offset(), "same as non-reflect &v.field")
+ ptr := add(v.ptr, field.offset, "same as non-reflect &v.field")
return Value{typ, ptr, fl}
}
diff --git a/src/internal/reflectlite/type.go b/src/internal/reflectlite/type.go
index bc6fc94773..21e3c1278d 100644
--- a/src/internal/reflectlite/type.go
+++ b/src/internal/reflectlite/type.go
@@ -269,17 +269,13 @@ type sliceType struct {
// Struct field
type structField struct {
- name name // name is always non-empty
- typ *rtype // type of field
- offsetEmbed uintptr // byte offset of field<<1 | isEmbedded
-}
-
-func (f *structField) offset() uintptr {
- return f.offsetEmbed >> 1
+ name name // name is always non-empty
+ typ *rtype // type of field
+ offset uintptr // byte offset of field
}
func (f *structField) embedded() bool {
- return f.offsetEmbed&1 != 0
+ return f.name.embedded()
}
// structType represents a struct type.
@@ -328,6 +324,10 @@ func (n name) hasTag() bool {
return (*n.bytes)&(1<<1) != 0
}
+func (n name) embedded() bool {
+ return (*n.bytes)&(1<<3) != 0
+}
+
// readVarint parses a varint as encoded by encoding/binary.
// It returns the number of encoded bytes and the encoded value.
func (n name) readVarint(off int) (int, int) {
@@ -947,7 +947,10 @@ func haveIdenticalUnderlyingType(T, V *rtype, cmpTags bool) bool {
if cmpTags && tf.name.tag() != vf.name.tag() {
return false
}
- if tf.offsetEmbed != vf.offsetEmbed {
+ if tf.offset != vf.offset {
+ return false
+ }
+ if tf.embedded() != vf.embedded() {
return false
}
}
diff --git a/src/internal/testenv/testenv.go b/src/internal/testenv/testenv.go
index 1feb630cf5..b7cb95063b 100644
--- a/src/internal/testenv/testenv.go
+++ b/src/internal/testenv/testenv.go
@@ -16,6 +16,7 @@ import (
"flag"
"fmt"
"internal/cfg"
+ "internal/testmath"
"os"
"os/exec"
"path/filepath"
@@ -463,3 +464,67 @@ func RunWithTimeout(t testing.TB, cmd *exec.Cmd) ([]byte, error) {
return b.Bytes(), err
}
+
+// CheckLinear checks if the function produced by f scales linearly.
+//
+// f must accept a scale factor which causes the input to the function it
+// produces to scale by that factor.
+func CheckLinear(t *testing.T, f func(scale float64) func(*testing.B)) {
+ MustHaveExec(t)
+
+ if os.Getenv("GO_PERF_UNIT_TEST") == "" {
+ // Invoke the same test as a subprocess with the GO_PERF_UNIT_TEST environment variable set.
+ // We create a subprocess for two reasons:
+ //
+ // 1. There's no other way to set the benchmarking parameters of testing.Benchmark.
+ // 2. Since we're effectively running a performance test, running in a subprocess grants
+ // us a little bit more isolation than using the same process.
+ //
+ // As an alternative, we could fairly easily reimplement the timing code in testing.Benchmark,
+ // but a subprocess is just as easy to create.
+
+ selfCmd := CleanCmdEnv(exec.Command(os.Args[0], "-test.v", fmt.Sprintf("-test.run=^%s$", t.Name()), "-test.benchtime=1x"))
+ selfCmd.Env = append(selfCmd.Env, "GO_PERF_UNIT_TEST=1")
+ output, err := RunWithTimeout(t, selfCmd)
+ if err != nil {
+ t.Error(err)
+ t.Logf("--- subprocess output ---\n%s", string(output))
+ }
+ if bytes.Contains(output, []byte("insignificant result")) {
+ t.Skip("insignificant result")
+ }
+ return
+ }
+
+ // Pick a reasonable sample count.
+ const count = 10
+
+ // Collect samples for scale factor 1.
+ x1 := make([]testing.BenchmarkResult, 0, count)
+ for i := 0; i < count; i++ {
+ x1 = append(x1, testing.Benchmark(f(1.0)))
+ }
+
+ // Collect samples for scale factor 2.
+ x2 := make([]testing.BenchmarkResult, 0, count)
+ for i := 0; i < count; i++ {
+ x2 = append(x2, testing.Benchmark(f(2.0)))
+ }
+
+ // Run a t-test on the results.
+ r1 := testmath.BenchmarkResults(x1)
+ r2 := testmath.BenchmarkResults(x2)
+ result, err := testmath.TwoSampleWelchTTest(r1, r2, testmath.LocationDiffers)
+ if err != nil {
+ t.Fatalf("failed to run t-test: %v", err)
+ }
+ if result.P > 0.005 {
+ // Insignificant result.
+ t.Skip("insignificant result")
+ }
+
+ // Let ourselves be within 3x; 2x is too strict.
+ if m1, m2 := r1.Mean(), r2.Mean(); 3.0*m1 < m2 {
+ t.Fatalf("failure to scale linearly: µ_1=%s µ_2=%s p=%f", time.Duration(m1), time.Duration(m2), result.P)
+ }
+}
diff --git a/src/internal/testmath/bench.go b/src/internal/testmath/bench.go
new file mode 100644
index 0000000000..6f034b4685
--- /dev/null
+++ b/src/internal/testmath/bench.go
@@ -0,0 +1,38 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package testmath
+
+import (
+ "math"
+ "testing"
+ "time"
+)
+
+type BenchmarkResults []testing.BenchmarkResult
+
+func (b BenchmarkResults) Weight() float64 {
+ var weight int
+ for _, r := range b {
+ weight += r.N
+ }
+ return float64(weight)
+}
+
+func (b BenchmarkResults) Mean() float64 {
+ var dur time.Duration
+ for _, r := range b {
+ dur += r.T * time.Duration(r.N)
+ }
+ return float64(dur) / b.Weight()
+}
+
+func (b BenchmarkResults) Variance() float64 {
+ var num float64
+ mean := b.Mean()
+ for _, r := range b {
+ num += math.Pow(float64(r.T)-mean, 2) * float64(r.N)
+ }
+ return float64(num) / b.Weight()
+}
diff --git a/src/internal/testmath/ttest.go b/src/internal/testmath/ttest.go
new file mode 100644
index 0000000000..d15d2deebb
--- /dev/null
+++ b/src/internal/testmath/ttest.go
@@ -0,0 +1,213 @@
+// Copyright 2022 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package testmath
+
+import (
+ "errors"
+ "math"
+)
+
+// A TTestSample is a sample that can be used for a one or two sample
+// t-test.
+type TTestSample interface {
+ Weight() float64
+ Mean() float64
+ Variance() float64
+}
+
+var (
+ ErrSampleSize = errors.New("sample is too small")
+ ErrZeroVariance = errors.New("sample has zero variance")
+ ErrMismatchedSamples = errors.New("samples have different lengths")
+)
+
+// TwoSampleWelchTTest performs a two-sample (unpaired) Welch's t-test
+// on samples x1 and x2. This t-test does not assume the distributions
+// have equal variance.
+func TwoSampleWelchTTest(x1, x2 TTestSample, alt LocationHypothesis) (*TTestResult, error) {
+ n1, n2 := x1.Weight(), x2.Weight()
+ if n1 <= 1 || n2 <= 1 {
+ // TODO: Can we still do this with n == 1?
+ return nil, ErrSampleSize
+ }
+ v1, v2 := x1.Variance(), x2.Variance()
+ if v1 == 0 && v2 == 0 {
+ return nil, ErrZeroVariance
+ }
+
+ dof := math.Pow(v1/n1+v2/n2, 2) /
+ (math.Pow(v1/n1, 2)/(n1-1) + math.Pow(v2/n2, 2)/(n2-1))
+ s := math.Sqrt(v1/n1 + v2/n2)
+ t := (x1.Mean() - x2.Mean()) / s
+ return newTTestResult(int(n1), int(n2), t, dof, alt), nil
+}
+
+// A TTestResult is the result of a t-test.
+type TTestResult struct {
+ // N1 and N2 are the sizes of the input samples. For a
+ // one-sample t-test, N2 is 0.
+ N1, N2 int
+
+ // T is the value of the t-statistic for this t-test.
+ T float64
+
+ // DoF is the degrees of freedom for this t-test.
+ DoF float64
+
+ // AltHypothesis specifies the alternative hypothesis tested
+ // by this test against the null hypothesis that there is no
+ // difference in the means of the samples.
+ AltHypothesis LocationHypothesis
+
+ // P is p-value for this t-test for the given null hypothesis.
+ P float64
+}
+
+func newTTestResult(n1, n2 int, t, dof float64, alt LocationHypothesis) *TTestResult {
+ dist := TDist{dof}
+ var p float64
+ switch alt {
+ case LocationDiffers:
+ p = 2 * (1 - dist.CDF(math.Abs(t)))
+ case LocationLess:
+ p = dist.CDF(t)
+ case LocationGreater:
+ p = 1 - dist.CDF(t)
+ }
+ return &TTestResult{N1: n1, N2: n2, T: t, DoF: dof, AltHypothesis: alt, P: p}
+}
+
+// A LocationHypothesis specifies the alternative hypothesis of a
+// location test such as a t-test or a Mann-Whitney U-test. The
+// default (zero) value is to test against the alternative hypothesis
+// that they differ.
+type LocationHypothesis int
+
+const (
+ // LocationLess specifies the alternative hypothesis that the
+ // location of the first sample is less than the second. This
+ // is a one-tailed test.
+ LocationLess LocationHypothesis = -1
+
+ // LocationDiffers specifies the alternative hypothesis that
+ // the locations of the two samples are not equal. This is a
+ // two-tailed test.
+ LocationDiffers LocationHypothesis = 0
+
+ // LocationGreater specifies the alternative hypothesis that
+ // the location of the first sample is greater than the
+ // second. This is a one-tailed test.
+ LocationGreater LocationHypothesis = 1
+)
+
+// A TDist is a Student's t-distribution with V degrees of freedom.
+type TDist struct {
+ V float64
+}
+
+// PDF returns the value at x of the probability distribution function for the
+// distribution.
+func (t TDist) PDF(x float64) float64 {
+ return math.Exp(lgamma((t.V+1)/2)-lgamma(t.V/2)) /
+ math.Sqrt(t.V*math.Pi) * math.Pow(1+(x*x)/t.V, -(t.V+1)/2)
+}
+
+// CDF returns the value at x of the cumulative distribution function for the
+// distribution.
+func (t TDist) CDF(x float64) float64 {
+ if x == 0 {
+ return 0.5
+ } else if x > 0 {
+ return 1 - 0.5*betaInc(t.V/(t.V+x*x), t.V/2, 0.5)
+ } else if x < 0 {
+ return 1 - t.CDF(-x)
+ } else {
+ return math.NaN()
+ }
+}
+
+func (t TDist) Bounds() (float64, float64) {
+ return -4, 4
+}
+
+func lgamma(x float64) float64 {
+ y, _ := math.Lgamma(x)
+ return y
+}
+
+// betaInc returns the value of the regularized incomplete beta
+// function Iₓ(a, b) = 1 / B(a, b) * ∫₀ˣ tᵃ⁻¹ (1-t)ᵇ⁻¹ dt.
+//
+// This is not to be confused with the "incomplete beta function",
+// which can be computed as BetaInc(x, a, b)*Beta(a, b).
+//
+// If x < 0 or x > 1, returns NaN.
+func betaInc(x, a, b float64) float64 {
+ // Based on Numerical Recipes in C, section 6.4. This uses the
+ // continued fraction definition of I:
+ //
+ // (xᵃ*(1-x)ᵇ)/(a*B(a,b)) * (1/(1+(d₁/(1+(d₂/(1+...))))))
+ //
+ // where B(a,b) is the beta function and
+ //
+ // d_{2m+1} = -(a+m)(a+b+m)x/((a+2m)(a+2m+1))
+ // d_{2m} = m(b-m)x/((a+2m-1)(a+2m))
+ if x < 0 || x > 1 {
+ return math.NaN()
+ }
+ bt := 0.0
+ if 0 < x && x < 1 {
+ // Compute the coefficient before the continued
+ // fraction.
+ bt = math.Exp(lgamma(a+b) - lgamma(a) - lgamma(b) +
+ a*math.Log(x) + b*math.Log(1-x))
+ }
+ if x < (a+1)/(a+b+2) {
+ // Compute continued fraction directly.
+ return bt * betacf(x, a, b) / a
+ } else {
+ // Compute continued fraction after symmetry transform.
+ return 1 - bt*betacf(1-x, b, a)/b
+ }
+}
+
+// betacf is the continued fraction component of the regularized
+// incomplete beta function Iₓ(a, b).
+func betacf(x, a, b float64) float64 {
+ const maxIterations = 200
+ const epsilon = 3e-14
+
+ raiseZero := func(z float64) float64 {
+ if math.Abs(z) < math.SmallestNonzeroFloat64 {
+ return math.SmallestNonzeroFloat64
+ }
+ return z
+ }
+
+ c := 1.0
+ d := 1 / raiseZero(1-(a+b)*x/(a+1))
+ h := d
+ for m := 1; m <= maxIterations; m++ {
+ mf := float64(m)
+
+ // Even step of the recurrence.
+ numer := mf * (b - mf) * x / ((a + 2*mf - 1) * (a + 2*mf))
+ d = 1 / raiseZero(1+numer*d)
+ c = raiseZero(1 + numer/c)
+ h *= d * c
+
+ // Odd step of the recurrence.
+ numer = -(a + mf) * (a + b + mf) * x / ((a + 2*mf) * (a + 2*mf + 1))
+ d = 1 / raiseZero(1+numer*d)
+ c = raiseZero(1 + numer/c)
+ hfac := d * c
+ h *= hfac
+
+ if math.Abs(hfac-1) < epsilon {
+ return h
+ }
+ }
+ panic("betainc: a or b too big; failed to converge")
+}
diff --git a/src/io/example_test.go b/src/io/example_test.go
index e4b20bd981..419e449982 100644
--- a/src/io/example_test.go
+++ b/src/io/example_test.go
@@ -6,7 +6,6 @@ package io_test
import (
"bytes"
- "errors"
"fmt"
"io"
"log"
@@ -284,16 +283,3 @@ func ExampleReadAll() {
// Output:
// Go is a general-purpose language designed with systems programming in mind.
}
-
-func ExampleLimitedReader() {
- r := strings.NewReader("some io.Reader stream to be read\n")
- sentinel := errors.New("reached read limit")
- lr := &io.LimitedReader{R: r, N: 4, Err: sentinel}
-
- if _, err := io.Copy(os.Stdout, lr); err != sentinel {
- log.Fatal(err)
- }
-
- // Output:
- // some
-}
diff --git a/src/io/io.go b/src/io/io.go
index 830779e79d..db88125f50 100644
--- a/src/io/io.go
+++ b/src/io/io.go
@@ -455,26 +455,20 @@ func copyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error) {
// LimitReader returns a Reader that reads from r
// but stops with EOF after n bytes.
// The underlying implementation is a *LimitedReader.
-func LimitReader(r Reader, n int64) Reader { return &LimitedReader{r, n, nil} }
+func LimitReader(r Reader, n int64) Reader { return &LimitedReader{r, n} }
// A LimitedReader reads from R but limits the amount of
// data returned to just N bytes. Each call to Read
// updates N to reflect the new amount remaining.
-// Read returns Err when N <= 0.
-// If Err is nil, it returns EOF instead.
+// Read returns EOF when N <= 0 or when the underlying R returns EOF.
type LimitedReader struct {
- R Reader // underlying reader
- N int64 // max bytes remaining
- Err error // error to return on reaching the limit
+ R Reader // underlying reader
+ N int64 // max bytes remaining
}
func (l *LimitedReader) Read(p []byte) (n int, err error) {
if l.N <= 0 {
- err := l.Err
- if err == nil {
- err = EOF
- }
- return 0, err
+ return 0, EOF
}
if int64(len(p)) > l.N {
p = p[0:l.N]
diff --git a/src/net/dial.go b/src/net/dial.go
index b24bd2f5f4..c538342566 100644
--- a/src/net/dial.go
+++ b/src/net/dial.go
@@ -341,6 +341,7 @@ func DialTimeout(network, address string, timeout time.Duration) (Conn, error) {
type sysDialer struct {
Dialer
network, address string
+ testHookDialTCP func(ctx context.Context, net string, laddr, raddr *TCPAddr) (*TCPConn, error)
}
// Dial connects to the address on the named network.
diff --git a/src/net/dial_test.go b/src/net/dial_test.go
index afec31f636..1256867da8 100644
--- a/src/net/dial_test.go
+++ b/src/net/dial_test.go
@@ -9,6 +9,8 @@ package net
import (
"bufio"
"context"
+ "errors"
+ "fmt"
"internal/testenv"
"io"
"os"
@@ -175,31 +177,9 @@ func dialClosedPort(t *testing.T) (dialLatency time.Duration) {
}
func TestDialParallel(t *testing.T) {
- testenv.MustHaveExternalNetwork(t)
-
- if !supportsIPv4() || !supportsIPv6() {
- t.Skip("both IPv4 and IPv6 are required")
- }
-
- closedPortDelay := dialClosedPort(t)
-
const instant time.Duration = 0
const fallbackDelay = 200 * time.Millisecond
- // Some cases will run quickly when "connection refused" is fast,
- // or trigger the fallbackDelay on Windows. This value holds the
- // lesser of the two delays.
- var closedPortOrFallbackDelay time.Duration
- if closedPortDelay < fallbackDelay {
- closedPortOrFallbackDelay = closedPortDelay
- } else {
- closedPortOrFallbackDelay = fallbackDelay
- }
-
- origTestHookDialTCP := testHookDialTCP
- defer func() { testHookDialTCP = origTestHookDialTCP }()
- testHookDialTCP = slowDialTCP
-
nCopies := func(s string, n int) []string {
out := make([]string, n)
for i := 0; i < n; i++ {
@@ -223,31 +203,21 @@ func TestDialParallel(t *testing.T) {
// Primary is slow; fallback should kick in.
{[]string{slowDst4}, []string{"::1"}, "", true, fallbackDelay},
// Skip a "connection refused" in the primary thread.
- {[]string{"127.0.0.1", "::1"}, []string{}, "tcp4", true, closedPortDelay},
- {[]string{"::1", "127.0.0.1"}, []string{}, "tcp6", true, closedPortDelay},
+ {[]string{"127.0.0.1", "::1"}, []string{}, "tcp4", true, instant},
+ {[]string{"::1", "127.0.0.1"}, []string{}, "tcp6", true, instant},
// Skip a "connection refused" in the fallback thread.
- {[]string{slowDst4, slowDst6}, []string{"::1", "127.0.0.1"}, "tcp6", true, fallbackDelay + closedPortDelay},
+ {[]string{slowDst4, slowDst6}, []string{"::1", "127.0.0.1"}, "tcp6", true, fallbackDelay},
// Primary refused, fallback without delay.
- {[]string{"127.0.0.1"}, []string{"::1"}, "tcp4", true, closedPortOrFallbackDelay},
- {[]string{"::1"}, []string{"127.0.0.1"}, "tcp6", true, closedPortOrFallbackDelay},
+ {[]string{"127.0.0.1"}, []string{"::1"}, "tcp4", true, instant},
+ {[]string{"::1"}, []string{"127.0.0.1"}, "tcp6", true, instant},
// Everything is refused.
- {[]string{"127.0.0.1"}, []string{}, "tcp4", false, closedPortDelay},
+ {[]string{"127.0.0.1"}, []string{}, "tcp4", false, instant},
// Nothing to do; fail instantly.
{[]string{}, []string{}, "", false, instant},
// Connecting to tons of addresses should not trip the deadline.
{nCopies("::1", 1000), []string{}, "", true, instant},
}
- handler := func(dss *dualStackServer, ln Listener) {
- for {
- c, err := ln.Accept()
- if err != nil {
- return
- }
- c.Close()
- }
- }
-
// Convert a list of IP strings into TCPAddrs.
makeAddrs := func(ips []string, port string) addrList {
var out addrList
@@ -262,76 +232,73 @@ func TestDialParallel(t *testing.T) {
}
for i, tt := range testCases {
- dss, err := newDualStackServer()
- if err != nil {
- t.Fatal(err)
- }
- defer dss.teardown()
- if err := dss.buildup(handler); err != nil {
- t.Fatal(err)
- }
- if tt.teardownNetwork != "" {
- // Destroy one of the listening sockets, creating an unreachable port.
- dss.teardownNetwork(tt.teardownNetwork)
- }
+ i, tt := i, tt
+ t.Run(fmt.Sprint(i), func(t *testing.T) {
+ dialTCP := func(ctx context.Context, network string, laddr, raddr *TCPAddr) (*TCPConn, error) {
+ n := "tcp6"
+ if raddr.IP.To4() != nil {
+ n = "tcp4"
+ }
+ if n == tt.teardownNetwork {
+ return nil, errors.New("unreachable")
+ }
+ if r := raddr.IP.String(); r == slowDst4 || r == slowDst6 {
+ <-ctx.Done()
+ return nil, ctx.Err()
+ }
+ return &TCPConn{}, nil
+ }
- primaries := makeAddrs(tt.primaries, dss.port)
- fallbacks := makeAddrs(tt.fallbacks, dss.port)
- d := Dialer{
- FallbackDelay: fallbackDelay,
- }
- startTime := time.Now()
- sd := &sysDialer{
- Dialer: d,
- network: "tcp",
- address: "?",
- }
- c, err := sd.dialParallel(context.Background(), primaries, fallbacks)
- elapsed := time.Since(startTime)
+ primaries := makeAddrs(tt.primaries, "80")
+ fallbacks := makeAddrs(tt.fallbacks, "80")
+ d := Dialer{
+ FallbackDelay: fallbackDelay,
+ }
+ const forever = 60 * time.Minute
+ if tt.expectElapsed == instant {
+ d.FallbackDelay = forever
+ }
+ startTime := time.Now()
+ sd := &sysDialer{
+ Dialer: d,
+ network: "tcp",
+ address: "?",
+ testHookDialTCP: dialTCP,
+ }
+ c, err := sd.dialParallel(context.Background(), primaries, fallbacks)
+ elapsed := time.Since(startTime)
- if c != nil {
- c.Close()
- }
+ if c != nil {
+ c.Close()
+ }
- if tt.expectOk && err != nil {
- t.Errorf("#%d: got %v; want nil", i, err)
- } else if !tt.expectOk && err == nil {
- t.Errorf("#%d: got nil; want non-nil", i)
- }
+ if tt.expectOk && err != nil {
+ t.Errorf("#%d: got %v; want nil", i, err)
+ } else if !tt.expectOk && err == nil {
+ t.Errorf("#%d: got nil; want non-nil", i)
+ }
- // We used to always use 95 milliseconds as the slop,
- // but that was flaky on Windows. See issue 35616.
- slop := 95 * time.Millisecond
- if half := tt.expectElapsed / 2; half > slop {
- slop = half
- }
- expectElapsedMin := tt.expectElapsed - slop
- expectElapsedMax := tt.expectElapsed + slop
- if elapsed < expectElapsedMin {
- t.Errorf("#%d: got %v; want >= %v", i, elapsed, expectElapsedMin)
- } else if elapsed > expectElapsedMax {
- t.Errorf("#%d: got %v; want <= %v", i, elapsed, expectElapsedMax)
- }
+ if elapsed < tt.expectElapsed || elapsed >= forever {
+ t.Errorf("#%d: got %v; want >= %v, < forever", i, elapsed, tt.expectElapsed)
+ }
- // Repeat each case, ensuring that it can be canceled quickly.
- ctx, cancel := context.WithCancel(context.Background())
- var wg sync.WaitGroup
- wg.Add(1)
- go func() {
- time.Sleep(5 * time.Millisecond)
- cancel()
- wg.Done()
- }()
- startTime = time.Now()
- c, err = sd.dialParallel(ctx, primaries, fallbacks)
- if c != nil {
- c.Close()
- }
- elapsed = time.Now().Sub(startTime)
- if elapsed > 100*time.Millisecond {
- t.Errorf("#%d (cancel): got %v; want <= 100ms", i, elapsed)
- }
- wg.Wait()
+ // Repeat each case, ensuring that it can be canceled.
+ ctx, cancel := context.WithCancel(context.Background())
+ var wg sync.WaitGroup
+ wg.Add(1)
+ go func() {
+ time.Sleep(5 * time.Millisecond)
+ cancel()
+ wg.Done()
+ }()
+ // Ignore errors, since all we care about is that the
+ // call can be canceled.
+ c, _ = sd.dialParallel(ctx, primaries, fallbacks)
+ if c != nil {
+ c.Close()
+ }
+ wg.Wait()
+ })
}
}
@@ -438,6 +405,16 @@ func TestDialParallelSpuriousConnection(t *testing.T) {
t.Fatal(err)
}
+ // Workaround for https://go.dev/issue/37795.
+ // On arm64 macOS (current as of macOS 12.4),
+ // reading from a socket at the same time as the client
+ // is closing it occasionally hangs for 60 seconds before
+ // returning ECONNRESET. Sleep for a bit to give the
+ // socket time to close before trying to read from it.
+ if runtime.GOOS == "darwin" && runtime.GOARCH == "arm64" {
+ time.Sleep(10 * time.Millisecond)
+ }
+
// The client should close itself, without sending data.
c.SetReadDeadline(readDeadline)
var b [1]byte
diff --git a/src/net/http/status.go b/src/net/http/status.go
index 75fea0ca35..cd90877ef0 100644
--- a/src/net/http/status.go
+++ b/src/net/http/status.go
@@ -7,68 +7,68 @@ package http
// HTTP status codes as registered with IANA.
// See: https://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml
const (
- StatusContinue = 100 // RFC 7231, 6.2.1
- StatusSwitchingProtocols = 101 // RFC 7231, 6.2.2
+ StatusContinue = 100 // RFC 9110, 15.2.1
+ StatusSwitchingProtocols = 101 // RFC 9110, 15.2.2
StatusProcessing = 102 // RFC 2518, 10.1
StatusEarlyHints = 103 // RFC 8297
- StatusOK = 200 // RFC 7231, 6.3.1
- StatusCreated = 201 // RFC 7231, 6.3.2
- StatusAccepted = 202 // RFC 7231, 6.3.3
- StatusNonAuthoritativeInfo = 203 // RFC 7231, 6.3.4
- StatusNoContent = 204 // RFC 7231, 6.3.5
- StatusResetContent = 205 // RFC 7231, 6.3.6
- StatusPartialContent = 206 // RFC 7233, 4.1
+ StatusOK = 200 // RFC 9110, 15.3.1
+ StatusCreated = 201 // RFC 9110, 15.3.2
+ StatusAccepted = 202 // RFC 9110, 15.3.3
+ StatusNonAuthoritativeInfo = 203 // RFC 9110, 15.3.4
+ StatusNoContent = 204 // RFC 9110, 15.3.5
+ StatusResetContent = 205 // RFC 9110, 15.3.6
+ StatusPartialContent = 206 // RFC 9110, 15.3.7
StatusMultiStatus = 207 // RFC 4918, 11.1
StatusAlreadyReported = 208 // RFC 5842, 7.1
StatusIMUsed = 226 // RFC 3229, 10.4.1
- StatusMultipleChoices = 300 // RFC 7231, 6.4.1
- StatusMovedPermanently = 301 // RFC 7231, 6.4.2
- StatusFound = 302 // RFC 7231, 6.4.3
- StatusSeeOther = 303 // RFC 7231, 6.4.4
- StatusNotModified = 304 // RFC 7232, 4.1
- StatusUseProxy = 305 // RFC 7231, 6.4.5
- _ = 306 // RFC 7231, 6.4.6 (Unused)
- StatusTemporaryRedirect = 307 // RFC 7231, 6.4.7
- StatusPermanentRedirect = 308 // RFC 7538, 3
+ StatusMultipleChoices = 300 // RFC 9110, 15.4.1
+ StatusMovedPermanently = 301 // RFC 9110, 15.4.2
+ StatusFound = 302 // RFC 9110, 15.4.3
+ StatusSeeOther = 303 // RFC 9110, 15.4.4
+ StatusNotModified = 304 // RFC 9110, 15.4.5
+ StatusUseProxy = 305 // RFC 9110, 15.4.6
+ _ = 306 // RFC 9110, 15.4.7 (Unused)
+ StatusTemporaryRedirect = 307 // RFC 9110, 15.4.8
+ StatusPermanentRedirect = 308 // RFC 9110, 15.4.9
- StatusBadRequest = 400 // RFC 7231, 6.5.1
- StatusUnauthorized = 401 // RFC 7235, 3.1
- StatusPaymentRequired = 402 // RFC 7231, 6.5.2
- StatusForbidden = 403 // RFC 7231, 6.5.3
- StatusNotFound = 404 // RFC 7231, 6.5.4
- StatusMethodNotAllowed = 405 // RFC 7231, 6.5.5
- StatusNotAcceptable = 406 // RFC 7231, 6.5.6
- StatusProxyAuthRequired = 407 // RFC 7235, 3.2
- StatusRequestTimeout = 408 // RFC 7231, 6.5.7
- StatusConflict = 409 // RFC 7231, 6.5.8
- StatusGone = 410 // RFC 7231, 6.5.9
- StatusLengthRequired = 411 // RFC 7231, 6.5.10
- StatusPreconditionFailed = 412 // RFC 7232, 4.2
- StatusRequestEntityTooLarge = 413 // RFC 7231, 6.5.11
- StatusRequestURITooLong = 414 // RFC 7231, 6.5.12
- StatusUnsupportedMediaType = 415 // RFC 7231, 6.5.13
- StatusRequestedRangeNotSatisfiable = 416 // RFC 7233, 4.4
- StatusExpectationFailed = 417 // RFC 7231, 6.5.14
- StatusTeapot = 418 // RFC 7168, 2.3.3
- StatusMisdirectedRequest = 421 // RFC 7540, 9.1.2
- StatusUnprocessableEntity = 422 // RFC 4918, 11.2
+ StatusBadRequest = 400 // RFC 9110, 15.5.1
+ StatusUnauthorized = 401 // RFC 9110, 15.5.2
+ StatusPaymentRequired = 402 // RFC 9110, 15.5.3
+ StatusForbidden = 403 // RFC 9110, 15.5.4
+ StatusNotFound = 404 // RFC 9110, 15.5.5
+ StatusMethodNotAllowed = 405 // RFC 9110, 15.5.6
+ StatusNotAcceptable = 406 // RFC 9110, 15.5.7
+ StatusProxyAuthRequired = 407 // RFC 9110, 15.5.8
+ StatusRequestTimeout = 408 // RFC 9110, 15.5.9
+ StatusConflict = 409 // RFC 9110, 15.5.10
+ StatusGone = 410 // RFC 9110, 15.5.11
+ StatusLengthRequired = 411 // RFC 9110, 15.5.12
+ StatusPreconditionFailed = 412 // RFC 9110, 15.5.13
+ StatusRequestEntityTooLarge = 413 // RFC 9110, 15.5.14
+ StatusRequestURITooLong = 414 // RFC 9110, 15.5.15
+ StatusUnsupportedMediaType = 415 // RFC 9110, 15.5.16
+ StatusRequestedRangeNotSatisfiable = 416 // RFC 9110, 15.5.17
+ StatusExpectationFailed = 417 // RFC 9110, 15.5.18
+ StatusTeapot = 418 // RFC 9110, 15.5.19 (Unused)
+ StatusMisdirectedRequest = 421 // RFC 9110, 15.5.20
+ StatusUnprocessableEntity = 422 // RFC 9110, 15.5.21
StatusLocked = 423 // RFC 4918, 11.3
StatusFailedDependency = 424 // RFC 4918, 11.4
StatusTooEarly = 425 // RFC 8470, 5.2.
- StatusUpgradeRequired = 426 // RFC 7231, 6.5.15
+ StatusUpgradeRequired = 426 // RFC 9110, 15.5.22
StatusPreconditionRequired = 428 // RFC 6585, 3
StatusTooManyRequests = 429 // RFC 6585, 4
StatusRequestHeaderFieldsTooLarge = 431 // RFC 6585, 5
StatusUnavailableForLegalReasons = 451 // RFC 7725, 3
- StatusInternalServerError = 500 // RFC 7231, 6.6.1
- StatusNotImplemented = 501 // RFC 7231, 6.6.2
- StatusBadGateway = 502 // RFC 7231, 6.6.3
- StatusServiceUnavailable = 503 // RFC 7231, 6.6.4
- StatusGatewayTimeout = 504 // RFC 7231, 6.6.5
- StatusHTTPVersionNotSupported = 505 // RFC 7231, 6.6.6
+ StatusInternalServerError = 500 // RFC 9110, 15.6.1
+ StatusNotImplemented = 501 // RFC 9110, 15.6.2
+ StatusBadGateway = 502 // RFC 9110, 15.6.3
+ StatusServiceUnavailable = 503 // RFC 9110, 15.6.4
+ StatusGatewayTimeout = 504 // RFC 9110, 15.6.5
+ StatusHTTPVersionNotSupported = 505 // RFC 9110, 15.6.6
StatusVariantAlsoNegotiates = 506 // RFC 2295, 8.1
StatusInsufficientStorage = 507 // RFC 4918, 11.5
StatusLoopDetected = 508 // RFC 5842, 7.2
diff --git a/src/net/lookup_windows.go b/src/net/lookup_windows.go
index 051f47da39..9ff39c74a4 100644
--- a/src/net/lookup_windows.go
+++ b/src/net/lookup_windows.go
@@ -329,7 +329,7 @@ func (r *Resolver) lookupNS(ctx context.Context, name string) ([]*NS, error) {
func (r *Resolver) lookupTXT(ctx context.Context, name string) ([]string, error) {
if r.preferGoOverWindows() {
- return r.lookupTXT(ctx, name)
+ return r.goLookupTXT(ctx, name)
}
// TODO(bradfitz): finish ctx plumbing. Nothing currently depends on this.
acquireThread()
diff --git a/src/net/lookup_windows_test.go b/src/net/lookup_windows_test.go
index 9254733364..823ec088b8 100644
--- a/src/net/lookup_windows_test.go
+++ b/src/net/lookup_windows_test.go
@@ -6,6 +6,7 @@ package net
import (
"bytes"
+ "context"
"encoding/json"
"errors"
"fmt"
@@ -26,104 +27,117 @@ func toJson(v any) string {
return string(data)
}
+func testLookup(t *testing.T, fn func(*testing.T, *Resolver, string)) {
+ for _, def := range []bool{true, false} {
+ def := def
+ for _, server := range nslookupTestServers {
+ server := server
+ var name string
+ if def {
+ name = "default/"
+ } else {
+ name = "go/"
+ }
+ t.Run(name+server, func(t *testing.T) {
+ t.Parallel()
+ r := DefaultResolver
+ if !def {
+ r = &Resolver{PreferGo: true}
+ }
+ fn(t, r, server)
+ })
+ }
+ }
+}
+
func TestNSLookupMX(t *testing.T) {
testenv.MustHaveExternalNetwork(t)
- for _, server := range nslookupTestServers {
- mx, err := LookupMX(server)
+ testLookup(t, func(t *testing.T, r *Resolver, server string) {
+ mx, err := r.LookupMX(context.Background(), server)
if err != nil {
- t.Error(err)
- continue
+ t.Fatal(err)
}
if len(mx) == 0 {
- t.Errorf("no results")
- continue
+ t.Fatal("no results")
}
expected, err := nslookupMX(server)
if err != nil {
- t.Logf("skipping failed nslookup %s test: %s", server, err)
+ t.Skipf("skipping failed nslookup %s test: %s", server, err)
}
sort.Sort(byPrefAndHost(expected))
sort.Sort(byPrefAndHost(mx))
if !reflect.DeepEqual(expected, mx) {
t.Errorf("different results %s:\texp:%v\tgot:%v", server, toJson(expected), toJson(mx))
}
- }
+ })
}
func TestNSLookupCNAME(t *testing.T) {
testenv.MustHaveExternalNetwork(t)
- for _, server := range nslookupTestServers {
- cname, err := LookupCNAME(server)
+ testLookup(t, func(t *testing.T, r *Resolver, server string) {
+ cname, err := r.LookupCNAME(context.Background(), server)
if err != nil {
- t.Errorf("failed %s: %s", server, err)
- continue
+ t.Fatalf("failed %s: %s", server, err)
}
if cname == "" {
- t.Errorf("no result %s", server)
+ t.Fatalf("no result %s", server)
}
expected, err := nslookupCNAME(server)
if err != nil {
- t.Logf("skipping failed nslookup %s test: %s", server, err)
- continue
+ t.Skipf("skipping failed nslookup %s test: %s", server, err)
}
if expected != cname {
t.Errorf("different results %s:\texp:%v\tgot:%v", server, expected, cname)
}
- }
+ })
}
func TestNSLookupNS(t *testing.T) {
testenv.MustHaveExternalNetwork(t)
- for _, server := range nslookupTestServers {
- ns, err := LookupNS(server)
+ testLookup(t, func(t *testing.T, r *Resolver, server string) {
+ ns, err := r.LookupNS(context.Background(), server)
if err != nil {
- t.Errorf("failed %s: %s", server, err)
- continue
+ t.Fatalf("failed %s: %s", server, err)
}
if len(ns) == 0 {
- t.Errorf("no results")
- continue
+ t.Fatal("no results")
}
expected, err := nslookupNS(server)
if err != nil {
- t.Logf("skipping failed nslookup %s test: %s", server, err)
- continue
+ t.Skipf("skipping failed nslookup %s test: %s", server, err)
}
sort.Sort(byHost(expected))
sort.Sort(byHost(ns))
if !reflect.DeepEqual(expected, ns) {
t.Errorf("different results %s:\texp:%v\tgot:%v", toJson(server), toJson(expected), ns)
}
- }
+ })
}
func TestNSLookupTXT(t *testing.T) {
testenv.MustHaveExternalNetwork(t)
- for _, server := range nslookupTestServers {
- txt, err := LookupTXT(server)
+ testLookup(t, func(t *testing.T, r *Resolver, server string) {
+ txt, err := r.LookupTXT(context.Background(), server)
if err != nil {
- t.Errorf("failed %s: %s", server, err)
- continue
+ t.Fatalf("failed %s: %s", server, err)
}
if len(txt) == 0 {
- t.Errorf("no results")
- continue
+ t.Fatalf("no results")
}
expected, err := nslookupTXT(server)
if err != nil {
- t.Logf("skipping failed nslookup %s test: %s", server, err)
- continue
+ t.Skipf("skipping failed nslookup %s test: %s", server, err)
}
sort.Strings(expected)
sort.Strings(txt)
if !reflect.DeepEqual(expected, txt) {
t.Errorf("different results %s:\texp:%v\tgot:%v", server, toJson(expected), toJson(txt))
}
- }
+ })
}
func TestLookupLocalPTR(t *testing.T) {
diff --git a/src/net/netip/netip.go b/src/net/netip/netip.go
index eae9c29ea7..bb83371a55 100644
--- a/src/net/netip/netip.go
+++ b/src/net/netip/netip.go
@@ -1310,14 +1310,14 @@ func ParsePrefix(s string) (Prefix, error) {
bitsStr := s[i+1:]
bits, err := strconv.Atoi(bitsStr)
if err != nil {
- return Prefix{}, errors.New("netip.ParsePrefix(" + strconv.Quote(s) + ": bad bits after slash: " + strconv.Quote(bitsStr))
+ return Prefix{}, errors.New("netip.ParsePrefix(" + strconv.Quote(s) + "): bad bits after slash: " + strconv.Quote(bitsStr))
}
maxBits := 32
if ip.Is6() {
maxBits = 128
}
if bits < 0 || bits > maxBits {
- return Prefix{}, errors.New("netip.ParsePrefix(" + strconv.Quote(s) + ": prefix length out of range")
+ return Prefix{}, errors.New("netip.ParsePrefix(" + strconv.Quote(s) + "): prefix length out of range")
}
return PrefixFrom(ip, bits), nil
}
diff --git a/src/net/tcpsock_plan9.go b/src/net/tcpsock_plan9.go
index 768d03b06c..435335e92e 100644
--- a/src/net/tcpsock_plan9.go
+++ b/src/net/tcpsock_plan9.go
@@ -15,8 +15,11 @@ func (c *TCPConn) readFrom(r io.Reader) (int64, error) {
}
func (sd *sysDialer) dialTCP(ctx context.Context, laddr, raddr *TCPAddr) (*TCPConn, error) {
- if testHookDialTCP != nil {
- return testHookDialTCP(ctx, sd.network, laddr, raddr)
+ if h := sd.testHookDialTCP; h != nil {
+ return h(ctx, sd.network, laddr, raddr)
+ }
+ if h := testHookDialTCP; h != nil {
+ return h(ctx, sd.network, laddr, raddr)
}
return sd.doDialTCP(ctx, laddr, raddr)
}
diff --git a/src/net/tcpsock_posix.go b/src/net/tcpsock_posix.go
index bc3d324e6b..1c91170c50 100644
--- a/src/net/tcpsock_posix.go
+++ b/src/net/tcpsock_posix.go
@@ -55,8 +55,11 @@ func (c *TCPConn) readFrom(r io.Reader) (int64, error) {
}
func (sd *sysDialer) dialTCP(ctx context.Context, laddr, raddr *TCPAddr) (*TCPConn, error) {
- if testHookDialTCP != nil {
- return testHookDialTCP(ctx, sd.network, laddr, raddr)
+ if h := sd.testHookDialTCP; h != nil {
+ return h(ctx, sd.network, laddr, raddr)
+ }
+ if h := testHookDialTCP; h != nil {
+ return h(ctx, sd.network, laddr, raddr)
}
return sd.doDialTCP(ctx, laddr, raddr)
}
diff --git a/src/os/file.go b/src/os/file.go
index ab017d4af7..9f388921ae 100644
--- a/src/os/file.go
+++ b/src/os/file.go
@@ -620,8 +620,10 @@ func isWindowsNulName(name string) bool {
// operating system will begin with "/prefix": DirFS("/prefix").Open("file") is the
// same as os.Open("/prefix/file"). So if /prefix/file is a symbolic link pointing outside
// the /prefix tree, then using DirFS does not stop the access any more than using
-// os.Open does. DirFS is therefore not a general substitute for a chroot-style security
-// mechanism when the directory tree contains arbitrary content.
+// os.Open does. Additionally, the root of the fs.FS returned for a relative path,
+// DirFS("prefix"), will be affected by later calls to Chdir. DirFS is therefore not
+// a general substitute for a chroot-style security mechanism when the directory tree
+// contains arbitrary content.
//
// The result implements fs.StatFS.
func DirFS(dir string) fs.FS {
diff --git a/src/reflect/abi.go b/src/reflect/abi.go
index 9957d23768..32cb314188 100644
--- a/src/reflect/abi.go
+++ b/src/reflect/abi.go
@@ -237,7 +237,7 @@ func (a *abiSeq) regAssign(t *rtype, offset uintptr) bool {
st := (*structType)(unsafe.Pointer(t))
for i := range st.fields {
f := &st.fields[i]
- if !a.regAssign(f.typ, offset+f.offset()) {
+ if !a.regAssign(f.typ, offset+f.offset) {
return false
}
}
diff --git a/src/reflect/all_test.go b/src/reflect/all_test.go
index febbd5f5a7..56d91105a6 100644
--- a/src/reflect/all_test.go
+++ b/src/reflect/all_test.go
@@ -5891,6 +5891,87 @@ func TestStructOfDifferentPkgPath(t *testing.T) {
})
}
+func TestStructOfTooLarge(t *testing.T) {
+ t1 := TypeOf(byte(0))
+ t2 := TypeOf(int16(0))
+ t4 := TypeOf(int32(0))
+ t0 := ArrayOf(0, t1)
+
+ // 2^64-3 sized type (or 2^32-3 on 32-bit archs)
+ bigType := StructOf([]StructField{
+ {Name: "F1", Type: ArrayOf(int(^uintptr(0)>>1), t1)},
+ {Name: "F2", Type: ArrayOf(int(^uintptr(0)>>1-1), t1)},
+ })
+
+ type test struct {
+ shouldPanic bool
+ fields []StructField
+ }
+
+ tests := [...]test{
+ {
+ shouldPanic: false, // 2^64-1, ok
+ fields: []StructField{
+ {Name: "F1", Type: bigType},
+ {Name: "F2", Type: ArrayOf(2, t1)},
+ },
+ },
+ {
+ shouldPanic: true, // overflow in total size
+ fields: []StructField{
+ {Name: "F1", Type: bigType},
+ {Name: "F2", Type: ArrayOf(3, t1)},
+ },
+ },
+ {
+ shouldPanic: true, // overflow while aligning F2
+ fields: []StructField{
+ {Name: "F1", Type: bigType},
+ {Name: "F2", Type: t4},
+ },
+ },
+ {
+ shouldPanic: true, // overflow while adding trailing byte for zero-sized fields
+ fields: []StructField{
+ {Name: "F1", Type: bigType},
+ {Name: "F2", Type: ArrayOf(2, t1)},
+ {Name: "F3", Type: t0},
+ },
+ },
+ {
+ shouldPanic: true, // overflow while aligning total size
+ fields: []StructField{
+ {Name: "F1", Type: t2},
+ {Name: "F2", Type: bigType},
+ },
+ },
+ }
+
+ for i, tt := range tests {
+ func() {
+ defer func() {
+ err := recover()
+ if !tt.shouldPanic {
+ if err != nil {
+ t.Errorf("test %d should not panic, got %s", i, err)
+ }
+ return
+ }
+ if err == nil {
+ t.Errorf("test %d expected to panic", i)
+ return
+ }
+ s := fmt.Sprintf("%s", err)
+ if s != "reflect.StructOf: struct size would exceed virtual address space" {
+ t.Errorf("test %d wrong panic message: %s", i, s)
+ return
+ }
+ }()
+ _ = StructOf(tt.fields)
+ }()
+ }
+}
+
func TestChanOf(t *testing.T) {
// check construction and use of type not in binary
type T string
diff --git a/src/reflect/export_test.go b/src/reflect/export_test.go
index a5a3c1c271..f7d2cc362d 100644
--- a/src/reflect/export_test.go
+++ b/src/reflect/export_test.go
@@ -140,7 +140,7 @@ func IsExported(t Type) bool {
}
func ResolveReflectName(s string) {
- resolveReflectName(newName(s, "", false))
+ resolveReflectName(newName(s, "", false, false))
}
type Buffer struct {
diff --git a/src/reflect/type.go b/src/reflect/type.go
index 97040b5188..a52d3129df 100644
--- a/src/reflect/type.go
+++ b/src/reflect/type.go
@@ -433,17 +433,13 @@ type sliceType struct {
// Struct field
type structField struct {
- name name // name is always non-empty
- typ *rtype // type of field
- offsetEmbed uintptr // byte offset of field<<1 | isEmbedded
-}
-
-func (f *structField) offset() uintptr {
- return f.offsetEmbed >> 1
+ name name // name is always non-empty
+ typ *rtype // type of field
+ offset uintptr // byte offset of field
}
func (f *structField) embedded() bool {
- return f.offsetEmbed&1 != 0
+ return f.name.embedded()
}
// structType represents a struct type.
@@ -460,6 +456,7 @@ type structType struct {
// 1<<0 the name is exported
// 1<<1 tag data follows the name
// 1<<2 pkgPath nameOff follows the name and tag
+// 1<<3 the name is of an embedded (a.k.a. anonymous) field
//
// Following that, there is a varint-encoded length of the name,
// followed by the name itself.
@@ -496,6 +493,10 @@ func (n name) hasTag() bool {
return (*n.bytes)&(1<<1) != 0
}
+func (n name) embedded() bool {
+ return (*n.bytes)&(1<<3) != 0
+}
+
// readVarint parses a varint as encoded by encoding/binary.
// It returns the number of encoded bytes and the encoded value.
func (n name) readVarint(off int) (int, int) {
@@ -565,7 +566,7 @@ func (n name) pkgPath() string {
return pkgPathName.name()
}
-func newName(n, tag string, exported bool) name {
+func newName(n, tag string, exported, embedded bool) name {
if len(n) >= 1<<29 {
panic("reflect.nameFrom: name too long: " + n[:1024] + "...")
}
@@ -586,6 +587,9 @@ func newName(n, tag string, exported bool) name {
l += tagLenLen + len(tag)
bits |= 1 << 1
}
+ if embedded {
+ bits |= 1 << 3
+ }
b := make([]byte, l)
b[0] = bits
@@ -1256,7 +1260,7 @@ func (t *structType) Field(i int) (f StructField) {
if tag := p.name.tag(); tag != "" {
f.Tag = StructTag(tag)
}
- f.Offset = p.offset()
+ f.Offset = p.offset
// NOTE(rsc): This is the only allocation in the interface
// presented by a reflect.Type. It would be nice to avoid,
@@ -1472,7 +1476,7 @@ func (t *rtype) ptrTo() *rtype {
prototype := *(**ptrType)(unsafe.Pointer(&iptr))
pp := *prototype
- pp.str = resolveReflectName(newName(s, "", false))
+ pp.str = resolveReflectName(newName(s, "", false, false))
pp.ptrToThis = 0
// For the type structures linked into the binary, the
@@ -1739,7 +1743,10 @@ func haveIdenticalUnderlyingType(T, V *rtype, cmpTags bool) bool {
if cmpTags && tf.name.tag() != vf.name.tag() {
return false
}
- if tf.offsetEmbed != vf.offsetEmbed {
+ if tf.offset != vf.offset {
+ return false
+ }
+ if tf.embedded() != vf.embedded() {
return false
}
}
@@ -1891,7 +1898,7 @@ func ChanOf(dir ChanDir, t Type) Type {
ch := *prototype
ch.tflag = tflagRegularMemory
ch.dir = uintptr(dir)
- ch.str = resolveReflectName(newName(s, "", false))
+ ch.str = resolveReflectName(newName(s, "", false, false))
ch.hash = fnv1(typ.hash, 'c', byte(dir))
ch.elem = typ
@@ -1934,7 +1941,7 @@ func MapOf(key, elem Type) Type {
// in ../cmd/compile/internal/reflectdata/reflect.go:writeType.
var imap any = (map[unsafe.Pointer]unsafe.Pointer)(nil)
mt := **(**mapType)(unsafe.Pointer(&imap))
- mt.str = resolveReflectName(newName(s, "", false))
+ mt.str = resolveReflectName(newName(s, "", false, false))
mt.tflag = 0
mt.hash = fnv1(etyp.hash, 'm', byte(ktyp.hash>>24), byte(ktyp.hash>>16), byte(ktyp.hash>>8), byte(ktyp.hash))
mt.key = ktyp
@@ -2113,7 +2120,7 @@ func FuncOf(in, out []Type, variadic bool) Type {
}
// Populate the remaining fields of ft and store in cache.
- ft.str = resolveReflectName(newName(str, "", false))
+ ft.str = resolveReflectName(newName(str, "", false, false))
ft.ptrToThis = 0
return addToCache(&ft.rtype)
}
@@ -2225,7 +2232,7 @@ func hashMightPanic(t *rtype) bool {
}
}
-// Make sure these routines stay in sync with ../../runtime/map.go!
+// Make sure these routines stay in sync with ../runtime/map.go!
// These types exist only for GC, so we only fill out GC relevant info.
// Currently, that's just size and the GC program. We also fill in string
// for possible debugging use.
@@ -2290,7 +2297,7 @@ func bucketOf(ktyp, etyp *rtype) *rtype {
gcdata: gcdata,
}
s := "bucket(" + ktyp.String() + "," + etyp.String() + ")"
- b.str = resolveReflectName(newName(s, "", false))
+ b.str = resolveReflectName(newName(s, "", false, false))
return b
}
@@ -2369,7 +2376,7 @@ func SliceOf(t Type) Type {
prototype := *(**sliceType)(unsafe.Pointer(&islice))
slice := *prototype
slice.tflag = 0
- slice.str = resolveReflectName(newName(s, "", false))
+ slice.str = resolveReflectName(newName(s, "", false, false))
slice.hash = fnv1(typ.hash, '[')
slice.elem = typ
slice.ptrToThis = 0
@@ -2628,11 +2635,17 @@ func StructOf(fields []StructField) Type {
comparable = comparable && (ft.equal != nil)
offset := align(size, uintptr(ft.align))
+ if offset < size {
+ panic("reflect.StructOf: struct size would exceed virtual address space")
+ }
if ft.align > typalign {
typalign = ft.align
}
size = offset + ft.size
- f.offsetEmbed |= offset << 1
+ if size < offset {
+ panic("reflect.StructOf: struct size would exceed virtual address space")
+ }
+ f.offset = offset
if ft.size == 0 {
lastzero = size
@@ -2648,6 +2661,9 @@ func StructOf(fields []StructField) Type {
// zero-sized field can't manufacture a pointer to the
// next object in the heap. See issue 9401.
size++
+ if size == 0 {
+ panic("reflect.StructOf: struct size would exceed virtual address space")
+ }
}
var typ *structType
@@ -2690,7 +2706,11 @@ func StructOf(fields []StructField) Type {
str := string(repr)
// Round the size up to be a multiple of the alignment.
- size = align(size, uintptr(typalign))
+ s := align(size, uintptr(typalign))
+ if s < size {
+ panic("reflect.StructOf: struct size would exceed virtual address space")
+ }
+ size = s
// Make the struct type.
var istruct any = struct{}{}
@@ -2698,7 +2718,7 @@ func StructOf(fields []StructField) Type {
*typ = *prototype
typ.fields = fs
if pkgpath != "" {
- typ.pkgPath = newName(pkgpath, "", false)
+ typ.pkgPath = newName(pkgpath, "", false, false)
}
// Look in cache.
@@ -2742,7 +2762,7 @@ func StructOf(fields []StructField) Type {
}
}
- typ.str = resolveReflectName(newName(str, "", false))
+ typ.str = resolveReflectName(newName(str, "", false, false))
typ.tflag = 0 // TODO: set tflagRegularMemory
typ.hash = hash
typ.size = size
@@ -2774,14 +2794,14 @@ func StructOf(fields []StructField) Type {
continue
}
// Pad to start of this field with zeros.
- if ft.offset() > off {
- n := (ft.offset() - off) / goarch.PtrSize
+ if ft.offset > off {
+ n := (ft.offset - off) / goarch.PtrSize
prog = append(prog, 0x01, 0x00) // emit a 0 bit
if n > 1 {
prog = append(prog, 0x81) // repeat previous bit
prog = appendVarint(prog, n-1) // n-1 times
}
- off = ft.offset()
+ off = ft.offset
}
prog = appendGCProg(prog, ft.typ)
@@ -2803,8 +2823,8 @@ func StructOf(fields []StructField) Type {
if comparable {
typ.equal = func(p, q unsafe.Pointer) bool {
for _, ft := range typ.fields {
- pi := add(p, ft.offset(), "&x.field safe")
- qi := add(q, ft.offset(), "&x.field safe")
+ pi := add(p, ft.offset, "&x.field safe")
+ qi := add(q, ft.offset, "&x.field safe")
if !ft.typ.equal(pi, qi) {
return false
}
@@ -2841,16 +2861,11 @@ func runtimeStructField(field StructField) (structField, string) {
}
}
- offsetEmbed := uintptr(0)
- if field.Anonymous {
- offsetEmbed |= 1
- }
-
resolveReflectType(field.Type.common()) // install in runtime
f := structField{
- name: newName(field.Name, string(field.Tag), field.IsExported()),
- typ: field.Type.common(),
- offsetEmbed: offsetEmbed,
+ name: newName(field.Name, string(field.Tag), field.IsExported(), field.Anonymous),
+ typ: field.Type.common(),
+ offset: 0,
}
return f, field.PkgPath
}
@@ -2874,7 +2889,7 @@ func typeptrdata(t *rtype) uintptr {
return 0
}
f := st.fields[field]
- return f.offset() + f.typ.ptrdata
+ return f.offset + f.typ.ptrdata
default:
panic("reflect.typeptrdata: unexpected type, " + t.String())
@@ -2917,7 +2932,7 @@ func ArrayOf(length int, elem Type) Type {
prototype := *(**arrayType)(unsafe.Pointer(&iarray))
array := *prototype
array.tflag = typ.tflag & tflagRegularMemory
- array.str = resolveReflectName(newName(s, "", false))
+ array.str = resolveReflectName(newName(s, "", false, false))
array.hash = fnv1(typ.hash, '[')
for n := uint32(length); n > 0; n >>= 8 {
array.hash = fnv1(array.hash, byte(n))
@@ -3097,7 +3112,7 @@ func funcLayout(t *funcType, rcvr *rtype) (frametype *rtype, framePool *sync.Poo
} else {
s = "funcargs(" + t.String() + ")"
}
- x.str = resolveReflectName(newName(s, "", false))
+ x.str = resolveReflectName(newName(s, "", false, false))
// cache result for future callers
framePool = &sync.Pool{New: func() any {
@@ -3165,7 +3180,7 @@ func addTypeBits(bv *bitVector, offset uintptr, t *rtype) {
tt := (*structType)(unsafe.Pointer(t))
for i := range tt.fields {
f := &tt.fields[i]
- addTypeBits(bv, offset+f.offset(), f.typ)
+ addTypeBits(bv, offset+f.offset, f.typ)
}
}
}
diff --git a/src/reflect/value.go b/src/reflect/value.go
index 5abdca2820..74554a3ac8 100644
--- a/src/reflect/value.go
+++ b/src/reflect/value.go
@@ -1287,7 +1287,7 @@ func (v Value) Field(i int) Value {
// In the former case, we want v.ptr + offset.
// In the latter case, we must have field.offset = 0,
// so v.ptr + field.offset is still the correct address.
- ptr := add(v.ptr, field.offset(), "same as non-reflect &v.field")
+ ptr := add(v.ptr, field.offset, "same as non-reflect &v.field")
return Value{typ, ptr, fl}
}
diff --git a/src/regexp/backtrack.go b/src/regexp/backtrack.go
index 41ae59bcaa..0739f5ff58 100644
--- a/src/regexp/backtrack.go
+++ b/src/regexp/backtrack.go
@@ -163,7 +163,7 @@ func (re *Regexp) tryBacktrack(b *bitState, i input, pc uint32, pos int) bool {
}
Skip:
- inst := re.prog.Inst[pc]
+ inst := &re.prog.Inst[pc]
switch inst.Op {
default:
diff --git a/src/regexp/exec.go b/src/regexp/exec.go
index 4411e4c3e6..3fc4b684fe 100644
--- a/src/regexp/exec.go
+++ b/src/regexp/exec.go
@@ -427,7 +427,7 @@ func (re *Regexp) doOnePass(ir io.RuneReader, ib []byte, is string, pos, ncap in
flag = i.context(pos)
}
pc := re.onepass.Start
- inst := re.onepass.Inst[pc]
+ inst := &re.onepass.Inst[pc]
// If there is a simple literal prefix, skip over it.
if pos == 0 && flag.match(syntax.EmptyOp(inst.Arg)) &&
len(re.prefix) > 0 && i.canCheckPrefix() {
@@ -442,7 +442,7 @@ func (re *Regexp) doOnePass(ir io.RuneReader, ib []byte, is string, pos, ncap in
pc = int(re.prefixEnd)
}
for {
- inst = re.onepass.Inst[pc]
+ inst = &re.onepass.Inst[pc]
pc = int(inst.Out)
switch inst.Op {
default:
@@ -470,7 +470,7 @@ func (re *Regexp) doOnePass(ir io.RuneReader, ib []byte, is string, pos, ncap in
}
// peek at the input rune to see which branch of the Alt to take
case syntax.InstAlt, syntax.InstAltMatch:
- pc = int(onePassNext(&inst, r))
+ pc = int(onePassNext(inst, r))
continue
case syntax.InstFail:
goto Return
diff --git a/src/runtime/HACKING.md b/src/runtime/HACKING.md
index af9fe288aa..61b5a51959 100644
--- a/src/runtime/HACKING.md
+++ b/src/runtime/HACKING.md
@@ -41,8 +41,20 @@ All `g`, `m`, and `p` objects are heap allocated, but are never freed,
so their memory remains type stable. As a result, the runtime can
avoid write barriers in the depths of the scheduler.
-User stacks and system stacks
------------------------------
+`getg()` and `getg().m.curg`
+----------------------------
+
+To get the current user `g`, use `getg().m.curg`.
+
+`getg()` alone returns the current `g`, but when executing on the
+system or signal stacks, this will return the current M's "g0" or
+"gsignal", respectively. This is usually not what you want.
+
+To determine if you're running on the user stack or the system stack,
+use `getg() == getg().m.curg`.
+
+Stacks
+======
Every non-dead G has a *user stack* associated with it, which is what
user Go code executes on. User stacks start small (e.g., 2K) and grow
@@ -63,17 +75,33 @@ non-preemptible and the garbage collector does not scan system stacks.
While running on the system stack, the current user stack is not used
for execution.
-`getg()` and `getg().m.curg`
-----------------------------
+nosplit functions
+-----------------
-To get the current user `g`, use `getg().m.curg`.
+Most functions start with a prologue that inspects the stack pointer
+and the current G's stack bound and calls `morestack` if the stack
+needs to grow.
-`getg()` alone returns the current `g`, but when executing on the
-system or signal stacks, this will return the current M's "g0" or
-"gsignal", respectively. This is usually not what you want.
+Functions can be marked `//go:nosplit` (or `NOSPLIT` in assembly) to
+indicate that they should not get this prologue. This has several
+uses:
-To determine if you're running on the user stack or the system stack,
-use `getg() == getg().m.curg`.
+- Functions that must run on the user stack, but must not call into
+ stack growth, for example because this would cause a deadlock, or
+ because they have untyped words on the stack.
+
+- Functions that must not be preempted on entry.
+
+- Functions that may run without a valid G. For example, functions
+ that run in early runtime start-up, or that may be entered from C
+ code such as cgo callbacks or the signal handler.
+
+Splittable functions ensure there's some amount of space on the stack
+for nosplit functions to run in and the linker checks that any static
+chain of nosplit function calls cannot exceed this bound.
+
+Any function with a `//go:nosplit` annotation should explain why it is
+nosplit in its documentation comment.
Error handling and reporting
============================
diff --git a/src/runtime/alg.go b/src/runtime/alg.go
index 5d7d1c77f4..2a413eeef3 100644
--- a/src/runtime/alg.go
+++ b/src/runtime/alg.go
@@ -182,7 +182,7 @@ func typehash(t *_type, p unsafe.Pointer, h uintptr) uintptr {
if f.name.isBlank() {
continue
}
- h = typehash(f.typ, add(p, f.offset()), h)
+ h = typehash(f.typ, add(p, f.offset), h)
}
return h
default:
diff --git a/src/runtime/asm_ppc64x.s b/src/runtime/asm_ppc64x.s
index ec17dc9ce3..c6bcf82ec0 100644
--- a/src/runtime/asm_ppc64x.s
+++ b/src/runtime/asm_ppc64x.s
@@ -106,7 +106,7 @@ DATA runtime·mainPC+0(SB)/8,$runtime·main