- Build cmd with CGO_ENABLED=0. Doing so removes the C compiler
toolchain from the reproducibility perimeter and also results in
cmd/go and cmd/pprof binaries that are statically linked,
so that they will run on a wider variety of systems.
In particular the Linux versions will run on Alpine and NixOS
without needing a simulation of libc.so.6.
The potential downside of disabling cgo is that cmd/go and cmd/pprof
use the pure Go network resolver instead of the host resolver on
Unix systems. This means they will not be able to use non-DNS
resolver mechanisms that may be specified in /etc/resolv.conf,
such as mDNS. Neither program seems likely to need non-DNS names
like those, however.
macOS and Windows systems still use the host resolver, which they
access without cgo.
- Build cmd with -trimpath when building a release.
Doing so removes $GOPATH from the file name prefixes stored in the
binary, so that the build directory does not leak into the final artifacts.
- When CC and CXX are empty, do not pick values to hard-code into
the source tree and binaries. Instead, emit code that makes the
right decision at runtime. In addition to reproducibility, this
makes cross-compiled toolchains work better. A macOS toolchain
cross-compiled on Linux will now correctly look for clang,
instead of looking for gcc because it was built on Linux.
- Convert \ to / in file names stored in .a files.
These are converted to / in the final binaries, but the hashes of
the .a files affect the final build ID of the binaries. Without this
change, builds of a Windows toolchain on Windows and non-Windows
machines produce identical binaries except for the input hash part
of the build ID.
- Due to the conversion of \ to / in .a files, convert back when
reading inline bodies on Windows to preserve output file names
in error messages.
Combined, these four changes (along with Go 1.20's removal of
installed pkg/**.a files and conversion of macOS net away from cgo)
make the output of make.bash fully reproducible, even when
cross-compiling: a released macOS toolchain built on Linux or Windows
will contain exactly the same bits as a released macOS toolchain
built on macOS.
The word "released" in the previous sentence is important.
For the build IDs in the binaries to work out the same on
both systems, a VERSION file must exist to provide a consistent
compiler build ID (instead of using a content hash of the binary).
For #24904.
Fixes#57007.
Change-Id: I665e1ef4ff207d6ff469452347dca5bfc81050e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/454836
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
We have a make.bash-time generation capability,
so use it to generate the embedded zip file for time/tzdata.
This is one less file to try to review in CLs like CL 455356.
For #22487.
Fixes#43350.
Change-Id: I2fcd0665fa0b1c830baec5fb4cd714483fea25a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/455357
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
These environment variables affect cmd/dist, and in turn run.bash.
They exist primarily for the Go build system, but are still needed
sometimes when investigating problems. Document them in one place.
Fixes#46054.
Change-Id: I5dea2ac79b0d203d2f3c9ec2980382f62bead5cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/455517
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
pkg/obj will be empty, so just remove it.
pkg/$GOOS_$GOARCH will be empty unless the user has specified
GODEBUG=installgoroot=all, so check if it's empty, and if so, delete
it.
Also remove xreaddirfiles, which is unused.
Also remove the copy of pkg/$GOOS_$GOARCH in the cmd/go test
TestNewReleaseRebuildsStalePackagesInGOPATH. The directory is empty so
copying it has no effect.
For #47257
Change-Id: Ief90b882d157bd16078cd5d2b83a915bfc831f9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/453496
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
No one ever runs 'go tool api', because the invocation
has gotten unwieldy enough that it's not practical.
And we don't support it as a standalone tool for other
packages - it's not even in the distribution.
Making it an ordinary package test lets us invoke it
more easily from cmd/dist (as go test cmd/api -check)
and avoids the increasingly baroque code in run.go to
build a command line.
Left in cmd/api even though it's no longer a command
because (1) it uses a package from cmd/vendor and
(2) it uses internal/testenv. Otherwise it could be misc/api.
Fixes#56845.
Change-Id: I00a13d9c19b1e259fa0e6bb93d1a4dca25f0e8c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/453258
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Revise the code added in CL 452763 that skips some of the -race tests
on older windows builders. The old-style skip was doing a log.Printf,
which wound up being interpreted in "-list" mode. Fix is to pass in a
special rtPreFunc when registering the test (thanks Austin for the
fix suggestion).
Updates #56904.
Change-Id: Ia6ea31fb7f011b539173f47357ab3bf7389f256d
Reviewed-on: https://go-review.googlesource.com/c/go/+/452769
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Than McIntosh <thanm@google.com>
Add a skip for the external-linkage part of the race detector test for
elderly versions of windows, until the underlying cause for the problem
can be determined.
Updates #56904.
Change-Id: I3e8650ff66f34efefabcd6bc343d57124539901b
Reviewed-on: https://go-review.googlesource.com/c/go/+/452763
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
Auto-Submit: Than McIntosh <thanm@google.com>
The gcc toolchain on the aix-ppc64 builder apparently does not achieve
reproducible builds for packages that use cgo, which causes the
binaries in cmd that use package "net" (cmd/go, cmd/pprof, and
cmd/trace) to appear stale whenever the Go build cache is cleared.
For now, we work around the staleness by rebuilding std and simply not
checking whether cmd is stale.
For #56896.
Updates #47257.
Change-Id: I15f86e72dee53904b881710d5d5d613872361510
Reviewed-on: https://go-review.googlesource.com/c/go/+/452680
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ayappan Perumal <ayappanec@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
If GOMAXPROCS=1, force serial execution, which is better for
debugging build problems and also minimizes footprint, if that
happens to matter.
This wasn't good when the bootstrap was 1.4 because there
default GOMAXPROCS=1, but that is no longer the bootstrap
version.
Change-Id: I637e25c8acb4758795fceef63921eda359a7be29
Reviewed-on: https://go-review.googlesource.com/c/go/+/452556
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Now that we don't install the .a files, the installed
runtime/internal/sys.a no longer exists. Stop trying to access it.
Change-Id: Ic437e7ad95be6109d0b2fcfb81e840f5cb476be8
Reviewed-on: https://go-review.googlesource.com/c/go/+/451360
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Replace explicit Go version names where possible with generic reference
to Go bootstrap version.
Updates #44505
Change-Id: I4a6439576efd40e72acd26fcc1472a1a8b0b06e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/344330
Auto-Submit: Martin Möhrmann <martin@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Change-Id: I69065f8adf101fdb28682c55997f503013a50e29
Reviewed-on: https://go-review.googlesource.com/c/go/+/449757
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joedian Reid <joedian@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
In -race mode, the dist test command only registers the std, race,
osusergo, and amd64ios tests before returning early from
(*tester).registerTests. Prior to CL 450018, the osusergo and amd64ios
tests weren't even affected by -race mode, so it seems their inclusion
was unintentional. CL 450018 lifted the logic to run tests in race
mode, which means these tests went from running without -race to
running with -race. Unfortunately, amd64ios is not compatible with
-race, so it is now failing on the darwin-amd64-race builder.
Fix this by omitting the osusergo and amd64ios tests from -race mode,
since it seems like they were really intended to be included anyway.
This should fix the darwin-amd64-race builder.
Updates #37486.
Change-Id: I554bb60bc729dbb6f1bc926f1ea329768b0d6d81
Reviewed-on: https://go-review.googlesource.com/c/go/+/451437
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Now that all uses of "go test" have been converted over to the new
abstraction, we can delete the old helpers for building "go test"
commands and simplify some code that's only used by the new
abstraction now.
For #37486.
Change-Id: I770cd457e018160d694abcc0b6ac80f7dc2e8425
Reviewed-on: https://go-review.googlesource.com/c/go/+/450020
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
This adds support for host tests to goTest and registerTest and
modifies all uses of registerHostTest to use goTest and registerTest.
This eliminates the last case where go test command lines are
constructed by hand. Next we'll clean up all of the infrastructure
support for that.
I traced all exec calls from cmd/dist on linux/amd64 and this makes
only no-op changes (such as re-arranging the order of flags).
Preparation for #37486.
Change-Id: Icb7ec8efdac72bdb819ae24b2f585375d9d9d5b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/450019
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This converts most of the remaining manual "go test" command line
construction in cmd/dist to use the goTest abstraction and
registerTest.
At this point, the only remaining place that directly constructs go
test command lines is runHostTest.
This fixes a bug in the "nolibgcc:os/user" test. It was clearly
supposed to pass "-run=^Test[^CS]", but the logic to override the
"-run" flag for "nolibgcc:net" caused "nolibgcc:os/user" to pass
*both* "-run=^Test[^CS]" and "-run=". This was then rewritten into
just "-run=" by flattenCmdline, which caused all os/user tests to run,
and not actually skip the expensive tests as intended. (This is a
great example of why the new abstraction is much more robust than
command line construction.)
I traced all exec calls from cmd/dist on linux/amd64 and, other than
the fix to nolibgcc:os/user, this makes only no-op changes (such as
re-arranging the order of flags).
For #37486.
Change-Id: Ie8546bacc56640ea39f2804a87795c14a3fe4c7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/450018
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Currently, dist test has a single test called "cgo_test" that runs a
large number of different "go test"s.
This commit restructures cgo_test into several individual tests, each
of which runs a single "go test" that can be described by a goTest
object and registered with registerTest. Since this lets us raise the
abstraction level of constructing these tests and these tests are
mostly covering the Cartesian product of a small number of orthogonal
dimensions, we pull the common logic for constructing these tests into
a helper function.
For consistency, we now pass -tags=static to the static testtls and
nocgo tests, but this tag doesn't affect the build of these tests at
all. I traced all exec calls from cmd/dist on linux/amd64 and this is
the only non-trivial change.
For #37486.
Change-Id: I53c1efa1c38d785dc71968f05e8d7d636b553e96
Reviewed-on: https://go-review.googlesource.com/c/go/+/450017
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
The overall goal is to make registerTest the primary entry point for
adding dist tests and to convert nearly all dist tests to be
represented by a goTest, registered via registerTest. This will
centralize the logic for creating dist tests corresponding to go tool
tests.
I traced all exec calls from cmd/dist on linux/amd64 and this makes
only no-op changes (such as re-arranging the order of flags).
For #37486.
Change-Id: I4749e6f3666134d3259b54ee6055d76a4235c60c
Reviewed-on: https://go-review.googlesource.com/c/go/+/450016
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This CL rewrites everywhere in dist that manually constructs an
exec.Cmd to run "go test" to use the goTest abstraction. All remaining
invocations of "go test" after this CL construct the command line
manually, but ultimately use addCmd to execute it.
I traced all exec calls from cmd/dist on linux/amd64 and this makes
only no-op changes (such as re-arranging the order of flags).
For #37486.
Change-Id: Idc7497e39bac04def7ddaf2010881c9623e76fd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/450015
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Austin Clements <austin@google.com>
This introduces an abstraction for constructing and running "go test"
commands. Currently, dist test is basically a shell script written in
Go syntax: it mostly just invokes lots of subprocesses, almost all of
which are "go test" invocations, and it constructs those command lines
directly from strings all over the place.
This CL raises the level of abstraction of invoking go test. The
current level of abstraction is not serving us very well: it's
conveniently terse, but the actual logic for constructing a command
line is typically so spread out that it's difficult to predict what
command will actually run. For example, the `gotest` function
constructs the basic command, but many tests want to override at least
some of these flags, so flattenCmdLine has logic specific to `go test`
for eliminating duplicate flags that `go test` itself would reject. At
the same time, the logic for constructing many common flags is
conditional, leading to a bevy of helpers for constructing flags like
`-short` and `-timeout` and `-run` that are scattered throughout
test.go and very easy to forget to call.
This CL centralizes and flattens all of this knowledge into a new
`goTest` type. This type gives dist a single, unified point where we
can change anything about how it invokes "go test".
There's currently some "unnecessary" abstraction in the implementation
of the goTest type to separate "build" and "run" flags. This will
become important later when we convert host tests and to do separate
build and run steps.
The following CLs will convert dist test to use this type rather than
directly constructing "go test" command lines. Finally, we'll strip
out the scattered helper logic for building command lines.
For #37486.
Change-Id: I9f1633fe6c0921696419ce8127ed2ca7b7a4e01b
Reviewed-on: https://go-review.googlesource.com/c/go/+/448802
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Currently, cmd/go's testterminal18153 is implemented as a special test
that doesn't run as part of cmd/go's regular tests. Because the test
requires stdout and stderr to be a terminal, it is currently run
directly by "dist test" so it can inherit the terminal of all.bash.
This has a few problems. It's yet another special case in dist test.
dist test also has to be careful to not apply its own buffering to
this test, so it can't run in parallel and it limits dist test's own
scheduler. It doesn't run as part of regular "go test", which means it
usually only gets coverage from running all.bash. And since we have to
skip it if all.bash wasn't run at a terminal, I'm sure it often gets
skipped even when running all.bash.
Fix all of this by rewriting this test to create its own PTY and
re-exec "go test" to check that PTY passes through go test. This makes
the test self-contained, so it can be a regular cmd/go test, we can
drop it from dist test, and it's not sensitive to the environment of
all.bash.
Preparation for #37486.
Updates #18153.
Change-Id: I6493dbb0143348e299718f6e311ac8a63f5d69c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/449503
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The registerTest function has a special case for commands that start
with "time", but we don't use this case anywhere. Delete this special
case and its support code.
Preparation for #37486.
Change-Id: Ica180417e7aa4e4fc260cb97467942bae972fdb6
Reviewed-on: https://go-review.googlesource.com/c/go/+/448801
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This is unused, and eliminating it lets us simplify the whole
registerTest mechanism.
Preparation for #37486.
Change-Id: Ia6221e48192cd17775a5d662bdb389d67a9265bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/448800
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
I keep having to reconstruct how dist test's command system works.
This CL documents the main functions to help with this.
Change-Id: Ic1102496d3c7f356e73f3fa2ea76498251f525d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/448799
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
The cmd/dist cgo_test enumerates a large number of platforms in
various special cases. Some combinations are suspiciously absent. This
CL completes the combinations.
I've confirmed using trybots that the newly-enabled tests pass on
android/* (this is not surprising because the gohostos is never
"android" anyway), windows/arm64, linux/ppc64 (no cgo), linux/loong64
(except for one test, filed #56623), linux/mips*, netbsd/arm (except
for one test, filed #56629), and netbsd/arm64. The windows/arm builder
is out to lunch, so I'm assuming that works. Since netbsd/arm and
arm64 mostly passed these tests, I've also enabled them on netbsd/386
and netbsd/amd64, where they seem to work fine as well.
Preparation for #37486.
Change-Id: I04c3348e4f422d74d51e714647ca3db379e6e919
Reviewed-on: https://go-review.googlesource.com/c/go/+/448016
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This can be used to provide better instruction selection for assembly
implementations without having to implement two variants and dynamic
runtime selections when a newer GOPPC64 value is used.
Change-Id: I4331037d57b128137280aa7904d08d362391f81e
Reviewed-on: https://go-review.googlesource.com/c/go/+/449115
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Now that misc/cgo/test doesn't invoke any Go builds itself, we don't
need to use GOFLAGS to thread build flags down into sub-builds.
Preparation for #37486.
Change-Id: I9ecdd215da5c21339cbf76765f8be2190d19539c
Reviewed-on: https://go-review.googlesource.com/c/go/+/447356
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
We reused p so we were deleting the same directory twice instead of two
different directories. Fix that.
For #47257
Change-Id: I315ad87d0a9182e00ae4c11b82986227e2b02e17
Reviewed-on: https://go-review.googlesource.com/c/go/+/447115
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
The go1 benchmark suite does a lot of work at package init time, which
makes it take quite a while to run even if you're not running any of
the benchmarks, or if you're only running a subset of them. This leads
to an awkward workaround in dist test to compile but not run the
package, unlike roughly all other packages. It also reduces isolation
between benchmarks by affecting the starting heap size of all
benchmarks.
Fix this by initializing all data required by a benchmark when that
benchmark runs, and keeping it local so it gets freed by the GC and
doesn't leak between benchmarks. Now, none of the benchmarks depend on
global state.
Re-initializing the data on each benchmark run does add overhead to an
actual benchmark run, as each benchmark function is called several
times with different values of b.N. A full run of all benchmarks at
the default -benchtime=1s now takes ~10% longer; higher -benchtimes
would be less. It would be quite difficult to cache this data between
invocations of the same benchmark function without leaking between
different benchmarks and affecting GC overheads, as the testing
package doesn't provide any mechanism for this.
This reduces the time to run the binary with no benchmarks from 1.5
seconds to 10 ms, and also reduces the memory required to do this from
342 MiB to 17 MiB.
To make sure data was not leaking between different benchmarks, I ran
the benchmarks with -shuffle=on. The variance remained low: mostly
under 3%. A few benchmarks had higher variance, but in all cases it
was similar to the variance between this change.
This CL naturally changes the measured performance of several of the
benchmarks because it dramatically changes the heap size and hence GC
overheads. However, going forward the benchmarks should be much better
isolated.
For #37486.
Change-Id: I252ebea703a9560706cc1990dc5ad22d1927c7a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/443336
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Packages in GOROOT that don't use cgo will not be installed in
GOROOT/pkg, and will instead be cached as usual like other Go
packages.
- add a internal/buildinternal package to hold the identities of the
five packages that use cgo
- update dist's test code to do a go build std cmd before checking
staleness on builders. Because most of those packages no longer have
install locations, and have dependencies that don't either, the
packages need to be cached to not be stale.
- fix index_test to import packages with the path "." when preparing
the "want" values to compare the indexed data to. (the module index
matches the behavior of build.ImportDir, which always passes in "."
as the path.
- In both the index and go/build Importers, don't set
PkgObj for GOROOT packages which will no longer have install
targets. PkgTargetRoot will still be set to compute target paths,
which will still be needed in buildmode=shared.
- "downgrade" all install actions that don't have a target to build
actions. (The target should already not be set for packages that
shouldn't be installed).
For #47257
Change-Id: Ia5aee6b3b20b58e028119cf0352a4c4a2f10f6b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/432535
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Before this change, the .a files for the intermediate go toolchains
were produced in the same location as the install target. This change
has them produced in a temporary location instead. This change,
combined with go install not producing .a files for most stdlib
packages by default results in a build of the distribution only
producing .a's for the five packages in std that require cgo. (Before
this change, the .a's for the final build were not being produced, but
stale ones from the intermediate builds were left behind.)
For #47257
Change-Id: I91b826cd1ce9aad9c492fb865e36d34dc8bb188e
Reviewed-on: https://go-review.googlesource.com/c/go/+/436135
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
go/build and cmd/go will stop returing Targets for stdlib .a files, and
stop producing the .a files is pkg/GOOS_GOARCH. update tests to
anticipate that and to pass in importcfgs instead of expecting the
compiler can find .a files in their old locations.
Adds code to determine locations of .a files to internal/goroot. Also
adds internal/goroot to dist's bootstrap directories and changes
internal/goroot to build with a bootstrap version of Go.
Change-Id: Ie81e51105bddb3f0e374cbf47e81c23edfb67fa5
Reviewed-on: https://go-review.googlesource.com/c/go/+/442303
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Previously we used Go 1.17, but we realized thanks to tickling
a pre-Go1.17.3 bug that if we are going to change the bootstrap
toolchain that we should default to the latest available point release
at the time we make the switch, not the initial major release, so as
to avoid bugs that were fixed in the point releases.
This CL updates the default search locations and the release notes.
Users who run make.bash and depend on finding $HOME/sdk/go1.17
may need to run
go install golang.org/dl/go1.17.13@latest
go1.17.13 download
to provide a Go 1.17.13 toolchain to their builds.
Change-Id: I3a2511f088cf852470a7216a5a41ae775fb561b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/439419
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Currently, the entry-point to this test is a Bash script that smoke
tests the FORTRAN compiler and then runs a FORTRAN-containing Go test.
This CL rearranges things so a pure Go Go test smoke tests the FORTRAN
compiler and then runs a non-test FORTRAN-containing Go binary.
While we're here, we fix a discrepancy when the host is GOARCH=amd64,
but the target is GOARCH=386. Previously, we would pick the wrong
libgfortran path because we didn't account for the cross-compilation,
causing the link to fail. Except for some reason this was ignored and
the test nevertheless "passed". In the new test we're a little more
strict, so this build failure will cause the test to fail, so we add a
little logic to account for cross-compilation with the host toolchain.
For #37486.
Change-Id: Ie6f70066885d6fbb4e1b5a2b1e13b85dee5b359b
Reviewed-on: https://go-review.googlesource.com/c/go/+/443069
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
This migrates testsigfwd, which uses some one-off build
infrastructure, to be part of the runtime's testprogcgo.
The test is largely unchanged. Because it's part of a larger binary,
this CL renames a few things and gates the constructor-time signal
handler registration on an environment variable. This CL also replaces
an errant fmt.Errorf with fmt.Fprintf.
For #37486, since it eliminates a non-go-test from dist.
Change-Id: I0efd146ea0a0a3f0b361431349a419af0f0ecc61
Reviewed-on: https://go-review.googlesource.com/c/go/+/443068
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The testasan test was added back in 2013 (CL 10126044), many years
before Go added ASAN support in 2021 (CL 298611). So, in fact,
testasan does not test Go ASAN support at all, as you might expect
(misc/cgo/testsanitizers does that). It's intended to test whether the
Go memory allocator works in a mixed C/Go binary where the C code is
compiled with ASAN. The test doesn't actually use ASAN in any way; it
just simulates where ASAN of 2013 put its shadow mappings. This made
sense to test at the time because Go was picky about where its heap
landed and ASAN happened to put its mappings exactly where Go wanted
to put its heap. These days, Go is totally flexible about its heap
placement, and I wouldn't be surprised if ASAN also works differently.
Given all of this, this test adds almost no value today. Drop it.
For #37486, since it eliminates a non-go-test from dist.
Change-Id: I0292f8efbdc0e1e39650715604535c445fbaa87f
Reviewed-on: https://go-review.googlesource.com/c/go/+/443067
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
We haven't used this in a while and it's going to complicate later
changes to dist, so drop support. This was primarily for supporting
slow QEMU-based builders, but an alternative and simpler way to do
that if we need to in the future is to supply a go_exec wrapper to run
tests in QEMU, like we do for other emulated platforms.
Simplification for #37486.
Change-Id: Idc0383f59c61d8546ea3b4d2eede4acdaf30d9b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/431256
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Adjust os/arch checks to enable msan tests on freebsd/amd64.
R=go1.20
For #53298
Change-Id: I3d0f5259db73d526d006a12de5ba6284528cf819
Reviewed-on: https://go-review.googlesource.com/c/go/+/411276
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Separate out the functions from cmd/internal/sys/support.go and
migrate them to a new package internal/platform, so that functions such as
"RaceDetectorSupported" can be called from tests in std as well as in
cmd. This isn't a complete move of everything in cmd/internal/sys;
there are still many functions left.
The original version of this CL (patch set 1) called the new package
"internal/sys", but for packages that needed both "internal/sys" and
"cmd/internal/sys" the import of the former had to be done with a
different name, which was confusing and also required a hack in
cmd/dist.
Updates #56006.
Change-Id: I866d62e75adbf3a640a06e2c7386a6e9e2a18d91
Reviewed-on: https://go-review.googlesource.com/c/go/+/438475
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
The flag is now removed from `go build` and `go test`.
It has been deprecated since Go 1.16, printing a warning message.
The idea was to fully delete it in Go 1.17, but that didn't happen.
First, delete the BuildI variable and its flag declarations,
as well as all the bits of docs that mentioned the flag.
Second, delete or simplify the code paths that used BuildI.
Third, adapt the tests to the removed flag.
Some of them are removed, like test_relative_import_dash_i.txt and
TestGoTestDashIDashOWritesBinary, as they centered around the flag.
The rest are modified to not cover or use the flag.
Finally, change cmd/dist to no longer use `go install -i`.
The purpose of the flag was that, when bootstrapping the toolchain,
all of its dependencies would also be installed as object files.
When removing the use of the -i flags, the checkNotStale call right
after building toolchain3 would fail as expected,
because runtime/internal/sys is now only up to date in the build cache.
Luckily, that's not a problem: we run `go install std cmd` right after,
so all standard library packages will be installed as object files.
Move the checkNotStale call after that install command.
Fixes#41696.
Change-Id: I5d8139f18aaee07da886d483e663f3f2f00d5f3a
Reviewed-on: https://go-review.googlesource.com/c/go/+/416094
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Updates #53466
Change-Id: I6643b4254dc707351d397018cee485bb508dde94
Reviewed-on: https://go-review.googlesource.com/c/go/+/431659
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Dmitri Goutnik <dgoutnik@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Change-Id: Ia0896bd1edf2558821244fecd1c297b599472f47
GitHub-Last-Rev: cfd1e1091a
GitHub-Pull-Request: golang/go#55944
Reviewed-on: https://go-review.googlesource.com/c/go/+/436637
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Most tests will be skipped anyway because 'go build' is not available,
but this ensures cmd will be build tested by TryBots for js/wasm.
For #25911
For #35220
For #54219
Change-Id: I09c75905c36311810eb1ae75eeee2fa6102c1c0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/435237
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Updates #53466
Change-Id: I17a41adb93c22fa84a7bdcec9d8c65001c5a8ba9
Reviewed-on: https://go-review.googlesource.com/c/go/+/431657
Reviewed-by: Dmitri Goutnik <dgoutnik@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
1. replace [0-9] with \d in regexps
2. replace [a-zA-Z0-9_] with \w in regexps
Change-Id: I9e260538252a0c1071e76aeb1c5f885c6843a431
GitHub-Last-Rev: 286e1a4619
GitHub-Pull-Request: golang/go#54874
Reviewed-on: https://go-review.googlesource.com/c/go/+/428435
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Adds a -coveragecfg=<configfile> command line option to the compiler
to help support a cooperative "tool and compiler" mode for coverage
instrumentation. In this mode the cmd/cover tool generates most of the
counter instrumentation via source-to-source rewriting, but the
compiler fixes up the result if passed the "-coveragecfg" option. The
fixups include:
- reclassifying counter variables (special storage class)
- marking meta-data variables are read-only
- adding in an init call to do registation
Updates #51430.
Change-Id: Iead72b85209725ee044542374465f118a3ee72e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/395895
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Russ Cox
Dmitri Shuralyov
Michael Matloob
Than McIntosh
Bryan C. Mills
David Chase
Cherry Mui
Martin Möhrmann
cui fliter
Austin Clements
Paul E. Murphy
Raj Barik
Dmitri Goutnik
Robert Griesemer
eric fang
Daniel Martí
Mikael Urankar
Tobias Klauser
TomCao New Macbook Pro