The -p flag specifies the import path of the package being compiled.
This CL makes it required when invoking the compiler and
adjusts tests that invoke the compiler directly to conform to this
new requirement. The go command already passes the flag, so it
is unmodified in this CL. It is expected that any other Go build systems
also already pass -p, or else they will need to arrange to do so before
updating to Go 1.19. Of particular note, Bazel already does for rules
with an importpath= attribute, which includes all Gazelle-generated rules.
There is more cleanup possible now in cmd/compile, cmd/link,
and other consumers of Go object files, but that is left to future CLs.
Additional historical background follows but can be ignored.
Long ago, before the go command, or modules, or any kind of
versioning, symbols in Go archive files were named using just the
package name, so that for example func F in math/rand and func F in
crypto/rand would both be the object file symbol 'rand.F'. This led to
collisions even in small source trees, which made certain packages
unusable in the presence of other packages and generally was a problem
for Go's goal of scaling to very large source trees.
Fixing this problem required changing from package names to import
paths in symbol names, which was mostly straightforward. One wrinkle,
though, is that the compiler did not know the import path of the
package being compiled; it only knew the package name. At the time,
there was no go command, just Makefiles that people had invoking 6g
(now “go tool compile”) and then copying the resulting object file to
an importable location. That is, everyone had a custom build setup for
Go, because there was no standard one. So it was not particularly
attractive to change how the compiler was invoked, since that would
break approximately every Go user at the time. Instead, we arranged
for the compiler to emit, and other tools reading object files to
recognize, a special import path (the empty string, it turned out)
denoting “the import path of this object file”. This worked well
enough at the time and maintained complete command-line compatibility
with existing Go usage.
The changes implementing this transition can be found by searching
the Git history for “package global name space”, which is what they
eliminated. In particular, CL 190076 (a6736fa4), CL 186263 (758f2bc5),
CL 193080 (1cecac81), CL 194053 (19126320), and CL 194071 (531e6b77)
did the bulk of this transformation in January 2010.
Later, in September 2011, we added the -p flag to the compiler for
diagnostic purposes. The problem was that it was easy to create import
cycles, especially in tests, and these could not be diagnosed until
link time. You'd really want the compiler to diagnose these, for
example if the compilation of package sort noticed it was importing a
package that itself imported "sort". But the compilation of package
sort didn't know its own import path, and so it could not tell whether
it had found itself as a transitive dependency. Adding the -p flag
solved this problem, and its use was optional, since the linker would
still diagnose the import cycle in builds that had not updated to
start passing -p. This was CL 4972057 (1e480cd1).
There was still no go command at this point, but when we introduced
the go command we made it pass -p, which it has for many years at this
point.
Over time, parts of the compiler began to depend on the presence of
the -p flag for various reasonable purposes. For example:
In CL 6497074 (041fc8bf; Oct 2012), the race detector used -p to
detect packages that should not have race annotations, such as
runtime/race and sync/atomic.
In CL 13367052 (7276c02b; Sep 2013), a bug fix used -p to detect the
compilation of package reflect.
In CL 30539 (8aadcc55; Oct 2016), the compiler started using -p to
identify package math, to be able to intrinsify calls to Sqrt inside
that package.
In CL 61019 (9daee931; Sep 2017), CL 71430 (2c1d2e06; Oct 2017), and
later related CLs, the compiler started using the -p value when
creating various DWARF debugging information.
In CL 174657 (cc5eaf93; May 2019), the compiler started writing
symbols without the magic empty string whenever -p was used, to reduce
the amount of work required in the linker.
In CL 179861 (dde7c770; Jun 2019), the compiler made the second
argument to //go:linkname optional when -p is used, because in that
case the compiler can derive an appropriate default.
There are more examples. Today it is impossible to compile the Go
standard library without using -p, and DWARF debug information is
incomplete without using -p.
All known Go build systems pass -p. In particular, the go command
does, which is what nearly all Go developers invoke to build Go code.
And Bazel does, for go_library rules that set the importpath
attribute, which is all rules generated by Gazelle.
Gccgo has an equivalent of -p and has required its use in order to
disambiguate packages with the same name but different import paths
since 2010.
On top of all this, various parts of code generation for generics
are made more complicated by needing to cope with the case where -p
is not specified, even though it's essentially always specified.
In summary, the current state is:
- Use of the -p flag with cmd/compile is required for building
the standard library, and for complete DWARF information,
and to enable certain linker speedups.
- The go command and Bazel, which we expect account for just
about 100% of Go builds, both invoke cmd/compile with -p.
- The code in cmd/compile to support builds without -p is
complex and has become more complex with generics, but it is
almost always dead code and therefore not worth maintaining.
- Gccgo already requires its equivalent of -p in any build
where two packages have the same name.
All this supports the change in this CL, which makes -p required
and adjusts tests that invoke cmd/compile to add -p appropriately.
Future CLs will be able to remove all the code dealing with the
possibility of -p not having been specified.
Change-Id: I6b95b9d4cffe59c7bac82eb273ef6c4a67bb0e43
Reviewed-on: https://go-review.googlesource.com/c/go/+/391014
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.
Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.
A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.
Fixes#49884.
Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Go 1.15 pack's r command creates the output file if it does not
exist. The system "ar" command does this as well. Do the same.
For bazelbuild/rules_go#2762.
Change-Id: Icd88396b5c714b735c859a29ab29851e4301f4d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/279516
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Treat the compiler's -linkobj output as "compiler object, which
means "pack c" will "see through" the file and add individual
entry to the new archive, instead of the object as a whole.
This is somewhat peculiar. But Go 1.15's cmd/pack does this,
although seemingly accidental. We just do the same. FWIW, it
does make things more consistent with/without -linkobj flag.
Fixes#43271.
Change-Id: I6b2d99256db7ebf0fa430f85afa7464e334f6bcb
Reviewed-on: https://go-review.googlesource.com/c/go/+/279483
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)
Update the Go tree to use the preferred names.
As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.
ReadDir changes are in a separate CL, because they are not a
simple search and replace.
For #42026.
Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The old os references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
For #41190.
Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/243907
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Rewrite part of cmd/pack to use the cmd/internal/archive package.
Change-Id: Ia7688810d3ea4d0277056870091f59cf09cffcad
Reviewed-on: https://go-review.googlesource.com/c/go/+/247917
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Shorten some of the longest tests that run during all.bash.
Removes 7r 50u 21s from all.bash.
After this change, all.bash is under 5 minutes again on my laptop.
For #26473.
Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79
Reviewed-on: https://go-review.googlesource.com/c/go/+/177559
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
By always writing out pack files, the object file format can be
simplified somewhat. In particular, the export data format will no
longer require escaping, because the pack file provides appropriate
framing.
This CL does not affect build systems that use -pack, which includes
all major Go build systems (cmd/go, gb, bazel).
Also, existing package import logic already distinguishes pack/object
files based on file contents rather than file extension.
The only exception is cmd/pack, which specially handled object files
created by cmd/compile when used with the 'c' mode. This mode is
extended to now recognize the pack files produced by cmd/compile and
handle them as before.
Passes toolstash-check.
Updates #21705.
Updates #24512.
Change-Id: Idf131013bfebd73a5cde7e087eb19964503a9422
Reviewed-on: https://go-review.googlesource.com/102236
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The binary export data format includes escaping to prevent "\n$$" from
appearing internally, but not "\n!\n". This could result in a false
positive when cmd/pack searched for "\n!\n" as the delimiter between
package definition and linker object.
To address this, this CL changes cmd/pack to also be aware of the
"\n$$" markers, and to ignore "\n!\n" within the export data.
Fixes#21703.
Change-Id: I71ea8ba49dbd066c7afb7717ddc0190e38fe5649
Reviewed-on: https://go-review.googlesource.com/60773
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change makes sure that tests are run with the correct
version of the go tool. The correct version is the one that
we invoked with "go test", not the one that is first in our path.
Fixes#16577
Change-Id: If22c8f8c3ec9e7c35d094362873819f2fbb8559b
Reviewed-on: https://go-review.googlesource.com/28089
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL/19862 (f79b50b8d5) recently introduced the constants
SeekStart, SeekCurrent, and SeekEnd to the io package. We should use these constants
consistently throughout the code base.
Updates #15269
Change-Id: If7fcaca7676e4a51f588528f5ced28220d9639a2
Reviewed-on: https://go-review.googlesource.com/22097
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <joetsai@digital-static.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL removes the remaining visible uses of the "architecture letter" concept.
(They are no longer in tool names nor in source directory names.)
Because the architecture letter concept is now gone, delete GOCHAR
from "go env" output, and change go/build.ArchChar to return an
error always.
The architecture letter is still used in the compiler and linker sources
as a clumsy architecture enumeration, but that use is not visible to
Go users and can be cleaned up separately.
Change-Id: I4d97a38f372003fb610c9c5241bea440d9dbeb8d
Reviewed-on: https://go-review.googlesource.com/10289
Reviewed-by: Rob Pike <r@golang.org>
This CL fixes the build to use the newly created go tool compile
and go tool link in place of go tool 5g, go tool 5l, and so on.
See golang-dev thread titled "go tool compile, etc" for background.
Although it was not a primary motivation, this conversion does
reduce the wall clock time and cpu time required for make.bash
by about 10%.
Change-Id: I79cbbdb676cab029db8aeefb99a53178ff55f98d
Reviewed-on: https://go-review.googlesource.com/10288
Reviewed-by: Rob Pike <r@golang.org>
Just like darwin/arm.
Change-Id: I5ed26975670d4189a46b585a56c66c199905d168
Reviewed-on: https://go-review.googlesource.com/8823
Reviewed-by: Minux Ma <minux@golang.org>
Use log.Fatalf for formatting directives instead of log.Fatal
Change-Id: Ia207b320f5795c63cdfa71f92c19ca6d05cc833f
Reviewed-on: https://go-review.googlesource.com/2160
Reviewed-by: Minux Ma <minux@golang.org>
There were at least two bugs:
1) It would overwrite a non-archive.
2) It would truncate a non-archive and then fail.
In general the file handling was too clever to be correct.
Make it more straightforward, doing the creation
separately from archive management.
Fixes#8369.
LGTM=adg, iant
R=golang-codereviews, adg, iant
CC=golang-codereviews
https://golang.org/cl/147010046
Add nacl.bash, the NaCl version of all.bash.
It's a separate script because it builds a variant of package syscall
with a large zip file embedded in it, containing all the input files
needed for tests.
Disable various tests new since the last round, mostly the ones using os/exec.
Fixes#7945.
LGTM=dave
R=golang-codereviews, remyoudompheng, dave, bradfitz
CC=golang-codereviews
https://golang.org/cl/100590044
TestLargeDefs was issuing over one million small writes to
create a 7MB file (large.go). This is quite slow on Plan 9
since our disk file systems aren't very fast and they're
usually accessed over the network.
Buffering the writes makes the test about six times faster.
Even on Linux, it's about 1.5 times faster.
Here are the results on a slow Plan 9 machine:
Before:
% ./pack.test -test.v -test.run TestLargeDefs
=== RUN TestLargeDefs
--- PASS: TestLargeDefs (125.11 seconds)
PASS
After:
% ./pack.test -test.v -test.run TestLargeDefs
=== RUN TestLargeDefs
--- PASS: TestLargeDefs (20.835 seconds)
PASS
LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/95040044
It is possible to use ./ imports on Windows but it
requires some extra command-line work
('go build' does this automatically, but we can't use 'go build' here).
Instead, use an ordinary import and -I/-L, which are easier to use.
LGTM=iant
R=iant
CC=golang-codereviews
https://golang.org/cl/89040043
My cmd/go got in a weird state where it started invoking pack grcP.
Change pack to print a 1-line explanation of the usage problem
before the generic usage message.
LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/87770047
Match used len(ar.files) == 0 to mean "match everything"
but it also deleted matched things from the list, so once you
had matched everything you asked for, match returned true
for whatever was left in the archive too.
Concretely, if you have an archive containing f1, f2, then
pack t foo.a f1
would match f1 and then, because len(ar.files) == 0 after
deleting f1 from the match list, also match f2.
Avoid the problem by recording explicitly whether match
matches everything.
LGTM=r, dsymonds
R=r, dsymonds
CC=golang-codereviews
https://golang.org/cl/65630046
Get more information to help understand build failure on Plan 9.
Also Windows.
(TestHello is failing because GOCHAR does not appear in output.
What does?)
Update #7362
LGTM=bradfitz
R=rsc, bradfitz
CC=golang-codereviews
https://golang.org/cl/66070044
When Go 1.3 is released, this will keep existing
Go 1.2 build scripts that use 'go tool pack grc' working.
For efficiency, such scripts should be changed to
use 6g -pack instead, but keeping the old behavior
available enables a more graceful transition.
LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/66130043
Replace the pack command, a C program, with a clean reimplementation in Go.
It does not need to reproduce the full feature set and it is no longer used by
the build chain, but has a role in looking inside archives created by the build
chain directly.
Since it's not in C, it is no longer build by dist, so remove it from cmd/dist and
make it a "tool" in cmd/go terminology.
Fixes#2705
R=rsc, dave, minux.ma, josharian
CC=golang-codereviews
https://golang.org/cl/52310044
These no longer work; removing them makes other refactoring easier.
The code for pack P being deleted in this CL does not work either.
I created issue 6989 to track restoring this functionality (probably not
until pack is written in Go).
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/44300043
If you thought gcc -ansi -pedantic was pedantic, just wait
until you meet clang -fsanitize=undefined.
I think this addresses all the reported "errors", but we'll
need another run to be sure.
all.bash still passes.
Update #5764
Dave, can you please try again?
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/13334049
Also introduce BGET2/4, BPUT2/4 as they are widely used.
Slightly improve BGETC/BPUTC implementation.
This gives ~5% CPU time improvement on go install -a -p1 std.
Before:
real user sys
0m23.561s 0m16.625s 0m5.848s
0m23.766s 0m16.624s 0m5.846s
0m23.742s 0m16.621s 0m5.868s
after:
0m22.999s 0m15.841s 0m5.889s
0m22.845s 0m15.808s 0m5.850s
0m22.889s 0m15.832s 0m5.848s
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/12745047
Also:
- faster code for example extraction
- simplify handling of command documentation:
all "main" packages are treated as commands
- various minor cleanups along the way
For commands written in Go, any doc.go file containing
documentation must now be part of package main (rather
then package documentation), otherwise the documentation
won't show up in godoc (it will still build, though).
For commands written in C, documentation may still be
in doc.go files defining package documentation, but the
recommended way is to explicitly ignore those files with
a +build ignore constraint to define package main.
Fixes#4806.
R=adg, rsc, dave, bradfitz
CC=golang-dev
https://golang.org/cl/7333046
This fixes a problem with ELF tools thinking they know the
format of the symbol table, as we do not use any of the
standard formats for that table.
This change will probably annoy the Plan 9 users, but I
believe there are other incompatibilities already that mean
they have to use a Go-specific nm.
Fixes#3473.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/6500117
Russ Cox
Tobias Klauser
Cherry Zhang
Daniel Martí
Matthew Dempsky
Keith Randall
Joe Tsai
Hyang-Ah Hana Kim
David Crawshaw
Rob Pike
Brad Fitzpatrick
Emil Hessman
David Crawshaw
Anthony Martin
Robert Hencke
Ian Lance Taylor
Alex Brainman
Dave Cheney
Dmitriy Vyukov
Robert Griesemer
Shenghou Ma