Today the linker keeps all methods of reachable types. This is
necessary if a program uses reflect.Value.Call. But while use of
reflection is widespread in Go for encoders and decoders, using
it to call a method is rare.
This CL looks for the use of reflect.Value.Call in a program, and
if it is absent, adopts a (reasonably conservative) method pruning
strategy as part of dead code elimination. Any method that is
directly called is kept, and any method that matches a used
interface's method signature is kept.
Whether or not a method body is kept is determined by the relocation
from its receiver's *rtype to its *rtype. A small change in the
compiler marks these relocations as R_METHOD so they can be easily
collected and manipulated by the linker.
As a bonus, this technique removes the text segment of methods that
have been inlined. Looking at the output of building cmd/objdump with
-ldflags=-v=2 shows that inlined methods like
runtime.(*traceAllocBlockPtr).ptr are removed from the program.
Relatively little work is necessary to do this. Linking two
examples, jujud and cmd/objdump show no more than +2% link time.
Binaries that do not use reflect.Call.Value drop 4 - 20% in size:
addr2line: -793KB (18%)
asm: -346KB (8%)
cgo: -490KB (10%)
compile: -564KB (4%)
dist: -736KB (17%)
fix: -404KB (12%)
link: -328KB (7%)
nm: -827KB (19%)
objdump: -712KB (16%)
pack: -327KB (14%)
yacc: -350KB (10%)
Binaries that do use reflect.Call.Value see a modest size decrease
of 2 - 6% thanks to pruning of unexported methods:
api: -151KB (3%)
cover: -222KB (4%)
doc: -106KB (2.5%)
pprof: -314KB (3%)
trace: -357KB (4%)
vet: -187KB (2.7%)
jujud: -4.4MB (5.8%)
cmd/go: -384KB (3.4%)
The trivial Hello example program goes from 2MB to 1.68MB:
package main
import "fmt"
func main() {
fmt.Println("Hello, 世界")
}
Method pruning also helps when building small binaries with
"-ldflags=-s -w". The above program goes from 1.43MB to 1.2MB.
Unfortunately the linker can only tell if reflect.Value.Call has been
statically linked, not if it is dynamically used. And while use is
rare, it is linked into a very common standard library package,
text/template. The result is programs like cmd/go, which don't use
reflect.Value.Call, see limited benefit from this CL. If binary size
is important enough it may be possible to address this in future work.
For #6853.
Change-Id: Iabe90e210e813b08c3f8fd605f841f0458973396
Reviewed-on: https://go-review.googlesource.com/20483
Reviewed-by: Russ Cox <rsc@golang.org>
Drops cmd/binary size from 14.41 MiB to 11.42 MiB.
Before:
text data bss dec hex filename
8121210 3521696 737960 12380866 bceac2 ../pkg/tool/linux_amd64/compile
bradfitz@dev-bradfitz-debian2:~/go/src$ ls -l ../pkg/tool/linux_amd64/compile
-rwxr-xr-x 1 bradfitz bradfitz 15111272 Mar 8 23:32 ../pkg/tool/linux_amd64/compile
a2afc0 51312 R html.statictmp_0085
6753f0 56592 T cmd/internal/obj/x86.doasm
625480 58080 T cmd/compile/internal/gc.typecheck1
f34c40 65688 D runtime.trace
be0a20 133552 D cmd/compile/internal/ppc64.varianttable
c013e0 265856 D cmd/compile/internal/arm.progtable
c42260 417280 D cmd/compile/internal/amd64.progtable
ca8060 417280 D cmd/compile/internal/x86.progtable
f44ce0 500640 D cmd/internal/obj/arm64.oprange
d0de60 534208 D cmd/compile/internal/ppc64.progtable
d90520 667520 D cmd/compile/internal/arm64.progtable
e334a0 790368 D cmd/compile/internal/mips64.progtable
a3e8c0 1579362 r runtime.pclntab
After:
text data bss dec hex filename
8128226 375954 246432 8750612 858614 ../pkg/tool/linux_amd64/compile
-rwxr-xr-x 1 bradfitz bradfitz 11971432 Mar 8 23:35 ../pkg/tool/linux_amd64/compile
6436d0 43936 T cmd/compile/internal/gc.walkexpr
c13ca0 45056 D cmd/compile/internal/ssa.opcodeTable
5d8ea0 50256 T cmd/compile/internal/gc.(*state).expr
818c50 50448 T cmd/compile/internal/ssa.rewriteValueAMD64_OpMove
a2d0e0 51312 R html.statictmp_0085
6753d0 56592 T cmd/internal/obj/x86.doasm
625460 58080 T cmd/compile/internal/gc.typecheck1
c38fe0 65688 D runtime.trace
a409e0 1578810 r runtime.pclntab
Fixes#14703
Change-Id: I2177596d5c7fd67db0a3c423cd90801cf52adb12
Reviewed-on: https://go-review.googlesource.com/20450
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Deleting the string merging pass makes the linker 30-35% faster
but makes jujud (using the github.com/davecheney/benchjuju snapshot) 2.5% larger.
Two optimizations bring the space overhead down to 0.6%.
First, change the default alignment for string data to 1 byte.
(It was previously defaulting to larger amounts, usually pointer width.)
Second, write out the type string for T (usually a bigger expression) as "*T"[1:],
so that the type strings for T and *T share storage.
Combined, these obtain the bulk of the benefit of string merging
at essentially no cost. The remaining benefit from string merging
is not worth the excessive cost, so delete it.
As penance for making the jujud binary 0.6% larger,
the next CL in this sequence trims the reflect functype
information enough to make the jujud binary overall 0.75% smaller
(that is, that CL has a net -1.35% effect).
For #6853.
Fixes#14648.
Change-Id: I3fdd74c85410930c36bb66160ca4174ed540fc6e
Reviewed-on: https://go-review.googlesource.com/20334
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Currently, package obj reserves a range of 1<<12 opcodes for each
target architecture. E.g., mips64 has [6<<12, 7<<12).
However, because mips.ABEQ and mips.ALAST are both within that range,
the expression mips.ABEQ+mips.ALAST in turn falls (far) outside that
range around 12<<12, meaning it could theoretically collide with
another arch's opcodes.
More practically, it's a problem because 12<<12 overflows an int16,
which hampers fixing #14692. (We could also just switch to uint16 to
avoid the overflow, but that still leaves the first problem.)
As a workaround, use Michael Hudson-Doyle's solution from
https://golang.org/cl/20182 and use negative values for these variant
instructions.
Passes toolstash -cmp for GOARCH=arm and GOARCH=mips64.
Updates #14692.
Change-Id: Iad797d10652360109fa4db19d4d1edb6529fc2c0
Reviewed-on: https://go-review.googlesource.com/20345
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix some test output while we're here.
Change-Id: I265cedc222e078eff120f268b92451e12b0400b2
Reviewed-on: https://go-review.googlesource.com/20294
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
No immediate reduction in the size of Addr.
Passes toolstash -cmp.
Change-Id: I78ea4c6e181b6e571ce70a5f1ae8158844eb197d
Reviewed-on: https://go-review.googlesource.com/20276
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Add tests to ensure that the size of important types don't change
unexpectedly.
Skip the test on nacl platforms because of their unusual padding
requirements.
Change-Id: Iddb127a99499e089a309b721f5073356c0da8b24
Reviewed-on: https://go-review.googlesource.com/20285
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Dave Cheney <dave@cheney.net>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is mostly changing the opXXX helpers to take an int16 (matching Prog.As)
argument and return a uint32. The only bit that's not completely trivial is
passing -p.As to opirr to signal operating on a shifted constant, because AADD
+ ALAST overflows int16.
Change-Id: I69133800bbe41c38fa4a89bbbf49823043b3419c
Reviewed-on: https://go-review.googlesource.com/20182
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a subset of https://golang.org/cl/20022 with only the copyright
header lines, so the next CL will be smaller and more reviewable.
Go policy has been single space after periods in comments for some time.
The copyright header template at:
https://golang.org/doc/contribute.html#copyright
also uses a single space.
Make them all consistent.
Change-Id: Icc26c6b8495c3820da6b171ca96a74701b4a01b0
Reviewed-on: https://go-review.googlesource.com/20111
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The Go toolchain stopped creating them before Go 1.3, so no point in
worrying about them today.
History:
- Git commit 250a091 added cmd/ar, which wrote Plan 9 __.SYMDEF
entries into archive files.
- golang.org/cl/6500117 renamed __.SYMDEF to __.GOSYMDEF. (Notably,
the commit message suggests users need to use Go nm to read symbols,
but even back then the toolchain did nothing with __.(GO)?SYMDEF files
except skip over them.)
- golang.org/cl/42880043 added the -pack flag to cmd/gc to directly
produce archives by the Go compiler, and did not write __.GOSYMDEF
entries.
- golang.org/cl/52310044 rewrote cmd/pack in Go, and removed support
for producing __.GOSYMDEF entries.
Change-Id: I255edf40d0d3690e3447e488039fcdef73c6d6b1
Reviewed-on: https://go-review.googlesource.com/19924
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We can't drop Prog entries when we want to print disassembly.
Added a test for -S.
Fixes#14515
Change-Id: I44c72f70f7a3919acc01c559d30335d26669e76f
Reviewed-on: https://go-review.googlesource.com/19930
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Our stack frame sizes look pretty good now. Lower the stack
guard from 1024 to 720.
Tip is currently using 720.
We could go lower (to 640 at least) except PPC doesn't like that.
Change-Id: Ie5f96c0e822435638223f1e8a2bd1a1eed68e6aa
Reviewed-on: https://go-review.googlesource.com/19922
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
I can't remember just how this happened to me, but I got an unfortunate
crash with some set of cmd/compile debug options and source code.
Change-Id: Ibef6129c50b68dad0594ac439466bfbc4b32a095
Reviewed-on: https://go-review.googlesource.com/19920
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Simplifies some code as ptrToThis was unreliable under dynamic
linking. Now the same type lookup is used regardless of execution
mode.
A synthetic relocation, R_USETYPE, is introduced to make sure the
linker includes *T on use of T, if *T is carrying methods.
Changes the heap dump format. Anything reading the format needs to
look at the last bool of a type of an interface value to determine
if the type should be the pointer-to type.
Reduces binary size of cmd/go by 0.2%.
For #6853.
Change-Id: I79fcb19a97402bdb0193f3c7f6d94ddf061ee7b2
Reviewed-on: https://go-review.googlesource.com/19695
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Only tests do this, provide them a hook to disable freeing
after flush.
Change-Id: I810c6c51414a93f476a18ba07b807e16092bf8cf
Reviewed-on: https://go-review.googlesource.com/19907
Reviewed-by: Keith Randall <khr@golang.org>
Don't accumulate a massive list of Prog structs during
compilation and write them all out at the end of compilation.
Instead, convert them to code+relocs (or data+relocs) after each
function is compiled.
Track down a few other places that were keeping Progs alive
and nil them out so the Progs get GCd promptly.
Saves ~20% in peak memory usage for the compiler. Surprisingly not much
help speed-wise (only because we end up doing more GCs. With a
compensating GOGC=120, it does help a bit), but this provides a base for
more changes (e.g. reusing a cache of Progs).
Change-Id: I838e01017c228995a687a8110d0cd67bf8596407
Reviewed-on: https://go-review.googlesource.com/19867
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When -N, make sure we don't drop every instruction from
a block, even ones which would otherwise be empty.
Helps keep line numbers around for debugging, particularly
for break and continue statements (which often compile
down to nothing).
Fixes#14379
Change-Id: I33722c4f0dcd502f146fa48af262ba3a477c959a
Reviewed-on: https://go-review.googlesource.com/19854
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Change-Id: I38e59088c37426d914ce2b4dfc79f3d476e06f49
Reviewed-on: https://go-review.googlesource.com/19617
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
These have no accepted input syntax and,
as far as I can tell, do not actually exist.
Change-Id: Iafdfb71adccad76230191d922eb7ddf78b7d5898
Reviewed-on: https://go-review.googlesource.com/19612
Reviewed-by: Ian Lance Taylor <iant@golang.org>
3DNotAnymore!
These only ever existed on AMD (not Intel) processors,
and AMD cancelled support for them in August 2010.
Change-Id: Ia362259add9d4f5788fd151fb373f91288677407
Reviewed-on: https://go-review.googlesource.com/19611
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Semi-regular merge from tip to dev.ssa.
Two fixes:
1) Mark selectgo as not returning. This caused problems
because there are no VARKILL ops on the selectgo path,
causing things to be marked live that shouldn't be.
2) Tell the amd64 assembler that addressing modes like
name(SP)(AX*4) are ok.
Change-Id: I9ca81c76391b1a65cc47edc8610c70ff1a621913
And update two imports in cmd/internal/objfile/disasm.go.
This makes GO15VENDOREXPERIMENT=0 ./make.bash work.
For Go 1.7 we will move it back.
Fixes#14236.
Change-Id: I429c9af4baff8496f83d113b1b03b90e309f4f48
Reviewed-on: https://go-review.googlesource.com/19384
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
LEAQ symbol+100(SB), AX
Under dynamic link, rewrites to
MOVQ symbol@GOT(SB), AX
ADDQ $100, AX
but ADDQ clobbers flags, whereas the original LEAQ (when not dynamic
linking) doesn't.
Use LEAQ instead of ADDQ to add that constant in so we preserve flags.
Change-Id: Ibb055403d94a4c5163e1c7d2f45da633ffd0b6a3
Reviewed-on: https://go-review.googlesource.com/19230
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It's causing the darwin-386 builder to fail with:
--- FAIL: TestDynlink (0.07s)
obj6_test.go:118: error exit status 3 output go tool: no such tool "asm"
FAIL
FAIL cmd/internal/obj/x86 0.073s
So skip it for now. It's tested in enough other places.
Change-Id: I9a98ad7b8be807005750112d892ac6c676c17dd5
Reviewed-on: https://go-review.googlesource.com/18989
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
The current code delays the literal pool until the very last moment,
but based on the assumption that span-dependent jumps are as
short as possible. If they need to be enlarged in a later round, that
very last moment may be too late. Flush a little early to prevent that.
Fixes#13579.
Change-Id: I759b5db5c43a977bf2b940872870cbbc436ad141
Reviewed-on: https://go-review.googlesource.com/18972
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Run-TryBot: Russ Cox <rsc@golang.org>
The x86 backend automatically rewrites MOV $0, AX to
XOR AX, AX. That rewrite isn't ok when the flags register
is live across the MOV. Keep track of which moves care
about preserving flags, then disable this rewrite for them.
On x86, Prog.Mark was being used to hold the length of the
instruction. We already store that in Prog.Isize, so no
need to store it in Prog.Mark also. This frees up Prog.Mark
to hold a bitmask on x86 just like all the other architectures.
Update #12405
Change-Id: Ibad8a8f41fc6222bec1e4904221887d3cc3ca029
Reviewed-on: https://go-review.googlesource.com/18861
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Was part of #13822 but not in the first message, so I missed it.
Fixes#13822 again.
Change-Id: I775004fa8d47b6af293124605521ec396573e267
Reviewed-on: https://go-review.googlesource.com/18900
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Add test for assembly errors, to verify fix.
Make sure invalid instruction errors are printed just once
(was printing them once per span iteration, so typically twice).
Fixes#13282.
Change-Id: Id5f66f80a80b3bc4832e00084b0a91f1afec7f8f
Reviewed-on: https://go-review.googlesource.com/18858
Reviewed-by: Rob Pike <r@golang.org>
Add amd64 instructions I promised to add for Go 1.6
at the beginning of January.
These may be the last instructions added by hand.
I intend to generate the whole set mechanically for Go 1.7.
Fixes#13822.
Change-Id: I8c6bae2efd25f717f9ec750402e50f408a911d2b
Reviewed-on: https://go-review.googlesource.com/18853
Reviewed-by: Rob Pike <r@golang.org>
Use the standard names, for discoverability.
Use the standard register arguments, for correctness.
Implement all possible arguments, for completeness.
Enable the corresponding tests now that everything is standard.
Update the uses in package runtime.
Fixes#14068.
Change-Id: I8e1af9a41e7d02d98c2a82af3d4cdb3e9204824f
Reviewed-on: https://go-review.googlesource.com/18852
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Not much testing yet, but the test now exists.
Another step toward #13822.
Change-Id: Idb2b06bf53a6113c83008150b4c0b631bb195279
Reviewed-on: https://go-review.googlesource.com/18844
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Tests for this and many other instructions are in a separate followup CL.
For #14068.
Change-Id: I6955315996a34d7fb79369b9d9a0119d11745e85
Reviewed-on: https://go-review.googlesource.com/18849
Reviewed-by: Rob Pike <r@golang.org>
Ilya added POPCNT in a CL earlier this month but it's really only POPCNTQ.
The other forms still need to be added.
For #4816.
Change-Id: I1186850d32ad6d5777475c7808e6fc9d9133e118
Reviewed-on: https://go-review.googlesource.com/18848
Reviewed-by: Rob Pike <r@golang.org>
Not recognized in any instructions yet, but this lets the
assembler parse them at least.
For #14068.
Change-Id: Id4f7329a969b747a867ce261b20165fab2cdcab8
Reviewed-on: https://go-review.googlesource.com/18846
Reviewed-by: Rob Pike <r@golang.org>
Also, remove output file if there are encoding errors.
The extra reports are convenient.
Removing the output file is very important.
Noticed while testing.
Change-Id: I0fab17d4078f93c5a0d6d1217d8d9a63ac789696
Reviewed-on: https://go-review.googlesource.com/18845
Reviewed-by: Rob Pike <r@golang.org>
Instead of two parallel files that look almost identical,
mark the expected differences in the original file.
The annotations being added here keep the tests passing,
but they also make clear a number of printing or parsing
errors that were not as easily seen when the data was
split across two files.
Fix a few diagnostic problems in cmd/internal/obj as well.
A step toward #13822.
Change-Id: I997172681ea6fa7da915ff0f0ab93d2b76f8dce2
Reviewed-on: https://go-review.googlesource.com/18823
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
David Crawshaw
Brad Fitzpatrick
Matthew Dempsky
Russ Cox
Josh Bleecher Snyder
Dave Cheney
Michael Hudson-Doyle
Keith Randall
Ian Lance Taylor
Martin Möhrmann
Shawn Smith
Shenghou Ma