Commit Graph

400 Commits

Author SHA1 Message Date
Than McIntosh 85d62a91ee [dev.link] cmd/link: fix up ctxt.Textp2 handling in AssignTextSymbolOrder
Change the loader method AssignTextSymbolOrder to return a slice of
all reachable textp symbols, since it will be needed in second-phase
DWARF gen.

Change-Id: Iaf16ee9cf0d5266aeb0d3df596e8117263b35d8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/220985
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-05 16:20:42 +00:00
Jeremy Faller 7d8aef689d [dev.link] cmd/link: remove *Link from relocsym
This is the last step requried before relocsym can go parallel.

Change-Id: Id1c1c530c2b9277917208c3767eeb29e02c17a9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/220841
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-04 20:36:34 +00:00
Jeremy Faller 4ab7ab54b0 [dev.link] cmd/link: remove *Link from architecture relocers
Also correct a few places I hadn't switched over to Target from *Link
context.

Change-Id: I6de6b39723762de3fa5f8d12800085421aca76fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/220840
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-04 18:55:08 +00:00
Cherry Zhang 4df1db99c4 [dev.link] cmd/link: convert stack bound check pass to new symbol representation
This CL converts the stack bound check pass to use the loader and
new symbol representation.

Change-Id: I0e4c102909db20dc2d5a367379146728862e53cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/220059
Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-04 15:33:13 +00:00
Jeremy Faller b9642cbb8d [dev.link] cmd/link: add Target to relocation functions
We need to propogate Target through the relocation functions.

Change-Id: I1192d54a20a34569fd13c1511d2b21ebf2d1272e
Reviewed-on: https://go-review.googlesource.com/c/go/+/220839
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-03 15:09:44 +00:00
Than McIntosh b7d5346889 [dev.link] cmd/link: remove -newdw flag for phase 1 DWARF gen
Remove the "-newdw" command line option, a temporary placeholder/gate
for the first phase of new DWARF gen, along with some portions of the
sym.Symbol based DWARF type gen code. The remainder of the old code
will be removed in a subsequent patch.

Change-Id: I7870539f020fbf05d0345d3af2acf383fae94892
Reviewed-on: https://go-review.googlesource.com/c/go/+/220984
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-03-02 13:35:06 +00:00
Alex Brainman 95f3821390 cmd/go, cmd/link: implement -buildmode=pie on windows
This CL implements windows version of -buildmode=pie code in both
cmd/go and cmd/link.

Windows executables built with -buildmode=pie set (unlike the one
built with -buildmode=exe) will have extra .reloc PE section, and
will have no IMAGE_FILE_RELOCS_STRIPPED flag set. They will also
have IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE flag set, and
IMAGE_DLLCHARACTERISTICS_HIGH_ENTROPY_VA flag set for windows/amd64.

Both cgo and non-cgo versions are implemented. And TestBuildmodePIE
is extended to test both cgo and non-cgo versions on windows and
linux.

This CL used some code from CLs 152759 and 203602.

RELNOTE=yes

Fixes #27144
Updates #35192

Change-Id: I1249e4ffbd79bd4277efefb56db321c390c0f76f
Reviewed-on: https://go-review.googlesource.com/c/go/+/214397
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-03-01 05:00:10 +00:00
Cherry Zhang 99eb14a7f4 [dev.link] all: merge branch 'master' into dev.link
Clean merge.

Change-Id: I8098f15f249f7a1c054c708d22dac41e5951cddb
2020-02-28 15:28:43 -05:00
Elias Naur e6d7326fb6 cmd/link: default to internal linking for android/arm64
The bootstrapping process (make.bash) on all other platforms use
internal linking. This change brings android/arm64 in line, fixing the
scary warning on our self-hosted Corellium builders:

warning: unable to find runtime/cgo.a

The linkmode default is changed to internal for all Android programs,
but in practice that won't matter outside our builders: using Go with
Android apps requires buildmode=c-shared which uses linkmode external.

Fixes #31343
Updates #31819

Change-Id: I3b3ada5ed69a7989e6d8e5960bbebf5e1c22aada
Reviewed-on: https://go-review.googlesource.com/c/go/+/207299
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-26 09:47:21 +00:00
Cherry Zhang 490bc6097a cmd/link: stop requiring the math package on ARM
It was needed for the old softfloat implementation, which has
long gone.

Change-Id: Ib8d53689209a3b003d62f84c7c6047d8ec5df859
Reviewed-on: https://go-review.googlesource.com/c/go/+/220719
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-25 18:45:59 +00:00
Than McIntosh 25f1b093fe [dev.link] cmd/link: convert DWARF type generation to use loader
Converts the portion of DWARF generation that deals with creation of
type DIEs and constant DIEs to use the new loader interfaces. Creation
of subprogram DIE and compilation unit DIE content still operates on
sym.Symbols at the moment, and happens much later in the linker.

The new code for type DIE generation is gated/guarded by the linker
flag "-newdw", which currently defaults to true. At some point in the
near future this flag should be removed, but it is handy for triage at
the moment.

This patch also includes shim code designed to run after loadlibfull()
that walks through the DIE chains and to converts loader.Sym
references back into sym.Symbol references for the remainder of the
compilation, since the second phase of DWARF has not yet been
converted.

Change-Id: I681a00fb8a1f3c37884a79b373d86411332e07c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/208230
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-19 21:15:12 +00:00
Jeremy Faller 860f12a2fc [dev.link] cmd/link: dostrdata and fieldtrack with new syms
Move the wavefront past fieldtrack and dostrdata.

Change-Id: Ia327ece0202e24031fec7e1f70b40e15fbb4f728
Reviewed-on: https://go-review.googlesource.com/c/go/+/219226
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-18 15:29:12 +00:00
Cherry Zhang 112a7cb82c [dev.link] cmd/link: remove the second result of MakeSymbolUpdater
With unique global indices, MakeSymbolUpdater will not change the
symbol's index. So no need to return a new index.

Change-Id: I5b4fd6a0167cc74476880bbf4382c524ecde7721
Reviewed-on: https://go-review.googlesource.com/c/go/+/219227
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-13 16:56:56 +00:00
Cherry Zhang 5a8b1509dd [dev.link] cmd/link: clean up unused function and arguments
Change-Id: Idacb027c5ae11e029ee4ab10f6c9a76ff435e5b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/219228
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-13 15:47:21 +00:00
Cherry Zhang b0275bfbf4 [dev.link] cmd/link: add defined package symbols first, before any non-package symbols
Currently, the loader adds defined package symbols and
non-package symbols to the global index space object by object.
This CL changes it to add all the defined package symbols first,
then all the non-package symbols. The advantage of doing this is
that when adding package symbols, by definition they cannot be
dup to each other, so we don't need to do a name lookup when
adding them.

We still add them to the lookup table (for now), since they may
still be referenced by name (e.g. through linkname). This CL is
also a prerequisite if we want to move to not adding package
symbols to the lookup table entirely (e.g. by using
pre-generated in-file lookup table).

Also update some comments to reflect the current state.

Change-Id: Ib757e070b48a9ef6215e47dc3421fc5c055b746c
Reviewed-on: https://go-review.googlesource.com/c/go/+/219078
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-02-12 22:07:52 +00:00
Jeremy Faller 1152c6d3f0 [dev.link] cmd/link: remove old host object loader vestiges
Change-Id: Ic27d1d62d1b411fa7cdf66af44f9843aa644e776
Reviewed-on: https://go-review.googlesource.com/c/go/+/217558
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-06 20:15:14 +00:00
Cherry Zhang 29d95be875 [dev.link] cmd/link: simplify named symbol resolution
Now that we have local-global index mappings, just use that for
symbol reference resolution.

Change-Id: I6bc5405853fe040ff21b624ccd8da7965d66ec8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/217065
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-02-05 20:50:45 +00:00
Jeremy Faller 83ba044be6 [dev.link] cmd/link: use new sym format in pe loader
Change-Id: Ib784b8432ff4355b7ff4068801a0bcfcaf108950
Reviewed-on: https://go-review.googlesource.com/c/go/+/216718
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-02-05 20:12:48 +00:00
Jeremy Faller 8896a6b8aa [dev.link] cmd/link: port xcoff to new loader syntax
Change-Id: I074dd726640f2bcf7aa50b5e10e0b3a278489cd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/216038
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-28 18:26:28 +00:00
Jeremy Faller 76d0977745 [dev.link] cmd/link: remove flag FlagNewLdElf
No real reason to keep the old code around as the new stuff is working
just fine.

Change-Id: I5e886f8274344738663a0ead181c5d58d9f5a45f
Reviewed-on: https://go-review.googlesource.com/c/go/+/215997
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-28 16:26:14 +00:00
Jeremy Faller 4ad94a5db9 [dev.link] cmd/link: use new sym builders in macho loader
Change-Id: Ia055559d1eb12736d0bdd5a30103cd4b9788d36e
Reviewed-on: https://go-review.googlesource.com/c/go/+/215917
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-28 16:26:01 +00:00
Than McIntosh 5279166402 [dev.link] cmd/link: convert Link.linksetup method to work with loader
Switch the linker's Link.linksetup method to work with loader.Sym
instead of sym.Symbols. Currently enabled when the new ELF host object
loader is turned on.

Change-Id: I336cc9f36166767baac574455531e195b6f1ac57
Reviewed-on: https://go-review.googlesource.com/c/go/+/213423
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-10 12:36:22 +00:00
Than McIntosh fa284af720 [dev.link] cmd/link: convert ELF host object loading to new loader
This is a rewrite of the ELF host object loader to use just the Loader
interfaces for symbol creation, without constructing sym.Symbols. At
the moment this is gated under the temporary linker command line
option "-newldelf".  This version is able to get through all.bash
on linux/amd64.

Change-Id: I99f41368f75b0df9e35ef3c2cf2a702b732540c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/210779
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-09 20:48:48 +00:00
Than McIntosh b720014743 [dev.link] cmd/link: add SymbolBuilder helper
Add SymbolBuilder helper type -- this type provides a set of methods
intended to make it easy to manipulate the content of a symbol (type,
relocations, data, etc).

Change-Id: I579bf8d04650e66d33a9780a6c2347a576c94c6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/210178
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-12-27 15:05:39 +00:00
Jeremy Faller 664accc7be [dev.link] cmd/link: remove sym.Symbols from loader
Pushing the wavefront forward requires a single source of truth for
symbols. This CL removes sym.Symbols from the loader for host object
loaders, allowing us to have the single source of truth for symbols be
the loader, not some strange combination of sym.Symbols and the loader.

Change-Id: Ib8ff0117ebe6040fade346047535ff342d781081
Reviewed-on: https://go-review.googlesource.com/c/go/+/209217
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-12-12 20:57:54 +00:00
Cherry Zhang 27c0aeee3f [dev.link] all: merge branch 'master' into dev.link
Clean merge.

Change-Id: I4a73cd4bac2f4d35d2c07c39f2bc01c6075fadab
2019-11-15 16:33:22 -05:00
Elias Naur d3c2b1f176 cmd/link/internal/ld,misc/cgo/testcshared: don't -fuse-ld=gold on Android
The NDK is switching to ldd, and will stop including the gold linker.

Change-Id: If74168017c9874134b34010906ab1d94001528b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/206840
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-14 17:43:48 +00:00
Cherry Zhang 62dfb27827 [dev.link] cmd/link: delete old object file format support
There are more cleanups to do, but I want to keep this CL mostly
a pure deletion.

Change-Id: I30f4891a2ea54545fd6b83041746ab65895537e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/206558
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-11-12 19:59:04 +00:00
Jeremy Faller aacffb6576 [dev.link] cmd/link: remove old objs from host obj loaders
Change-Id: I9ee853e37090ea015aba817d9aeb2f4cc9ff5a1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/206539
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-11 20:19:37 +00:00
Than McIntosh 795e8c2317 cmd/go: address DWARF linker issues with -buildmode=plugin on Darwin
Assorted fixups in the linker needed to enable turning back on
DWARF generation when building plugins for Darwin. Includes:

 - don't suppress import of runtime/cgo in the linker for
   Darwin if we are linking in plugin mode

 - in calcCompUnitRanges handle the case where we encounter
   linker-generated functions that have no associated Unit (and
   also have no DWARF)

 - generalize a guard in relocsym() include so as to avoid
   triggering a spurious error on go.info symbols in plugin mode

Updates #21647.
Updates #27502.

Change-Id: I317fea97bef2f3461e31498e63f9fd6d8b8f4b23
Reviewed-on: https://go-review.googlesource.com/c/go/+/182959
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-11 17:37:48 +00:00
Gerrit Code Review bababde766 Merge "cmd: merge branch 'dev.link' into master" 2019-11-08 20:24:43 +00:00
Joel Sing a2b1dc863f cmd/link,cmd/internal/objabi: factor out direct call identification
Factor out the direct CALL identification code from objabi.IsDirectJump and
use this in two places that have separately maintained lists of reloc types.
Provide an objabi.IsDirectCallOrJump function that implements the original
behaviour of objabi.IsDirectJump.

Change-Id: I48131bae92b2938fd7822110d53df0b4ffb35766
Reviewed-on: https://go-review.googlesource.com/c/go/+/196577
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-06 17:03:51 +00:00
Elias Naur 1bd974eeb7 cmd/link/internal/ld: omit bitcode-incompatible flags on iOS simulator
The -Wl,-headerpad, -Wl,-no_pie, -Wl,-pagezero_size flags are
incompatible with the bitcode-related flags used for iOS.

We already omitted the flags on darwin/arm and darwin/arm64; this change
omits the flags on all platforms != macOS so that building for the iOS
simulator works.

Updates #32963

Change-Id: Ic9af0daf01608f5ae0f70858e3045e399de7e95b
Reviewed-on: https://go-review.googlesource.com/c/go/+/205340
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-06 14:56:38 +00:00
Cherry Zhang bbae923d20 cmd: merge branch 'dev.link' into master
In the dev.link branch we implemented the new object file format
and (part of) the linker improvements described in
https://golang.org/s/better-linker

The new object file is index-based and provides random access.
The linker maps the object files into read-only memory, and
access symbols on-demand using indices, as opposed to reading
all object files sequentially into the heap with the old format.

The linker carries symbol informations using indices (as opposed
to Symbol data structure). Symbols are created after the
reachability analysis, and only created for reachable symbols.
This reduces the linker's memory usage.

Linking cmd/compile, it creates ~25% fewer Symbols, and reduces
memory usage (inuse_space) by ~15%. (More results from Than.)

Currently, both the old and new object file formats are supported.
The old format is used by default. The new format can be turned
on by using the compiler/assembler/linker's -newobj flag. Note
that the flag needs to be specified consistently to all
compilations, i.e.

go build -gcflags=all=-newobj -asmflags=all=-newobj -ldflags=-newobj

Change-Id: Ia0e35306b5b9b5b19fdc7fa7c602d4ce36fa6abd
2019-11-05 14:57:48 -05:00
Matthew Dempsky e341e93c51 cmd/compile, cmd/link: add coverage instrumentation for libfuzzer
This CL adds experimental coverage instrumentation similar to what
github.com/dvyukov/go-fuzz produces in its -libfuzzer mode. The
coverage can be enabled by compiling with -d=libfuzzer. It's intended
to be used in conjunction with -buildmode=c-archive to produce an ELF
archive (.a) file that can be linked with libFuzzer. See #14565 for
example usage.

The coverage generates a unique 8-bit counter for each basic block in
the original source code, and emits an increment operation. These
counters are then collected into the __libfuzzer_extra_counters ELF
section for use by libFuzzer.

Updates #14565.

Change-Id: I239758cc0ceb9ca1220f2d9d3d23b9e761db9bf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/202117
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-05 00:00:36 +00:00
Than McIntosh bf7e55b618 cmd/link: directly exec archive command if external tmpdir
When linking a Go archive, if the archiver invocation is the very last
thing that needs to happen in the link (no "atexit" cleanups required
remove the locally created tmpdir) then call syscall.Exec to invoke
the archiver command instead of the usual exec.Command. This has the
effect of reducing peak memory use for the linker overall, since we
don't be holding onto all of the linker's live memory while the
archiver is running.

Change-Id: Ibbe22d8d67a70cc2a4f91c68aab56d19fb77c393
Reviewed-on: https://go-review.googlesource.com/c/go/+/203821
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-04 18:51:13 +00:00
Cherry Zhang d3660e8ef8 cmd/link: enable PIE on darwin/arm
We used to pass -no_pie to external linker on darwin/arm, which
is incompatible with -fembed-bitcode. CL 201358 attempted to
remove the -no_pie flag, but it resulted the darwin linker to
complain about absolute addressing in TEXT segment.

On darwin/arm, we already get away from absolute addressing in
the TEXT section. The complained absolute addressing is in
RODATA, which was embedded in the TEXT segment. This CL moves
RODATA to the DATA segment, like what we already did on ARM64
and on AMD64 in c-archive/c-shared buildmodes for the same reason.
So there is no absolute addressing in the TEXT segment, which
allows us to remove -no_pie flag.

Fixes #35252.
Updates #32963.

Change-Id: Id6e3a594cb066d257d4f58fadb4a3ee4672529f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/205060
Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-11-04 15:18:34 +00:00
Cherry Zhang db75205a9b [dev.link] cmd/link: restore -strictdups flag in newobj mode
Change-Id: I93ad769595fa343400afa342af12e1445abff084
Reviewed-on: https://go-review.googlesource.com/c/go/+/204918
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-03 20:19:00 +00:00
Cherry Zhang 24ea07d592 [dev.link] cmd/link: add internal packages at the end
Currently in the linker we load internal packges first, then the
main package, and then load imported packages following the
dependency graph. As a result, packages are loaded mostly in the
dependency order, except the internal packages. The global symbol
indices are assigned the same way.

By loading the internal packages at the end, the packages are
loaded in the dependency order, so are the global indices. This
way, a relocation edge is mostly either within a packge or a
forward edge from a smaller index to a larger one. This allows
us to use a min-heap work queue in the deadcode pass, to achieve
better spatial locality (in the next CL).

Change-Id: I01fa9b3cf0c9e9e66006040f6378a51fd78f0f39
Reviewed-on: https://go-review.googlesource.com/c/go/+/204437
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-11-02 19:21:37 +00:00
Jeremy Faller 496e4273e4 [dev.link] cmd/link: fix loadpe to work with new obj file
Change-Id: I0fe88df182f13e7f04c8de0b82e111db441a26e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204341
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-02 19:21:08 +00:00
Cherry Zhang 48a0b97902 [dev.link] cmd/link: set cgo attributes early when internal cgo linking
In newobj mode, cgo attributes are typically set later, as we
create sym.Symbols later. But when internal cgo linking, the
host object loaders still work with sym.Symbols, and the cgo
attributes need to be set for them to work properly. Therefore,
set them early. This will cause creating some Symbols eagerly,
but they are mostly host object symbols and will need to be
created anyway.

Now all cgo internal linking tests pass on ELF systems.

Change-Id: I023a4df4429acc8ebf5e185f62e6809198497a78
Reviewed-on: https://go-review.googlesource.com/c/go/+/204857
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-02 00:36:59 +00:00
Jeremy Faller e961b26c27 [dev.link] cmd/link: fix xcoff loader for new obj format
config.go needs to be removed from this CL.

Change-Id: I04a267feeae1551bb18f6a03a725adc9db593fdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/204099
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-01 15:36:44 +00:00
Than McIntosh c0555a2a7a [dev.link] all: merge branch 'master' into dev.link
Fixed a couple of minor conflicts in lib.go and deadcode.go
relating to debug logging.

Change-Id: I58335fc42ab1f1f3409fd8354da4f26419e8fb22
2019-11-01 10:45:24 -04:00
Than McIntosh 219922e95b [dev.link] cmd/link: add support to new deadcode for field tracking
Fix up the new dead code pass to include support for populating the
ctxt "Reachparent" map, which is needed to support field tracking.
Since we don't have sym.Symbols created at the point where new dead
code runs, keep track of reachability using global symbol indices, and
then once loader.LoadFull is complete we can translate the index
mappings into symbol mappings.

The fieldtracking output is unfortunately different relative to
master, due to differences in the order in which symbols are
encountered in deadcode, but I have eyeballed the results to make sure
they look reasonable.

Change-Id: I48c7a4597f05c00f15af3bfd37fc15ab4d0017c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204342
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-01 13:54:58 +00:00
Cherry Zhang 63815923fc [dev.link] cmd/link: resolve ABI aliases for external symbols
ABI alias references in Go symbols are resolved during
loadObjFull. But for external symbols they are not resolved. If
there is a reference from an external symbol to a Go ABIInternal
symbol, this reference will be invalid as it is not resolved.

The old code resolve ABI aliases in the deadcode pass. But the
new deadcode pass doesn't do it, as it works with indices instead
of Symbols. We do this in LoadFull.

This makes all internal cgo linking tests pass on Mach-O.

Change-Id: Iac6c084c03f5ddbcc9455527800ce7ed7313f9a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/204698
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-01 13:51:36 +00:00
Cherry Zhang 4a43a50791 [dev.link] cmd/link: process cgo_import_static before host object loading
In internal linking mode, we need to process cgo_import_static
directives before loading host objects, because the directive is
to tell the host object loader how to deal with imported symbols.

This should fix linking with old object files. I think there
needs some similar logic for new object files, but I'll leave
that for later.

Change-Id: Icaa286de626ea1876086dbdd015047084c92caf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/204697
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-01 03:40:20 +00:00
Jeremy Faller 396a9b98ba [dev.link] cmd/link: fix macho host objects
Small updates to macho loader, fixing some misunderstandings I had about
using the new object file format.

Change-Id: I9224b01ca327e3a087ebfa36800bd6eef6abcc80
Reviewed-on: https://go-review.googlesource.com/c/go/+/204097
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-31 15:22:28 +00:00
Jeremy Faller 0e76e2f4e5 [dev.link] cmd/link: elf host obj support w/ new obj files
Add support for elf host objects with new object file format.

Change-Id: Ic5be1953359b9b6b78d9a0b715af69763aefd227
Reviewed-on: https://go-review.googlesource.com/c/go/+/201728
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-31 13:11:51 +00:00
Cherry Zhang 43d25a61c1 [dev.link] cmd/link: ensure deterministic order in dynexp
dynexp is used for generating the dynamic symbol table. It is
created from a map. Sort it to ensure deterministic order.

Should fix solaris build.

Change-Id: I561b9da3a4136a7ea41139073f76c98fb069d4fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/204378
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-31 01:09:34 +00:00
Elias Naur f4e32aeed1 cmd/link/internal/ld: remove flags incompatible with -fembed-bitcode
The flags -headerpad, -Wl,-no_pie and -pagezero_size are incompatible with
the -fembed-bitcode flag used by `gomobile build`. Than McIntosh
suggested we might not need the offending flags; this change removes
the flags on darwin/arm64 and -headerpad, -pagezero_size on darwin/arm.

The -Wl,-no_pie flag is left for darwin/arm because linking fails
without it:

ld: warning: PIE disabled. Absolute addressing (perhaps -mdynamic-no-pic) not allowed in code signed PIE, but used in _runtime.rodata from /var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/tmp/go-link-225285265/go.o. To fix this warning, don't compile with -mdynamic-no-pic or link with -Wl,-no_pie

Discussion: https://groups.google.com/d/msg/golang-dev/U1jK3xmmGAk/j0_ty46EDAAJ

I've verified the CL on the builders, built the "flappy" example from
gomobile with `gomobile build`, and verified that flappy runs on an
iPhone 5S.

Updates #32963

Change-Id: I783abc93ccf3c1d2b7ca00144b7164ba223d3529
Reviewed-on: https://go-review.googlesource.com/c/go/+/201358
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-30 08:17:29 +00:00
John Papandriopoulos 59a6847039 cmd/link: pass-through undefined call targets in external link mode
Allows Go asm calls referencing a function in a .syso file to be
passed through to the external linker, that would have otherwise
raised a "relocation target X not defined" error in cmd/link.

Fixes #33139

Change-Id: I2a8eb6063ebcd05fac96f141acf7652cf9189766
Reviewed-on: https://go-review.googlesource.com/c/go/+/198798
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-28 22:10:06 +00:00
Dmitri Goutnik 152dddee7e cmd/link: switch to ld.bfd on freebsd/arm64
Updates golang/go#35197

Change-Id: I4fd85c84475761d71d2c17e62796e0a411cf91d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/203519
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2019-10-28 09:26:37 +00:00
Jason A. Donenfeld 31bfab4ac6 cmd/link: enable DEP for Windows executables
There's no reason not to enable DEP in 2019, especially given Go's
minimum operating system level.

RELNOTE=yes

Change-Id: I9c3bbc5b05a1654876a218123dd57b9c9077b780
Reviewed-on: https://go-review.googlesource.com/c/go/+/203601
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2019-10-27 09:49:02 +00:00
nikita-vanyasin c3e8a20a65 cmd/link/internal: eliminate all ld.Cputime() usages
Also a similar 'elapsed' function and its usages were deleted.

Fixes #19865.

Change-Id: Ib125365e69cf2eda60de64fa74290c8c7d1fd65a
Reviewed-on: https://go-review.googlesource.com/c/go/+/171730
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-26 05:40:14 +00:00
Cherry Zhang d77b809df9 [dev.link] all: merge branch 'master' into dev.link
The only conflict is in cmd/internal/obj/link.go and the
resolution is trivial.

Change-Id: Ic79b760865a972a0ab68291d06386531d012de86
2019-10-25 13:41:36 -04:00
Alex Brainman e4c3925925 cmd/link: use libmsvcrt.a during internal link
When using recent versions of gcc with cgo, internal link fails with

c:\>go test debug/pe
--- FAIL: TestInternalLinkerDWARF (0.94s)
    file_test.go:394: building test executable for linktype 2 failed: exit status 2 # command-line-arguments
        runtime/cgo(.text): relocation target __acrt_iob_func not defined for ABI0 (but is defined for ABI0)
        runtime/cgo(.text): relocation target __acrt_iob_func not defined for ABI0 (but is defined for ABI0)
        runtime/cgo(.text): relocation target __acrt_iob_func not defined for ABI0 (but is defined for ABI0)
FAIL
FAIL    debug/pe        4.572s
FAIL

It appears that __acrt_iob_func is defined in libmsvcrt.a. And this
change adds libmsvcrt.a to the list of libraries always used byi
internal linker.

libmsvcrt.a also implements __imp___acrt_iob_func. So this change
also prevents rewriting __imp___acrt_iob_func name into
__acrt_iob_func, otherwise we end up with duplicate __acrt_iob_func
symbol error.

Fixes #23649

Change-Id: Ie9864cd17e907501e9a8a3672bbc33e02ca20e5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/197977
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-20 06:38:39 +00:00
Cherry Zhang bd229936cf [dev.link] cmd/link: restore export data hash
With the previous CL, the export data will not change whether it
is compiled with -dynlink flag or not. Restore the export data
hash, and reenable plugin version check.

TODO: it may be still better to just generate a fingerprint for
each package at compile time.

Change-Id: I1f298ac97c3ab9b8d05d1c95e8be74d10ca7cd0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/201720
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-17 20:28:28 +00:00
Cherry Zhang 2bbf2e0233 [dev.link] cmd/link: add basic shared library support in newobj mode
This CL adds basic shared library support in newobj mode. This is
not complete -- there are still tests in misc/cgo/testshared
failing. But at least a simple program works, and some tests
there pass.

Add the mechanism of loading external symbols with contents.
(Before, external symbols are always contentless.) This may
potentially be also used for other host objects.

Change-Id: I68dbf71e7949cc01ebf37ea159084e798ae16925
Reviewed-on: https://go-review.googlesource.com/c/go/+/201537
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-10-16 22:28:18 +00:00
Cherry Zhang 5caac2f73e [dev.link] cmd: default to new object files
Switch the default to new object files.

Internal linking cgo is disabled for now, as it does not work yet
in newobj mode.

Shared libraries are also broken.

Disable some tests that are known broken for now.

Change-Id: I8ca74793423861d607a2aa7b0d89a4f4d4ca7671
Reviewed-on: https://go-review.googlesource.com/c/go/+/200161
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-10-16 15:57:07 +00:00
Than McIntosh c9470b0483 [dev.link] cmd/link/internal/objfile: relocate loader to new package
Third change of several to update the loader API to reflect the final
consensus version of the loader API as described in Cherry's doc.
This piece:

   - move objfile.Loader into its own separate package, and update
     clients accordingly.

This includes a few minor cleanups, including converting a couple
of loader-related functions to methods, and privatizing some of the
loader methods such as ToGlobal/ToLocal.

Change-Id: Iae20585751a45491d8b19dcffc096aadae6bbfc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/200998
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-16 13:24:55 +00:00
Cherry Zhang cab29ebd84 [dev.link] cmd/link: create sym.Symbols after deadcode in newobj mode
With the new object files, now we can run the deadcode pass on
indices instead of Symbol structs, so we can delay creating
Symbols after the deadcode pass. Then we only need to create
reachable symbols.

Not create Symbols in LoadNew and LoadRefs, and recombine
LoadReloc into LoadFull.

Split loadcgo into two parts: the first finds root symbols, the
second create Symbols and sets attributes. The first runs before
the deadcode pass, while the second runs after.

TODO: currently there are still symbols that are not marked
reachable but still used. This includes DWARF symbols, file
symbols, and type symbols that are referenced by DWARF symbols.
We still need to create them (conservatively).

Change-Id: I695779c9312be9d49ab1683957ac3e72e1f65a1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/199643
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-15 18:55:32 +00:00
Jeremy Faller cc21e4d130 [dev.link] cmd/link: move macho host files to new loader format
Change-Id: I823b19c0742992dd760c6372428a1936bb7c7e70
Reviewed-on: https://go-review.googlesource.com/c/go/+/200768
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-14 14:03:56 +00:00
Cherry Zhang 8a9be4921a [dev.link] cmd/link: use index for deadcode
Switch the deadcode pass to use indices instead of Symbol
structures when using new object file format. Delay loading
symbol relocations and contents fully after the deadcode pass.
The next step is not to create Symbol structures until deadcode
is done.

Method tracking logic hasn't been implemented. Currently, all
methods of a reachable type are live.

Change-Id: Iffcd06ff84e6e52bd9eb24d1220d94234d18ab6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/198199
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-10 22:05:09 +00:00
Cherry Zhang 65a649c565 [dev.link] cmd/link, cmd/internal/goobj2: adopt new DWARF compilation unit logic with new object file
The dev.link branch was not sync'd with the new DWARF compilation
unit logic change on the master branch, and the new object file
format didn't support this.

This CL adds the new DWARF CU and file table support to the new
object file format. In the old object file, the DWARF file table
is a separate section. For now, we do the same with the new
object file, keeping it as a separate block.

While here, also refactor the loader code so it is easier for the
loader to carry per-object informations.

Change-Id: I4c317941fc0a5831acbc11ce8c2a8b7421471372
Reviewed-on: https://go-review.googlesource.com/c/go/+/198198
Reviewed-by: Austin Clements <austin@google.com>
2019-10-09 21:05:20 +00:00
Cherry Zhang 0d7404c852 [dev.link] cmd/link, cmd/internal/goobj2: mmap object file in -newobj mode
With the old object file format, we use mmap (if supported) to
read object files and back symbol data with mapped read-only
memory.

Do the same with the new object file format. This also
significantly reduces number of syscalls made to read object
files.

Currently we still do mmap in object file level, not archive
level. This is probably ok, as there shouldn't be many archives
that contain more than one object. If this is a problem we can
change that later.

Change-Id: Icae3ef14d8ed6adbee1b5b48d420e2af22fd9604
Reviewed-on: https://go-review.googlesource.com/c/go/+/197797
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-09 21:00:08 +00:00
Cherry Zhang 2495095275 [dev.link] cmd/link: load full symbol contents after deadcode pass
If the new object file format is used, now we load full symbol
contents after the deadcode pass, for reachable symbols only.
We still load some informations early, like relocations and the
contents of type symbols, which are used in the deadcode pass.
If we rewrite deadcode to use index directly, we could delay more
of the loading (to sym.Symbol), and perhaps delay the creation of
sym.Symbol.

TODO: internal linking with host objects doesn't work yet.

Change-Id: I7d4880e8f150e8709ffac277e62191623440e4cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/197258
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-09 16:05:40 +00:00
Cherry Zhang 48151b3f0f [dev.link] cmd/link: add symbols to Textp after deadcode pass
Currently we add all text symbols to ctxt.Textp at load time,
then the deadcode pass filters out unreachable ones. This CL
delays adding symbols to ctxt.Textp to the end of the deadcode
pass, where we only add reachable ones.

Change-Id: Ie83b2958f915c5aaa004b8c5ed1f1bc275f4d1db
Reviewed-on: https://go-review.googlesource.com/c/go/+/197257
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-08 19:15:46 +00:00
Cherry Zhang e63c1df348 [dev.link] cmd/link: move some work after deadcode
This is in preparation of delaying loading symbol content to
after dead code elimination.

Change-Id: Ia4ce82d9a42cc1a76c82f6d0eb9a54b0325ee3ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/197061
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-10-04 20:43:56 +00:00
Cherry Zhang 6051fd0ad9 [dev.link] cmd/link: support new object file format
Parse new object file format in the linker. At least we can link
a hello-world program.

Add a basic "loader", which handles symbol references in the
object file.
- mapping between local and global indices
- resolve by-name references
  (TODO: the overwrite logic isn't implemented yet)

Currently we still create sym.Symbol rather early, and, after all
the object files are loaded and indexed references are resolved,
add all symbols to sym.Symbols.

The code here is probably not going in the final version. This
is basically only for debugging purposes -- to make sure the
writer and the reader work as expected.

Change-Id: I895aeea68326fabdb7e5aa1371b8cac7211a09dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/196032
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-04 15:21:40 +00:00
Andrew Bonventre 1820cca7d7 Revert "cmd/link: load symbols from .syso in external link mode"
This reverts CL 186417.

Reason for revert: Broke darwin (10_14), linux (ppc), aix (ppc)

Updates #33139

Change-Id: I8bf3c817a96a0e57e45754a097cea7062b2fcdfd
Reviewed-on: https://go-review.googlesource.com/c/go/+/198177
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-01 13:20:16 +00:00
John Papandriopoulos 5e514b76e2 cmd/link: load symbols from .syso in external link mode
Fix linking with a package having a .syso file in external link mode,
that would otherwise cause an error before executing the external
linker because it can't find symbols that are exported in the said
.syso file.

Fixes #33139

Change-Id: Id3ee737fba1c6f1e37910593dfedf9c84486d398
Reviewed-on: https://go-review.googlesource.com/c/go/+/186417
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-01 00:02:11 +00:00
Than McIntosh 70a1efbb5c cmd/link: remove reading/processing of function Autom records
Remove linker reading and processing of automs (no longer needed, now
that the compiler is emitting R_USETYPE relocations on functions). So
as to avoid changing the object file format, the object still contains
a count of automs, but this count is required to be zero.

Updates #34554.

Change-Id: I10230e191057c5c5705541eeb06f747d5f73c42d
Reviewed-on: https://go-review.googlesource.com/c/go/+/197500
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2019-09-27 13:59:05 +00:00
Jeremy Faller 78a3734714 cmd/link: add notion of multiple compilation units per package
As we move the debug_line generation into the compiler, we need to
upgrade the notion of compilationUnit to not just be on a per package
basis.  That won't be the case as it will be impossible for all
compilationUnits to have the same set of files names used to build the
debug_lines table. (For example, assembled files in a package don't know
about any files but themselves, so the debug_lines table could only
reference themseves. As such, we need to break the 1:1 relationship
between compUnit and package.)

Change-Id: I2e517bb6c01de0115bbf777af828a2fe59c09ce8
Reviewed-on: https://go-review.googlesource.com/c/go/+/189618
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-24 21:33:10 +00:00
Jeremy Faller 8f816259ee cmd/link: make PCIter available to compiler
I'm branching this off cl/187117, and will be reworking that diff stack.

Testing: I've run go build -toolexec 'toolstash -cmp'

Change-Id: I922a97d0f25d52ea70cd974008a063d4e7af34a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/188023
Reviewed-by: Austin Clements <austin@google.com>
2019-09-23 19:02:02 +00:00
Yuval Pavel Zholkover c78ac39ae9 cmd/link: use gold when dynamic linking on arm,arm64 only on Linux and Android
Fixes freebsd/arm build.

Change-Id: Id9b1905a5335f86d317dab7514e0ce7cb74aba1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/193537
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-09-06 17:41:07 +00:00
Than McIntosh b1a20253fe cmd/link: memoize/cache whether plugin.Open symbol available
Perform a single lookup of "plugin.Open" at the point where we set the
loaded flag for the context, then cache whether the result is nil, so
that we can consult this cached value later on (instead of having to
look up the symbol each time). This helps speed up the DynLinkingGo()
context method, which is called from within some very hot loops in the
linker (when linking 'hyperkube' from kubernetes, reduces total calls
to "sym.(*Symbols).ROLookup" from 6.5M to 4.3M)

Change-Id: I92a2ea2b21d24f67aec0a7afeef4acc77c095adf
Reviewed-on: https://go-review.googlesource.com/c/go/+/193260
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-06 12:16:32 +00:00
Jason A. Donenfeld c96d794f66 ld: ensure that PE versions sync for internal and external linkage
Previously users who opted into cgo might have received a bit of a
behavior surprise when their mingw installation defaulted to a
potentially older and different set of compatibility hacks. Since Go is
explicitly targeting >=6.1 for internal linkage, propagate these changes
to external linkage too.

While we're at it, we move these values into constant variables so that
they don't become out of sync and allow for easy updating as Go
gradually drops compatibility for older operating systems.

Change-Id: I41e654d135be6e3db9088e73efeb414933e36caa
Reviewed-on: https://go-review.googlesource.com/c/go/+/191842
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2019-08-31 14:56:29 +00:00
Jason A. Donenfeld 8fc35238a7 ld: mark PE executables as terminal services aware
This has been the default in MSVC for a very long time, and it's hard to
imagine modern programs actually wanting the old legacy behavior. For
example, no modern programs try to install their junk into C:\windows
and therefore need to have an emulated writable windows directory.
That's not really even allowed by ACLs on modern systems.

Change-Id: Iadaca6815e39ea5c6b05c1cac5a95cfc35e5b48a
Reviewed-on: https://go-review.googlesource.com/c/go/+/191840
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2019-08-31 14:54:31 +00:00
Ian Lance Taylor 951143cf14 cmd/link: increase the function call limit in stkcheck
There is real (albeit generated) code that exceeds the limit.

Fixes #33555

Change-Id: I668e85825d3d2a471970e869abe63f3492213cc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/189697
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-08-09 20:22:03 +00:00
Ian Lance Taylor 37b194a422 cmd/link: put shlib ".type" functions in internal ABI
These functions are compiler generated, and as such are only available
in the internal ABI. Doing this avoids generating an alias symbol.
Doing that avoids confusion between unmangled and mangled type symbols.

Fixes #30768

Change-Id: I197a5ba6403aac11989ffa951dbe35bd0506de91
Reviewed-on: https://go-review.googlesource.com/c/go/+/186077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-07-15 22:54:39 +00:00
Russ Cox ffd7eba20a cmd/internal/bio: rename Reader.Seek to MustSeek
Renaming the method makes clear, both to readers and to vet,
that this method is not the implementation of io.Seeker:
it cannot fail.

Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

For #31916.

Change-Id: I3e6ad7264cb0121b4b76935450cccb71d533e96b
Reviewed-on: https://go-review.googlesource.com/c/go/+/176108
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 16:56:31 +00:00
erifan01 d2765de863 cmd/link: support PIE mode with internal link on linux arm64
This CL improves internal link to provide basic support for cgo and PIE:
1, add support for GOT, PLT and GOTPLT.
2, add support for following ELF relocation types which have been used by std
   packages:
     R_AARCH64_ADR_GOT_PAGE
     R_AARCH64_LD64_GOT_LO12_NC
     R_AARCH64_ADR_PREL_PG_HI21
     R_AARCH64_ADD_ABS_LO12_NC
     R_AARCH64_LDST8_ABS_LO12_NC
     R_AARCH64_LDST32_ABS_LO12_NC
     R_AARCH64_LDST64_ABS_LO12_NC
     R_AARCH64_JUMP26
     R_AARCH64_ABS64
     R_AARCH64_PREL32
     R_AARCH64_PREL64

With this change, Go toolchain can be built in internal linking mode, and
pure Go programs can be built with PIE mode in internal linking mode on arm64.

Updates #10373
The prototype of this CL is contributed by Wei Xiao <wei.xiao@arm.com>

Change-Id: I2253923c69e855fd1524d54def309a961dce6247
Reviewed-on: https://go-review.googlesource.com/c/go/+/163579
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
2019-05-02 15:38:36 +00:00
Clément Chigot 0c9e0c257b cmd/link: add .go.buildinfo in XCOFF symbol table
.go.buildinfo must be added to the symbol table on AIX. Otherwise, ld
won't be able to handle its relocations.

This patch also make ".data" the default section for all symbols inside
the data segment.

Change-Id: I83ac2bf1050e0ef6ef9c96ff793efd4ddc8e98d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/174298
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-30 16:54:38 +00:00
Elias Naur a74e01204e cmd/link/internal/ld: consolidate macho platform setup
Determine the macho platform once and use that the two places that
need it. This makes it easier to add a third platform check for a
follow-up change.

Updates #31447

Change-Id: I522a5fface647ab8e608f816c5832d531534df7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/174198
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-04-27 21:29:58 +00:00
Cherry Zhang b2f94d3e88 cmd/link: mmap output file
Use mmap for writing most of the output file content,
specifically, the sections and segments. After layout, we
already know the sizes and file offsets for the sections and
segments. So we can just write the bytes by copying to a mmap'd
backing store.

The writing of the output file is split into two parts. The first
part writes the sections and segments to the mmap'd region. The
second part writes some extra content, for which we don't know
the size, so we use direct file IO.

This is in preparation for mmap'ing input files read-only.

Change-Id: I9f3b4616a9f96bfd5c940d74c50aacd6d330f7d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/170738
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-04-19 18:24:44 +00:00
Josh Bleecher Snyder 6d64dd731b cmd/link: make Pciter more idiomatic
Rename it to PCIter and convert it to use methods.

Set pcscale once, during construction, to make call sites clearer.

Change some ints to bools.

Use a simple iteration termination condition,
instead of the cap comparison from the c2go translation.

Instead of requiring a Pcdata, which requires one caller
to synthesize a fake Pcdata, just ask for a byte slice.

Passes toolstash-check.

Change-Id: I811da0e929cf4a806bd6d70357ccf2911cd0c737
Reviewed-on: https://go-review.googlesource.com/c/go/+/171770
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-12 21:41:09 +00:00
Than McIntosh cacab64555 cmd/link: change -strictdups checking to handle mix of flags
Update the recently-added '-strictdups' sanity checking to avoid
failing the link in cases where we have objects feeding into the link
with a mix of command line flags (and in particular some with "-N" and
some without). This scenario will trigger errors/warnings due to
inlinable functions and wrapper functions that have different sizes
due to presence or lack of optimization.

Update #31034.

Change-Id: I1dd9e37c2f9bea5da0ab82e32e6fc210aebf6a65
Reviewed-on: https://go-review.googlesource.com/c/go/+/169160
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2019-03-27 17:53:27 +00:00
Clément Chigot 697f4183e6 cmd/link: allow buildmode c-archive for aix/ppc64
Change-Id: Ia268b0d64dc89866aa09bfffcaa109741088a904
Reviewed-on: https://go-review.googlesource.com/c/go/+/169119
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-27 17:39:13 +00:00
Elias Naur 90a3ce02dc cmd/link/internal/ld: skip TLS section on Android
We don't use the TLS section on android, and dropping it avoids
complaints about underalignment from the Android Q linker.

Updates #29674

Change-Id: I91dabf2a58e6eb1783872639a6a144858db09cef
Reviewed-on: https://go-review.googlesource.com/c/go/+/169618
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-27 14:46:21 +00:00
Than McIntosh d923309a17 cmd/link: add optional sanity checking for duplicate symbols
Introduce a new linker command line option "-strictdups", which
enables sanity checking of "ok to duplicate" symbols, especially
DWARF info symbols. Acceptable values are 0 (no checking) 1 (issue
warnings) and 2 (issue a fatal error checks fail).

Currently if we read a DWARF symbol (such as "go.info.PKG.FUNCTION")
from one object file, and then encounter the same symbol later on
while reading another object file, we simply discard the second one
and move on with the link, since the two should in theory be
identical.

If as a result of a compiler bug we wind up with symbols that are not
identical, this tends to (silently) result in incorrect DWARF
generation, which may or may not be discovered depending on who is
consuming the DWARF and what's being done with it.

When this option is turned on, at the point where a duplicate
symbol is detected in the object file reader, we check to make sure
that the length/contents of the symbol are the same as the previously
read symbol, and print a descriptive warning (or error) if not.

For the time being this can be used for one-off testing to find
problems; at some point it would be nice if we can enable it by
default.

Updates #30908.

Change-Id: I64c4e07c326b4572db674ff17c93307e2eec607c
Reviewed-on: https://go-review.googlesource.com/c/go/+/168410
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-03-22 17:26:03 +00:00
Clément Chigot cf952e9e62 cmd: enable -buildmode=pie on aix/ppc64
Change-Id: I939518462c931ba9feb125b2f299ef0706b124ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/168879
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-22 14:24:53 +00:00
Elias Naur 0fe1986a72 cmd/link/internal/ld: extract Mach-O load command parsing
We're going to need the ability to extract the LC_VERSION_MIN_* and
LC_BUILD_VERSION load commands. This CL adds peekMachoPlatform to do
that and in the process simplifies machoCombineDwarf.

While here, disable DWARF combining for Apple platforms other than
macOS (watchOS, tvOS, bridgeOS), not just iOS.

Updates #22395

Change-Id: I4862b0f15ccc87b7be1a6532b4d37b47c8f7f243
Reviewed-on: https://go-review.googlesource.com/c/go/+/168459
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-21 01:35:21 +00:00
Clément Chigot 0cd74d1f11 cmd, runtime: fix C trampolines on aix/ppc64
C trampolines are made by fixup CSECTS which are added between two
symbols. If such CSECTS is added inside Go functions, all method
offsets stored in moduledatas will be wrong.
In order to prevent this, every C code is moved at the end of the
executable and long calls are created for GO functions called by C
code.
The main function can't longer be made in Go as AIX __start isn't using
a long call to branch on it. Therefore, a main is defined on
runtime/cgo.

Change-Id: I214b18decdb83107cf7325b298609eef9f9d1330
Reviewed-on: https://go-review.googlesource.com/c/go/+/164010
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-19 03:18:05 +00:00
Clément Chigot cb8054a6b6 cmd/link: fix trampoline generation for aix/ppc64
This commit fixes trampoline generation on aix/ppc64 which must use TOC
symbols.
It also adds a size to runtime.text.X symbols to prevent ld from moving
them, like runtime.text.

Change-Id: Ida033ec20ad8d7b7fb3faeb0ec4fa7bc4ce86b7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/164009
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-15 04:15:59 +00:00
Clément Chigot a891f2e2ae cmd: always allow bigtoc generation with gcc on aix/ppc64
-mcmodel=large and -Wl,-bbigtoc must always be passed to gcc in order to
prevent TOC overflow error. However, a warning is still issued by ld. It
is removed as it doesn't give any useful information.

Change-Id: I95a78e8993cc7b5c0f329654d507409785f7eea6
Reviewed-on: https://go-review.googlesource.com/c/go/+/164008
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-13 04:17:56 +00:00
Clément Chigot 0ff9df6b53 cmd: disable DWARF with old ld on aix/ppc64
DWARF relocations isn't working with some older ld, because of
-Wl,-bnoobjreorder which is needed on Go.
This commit checks ld's version and disable DWARF generation in cmd/link
if it's too old. Some tests must therefore be skipped.

Change-Id: I2e794c263eb0dfe0b42e7062fb80c26f086b44d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/164007
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-13 02:33:58 +00:00
Clément Chigot 359fb4a9af cmd/link: on AIX generate export file for host linker
Change-Id: I6638cb0f9ed751c76a29cae62a93a923f18f14f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/164005
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-08 21:37:03 +00:00
Clément Chigot f0749c74fd cmd/link: fix moduledata symbols for aix/ppc64 and external linking
Moduledata symbols like runtime.data or runtime.text must have the
same position in the final executable (as some symbol accesses are made
by offset from them).
ld on AIX might move them randomly if there are nil size symbols.

ld will also remove unreachable symbols like runtime.epclntab or
runtime.rodata. In order to keep them, R_REF relocations are created
between firstmoduledata and these symbols. This relocation tells ld to
keep these symbols even if there aren't reachable.

Change-Id: Ie5a28cf406977131cec6442f7f5b6fd89fb775a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/164004
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-06 15:56:41 +00:00
Clément Chigot 80f10965ee cmd/link, runtime: allow external linking for aix/ppc64
This commit adds external linking in cmd/link for aix/ppc64.
As relocations on .text data aren't possible on AIX, Segrelrodata is
used to move all these datas to .data section.

Change-Id: I4d1361c1fc9290e11e6f5560864460c76551dbeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/164003
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-06 15:32:25 +00:00
Clément Chigot 20ac64a2dd cmd/dist, cmd/link, runtime: fix stack size when cross-compiling aix/ppc64
This commit allows to cross-compiling aix/ppc64. The nosplit limit must
twice as large as on others platforms because of AIX syscalls.
The stack limit, especially stackGuardMultiplier, was set by cmd/dist
during the bootstrap and doesn't depend on GOOS/GOARCH target.

Fixes #29572

Change-Id: Id51e38885e1978d981aa9e14972eaec17294322e
Reviewed-on: https://go-review.googlesource.com/c/157117
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-09 22:06:51 +00:00