Commit Graph

491 Commits

Author SHA1 Message Date
Paul E. Murphy e4253cd023 cmd/link: Add section data slice to Archrelocvariant
PPC64 needs to preserve bits when applying some relocations. DS form
relocations must preserve the lower two bits, and thus needs to inspect
the section data as it streams out.

Similarly, the overflow checking requires inspecting the primary
opcode to see if the value is sign or zero extended.

The existing PPC64 code no longer works as the slice returned by
(loader*).Data is cleared as we layout the symbol and process
relocations.  This data is always the section undergoing relocation,
thus we can directly inspect the contents to preserve bits or
check for overflows.

Change-Id: I239211f7e5e96208673663b6553b3017adae7e01
Reviewed-on: https://go-review.googlesource.com/c/go/+/300555
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-18 19:15:12 +00:00
Austin Clements eaa1ddee84 all: explode GOEXPERIMENT=regabi into 5 sub-experiments
This separates GOEXPERIMENT=regabi into five sub-experiments:
regabiwrappers, regabig, regabireflect, regabidefer, and regabiargs.
Setting GOEXPERIMENT=regabi now implies the working subset of these
(currently, regabiwrappers, regabig, and regabireflect).

This simplifies testing, helps derisk the register ABI project,
and will also help with performance comparisons.

This replaces the -abiwrap flag to the compiler and linker with
the regabiwrappers experiment.

As part of this, regabiargs now enables registers for all calls
in the compiler. Previously, this was statically disabled in
regabiEnabledForAllCompilation, but now that we can control it
independently, this isn't necessary.

For #40724.

Change-Id: I5171e60cda6789031f2ef034cc2e7c5d62459122
Reviewed-on: https://go-review.googlesource.com/c/go/+/302070
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
2021-03-18 16:51:27 +00:00
Cherry Zhang db4adb1a9b cmd/link: print symbol versions in stack bound check
When the stack bound check fails, print the call chain with
symbol versions (along with the names). Now that we have ABI
wrappers and wrappers do consume stack space, it is clearer to
distinguish the wrappers vs. the underlying functions.

Change-Id: Id1d922e3e7934b31317f233aff3d9667b6ac90c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/302869
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-03-18 14:09:57 +00:00
Brad Fitzpatrick 18510ae88f runtime, cmd/link/internal/ld: disable memory profiling when data unreachable
If runtime.MemProfile is unreachable, default to not collecting any
memory profiling samples, to save memory on the hash table.

Fixes #42347

Change-Id: I9a4894a5fc77035fe59b1842e1ec77a1182e70c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/299671
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Keith Randall <khr@golang.org>
2021-03-09 20:09:52 +00:00
Jason A. Donenfeld f41460145e cmd/link: recognize ARM64 PE files and relocations
For now, this only add a single relocation type, which is sufficient for
Windows resources. Later we'll see if we need more for cgo.

In order to ensure these code paths are actually tested, this expands
the rsrc tests to include all the architectures of PE objects that we
need to be recognizing, and splits things more clearly between binutils
and llvm objects, which have a slightly different layout, so that we
test both.

This CL is part of a stack adding windows/arm64
support (#36439), intended to land in the Go 1.17 cycle.

Change-Id: Ia1ee840265e9d12c0b12dd1c5d0810f8b300e557
Reviewed-on: https://go-review.googlesource.com/c/go/+/289429
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-02-26 18:15:09 +00:00
Jason A. Donenfeld 811167e2c9 cmd/link: do not pass -Bsymbolic for PE DLLs
This is only a valid option on ELF. Binutils accepts it, but LLVM
rejects it, so for Windows, it's best to just omit it.

Updates #44250.
Updates #39326.
Updates #38755.
Updates #36439.
Updates #43800.

Change-Id: Iffd2345d757f23dd737e63bd464cd412527077c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/291632
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-02-23 15:03:39 +00:00
Cherry Zhang 59703d53e2 [dev.regabi] cmd/link: stop using ABI aliases if wrapper is enabled
If ABI wrappers are enabled, we should not see ABI aliases at
link time. Stop resolving them. One exception is shared linkage,
where we still use ABI aliases as we don't always know the ABI
for symbols from shared libraries.

Change-Id: Ia89a788094382adeb4c4ef9b0312aa6e8c2f79ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/290032
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
2021-02-10 15:40:37 +00:00
Cherry Zhang a21de9ec73 [dev.regabi] cmd/link: resolve symbol ABI in shared linkage
In shared build mode and linkage, currently we assume all
function symbols are ABI0 (except for generated type algorithm
functions), and alias them to ABIInternal. When the two ABIs
actually differ (as it is now), this is not actually correct.
This CL resolves symbol ABI based on their mangled names.
If the symbol's name has a ".abi0" or ".abiinternal" suffix, it
is of the corresponding ABI. The symbol without the suffix is
the other ABI. For functions without ABI wrapper generated,
only one ABI exists but we don't know what it is, so we still
use alias (for now).

Change-Id: I2165f149bc83d513e81eb1eb4ee95464335b0e75
Reviewed-on: https://go-review.googlesource.com/c/go/+/289289
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-02-08 15:50:35 +00:00
Roland Shoemaker 953d1feca9 all: introduce and use internal/execabs
Introduces a wrapper around os/exec, internal/execabs, for use in
all commands. This wrapper prevents exec.LookPath and exec.Command from
running executables in the current directory.

All imports of os/exec in non-test files in cmd/ are replaced with
imports of internal/execabs.

This issue was reported by RyotaK.

Fixes CVE-2021-3115
Fixes #43783

Change-Id: I0423451a6e27ec1e1d6f3fe929ab1ef69145c08f
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/955304
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Katie Hockman <katiehockman@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/284783
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
2021-01-21 19:10:18 +00:00
Joel Sing d047c91a6c cmd/link,runtime: switch openbsd/amd64 to pthreads
This switches openbsd/amd64 to thread creation via pthreads, rather than doing
direct system calls.

Update #36435

Change-Id: I1105d5c392aa3e4c445d99c8cb80b927712e3529
Reviewed-on: https://go-review.googlesource.com/c/go/+/250180
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-01-19 12:49:13 +00:00
Cherry Zhang eb330020dc cmd/dist, cmd/go: pass -arch for C compilation on Darwin
On Apple Silicon Mac, the C compiler has an annoying default
target selection, depending on the ancestor processes'
architecture. In particular, if the shell or IDE is x86, when
running "go build" even with a native ARM64 Go toolchain, the C
compiler defaults to x86, causing build failures. We pass "-arch"
flag explicitly to avoid this situation.

Fixes #43692.
Fixes #43476.
Updates golang/vscode-go#1087.

Change-Id: I80b6a116a114e11e273c6886e377a1cc969fa3f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/283812
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-14 21:55:29 +00:00
Jason A. Donenfeld c9fb4eb0a2 cmd/link: handle grouped resource sections
The Go PE linker does not support enough generalized PE logic to
properly handle .rsrc sections gracefully. Instead a few things are
special cased for these. The linker also does not support PE's "grouped
sections" features, in which input objects have several named sections
that are sorted, merged, and renamed in the output file. In the past,
more sophisticated support for resources or for PE features like grouped
sections have not been necessary, as Go's own object formats are pretty
vanilla, and GNU binutils also produces pretty vanilla objects where all
sections are already merged.

However, GNU binutils is lagging with arm support, and here LLVM has
picked up the slack. In particular, LLVM has its own rc/cvtres combo,
which are glued together in mingw LLVM distributions as windres, a
command line compatible tool with binutils' windres, which supports arm
and arm64. But there's a key difference between binutils' windres and
LLVM's windres: the LLVM one uses proper grouped sections.

So, this commit adds grouped sections support for resource sections to
the linker. We don't attempt to plumb generic support for grouped
sections, just as there isn't generic support already for what resources
require. Instead we augment the resource handling logic to deal with
standard two-section resource objects.

We also add a test for this, akin to the current test for more vanilla
binutils resource objects, and make sure that the rsrc tests are always
performed.

Fixes #42866.
Fixes #43182.

Change-Id: I059450021405cdf2ef1c195ddbab3960764ad711
Reviewed-on: https://go-review.googlesource.com/c/go/+/268337
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-12-22 20:00:00 +00:00
Than McIntosh 740851baca cmd/link: avoid use of -T when linking with lld
When doing external linking on Windows, auto-detect the linker flavor
(bfd vs gold vs lld) and when linking with "lld", avoid the use of
"-T" (linker script), since this option is not supported by lld.
[Note: the Go linker currently employs -T to ensure proper placement
of the .debug_gdb_scripts section, to work around issues in older
versions of binutils; LLD recognizes this section and does place it
properly].

Updates #39326.

Change-Id: I3ea79cdceef2316bf86eccdb60188ac3655264ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/278932
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-12-18 00:16:17 +00:00
Cherry Zhang a318d56c1e cmd/link: pass arch-specific flags to external linker when testing supported flag
When testing if a flag (e.g. "-no-pie") is supported by the
external linker, pass arch-specific flags (like "-marm").

In particular, on the ARM builder, if CGO_LDFLAGS=-march=armv6
is set, the C toolchain fails to build if -marm is not passed.

	# cc -march=armv6 1.c
	1.c: In function 'main':
	1.c:3:1: sorry, unimplemented: Thumb-1 hard-float VFP ABI
	 int main() {
	 ^~~

This makes the Go linker think "-no-pie" is not supported when it
actually is.

Passing -marm makes it work.

Fixes #43202.

Change-Id: I4e8b71f08818993cbbcb2494b310c68d812d6b50
Reviewed-on: https://go-review.googlesource.com/c/go/+/278592
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-12-16 16:37:40 +00:00
Jason A. Donenfeld 033390d9ad cmd/link: recognize arm header of PE objects
The linker recognizes headers for 386 and amd64 PE objects, but not arm
objects. This is easily overlooked, since its the same as the 386 header
value, except the two nibbles of the first word are swapped. This commit
simply adds the check for this. Without it, .syso objects are rejected,
which means Windows binaries can't have resources built into them. At
the same time, we add comments to better indicate which condition
applies to which arch.

Change-Id: I210411d978504c1a9540e23abc5a180e24f159ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/268237
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-12-14 22:34:47 +00:00
Cherry Zhang 6f84993e90 cmd/link: code-sign on darwin/arm64
This CL lets the linker code-sign output binaries on
darwin/arm64, as the kernel requires binaries must be signed in
order to run.

This signature will likely be invalidated when we stamp the
buildid after linking. We still do it in the linker, for
- plain "go tool link" works.
- the linker generates the LC_CODE_SIGNATURE load command with
  the right size and offset, so we don't need to update it when
  stamping the buildid.

Updates #38485, #42684.

Change-Id: Ia306328906d73217221ba31093fe61a935a46122
Reviewed-on: https://go-review.googlesource.com/c/go/+/272256
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-12-01 23:14:14 +00:00
Quim Muntal e463c28cc1 cmd/link: avoid exporting all symbols on windows buildmode=pie
Marking one functions with __declspec(dllexport) forces mingw to
create .reloc section without having to export all symbols.

See https://insights.sei.cmu.edu/cert/2018/08/when-aslr-is-not-really-aslr---the-case-of-incorrect-assumptions-and-bad-defaults.html for more info.

This change cuts 73kb of a "hello world" pie binary.

Updates #6853
Fixes #40795

Change-Id: I3cc57c3b64f61187550bc8751dfa085f106c8475
Reviewed-on: https://go-review.googlesource.com/c/go/+/264459
Trust: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-11-02 00:46:44 +00:00
Meng Zhuo 7be9158ce5 cmd/link: remove all constants of elf
Use debug/elf instead.

Related:
CL 252478
CL 265317

Change-Id: If63b0458d9a6e825b40616bfb7a5a2c2e32402b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/265318
Trust: Meng Zhuo <mzh@golangcn.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-28 01:18:38 +00:00
Meng Zhuo 7eba75ab60 Revert "cmd/link: remove all constants of elf"
This reverts CL 252478.

Reason for revert: debug/Elfhdr has no Flags fields, some other CLs has removed it.

Change-Id: Ie199ac29f382c56aaf37a2e8338f2dafe6e79297
Reviewed-on: https://go-review.googlesource.com/c/go/+/265317
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Meng Zhuo <mzh@golangcn.org>
2020-10-27 05:19:46 +00:00
Meng Zhuo db7d42acac cmd/link: remove all constants of elf
Use debug/elf instead.

Change-Id: Ia6580648b6440e4a352f5c5ed59ac4d1c95e0175
Reviewed-on: https://go-review.googlesource.com/c/go/+/252478
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-27 05:07:21 +00:00
Meng Zhuo 7e25bdba5e cmd/link: use xcode strip for macho combine dwarf
The GNU strip will shrink text section while xcodetool strip don't.
We have to use xcodetool strip from system explicitly.

Fixes #41967

Change-Id: Ida372869e0ebc9e93f883640b1614836cea3672f
Reviewed-on: https://go-review.googlesource.com/c/go/+/262398
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Meng Zhuo <mzh@golangcn.org>
2020-10-23 15:54:44 +00:00
Joel Sing 15eaa870e1 cmd/link: add support for external linking on linux/riscv64
Fixes #36739

Change-Id: Id7573b343786360c72524f9f27d2a8f08d379cf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/243517
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-18 08:30:18 +00:00
Cherry Zhang f46a5b1e45 cmd/link: support PIE internal linking on darwin/amd64
This CL adds support of PIE internal linking on darwin/amd64.

This is also preparation for supporting internal linking on
darwin/arm64 (macOS), which requires PIE for everything.

Updates #38485.

Change-Id: I2ed58583dcc102f5e0521982491fc7ba6f2754ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/261642
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-14 00:19:23 +00:00
Cherry Zhang f8e554021b cmd/link: support C-shared buildmode on macOS/ARM64
It just works, after the plugin work.

Updates #38485.

Change-Id: I55aa11b380a33a729fccb731b77f48bc7d0dea2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/259443
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-06 22:56:00 +00:00
Cherry Zhang 7d6b304f12 cmd/link: support plugin on macOS/ARM64
Updates #38485.

Change-Id: I8295f7fad55b1f9701162f9d2902b3499137c64d
Reviewed-on: https://go-review.googlesource.com/c/go/+/259441
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-06 22:55:23 +00:00
Gerrit Code Review 89f687d6db Merge "cmd/link: merge branch 'dev.link' into master" 2020-10-01 15:16:43 +00:00
Quim 56dac60074 cmd/link: enable ASLR on windows binaries built with -buildmode=c-shared
Windows binaries built with -buildmode=c-shared set will have
IMAGE_DLLCHARACTERISTICS_DYNAMIC_BASE flag set, and
IMAGE_DLLCHARACTERISTICS_HIGH_ENTROPY_VA flag set for windows/amd64.

ASLR can be disabled on windows by using the new linker -aslr flag.

RELNOTE=yes

Fixes #41421

Change-Id: I62bd88c6d7e0f87173b093a0ad8e1a4d269ec790
Reviewed-on: https://go-review.googlesource.com/c/go/+/255259
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-01 14:26:35 +00:00
Jeremy Faller 91e4d2d57b [dev.link] Merge branch 'master' into dev.link
2 conflicts, that make sense.
	src/cmd/internal/obj/objfile.go
	src/cmd/link/internal/loader/loader.go

Change-Id: Ib224e2d248cb568fa1e888af79dd908b2f5e05ff
2020-09-30 18:00:58 -04:00
Cherry Zhang 7e54aa2c25 cmd/link: don't mark a symbol's GoType reachable when -linkshared
In CL 231397, we stopped marking symbols' GoType reachable in
general, but not when -linkshared. It was left as a TODO. This CL
addresses it.

The problem was that the type names are mangled in the shared
library, so we need to mangle the name consistently in the
executable as well (regardless of whether the symbol is reachable
or not), so that the GCProg generation code can find the
corresponding symbol from the shared library.

Change-Id: I1040747402929a983ec581109f1681a77893682e
Reviewed-on: https://go-review.googlesource.com/c/go/+/255964
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-18 19:41:09 +00:00
Cherry Zhang 3ab22052fb [dev.link] all: merge branch 'master' into dev.link
Clean merge.

Change-Id: Ib773b0bc00fd99d494f9331c3613bcc8285e48e3
2020-09-11 12:07:44 -04:00
Cherry Zhang 1e6ad65b43 cmd/link: enable DWARF combining on macOS ARM64
It appears the machoCalcStart function is meant to align the
segment, but it doesn't. Replace it with an actual alignment
calculation. Also, use the alignment from the configuration,
instead of hardcode.

With this fix we could enable DWARF combining on macOS ARM64.

Change-Id: I19ec771b77d752b83a54c53b6ee65af78a31b8ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/253558
Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-08 18:52:38 +00:00
Cherry Zhang e8f918535e cmd/link: strip STAB (symbolic debugging) symbols on darwin
On darwin, with external linking, the system linker produces STAB
(symbolic debugging) symbols in the binary's symbol table. These
include paths of the intermediate object files, like
<tmpdir>/go.o, which changes from run to run, making the build
non-reproducible.

Since we run dsymutil to produce debug info and combine them
back into the binary, we don't need those STAB symbols anymore.
Strip them after running dsymutil.

If DWARF is not enabled, we don't run dsymutil. We can pass
"-Wl,-S" to let the system linker not generate those symbols.

While here, also make it more consistent about DWARF combining.
Currently we only do DWARF combining on macOS/AMD64, when DWARF
is enabled. On ARM64, we run dsymutil, but then throw the result
away. This CL changes it to not run dsymutil (and strip) on
ARM64.

TODO: add a test. We don't do it here as it fails on some
(non-darwin) platforms.

Fixes #40979.

Change-Id: If770f7828cdb858857d6079e0585bf067f8f7a92
Reviewed-on: https://go-review.googlesource.com/c/go/+/250944
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-08 15:09:48 +00:00
Cherry Zhang 612b119447 cmd/link: pass darwin/amd64-specific flags only on AMD64
The linker assumed macOS is AMD64 (and 386 in the past). It
passes darwin/amd64-specific flags to the external linker when
building for macOS. They don't work for ARM64-based macOS. So
only pass them on AMD64.

Disable DWARF combining for macOS ARM64 for now. The generated
binary doesn't run. (TODO: fix.)

For macOS ARM64 port. External linking now works.

Change-Id: Iab53bc48f4fadd9b91de8898b4b450ea442667a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/253019
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-03 21:50:25 +00:00
Jeremy Faller 5402d40d5b [dev.link] cmd/link: fix memory growth on dev.link
CL 247399 caused memory growth in the linker. Fix this by adjusting how
we preallocate the number of symbols we'll need.

cmd/compile (Darwin), alloc/op:
Loadlib_GC                   33.5MB ± 0%    27.3MB ± 0%

Change-Id: I34997329ea4412716114df97fc9dad6ad0c171ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/249024
Run-TryBot: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-03 15:50:46 +00:00
Keith Randall 5c2c6d3fbf runtime: framepointers are no longer an experiment - hard code them
I think they are no longer experimental status. Might as well promote
them to permanent.

Change-Id: Id1259601b3dd2061dd60df86ee48080bfb575d2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/249857
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2020-08-27 21:15:47 +00:00
Jeremy Faller 5387cdcb24 [dev.link] cmd/link, cmd/compile: create content addressable pcdata syms
Switch pcdata over to content addressable symbols. This is the last
step before removing these from pclntab_old.

No meaningful benchmarks changes come from this work.

Change-Id: I3f74f3d6026a278babe437c8010e22992c92bd89
Reviewed-on: https://go-review.googlesource.com/c/go/+/247399
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-08-13 16:47:08 +00:00
Cherry Zhang 27e3778793 [dev.link] cmd: remove "2", another round
Rename the goobj2 package to goobj.

Change-Id: Iff97b5575cbac45ac44de96b6bd9d555b9a4a12a
Reviewed-on: https://go-review.googlesource.com/c/go/+/246444
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-08-11 18:32:23 +00:00
Cherry Zhang c4ee16eda9 [dev.link] all: merge branch 'master' into dev.link
Clean merge.

Change-Id: If78d97b9ac69511e4de7aa9532257d4fabebdcbc
2020-07-31 13:21:24 -04:00
Jeremy Faller 6ac9914383 [dev.link] create runtime.funcnametab
Move the function names out of runtime.pclntab_old, creating
runtime.funcnametab.  There is an unfortunate artifact in this change in
that calculating the funcID still requires loading the name. Future work
will likely pull this out and put it into the object file Funcs.

ls -l cmd/compile (darwin):
  before: 18524016
  after:  18519952

The difference in size can be attributed to alignment in pclntab_old.

Change-Id: Ibcbb230d4632178f8fcd0667165f5335786381f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/243223
Reviewed-by: Austin Clements <austin@google.com>
2020-07-31 13:55:07 +00:00
Cherry Zhang 4a92371291 [dev.link] cmd/link: remove "2", another round
Rename Reloc2 to Reloc, At2 to At, Aux2 to Aux.

Change-Id: Ic98d83c080e8cd80fbe1837c8f0aa134033508ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/245578
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-07-30 16:37:23 +00:00
Cherry Zhang 80b287fd28 [dev.link] cmd/link: remove loader.Reloc
We have Reloc and Reloc2. Reloc2 is the better approach and most
code uses Reloc2. There are still uses of Reloc. This CL migrates
them to Reloc2, and removes Reloc.

Change-Id: Id5f6a6019e1e044add682d05e70ebb1548ec58d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/245577
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-07-30 16:37:04 +00:00
Cherry Zhang 317c1ca9f2 [dev.link] cmd/link: refactor ExtReloc data structures
We used to generate all external relocations in memory, then emit
the relocation records at a later pass. The data structures were
chosen so that it takes as little memory as possible. Now we just
stream out external relocations, and ExtReloc is just a local
variable. Change the data structure to avoid repeated read of
some fields. Also get rid of ExtRelocView, as it is no longer
necessary.

Change-Id: I40209bbe4387af231b29788125c3b4ebb0ff4a33
Reviewed-on: https://go-review.googlesource.com/c/go/+/245479
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-07-30 16:36:25 +00:00
Cherry Zhang 19a932ceb8 cmd/link: don't mark shared library symbols reachable unconditionally
During the transitioning period, we mark symbols from Go shared
libraries reachable unconditionally. That might be useful when
there was still a large portion of the linker using sym.Symbols,
and only reachable symbols were converted to sym.Symbols. Marking
them reachable brings them to the dynamic symbol table, even if
they are not needed, increased the binary size unexpectedly.

That time has passed. Now we largely operate on loader symbols,
and it is not needed to mark them reachable anymore.

Fixes #40416.

Change-Id: I1e2bdb93a960ba7dc96575fabe15af93d8e95329
Reviewed-on: https://go-review.googlesource.com/c/go/+/244839
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-07-27 16:04:55 +00:00
Cherry Zhang 4b3cfcee58 [dev.link] cmd/link: remove non-streaming external relocation code
Now we support streaming external relocations everywhere.

Change-Id: I8d107c8239fe979bd5410e6a7f3fe471ac3e8b35
Reviewed-on: https://go-review.googlesource.com/c/go/+/244764
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-07-24 19:08:19 +00:00
Cherry Zhang bf1816c7b7 [dev.link] cmd/link: stream external relocations on ARM64 and on Darwin
Support streaming external relocations on ARM64. Support
architecture-specific relocations.

Also support streaming external relocations on Darwin. Do it in
the same CL so ARM64's archreloc doesn't need to support both
streaming and non-streaming.

Change-Id: Ia7fee9957892f98c065022c69a51f47402f4d6e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/243644
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-07-21 21:15:02 +00:00
Cherry Zhang f3a29d3a34 [dev.link] cmd/link: remove ARM64 special handling of GC data in shared libraries
The special case is no longer needed, didn't actually work, and
we no longer even save this map anywhere (see CL 240621 for more
information).

Change-Id: I19bcf32cace22decf50fd6414d4519cc51cbb0be
Reviewed-on: https://go-review.googlesource.com/c/go/+/241982
Reviewed-by: Austin Clements <austin@google.com>
2020-07-10 20:48:45 +00:00
Cherry Zhang 86f53c2a3c [dev.link] all: merge branch 'master' into dev.link
Change-Id: I644cfc9dfd6e024a0d62bbd226db8639e4a87dc7
2020-07-10 14:54:13 -04:00
Cherry Zhang 59a702aa6a [dev.link] cmd/link: emit Mach-O relocations in mmap
Following CL 240399 and CL 240400, do the same for Mach-O.

Linking cmd/compile with external linking,

name         old time/op    new time/op    delta
Asmb2_GC       32.7ms ± 2%    13.5ms ± 6%   -58.56%  (p=0.008 n=5+5)

name         old alloc/op   new alloc/op   delta
Asmb2_GC       16.5MB ± 0%     6.4MB ± 0%   -61.15%  (p=0.008 n=5+5)

Change-Id: I0fd7019d8713d1940e5fbbce4ee8eebd926451a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/241178
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-07-07 14:44:43 +00:00
Cherry Zhang 8c46cb1bf5 [dev.link] cmd/link: write ELF relocations in mmap on all architectures
In CL 240399 we changed to precompute the size for ELF relocation
records and use mmap to write them, but we left architectures
where elfreloc1 write non-fixed number of bytes. This CL handles
those architectures. When a Go relocation will turn into multiple
ELF relocations, in relocsym we account this difference and add
it to the size calculation. So when emitting ELF relocations, we
know the number of ELF relocations to be emitted.

Change-Id: I6732ab674b442f4618405e5412a77f6e4a3315d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/241079
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-07-06 19:58:23 +00:00
Cherry Zhang 36939aef77 [dev.link] cmd/link: parallelize ELF relocation writing
Now that we write ELF relocation records in mapped memory with
known sizes and offsets, we can write them in parallel.

Further speed up Asmb2 pass. Linking cmd/compile with external
linking,

Asmb2        141ms ± 4%      97ms ± 5%  -30.98%  (p=0.000 n=10+9)

Change-Id: I52c2b9230e90ed4421c21d7ef13a4f1e996f6054
Reviewed-on: https://go-review.googlesource.com/c/go/+/240400
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-07-05 23:57:04 +00:00