For FIPS init-time code+data verification, we need to arrange to
put the FIPS symbols into contiguous regions of the executable
and then record those sections along with the expected checksum.
The cmd/internal/obj changes identify the FIPS symbols and give
them distinguished types, which the linker then places in contiguous
regions. The linker also writes out information to use at run time
to find the FIPS sections, along with the expected hash.
See cmd/internal/obj/fips.go and cmd/link/internal/ld/fips.go
for more details.
The code is disabled in this commit.
CL 625998 and 625999 adds tests.
CL 626000 enables the code.
For #69536.
Change-Id: I48da6db94bc0bea7428c43d4abcf999527bccfcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/625997
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
These will be necessary when we start using the new FIPS symbols.
Split into a separate CL so that these refactoring changes can be
tested separate from any FIPS-specific changes.
Passes golang.org/x/tools/cmd/toolstash/buildall.
Change-Id: I73e5873fcb677f1f572f0668b4dc6f3951d822bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/625996
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
CL 402595 changes all usages of 16 bytes hash to 32 bytes hash by using
notsha256.
However, since CL 454836, notsha256 is not necessary anymore, so this CL
reverts those changes to 16 bytes hash using cmd/internal/hash package.
Updates #51940
Updates #64751
Change-Id: Ic015468ca4a49d0c3b1fb9fdbed93fddef3c838f
Reviewed-on: https://go-review.googlesource.com/c/go/+/610598
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
CL 402595 used notsha256 to prevent the compiler from depending on
cgo-based implementations of sha1 and sha256.
However, since CL 454836, cmd is built with CGO_ENABLED=0, which
will disable boringcrypto. Thus all usages of notsha256 is not necessary
anymore.
Updates #51940
Updates #64751
Change-Id: I503090f7a2efb5723e8a79523b143dc7cdb4edd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/610596
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
This CL adds a compiler directive go:wasmexport, which applies to
a Go function and makes it an exported function of the Wasm module
being built, so it can be called directly from the host. As
proposed in #65199, parameter and result types are limited to
32-bit and 64-bit integers and floats, and there can be at most
one result.
As the Go and Wasm calling conventions are different, for a
wasmexport function we generate a wrapper function does the ABI
conversion at compile time.
Currently this CL only adds basic support. In particular,
- it only supports executable mode, i.e. the Go wasm module calls
into the host via wasmimport, which then calls back to Go via
wasmexport. Library (c-shared) mode is not implemented yet.
- only supports wasip1, not js.
- if the exported function unwinds stacks (goroutine switch, stack
growth, etc.), it probably doesn't work.
TODO: support stack unwinding, c-shared mode, js.
For #65199.
Change-Id: Id1777c2d44f7d51942c1caed3173c0a82f120cc4
Reviewed-on: https://go-review.googlesource.com/c/go/+/603055
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Randy Reddig <randy.reddig@fastly.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This CL does some minor refactoring of the code handling
wasmimport.
- Put the WasmImport aux reading and writing code together for
symmetry.
- Define WasmFuncType, embedded in WasmImport. WasmFuncType could
also be used (later) for wasmexport.
- Move code generation code to a separate function. The containing
function is already pretty large.
- Simplify linker code a little bit. The loader convention is to
return the 0 Sym for nonexistent symbol, instead of a separate
boolean.
No change in generated code. Passes toolstash -cmp
(GOARCH=wasm GOOS=wasip1 go build -toolexec "toolstash -cmp" -a std cmd).
Change-Id: Idc2514f84a08621333841ae4034b81130e0ce411
Reviewed-on: https://go-review.googlesource.com/c/go/+/603135
Reviewed-by: Than McIntosh <thanm@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Currently, the SEH symbol is defined as an aux symbol of the
function symbol, without adding to ctxt.Data. Each function has
its own SEH symbol. As there are a lot of duplications of the
SEH symbol contents, currently a Go object file may contain many
copies of identical SEH symbols. They are deduplicated at link
time. But it does make the linker do redundant work, and make it
hard to reason about the SEH symbol writing in the object file
writer, and its resolution in the linker. In fact, in the object
file writer, the same SEH symbol may be added to the ctxt.defs
multiple times (as it is the aux of multiple function symbols),
which is not expected.
In fact, "aux symbol" is just a mechanism to associate auxiliary
data to another symbol. The auxiliary data symbol itself can be an
ordinary data symbol, even a content-addressable symbol. Define
the SEH symbol as a conntent-addressable symbol and add it to
ctxt.Data. This way there is only one definition of each unique
SEH symbol, which can be the aux of many functions.
While here, add a check to ensure that we add a symbol at most
once to the defs list.
Change-Id: Ie7a0cf02ca114060423e025931b30de97ca330fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/585656
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
The old Go object file format used linker symbols like "gofile..foo"
to record references to the filename "foo". But the current object
file format has a dedicated section for file names, so we don't need
these useless prefixes anymore.
Also, change DWARF generation to pass around the src.Pos directly,
rather than the old file symbols, which it just turned back into a
file index before writing out anyway.
Finally, directly record the FileIndex into src.PosBase, so that we
can skip the map lookups.
Change-Id: Ia4a5ebfa95da271f2522e45befdb9f137c16d373
Reviewed-on: https://go-review.googlesource.com/c/go/+/523378
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
cmd/asm and cmd/compile now always create symbols with the appropriate
package prefixes, so cmd/internal/dwarf and cmd/internal/obj can stop
worrying about qualifying names itself.
Change-Id: I9aee5d759bf0d41a61722c777e7f66fce957e79e
Reviewed-on: https://go-review.googlesource.com/c/go/+/523338
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Load large constants into vector registers from rodata, instead of placing them
in the literal pool. This treats VMOVQ/VMOVD/VMOVS the same as FMOVD/FMOVS and
makes use of the existing mechanism for storing values in rodata. Two additional
instructions are required for a load, however these instructions are used
infrequently and already have a high latency.
Updates #59615
Change-Id: I54226730267689963d73321e548733ae2d66740e
Reviewed-on: https://go-review.googlesource.com/c/go/+/515617
Reviewed-by: Eric Fang <eric.fang@arm.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
sort.Slice was being used to sort some newly added entries by name
to make the ctxt.Data slice reproducible, but some existing entries
have the same name, and the effect was to take the non-determinism
of the tail entries and scatter it into the earlier, deterministic section
when multiple entries had the same name.
The specific entries with the same name are type SDWARFVAR, which
all have an empty name but different relocations. If they are shuffled,
then the relocation symbols are visited in a different order, which
enters them into the string table in a different order, which results in
different object files, different object file hashes, and different build IDs
for the final executables.
Use sort.SliceStable to avoid reordering entries we don't mean to reorder.
Also add a simple test for scheduling-related non-determinism.
I debugged this originally using 'go install -race cmd/compile',
but that was slow and turned out not to be terribly reliable.
Using a few different GOMAXPROCS settings turns out to be a much more
effective (and faster) way to scramble scheduling decisions.
Fixes#60759.
Change-Id: Ia966b02b9fdaefa971f998a09319ca375bdf8604
Reviewed-on: https://go-review.googlesource.com/c/go/+/502755
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Currently, for a global variable, its debug info symbol is a named
symbol with the variable's name with a special prefix. And the
linker looks it up by name. This CL makes the debug info symbol an
aux symbol of the variable symbol.
Change-Id: I55614d0ef2af9c53eb40144ad80e09339bf3cbee
Reviewed-on: https://go-review.googlesource.com/c/go/+/490816
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
This CL updates the Go compiler so it generate SEH unwind info [1] as a
function auxiliary symbol when building for windows/amd64.
A follow up CL will teach the Go linker how to assemble these codes
into the PE .xdata section.
Updates #57302
[1] https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64#struct-unwind_info
Change-Id: I40ae0437bfee326c1a67c2b5e1496f0bf3ecea17
Reviewed-on: https://go-review.googlesource.com/c/go/+/461749
Reviewed-by: Davis Goodin <dagood@microsoft.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Go programs can now use the //go:wasmimport module_name function_name
directive to import functions from the WebAssembly runtime.
For now, the directive is restricted to the runtime and syscall/js
packages.
* Derived from CL 350737
* Original work modified to work with changes to the IR conversion code.
* Modification of CL 350737 changes to fully exist in Unified IR path (emp)
* Original work modified to work with changes to the ABI configuration code.
* Fixes#38248
Co-authored-by: Vedant Roy <vroy101@gmail.com>
Co-authored-by: Richard Musiol <mail@richard-musiol.de>
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Change-Id: I740719735d91c306ac718a435a78e1ee9686bc16
Reviewed-on: https://go-review.googlesource.com/c/go/+/463018
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Make linkgetlineFromPos and getFileIndexAndLine methods on Link, and
give the former a more descriptive name.
The docs are expanded to make it more clear that these are final
file/line visible in programs.
In getFileSymbolAndLine use ctxt.InnermostPos instead of ctxt.PosTable
direct, which makes it more clear that we want the semantics of
InnermostPos.
Change-Id: I7c3d344dec60407fa54b191be8a09c117cb87dd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/446301
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
It's all local to a single file and responsible for 1.7% of total
space allocated summed over compilation of the bent benchmarks.
Showing nodes accounting for 27.16GB, 27.04% of 100.44GB total
Dropped 1622 nodes (cum <= 0.50GB)
Showing top 10 nodes out of 321
flat flat% sum% cum cum%
4.87GB 4.85% 4.85% 4.87GB 4.85% cmd/compile/internal/objw.init
4.79GB 4.77% 9.62% 4.81GB 4.79% runtime.allocm
4.72GB 4.70% 14.32% 4.72GB 4.70% cmd/compile/internal/types.newType
3.10GB 3.09% 17.41% 3.17GB 3.15% cmd/compile/internal/ssagen.InitConfig
1.86GB 1.85% 19.26% 2.61GB 2.60% cmd/compile/internal/ssa.cse
1.72GB 1.71% 20.97% 2.25GB 2.24% cmd/internal/obj.(*Link).traverseFuncAux
1.66GB 1.65% 22.62% 1.66GB 1.65% runtime.malg
1.61GB 1.61% 24.23% 1.64GB 1.63% cmd/compile/internal/ssa.schedule
1.42GB 1.41% 25.64% 1.42GB 1.41% cmd/compile/internal/ir.NewNameAt
Change-Id: Id18ee3b83cb23a7042d59714a0c1ca074e7bc7a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/437297
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: David Chase <drchase@google.com>
We used to use SHA1 for content hashes, but CL 402595 changed
all the “don't care” hashes to cmd/internal/notsha256 (negated SHA256).
This made object files a little bit bigger: fmt.a on my Mac laptop grows
from 910678 to 937612 bytes (+3%).
To remove that growth, truncate the hash we use for these purposes
to 128 bits (half a SHA256), and also use base64 instead of hex for
encoding it when a string form is needed. This brings fmt.a down to
901706 bytes (-1% from original, -4% from current).
Change-Id: Id81da1cf3ee85ed130b3cda73aa697d8c0053a62
Reviewed-on: https://go-review.googlesource.com/c/go/+/404294
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
When we add GOEXPERIMENT=boringcrypto, the bootstrap process
will not converge if the compiler itself depends on the boringcrypto
cgo-based implementations of sha1 and sha256.
Using notsha256 avoids boringcrypto and makes bootstrap converge.
Removing md5 is not strictly necessary but it seemed worthwhile to
be consistent.
For #51940.
Change-Id: Iba649507e0964d1a49a1d16e463dd23c4e348f14
Reviewed-on: https://go-review.googlesource.com/c/go/+/402595
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
It'll be used in second place in a subsequent change.
No functional changes.
Change-Id: I58dd12d7dde45b36995d031fc7fbb27d6eaf48d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/353670
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Trust: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
When writing an object file, most symbols are indexed in
NumberSyms. Currently, pcdata symbols are indexed late and
separately. This is not really necessary, as pcdata symbols
already exist at the time of NumberSyms. Just do it there.
As pcdata symbols are laid out in the pclntab in a special way at
link time, distinguish them from other symbols in the content
hash. (In the old code this was partly achieved by indexing them
late.)
Change-Id: Ie9e721382b0af2cfb39350d031e2e66d79095a3c
Reviewed-on: https://go-review.googlesource.com/c/go/+/352611
Trust: Cherry Mui <cherryyz@google.com>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
The goal of this change is to improve the documentation
and make it easier to keep Link.NumberSyms and writer.contentHash aligned.
No functional changes.
A subsequent change will add conditions to contentHashSection.
Change-Id: I0a274f6974459d34d5a8553081f33ea4cd87f248
Reviewed-on: https://go-review.googlesource.com/c/go/+/352669
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Change-Id: I8a51f054e017e0116dee4e435b60c08d72e998e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/351331
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The go/build package needs access to this configuration,
so move it into a new package available to the standard library.
Change-Id: I868a94148b52350c76116451f4ad9191246adcff
Reviewed-on: https://go-review.googlesource.com/c/go/+/310731
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Use sym.Size, instead of len(sym.P), to decide whether a
content-addressable symbol is "short" and hashed as Hashed64.
So we don't dedup a small symbol with a gigantic almost-zero
symbol.
Fixes#42140.
Change-Id: Ic65869e1eaf51947517b3ece49c8b0be1b94bb75
Reviewed-on: https://go-review.googlesource.com/c/go/+/264337
Trust: Cherry Zhang <cherryyz@google.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
This creates space for a different kind of extension field
in LSym without making the struct any larger.
(There are many LSym, so we care about keeping the struct small.)
Change-Id: Ib16edb9e15f54c2a7351c8b875e19684058711e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/243943
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Type namedata symbols are for type/field/method names and package
paths. We can use content-addressable symbol mechanism for them.
Change-Id: I923fda17b7094c7a0e46aad7c450622eb3826294
Reviewed-on: https://go-review.googlesource.com/c/go/+/257960
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
I think they are no longer experimental status. Might as well promote
them to permanent.
Change-Id: Id1259601b3dd2061dd60df86ee48080bfb575d2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/249857
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
CL 245485 introduced a map for used files in a function. When
numbering symbols, make sure we traverse the files in
deterministic order.
Should fix longtest builders.
Change-Id: I1006bc5425116ab40e33a61e8f5acd1bdb4abad9
Reviewed-on: https://go-review.googlesource.com/c/go/+/247997
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In order to prevent renumbering of filenames in pclntab generation, use
the per-package file list (previously only used for DWARF generation) as
file-indices. This is the largest step to eliminate renumbering of
filenames in pclntab.
Note, this is probably not the final state of the file table within the
object file. In this form, the linker loads all filenames for all
objects. I'll move to storing the filenames as regular string
symbols,and defaulting all string symbols to using the larger hash value
to make generation of pcln simplest, and most memory friendly.
Change-Id: I23daafa3f4b4535076e23100200ae0e7163aafe0
Reviewed-on: https://go-review.googlesource.com/c/go/+/245485
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
For content-addressable symbols with relocations, we build a
content hash based on its content and relocations. Depending on
the category of the referenced symbol, we choose different hash
algorithms such that the hash is globally consistent.
For now, we only support content-addressable symbols with
relocations when the current package's import path is known, so
that the symbol names are fully expanded. Otherwise, if the
referenced symbol is a named symbol whose name is not fully
expanded, the hash won't be globally consistent, and can cause
erroneous collisions. This is fine for now, as the deduplication
is just an optimization, not a requirement for correctness (until
we get to type descriptors).
Change-Id: I639e4e03dd749b5d71f0a55c2525926575b1ac30
Reviewed-on: https://go-review.googlesource.com/c/go/+/243142
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
For now, we only do this for symbols without relocations.
Mark static temps "local", as they are not referenced across DSO
boundaries. And deduplicating a local symbol and a non-local
symbol can be problematic.
Change-Id: I0a3dc4138aaeea7fd4f326998f32ab6305da8e4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/243141
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
For symbols of size 8 bytes or below, we can map them to 64-bit
hash values using the identity function. There is no need to use
longer and more expensive hash functions.
For them, we introduce another pseudo-package, PkgIdxHashed64. It
is like PkgIdxHashed except that the hash function is different.
Note that the hash value is not affected with trailing zeros,
e.g. "A" and "A\0\0\0" have the same hash value. This allows
deduplicating a few more symbols. When deduplicating them, we
need to keep the longer one.
Change-Id: Iad0c2e9e569b6a59ca6a121fb8c8f0c018c6da03
Reviewed-on: https://go-review.googlesource.com/c/go/+/242362
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Fill in the data at compile time, and get rid of the preprocess
function in the linker.
We need to be careful with symbol alignment: data symbols are
generally naturally aligned, except for string symbols which are
not aligned. When deduplicating two symbols with same content but
different alignments, we need to keep the biggest alignment.
Change-Id: I4bd96adfdc5f704b5bf3a0e723457c9bfe16a684
Reviewed-on: https://go-review.googlesource.com/c/go/+/242081
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
This CL introduces content-addressable symbols (a.k.a. hashed
symbols) to object files. Content-addressable symbols are
identified and referenced by their content hashes, instead of by
names.
In the object file, a new pseudo-package index PkgIdxHashed is
introduced, for content-addressable symbols, and a new block is
added to store their hashes. The hashes are used by the linker to
identify and deduplicate the symbols.
For now, we only support content-addressable symbols that are
always locally defined (i.e. no cross-package references).
As a proof of concept, make string constant symbols content-
addressable.
Change-Id: Iaf53efd74c0ffb54fa95f784628cc84e95844536
Reviewed-on: https://go-review.googlesource.com/c/go/+/242079
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Mark compiler-generated ".stmp_%d" and "<fn>.stkobj" symbols as
AttrStatic, so as to tell the linker that they do not need to be
inserted into its name lookup tables.
Change-Id: I59ffd11659b2c54c2d0ad41275d05c3f919e3b88
Reviewed-on: https://go-review.googlesource.com/c/go/+/240497
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Merge conflicts are mostly recently changed nm/objdump output
format and its tests. Resolved easily (mostly just using the
format on master branch).
Change-Id: I99d8410a9a02947ecf027d9cae5762861562baf5
Currently, the compiler generates a fingerprint for each package,
which is used by the linker for index consistency check.
When building plugin or shared object, currently the linker also
generates a hash, by hashing the export data. At run time, when
a package is referenced by multiple DSOs, this hash is compared
to ensure consistency.
It would be good if we can unify this two hashes. This way, the
linker doesn't need to read the export data (which is intended
for the compiler only, and is not always available for the
linker). The export data hash is sufficient for both purposes.
It is consistent with the current hash geneated by the linker.
And the export data includes indices for exported symbols, so its
hash can be used to catch index mismatches.
Updates #33820.
Change-Id: I2bc0d74930746f54c683a10dfd695d50ea3f5a38
Reviewed-on: https://go-review.googlesource.com/c/go/+/236118
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Now we have ctxt.IsAsm, use that, instead of passing in a
parameter.
Change-Id: I81dedbe6459424fa9a4c2bfbd9abd83d83f3a107
Reviewed-on: https://go-review.googlesource.com/c/go/+/234492
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
We are not going to merge to master until Go 1.16 cycle. The old
object support can go now.
Change-Id: I93e6f584974c7749d0a0c2e7a96def35134dc566
Reviewed-on: https://go-review.googlesource.com/c/go/+/231918
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The new object files use indices for symbol references, instead
of names. Fundamental to the design, it requires that the
importing and imported packages have consistent view of symbol
indices. The Go command should already ensure this, when using
"go build". But in case it goes wrong, it could lead to obscure
errors like run-time crashes. It would be better to check the
index consistency at build time.
To do that, we add a fingerprint to each object file, which is
a hash of symbol indices. In the object file it records the
fingerprints of all imported packages, as well as its own
fingerprint. At link time, the linker checks that a package's
fingerprint matches the fingerprint recorded in the importing
packages, and issue an error if they don't match.
This CL does the first part: introducing the fingerprint in the
object file, and propagating fingerprints through
importing/exporting by the compiler. It is not yet used by the
linker. Next CL will do.
Change-Id: I0aa372da652e4afb11f2867cb71689a3e3f9966e
Reviewed-on: https://go-review.googlesource.com/c/go/+/229617
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Add back the newobj flag, renamed to go115newobj, for feature
gating. The flag defaults to true.
This essentially reverts CL 206398 as well as CL 220060.
The old object format isn't working yet. Will fix in followup CLs.
Change-Id: I1ace2a9cbb1a322d2266972670d27bda4e24adbc
Reviewed-on: https://go-review.googlesource.com/c/go/+/224623
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Switch the primary subprogram die DWARF symbol emitted by the compiler
from named+dupOK to anonymous aux. This should help performance wise
by not having to add these symbols to the linker's symbol name lookup
tables.
Change-Id: Idf66662b8bf60b3dee9a55e6cd5137b24a9f5ab6
Reviewed-on: https://go-review.googlesource.com/c/go/+/223669
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Generalize symbol traversal code for aux symbols to allow for client
control over whether the walk incldues symbols referenced by
relocations on visited aux syms. This is not needed just yet but will
be required in order to support anonymous aux syms that have
relocations.
Change-Id: I898c1f398213c8d9d777dd3c40524a013b25e348
Reviewed-on: https://go-review.googlesource.com/c/go/+/223668
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Convert DWARF .debug_line symbols to anonymous aux syms, so as
to save space in object files and reduce the number of symbols
that have to be added to the linker's lookup tables.
Change-Id: I5b350f036e21a7a7128cb08148ab7c243aaf0d0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/223018
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
For compiler developers interested in seeing DWARF generation details,
this patch provides symbol "debug asm" dumps for DWARF aux symbols
when -S=2 is in effect.
Change-Id: I5a0b6b65ce7b708948cbbf23c6b0d279bd4f8d9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/223017
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
There are more cleanups to do, but I want to keep this CL mostly
a pure deletion.
Change-Id: Icd2ff0a4b648eb4adf3d29386542617e49620818
Reviewed-on: https://go-review.googlesource.com/c/go/+/206398
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Russ Cox
Cuong Manh Le
Cherry Mui
Matthew Dempsky
Joel Sing
qmuntal
Evan Phoenix
cui fliter
Michael Pratt
David Chase
Josh Bleecher Snyder
John Bampton
Keith Randall
Jeremy Faller
Than McIntosh
Tobias Klauser