go/src
Cherry Mui 21c2fdd91c cmd/link: use slice and bitmap for some attributes
Currently, a symbol's outer symbol, the "special" attribute, and
whether a symbol is a generator symbol are represented as maps,
and are accessed in some loops over nearly all reachable symbols.
The map lookups are a bit expensive.

For outer symbol, a non-trivial portion of the symbols have outer
symbol set (e.g. type symbols, which we put into container symbols
like "type:*"). Using a slice to access more efficiently.

For the special and generator symbol attributes, use a bitmap.
There are not many symbols have those attributes, so the bitmap is
quite sparse. The bitmap is not too large anyway, so use it for
now. If we want to further reduce memory usage we could consider
some other data structure like a Bloom filter.

Linking cmd/compile in external linking mode (on macOS/amd64)

Symtab   12.9ms ± 9%     6.4ms ± 5%   -50.08%  (p=0.000 n=19+18)
Dodata   64.9ms ±12%    57.1ms ±12%   -11.90%  (p=0.000 n=20+20)
Asmb     36.7ms ±11%    32.8ms ± 9%   -10.61%  (p=0.000 n=20+18)
Asmb2    26.6ms ±15%    21.9ms ±12%   -17.75%  (p=0.000 n=20+18)

There is some increase of memory usage

Munmap_GC   40.9M ± 1%     43.2M ± 0%    +5.54%  (p=0.000 n=20+19)

The next CL will bring the memory usage back.

Change-Id: Ie4347eb96c51f008b9284270de37fc880bb52d2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/487415
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2023-04-24 14:00:39 +00:00
..
archive
arena
bufio
builtin
bytes std: fix various nilness findings 2023-04-20 01:07:29 +00:00
cmd cmd/link: use slice and bitmap for some attributes 2023-04-24 14:00:39 +00:00
compress
container
context context: add AfterFunc 2023-04-19 19:13:01 +00:00
crypto crypto/tls: fix PSK binder calculation 2023-04-24 13:35:52 +00:00
database/sql
debug debug/elf: support zstd compression 2023-04-18 20:34:36 +00:00
embed embed/internal/embedtest: test openFile.ReadAt with non-zero offset 2023-04-14 20:10:16 +00:00
encoding std: fix various nilness findings 2023-04-20 01:07:29 +00:00
errors
expvar
flag flag: panic if a flag is defined after being set 2023-04-21 22:25:02 +00:00
fmt
go go/types, types2: abort type unification if no progress is made 2023-04-21 12:32:24 +00:00
hash
html all: re-run stringer 2023-04-11 20:24:07 +00:00
image
index/suffixarray
internal internal/testenv: in HasExec, try to actually exec on ios and wasm platforms 2023-04-22 00:41:24 +00:00
io io: ReadAll: do not check for realloc in first round 2023-04-20 16:22:09 +00:00
log log: avoid leaking goroutines in TestOutputRace 2023-04-21 20:04:37 +00:00
maps maps: replace slicesEqual with slices.Equal 2023-04-18 13:37:52 +00:00
math cmd/internal/obj/ppc64: modify PCALIGN to ensure alignment 2023-04-21 16:47:45 +00:00
mime cmd/go: add check for unknown godebug setting 2023-04-18 13:19:19 +00:00
net std: fix various nilness findings 2023-04-20 01:07:29 +00:00
os os/user: skip tests that invoke Current if it returns an expected error 2023-04-21 15:59:43 +00:00
path path/filepath,cmd/go: skip tests involving double-symlinks on ios 2023-04-14 17:01:07 +00:00
plugin plugin: fix duplicated word in comment 2023-04-18 13:43:02 +00:00
reflect reflect: remove typedmemmovepartial as it is unused 2023-04-10 18:58:50 +00:00
regexp regexp: fix copy-paste typo on Regexp.UnmarshalText doc 2023-04-14 16:28:44 +00:00
runtime runtime: call _exit, not exit, on AIX and Solaris 2023-04-24 05:12:17 +00:00
slices slices: optimize Index and Compact for large types 2023-04-19 19:09:29 +00:00
sort
strconv
strings
sync
syscall syscall: add a sync.Pool for a NetlinkRIB scratch buffer 2023-04-18 14:42:52 +00:00
testdata
testing std: fix various nilness findings 2023-04-20 01:07:29 +00:00
text
time time: add wasip1 support 2023-04-11 20:56:10 +00:00
unicode
unsafe
vendor all: update vendored golang.org/x/net 2023-04-11 22:42:18 +00:00
Make.dist
README.vendor
all.bash make.{bash,bat}: check unmodified $PATH for $GOROOT/bin presence 2023-04-19 14:36:22 +00:00
all.bat make.{bash,bat}: check unmodified $PATH for $GOROOT/bin presence 2023-04-19 14:36:22 +00:00
all.rc
bootstrap.bash
buildall.bash
clean.bash
clean.bat
clean.rc
cmp.bash
go.mod all: update vendored golang.org/x/net 2023-04-11 22:42:18 +00:00
go.sum all: update vendored golang.org/x/net 2023-04-11 22:42:18 +00:00
make.bash
make.bat cmd/dist: add -distpack flag to build distribution archives 2023-04-11 21:29:13 +00:00
make.rc
race.bash
race.bat
run.bash
run.bat
run.rc

README.vendor

Vendoring in std and cmd
========================

The Go command maintains copies of external packages needed by the
standard library in the src/vendor and src/cmd/vendor directories.

There are two modules, std and cmd, defined in src/go.mod and
src/cmd/go.mod. When a package outside std or cmd is imported
by a package inside std or cmd, the import path is interpreted
as if it had a "vendor/" prefix. For example, within "crypto/tls",
an import of "golang.org/x/crypto/cryptobyte" resolves to
"vendor/golang.org/x/crypto/cryptobyte". When a package with the
same path is imported from a package outside std or cmd, it will
be resolved normally. Consequently, a binary may be built with two
copies of a package at different versions if the package is
imported normally and vendored by the standard library.

Vendored packages are internally renamed with a "vendor/" prefix
to preserve the invariant that all packages have distinct paths.
This is necessary to avoid compiler and linker conflicts. Adding
a "vendor/" prefix also maintains the invariant that standard
library packages begin with a dotless path element.

The module requirements of std and cmd do not influence version
selection in other modules. They are only considered when running
module commands like 'go get' and 'go mod vendor' from a directory
in GOROOT/src.

Maintaining vendor directories
==============================

Before updating vendor directories, ensure that module mode is enabled.
Make sure that GO111MODULE is not set in the environment, or that it is
set to 'on' or 'auto'.

Requirements may be added, updated, and removed with 'go get'.
The vendor directory may be updated with 'go mod vendor'.
A typical sequence might be:

    cd src
    go get golang.org/x/net@latest
    go mod tidy
    go mod vendor

Use caution when passing '-u' to 'go get'. The '-u' flag updates
modules providing all transitively imported packages, not only
the module providing the target package.

Note that 'go mod vendor' only copies packages that are transitively
imported by packages in the current module. If a new package is needed,
it should be imported before running 'go mod vendor'.