Commit Graph

65 Commits

Author SHA1 Message Date
Ian Lance Taylor 122a22e0e9 internal/syscall/unix: use runtime.gostring for Gostring
Under the race detector, checkptr flags uses of unsafe.Slice that
result in slices that straddle multiple Go allocations.
Avoid that scenario by calling existing runtime code.

This fixes a failure on the darwin-.*-race builders introduced in
CL 446178.

Change-Id: I6e0fdb37e3c3f38d97939a8799bb4d10f519c5b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/449936
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
2022-11-11 23:24:12 +00:00
Cuong Manh Le 7ffbcd1987 runtime: replace stringStruct with unsafe.String where appropriate
Simplify the code a bit, no significant performance changes.

name                             old time/op    new time/op    delta
HashStringSpeed-8                  9.64ns ±11%    8.91ns ± 9%   -7.60%  (p=0.007 n=10+10)
HashStringArraySpeed-8             19.8ns ± 3%    19.5ns ± 2%     ~     (p=0.085 n=10+10)
MapStringKeysEight_16-8            10.7ns ± 3%    10.2ns ± 2%   -4.48%  (p=0.000 n=10+10)
MapStringKeysEight_32-8            8.89ns ± 3%    8.71ns ± 3%     ~     (p=0.082 n=9+10)
MapStringKeysEight_64-8            8.84ns ± 2%    8.60ns ± 3%   -2.73%  (p=0.004 n=9+10)
MapStringKeysEight_1M-8            8.90ns ± 3%    8.62ns ± 3%   -3.15%  (p=0.000 n=10+10)
MapStringConversion/32/simple-8    8.62ns ± 3%    8.61ns ± 2%     ~     (p=0.895 n=10+10)
MapStringConversion/32/struct-8    8.53ns ± 2%    8.63ns ± 2%     ~     (p=0.123 n=10+10)
MapStringConversion/32/array-8     8.54ns ± 2%    8.50ns ± 1%     ~     (p=0.590 n=9+9)
MapStringConversion/64/simple-8    8.44ns ± 2%    8.38ns ± 2%     ~     (p=0.353 n=10+10)
MapStringConversion/64/struct-8    8.41ns ± 2%    8.48ns ± 2%     ~     (p=0.143 n=10+10)
MapStringConversion/64/array-8     8.42ns ± 2%    8.44ns ± 2%     ~     (p=0.739 n=10+10)
MapInterfaceString-8               13.6ns ±26%    13.6ns ±20%     ~     (p=0.736 n=10+9)
AppendGrowString-8                 38.9ms ± 9%    40.2ms ±13%     ~     (p=0.481 n=10+10)
CompareStringEqual-8               3.03ns ± 2%    2.86ns ± 3%   -5.58%  (p=0.000 n=10+10)
CompareStringIdentical-8           1.20ns ± 3%    1.01ns ± 4%  -16.16%  (p=0.000 n=10+10)
CompareStringSameLength-8          2.11ns ± 3%    1.85ns ± 3%  -12.33%  (p=0.000 n=10+10)
CompareStringDifferentLength-8     0.30ns ± 0%    0.30ns ± 0%     ~     (p=0.508 n=10+9)
CompareStringBigUnaligned-8        43.0µs ± 1%    42.8µs ± 2%     ~     (p=0.165 n=10+10)
CompareStringBig-8                 43.2µs ± 2%    43.4µs ± 2%     ~     (p=0.661 n=9+10)
ConcatStringAndBytes-8             15.1ns ± 1%    14.9ns ± 1%   -1.57%  (p=0.001 n=8+10)
SliceByteToString/1-8              2.45ns ± 2%    2.39ns ± 2%   -2.64%  (p=0.000 n=10+10)
SliceByteToString/2-8              10.9ns ± 2%    10.8ns ± 4%     ~     (p=0.060 n=10+10)
SliceByteToString/4-8              11.9ns ± 0%    11.8ns ± 1%   -0.97%  (p=0.000 n=8+8)
SliceByteToString/8-8              13.9ns ± 1%    13.9ns ± 1%   +0.57%  (p=0.009 n=9+9)
SliceByteToString/16-8             18.0ns ± 3%    18.6ns ± 5%   +2.78%  (p=0.001 n=9+10)
SliceByteToString/32-8             20.1ns ± 3%    20.5ns ± 5%   +2.10%  (p=0.034 n=10+10)
SliceByteToString/64-8             24.3ns ± 3%    24.9ns ± 3%   +2.28%  (p=0.001 n=9+10)
SliceByteToString/128-8            33.8ns ± 1%    34.5ns ± 4%     ~     (p=0.264 n=8+10)

Updates #54854

Change-Id: I7ce57a92c5f590fa8cb31a48969d281147eb05f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/428759
Reviewed-by: hopehook <hopehook@golangcn.org>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-09-13 16:42:28 +00:00
Abirdcfly 64f2829c9c runtime: fix typo in function comments
The correct word to use here is 'receive' not 'recieve'

Change-Id: Ia33e2a91c1c2da6dbe479a05518dbb9b8733d20d
GitHub-Last-Rev: e3c3e211dd
GitHub-Pull-Request: golang/go#54025
Reviewed-on: https://go-review.googlesource.com/c/go/+/419316
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
2022-07-24 21:05:02 +00:00
Russ Cox 946b4baaf6 all: gofmt main repo
Excluding vendor and testdata.
CL 384268 already reformatted most, but these slipped past.

The struct in the doc comment in debug/dwarf/type.go
was fixed up by hand to indent the first and last lines as well.

For #51082.

Change-Id: Iad020f83aafd671ff58238fe491907e85923d0c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/407137
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-19 15:49:05 +00:00
Michael Anthony Knyszek 0feebe6eb5 runtime: add byte count parser for GOMEMLIMIT
This change adds a parser for the GOMEMLIMIT environment variable's
input. This environment variable accepts a number followed by an
optional prefix expressing the unit. Acceptable units include
B, KiB, MiB, GiB, TiB, where *iB is a power-of-two byte unit.

For #48409.

Change-Id: I6a3b4c02b175bfcf9c4debee6118cf5dda93bb6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/393400
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-05-03 15:11:55 +00:00
Russ Cox 19309779ac all: gofmt main repo
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]

Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.

For #51082.

Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-11 16:34:30 +00:00
Russ Cox 9839668b56 all: separate doc comment from //go: directives
A future change to gofmt will rewrite

	// Doc comment.
	//go:foo

to

	// Doc comment.
	//
	//go:foo

Apply that change preemptively to all comments (not necessarily just doc comments).

For #51082.

Change-Id: Iffe0285418d1e79d34526af3520b415a12203ca9
Reviewed-on: https://go-review.googlesource.com/c/go/+/384260
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-05 17:54:15 +00:00
Russ Cox 7d87ccc860 all: fix various doc comment formatting nits
A run of lines that are indented with any number of spaces or tabs
format as a <pre> block. This commit fixes various doc comments
that format badly according to that (standard) rule.

For example, consider:

	// - List item.
	//   Second line.
	// - Another item.

Because the - lines are unindented, this is actually two paragraphs
separated by a one-line <pre> block. This CL rewrites it to:

	//  - List item.
	//    Second line.
	//  - Another item.

Today, that will format as a single <pre> block.
In a future release, we hope to format it as a bulleted list.

Various other minor fixes as well, all in preparation for reformatting.

For #51082.

Change-Id: I95cf06040d4186830e571cd50148be3bf8daf189
Reviewed-on: https://go-review.googlesource.com/c/go/+/384257
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01 18:18:01 +00:00
fanzha02 6f327f7b88 runtime, syscall: add calls to asan functions
Add explicit address sanitizer instrumentation to the runtime and
syscall packages. The compiler does not instrument the runtime
package. It does instrument the syscall package, but we need to add
a couple of cases that it can't see.

Refer to the implementation of the asan malloc runtime library,
this patch also allocates extra memory as the redzone, around the
returned memory region, and marks the redzone as unaddressable to
detect the overflows or underflows.

Updates #44853.

Change-Id: I2753d1cc1296935a66bf521e31ce91e35fcdf798
Reviewed-on: https://go-review.googlesource.com/c/go/+/298614
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: fannie zhang <Fannie.Zhang@arm.com>
2021-11-02 05:35:11 +00:00
Michael Anthony Knyszek 9a93072a07 [dev.typeparams] runtime/internal/sys: replace BigEndian with goarch.BigEndian [generated]
[git-generate]
cd src/runtime/internal/atomic
gofmt -w -r "sys.BigEndian -> goarch.BigEndian" .
goimports -w *.go
cd ../..
gofmt -w -r "sys.BigEndian -> goarch.BigEndian" .
goimports -w *.go

Change-Id: Iad35d2b367d8defb081a77ca837e7a7c805c2b7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/329190
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-06-17 20:42:35 +00:00
Cherry Mui 626e89c261 [dev.typeparams] runtime: replace funcPC with internal/abi.FuncPCABIInternal
At this point all funcPC references are ABIInternal functions.
Replace with the intrinsics.

Change-Id: I3ba7e485c83017408749b53f92877d3727a75e27
Reviewed-on: https://go-review.googlesource.com/c/go/+/321954
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-05-21 22:40:36 +00:00
David Chase 35a8bbc9ea runtime: make concatstring{2,3,4,5} consistent w/ compiler's use
Internally the compiler uses a0,a1,a3 string, not [3]string,
and this lead to different parameter passing (memory, versus registers)
which of course did not work.

Updates #40724.

Change-Id: I1dbf479b88134559ba54b4b00a042b9a0fd128b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/306910
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-04-02 20:26:12 +00:00
Austin Clements 7bbd5ca5a6 runtime: replace index and contains with bytealg calls
The runtime has its own implementation of string indexing. To reduce
code duplication and cognitive load, replace this with calls to the
internal/bytealg package. We can't do this on Plan 9 because it needs
string indexing in a note handler (which isn't allowed to use the
optimized bytealg version because it uses SSE), so we can't just
eliminate the index function, but this CL does down-scope it so make
it clear it's only for note handlers on Plan 9.

Change-Id: Ie1a142678262048515c481e8c26313b80c5875df
Reviewed-on: https://go-review.googlesource.com/c/go/+/244537
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2020-08-17 13:20:03 +00:00
Austin Clements 92bda33d27 runtime: revert signal stack mlocking
Go 1.14 included a (rather awful) workaround for a Linux kernel bug
that corrupted vector registers on x86 CPUs during signal delivery
(https://bugzilla.kernel.org/show_bug.cgi?id=205663). This bug was
introduced in Linux 5.2 and fixed in 5.3.15, 5.4.2 and all 5.5 and
later kernels. The fix was also back-ported by major distros. This
workaround was necessary, but had unfortunate downsides, including
causing Go programs to exceed the mlock ulimit in many configurations
(#37436).

We're reasonably confident that by the Go 1.16 release, the number of
systems running affected kernels will be vanishingly small. Hence,
this CL removes this workaround.

This effectively reverts CLs 209597 (version parser), 209899 (mlock
top of signal stack), 210299 (better failure message), 223121 (soft
mlock failure handling), and 244059 (special-case patched Ubuntu
kernels). The one thing we keep is the osArchInit function. It's empty
everywhere now, but is a reasonable hook to have.

Updates #35326, #35777 (the original register corruption bugs).
Updates #40184 (request to revert in 1.15).
Fixes #35979.

Change-Id: Ie213270837095576f1f3ef46bf3de187dc486c50
Reviewed-on: https://go-review.googlesource.com/c/go/+/246200
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-13 02:17:17 +00:00
Josh Bleecher Snyder b6feb03b24 cmd/compile,runtime: pass only ptr and len to some runtime calls
Some runtime calls accept a slice, but only use ptr and len.
This change modifies most such routines to accept only ptr and len.

After this change, the only runtime calls that accept an unnecessary
cap arg are concatstrings and slicerunetostring.
Neither is particularly common, and both are complicated to modify.

Negligible compiler performance impact. Shrinks binaries a little.
There are only a few regressions; the one I investigated was
due to register allocation fluctuation.

Passes 'go test -race std cmd', modulo #38265 and #38266.
Wow, does that take a long time to run.

Updates #36890

file      before    after     Δ       %       
compile   19655024  19655152  +128    +0.001% 
cover     5244840   5236648   -8192   -0.156% 
dist      3662376   3658280   -4096   -0.112% 
link      6680056   6675960   -4096   -0.061% 
pprof     14789844  14777556  -12288  -0.083% 
test2json 2824744   2820648   -4096   -0.145% 
trace     11647876  11639684  -8192   -0.070% 
vet       8260472   8256376   -4096   -0.050% 
total     115163736 115118808 -44928  -0.039% 

Change-Id: Idb29fa6a81d6a82bfd3b65740b98cf3275ca0a78
Reviewed-on: https://go-review.googlesource.com/c/go/+/227163
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-04-08 22:19:53 +00:00
Diogo Pinela bda42a7a78 runtime: use staticuint64s instead of staticbytes for 1-length strings
This was the last remaining use of staticbytes, so we can now
delete it.

The new code appears slightly faster on amd64:

name                   old time/op  new time/op  delta
SliceByteToString/1-4  6.29ns ± 2%  5.89ns ± 1%  -6.46%  (p=0.000 n=14+14)

This may not be the case on the big-endian architectures, since they have
to do an extra addition.

Updates #37612

Change-Id: Icb84c5911ba025f798de152849992a55be99e4f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/221979
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-05 19:59:18 +00:00
Austin Clements fa3a121a79 runtime: add a simple version number parser
This will be used to parse the Linux kernel versions, but this code is
generic and can be tested on its own.

For #35777.

Change-Id: If1df48d07250e5855dde45bc9d57c66f777b9fb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/209597
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-12-05 01:48:12 +00:00
Austin Clements f5e5bc1a42 runtime: mark all Go symbols called from assembly in other packages
This marks all Go symbols called from assembly in other packages with
"go:linkname" directives to ensure they get ABI wrappers.

Now that we have this go:linkname convention, this also removes the
abi0Syms definition in the runtime, which was used to give morestackc
an ABI0 wrapper. Instead, we now just mark morestackc with a
go:linkname directive.

This was tested with buildall.bash in the default configuration, with
-race, and with -gcflags=all=-d=ssa/intrinsics/off. Since I couldn't
test cgo on non-Linux configurations, I manually grepped for runtime
symbols in runtime/cgo.

Updates #31230.

Change-Id: I6c8aa56be2ca6802dfa2bf159e49c411b9071bf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/179862
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-06-06 19:44:12 +00:00
Martin Möhrmann a0f57c3fd0 cmd/compile: avoid string allocations when map key is struct or array literal
x = map[string(byteslice)] is already optimized by the compiler to avoid a
string allocation. This CL generalizes this optimization to:

x = map[T1{ ... Tn{..., string(byteslice), ...} ... }]
where T1 to Tn is a nesting of struct and array literals.

Found in a hot code path that used a struct of strings made from []byte
slices to make a map lookup.

There are no uses of the more generalized optimization in the standard library.
Passes toolstash -cmp.

MapStringConversion/32/simple    21.9ns ± 2%    21.9ns ± 3%      ~     (p=0.995 n=17+20)
MapStringConversion/32/struct    28.8ns ± 3%    22.0ns ± 2%   -23.80%  (p=0.000 n=20+20)
MapStringConversion/32/array     28.5ns ± 2%    21.9ns ± 2%   -23.14%  (p=0.000 n=19+16)
MapStringConversion/64/simple    21.0ns ± 2%    21.1ns ± 3%      ~     (p=0.072 n=19+18)
MapStringConversion/64/struct    72.4ns ± 3%    21.3ns ± 2%   -70.53%  (p=0.000 n=20+20)
MapStringConversion/64/array     72.8ns ± 1%    21.0ns ± 2%   -71.13%  (p=0.000 n=17+19)

name                           old allocs/op  new allocs/op  delta
MapStringConversion/32/simple      0.00           0.00           ~     (all equal)
MapStringConversion/32/struct      0.00           0.00           ~     (all equal)
MapStringConversion/32/array       0.00           0.00           ~     (all equal)
MapStringConversion/64/simple      0.00           0.00           ~     (all equal)
MapStringConversion/64/struct      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=20+20)
MapStringConversion/64/array       1.00 ± 0%      0.00       -100.00%  (p=0.000 n=20+20)

Change-Id: I483b4d84d8d74b1025b62c954da9a365e79b7a3a
Reviewed-on: https://go-review.googlesource.com/c/116275
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-15 19:22:07 +00:00
Martin Möhrmann b0dc54697b runtime: replace calls to hasprefix with hasPrefix
The hasprefix function is redundant and can be removed since it has
the same implementation as hasPrefix modulo variable names.

Fixes #25688

Change-Id: I499cc24a2b5c38d1301718a4e66f555fd138386f
Reviewed-on: https://go-review.googlesource.com/115835
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
2018-08-22 19:44:26 +00:00
Josh Bleecher Snyder 2f2e8f9c81 runtime: use staticbytes in intstring for small v
Triggers 21 times during make.bash.

Change-Id: I7efb34200439256151304bb66cd309913f7c9c9e
Reviewed-on: https://go-review.googlesource.com/110557
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-01 18:03:57 +00:00
David du Colombier 523f2ea77b runtime: don't use floating point in findnull on Plan 9
In CL 98015, findnull was rewritten so it uses bytes.IndexByte.

This broke the build on plan9/amd64 because the implementation
of bytes.IndexByte on AMD64 relies on SSE instructions while
floating point instructions are not allowed in the note handler.

This change fixes findnull by using the former implementation
on Plan 9, so it doesn't use bytes.IndexByte.

Fixes #24387.

Change-Id: I084d1a44d38d9f77a6c1ad492773f0a98226be16
Reviewed-on: https://go-review.googlesource.com/100577
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2018-03-14 13:15:52 +00:00
Ilya Tocar 91102bf723 runtime: use bytes.IndexByte in findnull
bytes.IndexByte is heavily optimized. Use it in findnull.
This is second attempt, similar to CL97523.
In this version we never call IndexByte on region of memory,
that crosses page boundary. A bit slower than CL97523,
but still fast:

name        old time/op  new time/op  delta
GoString-6   164ns ± 2%   118ns ± 0%  -28.00%  (p=0.000 n=10+6)

findnull is also used in gostringnocopy,
which is used in many hot spots in the runtime.

Fixes #23830

Change-Id: Id843dd4f65a34309d92bdd8df229e484d26b0cb2
Reviewed-on: https://go-review.googlesource.com/98015
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-03-09 19:37:39 +00:00
Brad Fitzpatrick 1fadbc1a76 Revert "runtime: use bytes.IndexByte in findnull"
This reverts commit 7365fac2db.

Reason for revert: breaks the build on some architectures, reading unmapped pages?

Change-Id: I3a8c02dc0b649269faacea79ecd8213defa97c54
Reviewed-on: https://go-review.googlesource.com/97995
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-03-01 22:22:51 +00:00
Josh Bleecher Snyder 7365fac2db runtime: use bytes.IndexByte in findnull
bytes.IndexByte is heavily optimized.
Use it in findnull.

name        old time/op  new time/op  delta
GoString-8  65.5ns ± 1%  40.2ns ± 1%  -38.62%  (p=0.000 n=19+19)

findnull is also used in gostringnocopy,
which is used in many hot spots in the runtime.

Fixes #23830

Change-Id: I2e6cb279c7d8078f8844065de684cc3567fe89d7
Reviewed-on: https://go-review.googlesource.com/97523
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-03-01 20:34:07 +00:00
Josh Bleecher Snyder 9372e3f5ef runtime: don't allocate to build strings of length 1
Use staticbytes instead.
Instrumenting make.bash shows approx 0.5%
of all slicebytetostrings have a buffer of length 1.

name                     old time/op  new time/op  delta
SliceByteToString/1-8    14.1ns ± 1%   4.1ns ± 1%  -71.13%  (p=0.000 n=17+20)
SliceByteToString/2-8    15.5ns ± 2%  15.5ns ± 1%     ~     (p=0.061 n=20+18)
SliceByteToString/4-8    14.9ns ± 1%  15.0ns ± 2%   +1.25%  (p=0.000 n=20+20)
SliceByteToString/8-8    17.1ns ± 1%  17.5ns ± 1%   +2.16%  (p=0.000 n=19+19)
SliceByteToString/16-8   23.6ns ± 1%  23.9ns ± 1%   +1.41%  (p=0.000 n=20+18)
SliceByteToString/32-8   26.0ns ± 1%  25.8ns ± 0%   -1.05%  (p=0.000 n=19+16)
SliceByteToString/64-8   30.0ns ± 0%  30.2ns ± 0%   +0.56%  (p=0.000 n=16+18)
SliceByteToString/128-8  38.9ns ± 0%  39.0ns ± 0%   +0.23%  (p=0.019 n=19+15)

Fixes #24172

Change-Id: I3dfa14eefbf9fb4387114e20c9cb40e186abe962
Reviewed-on: https://go-review.googlesource.com/97717
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-03-01 17:38:06 +00:00
Martin Möhrmann dfb0e4f6c7 runtime: avoid clearing memory during byte slice allocation in gobytes
Avoid using make in gobytes which clears the byte slice backing
array unnecessarily since the content is overwritten immediately again.

Check that the user provided length is positive and below the maximum
allowed allocation size explicitly in gobytes as this was done in makeslice
before this change.

Fixes #23634

Change-Id: Id852619e932aabfc468871c42ad07d34da91f45c
Reviewed-on: https://go-review.googlesource.com/94760
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-02-19 05:58:51 +00:00
Austin Clements d7691d055a runtime: replace _MaxMem with maxAlloc
Now that we have memLimit, also having _MaxMem is a bit confusing.

Replace it with maxAlloc, which better conveys what it limits. We also
define maxAlloc slightly differently: since it's now clear that it
limits allocation size, we can account for a subtle difference between
32-bit and 64-bit.

Change-Id: Iac39048018cc0dae7f0919e25185fee4b3eed529
Reviewed-on: https://go-review.googlesource.com/85890
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2018-02-15 21:12:26 +00:00
Austin Clements 229aaac19e runtime: remove getcallerpc argument
Now that getcallerpc is a compiler intrinsic on x86 and non-x86
platforms don't need the argument, we can drop it.

Sadly, this doesn't let us remove any dummy arguments since all of
those cases also use getcallersp, which still takes the argument
pointer, but this is at least an improvement.

Change-Id: I9c34a41cf2c18cba57f59938390bf9491efb22d2
Reviewed-on: https://go-review.googlesource.com/65474
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2017-09-22 22:17:15 +00:00
Josh Bleecher Snyder 23be728950 runtime: optimize slicebytestostring
Inline rawstringtmp and simplify.
Use memmove instead of copy.

name                     old time/op  new time/op  delta
SliceByteToString/1-8    19.4ns ± 2%  14.1ns ± 1%  -27.04%  (p=0.000 n=20+17)
SliceByteToString/2-8    20.8ns ± 2%  15.5ns ± 2%  -25.46%  (p=0.000 n=20+20)
SliceByteToString/4-8    20.7ns ± 1%  14.9ns ± 1%  -28.30%  (p=0.000 n=20+20)
SliceByteToString/8-8    23.2ns ± 1%  17.1ns ± 1%  -26.22%  (p=0.000 n=19+19)
SliceByteToString/16-8   29.4ns ± 1%  23.6ns ± 1%  -19.76%  (p=0.000 n=17+20)
SliceByteToString/32-8   31.4ns ± 1%  26.0ns ± 1%  -17.11%  (p=0.000 n=16+19)
SliceByteToString/64-8   36.1ns ± 0%  30.0ns ± 0%  -16.96%  (p=0.000 n=16+16)
SliceByteToString/128-8  46.9ns ± 0%  38.9ns ± 0%  -17.15%  (p=0.000 n=17+19)

Change-Id: I422e688830e4a9bd21897d1f74964625b735f436
Reviewed-on: https://go-review.googlesource.com/37791
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marvin Stenger <marvin.stenger94@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-08 22:05:52 +00:00
Martin Möhrmann d7b34d5f29 runtime: improve atoi implementation
- Adds overflow checks
- Adds parsing of negative integers
- Adds boolean return value to signal parsing errors
- Adds atoi32 for parsing of integers that fit in an int32
- Adds tests

Handling of errors to provide error messages
at the call sites is left to future CLs.

Updates #17718

Change-Id: I3cacd0ab1230b9efc5404c68edae7304d39bcbc0
Reviewed-on: https://go-review.googlesource.com/32390
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-11-01 14:04:39 +00:00
Austin Clements 87e48c5afd runtime, cmd/compile: rename memclr -> memclrNoHeapPointers
Since barrier-less memclr is only safe in very narrow circumstances,
this commit renames memclr to avoid accidentally calling memclr on
typed memory. This can cause subtle, non-deterministic bugs, so it's
worth some effort to prevent. In the near term, this will also prevent
bugs creeping in from any concurrent CLs that add calls to memclr; if
this happens, whichever patch hits master second will fail to compile.

This also adds the other new memclr variants to the compiler's
builtin.go to minimize the churn on that binary blob. We'll use these
in future commits.

Updates #17503.

Change-Id: I00eead049f5bd35ca107ea525966831f3d1ed9ca
Reviewed-on: https://go-review.googlesource.com/31369
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2016-10-28 18:20:33 +00:00
Martin Möhrmann b679665a18 cmd/compile: move stringtoslicebytetmp to the backend
- removes the runtime function stringtoslicebytetmp
- removes the generation of calls to stringtoslicebytetmp from the frontend
- adds handling of OSTRARRAYBYTETMP in the backend

This reduces binary sizes and avoids function call overhead.

Change-Id: Ib9988d48549cee663b685b4897a483f94727b940
Reviewed-on: https://go-review.googlesource.com/32158
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-10-28 07:58:47 +00:00
Martin Möhrmann d295174030 runtime: speed up non-ASCII rune decoding
Copies utf8 constants and EncodeRune implementation from unicode/utf8.

Adds a new decoderune implementation that is used by the compiler
in code generated for ranging over strings. It does not handle
ASCII runes since these are handled directly before calls to decoderune.

The DecodeRuneInString implementation from unicode/utf8 is not used
since it uses a lookup table that would increase the use of cpu caches.

Adds more tests that check decoding of valid and invalid utf8 sequences.

name                              old time/op  new time/op  delta
RuneIterate/range2/ASCII-4        7.45ns ± 2%  7.45ns ± 1%     ~     (p=0.634 n=16+16)
RuneIterate/range2/Japanese-4     53.5ns ± 1%  49.2ns ± 2%   -8.03%  (p=0.000 n=20+20)
RuneIterate/range2/MixedLength-4  46.3ns ± 1%  41.0ns ± 2%  -11.57%  (p=0.000 n=20+20)

new:
"".decoderune t=1 size=423 args=0x28 locals=0x0
old:
"".charntorune t=1 size=666 args=0x28 locals=0x0

Change-Id: I1df1fdb385bb9ea5e5e71b8818ea2bf5ce62de52
Reviewed-on: https://go-review.googlesource.com/28490
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-17 11:25:22 +00:00
Martin Möhrmann 150de948ee cmd/compile: intrinsify slicebytetostringtmp when not instrumenting
when not instrumenting:
- Intrinsify uses of slicebytetostringtmp within the runtime package
  in the ssa backend.
- Pass OARRAYBYTESTRTMP nodes to the compiler backends for lowering
  instead of generating calls to slicebytetostringtmp.

name                    old time/op  new time/op  delta
ConcatStringAndBytes-4  27.9ns ± 2%  24.7ns ± 2%  -11.52%  (p=0.000 n=43+43)

Fixes #17044

Change-Id: I51ce9c3b93284ce526edd0234f094e98580faf2d
Reviewed-on: https://go-review.googlesource.com/29017
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2016-09-14 21:58:14 +00:00
Martin Möhrmann 252093f120 runtime: remove maxstring
Before this CL the runtime prevented printing of overlong strings with the print
function when the length of the string was determined to be corrupted.
Corruption was checked by comparing the string size against the limit
which was stored in maxstring.

However maxstring was not updated everywhere were go strings were created
e.g. for string constants during compile time. Thereby the check for maximum
string length prevented the printing of some valid strings.

The protection maxstring provided did not warrant the bookkeeping
and global synchronization needed to keep maxstring updated to the
correct limit everywhere.

Fixes #16999

Change-Id: I62cc2f4362f333f75b77f199ce1a71aac0ff7aeb
Reviewed-on: https://go-review.googlesource.com/28813
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-09-08 15:57:01 +00:00
Martin Möhrmann 0dae9dfb08 cmd/compile: improve string iteration performance
Generate a for loop for ranging over strings that only needs to call
the runtime function charntorune for non ASCII characters.

This provides faster iteration over ASCII characters and slightly
faster iteration for other characters.

The runtime function charntorune is changed to take an index from where
to start decoding and returns the index after the last byte belonging
to the decoded rune.

All call sites of charntorune in the runtime are replaced by a for loop
that will be transformed by the compiler instead of calling the charntorune
function directly.

go binary size decreases by 80 bytes.
godoc binary size increases by around 4 kilobytes.

runtime:

name                           old time/op  new time/op  delta
RuneIterate/range/ASCII-4      43.7ns ± 3%  10.3ns ± 4%  -76.33%  (p=0.000 n=44+45)
RuneIterate/range/Japanese-4   72.5ns ± 2%  62.8ns ± 2%  -13.41%  (p=0.000 n=49+50)
RuneIterate/range1/ASCII-4     43.5ns ± 2%  10.4ns ± 3%  -76.18%  (p=0.000 n=50+50)
RuneIterate/range1/Japanese-4  72.5ns ± 2%  62.9ns ± 2%  -13.26%  (p=0.000 n=50+49)
RuneIterate/range2/ASCII-4     43.5ns ± 3%  10.3ns ± 2%  -76.22%  (p=0.000 n=48+47)
RuneIterate/range2/Japanese-4  72.4ns ± 2%  62.7ns ± 2%  -13.47%  (p=0.000 n=50+50)

strings:

name                 old time/op    new time/op    delta
IndexRune-4            64.7ns ± 5%    22.4ns ± 3%  -65.43%  (p=0.000 n=25+21)
MapNoChanges-4          269ns ± 2%     157ns ± 2%  -41.46%  (p=0.000 n=23+24)
Fields-4               23.0ms ± 2%    19.7ms ± 2%  -14.35%  (p=0.000 n=25+25)
FieldsFunc-4           23.1ms ± 2%    19.6ms ± 2%  -14.94%  (p=0.000 n=25+24)

name                 old speed      new speed      delta
Fields-4             45.6MB/s ± 2%  53.2MB/s ± 2%  +16.87%  (p=0.000 n=24+25)
FieldsFunc-4         45.5MB/s ± 2%  53.5MB/s ± 2%  +17.57%  (p=0.000 n=25+24)

Updates #13162

Change-Id: I79ffaf828d82bf9887592f08e5cad883e9f39701
Reviewed-on: https://go-review.googlesource.com/27853
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
2016-08-30 18:17:20 +00:00
Josh Bleecher Snyder 880c967ccd runtime: minor string/rune optimizations
Eliminate a spill in concatstrings.
Provide bounds elim hints in runetochar.
No significant benchmark movement.

Before:
"".runetochar t=1 size=412 args=0x28 locals=0x0
"".concatstrings t=1 size=736 args=0x30 locals=0x98

After:
"".runetochar t=1 size=337 args=0x28 locals=0x0
"".concatstrings t=1 size=711 args=0x30 locals=0x90

Change-Id: Icce646976cb20a223163b7e72a54761193ac17e3
Reviewed-on: https://go-review.googlesource.com/27460
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-08-22 15:19:31 +00:00
Keith Randall 6f3f02f80d runtime: zero tmpbuf between len and cap
Zero the entire buffer so we don't need to
lower its capacity upon return.  This lets callers
do some appending without allocation.

Zeroing is cheap, the byte buffer requires only
4 extra instructions.

Fixes #14235

Change-Id: I970d7badcef047dafac75ac17130030181f18fe2
Reviewed-on: https://go-review.googlesource.com/22424
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-04-25 21:16:52 +00:00
Keith Randall 001e8e8070 runtime: simplify mallocgc flag argument
mallocgc can calculate noscan itself.  The only remaining
flag argument is needzero, so we just make that a boolean arg.

Fixes #15379

Change-Id: I839a70790b2a0c9dbcee2600052bfbd6c8148e20
Reviewed-on: https://go-review.googlesource.com/22290
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-04-20 14:02:22 +00:00
Matthew Dempsky e6066711a0 cmd/compile, runtime: fix pedantic int->string conversions
Previously, cmd/compile rejected constant int->string conversions if
the integer value did not fit into an "int" value. Also, runtime
incorrectly truncated 64-bit values to 32-bit before checking if
they're a valid Unicode code point. According to the Go spec, both of
these cases should instead yield "\uFFFD".

Fixes #15039.

Change-Id: I3c8a3ad9a0780c0a8dc1911386a523800fec9764
Reviewed-on: https://go-review.googlesource.com/21344
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-03-31 10:28:23 +00:00
Matthew Dempsky a03bdc3e6b runtime: eliminate unnecessary type conversions
Automated refactoring produced using github.com/mdempsky/unconvert.

Change-Id: Iacf871a4f221ef17f48999a464ab2858b2bbaa90
Reviewed-on: https://go-review.googlesource.com/20071
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-07 20:53:27 +00:00
Dmitry Vyukov db44223fde runtime: fix getcallerpc args
Change-Id: I6b14b8eecf125dd74bd40f4e7fff6b49de150e42
Reviewed-on: https://go-review.googlesource.com/19897
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2016-02-25 18:57:28 +00:00
Keith Randall 4d02b12417 runtime: don't expose stack buffer in stringto{byte,rune}slice
When using a stack-allocated buffer for the result, don't
expose the uninitialized portion of it by restricting its
capacity to its length.

The other option is to zero the portion between len and cap.
That seems like more work, but might be worth it if the caller
then appends some stuff to the result.  But this close to 1.6,
I'm inclined to do the simplest fix possible.

Fixes #14232

Change-Id: I21c50d3cda02fd2df4d60ba5e2cfe2efe272f333
Reviewed-on: https://go-review.googlesource.com/19231
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-02-04 23:57:10 +00:00
Michael Matloob 67faca7d9c runtime: break atomics out into package runtime/internal/atomic
This change breaks out most of the atomics functions in the runtime
into package runtime/internal/atomic. It adds some basic support
in the toolchain for runtime packages, and also modifies linux/arm
atomics to remove the dependency on the runtime's mutex. The mutexes
have been replaced with spinlocks.

all trybots are happy!
In addition to the trybots, I've tested on the darwin/arm64 builder,
on the darwin/arm builder, and on a ppc64le machine.

Change-Id: I6698c8e3cf3834f55ce5824059f44d00dc8e3c2f
Reviewed-on: https://go-review.googlesource.com/14204
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2015-11-10 17:38:04 +00:00
Ian Lance Taylor 73f329f472 runtime, syscall: add calls to msan functions
Add explicit memory sanitizer instrumentation to the runtime and syscall
packages.  The compiler does not instrument the runtime package.  It
does instrument the syscall package, but we need to add a couple of
cases that it can't see.

Change-Id: I2d66073f713fe67e33a6720460d2bb8f72f31394
Reviewed-on: https://go-review.googlesource.com/16164
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2015-10-21 19:17:46 +00:00
Matthew Dempsky d4a7ea1b71 runtime: add stringStructOf helper function
Instead of open-coding conversions from *string to unsafe.Pointer then
to *stringStruct, add a helper function to add some type safety.
Bonus: This caught two **string values being converted to
*stringStruct in heapdump.go.

While here, get rid of the redundant _string type, but add in a
stringStructDWARF type used for generating DWARF debug info.

Change-Id: I8882f8cca66ac45190270f82019a5d85db023bd2
Reviewed-on: https://go-review.googlesource.com/16131
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-10-20 23:13:27 +00:00
Nodir Turakulov 881b0e7880 runtime: merge string1.go into string.go
string1.go contents are appended to string.go as is

Updates #12952

Change-Id: I30083ba7fdd362d4421e964a494c76ca865bedc2
Reviewed-on: https://go-review.googlesource.com/15951
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-10-16 15:46:02 +00:00
Michael Hudson-Doyle ab4df700b8 runtime: merge slice and sliceStruct
By removing type slice, renaming type sliceStruct to type slice and
whacking until it compiles.

Has a pleasing net reduction of conversions.

Fixes #10188

Change-Id: I77202b8df637185b632fd7875a1fdd8d52c7a83c
Reviewed-on: https://go-review.googlesource.com/8770
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-04-15 16:59:49 +00:00
Dmitry Vyukov 9568126f35 cmd/gc: allocate buffers for non-escaping string conversions on stack
Support the following conversions in escape analysis:
[]rune("foo")
[]byte("foo")
string([]rune{})

If the result does not escape, allocate temp buffer on stack
and pass it to runtime functions.

Change-Id: I1d075907eab8b0109ad7ad1878104b02b3d5c690
Reviewed-on: https://go-review.googlesource.com/3590
Reviewed-by: Russ Cox <rsc@golang.org>
2015-02-12 08:29:53 +00:00