Currently, zeroing generates an ssa.OpZero, which never has write
barriers, even if the assignment is an OASWB. The hybrid barrier
requires write barriers on zeroing, so change OASWB to generate an
ssa.OpZeroWB when assigning the zero value, which turns into a
typedmemclr.
Updates #17503.
Change-Id: Ib37ac5e39f578447dbd6b36a6a54117d5624784d
Reviewed-on: https://go-review.googlesource.com/31451
Reviewed-by: Cherry Zhang <cherryyz@google.com>
- removes the runtime function stringtoslicebytetmp
- removes the generation of calls to stringtoslicebytetmp from the frontend
- adds handling of OSTRARRAYBYTETMP in the backend
This reduces binary sizes and avoids function call overhead.
Change-Id: Ib9988d48549cee663b685b4897a483f94727b940
Reviewed-on: https://go-review.googlesource.com/32158
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When we do
var x []byte = ...
y := x[i:]
We can't just use y.ptr = x.ptr + i, as the new pointer may point to the
next object in memory after the backing array.
We used to fix this by doing:
y.cap = x.cap - i
delta := i
if y.cap == 0 {
delta = 0
}
y.ptr = x.ptr + delta
That generates a branch in what is otherwise straight-line code.
Better to do:
y.cap = x.cap - i
mask := (y.cap - 1) >> 63 // -1 if y.cap==0, 0 otherwise
y.ptr = x.ptr + i &^ mask
It's about the same number of instructions (~4, depending on what
parts are constant, and the target architecture), but it is all
inline. It plays nicely with CSE, and the mask can be computed
in parallel with the index (in cases where a multiply is required).
It is a minor win in both speed and space.
Change-Id: Ied60465a0b8abb683c02208402e5bb7ac0e8370f
Reviewed-on: https://go-review.googlesource.com/32022
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When the compiler insert write barriers, the frontend makes
conservative decisions at an early stage. This may have false
positives which result in write barriers for stack writes.
A new phase, writebarrier, is added to the SSA backend, to delay
the decision and eliminate false positives. The frontend still
makes conservative decisions. When building SSA, instead of
emitting runtime calls directly, it emits WB ops (StoreWB,
MoveWB, etc.), which will be expanded to branches and runtime
calls in writebarrier phase. Writes to static locations on stack
are detected and write barriers are removed.
All write barriers of stack writes found by the script from
issue #17330 are eliminated (except two false positives).
Fixes#17330.
Change-Id: I9bd66333da9d0ceb64dcaa3c6f33502798d1a0f8
Reviewed-on: https://go-review.googlesource.com/31131
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Adapt old test for prove's bounds check elimination.
Added missing rule to generic rules that lead to differences
between 32 and 64 bit platforms on sliceopt test.
Added debugging to prove.go that was helpful-to-necessary to
discover that missing rule.
Lowered debugging level on prove.go from 3 to 1; no idea
why it was previously 3.
Change-Id: I09de206aeb2fced9f2796efe2bfd4a59927eda0c
Reviewed-on: https://go-review.googlesource.com/23290
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This pragma cancels the effect of go:nowritebarrierrec. This is useful
in the scheduler because there are places where we enter a function
without a valid P (and hence cannot have write barriers), but then
obtain a P. This allows us to annotate the function with
go:nowritebarrierrec and split out the part after we've obtained a P
into a go:yeswritebarrierrec function.
Change-Id: Ic8ce4b6d3c074a1ecd8280ad90eaf39f0ffbcc2a
Reviewed-on: https://go-review.googlesource.com/30938
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
"abc"[1] is not like 'b', in that -"abc"[1] is uint8 math, not ideal constant math.
Delay the constantification until after ideal constant folding is over.
Fixes#11370.
Change-Id: Iba2fc00ca2455959e7bab8f4b8b4aac14b1f9858
Reviewed-on: https://go-review.googlesource.com/15740
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This, along with CL 30140, removes ~50% of stack write barriers
mentioned in issue #17330. The remaining are most due to Phi and
FwdRef, which is not resolved when building SSA. We might be
able to do it at a later stage where Phi and Copy propagations
are done, but matching an if-(store-store-call)+ sequence seems
not very pleasant.
Updates #17330.
Change-Id: Iaa36c7b1f4c4fc3dc10a27018a3b0e261094cb21
Reviewed-on: https://go-review.googlesource.com/30290
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Identify live stack variables during SSA and compute the stack frame
layout earlier so that we can emit instructions with the correct
offsets upfront.
Passes toolstash/buildall.
Change-Id: I191100dba274f1e364a15bdcfdc1d1466cdd1db5
Reviewed-on: https://go-review.googlesource.com/30216
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Probably a holdover from linked list vs. slice.
Change-Id: Ib2540b08ef0ae48707d44a5d57bc23f8d65c760d
Reviewed-on: https://go-review.googlesource.com/30256
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Should be more asymptotically happy.
We process each variable in turn to find all the
locations where it needs a phi (the dominance frontier
of all of its definitions). Then we add all those phis.
This takes O(n * #variables), although hopefully much less.
Then we do a single tree walk to match all the
FwdRefs with the nearest definition or phi.
This takes O(n) time.
The one remaining inefficiency is that we might end up
introducing a bunch of dead phis in the first step.
A TODO is to introduce phis only where they might be
used by a read.
The old algorithm is still faster on small functions,
so there's a cutover size (currently 500 blocks).
This algorithm supercedes the David's sparse phi
placement algorithm for large functions.
Lowers compile time of example from #14934 from
~10 sec to ~4 sec.
Lowers compile time of example from #16361 from
~4.5 sec to ~3 sec.
Lowers #16407 from ~20 min to ~30 sec.
Update #14934
Update #16361Fixes#16407
Change-Id: I1cff6364e1623c143190b6a924d7599e309db58f
Reviewed-on: https://go-review.googlesource.com/30163
Reviewed-by: David Chase <drchase@google.com>
At this point in the compiler we haven't assigned Xoffset values for
PAUTO variables anyway, so just immediately store the stack offsets
into Xoffset rather than into a global map.
Change-Id: I61eb471c857c8b145fd0895cbd98fd4e8d3c3365
Reviewed-on: https://go-review.googlesource.com/30081
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
var x *X = ...
defer x.foo()
As part of the defer, we need to calculate &(*X).foo·f. This expression
is the address of the static closure that will call (*X).foo when a
pointer to that closure is used in a call/defer/go. This pointer is not
currently properly typed in SSA. It is a pointer type, but the base
type is nil, not a proper type.
This turns out not to be a problem currently because we never use the
type of these SSA values. But I'm trying to change that (to be able to
spill them) in CL 28391. To fix, use uint8 as the fake type of the
closure.
Change-Id: Ieee388089c9af398ed772ee8c815122c347cb633
Reviewed-on: https://go-review.googlesource.com/29444
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Also adds the 'find leftmost one' instruction (FLOGR) and replaces the
WORD-encoded use of FLOGR in math/big with it.
Change-Id: I18e7cd19e75b8501a6ae8bd925471f7e37ded206
Reviewed-on: https://go-review.googlesource.com/29372
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We're dropping this behavior in favor of runtime.KeepAlive.
Implement runtime.KeepAlive as an intrinsic.
Update #15843
Change-Id: Ib60225bd30d6770ece1c3c7d1339a06aa25b1cbc
Reviewed-on: https://go-review.googlesource.com/28310
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Teach SSA about the cmd/internal/obj/$ARCH register numbering.
It can then return that numbering when requested. Each architecture
now does not need to know anything about the internal SSA numbering
of registers.
Change-Id: I34472a2736227c15482e60994eebcdd2723fa52d
Reviewed-on: https://go-review.googlesource.com/29249
Reviewed-by: David Chase <drchase@google.com>
Follow up to CL 29134. Generated with gofmt -r 'Nod -> nod', plus
three manual adjustments to the comments in syntax/parser.go
Change-Id: I02920f7ab10c70b6e850457b42d5fe35f1f3821a
Reviewed-on: https://go-review.googlesource.com/29136
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Binary search remains our friend.
Suppose you add an ought-to-be-benign pattern to PPC64.rules,
and make.bash starts crashing. You can guard the pattern(s)
with config.DebugTest:
(Eq8 x y) && config.DebugTest && isSigned(x.Type) &&
isSigned(y.Type) ->
(Equal (CMPW (SignExt8to32 x) (SignExt8to32 y)))
and then
gossahash -s ./make.bash
...
(go drink beer while silicon minions toil)
...
Trying ./make.bash args=[], env=[GOSSAHASH=100110010111110]
./make.bash failed (1 distinct triggers): exit status 1
Trigger string is 'GOSSAHASH triggered (*importReader).readByte',
repeated 1 times
Review GSHS_LAST_FAIL.0.log for failing run
Finished with GOSSAHASH=100110010111110
Change-Id: I4eff46ebaf496baa2acedd32e217005cb3ac1c62
Reviewed-on: https://go-review.googlesource.com/29273
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
GOSSAFUNC=foo had previously only done printing for the
single function foo, and didn't quite clean up after itself
properly. Changes ensures that Config.HTML != nil iff
GOSSAFUNC==name-of-current-function.
Change-Id: I255e2902dfc64f715d93225f0d29d9525c06f764
Reviewed-on: https://go-review.googlesource.com/29250
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
After the removal of the old backend many types are no longer referenced
outside internal/gc. Make these functions private so that tools like
honnef.co/go/unused can spot when they become dead code. In doing so
this CL identified several previously public helpers which are no longer
used, so removes them.
This should be the last of the public functions.
Change-Id: I7e9c4e72f86f391b428b9dddb6f0d516529706c3
Reviewed-on: https://go-review.googlesource.com/29134
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Get rid of BlockCheck. Josh goaded me into it, and I went
down a rabbithole making it happen.
NilCheck now panics if the pointer is nil and returns void, as before.
BlockCheck is gone, and NilCheck is no longer a Control value for
any block. It just exists (and deadcode knows not to throw it away).
I rewrote the nilcheckelim pass to handle this case. In particular,
there can now be multiple NilCheck ops per block.
I moved all of the arch-dependent nil check elimination done as
part of ssaGenValue into its own proper pass, so we don't have to
duplicate that code for every architecture.
Making the arch-dependent nil check its own pass means I needed
to add a bunch of flags to the opcode table so I could write
the code without arch-dependent ops everywhere.
Change-Id: I419f891ac9b0de313033ff09115c374163416a9f
Reviewed-on: https://go-review.googlesource.com/29120
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
when not instrumenting:
- Intrinsify uses of slicebytetostringtmp within the runtime package
in the ssa backend.
- Pass OARRAYBYTESTRTMP nodes to the compiler backends for lowering
instead of generating calls to slicebytetostringtmp.
name old time/op new time/op delta
ConcatStringAndBytes-4 27.9ns ± 2% 24.7ns ± 2% -11.52% (p=0.000 n=43+43)
Fixes#17044
Change-Id: I51ce9c3b93284ce526edd0234f094e98580faf2d
Reviewed-on: https://go-review.googlesource.com/29017
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Rip out the code that allows SSA to be used conditionally.
No longer exists:
ssa=0 flag
GOSSAHASH
GOSSAPKG
SSATEST
GOSSAFUNC now only controls the printing of the IR/html.
Still need to rip out all of the old backend. It should no longer be
callable after this CL.
Update #16357
Change-Id: Ib30cc18fba6ca52232c41689ba610b0a94aa74f5
Reviewed-on: https://go-review.googlesource.com/29155
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
It passed tests once, if anything's wrong, better to fail
sooner than later.
Change-Id: Ibb1c5db3f4c5535a4ff4681fd157db77082c5041
Reviewed-on: https://go-review.googlesource.com/28982
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The new SSA backend modifies the ABI slightly: R0 is now a usable
general purpose register.
Fixes#16677.
Change-Id: I367435ce921e0c7e79e021c80cf8ef5d1d1466cf
Reviewed-on: https://go-review.googlesource.com/28978
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Also add assembly implementation, in case intrinsics is disabled.
Change-Id: Iff0a8a8ce326651bd29f6c403f5ec08dd3629993
Reviewed-on: https://go-review.googlesource.com/28979
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
No need for it, we can treat calls as (mostly) normal values
that take a memory and return a memory.
Lowers the number of basic blocks needed to represent a function.
"go test -c net/http" uses 27% fewer basic blocks.
Probably doesn't affect generated code much, but should help
various passes whose running time and/or space depends on
the number of basic blocks.
Fixes#15631
Change-Id: I0bf21e123f835e2cfa382753955a4f8bce03dfa6
Reviewed-on: https://go-review.googlesource.com/28950
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
- also consistently use %v instead of %s when we have a (gc) Formatter
- rewrite done automatically using Formats test in -u (update) mode
- manual update of format strings that were not single string constants
- updated fmt.go, fmt_test.go accordingly
- fmt_test: permit "%T" always
Change-Id: I8f0704286aba5704600ad0c4a4484005b79b905d
Reviewed-on: https://go-review.googlesource.com/28954
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Atomic ops on ARM are implemented with kernel calls, so they are
not intrinsified.
Change-Id: I0e7cc2e5526ae1a3d24b4b89be1bd13db071f8ef
Reviewed-on: https://go-review.googlesource.com/28977
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This list now matches the one in popt.go.
Change-Id: Ib24de531cc35252f0ef276e5c6d247654b021533
Reviewed-on: https://go-review.googlesource.com/28965
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When the divisor is known to be a constant
non-zero, don't insert panicdivide calls
that will just be eliminated later.
The main benefit here is readability of the SSA
form for compiler developers.
Change-Id: Icb7d07fc996941fbaff84524ac3e4b53d8e75fda
Reviewed-on: https://go-review.googlesource.com/28530
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
It should alias to Xchg instead of Swap. Found when testing #16985.
Change-Id: If9fd734a1f89b8b2656f421eb31b9d1b0d95a49f
Reviewed-on: https://go-review.googlesource.com/28512
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Does not pass toolstash -cmp due to changed export data,
but the cmd/go binary (which doesn't contain export data)
is bit-for-bit identical.
Change-Id: I6b12f9de18cf7da528e9207dccbf8f08c969f142
Reviewed-on: https://go-review.googlesource.com/26753
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reduce the duplication in every arch by moving the code into package gc.
Change-Id: Ia111add8316492571825431ecd4f0154c8792ae1
Reviewed-on: https://go-review.googlesource.com/28481
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Uses the same implementation as runtime/internal/atomic.
Reorganize the intrinsic detector to make it more table-driven.
Also works on amd64p32.
Change-Id: I7a5238951d6018d7d5d1bc01f339f6ee9282b2d0
Reviewed-on: https://go-review.googlesource.com/28076
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Atomic swap, add/and/or, compare and swap.
Also works on amd64p32.
Change-Id: Idf2d8f3e1255f71deba759e6e75e293afe4ab2ba
Reviewed-on: https://go-review.googlesource.com/27813
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add the following optimizations:
- fold constants
- fold address into load/store
- simplify extensions and conditional branches
- remove nil checks
Turn on SSA on MIPS64 by default, and toggle the tests.
Fixes#16359.
Change-Id: I7f1e38c2509e22e42cd024e712990ebbe47176bd
Reviewed-on: https://go-review.googlesource.com/27870
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Inline atomic reads and writes on amd64. There's no reason
to pay the overhead of a call for these.
To keep atomic loads from being reordered, we make them
return a <value,memory> tuple.
Change the meaning of resultInArg0 for tuple-generating ops
to mean the first part of the result tuple, not the second.
This means we can always put the store part of the tuple last,
matching how arguments are laid out. This requires reordering
the outputs of add32carry and sub32carry and their descendents
in various architectures.
benchmark old ns/op new ns/op delta
BenchmarkAtomicLoad64-8 2.09 0.26 -87.56%
BenchmarkAtomicStore64-8 7.54 5.72 -24.14%
TBD (in a different CL): Cas, Or8, ...
Change-Id: I713ea88e7da3026c44ea5bdb56ed094b20bc5207
Reviewed-on: https://go-review.googlesource.com/27641
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Now that we have ops that can return 2 results, have BSF return a result
and flags. We can then get rid of the redundant comparison and use CMOV
instead of CMOVconst ops.
Get rid of a bunch of the ops we don't use. Ctz{8,16}, plus all the Clzs,
and CMOVNEs. I don't think we'll ever use them, and they would be easy
to add back if needed.
Change-Id: I8858a1d017903474ea7e4002fc76a6a86e7bd487
Reviewed-on: https://go-review.googlesource.com/27630
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This time with the cherry-pick from the proper patch of
the old CL.
Stack size increased.
Corrected NaN-comparison glitches.
Marked g register as clobbered by calls.
Fixed shared libraries.
live_ssa.go still disabled because of differences.
Presumably turning on more optimization will fix
both the stack size and the live_ssa.go glitches.
Enhanced debugging output for shared libs test.
Rebased onto master.
Updates #16010.
Change-Id: I40864faf1ef32c118fb141b7ef8e854498e6b2c4
Reviewed-on: https://go-review.googlesource.com/27159
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
- Use machine instructions for uint64<->float conversions
- Do not enforce alignment on Zero/Move
ARM64 supports unaligned load/stores, but only aligned offset
or small offset can be encoded into instructions.
- Do combined loads
Change-Id: Iffca7dd0f13070b17b784861ce5a30af584680eb
Reviewed-on: https://go-review.googlesource.com/27086
Reviewed-by: David Chase <drchase@google.com>
When T is a scalar, there are no runtime calls
required, which makes this a clear win.
encoding/binary:
WriteInts-8 958ns ± 3% 864ns ± 2% -9.80% (p=0.000 n=15+15)
This also considerably shrinks a core fmt
routine:
Before: "".(*pp).printArg t=1 size=3952 args=0x20 locals=0xf0
After: "".(*pp).printArg t=1 size=2624 args=0x20 locals=0x98
Unfortunately, I find it very hard to get stable
numbers out of the fmt benchmarks due to thermal scaling.
Change-Id: I1278006b030253bf8e48dc7631d18985cdaa143d
Reviewed-on: https://go-review.googlesource.com/26659
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
autopkg == localpkg, so it appears to be a remnant of earlier code.
Change-Id: I65b6c074535e877317cbf9f1f35e94890f0ebf14
Reviewed-on: https://go-review.googlesource.com/26662
Reviewed-by: Keith Randall <khr@golang.org>
Passes ssa_test.
Requires a few new instructions and some scratchpad
memory to move data between G and F registers.
Also fixed comparisons to be correct in case of NaN.
Added missing instructions for run.bash.
Removed some FP registers that are apparently "reserved"
(but that are also apparently also unused except for a
gratuitous multiplication by two when y = x+x would work
just as well).
Currently failing stack splits.
Updates #16010.
Change-Id: I73b161bfff54445d72bd7b813b1479f89fc72602
Reviewed-on: https://go-review.googlesource.com/26813
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Add more ARM64 optimizations:
- use hardware zero register when it is possible.
- use shifted ops.
The assembler supports shifted ops but not documented, nor knows
how to print it. This CL adds them.
- enable fast division.
This was disabled because it makes the old backend generate slower
code. But with SSA it generates faster code.
Turn on SSA by default, also adjust tests.
Change-Id: I7794479954c83bb65008dcb457bc1e21d7496da6
Reviewed-on: https://go-review.googlesource.com/26950
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Last part of the 386 SSA port.
Modify the x86 backend to simulate SSE registers and
instructions with 387 registers and instructions.
The simulation isn't terribly performant, but it works,
and the old implementation wasn't very performant either.
Leaving to people who care about 387 to optimize if they want.
Turn on SSA backend for 386 by default.
Fixes#16358
Change-Id: I678fb59132620b2c47e993c1c10c4c21135f70c0
Reviewed-on: https://go-review.googlesource.com/25271
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
It's not a new backend, just a PtrSize==4 modification
of the existing AMD64 backend.
Change-Id: Icc63521a5cf4ebb379f7430ef3f070894c09afda
Reviewed-on: https://go-review.googlesource.com/25586
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
GOARCH=386 SSATEST=1 ./all.bash passes
Caveat: still needs changes to test/ files to use *_ssa.go versions. I
won't check those changes in with this CL because the builders will
complain as they don't have SSATEST=1.
Mostly minor fixes.
Implement float <-> uint32 in assembly. It seems the simplest option
for now.
GO386=387 does not work. That's why I can't make SSA the default for
386 yet.
Change-Id: Ic4d4402104d32bcfb1fd612f5bb6539f9acb8ae0
Reviewed-on: https://go-review.googlesource.com/25119
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Make tuple types and their SelectX ops fully generic.
These ops no longer need to be lowered.
Regalloc understands them and their tuple-generating arguments.
We can now have opcodes returning arbitrary pairs of results.
(And it would be easy to move to >2 results if needed.)
Update arm implementation to the new standard.
Implement just enough in 386 port to do 64-bit add.
Change-Id: I370ed5aacce219c82e1954c61d1f63af76c16f79
Reviewed-on: https://go-review.googlesource.com/24976
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
NaCl code runs in sandbox and there are restrictions for its
instruction uses
(https://developer.chrome.com/native-client/reference/sandbox_internals/arm-32-bit-sandbox).
Like the legacy backend, on NaCl,
- don't use R9, which is used as NaCl's "thread pointer".
- don't use Duff's device.
- don't use indexed load/stores.
- the assembler rewrites DIV/MOD to runtime calls, which on NaCl
clobbers R12, so R12 is marked as clobbered for DIV/MOD.
- other restrictions are satisfied by the assembler.
Enable SSA specific tests on nacl/arm, and disable non-SSA ones.
Updates #15365.
Change-Id: I9262693ec6756b89ca29d3ae4e52a96fe5403b02
Reviewed-on: https://go-review.googlesource.com/24859
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
As Josh mentioned in CL 24716, there has been requests for using SSA
for ARM. SSA can still be disabled by setting -ssa=0 for cmd/compile,
or partially enabled with GOSSAFUNC, GOSSAPKG, and GOSSAHASH.
Not enable SSA by default on NaCl, which is not supported yet.
Enable SSA-specific tests on ARM: live_ssa.go and nilptr3_ssa.go;
disable non-SSA tests: live.go, nilptr3.go, and slicepot.go.
Updates #15365.
Change-Id: Ic2ca8d166aeca8517b9d262a55e92f2130683a16
Reviewed-on: https://go-review.googlesource.com/23953
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
The comments were mostly duplicated; unify them.
Add a check that the required invariant holds.
Change-Id: I42fe09dcd1fac76d3c4e191f7a58c591c5ce429b
Reviewed-on: https://go-review.googlesource.com/24719
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Encode the size and the alignment into AuxInt of Zero and Move ops.
On AMD64, we simply don't look at the alignment. On ARM and PPC64, we
only generate aligned stores.
Updates #15365.
Change-Id: Ifdcc205c364f67c4516b9adebfe7d50d223b6863
Reviewed-on: https://go-review.googlesource.com/24511
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The frontend may emit node with line number missing. In this case,
use the parent line number. Instead of changing every call site of
pushLine, do it in pushLine itself.
Fixes#16214.
Change-Id: I80390550b56e4d690fc770b01ff725b892ffd6dc
Reviewed-on: https://go-review.googlesource.com/24641
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We had ~30 one way, and these four new occurrences the other way.
Updates #11626
Change-Id: Ic6403dc4905874916ae292ff739d33482ed8e5bf
Reviewed-on: https://go-review.googlesource.com/24683
Reviewed-by: Rob Pike <r@golang.org>
This adds the initial SSA implementation for PPC64.
Builds golang and all.bash runs correctly. Simple hello.go
builds but does not run.
Change-Id: I7cec211b934cd7a2dd75a6cdfaf9f71867063466
Reviewed-on: https://go-review.googlesource.com/24453
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This will be used verbatim in other architectures.
Change-Id: I307891ae597d797fd45f296b6a38ffe9fac6b975
Reviewed-on: https://go-review.googlesource.com/24155
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This will be needed by other architectures as well.
Put a cleaner encapsulation around it.
Change-Id: I0ac25d600378042b2233301678e9d037e20701d8
Reviewed-on: https://go-review.googlesource.com/24154
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
SSA treats SP as constant throughout a function, so as OffPtr [off] SP.
When the stack moves, spilled OffPtr values become invalid, if they are
not pointer-typed.
(Currently it is fine because of the optimization rules that folds OffPtr
into Load/Store. But it'd better be "optimization", not requirement.)
Updates #15365.
Change-Id: I76cf4008dfdc169e1cb5a55a2605b6678efc915d
Reviewed-on: https://go-review.googlesource.com/23941
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This matches the behavior of the legacy backend.
Fixes#15975 (if this is the intended behavior)
Change-Id: Id277959069b8b8bf9958fa8f2cbc762c752a1a19
Reviewed-on: https://go-review.googlesource.com/23820
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Make sure auto names don't conflict with function names. Before this CL,
we confused name a.len (the len field of the slice a) with a.len (the function
len declared on a).
Fixes#15961
Change-Id: I14913de697b521fb35db9a1b10ba201f25d552bb
Reviewed-on: https://go-review.googlesource.com/23789
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Machine supports (or the runtime simulates in soft float mode)
(u)int32<->float conversions. The frontend rewrites int64<->float
conversions to call to runtime function.
For int64->float32 conversion, the frontend generates
. . AS u(100) l(10) tc(1)
. . . NAME-main.~r1 u(1) a(true) g(1) l(9) x(8+0) class(PPARAMOUT) f(1) float32
. . . CALLFUNC u(100) l(10) tc(1) float32
. . . . NAME-runtime.int64tofloat64 u(1) a(true) x(0+0) class(PFUNC) tc(1) used(true) FUNC-func(int64) float64
The CALLFUNC node has type float32, whereas runtime.int64tofloat64
returns float64. The legacy backend implicitly makes a float64->float32
conversion. The SSA backend does not do implicit conversion, so we
insert an explicit CONV here.
All cmd/compile/internal/gc/testdata/*_ssa.go tests passed.
Progress on SSA for ARM. Still not complete.
Update #15365.
Change-Id: I30937c8ff977271246b068f48224693776804339
Reviewed-on: https://go-review.googlesource.com/23652
Reviewed-by: Keith Randall <khr@golang.org>
This CL adds support of Div, Mod, Convert, GetClosurePtr and 64-bit indexing
support to SSA backend for ARM.
Add tests for 64-bit indexing to cmd/compile/internal/gc/testdata/string_ssa.go.
Tests cmd/compile/internal/gc/testdata/*_ssa.go passed, except compound_ssa.go
and fp_ssa.go.
Progress on SSA for ARM. Still not complete. Essentially the only unsupported
part is floating point.
Updates #15365.
Change-Id: I269e88b67f641c25e7a813d910c96d356d236bff
Reviewed-on: https://go-review.googlesource.com/23542
Reviewed-by: David Chase <drchase@google.com>
Introduce dec64 rules to (generically) decompose 64-bit integer on
32-bit architectures. 64-bit integer is composed/decomposed with
Int64Make/Hi/Lo ops, as for complex types.
The idea of dealing with Add64 is the following:
(Add64 (Int64Make xh xl) (Int64Make yh yl))
->
(Int64Make
(Add32withcarry xh yh (Select0 (Add32carry xl yl)))
(Select1 (Add32carry xl yl)))
where Add32carry returns a tuple (flags,uint32). Select0 and Select1
read the first and the second component of the tuple, respectively.
The two Add32carry will be CSE'd.
Similarly for multiplication, Mul32uhilo returns a tuple (hi, lo).
Also add support of KeepAlive, to fix build after merge.
Tests addressed_ssa.go, array_ssa.go, break_ssa.go, chan_ssa.go,
cmp_ssa.go, ctl_ssa.go, map_ssa.go, and string_ssa.go in
cmd/compile/internal/gc/testdata passed.
Progress on SSA for ARM. Still not complete.
Updates #15365.
Change-Id: I7867c76785a456312de5d8398a6b3f7ca5a4f7ec
Reviewed-on: https://go-review.googlesource.com/23213
Reviewed-by: Keith Randall <khr@golang.org>
When we do *p = f(), we might need to copy the return value from
f to p with a write barrier. The write barrier itself is a call,
so we need to copy the return value of f to a temporary location
before we call the write barrier function. Otherwise, the call
itself (specifically, marshalling the args to typedmemmove) will
clobber the value we're trying to write.
Fixes#15854
Change-Id: I5703da87634d91a9884e3ec098d7b3af713462e7
Reviewed-on: https://go-review.googlesource.com/23522
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Also fix argument offset for runtime calls.
Also fix LoadReg/StoreReg by generating instructions by type.
Progress on SSA backend for ARM. Still not complete.
Tests append_ssa.go, assert_ssa.go, loadstore_ssa.go, short_ssa.go, and
deferNoReturn.go in cmd/compile/internal/gc/testdata passed.
Updates #15365.
Change-Id: I0f0a2398cab8bbb461772a55241a16a7da2ecedf
Reviewed-on: https://go-review.googlesource.com/23212
Reviewed-by: David Chase <drchase@google.com>
As in the elimination of PHEAP|PPARAM in CL 23393,
this is something the front end can trivially take care of
and then not bother the back ends with.
It also eliminates some suspect (and only lightly exercised)
code paths in the back ends.
I don't have a smoking gun for this one but it seems
more clearly correct.
Change-Id: I3b3f5e669b3b81d091ff1e2fb13226a6f14c69d5
Reviewed-on: https://go-review.googlesource.com/23431
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
The liveness computation of parameters generally was never
correct, but forcing all parameters to be live throughout the
function covered up that problem. The new SSA back end is
too clever: even though it currently keeps the parameter values live
throughout the function, it may find optimizations that mean
the current values are not written back to the original parameter
stack slots immediately or ever (for example if a parameter is set
to nil, SSA constant propagation may replace all later uses of the
parameter with a constant nil, eliminating the need to write the nil
value back to the stack slot), so the liveness code must now
track the actual operations on the stack slots, exposing these
problems.
One small problem in the handling of arguments is that nodarg
can return ONAME PPARAM nodes with adjusted offsets, so that
there are actually multiple *Node pointers for the same parameter
in the instruction stream. This might be possible to correct, but
not in this CL. For now, we fix this by using n.Orig instead of n
when considering PPARAM and PPARAMOUT nodes.
The major problem in the handling of arguments is general
confusion in the liveness code about the meaning of PPARAM|PHEAP
and PPARAMOUT|PHEAP nodes, especially as contrasted with PAUTO|PHEAP.
The difference between these two is that when a local variable "moves"
to the heap, it's really just allocated there to start with; in contrast,
when an argument moves to the heap, the actual data has to be copied
there from the stack at the beginning of the function, and when a
result "moves" to the heap the value in the heap has to be copied
back to the stack when the function returns
This general confusion is also present in the SSA back end.
The PHEAP bit worked decently when I first introduced it 7 years ago (!)
in 391425ae. The back end did nothing sophisticated, and in particular
there was no analysis at all: no escape analysis, no liveness analysis,
and certainly no SSA back end. But the complications caused in the
various downstream consumers suggest that this should be a detail
kept mainly in the front end.
This CL therefore eliminates both the PHEAP bit and even the idea of
"heap variables" from the back ends.
First, it replaces the PPARAM|PHEAP, PPARAMOUT|PHEAP, and PAUTO|PHEAP
variable classes with the single PAUTOHEAP, a pseudo-class indicating
a variable maintained on the heap and available by indirecting a
local variable kept on the stack (a plain PAUTO).
Second, walkexpr replaces all references to PAUTOHEAP variables
with indirections of the corresponding PAUTO variable.
The back ends and the liveness code now just see plain indirected
variables. This may actually produce better code, but the real goal
here is to eliminate these little-used and somewhat suspect code
paths in the back end analyses.
The OPARAM node type goes away too.
A followup CL will do the same to PPARAMREF. I'm not sure that
the back ends (SSA in particular) are handling those right either,
and with the framework established in this CL that change is trivial
and the result clearly more correct.
Fixes#15747.
Change-Id: I2770b1ce3cbc93981bfc7166be66a9da12013d74
Reviewed-on: https://go-review.googlesource.com/23393
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix hardcoded flag register mask in ssa/flagalloc.go by auto-generating
the mask.
Also fix a mistake (in previous CL) about conditional branches.
Progress on SSA backend for ARM. Still not complete. Now "container/ring"
package compiles and tests passed.
Updates #15365.
Change-Id: Id7c8805c30dbb8107baedb485ed0f71f59ed6ea8
Reviewed-on: https://go-review.googlesource.com/23093
Reviewed-by: Keith Randall <khr@golang.org>
Introduce a KeepAlive op which makes sure that its argument is kept
live until the KeepAlive. Use KeepAlive to mark pointer input
arguments as live after each function call and at each return.
We do this change only for pointer arguments. Those are the
critical ones to handle because they might have finalizers.
Doing compound arguments (slices, structs, ...) is more complicated
because we would need to track field liveness individually (we do
that for auto variables now, but inputs requires extra trickery).
Turn off the automatic marking of args as live. That way, when args
are explicitly nulled, plive will know that the original argument is
dead.
The KeepAlive op will be the eventual implementation of
runtime.KeepAlive.
Fixes#15277
Change-Id: I5f223e65d99c9f8342c03fbb1512c4d363e903e5
Reviewed-on: https://go-review.googlesource.com/22365
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
This adds a sparse method for locating nearest ancestors
in a dominator tree, and checks blocks with more than one
predecessor for differences and inserts phi functions where
there are.
Uses reversed post order to cut number of passes, running
it from first def to last use ("last use" for paramout and
mem is end-of-program; last use for a phi input from a
backedge is the source of the back edge)
Includes a cutover from old algorithm to new to avoid paying
large constant factor for small programs. This keeps normal
builds running at about the same time, while not running
over-long on large machine-generated inputs.
Add "phase" flags for ssa/build -- ssa/build/stats prints
number of blocks, values (before and after linking references
and inserting phis, so expansion can be measured), and their
product; the product governs the cutover, where a good value
seems to be somewhere between 1 and 5 million.
Among the files compiled by make.bash, this is the shape of
the tail of the distribution for #blocks, #vars, and their
product:
#blocks #vars product
max 6171 28180 173,898,780
99.9% 1641 6548 10,401,878
99% 463 1909 873,721
95% 152 639 95,235
90% 84 359 30,021
The old algorithm is indeed usually fastest, for 99%ile
values of usually.
The fix to LookupVarOutgoing
( https://go-review.googlesource.com/#/c/22790/ )
deals with some of the same problems addressed by this CL,
but on at least one bug ( #15537 ) this change is still
a significant help.
With this CL:
/tmp/gopath$ rm -rf pkg bin
/tmp/gopath$ time go get -v -gcflags -memprofile=y.mprof \
github.com/gogo/protobuf/test/theproto3/combos/...
...
real 4m35.200s
user 13m16.644s
sys 0m36.712s
and pprof reports 3.4GB allocated in one of the larger profiles
With tip:
/tmp/gopath$ rm -rf pkg bin
/tmp/gopath$ time go get -v -gcflags -memprofile=y.mprof \
github.com/gogo/protobuf/test/theproto3/combos/...
...
real 10m36.569s
user 25m52.286s
sys 4m3.696s
and pprof reports 8.3GB allocated in the same larger profile
With this CL, most of the compilation time on the benchmarked
input is spent in register/stack allocation (cumulative 53%)
and in the sparse lookup algorithm itself (cumulative 20%).
Fixes#15537.
Change-Id: Ia0299dda6a291534d8b08e5f9883216ded677a00
Reviewed-on: https://go-review.googlesource.com/22342
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Progress on SSA backend for ARM. Still not complete. Now "helloworld"
function compiles and runs.
Updates #15365.
Change-Id: I02f66983cefdf07a6aed262fb4af8add464d8e9a
Reviewed-on: https://go-review.googlesource.com/22854
Reviewed-by: Keith Randall <khr@golang.org>
If b has exactly one predecessor, as happens
frequently with static calls, we can make
lookupVarOutgoing generate less garbage.
Instead of generating a value that is just
going to be an OpCopy and then get eliminated,
loop. This can lead to lots of looping.
However, this loop is way cheaper than generating
lots of ssa.Values and then eliminating them.
For a subset of the code in #15537:
Before:
28.31 real 36.17 user 1.68 sys
2282450944 maximum resident set size
After:
9.63 real 11.66 user 0.51 sys
638144512 maximum resident set size
Updates #15537.
Excitingly, it appears that this also helps
regular code:
name old time/op new time/op delta
Template 288ms ± 6% 276ms ± 7% -4.13% (p=0.000 n=21+24)
Unicode 143ms ± 8% 141ms ±10% ~ (p=0.287 n=24+25)
GoTypes 932ms ± 4% 874ms ± 4% -6.20% (p=0.000 n=23+22)
Compiler 4.89s ± 4% 4.58s ± 4% -6.46% (p=0.000 n=22+23)
MakeBash 40.2s ±13% 39.8s ± 9% ~ (p=0.648 n=23+23)
name old user-ns/op new user-ns/op delta
Template 388user-ms ±10% 373user-ms ± 5% -3.80% (p=0.000 n=24+25)
Unicode 203user-ms ± 6% 202user-ms ± 7% ~ (p=0.492 n=22+24)
GoTypes 1.29user-s ± 4% 1.17user-s ± 4% -9.67% (p=0.000 n=25+23)
Compiler 6.86user-s ± 5% 6.28user-s ± 4% -8.49% (p=0.000 n=25+25)
name old alloc/op new alloc/op delta
Template 51.5MB ± 0% 47.6MB ± 0% -7.47% (p=0.000 n=22+25)
Unicode 37.2MB ± 0% 37.1MB ± 0% -0.21% (p=0.000 n=25+25)
GoTypes 166MB ± 0% 138MB ± 0% -16.83% (p=0.000 n=25+25)
Compiler 756MB ± 0% 628MB ± 0% -16.96% (p=0.000 n=25+23)
name old allocs/op new allocs/op delta
Template 450k ± 0% 445k ± 0% -1.02% (p=0.000 n=25+25)
Unicode 356k ± 0% 356k ± 0% ~ (p=0.374 n=24+25)
GoTypes 1.31M ± 0% 1.25M ± 0% -4.18% (p=0.000 n=25+25)
Compiler 5.29M ± 0% 5.02M ± 0% -5.15% (p=0.000 n=25+23)
It also seems to help in other cases in which
phi insertion is a pain point (#14774, #14934).
Change-Id: Ibd05ed7b99d262117ece7bb250dfa8c3d1cc5dd2
Reviewed-on: https://go-review.googlesource.com/22790
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Turn SSAing of variables off when compiling with optimizations off.
This helps keep variable names around that would otherwise be
optimized away.
Fixes#14744
Change-Id: I31db8cf269c068c7c5851808f13e5955a09810ca
Reviewed-on: https://go-review.googlesource.com/22681
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
These symbols are de-duplicated in the linker but the compiler generates quite
many duplicates too: 2425 of 13769 total symbols for runtime.a for example.
De-duplicating them in the compiler saves the linker a bit of work.
Fixes#14983
Change-Id: I5f18e5f9743563c795aad8f0a22d17a7ed147711
Reviewed-on: https://go-review.googlesource.com/22293
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Updates #15462
Automatic refactor with sed -e.
Replace all oconv(op, 0) to string conversion with the raw op value
which fmt's %v verb can print directly.
The remaining oconv(op, FmtSharp) will be replaced with op.GoString and
%#v in the next CL.
Change-Id: I5e2f7ee0bd35caa65c6dd6cb1a866b5e4519e641
Reviewed-on: https://go-review.googlesource.com/22499
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The line numbers of ONAMEs are the location of their
declaration, not their use.
The line numbers of named OLITERALs are also the location
of their declaration.
Ignore both of these. Instead, we will inherit the line number from
the containing syntactic item.
Fixes#14742Fixes#15430
Change-Id: Ie43b5b9f6321cbf8cead56e37ccc9364d0702f2f
Reviewed-on: https://go-review.googlesource.com/22479
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Updates #15462
Semi automatic change with gofmt -r and hand fixups for callers outside
internal/gc.
All the uses of gc.Oconv outside cmd/compile/internal/gc were for the
Oconv(op, 0) form, which is already handled the Op.String method.
Replace the use of gc.Oconv(op, 0) with op itself, which will call
Op.String via the %v or %s verb. Unexport Oconv.
Change-Id: I84da2a2e4381b35f52efce427b2d6a3bccdf2526
Reviewed-on: https://go-review.googlesource.com/22496
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
As a nice side-effect, this allows us to
unify several code paths.
The terminology (low, high, max, simple slice expr,
full slice expr) is taken from the spec and
the examples in the spec.
This is a trial run. The plan, probably for Go 1.8,
is to change slice expressions to use Node.List
instead of OKEY, and to do some similar
tree structure changes for other ops.
Passes toolstash -cmp. No performance change.
all.bash passes with GO_GCFLAGS=-newexport.
Updates #15350
Change-Id: Ic1efdc36e79cdb95ae1636e9817a3ac8f83ab1ac
Reviewed-on: https://go-review.googlesource.com/22425
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Instead of switching on Ctype (which internally uses a type switch)
and then scattering lots of type assertions throughout the CTFOO case
clauses, just use type switches directly on the underlying constant
value.
Passes toolstash/buildall.
Change-Id: I9bc172cc67e5f391cddc15539907883b4010689e
Reviewed-on: https://go-review.googlesource.com/22384
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
They are guaranteed to be non-nil, no point in inserting
nil checks for them.
Fixes#15390
Change-Id: I3b9a0f2319affc2139dcc446d0a56c6785ae5a86
Reviewed-on: https://go-review.googlesource.com/22291
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Instead of using TARRAY for both arrays and slices, create a new
TSLICE kind to handle slices.
Also, get rid of the "DDDArray" distinction. While kinda ugly, it
seems likely we'll need to defer evaluating the constant bounds
expressions for golang.org/issue/13890.
Passes toolstash/buildall.
Change-Id: I8e45d4900e7df3a04cce59428ec8b38035d3cc3a
Reviewed-on: https://go-review.googlesource.com/22329
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Progress on SSA backend for ARM. Still not complete. It compiles a
Fibonacci function, but the caller picked the return value from an
incorrect offset. This CL adjusts it to match the stack frame layout
for architectures with link register.
Updates #15365.
Change-Id: I01e03c3e95f5503a185e8ac2b6d9caf4faf3d014
Reviewed-on: https://go-review.googlesource.com/22186
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
No point in passing the slice type to these functions.
All they need is the element type. One less indirection,
maybe a few less []T type descriptors in the binary.
Change-Id: Ib0b83b5f14ca21d995ecc199ce8ac00c4eb375e6
Reviewed-on: https://go-review.googlesource.com/22275
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Some of the Debug[x] flags are actually boolean too, but not all, so
they need to be handled separately.
While here, change some obj.Flagstr and obj.Flagint64 calls to
directly use flag.StringVar and flag.Int64Var instead.
Change-Id: Iccedf6fed4328240ee2257f57fe6d66688f237c4
Reviewed-on: https://go-review.googlesource.com/22052
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
When a struct is SSAable, we will name its component parts
by their field names. For example,
type T struct {
a, b, c int
}
If we ever need to spill a variable x of type T, we will
spill its individual components to variables named x.a, x.b,
and x.c.
Change-Id: I857286ff1f2597f2c4bbd7b4c0b936386fb37131
Reviewed-on: https://go-review.googlesource.com/21389
Reviewed-by: David Chase <drchase@google.com>
Be more careful about inserting instrumentation in racewalk.
If the node being instrumented is an OAS, and it has a non-
empty Ninit, then append instrumentation to the Ninit list
rather than letting it be inserted before the OAS (and the
compilation of its init list). This deals with the case that
the Ninit list defines a variable used in the RHS of the OAS.
Fixes#15091.
Change-Id: Iac91696d9104d07f0bf1bd3499bbf56b2e1ef073
Reviewed-on: https://go-review.googlesource.com/21771
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: David Chase <drchase@google.com>
Information about CPU architectures (e.g., name, family, byte
ordering, pointer and register size) is currently redundantly
scattered around the source tree. Instead consolidate the basic
information into a single new package cmd/internal/sys.
Also, introduce new sys.I386, sys.AMD64, etc. names for the constants
'8', '6', etc. and replace most uses of the latter. The notable
exceptions are a couple of error messages that still refer to the old
char-based toolchain names and function reltype in cmd/link.
Passes toolstash/buildall.
Change-Id: I8a6f0cbd49577ec1672a98addebc45f767e36461
Reviewed-on: https://go-review.googlesource.com/21623
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Many of Type's fields are etype-specific.
This CL organizes them into their own auxiliary types,
duplicating a few fields as necessary,
and adds an Extra field to hold them.
It also sorts the remaining fields for better struct packing.
It also improves documentation for most fields.
This reduces the size of Type at the cost of some extra allocations.
There's no CPU impact; memory impact below.
It also makes the natural structure of Type clearer.
Passes toolstash -cmp on all architectures.
Ideas for future work in this vein:
(1) Width and Align probably only need to be
stored for Struct and Array types.
The refactoring to accomplish this would hopefully
also eliminate TFUNCARGS and TCHANARGS entirely.
(2) Maplineno is sparsely used and could probably better be
stored in a separate map[*Type]int32, with mapqueue updated
to store both a Node and a line number.
(3) The Printed field may be removable once the old (non-binary)
importer/exported has been removed.
(4) StructType's fields field could be changed from *[]*Field to []*Field,
which would remove a common allocation.
(5) I believe that Type.Nod can be moved to ForwardType. Separate CL.
name old alloc/op new alloc/op delta
Template 57.9MB ± 0% 55.9MB ± 0% -3.43% (p=0.000 n=50+50)
Unicode 38.3MB ± 0% 37.8MB ± 0% -1.39% (p=0.000 n=50+50)
GoTypes 185MB ± 0% 180MB ± 0% -2.56% (p=0.000 n=50+50)
Compiler 824MB ± 0% 806MB ± 0% -2.19% (p=0.000 n=50+50)
name old allocs/op new allocs/op delta
Template 486k ± 0% 497k ± 0% +2.25% (p=0.000 n=50+50)
Unicode 377k ± 0% 379k ± 0% +0.55% (p=0.000 n=50+50)
GoTypes 1.39M ± 0% 1.42M ± 0% +1.63% (p=0.000 n=50+50)
Compiler 5.52M ± 0% 5.57M ± 0% +0.84% (p=0.000 n=47+50)
Change-Id: I828488eeb74902b013d5ae4cf844de0b6c0dfc87
Reviewed-on: https://go-review.googlesource.com/21611
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
No point in doing anything for x=x assignments.
In addition, skipping these assignments prevents generating:
VARDEF x
COPY x -> x
which is bad because x is incorrectly considered
dead before the vardef.
Fixes#14904
Change-Id: I6817055ec20bcc34a9648617e0439505ee355f82
Reviewed-on: https://go-review.googlesource.com/21470
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Signed-off-by: Eric Engestrom <eric@engestrom.ch>
Change-Id: I91873aaebf79bdf1c00d38aacc1a1fb8d79656a7
Reviewed-on: https://go-review.googlesource.com/21433
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Replace isideal(t) with t.IsUntyped().
Replace Istype(t, k) with t.IsKind(k).
Replace isnilinter(t) with t.IsEmptyInterface().
Also replace a lot of t.IsKind(TFOO) with t.IsFoo().
Replacements prepared mechanically with gofmt -w -r.
Passes toolstash -cmp.
Change-Id: Iba48058f3cc863e15af14277b5ff5e729e67e043
Reviewed-on: https://go-review.googlesource.com/21424
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Compound AUTO types weren't named previously. That was because live
variable analysis (plive.go) doesn't handle spilling to compound types.
It can't handle them because there is no valid place to put VARDEFs when
regalloc is spilling compound types.
compound types = multiword builtin types: complex, string, slice, and
interface.
Instead, we split named AUTOs into individual one-word variables. For
example, a string s gets split into a byte ptr s.ptr and an integer
s.len. Those two variables can be spilled to / restored from
independently. As a result, live variable analysis can handle them
because they are one-word objects.
This CL will change how AUTOs are described in DWARF information.
Consider the code:
func f(s string, i int) int {
x := s[i:i+5]
g()
return lookup(x)
}
The old compiler would spill x to two consecutive slots on the stack,
both named x (at offsets 0 and 8). The new compiler spills the pointer
of x to a slot named x.ptr. It doesn't spill x.len at all, as it is a
constant (5) and can be rematerialized for the call to lookup.
So compound objects may not be spilled in their entirety, and even if
they are they won't necessarily be contiguous. Such is the price of
optimization.
Re-enable live variable analysis tests. One test remains disabled, it
fails because of #14904.
Change-Id: I8ef2b5ab91e43a0d2136bfc231c05d100ec0b801
Reviewed-on: https://go-review.googlesource.com/21233
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Replace Isfixedarray, Isslice, and Isinter with the IsArray, IsSlice,
and IsInterface methods added for SSA. Rewrite performed mechanically
using gofmt -w -r "Isfoo(t) -> t.IsFoo()".
Because the IsFoo methods panic when given a nil pointer, a handful of
call sites had to be modified to check for nil Type values. These
aren't strictly necessary, because nil Type values should only occur
in invalid Go source programs, so it would be okay if we panicked on
them and gave up type checking the rest of the package. However, there
are a couple regress tests that expect we continue, so add checks to
keep those tests passing. (See #15029.)
Passes toolstash -cmp.
Change-Id: I511c6ac4cfdf3f9cbdb3e52a5fa91b6d09d82f80
Reviewed-on: https://go-review.googlesource.com/21336
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This removes almost all direct access to
Type’s heavily overloaded Type field.
Mostly generated by eg, manually checked.
Significant manual changes:
* reflect.go's typPkg used Type indiscriminately.
Use it only for specific etypes.
* gen.go's visitComponents contained a usage of Type
with structs. Using Type for structs no longer
occurs, and the Fatal contained therein has not triggered,
so it has been axed.
* Scary code in cgen.go's cgen_slice is now explicitly scary.
Passes toolstash -cmp.
Change-Id: I2dbfb3c959da7ae239f964d83898c204affcabc6
Reviewed-on: https://go-review.googlesource.com/21331
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously, t.IsPtr() reported whether t was represented with a
pointer, but some of its callers expected it to report whether t is an
actual Go pointer. Resolve this by renaming t.IsPtr to t.IsPtrShaped
and adding a new t.IsPtr method to report Go pointer types.
Updated a couple callers in gc/ssa.go to use IsPtr instead of
IsPtrShaped.
Passes toolstash -cmp.
Updates #15028.
Change-Id: I0a8154b5822ad8a6ad296419126ad01a3d2a5dc5
Reviewed-on: https://go-review.googlesource.com/21232
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
One intrinsic was needed to help get the very best
performance out of a future GC; as long as that one was
being added, I also added Bswap since that is sometimes
a handy thing to have. I had intended to fill out the
bit-scan intrinsic family, but the mismatch between the
"scan forward" instruction and "count leading zeroes"
was large enough to cause me to leave it out -- it poses
a dilemma that I'd rather dodge right now.
These intrinsics are not exposed for general use.
That's a separate issue requiring an API proposal change
( https://github.com/golang/proposal )
All intrinsics are tested, both that they are substituted
on the appropriate architecture, and that they produce the
expected result.
Change-Id: I5848037cfd97de4f75bdc33bdd89bba00af4a8ee
Reviewed-on: https://go-review.googlesource.com/20564
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Start working on arm port. Gets close to correct
code for fibonacci:
func fib(n int) int {
if n < 2 {
return n
}
return fib(n-1) + fib(n-2)
}
Still a lot to do, but this is a good starting point.
Cleaned up some arch-specific dependencies in regalloc.
Change-Id: I4301c6c31a8402168e50dcfee8bcf7aee73ea9d5
Reviewed-on: https://go-review.googlesource.com/21000
Reviewed-by: David Chase <drchase@google.com>
Don't write back parts of a slicing operation if they
are unchanged from the source of the slice. For example:
x.s = x.s[0:5] // don't write back pointer or cap
x.s = x.s[:5] // don't write back pointer or cap
x.s = x.s[:5:7] // don't write back pointer
There is more to be done here, for example:
x.s = x.s[:len(x.s):7] // don't write back ptr or len
This CL can't handle that one yet.
Fixes#14855
Change-Id: Id1e1a4fa7f3076dc1a76924a7f1cd791b81909bb
Reviewed-on: https://go-review.googlesource.com/20954
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Also give them more idiomatic Go names. Adding godocs is outside the
scope of this CL. (Besides, the method names almost all directly
parallel an underlying math/big.Int or math/big.Float method.)
CL prepared mechanically with sed (for rewriting mpint.go/mpfloat.go)
and gofmt (for rewriting call sites).
Passes toolstash -cmp.
Change-Id: Id76f4aee476ba740f48db33162463e7978c2083d
Reviewed-on: https://go-review.googlesource.com/20909
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
The Node type ODOT and its variants all represent a selector, with a
simple name to the right of the dot. Before this change this was
represented by using an ONAME Node in the Right field. This ONAME node
served no useful purpose. This CL changes these Node types to store the
symbol in the Sym field instead, thus not requiring allocating a Node
for each selector.
When compiling x/tools/go/types this CL eliminates nearly 5000 calls to
newname and reduces the total number of Nodes allocated by about 6.6%.
It seems to cut compilation time by 1 to 2 percent.
Getting this right was somewhat subtle, and I added two dubious changes
to produce the exact same output as before. One is to ishairy in
inl.go: the ONAME node increased the cost of ODOT and friends by 1, and
I retained that, although really ODOT is not more expensive than any
other node. The other is to varexpr in walk.go: because the ONAME in
the Right field of an ODOT has no class, varexpr would always return
false for an ODOT, although in fact for some ODOT's it seemingly ought
to return true; I added an && false for now. I will send separate CLs,
that will break toolstash -cmp, to clean these up.
This CL passes toolstash -cmp.
Change-Id: I4af8a10cc59078c436130ce472f25abc3a9b2f80
Reviewed-on: https://go-review.googlesource.com/20890
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Make sure we don't generate write barriers in runtime
code that is marked to forbid write barriers.
Implement the optimization that if we're writing a sliced
slice back to the location it came from, we don't need a
write barrier.
Fixes#14784
Change-Id: I04b6a3b2ac303c19817e932a36a3b006de103aaa
Reviewed-on: https://go-review.googlesource.com/20791
Reviewed-by: Austin Clements <austin@google.com>
This is an automated rewrite of all the calls of the form:
for f, it := IterFields(t); f != nil; f = it.Next() { ... }
Followup CLs will work on cleaning up the remaining cases.
Change-Id: Ic1005ad45ae0b50c63e815e34e507e2d2644ba1a
Reviewed-on: https://go-review.googlesource.com/20794
Reviewed-by: David Crawshaw <crawshaw@golang.org>
For every string constant the compiler was creating 2 Sym's and 2
Node's. It would never refer to them again, but would keep them alive
in gostringpkg. This changes the code to just use obj.LSym's instead.
When compiling x/tools/go/types, this yields about a 15% reduction in
the number of calls to newname and a 3% reduction in the total number of
Node objects. Unfortunately I couldn't see any change in compile time,
but reducing memory usage is desirable anyhow.
Passes toolstash -cmp.
Change-Id: I24f1cb1e6cff0a3afba4ca66f7166874917a036b
Reviewed-on: https://go-review.googlesource.com/20792
Reviewed-by: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Keep track of how many uses each Value has. Each appearance in
Value.Args and in Block.Control counts once.
The number of uses of a value is generically useful to
constrain rewrite rules. For instance, we might want to
prevent merging index operations into loads if the same
index expression is used lots of times.
But I have one use in particular for which the use count is required.
We must make sure we don't combine ops with loads if the load has
more than one use. Otherwise, we may split a single load
into multiple loads and that breaks perceived behavior in
the presence of races. In particular, the load of m.state
in sync/mutex.go:Lock can't be done twice. (I have a separate
CL which triggers the mutex failure. This CL has a test which
demonstrates a similar failure.)
Change-Id: Icaafa479239f48632a069d0c3f624e6ebc6b1f0e
Reviewed-on: https://go-review.googlesource.com/20790
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
All of a struct's fields have to fit into memory anyway, so index them
with int instead of int64. This also makes it nicer for
cmd/compile/internal/gc to reuse the same NumFields function.
Change-Id: I210be804a0c33370ec9977414918c02c675b0fbe
Reviewed-on: https://go-review.googlesource.com/20691
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Turns out there were only two call sites that expected
t.Field(t.NumFields()) to return nil.
Change-Id: I4679988d38ee9d7c9d89883537a17046717b2a77
Reviewed-on: https://go-review.googlesource.com/20731
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Parts of the SSA compiler in package gc contain amd64-specific code,
most notably Prog generation. Move this code into package amd64, so that
other architectures can be added more easily.
In package gc, this change is just moving code. There are no functional
changes or even any larger structural changes beyond changing function
names (mostly for export).
In the cmd/compile/internal/ssa/gen tool, more information is included
in arch to remove the AMD64-specific behavior in the main portion of the
tool. The generated opGen.go is identical.
Change-Id: I8eb37c6e6df6de1b65fa7dab6f3bc32c29daf643
Reviewed-on: https://go-review.googlesource.com/20609
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Line numbers are always int32, so the Warnl function should take the
line number as an int32 as well. This matches gc.Warnl and removes
a cast every place it's used.
Change-Id: I5d6201e640d52ec390eb7174f8fd8c438d4efe58
Reviewed-on: https://go-review.googlesource.com/20662
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reuse auto symbols so cse can eliminate OpAddrs that refer to
them.
Change-Id: I69e6a3f77a3a33946459cf8c6eccf223f6125048
Reviewed-on: https://go-review.googlesource.com/20569
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The location of VARDEFs is incorrect for PPARAMOUT variables
which are also used as temporary locations. We put in VARDEFs
when setting the variable at return time, but when the location
is also used as a temporary the lifetime values are wrong.
Fix copyelim to update the names map properly. This is a
real name bug fix which, as a result, allows me to
write a reasonable test to trigger the PPARAMOUT bug.
This is kind of a band-aid fix for #14591. A more pricipled
fix (which allows values to be stored in the return variable
earlier than the return point) will be harder.
Fixes#14591
Change-Id: I7df8ae103a982d1f218ed704c080d7b83cdcfdd9
Reviewed-on: https://go-review.googlesource.com/20457
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
I would like to add a
func (t *Type) Elem() *Type
method to package gc, but that would collide with the existing
func (t *Type) Elem() ssa.Type
method needed to make *gc.Type implement ssa.Type. Because the latter
is much less widely used right now than the former will be, this CL
renames it to ElemType.
Longer term, hopefully gc and ssa will share a common Type interface,
and ElemType can go away.
Change-Id: I270008515dc4c01ef531cf715637a924659c4735
Reviewed-on: https://go-review.googlesource.com/20546
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Make sure we do any just-before-return cleanup on all paths out of a
function, including when recovering. Each exit path should include
deferreturn (if there are any defers) and then the exit
code (e.g. copying heap-escaping return values back to the stack).
Introduce a Defer SSA block type which has two outgoing edges - one the
fallthrough edge (the defer was queued successfully) and one which
immediately returns (the defer had a successful recover() call and
normal execution should resume at the return point).
Fixes#14725
Change-Id: Iad035c9fd25ef8b7a74dafbd7461cf04833d981f
Reviewed-on: https://go-review.googlesource.com/20486
Reviewed-by: David Chase <drchase@google.com>
This CL was mostly produced by a one-off automated rewrite tool
looking for statements like "for X := T.Type; X != nil; X = X.Down"
and a few minor variations.
Passes toolstash -cmp.
Change-Id: Ib22705e37d078ef97841ee2e08f60bdbcabb94ad
Reviewed-on: https://go-review.googlesource.com/20520
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Accessing the n'th field of a struct is fairly common, and in
particular accessing the 0'th field of the receiver parameter list is
very common. Add helper methods for both of these tasks and update
code to make use of them.
Change-Id: I81f551fecdca306b3800636caebcd0dc106f2ed7
Reviewed-on: https://go-review.googlesource.com/20498
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dave Cheney <dave@cheney.net>
Mix in several other minor cleanups, including adding some new methods
to Nodes: Index, Addr, SetIndex, SetNodes.
Passes toolstash -cmp.
Update #14473.
Change-Id: I8bd4ae3fde7c5e20ba66e7dd1654fbc70c3ddeb8
Reviewed-on: https://go-review.googlesource.com/20491
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL was automatically generated using a special-purpose AST
rewriting tool, followed by manual editing to put some comments back in
the right places and fix some bad line breaks.
The result is not perfect but it's a big step toward getting back to
sanity, and because it was automatically generated there is a decent
chance that it is correct.
Passes toolstash -cmp.
Update #14473.
Change-Id: I01c09078a6d78e2b008bc304d744b79469a38d3d
Reviewed-on: https://go-review.googlesource.com/20440
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
More idiomatic naming (in particular, matches the naming used for
go/types.Signature).
Also, convert more code to use these methods and/or IterFields.
(Still more to go; only made a quick pass for low hanging fruit.)
Passes toolstash -cmp.
Change-Id: I61831bfb1ec2cd50d4c7efc6062bca4e0dcf267b
Reviewed-on: https://go-review.googlesource.com/20451
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Eliminates type conversions in a bunch of Oconv(int(n.Op), ...) calls.
Notably, this identified a misuse of Oconv in amd64/gsubr.go to try to
print an assembly instruction op instead of a compiler node op.
Change-Id: I93b5aa49fe14a5eaf868b05426d3b8cd8ab52bc5
Reviewed-on: https://go-review.googlesource.com/20298
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
OffPtr allocates less and is easier to optimize.
With this change, the OffPtr collapsing opt
rule matches increase from 160k to 263k,
and the Load-after-Store opt rule matches
increase from 217 to 853.
Change-Id: I763426a3196900f22a367f7f6d8e8047b279653d
Reviewed-on: https://go-review.googlesource.com/20273
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
Reviewed-by: Keith Randall <khr@golang.org>
Instead make substArgTypes responsible for cloning the function
definition Node and the function signature Type tree.
Passes toolstash -cmp.
Change-Id: I9ec84c90a7ae83d164d3f578e84a91cf1490d8ab
Reviewed-on: https://go-review.googlesource.com/20239
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Also fix some uses of nodeSeqIterator.Len, and fix the implementation in
nodesIterator.
Passes toolstash -cmp.
Update #14473.
Change-Id: I228871470234b7f1314ffd2aae8a4c0624c35f98
Reviewed-on: https://go-review.googlesource.com/20231
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
To turn ssa compilation on or off altogether, use
-ssa=1 or -ssa=0. Default is on.
To turn on or off consistency checks, do
-d=ssa/check/on or -d=ssa/check/off. Default is on for now.
Change-Id: I277e0311f538981c8b9c62e7b7382a0c8755ce4c
Reviewed-on: https://go-review.googlesource.com/20217
Reviewed-by: David Chase <drchase@google.com>
- removed lots of unnecessary int(x) casts
- removed parserline() - was inconsistently used anyway
- minor simplifications in dcl.go
Change-Id: Ibf7de679eea528a31c9692ef1c76a1d9b3239211
Reviewed-on: https://go-review.googlesource.com/20131
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Do some easy TODOs.
Move a bunch of other TODOs into bugs.
Change-Id: Iaba9dad6221a2af11b3cbcc512875f4a85842873
Reviewed-on: https://go-review.googlesource.com/20114
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This check seems to apply to all code instrumentation, not just -race.
Fixes#14589.
Change-Id: I16ae07749ede7c1e6ed06f472711638d195034ac
Reviewed-on: https://go-review.googlesource.com/20113
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marvin Stenger <marvin.stenger94@gmail.com>
SSA is a real compiler now, don't trigger on function names.
Change-Id: Iaf6dd78248c1b6ca2630275bd59f7ea7b7576497
Reviewed-on: https://go-review.googlesource.com/20088
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add writeback code to each return location which copies
the final result back to the correct stack location.
Cgo plays tricky games by taking the address of a
in f(a int) (b int) and then using that address to
modify b. So for cgo-generated Go code, disable the
SSAing of output args.
Update #14511
Change-Id: I95cba727d53699d31124eef41db0e03935862be9
Reviewed-on: https://go-review.googlesource.com/19988
Reviewed-by: Todd Neal <todd@tneal.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It gets rewritten to an xor by the linker also.
Change-Id: Iae35130325d41bd1a09b7e971190cae6f4e17fac
Reviewed-on: https://go-review.googlesource.com/20058
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Movups is 1 byte smaller than movapd that we currently use.
Change-Id: I22f771f066529352722a28543535ec43497cb9c5
Reviewed-on: https://go-review.googlesource.com/19938
Reviewed-by: David Chase <drchase@google.com>
Avoid targeting a partial register with load;
ensure source of load (writebarrier) is aligned.
Better yet would be "CMPB $1,writebarrier" but that requires
wrestling with flagalloc (mem operand complicates moving
instruction around).
Didn't see a change in time for
benchcmd -n 10 Build go build net/http
Verified that we clean the code up properly:
0x20a8 <main.main+104>: mov 0xc30a2(%rip),%eax
# 0xc5150 <runtime.writeBarrier>
0x20ae <main.main+110>: test %al,%al
Change-Id: Id5fb8c260eaec27bd727cb0ae1476c60343b0986
Reviewed-on: https://go-review.googlesource.com/19998
Reviewed-by: Keith Randall <khr@golang.org>
plive doesn't like the fact that we put JMPs right
after CALL ops to select{send,recv}.
Fixes SSA -N build.
Change-Id: I9b3c9e5293196094fd5a6206dd2f99784951f7a9
Reviewed-on: https://go-review.googlesource.com/19982
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Just like we do for integer loads/stores.
Update #14511
Change-Id: Ic6ca6b54301438a5701ea5fb0be755451cb24d45
Reviewed-on: https://go-review.googlesource.com/19923
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
The -d compiler flag can also specify ssa phase and flag,
for example -d=ssa/generic_cse/time,ssa/generic_cse/stats
Spaces in the phase names can be specified with an
underscore. Flags currently parsed (not necessarily
recognized by the phases yet) are:
on, off, mem, time, debug, stats, and test
On, off and time are handled in the harness,
debug, stats, and test are interpreted by the phase itself.
The pass is now attached to the Func being compiled, and a
new method logStats(key, ...value) on *Func to encourage a
semi-standardized format for that output. Output fields
are separated by tabs to ease digestion by awk and
spreadsheets. For example,
if f.pass.stats > 0 {
f.logStat("CSE REWRITES", rewrites)
}
Change-Id: I16db2b5af64c50ca9a47efeb51d961147a903abc
Reviewed-on: https://go-review.googlesource.com/19885
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
Found looking at mapaccess1_faststr.
runtime.throw never returns.
Do x+y+c with an LEA.
Change-Id: I27ea6669324242a6302397cbdc73230891d97591
Reviewed-on: https://go-review.googlesource.com/19911
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Helps keep line numbers around for debugging, particularly
for break and continue statements (which often compile
down to nothing).
Update #14379
Change-Id: I6ea06aa887b0450d9ba4f11e319e5c263f5a98ba
Reviewed-on: https://go-review.googlesource.com/19848
Reviewed-by: David Chase <drchase@google.com>
The upper bits of 8/16/32 bit constants are undefined. We need to
truncate in order to prevent x86.oclass misidentifying the size of the
constant.
Fixes#14389
Change-Id: I3e5ff79cd904376572a93f489ba7e152a5cb6e60
Reviewed-on: https://go-review.googlesource.com/19740
Reviewed-by: Keith Randall <khr@golang.org>
Semi-regular merge from tip to dev.ssa.
Two fixes:
1) Mark selectgo as not returning. This caused problems
because there are no VARKILL ops on the selectgo path,
causing things to be marked live that shouldn't be.
2) Tell the amd64 assembler that addressing modes like
name(SP)(AX*4) are ok.
Change-Id: I9ca81c76391b1a65cc47edc8610c70ff1a621913
Panic doesn't return, so record that we immediately exit after a panic
call. This will help code analysis.
Change-Id: I4d1f67494f97b6aee130c43ff4e44307b2b0f149
Reviewed-on: https://go-review.googlesource.com/19303
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
The frontend does this for 32 bits and below, but SSA needs
to do it for 64 bits. The algorithms are all copied from
cgen.go:cgen_div.
Speeds up TimeFormat substantially: ~40% slower to ~10% slower.
Change-Id: I023ea2eb6040df98ccd9105e15ca6ea695610a7a
Reviewed-on: https://go-review.googlesource.com/19302
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
Use just a single write barrier flag test, even if there
are multiple pointer fields in a struct.
This helps move more of the wb-specific code (like the LEA
needed to materialize the write address) into the unlikely path.
Change-Id: Ic7a67145904369c4ff031e464d51267d71281c8f
Reviewed-on: https://go-review.googlesource.com/19085
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
If the output register is one of the input registers,
we can use a real add instead of LEA.
Change-Id: Ide58f1536afb077c0b939d3a8c7555807fd1c5e3
Reviewed-on: https://go-review.googlesource.com/19234
Reviewed-by: Alexandru Moșoi <alexandru@mosoi.ro>
Mostly indexed loads. A few more LEA cases.
Change-Id: Idc1d447ed0dd6e906cd48e70307a95e77f61cf5f
Reviewed-on: https://go-review.googlesource.com/19172
Reviewed-by: Todd Neal <todd@tneal.org>
Run-TryBot: Keith Randall <khr@golang.org>
Add the aux type to opcodes.
Add rematerializeable as a flag.
Change-Id: I906e19281498f3ee51bb136299bf26e13a54b2ec
Reviewed-on: https://go-review.googlesource.com/19088
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
Small optimizations I noticed while looking at Giovanni's test cases.
More shifts by constants.
Indexed stores for smaller types.
Fold LEA into loads/stores.
More extending loads.
CMP $0 of AND -> TEST
Fix order of TEST ops.
Giovanni's test cases at https://gist.github.com/rasky/62fba94e3a20d1b05b2a
Change-Id: I7077bc0b5319bf05767eeb39f401f4bb4b39f635
Reviewed-on: https://go-review.googlesource.com/19086
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
Reviewed-by: David Chase <drchase@google.com>
From memory profiling, about 3% reduction in allocation count.
Change-Id: I4b662d55b8a94fe724759a2b22f05a08d0bf40f8
Reviewed-on: https://go-review.googlesource.com/19103
Reviewed-by: Keith Randall <khr@golang.org>
Rename StoreConst to ValAndOff so we can use it for other ops.
Make ValAndOff print nicely.
Add some notes & checks related to my aborted attempt to
implement combined CMP+load ops.
Change-Id: I2f901d12d42bc5a82879af0334806aa184a97e27
Reviewed-on: https://go-review.googlesource.com/18947
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: David Chase <drchase@google.com>
The old write barriers used _nostore versions, which
don't work for Ian's cgo checker. Instead, we adopt the
same write barrier pattern as the default compiler.
It's a bit trickier to code up but should be more efficient.
Change-Id: I6696c3656cf179e28f800b0e096b7259bd5f3bb7
Reviewed-on: https://go-review.googlesource.com/18941
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Some tests make multiple Funcs per Config at once.
With value & block caching, we can't do that any more.
Change-Id: Ibdb60aa2fcf478f1726b3be0fcaa06b04433eb67
Reviewed-on: https://go-review.googlesource.com/19081
Reviewed-by: Keith Randall <khr@golang.org>
It is one of the slowest compiler phases right now, and we
run two of them.
Instead of using a map to make the initial partition, use a sort.
It is much less memory intensive.
Do a few optimizations to avoid work for size-1 equivalence classes.
Implement -N.
Change-Id: I1d2d85d3771abc918db4dd7cc30b0b2d854b15e1
Reviewed-on: https://go-review.googlesource.com/19024
Reviewed-by: David Chase <drchase@google.com>
The x86 backend automatically rewrites MOV $0, AX to
XOR AX, AX. That rewrite isn't ok when the flags register
is live across the MOV. Keep track of which moves care
about preserving flags, then disable this rewrite for them.
On x86, Prog.Mark was being used to hold the length of the
instruction. We already store that in Prog.Isize, so no
need to store it in Prog.Mark also. This frees up Prog.Mark
to hold a bitmask on x86 just like all the other architectures.
Update #12405
Change-Id: Ibad8a8f41fc6222bec1e4904221887d3cc3ca029
Reviewed-on: https://go-review.googlesource.com/18861
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
The conversion from -0.0 to +0.0 happens inside mpgetflt now.
The SSA code doesn't need this fix any more.
Change-Id: I6cd4f4a4e75b13cf284ebbb95b08af050ed9891c
Reviewed-on: https://go-review.googlesource.com/18942
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Empty blocks are introduced to remove critical edges.
After regalloc, we can remove any of the added blocks
that are still empty.
Change-Id: I0b40e95ac3a6cc1e632a479443479532b6c5ccd9
Reviewed-on: https://go-review.googlesource.com/18833
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Break small structs up into their components so they
can be registerized.
Change StructSelect to use field indexes instead of
field offsets, as field offsets aren't unique in the
presence of zero-sized fields.
Change-Id: I2f1dc89f7fa58e1cf58aa1a32b238959d53f62e4
Reviewed-on: https://go-review.googlesource.com/18570
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Redo how we keep track of forward references when building SSA.
When the forward reference is resolved, update the Value node
in place.
Improve the phi elimination pass so it can simplify phis of phis.
Give SSA package access to decoded line numbers. Fix line numbers
for constant booleans.
Change-Id: I3dc9896148d260be2f3dd14cbe5db639ec9fa6b7
Reviewed-on: https://go-review.googlesource.com/18674
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Distinguish move/load/store ops. Unify some of this code a bit.
Reduces Mandelbrot slowdown with SSA from 58% to 12%.
Change-Id: I3276eaebcbcdd9de3f8299c79b5f25c0429194c4
Reviewed-on: https://go-review.googlesource.com/18677
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
If a failure occurs in SSA processing, we always report the
last line of the function we're compiling. Modify the callbacks
from SSA to the GC compiler so we can pass a line number back
and use it in Fatalf.
Change-Id: Ifbfad50d5e167e997e0a96f0775bcc369f5c397e
Reviewed-on: https://go-review.googlesource.com/18599
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Add new constant-flags opcodes. These can be generated from
comparisons that we know the result of, like x&31 < 32.
Constant-fold the constant-flags opcodes into all flag users.
Reorder some CMPxconst args so they read in the comparison direction.
Reorg deadcode removal a bit - it needs to remove the OpCopy ops it
generates when strength-reducing Phi ops. So it needs to splice out all
the dead blocks and do a copy elimination before it computes live
values.
Change-Id: Ie922602033592ad8212efe4345394973d3b94d9f
Reviewed-on: https://go-review.googlesource.com/18267
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
In code that does:
var x, z int32
var y int64
z = phi(x, int32(y))
We silently drop the int32 cast because truncation is a no-op.
The phi operation needs to make sure it uses the size of the
phi, not the size of its arguments, when generating spills.
Change-Id: I1f7baf44f019256977a46fdd3dad1972be209042
Reviewed-on: https://go-review.googlesource.com/18390
Reviewed-by: David Chase <drchase@google.com>
We used to compile everything with SSA and then decide whether
to use the result or not. It was useful when we were working
on coverage without much regard for correctness, but not so much now.
Instead, let's decide what we're going to compile and go through
the SSA compiler for only those functions.
TODO: next CL: get rid of all the UnimplementedF stuff.
Change-Id: If629addd8b62cd38ef553fd5d835114137885ce0
Reviewed-on: https://go-review.googlesource.com/17763
Reviewed-by: David Chase <drchase@google.com>
Make sure that when a pointer value is live across a function
call, we save it as a pointer. (And similarly a uintptr
live across a function call should not be saved as a pointer.)
Add a nasty test case.
This is probably what is preventing the merge from master
to dev.ssa. Signs point to something like this bug happening
in mallocgc.
Change-Id: Ib23fa1251b8d1c50d82c6a448cb4a4fc28219029
Reviewed-on: https://go-review.googlesource.com/16830
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Panics are only distinguished by their type and line number, so
if we can trigger two of those panics in the same line, use the
same panic call. For example, in a[i]+b[j] we need only one
panicindex call that both bounds checks can use.
Change-Id: Ia2b6d3b1a67f2775df05fb72b8a1b149833572b7
Reviewed-on: https://go-review.googlesource.com/16772
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Might as well, for a && !(b && c) and the like.
Change-Id: I2548b6e6ee5870e074bcef6edd56a7db6e81d70f
Reviewed-on: https://go-review.googlesource.com/16600
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Declare a function's arguments as having already been
spilled so their use just requires a restore.
Allow spill locations to be portions of larger objects the stack.
Required to load portions of compound input arguments.
Rename the memory input to InputMem. Use Arg for the
pre-spilled argument values.
Change-Id: I8fe2a03ffbba1022d98bfae2052b376b96d32dda
Reviewed-on: https://go-review.googlesource.com/16536
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Be more consistent about this. There's no reason to do the
pointer arithmetic on a different type, as sizeof(int) >=
sizeof(ptr) on all of our platforms. It simplifies our
rewrite rules also, except for a few that need duplication.
Add some more constant folding to get constant indexing and
slicing to fold down to nothing.
Change-Id: I3e56cdb14b3dc1a6a0514f0333e883f92c19e3c7
Reviewed-on: https://go-review.googlesource.com/16586
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
For the statement
if a && b { target }
the old code allocated a new variable v and did:
v = a
if a {
v = b
}
if v { goto target }
The new code does:
if a {
if b { goto target }
}
The new arrangement tends to generate much more efficient code. In
particular, there is no temporary variable and there is only one join
point instead of two.
The old code is still used for ANDAND and OROR which are not
direct descendents of IF or FOR statements.
Change-Id: I082f246d27c823c6f32d1287300e4b0911607507
Reviewed-on: https://go-review.googlesource.com/16584
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Some tests disabled, some bifurcated into _ssa and not,
with appropriate logging added to compiler.
"tests/live.go" in particular needs attention.
SSA-specific testing removed, since it's all SSA now.
Added "-run_skips" option to tests/run.go to simplify
checking whether a test still fails (or how it fails)
on a skipped platform.
The compiler now compiles with SSA by default.
If you don't want SSA, specify GOSSAHASH=n (or N) as
an environment variable. Function names ending in "_ssa"
are always SSA-compiled.
GOSSAFUNC=fname retains its "SSA for fname, log to ssa.html"
GOSSAPKG=pkg only has an effect when GOSSAHASH=n
GOSSAHASH=10101 etc retains its name-hash-matching behavior
for purposes of debugging.
See #13068
Change-Id: I8217bfeb34173533eaeb391b5f6935483c7d6b43
Reviewed-on: https://go-review.googlesource.com/16299
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
This improves cse and works correctly now that divide by zero is checked
explicitly.
Change-Id: If54fbe403ed5230b897afc5def644ba9f0056dfd
Reviewed-on: https://go-review.googlesource.com/16454
Run-TryBot: Todd Neal <todd@tneal.org>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Added an explicit compare-zero and branch-to-panic for
integer division and mod so that other optimizations will
not be fooled by their implicit panics.
Change-Id: Ibf96f636b541c0088861907c537a6beb4b99fa4c
Reviewed-on: https://go-review.googlesource.com/16450
Reviewed-by: Keith Randall <khr@golang.org>
For debugging, spill values to named variables instead of autotmp_
variables if possible. We do this by keeping a name -> value map
for each function, keep it up-to-date during deadcode elim, and use
it to override spill decisions in stackalloc.
It might even make stack frames a bit smaller, as it makes it easy
to identify a set of spills which are likely not to interfere.
This just works for one-word variables for now. Strings/slices
will be a separate CL.
Change-Id: Ie89eba8cab16bcd41b311c479ec46dd7e64cdb67
Reviewed-on: https://go-review.googlesource.com/16336
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Introduce opcodes that store a constant value.
AuxInt now needs to hold both the value to be stored and the
constant offset at which to store it. Introduce a StoreConst
type to help encode/decode these parts to/from an AuxInt.
Change-Id: I1631883abe035cff4b16368683e1eb3d2ccb674d
Reviewed-on: https://go-review.googlesource.com/16170
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
If the closure pointer gets spilled, we need to spill it with
pointer type to make stack copy and GC happy.
Change-Id: Ic108748e6b9caecd45522141f02c9422567376e3
Reviewed-on: https://go-review.googlesource.com/16363
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Replace REP MOVSB with all the copying techniques used by the
old compiler. Copy in chunks, DUFFCOPY, etc.
Introduces MOVO opcodes and an Int128 type to move around
16 bytes at a time.
Change-Id: I1e73e68ca1d8b3dd58bb4af2f4c9e5d9bf13a502
Reviewed-on: https://go-review.googlesource.com/16174
Reviewed-by: Todd Neal <todd@tneal.org>
Run-TryBot: Keith Randall <khr@golang.org>
Use faulting loads instead of test/jeq to do nil checks.
Fold nil checks into a following load/store if possible.
Makes binaries about 2% smaller.
Change-Id: I54af0f0a93c853f37e34e0ce7e3f01dd2ac87f64
Reviewed-on: https://go-review.googlesource.com/16287
Reviewed-by: David Chase <drchase@google.com>
Modified GOSSA{HASH.PKG} environment variable filters to
make it easier to make/run with all SSA for testing.
Disable attempts at SSA for architectures that are not
amd64 (avoid spurious errors/unimplementeds.)
Removed easy out for unimplemented features.
Add convert op for proper liveness in presence of uintptr
to/from unsafe.Pointer conversions.
Tweaked stack sizes to get a pass on windows;
1024 instead 768, was observed to pass at least once.
Change-Id: Ida3800afcda67d529e3b1cf48ca4a3f0fa48b2c5
Reviewed-on: https://go-review.googlesource.com/16201
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
getg reads from memory, so it should really have a
memory arg. It is critical in functions which call setg
to make sure getg gets ordered correctly with setg.
Change-Id: Ief4875421f741fc49c07b0e1f065ce2535232341
Reviewed-on: https://go-review.googlesource.com/16100
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reuse the ArgSymbol for nodes so that the Aux values will be equal for cse.
Change-Id: Iaae80bd19ff2d3f51b6c9049fd860e04baa6f175
Reviewed-on: https://go-review.googlesource.com/15930
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Todd Neal <todd@tneal.org>
Changed racewalk/race detector to use FP in a more
sensible way.
Relaxed checks for CONVNOP when race detecting.
Modified tighten to ensure that GetClosurePtr cannot float
out of entry block (turns out this cannot be relaxed, DX is
sometimes stomped by other code accompanying race detection).
Added case for addr(CONVNOP)
Modified addr to take "bounded" flag to suppress nilchecks
where it is set (usually, by race detector).
Cannot leave unimplemented-complainer enabled because it
turns out we are optimistically running SSA on every platform.
Change-Id: Ife021654ee4065b3ffac62326d09b4b317b9f2e0
Reviewed-on: https://go-review.googlesource.com/15710
Reviewed-by: Keith Randall <khr@golang.org>
This can lead to multiple stores being live at once.
Do OINDEX and ODOT using addresses & loads instead of specific ops.
This keeps SSA values from containing unSSAable types.
Change-Id: I79567e9d43cdee09084eb89ea0bd7aa3aad48ada
Reviewed-on: https://go-review.googlesource.com/15654
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Turns out that these do occur after all, so did the obvious
refactoring into the addr method.
Also added better debugging for the case of unhandled
closure args.
Change-Id: I1cd8ac58f78848bae0b995736f1c744fd20a6c95
Reviewed-on: https://go-review.googlesource.com/15640
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
Changed tree generation to correctly use PARAMOUT instead
of PARAM.
Emit Func.Exit before any returns.
Change-Id: I2fa53cc7fad05fb4eea21081ba33d1f66db4ed49
Reviewed-on: https://go-review.googlesource.com/15610
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
One was OAPPEND of large types. We need to mem-mem copy them
instead of storing them.
Another was pointer-like struct and array types being put in the
data field of an eface. We need to use the underlying pointer
type for the load that fills in the eface.data field.
Change-Id: Id8278c0381904e52d59011a66ce46386b41b5521
Reviewed-on: https://go-review.googlesource.com/15552
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
For appending large types, we want to evaluate the
values being appended after the growslice call, not before.
Evaluating them before leads to phi operations on large types
which confuses the lowering pass.
The order pass has already removed any side-effects from the
values being appended, so it doesn't matter if we do this
last eval before or after the growslice call.
This change fixes a bunch (but not all) of our failed lowerings.
Change-Id: I7c697d4d5275d71b7ef4677b830fd86c52ba03a4
Reviewed-on: https://go-review.googlesource.com/15430
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Cleaned up first-block-in-function code.
Added cases for |PHEAP for PPARAM and PAUTO.
Made PPARAMOUT act more like PAUTO for purposes
of address generation and vardef placement.
Added cases for OCLOSUREVAR and Ops for getting closure
pointer. Closure ops are scheduled at top of entry block
to capture DX.
Wrote test that seems to show proper behavior for addressed
parameters, locals, and returns.
Change-Id: Iee93ebf9e3d9f74cfb4d1c1da8038eb278d8a857
Reviewed-on: https://go-review.googlesource.com/14650
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
There's no need for special ops for panicindex and panicslice.
Just use regular runtime calls.
Change-Id: I71b9b73f4f1ebce1220fdc1e7b7f65cfcf4b7bae
Reviewed-on: https://go-review.googlesource.com/14726
Reviewed-by: David Chase <drchase@google.com>
For variables which get SSA'd, SSA keeps track of all the def/kill.
It is only for on-stack variables that we need them.
This reduces stack frame sizes significantly because often the
only use of a variable was a varkill, and without that last use
the variable doesn't get allocated in the frame at all.
Fixes#12602
Change-Id: I3f00a768aa5ddd8d7772f375b25f846086a3e689
Reviewed-on: https://go-review.googlesource.com/14758
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We need to move any objects whose types are not SSA-able.
Fixes the "not lowered: Load ARRAY PTR64 mem" errors.
Change-Id: I7a0b609f917d7fb34bc9215fee4da15f9961cf6c
Reviewed-on: https://go-review.googlesource.com/14753
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The frontend rewrites most literals, so we see only zero
ones during SSA construction. We can implement those
using the existing zeroing behavior.
Change-Id: I390ad1be0a4b6729baf0c8936c7610aae2aef049
Reviewed-on: https://go-review.googlesource.com/14754
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Generate AUNDEF for every exit block, not just for certain
control values.
Change-Id: Ife500ac5159ee790bc1e70c0e9b0b1f854bc4c47
Reviewed-on: https://go-review.googlesource.com/14721
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
OCALLINTER, as well as ODEFER/OPROC with OCALLMETH/OCALLINTER.
Move all the call logic to its own routine, a lot of the
code is shared.
Change-Id: Ieac59596165e434cc6d1d7b5e46b78957e9c5ed3
Reviewed-on: https://go-review.googlesource.com/14464
Reviewed-by: Todd Neal <todd@tneal.org>
Reviewed-by: David Chase <drchase@google.com>
A simpler way to do iface/slice comparisons. Fixes some
cases of failed lowerings.
Change-Id: Ia252bc8648293a2d460f63c41f1591785543a1e9
Reviewed-on: https://go-review.googlesource.com/14493
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
For now, we only use typedmemmove. This can be optimized
in future CLs.
Also add a feature to help with binary searching bad compilations.
Together with GOSSAPKG, GOSSAHASH specifies the last few binary digits
of the hash of function names that should be compiled. So
GOSSAHASH=0110 means compile only those functions whose last 4 bits
of hash are 0110. By adding digits to the front we can binary search
for the function whose SSA-generated code is causing a test to fail.
Change-Id: I5a8b6b70c6f034f59e5753965234cd42ea36d524
Reviewed-on: https://go-review.googlesource.com/14530
Reviewed-by: Keith Randall <khr@golang.org>
Move the AST to SSA conversion to the caller.
This enables it to be used in contexts in which
the RHS is already an *ssa.Value.
Change-Id: Ibb87210fb9fda095a9b7c7f4ad1264a7cbd269bf
Reviewed-on: https://go-review.googlesource.com/14521
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
Move to implicit (mostly) instead of explicit exit blocks.
RET and RETJMP have no outgoing edges - they implicitly exit.
CALL only has one outgoing edge, as its exception edge is
implicit as well.
Exit blocks are only used for unconditionally panicking code,
like the failed branches of nil and bounds checks.
There may now be more than one exit block. No merges happen
at exit blocks.
The only downside is it is harder to find all the places code
can exit the method. See the reverse dominator code for an
example.
Change-Id: I42e2fd809a4bf81301ab993e29ad9f203ce48eb0
Reviewed-on: https://go-review.googlesource.com/14462
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
It was using 64-bit float comparison ops for complex64.
It should use 32-bit float comparison.
Fixes build.
Change-Id: I6452b227257fecc09e04cd092ccf328d1fc9917f
Reviewed-on: https://go-review.googlesource.com/14497
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
They were using the result type to look up the op, not the arg type.
Change-Id: I0641cba363fa6e7a66ad0860aa340106c10c2cea
Reviewed-on: https://go-review.googlesource.com/14469
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This matches existing behavior, see issue #2196
Change-Id: Ifa9359b7c821115389f337a57de355c5ec23be8f
Reviewed-on: https://go-review.googlesource.com/14261
Reviewed-by: Keith Randall <khr@golang.org>
Evaluating args can overwrite arg area, so we can't write argsize and func
until args are evaluated.
Fixes test/recover.go, test/recover1.go, and test/fixedbugs/issue4066.go
Change-Id: I862e4934ccdb8661431bcc3e1e93817ea834ea3f
Reviewed-on: https://go-review.googlesource.com/14405
Reviewed-by: David Chase <drchase@google.com>
Make sure that return blocks take a store as their control. Without
this, code was getting inserted between the return and exit blocks.
Use AEND to mark the end of code. The live variable analysis gets
confused when routines end like:
JMP earlier
RET
because the RET is unreachable. The RET was incorrectly added to the
last basic block, rendering the JMP invisible to the CFG builder.
Change-Id: I91b32c8b37075347243ff039b4e4385856fba7cd
Reviewed-on: https://go-review.googlesource.com/14398
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
The after-defer test jumps to a deferreturn site. Some functions
(those with infinite loops) have no deferreturn site. Add one
so we have one to jump to.
Change-Id: I505e7f3f888f5e7d03ca49a3477b41cf1f78eb8a
Reviewed-on: https://go-review.googlesource.com/14349
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
They are already handled by the frontend, we just need to
skip them when we see them in ssa.
Change-Id: I309d91552f96a761f8d429a2cab3a47d200ca9e5
Reviewed-on: https://go-review.googlesource.com/14341
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TODO: for now, just function calls. Do method and interface calls.
Change-Id: Ib262dfa31cb753996cde899beaad4dc2e66705ac
Reviewed-on: https://go-review.googlesource.com/14035
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This change is all about leveraging the gc bitmap generation
that is already done by the current compiler. We rearrange how
stack allocation is done so that we generate a variable declaration
for each spill. We also reorganize how args/locals are recorded
during SSA. Then we can use the existing allocauto/defframe to
allocate the stack frame and liveness to make the gc bitmaps.
With this change, stack copying works correctly and we no longer
need hacks in runtime/stack*.go to make tests work. GC is close
to working, it just needs write barriers.
Change-Id: I990fb4e3fbe98850c6be35c3185a1c85d9e1a6ba
Reviewed-on: https://go-review.googlesource.com/13894
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Change to table-driven instead of branchy code; leads to
net reduction in lines, easier to understand what happens,
easier to modify code if we want option to exclude generation
of branchy cases.
Doesn't appear to scale for 8x8 case of integer types.
Change-Id: Ib40104b149d30bb329c5782f6cac45c75743e768
Reviewed-on: https://go-review.googlesource.com/14163
Reviewed-by: Keith Randall <khr@golang.org>
Store floats in AuxInt to reduce allocations.
Change-Id: I101e6322530b4a0b2ea3591593ad022c992e8df8
Reviewed-on: https://go-review.googlesource.com/14320
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Store bools in AuxInt to reduce allocations.
Change-Id: Ibd26db67fca5e1e2803f53d7ef094897968b704b
Reviewed-on: https://go-review.googlesource.com/14276
Reviewed-by: Keith Randall <khr@golang.org>
Fallthrough return needs to be a return block before jumping
to the exit block.
Change-Id: I994de2064da5c326c9cade2c33cbb15bdbce5acb
Reviewed-on: https://go-review.googlesource.com/14256
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
It is confusing to have exceptional edges jump back into
real code. Distinguish return blocks, which execute acutal
code, and the exit block, which is a merge point for the regular
and exceptional return flow.
Prevent critical edge insertion from adding blocks on edges
into the exit block. These added blocks serve no purpose and
add a bunch of dead jumps to the assembly output. Furthermore,
live variable analysis is confused by these jumps.
Change-Id: Ifd69e6c00e90338ed147e7cb351b5100dc0364df
Reviewed-on: https://go-review.googlesource.com/14254
Reviewed-by: David Chase <drchase@google.com>
complex128 was being treated as a complex64
Fixes math/cmplx.
Change-Id: I2996915b4cb6b94198d41cf08a30bd8531b9fec5
Reviewed-on: https://go-review.googlesource.com/14206
Reviewed-by: Keith Randall <khr@golang.org>
Modified to use the truncating conversion.
Fixes reflect.
Change-Id: I47bf3200abc2d2c662939a2a2351e2ff84168f4a
Reviewed-on: https://go-review.googlesource.com/14167
Reviewed-by: David Chase <drchase@google.com>
Still to do:
details, more testing corner cases. (e.g. negative zero)
Includes small cleanups for previous CL.
Note: complex division is currently done in the runtime,
so the division code here is apparently not yet necessary
and also not tested. Seems likely better to open code
division and expose the widening/narrowing to optimization.
Complex64 multiplication and division is done in wide
format to avoid cancellation errors; for division, this
also happens to be compatible with pre-SSA practice
(which uses a single complex128 division function).
It would-be-nice to widen for complex128 multiplication
intermediates as well, but that is trickier to implement
without a handy wider-precision format.
Change-Id: I595a4300f68868fb7641852a54674c6b2b78855e
Reviewed-on: https://go-review.googlesource.com/14028
Reviewed-by: Keith Randall <khr@golang.org>
This also fixes the encoding/gob TestTopLevelNilPointer failure.
Change-Id: I9b29a6fddffd51af305c685f3a8e2a0594bfeeab
Reviewed-on: https://go-review.googlesource.com/14032
Reviewed-by: Keith Randall <khr@golang.org>
addEdge had two identical implementations so make it an exported method
on Block.
Change-Id: I8c21655a9dc5074fefd7f63b2f5b51897571e608
Reviewed-on: https://go-review.googlesource.com/14040
Reviewed-by: Keith Randall <khr@golang.org>
Frontend has already rewriten fallthrough statements, we just need to
ignore them.
Change-Id: Iadf89b06a9f8f9e6e2e1e87c934f31add77a19a1
Reviewed-on: https://go-review.googlesource.com/14029
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Fixes build. There may be no current block.
Change-Id: I0da8bab133dc207124556927698e7cd682e64ef5
Reviewed-on: https://go-review.googlesource.com/13989
Reviewed-by: Keith Randall <khr@golang.org>
Austin Clements
Martin Möhrmann
Keith Randall
Cherry Zhang
Michael Munday
shaharko
Josh Bleecher Snyder
David Chase
Anthony Canino
Hajime Hoshi
Matthew Dempsky
Dave Cheney
Robert Griesemer
Michael Pratt
Brad Fitzpatrick
Lynn Boger
Russ Cox
Michael Hudson-Doyle
Alexandru Moșoi
Eric Engestrom
Ian Lance Taylor
Todd Neal
Ilya Tocar
Gerrit Code Review
Alexandru Moșoi