Commit Graph

11 Commits

Author SHA1 Message Date
Joe Tsai 8eca08611a unicode/utf8: optimize ValidRune
Re-writing the switch statement as a single boolean expression
reduces the number of branches that the compiler generates.
It is also arguably easier to read as a pair of numeric ranges
that valid runes can exist in.

No test changes since the existing test does a good job of
testing all of the boundaries.

This change was to gain back some performance after a correctness
fix done in http://golang.org/cl/32123.

The correctness fix (CL/32123) slowed down the benchmarks slightly:
	benchmark                   old ns/op     new ns/op     delta
	BenchmarkIndexRune/10-4     19.3          21.6          +11.92%
	BenchmarkIndexRune/32-4     33.6          35.2          +4.76%

Since the fix relies on utf8.ValidRune, this CL improves benchmarks:
	benchmark                   old ns/op     new ns/op     delta
	BenchmarkIndexRune/10-4     21.6          20.0          -7.41%
	BenchmarkIndexRune/32-4     35.2          33.5          -4.83%

Change-Id: Ib1ca10a2e29c90e879a8ef9b7221c33e85d015d8
Reviewed-on: https://go-review.googlesource.com/32122
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-26 23:02:52 +00:00
Martin Möhrmann d295174030 runtime: speed up non-ASCII rune decoding
Copies utf8 constants and EncodeRune implementation from unicode/utf8.

Adds a new decoderune implementation that is used by the compiler
in code generated for ranging over strings. It does not handle
ASCII runes since these are handled directly before calls to decoderune.

The DecodeRuneInString implementation from unicode/utf8 is not used
since it uses a lookup table that would increase the use of cpu caches.

Adds more tests that check decoding of valid and invalid utf8 sequences.

name                              old time/op  new time/op  delta
RuneIterate/range2/ASCII-4        7.45ns ± 2%  7.45ns ± 1%     ~     (p=0.634 n=16+16)
RuneIterate/range2/Japanese-4     53.5ns ± 1%  49.2ns ± 2%   -8.03%  (p=0.000 n=20+20)
RuneIterate/range2/MixedLength-4  46.3ns ± 1%  41.0ns ± 2%  -11.57%  (p=0.000 n=20+20)

new:
"".decoderune t=1 size=423 args=0x28 locals=0x0
old:
"".charntorune t=1 size=666 args=0x28 locals=0x0

Change-Id: I1df1fdb385bb9ea5e5e71b8818ea2bf5ce62de52
Reviewed-on: https://go-review.googlesource.com/28490
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-10-17 11:25:22 +00:00
Martin Möhrmann fd41951c2b unicode/utf8: reduce bounds checks in EncodeRune
Provide bounds elim hints in EncodeRune.

name                  old time/op  new time/op  delta
EncodeASCIIRune-4     2.69ns ± 2%  2.69ns ± 2%    ~     (p=0.193 n=47+46)
EncodeJapaneseRune-4  5.97ns ± 2%  5.38ns ± 2%  -9.93%  (p=0.000 n=49+50)

Change-Id: I1a6dcffff3bdd64ab93c2130021e3b00981de4c8
Reviewed-on: https://go-review.googlesource.com/28492
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-09-03 20:05:36 +00:00
Brad Fitzpatrick 5fea2ccc77 all: single space after period.
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.

This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:

$ perl -i -npe 's,^(\s*// .+[a-z]\.)  +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.)  +([A-Z])')
$ go test go/doc -update

Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-03-02 00:13:47 +00:00
Marcel van Lohuizen 1dae47378c unicode/utf8: add test for FullRune
Check that it now properly handles \xC0 and \xC1.

Fixes #11733.

Change-Id: I66cfe0d43f9d123d4c4509a3fa18b9b6380dfc39
Reviewed-on: https://go-review.googlesource.com/17225
Reviewed-by: Russ Cox <rsc@golang.org>
2015-12-01 10:39:06 +00:00
Aaron Jacobs b39329bff5 unicode/utf8: don't imply that the empty string is incorrect UTF-8
Change-Id: Idd9523949ee4f2f304b12be39f8940ba34a420be
Reviewed-on: https://go-review.googlesource.com/16361
Reviewed-by: Russ Cox <rsc@golang.org>
2015-11-24 17:12:35 +00:00
Marcel van Lohuizen bf5b4e71be unicode/utf8: table-based algorithm for decoding
This simplifies covering all cases, reducing the number of branches
and making unrolling for simpler functions manageable.
This significantly improves performance of non-ASCII input.

This change will also allow addressing Issue #11733 in an efficient
manner.

RuneCountTenASCIIChars-8             13.7ns ± 4%  13.5ns ± 2%     ~     (p=0.116 n=7+8)
RuneCountTenJapaneseChars-8           153ns ± 3%    74ns ± 2%  -51.42%  (p=0.000 n=8+8)
RuneCountInStringTenASCIIChars-8     13.5ns ± 2%  12.5ns ± 3%   -7.13%  (p=0.000 n=8+7)
RuneCountInStringTenJapaneseChars-8   145ns ± 2%    68ns ± 2%  -53.21%  (p=0.000 n=8+8)
ValidTenASCIIChars-8                 14.1ns ± 3%  12.5ns ± 5%  -11.38%  (p=0.000 n=8+8)
ValidTenJapaneseChars-8               147ns ± 3%    71ns ± 4%  -51.72%  (p=0.000 n=8+8)
ValidStringTenASCIIChars-8           12.5ns ± 3%  12.3ns ± 3%     ~     (p=0.095 n=8+8)
ValidStringTenJapaneseChars-8         146ns ± 4%    70ns ± 2%  -51.62%  (p=0.000 n=8+7)
DecodeASCIIRune-8                    5.91ns ± 2%  4.83ns ± 3%  -18.28%  (p=0.001 n=7+7)
DecodeJapaneseRune-8                 12.2ns ± 7%   8.5ns ± 3%  -29.79%  (p=0.000 n=8+7)
FullASCIIRune-8                      5.95ns ± 3%  4.27ns ± 1%  -28.23%  (p=0.000 n=8+7)
FullJapaneseRune-8                   12.0ns ± 6%   4.3ns ± 3%  -64.39%  (p=0.000 n=8+8)

Change-Id: Iea1d6b0180cbbee1739659a0a38038126beecaca
Reviewed-on: https://go-review.googlesource.com/16940
Reviewed-by: Russ Cox <rsc@golang.org>
2015-11-16 21:16:51 +00:00
Marcel van Lohuizen 9b299c1efd unicode/utf8: removed uses of ranging over string
Ranging over string is much slower than using DecodeRuneInString.
See golang.org/issue/13162.

Replacing ranging over a string with the implementation of the Bytes
counterpart results in the following performance improvements:

RuneCountInStringTenASCIIChars-8     43.0ns ± 1%  16.4ns ± 2%  -61.80%  (p=0.000 n=7+8)
RuneCountInStringTenJapaneseChars-8   161ns ± 2%   154ns ± 2%   -4.58%  (p=0.000 n=8+8)
ValidStringTenASCIIChars-8           52.2ns ± 1%  13.2ns ± 1%  -74.62%  (p=0.001 n=7+7)
ValidStringTenJapaneseChars-8         173ns ± 2%   153ns ± 2%  -11.78%  (p=0.000 n=7+8)

Update golang/go#13162

Change-Id: Ifc40a6a94bb3317f1f2d929d310bd2694645e9f6
Reviewed-on: https://go-review.googlesource.com/16695
Reviewed-by: Russ Cox <rsc@golang.org>
2015-11-16 11:29:13 +00:00
Marcel van Lohuizen 3d198bd7be unicode/utf8: added benchmarks
Cover some functions that weren't benched before and add InString
variants if the underlying implementation is different.

Note: compare (Valid|RuneCount)InString* to their (Valid|RuneCount)*
counterparts. It shows, somewhat unexpectedly, that ranging over
a string is *much* slower than using calls to DecodeRune.

Results:
In order to avoid a discrepancy in measuring the performance
of core we could leave the names of the string-based measurements
unchanged and suffix the added alternatives with Bytes.

Compared to old:
BenchmarkRuneCountTenASCIIChars-8        44.3          12.4          -72.01%
BenchmarkRuneCountTenJapaneseChars-8     167           67.1          -59.82%
BenchmarkEncodeASCIIRune-8               3.37          3.44          +2.08%
BenchmarkEncodeJapaneseRune-8            7.19          7.24          +0.70%
BenchmarkDecodeASCIIRune-8               5.41          5.53          +2.22%
BenchmarkDecodeJapaneseRune-8            8.17          8.41          +2.94%

All benchmarks:
BenchmarkRuneCountTenASCIIChars-8           	100000000	        12.4 ns/op
BenchmarkRuneCountTenJapaneseChars-8        	20000000	        67.1 ns/op
BenchmarkRuneCountInStringTenASCIIChars-8   	30000000	        44.5 ns/op
BenchmarkRuneCountInStringTenJapaneseChars-8	10000000	       165 ns/op
BenchmarkValidTenASCIIChars-8               	100000000	        12.5 ns/op
BenchmarkValidTenJapaneseChars-8            	20000000	        71.1 ns/op
BenchmarkValidStringTenASCIIChars-8         	30000000	        50.0 ns/op
BenchmarkValidStringTenJapaneseChars-8      	10000000	       161 ns/op
BenchmarkEncodeASCIIRune-8                  	500000000	         3.44 ns/op
BenchmarkEncodeJapaneseRune-8               	200000000	         7.24 ns/op
BenchmarkDecodeASCIIRune-8                  	300000000	         5.53 ns/op
BenchmarkDecodeJapaneseRune-8               	200000000	         8.41 ns/op
BenchmarkFullASCIIRune-8                    	500000000	         3.91 ns/op
BenchmarkFullJapaneseRune-8                 	300000000	         4.22 ns/op

Change-Id: I674d2ee4917b975a37717bbfa1082cc84dcd275e
Reviewed-on: https://go-review.googlesource.com/14431
Reviewed-by: Russ Cox <rsc@golang.org>
2015-10-26 10:42:38 +00:00
Nigel Tao 2dcb613878 unicode/utf8: fix docs for DecodeRune(empty) and friends.
LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/157080043
2014-10-16 09:13:50 +11:00
Russ Cox c007ce824d build: move package sources from src/pkg to src
Preparation was in CL 134570043.
This CL contains only the effect of 'hg mv src/pkg/* src'.
For more about the move, see golang.org/s/go14nopkg.
2014-09-08 00:08:51 -04:00