The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.
For #41190.
Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This CL changes some unit test functions, making sure that these tests (and goroutines spawned during test) won't block.
Since they are just test functions, I use one CL to fix them all. I hope this won't cause trouble to reviewers and can save time for us.
There are three main categories of incorrect logic fixed by this CL:
1. Use testing.Fatal()/Fatalf() in spawned goroutines, which is forbidden by Go's document.
2. Channels are used in such a way that, when errors or timeout happen, the test will be blocked and never return.
3. Channels are used in such a way that, when errors or timeout happen, the test can return but some spawned goroutines will be leaked, occupying resource until all other tests return and the process is killed.
Change-Id: I3df931ec380794a0cf1404e632c1dd57c65d63e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/219380
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
An application that wants to reject non-canonical encodings is likely to
care about other sources of malleability.
Change-Id: I1d3a5b281d2631ca78df3f89b957a02687a534d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/188858
Reviewed-by: Katie Hockman <katie@golang.org>
Most of the decoding time is spent in the first Decode loop, since the
rest of the function only deals with the few remaining bytes. Any
unnecessary work done in that loop body matters tremendously.
One such unnecessary bottleneck was the use of the enc.decodeMap table.
Since enc is a pointer receiver, and the field is used within the
non-inlineable function decode64, the decoder must perform a nil check
at every iteration.
To fix that, move the enc.decodeMap uses to the parent function, where
we can lift the nil check outside the loop. That gives roughly a 15%
speed-up. The function no longer performs decoding per se, so rename it.
While at it, remove the now unnecessary receivers.
An unfortunate side effect of this change is that the loop now contains
eight bounds checks on src instead of just one. However, not having to
slice src plus the nil check removal well outweigh the added cost.
The other piece that made decode64 slow was that it wasn't inlined, and
had multiple branches. Use a simple bitwise-or trick suggested by Roger
Peppe, and collapse the rest of the bitwise logic into a single
expression. Inlinability and the reduced branching give a further 10%
speed-up.
Finally, add these two functions to TestIntendedInlining, since we want
them to stay inlinable.
Apply the same refactor to decode32 for consistency, and to let 32-bit
architectures see a similar performance gain for large inputs.
name old time/op new time/op delta
DecodeString/2-8 47.3ns ± 1% 45.8ns ± 0% -3.28% (p=0.002 n=6+6)
DecodeString/4-8 55.8ns ± 2% 51.5ns ± 0% -7.71% (p=0.004 n=5+6)
DecodeString/8-8 64.9ns ± 0% 61.7ns ± 0% -4.99% (p=0.004 n=5+6)
DecodeString/64-8 238ns ± 0% 198ns ± 0% -16.54% (p=0.002 n=6+6)
DecodeString/8192-8 19.5µs ± 0% 14.6µs ± 0% -24.96% (p=0.004 n=6+5)
name old speed new speed delta
DecodeString/2-8 84.6MB/s ± 1% 87.4MB/s ± 0% +3.38% (p=0.002 n=6+6)
DecodeString/4-8 143MB/s ± 2% 155MB/s ± 0% +8.41% (p=0.004 n=5+6)
DecodeString/8-8 185MB/s ± 0% 195MB/s ± 0% +5.29% (p=0.004 n=5+6)
DecodeString/64-8 369MB/s ± 0% 442MB/s ± 0% +19.78% (p=0.002 n=6+6)
DecodeString/8192-8 560MB/s ± 0% 746MB/s ± 0% +33.27% (p=0.004 n=6+5)
Updates #19636.
Change-Id: Ib839577b0e3f5a2bb201f5cae580c61365d92894
Reviewed-on: https://go-review.googlesource.com/c/go/+/151177
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: roger peppe <rogpeppe@gmail.com>
Base64-encoding 32 bytes results in a 44-byte string.
While in general a 44-byte string might decode to 33 bytes,
if you take a 44-byte string that actually only encodes 32 bytes,
and you try to decode it into 32 bytes, that should succeed.
Instead it fails trying to do a useless dst[33:] slice operation.
Delete that slice operation.
Noticed while preparing CL 156322.
Change-Id: I8024bf28a65e2638675b980732b2ff91c66c62cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/164628
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Most of the encoding time is spent in the first Encode loop, since the
rest of the function only deals with the few remaining bytes. Any
unnecessary work done in that loop body matters tremendously.
One such unnecessary bottleneck was the use of the enc.encode table.
Since enc is a pointer receiver, and the field is first used within the
loop, the encoder must perform a nil check at every iteration.
Add a dummy use of the field before the start of the loop, to move the
nil check there. After that line, the compiler now knows that enc can't
be nil, and thus the hot loop is free of nil checks.
name old time/op new time/op delta
EncodeToString-4 14.7µs ± 0% 13.7µs ± 1% -6.53% (p=0.000 n=10+10)
name old speed new speed delta
EncodeToString-4 559MB/s ± 0% 598MB/s ± 1% +6.99% (p=0.000 n=10+10)
Updates #20206.
Change-Id: Icbb523a7bd9e470a8be0a448d1d78ade97ed4ff6
Reviewed-on: https://go-review.googlesource.com/c/151158
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)
Updates #27864
Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
First, use a dummy slice access on decode64 and decode32 to ensure that
there is a single bounds check for src.
Second, move the PutUint64/PutUint32 calls out of these functions,
meaning that they are simpler and smaller. This may also open the door
to inlineability in the future, but for now, they both go past the
budget.
While at it, get rid of the ilen and olen variables, which have no
impact whatsoever on performance. At least, not measurable by any of the
benchmarks.
name old time/op new time/op delta
DecodeString/2-4 54.3ns ± 1% 55.2ns ± 2% +1.60% (p=0.017 n=5+6)
DecodeString/4-4 66.6ns ± 1% 66.8ns ± 2% ~ (p=0.903 n=6+6)
DecodeString/8-4 79.3ns ± 2% 79.6ns ± 1% ~ (p=0.448 n=6+6)
DecodeString/64-4 300ns ± 1% 281ns ± 3% -6.54% (p=0.002 n=6+6)
DecodeString/8192-4 27.4µs ± 1% 23.7µs ± 2% -13.47% (p=0.002 n=6+6)
name old speed new speed delta
DecodeString/2-4 73.7MB/s ± 1% 72.5MB/s ± 2% -1.55% (p=0.026 n=5+6)
DecodeString/4-4 120MB/s ± 1% 120MB/s ± 2% ~ (p=0.851 n=6+6)
DecodeString/8-4 151MB/s ± 2% 151MB/s ± 1% ~ (p=0.485 n=6+6)
DecodeString/64-4 292MB/s ± 1% 313MB/s ± 3% +7.03% (p=0.002 n=6+6)
DecodeString/8192-4 399MB/s ± 1% 461MB/s ± 2% +15.58% (p=0.002 n=6+6)
For #19636.
Change-Id: I0dfbdafa2a41dc4c582f63aef94b90b8e473731c
Reviewed-on: https://go-review.googlesource.com/113776
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The ascii85, base32 and base64 packages all contain a test called
TestDecoderBuffering. Each of these tests contain a loop that ignores
the error returned from the Read method of their decoders. The result
being that the tests loop for ever if the decoders actually return an
error. This commit fixes the issue by terminating the loops if an error
occurs and failing the tests with a suitable error message.
Change-Id: Idb385673cf9f3f6f8befe4288b4be366ab0985fd
Reviewed-on: https://go-review.googlesource.com/46010
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some of the _test.go files in the encoding packages contain a private
function called testEqual that calls testing.Errorf if the arguments
passed to it are unequal. The line numbers output by such calls to
Errorf identify the failure as being in testEqual itself which is not
very useful. This commit fixes the problem by adding a call to the
new t.Helper method in each of the testEqual functions. The line
numbers output when errors do occur now identify the real source of
the error.
Change-Id: I582d1934f40ef2b788116c3811074c67ea882021
Reviewed-on: https://go-review.googlesource.com/45871
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Replace 'does not contains' with 'does not contain' where it appears
in the source code.
Change-Id: Ie7266347c429512c8a41a7e19142afca7ead3922
Reviewed-on: https://go-review.googlesource.com/37887
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Document and check that the alphabet cannot contain '\n' or '\r'.
Document that the alphabet cannot contain the padding character.
Document that the padding character must be equal or bellow '\xff'.
Document that the padding character must not be '\n' or '\r'.
Fixes#19343Fixes#19318
Change-Id: I6de0034d347ffdf317d7ea55d6fe38b01c2c4c03
Reviewed-on: https://go-review.googlesource.com/37838
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix Decode to return the correct illegal data index from a corrupted
input that contains whitespaces.
Fixes#19406
Change-Id: Ib2b2b6ed7e41f024d0da2bd035caec4317c2869c
Reviewed-on: https://go-review.googlesource.com/37837
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If strict option is enabled, when decoding, instead of skip the padding
bits, it will do strict check to enforce they are set to zero.
Fixes#15656
Change-Id: I869fb725a39cc9dde44dbc4ff0046446e7abc642
Reviewed-on: https://go-review.googlesource.com/24964
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
While we're at it, add tests for EncodedLen and DecodedLen.
Fixes#14803.
Change-Id: I200c72cf11c51669b8d9f70c6e57ece359f7ae61
Reviewed-on: https://go-review.googlesource.com/20649
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a subset of https://golang.org/cl/20022 with only the copyright
header lines, so the next CL will be smaller and more reviewable.
Go policy has been single space after periods in comments for some time.
The copyright header template at:
https://golang.org/doc/contribute.html#copyright
also uses a single space.
Make them all consistent.
Change-Id: Icc26c6b8495c3820da6b171ca96a74701b4a01b0
Reviewed-on: https://go-review.googlesource.com/20111
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some applications use unpadded base64 format, omitting the trailing
'=' padding characters from the standard base64 format, either to
minimize size or (more justifiably) to avoid use of the '=' character.
Unpadded flavors are standard and documented in section 3.2 of RFC 4648.
To support these unpadded flavors, this change adds two predefined
encoding variables, RawStdEncoding and RawURLEncoding, for unpadded
encodings using the standard and URL character set, respectively.
The change also adds a function WithPadding() to customize the padding
character or disable padding in a custom Encoding.
Finally, I noticed that the existing base64 test-suite was only
exercising the StdEncoding, and not referencing URLEncoding at all.
This change adds test-suite functionality to exercise all four encodings
(the two existing ones and the two new unpadded flavors),
although it still doesn't run *every* test on all four encodings.
Naming: I used the "Raw" prefix because it's more concise than "Unpadded"
and seemed just as expressive, but I have no strong preferences here.
Another short alternative prefix would be "Min" ("minimal" encoding).
Change-Id: Ic0423e02589b39a6b2bb7d0763bd073fd244f469
Reviewed-on: https://go-review.googlesource.com/1511
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Russ Cox
Kirill Korotaev
Ziheng Liu
Filippo Valsorda
Daniel Martí
Leon Klingele
Mihai Todor
Brad Fitzpatrick
Josselin Costanzi
Mark Ryan
Kevin Burke
Xuyang Kang
Caleb Spare
Martin Möhrmann
Didier Spezia
Egon Elbre
Bryan Ford