A negative rune (other than NoPadding) makes no semantic sense.
Doing so relies on integer overflow of converting a rune to a byte
and would thus be equivalent to passing the positive byte value
of byte(padding).
This may cause existing code to panic.
An alternative is treat negative runes as equivalent to NoPadding.
However, the code already panics to report erroneous padding values,
so this is in line with the existing API.
Change-Id: I02499705519581598adc0c8525d90e25278dc056
Reviewed-on: https://go-review.googlesource.com/c/go/+/505236
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Implement append-like equivalent of Encode and Decode functions.
Fixes#53693
Change-Id: I79d8d834e3c8f77fad32be2fd391e33d4d1527ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/504884
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
hex is in fact immutable, declare it as a const to avoid accidental
modification, also for consistency with other packages.
Change-Id: I99f292e98c82d4c4526e46c9897d154d0c073da5
GitHub-Last-Rev: d2f06965e7
GitHub-Pull-Request: golang/go#62011
Reviewed-on: https://go-review.googlesource.com/c/go/+/519155
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Over the past few months as I read the standard library's documentation
I kept finding spots where godoc links would have helped me.
I kept adding to a stash of changes to fix them up bit by bit.
The stash has grown big enough by now, and we're nearing a release,
so I think it's time to merge to avoid git conflicts or bit rot.
Note that a few sentences are slightly reworded,
since "implements the Fooer interface" can just be "implements [Fooer]"
now that the link provides all the context needed to the user.
Change-Id: I01c31d3d3ff066d06aeb44f545f8dd0fb9a8d998
Reviewed-on: https://go-review.googlesource.com/c/go/+/508395
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Converts the 5 x 8-bit source byte to two 32-bit integers.
This will reduce the number of shift operations.
benchmark old ns/op new ns/op delta
BenchmarkEncode-10 9005 4426 -50.85%
BenchmarkEncodeToString-10 10739 6155 -42.69%
benchmark old MB/s new MB/s speedup
BenchmarkEncode-10 909.69 1850.81 2.03x
BenchmarkEncodeToString-10 762.84 1331.02 1.74x
Change-Id: I9418d3436b73f94a4eb4b2b525e4f83612ff4d47
Reviewed-on: https://go-review.googlesource.com/c/go/+/514095
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Change-Id: I9bafc7aa4e20e7cd994b75e7576156ca68f4fc8b
GitHub-Last-Rev: e037f689bd
GitHub-Pull-Request: golang/go#61746
Reviewed-on: https://go-review.googlesource.com/c/go/+/515855
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
The encoding/gob.TestLargeSlice test needs too much virtual memory
to run reliably on machines with a small address space, for example
the plan9-arm builders where user processes only have 1 gigabyte.
Fixes#60284
Change-Id: Ied88630e5ec6685e14d2060ae316abca1619f9b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/496138
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: David du Colombier <0intro@gmail.com>
For #60088
Change-Id: I2e471c76de62944b14472966b63f5778124b9b8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/514655
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
For #60088
Change-Id: Ib2589b994d304cca1f2e2081639959d80818ac7f
Reviewed-on: https://go-review.googlesource.com/c/go/+/514639
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
This avoids several mildly confusing Elem calls.
For #60088
Change-Id: If7b83d2ab10537c7e886a035b43cb272130c1669
Reviewed-on: https://go-review.googlesource.com/c/go/+/514455
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
benchmark old ns/op new ns/op delta
BenchmarkUnmarshalMap-10 218 172 -21.28%
benchmark old allocs new allocs delta
BenchmarkUnmarshalMap-10 15 12 -20.00%
benchmark old bytes new bytes delta
BenchmarkUnmarshalMap-10 328 256 -21.95%
Change-Id: Ie20ab62731c752eb0040c6d1591fedd7d12b1e0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/514100
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Change-Id: I534698008b46b23352d9f1fed891fd96dc0947b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/507115
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Replace reflect.PtrTo with reflect.PointerTo.
Fixes#59599
Change-Id: I49407193e2050543ef983cd637703acc682d9f51
Change-Id: I49407193e2050543ef983cd637703acc682d9f51
GitHub-Last-Rev: 7bc9ccf1dc
GitHub-Pull-Request: golang/go#61440
Reviewed-on: https://go-review.googlesource.com/c/go/+/511035
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Same as https://go-review.googlesource.com/c/go/+/510635, reduces risk of overflow
Change-Id: I18f5560d73af76c3e853464a89ad7e42dbbd5894
GitHub-Last-Rev: 652c8c6712
GitHub-Pull-Request: golang/go#61547
Reviewed-on: https://go-review.googlesource.com/c/go/+/512200
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Change-Id: I0a55cdc38ae496e2070f0b9ef317a41f82352afd
GitHub-Last-Rev: c19527a26b
GitHub-Pull-Request: golang/go#61407
Reviewed-on: https://go-review.googlesource.com/c/go/+/510635
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
The invention of base32 and base64 predates the invention of UTF-8
and was never meant to output valid UTF-8.
By default, the output is always valid ASCII (and thus valid UTF-8)
except when the user specifies an alphabet or padding value
that is larger than '\x7f'. If that is done,
then the exact byte symbol is used rather than the UTF-8 encoding.
Fixes#60689
Change-Id: I4ec88d974ec0658ad1a578bbd65a809e27c73ea7
Reviewed-on: https://go-review.googlesource.com/c/go/+/505237
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Size is defined as returning -1 if the type is not fixed-size.
Before this CL cases like Size((*[]int)(nil)) would crash.
Fixes#60892
Change-Id: Iee8e20a0aee24b542b78cb4160c3b2c5a3eb02c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/504575
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Fixes#10275
Change-Id: I2b3d54f3eb0f85d65324ddc3c3b2a797d42a16c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/496537
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This change wraps the errors from the CharsetReader function so the caller can distinguish different error conditions.
Context: I have an XML file with an unknown encoding which I like to handle separately. I like to use the CharsetReader for this but the error type has not been forwarded.
Change-Id: I6739a0dee04ec376cd20536be2806ce7f50c5213
GitHub-Last-Rev: ada9dd510f
GitHub-Pull-Request: golang/go#60199
Reviewed-on: https://go-review.googlesource.com/c/go/+/494897
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This updates the doc comment to reflect the behavior of the method.
On error the method returns a partial result.
Fixes#59991
Change-Id: I71e9dfa37e0488c85abd3eeede2a1a34cb74239b
GitHub-Last-Rev: 389488e536
GitHub-Pull-Request: golang/go#60084
Reviewed-on: https://go-review.googlesource.com/c/go/+/494055
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Found by running
$ go run golang.org/x/tools/go/analysis/passes/nilness/cmd/nilness@latest std
No actual bugs--other than one panic(nil)--but a
few places where error nilness was unclear.
Change-Id: Ia916ba30f46f29c1bcf928cc62280169b922463a
Reviewed-on: https://go-review.googlesource.com/c/go/+/486675
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Alan Donovan <adonovan@google.com>
Like https://go.dev/cl/481376 did for encoding/gob,
but now for encoding/xml, which had very similar code.
One minor difference is that encoding/xml now needs a SetLen before the
call to Index, as otherwise we index just past the length.
Still, calling Grow and SetLen is easier to understand,
and avoids needing to make a new zero value.
goos: linux
goarch: amd64
pkg: encoding/xml
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
Unmarshal-8 6.904µ ± 1% 6.980µ ± 1% +1.10% (p=0.009 n=6)
│ old │ new │
│ B/op │ B/op vs base │
Unmarshal-8 7.656Ki ± 0% 7.586Ki ± 0% -0.92% (p=0.002 n=6)
│ old │ new │
│ allocs/op │ allocs/op vs base │
Unmarshal-8 188.0 ± 0% 185.0 ± 0% -1.60% (p=0.002 n=6)
Change-Id: Id83feb467a9c59c80c7936aa892780aae7e8b809
Reviewed-on: https://go-review.googlesource.com/c/go/+/483135
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Test added.
Fixes#22860
Change-Id: I08304834a2b7b10b4ac729bf36761692eb4731da
Reviewed-on: https://go-review.googlesource.com/c/go/+/113075
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Fixes the misuse of "a" vs "an", according to English grammatical
expectations and using https://www.a-or-an.com/
Change-Id: I53ac724070e3ff3d33c304483fe72c023c7cda47
Reviewed-on: https://go-review.googlesource.com/c/go/+/480536
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Growing by one is a simpler, and often cheaper,
operation compared to appending one (newly created) zero value.
The method was introduced in Go 1.20.
growSlice in dec_helpers.go is left alone,
as it grows using the builtin append instead of reflect.Append.
No noticeable performance difference on any of our benchmarks,
as this code only runs for slices large enough to not fit in
saferio.SliceCap, and none of our benchmarks use data that large.
goos: linux
goarch: amd64
pkg: encoding/gob
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
DecodeBytesSlice-8 11.37µ ± 1% 11.46µ ± 4% ~ (p=0.315 n=10)
DecodeInterfaceSlice-8 96.49µ ± 1% 95.75µ ± 1% ~ (p=0.436 n=10)
geomean 33.12µ 33.12µ +0.01%
│ old │ new │
│ B/op │ B/op vs base │
DecodeBytesSlice-8 22.39Ki ± 0% 22.39Ki ± 0% ~ (p=1.000 n=10)
DecodeInterfaceSlice-8 80.25Ki ± 0% 80.25Ki ± 0% ~ (p=0.650 n=10)
geomean 42.39Ki 42.39Ki +0.00%
│ old │ new │
│ allocs/op │ allocs/op vs base │
DecodeBytesSlice-8 169.0 ± 0% 169.0 ± 0% ~ (p=1.000 n=10) ¹
DecodeInterfaceSlice-8 3.178k ± 0% 3.178k ± 0% ~ (p=1.000 n=10) ¹
geomean 732.9 732.9 +0.00%
Change-Id: I468aebf4ae6f197a1fd35f6fee809ca591c1788f
Reviewed-on: https://go-review.googlesource.com/c/go/+/481376
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
I almost exclusively use these benchmarks with -benchtime already.
Change-Id: I6539cbba6abbdb6b275502e122f4e16856d8b9e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/481375
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rob Pike <r@golang.org>
fieldType is a struct with only a string and an integer,
so its size will barely be three times that of a pointer.
The indirection doesn't save us any memory or append/grow cost,
but it does cause a significant amount of allocations at init time.
goos: linux
goarch: amd64
pkg: encoding/gob
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
EndToEndPipe-16 730.9n ± 5% 741.6n ± 5% ~ (p=0.529 n=10)
EncodingGob 173.7µ ± 0% 171.1µ ± 0% -1.46% (p=0.000 n=10)
geomean 11.27µ 11.26µ -0.01%
│ old │ new │
│ B/op │ B/op vs base │
EndToEndPipe-16 1.766Ki ± 0% 1.766Ki ± 0% ~ (p=1.000 n=10) ¹
EncodingGob 38.27Ki ± 0% 34.30Ki ± 0% -10.38% (p=0.000 n=10)
geomean 8.221Ki 7.782Ki -5.33%
¹ all samples are equal
│ old │ new │
│ allocs/op │ allocs/op vs base │
EndToEndPipe-16 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
EncodingGob 642.0 ± 0% 615.0 ± 0% -4.21% (p=0.000 n=10)
geomean 35.83 35.07 -2.13%
¹ all samples are equal
Change-Id: I852a799834d2e9b7b915da74e871a4052d13892e
Reviewed-on: https://go-review.googlesource.com/c/go/+/479400
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
After the previous commit, both calls use the non-pointer type,
so we can deduplicate. No noticeable difference in init cost.
Change-Id: I0f0fb91d42655787cb58b4442ad3da4194560af4
Reviewed-on: https://go-review.googlesource.com/c/go/+/479399
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
wireType itself is just a struct with seven pointer fields,
so an indirection doesn't feel necessary to noticeably reduce the amount
of memory that typeInfo takes for each Go type registered in gob.
The indirection does add a small amount of overhead though,
particularly one extra allocation when registering a type,
which is done a number of times as part of init.
For consistency, also update wireTypeUserInfo to not use a pointer.
Measuring via one of the end-to-end benchmarks and benchinit:
goos: linux
goarch: amd64
pkg: encoding/gob
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
EndToEndPipe-16 736.8n ± 5% 733.9n ± 5% ~ (p=0.971 n=10)
EncodingGob 177.6µ ± 0% 173.6µ ± 0% -2.27% (p=0.000 n=10)
geomean 11.44µ 11.29µ -1.34%
│ old │ new │
│ B/op │ B/op vs base │
EndToEndPipe-16 1.766Ki ± 0% 1.766Ki ± 0% ~ (p=1.000 n=10) ¹
EncodingGob 38.47Ki ± 0% 38.27Ki ± 0% -0.50% (p=0.000 n=10)
geomean 8.241Ki 8.220Ki -0.25%
¹ all samples are equal
│ old │ new │
│ allocs/op │ allocs/op vs base │
EndToEndPipe-16 2.000 ± 0% 2.000 ± 0% ~ (p=1.000 n=10) ¹
EncodingGob 652.0 ± 0% 642.0 ± 0% -1.53% (p=0.000 n=10)
geomean 36.11 35.83 -0.77%
¹ all samples are equal
Change-Id: I528080b7d990ed595683f155a1ae25dcd26394b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/479398
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
The reflect method was added in Go 1.13, in 2019.
gob's own version dates all the way back to 2011.
The behavior appears to be the same, and all tests still pass.
gob does have special cases like always encoding arrays even when they
are the zero value, but that is done via the sendZero boolean field.
Change-Id: I9057b7436963e231fdbf2f6c4b1edb58a2b13305
Reviewed-on: https://go-review.googlesource.com/c/go/+/479397
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Doing this work at init time does make the first encode or decode call
slightly faster, but the cost is still paid upfront.
However, not all programs which directly or indirectly import
encoding/gob end up encoding or decoding any values.
For example, a program might only be run with the -help flag,
or it might only use gob encoding when a specific mode is enabled.
Moreover, any work done at init time needs to happen sequentially and
before the main function can start, blocking the entire program.
Using benchinit, we see a moderate saving at init time:
goos: linux
goarch: amd64
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
EncodingGob 188.9µ ± 0% 175.4µ ± 0% -7.15% (p=0.000 n=10)
│ old │ new │
│ B/op │ B/op vs base │
EncodingGob 39.78Ki ± 0% 38.46Ki ± 0% -3.32% (p=0.000 n=10)
│ old │ new │
│ allocs/op │ allocs/op vs base │
EncodingGob 668.0 ± 0% 652.0 ± 0% -2.40% (p=0.000 n=10)
Change-Id: I75a5df18c9b1d02566e5885a966360d8a525913a
Reviewed-on: https://go-review.googlesource.com/c/go/+/479396
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
A GeneralizedTime value may contain an optional fractional seconds
element (according to X.680 46.2, restricted by X.690 11.7.3). This
change adds support for this fractional part, up to nine digits, so that
Unmarshal won't fail when decoding a DER encoded GeneralizedTime value
with fractional digits. Also, test cases related to this change have
been added.
X.680 and X.690 can be found at:
https://www.itu.int/rec/T-REC-X.680https://www.itu.int/rec/T-REC-X.690Fixes#15842
Change-Id: If217c007e01b686db508a940e9e2ed3bfb901879
Reviewed-on: https://go-review.googlesource.com/c/go/+/108355
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
I noticed the one in path/filepath while reading the docs,
and the other ones were found via some quick grepping.
Change-Id: I386f2f74ef816a6d18aa2f58ee6b64dbd0147c9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/478795
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
I edited dec_helpers.go without realizing that it is a generated file.
Fix the generator to generate the current version (which generates
a small comment change).
Change-Id: I70e3bc78eb0728d23c08972611218f288dc1d29c
Reviewed-on: https://go-review.googlesource.com/c/go/+/479117
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Andrew Ekstedt <andrew.ekstedt@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Fixes#59172
Change-Id: I54d5e724f10117a40ec5dd58c810f6bbb2475933
GitHub-Last-Rev: d1a986698c
GitHub-Pull-Request: golang/go#59173
Reviewed-on: https://go-review.googlesource.com/c/go/+/478215
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Use the explicit API for acquiring an empty available buffer,
rather than the hack that's implemented in terms of Bytes and Len.
Change-Id: If286ed42693acd61ffe28dc849ed4b76c3ae4434
Reviewed-on: https://go-review.googlesource.com/c/go/+/476337
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
The folded name logic (despite all attempts to optimize it)
was fundamentally an O(n) operation where every field in a struct
needed to be linearly scanned in order to find a match.
This made unmashaling of unknown fields always O(n).
Instead of optimizing the comparison for each field,
make it such that we can look up a name in O(1).
We accomplish this by maintaining a map keyed by pre-folded names,
which we can pre-calculate when processing the struct type.
Using a stack-allocated buffer, we can fold the input name and
look up its presence in the map.
Also, instead of mapping from names to indexes,
map directly to a pointer to the field information.
The memory cost of this is the same and avoids an extra slice index.
The new logic is both simpler and faster.
Performance:
name old time/op new time/op delta
CodeDecoder 2.47ms ± 4% 2.42ms ± 2% -1.83% (p=0.022 n=10+9)
UnicodeDecoder 259ns ± 2% 248ns ± 1% -4.32% (p=0.000 n=10+10)
DecoderStream 150ns ± 1% 149ns ± 1% ~ (p=0.516 n=10+10)
CodeUnmarshal 3.13ms ± 2% 3.09ms ± 2% -1.37% (p=0.022 n=10+9)
CodeUnmarshalReuse 2.50ms ± 1% 2.45ms ± 1% -1.96% (p=0.001 n=8+9)
UnmarshalString 67.1ns ± 5% 64.5ns ± 5% -3.90% (p=0.005 n=10+10)
UnmarshalFloat64 60.1ns ± 4% 58.4ns ± 2% -2.89% (p=0.002 n=10+8)
UnmarshalInt64 51.0ns ± 4% 49.2ns ± 1% -3.53% (p=0.001 n=10+8)
Issue10335 80.7ns ± 2% 79.2ns ± 1% -1.82% (p=0.016 n=10+8)
Issue34127 28.6ns ± 3% 28.8ns ± 3% ~ (p=0.388 n=9+10)
Unmapped 177ns ± 2% 177ns ± 2% ~ (p=0.956 n=10+10)
Change-Id: I478b2b958f5a63a69c9a991a39cd5ffb43244a2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/471196
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Use append for HTMLEscape similar to Indent and Compact.
Move it to indent.go alongside Compact, as it shares similar logic.
In a future CL, we will modify appendCompact to be written in terms
of appendHTMLEscape, but we need to first move the JSON decoder logic
out of the main loop of appendCompact.
Change-Id: I131c64cd53d5d2b4ca798b37349aeefe17b418c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/471198
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
v.SetZero() is faster than v.Set(reflect.Zero(v.Type()))
and was recently added in Go 1.20.
Benchmark numbers are largely unchanged since this mainly
affects the unmarshaling of large numbers of JSON nulls,
which our benchmarks do not heavily exercise.
Change-Id: I464f60f63c9027e63a99fd5da92e7ab782018329
Reviewed-on: https://go-review.googlesource.com/c/go/+/471195
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
With native support for fuzzing in the Go toolchain,
rely instead on the fuzz tests declared in fuzz_test.go.
Change-Id: I601842cd0bc7e64ea3bfdafbbbc3534df11acf59
Reviewed-on: https://go-review.googlesource.com/c/go/+/471197
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
The Grow method is generally a more efficient way to grow a slice.
The older approach of using reflect.MakeSlice has to
waste effort zeroing the elements overwritten by the older slice
and has to allocate the slice header on the heap.
Performance:
name old time/op new time/op delta
CodeDecoder 2.41ms ± 2% 2.42ms ± 2% ~
CodeUnmarshal 3.12ms ± 3% 3.13ms ± 3% ~
CodeUnmarshalReuse 2.49ms ± 3% 2.52ms ± 3% ~
name old alloc/op new alloc/op delta
CodeDecoder 2.00MB ± 1% 1.99MB ± 1% ~
CodeUnmarshal 3.05MB ± 0% 2.92MB ± 0% -4.23%
CodeUnmarshalReuse 1.68MB ± 0% 1.68MB ± 0% -0.32%
name old allocs/op new allocs/op delta
CodeDecoder 77.1k ± 0% 77.0k ± 0% -0.09%
CodeUnmarshal 92.7k ± 0% 91.3k ± 0% -1.47%
CodeUnmarshalReuse 77.1k ± 0% 77.0k ± 0% -0.07%
The Code benchmarks (which are the only ones that uses slices)
are largely unaffected. There is a slight reduction in allocations.
A histogram of slice lengths from the Code testdata is as follows:
≤1: 392
≤2: 256
≤4: 252
≤8: 152
≤16: 126
≤32: 78
≤64: 62
≤128: 46
≤256: 18
≤512: 10
≤1024: 8
A bulk majority of slice lengths are 8 elements or under.
Use of reflect.Value.Grow performs better for larger slices since
it can avoid the zeroing of memory and has a faster growth rate.
However, Grow grows starting from 1 element,
with a 2x growth rate until some threshold (currently 512),
Starting from 1 ensures better utilization of the heap,
but at the cost of more frequent regrowth early on.
In comparison, the previous logic always started
with a minimum of 4 elements, which leads to a wasted capacity
of 75% for the highly frequent case of a single element slice.
The older code always had a growth rate of 1.5x,
and so wastes less memory for number of elements below 512.
All in all, there are too many factors that hurt or help performance.
Rergardless, the simplicity of favoring reflect.Value.Grow
over manually managing growth rates is a welcome simplification.
Change-Id: I62868a7f112ece3c2da3b4f6bdf74d397110243c
Reviewed-on: https://go-review.googlesource.com/c/go/+/471175
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
As part of the effort to rely less on bytes.Buffer,
switch most operations to use more natural append-like operations.
This makes it easier to swap bytes.Buffer out with a buffer type
that only needs to support a minimal subset of operations.
As a simplification, we can remove the use of the scratch buffer
and use the available capacity of the buffer itself as the scratch.
Also, declare an inlineable mayAppendQuote function to conditionally
append a double-quote if necessary.
Performance:
name old time/op new time/op delta
CodeEncoder 405µs ± 2% 397µs ± 2% -1.94% (p=0.000 n=20+20)
CodeEncoderError 453µs ± 1% 444µs ± 4% -1.83% (p=0.000 n=19+19)
CodeMarshal 559µs ± 4% 548µs ± 2% -2.02% (p=0.001 n=19+17)
CodeMarshalError 724µs ± 3% 716µs ± 2% -1.13% (p=0.030 n=19+20)
EncodeMarshaler 24.9ns ±15% 22.9ns ± 5% ~ (p=0.086 n=20+17)
EncoderEncode 14.0ns ±27% 15.0ns ±20% ~ (p=0.365 n=20+20)
There is a slight performance gain across the board due to
the elimination of the scratch buffer. Appends are done directly
into the unused capacity of the underlying buffer,
avoiding an additional copy. See #53685
Updates #27735
Change-Id: Icf6d612a7f7a51ecd10097af092762dd1225d49e
Reviewed-on: https://go-review.googlesource.com/c/go/+/469558
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
This is part of the effort to reduce direct reliance on bytes.Buffer
so that we can use a buffer with better pooling characteristics.
Unify these two methods as a single version that uses generics
to reduce duplicated logic. Unfortunately, we lack a generic
version of utf8.DecodeRune (see #56948), so we cast []byte to string.
The []byte variant is slightly slower for multi-byte unicode since
casting results in a stack-allocated copy operation.
Fortunately, this code path is used only for TextMarshalers.
We can also delete TestStringBytes, which exists to ensure
that the two duplicate implementations remain in sync.
Performance:
name old time/op new time/op delta
CodeEncoder 399µs ± 2% 409µs ± 2% +2.59% (p=0.000 n=9+9)
CodeEncoderError 450µs ± 1% 451µs ± 2% ~ (p=0.684 n=10+10)
CodeMarshal 553µs ± 2% 562µs ± 3% ~ (p=0.075 n=10+10)
CodeMarshalError 733µs ± 3% 737µs ± 2% ~ (p=0.400 n=9+10)
EncodeMarshaler 24.9ns ±12% 24.1ns ±13% ~ (p=0.190 n=10+10)
EncoderEncode 12.3ns ± 3% 14.7ns ±20% ~ (p=0.315 n=8+10)
name old speed new speed delta
CodeEncoder 4.87GB/s ± 2% 4.74GB/s ± 2% -2.53% (p=0.000 n=9+9)
CodeEncoderError 4.31GB/s ± 1% 4.30GB/s ± 2% ~ (p=0.684 n=10+10)
CodeMarshal 3.51GB/s ± 2% 3.46GB/s ± 3% ~ (p=0.075 n=10+10)
CodeMarshalError 2.65GB/s ± 3% 2.63GB/s ± 2% ~ (p=0.400 n=9+10)
name old alloc/op new alloc/op delta
CodeEncoder 327B ±347% 447B ±232% +36.93% (p=0.034 n=9+10)
CodeEncoderError 142B ± 1% 143B ± 0% ~ (p=1.000 n=8+7)
CodeMarshal 1.96MB ± 2% 1.96MB ± 2% ~ (p=0.468 n=10+10)
CodeMarshalError 2.04MB ± 3% 2.03MB ± 1% ~ (p=0.971 n=10+10)
EncodeMarshaler 4.00B ± 0% 4.00B ± 0% ~ (all equal)
EncoderEncode 0.00B 0.00B ~ (all equal)
name old allocs/op new allocs/op delta
CodeEncoder 0.00 0.00 ~ (all equal)
CodeEncoderError 4.00 ± 0% 4.00 ± 0% ~ (all equal)
CodeMarshal 1.00 ± 0% 1.00 ± 0% ~ (all equal)
CodeMarshalError 6.00 ± 0% 6.00 ± 0% ~ (all equal)
EncodeMarshaler 1.00 ± 0% 1.00 ± 0% ~ (all equal)
EncoderEncode 0.00 0.00 ~ (all equal)
There is a very slight performance degradation for CodeEncoder
due to an increase in allocation sizes. However, the number of allocations
did not change. This is likely due to remote effects of the growth rate
differences between bytes.Buffer and the builtin append function.
We shouldn't overly rely on the growth rate of bytes.Buffer anyways
since that is subject to possibly change in #51462.
As the benchtime increases, the alloc/op goes down indicating
that the amortized memory cost is fixed.
Updates #27735
Change-Id: Ie35e480e292fe082d7986e0a4d81212c1d4202b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/469556
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
This is part of the effort to reduce direct reliance on bytes.Buffer
so that we can use a buffer with better pooling characteristics.
Avoid direct use of bytes.Buffer in Compact and Indent and
instead modify the logic to rely only on append.
This avoids reliance on the bytes.Buffer.Truncate method,
which makes switching to a custom buffer implementation easier.
Performance:
name old time/op new time/op delta
EncodeMarshaler 25.5ns ± 8% 25.7ns ± 9% ~ (p=0.724 n=10+10)
name old alloc/op new alloc/op delta
EncodeMarshaler 4.00B ± 0% 4.00B ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
EncodeMarshaler 1.00 ± 0% 1.00 ± 0% ~ (all equal)
Updates #27735
Change-Id: I8cded03fab7651d43b5a238ee721f3472530868e
Reviewed-on: https://go-review.googlesource.com/c/go/+/469555
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Bryan Mills <bcmills@google.com>
Joe Tsai
Jes Cok
Daniel Martí
korzhao
apocelipes
miller
Ian Lance Taylor
qiulaidongfeng
chanxuehong
Sean Liao
Thuy Linh Luu
IvoGoman
Alan Donovan
Constantin Konstantinidis
cui fliter
Mateusz Poliwczak
Michael Teichgraeber
Johan Abildskov