Commit Graph

677 Commits

Author SHA1 Message Date
Constantin Konstantinidis 2afaa01855 encoding/binary: add word size to the error message of the failed constraint
Test added.

Fixes #22860

Change-Id: I08304834a2b7b10b4ac729bf36761692eb4731da
Reviewed-on: https://go-review.googlesource.com/c/go/+/113075
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-04 22:12:53 +00:00
cui fliter d8d22664c6 all: fix misuses of "a" vs "an"
Fixes the misuse of "a" vs "an", according to English grammatical
expectations and using https://www.a-or-an.com/

Change-Id: I53ac724070e3ff3d33c304483fe72c023c7cda47
Reviewed-on: https://go-review.googlesource.com/c/go/+/480536
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-04-04 14:20:53 +00:00
Daniel Martí f62c9701b4 encoding/gob: use reflect.Value.Grow
Growing by one is a simpler, and often cheaper,
operation compared to appending one (newly created) zero value.
The method was introduced in Go 1.20.

growSlice in dec_helpers.go is left alone,
as it grows using the builtin append instead of reflect.Append.

No noticeable performance difference on any of our benchmarks,
as this code only runs for slices large enough to not fit in
saferio.SliceCap, and none of our benchmarks use data that large.

	goos: linux
	goarch: amd64
	pkg: encoding/gob
	cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
						   │     old     │                new                 │
						   │   sec/op    │   sec/op     vs base               │
	DecodeBytesSlice-8       11.37µ ± 1%   11.46µ ± 4%       ~ (p=0.315 n=10)
	DecodeInterfaceSlice-8   96.49µ ± 1%   95.75µ ± 1%       ~ (p=0.436 n=10)
	geomean                  33.12µ        33.12µ       +0.01%

						   │     old      │                 new                 │
						   │     B/op     │     B/op      vs base               │
	DecodeBytesSlice-8       22.39Ki ± 0%   22.39Ki ± 0%       ~ (p=1.000 n=10)
	DecodeInterfaceSlice-8   80.25Ki ± 0%   80.25Ki ± 0%       ~ (p=0.650 n=10)
	geomean                  42.39Ki        42.39Ki       +0.00%

						   │     old     │                 new                  │
						   │  allocs/op  │  allocs/op   vs base                 │
	DecodeBytesSlice-8        169.0 ± 0%    169.0 ± 0%       ~ (p=1.000 n=10) ¹
	DecodeInterfaceSlice-8   3.178k ± 0%   3.178k ± 0%       ~ (p=1.000 n=10) ¹
	geomean                   732.9         732.9       +0.00%

Change-Id: I468aebf4ae6f197a1fd35f6fee809ca591c1788f
Reviewed-on: https://go-review.googlesource.com/c/go/+/481376
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-04-04 07:17:19 +00:00
Daniel Martí 56e900d9f0 encoding/gob: report allocs in benchmarks
I almost exclusively use these benchmarks with -benchtime already.

Change-Id: I6539cbba6abbdb6b275502e122f4e16856d8b9e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/481375
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rob Pike <r@golang.org>
2023-04-04 07:16:59 +00:00
Daniel Martí f80e270bab encoding/gob: avoid pointers to fieldType
fieldType is a struct with only a string and an integer,
so its size will barely be three times that of a pointer.

The indirection doesn't save us any memory or append/grow cost,
but it does cause a significant amount of allocations at init time.

	goos: linux
	goarch: amd64
	pkg: encoding/gob
	cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
					│     old     │                new                 │
					│   sec/op    │   sec/op     vs base               │
	EndToEndPipe-16   730.9n ± 5%   741.6n ± 5%       ~ (p=0.529 n=10)
	EncodingGob       173.7µ ± 0%   171.1µ ± 0%  -1.46% (p=0.000 n=10)
	geomean           11.27µ        11.26µ       -0.01%

					│     old      │                  new                   │
					│     B/op     │     B/op      vs base                  │
	EndToEndPipe-16   1.766Ki ± 0%   1.766Ki ± 0%        ~ (p=1.000 n=10) ¹
	EncodingGob       38.27Ki ± 0%   34.30Ki ± 0%  -10.38% (p=0.000 n=10)
	geomean           8.221Ki        7.782Ki        -5.33%
	¹ all samples are equal

					│    old     │                 new                 │
					│ allocs/op  │ allocs/op   vs base                 │
	EndToEndPipe-16   2.000 ± 0%   2.000 ± 0%       ~ (p=1.000 n=10) ¹
	EncodingGob       642.0 ± 0%   615.0 ± 0%  -4.21% (p=0.000 n=10)
	geomean           35.83        35.07       -2.13%
	¹ all samples are equal

Change-Id: I852a799834d2e9b7b915da74e871a4052d13892e
Reviewed-on: https://go-review.googlesource.com/c/go/+/479400
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2023-03-29 23:48:11 +00:00
Daniel Martí 72013c4700 encoding/gob: reuse calls to TypeOf for wireType
After the previous commit, both calls use the non-pointer type,
so we can deduplicate. No noticeable difference in init cost.

Change-Id: I0f0fb91d42655787cb58b4442ad3da4194560af4
Reviewed-on: https://go-review.googlesource.com/c/go/+/479399
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2023-03-29 23:47:35 +00:00
Daniel Martí 4d296bcbc2 encoding/gob: avoid a pointer to wireType in typeInfo
wireType itself is just a struct with seven pointer fields,
so an indirection doesn't feel necessary to noticeably reduce the amount
of memory that typeInfo takes for each Go type registered in gob.

The indirection does add a small amount of overhead though,
particularly one extra allocation when registering a type,
which is done a number of times as part of init.

For consistency, also update wireTypeUserInfo to not use a pointer.

Measuring via one of the end-to-end benchmarks and benchinit:

	goos: linux
	goarch: amd64
	pkg: encoding/gob
	cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
					│     old     │                new                 │
					│   sec/op    │   sec/op     vs base               │
	EndToEndPipe-16   736.8n ± 5%   733.9n ± 5%       ~ (p=0.971 n=10)
	EncodingGob       177.6µ ± 0%   173.6µ ± 0%  -2.27% (p=0.000 n=10)
	geomean           11.44µ        11.29µ       -1.34%

					│     old      │                  new                  │
					│     B/op     │     B/op      vs base                 │
	EndToEndPipe-16   1.766Ki ± 0%   1.766Ki ± 0%       ~ (p=1.000 n=10) ¹
	EncodingGob       38.47Ki ± 0%   38.27Ki ± 0%  -0.50% (p=0.000 n=10)
	geomean           8.241Ki        8.220Ki       -0.25%
	¹ all samples are equal

					│    old     │                 new                 │
					│ allocs/op  │ allocs/op   vs base                 │
	EndToEndPipe-16   2.000 ± 0%   2.000 ± 0%       ~ (p=1.000 n=10) ¹
	EncodingGob       652.0 ± 0%   642.0 ± 0%  -1.53% (p=0.000 n=10)
	geomean           36.11        35.83       -0.77%
	¹ all samples are equal

Change-Id: I528080b7d990ed595683f155a1ae25dcd26394b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/479398
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-29 22:57:25 +00:00
Daniel Martí 6a51c000de encoding/gob: use reflect.Value.IsZero
The reflect method was added in Go 1.13, in 2019.
gob's own version dates all the way back to 2011.

The behavior appears to be the same, and all tests still pass.
gob does have special cases like always encoding arrays even when they
are the zero value, but that is done via the sendZero boolean field.

Change-Id: I9057b7436963e231fdbf2f6c4b1edb58a2b13305
Reviewed-on: https://go-review.googlesource.com/c/go/+/479397
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-29 22:56:35 +00:00
Daniel Martí 09408a5b45 encoding/gob: avoid filling userTypeCache at init time
Doing this work at init time does make the first encode or decode call
slightly faster, but the cost is still paid upfront.

However, not all programs which directly or indirectly import
encoding/gob end up encoding or decoding any values.
For example, a program might only be run with the -help flag,
or it might only use gob encoding when a specific mode is enabled.

Moreover, any work done at init time needs to happen sequentially and
before the main function can start, blocking the entire program.

Using benchinit, we see a moderate saving at init time:

	goos: linux
	goarch: amd64
	cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
				│     old     │                new                 │
				│   sec/op    │   sec/op     vs base               │
	EncodingGob   188.9µ ± 0%   175.4µ ± 0%  -7.15% (p=0.000 n=10)

				│     old      │                 new                 │
				│     B/op     │     B/op      vs base               │
	EncodingGob   39.78Ki ± 0%   38.46Ki ± 0%  -3.32% (p=0.000 n=10)

				│    old     │                new                │
				│ allocs/op  │ allocs/op   vs base               │
	EncodingGob   668.0 ± 0%   652.0 ± 0%  -2.40% (p=0.000 n=10)

Change-Id: I75a5df18c9b1d02566e5885a966360d8a525913a
Reviewed-on: https://go-review.googlesource.com/c/go/+/479396
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
2023-03-29 22:56:11 +00:00
Mateusz Poliwczak c292397160 encoding/asn1: improve memory efficiency of ObjectIdentifier.String
name                      old time/op    new time/op    delta
ObjectIdentifierString-4     670ns ± 9%     157ns ±14%  -76.59%  (p=0.000 n=10+9)

name                      old alloc/op   new alloc/op   delta
ObjectIdentifierString-4      184B ± 0%       32B ± 0%  -82.61%  (p=0.000 n=10+10)

name                      old allocs/op  new allocs/op  delta
ObjectIdentifierString-4      14.0 ± 0%       1.0 ± 0%  -92.86%  (p=0.000 n=10+10)

This also improves the x509 certificate parser performance by ~12-15%

name                           old time/op    new time/op    delta
ParseCertificate/ecdsa_leaf-4    24.5µs ± 8%    20.9µs ±11%  -14.66%  (p=0.000 n=10+10)
ParseCertificate/rsa_leaf-4      26.6µs ± 5%    23.5µs ± 7%  -11.83%  (p=0.000 n=8+10)

name                           old alloc/op   new alloc/op   delta
ParseCertificate/ecdsa_leaf-4    12.5kB ± 0%    12.0kB ± 0%   -3.72%  (p=0.000 n=10+10)
ParseCertificate/rsa_leaf-4      13.9kB ± 0%    13.4kB ± 0%   -3.34%  (p=0.000 n=10+10)

name                           old allocs/op  new allocs/op  delta
ParseCertificate/ecdsa_leaf-4       238 ± 0%       165 ± 0%  -30.67%  (p=0.000 n=10+10)
ParseCertificate/rsa_leaf-4         262 ± 0%       189 ± 0%  -27.86%  (p=0.000 n=10+10)

Change-Id: I49905bbf8319b840e9211da73570db35d1445217
GitHub-Last-Rev: 361d68dc9b
GitHub-Pull-Request: golang/go#59198
Reviewed-on: https://go-review.googlesource.com/c/go/+/478836
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Mateusz Poliwczak <mpoliwczak34@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-29 18:24:36 +00:00
Michael Teichgraeber f7156f63b5 encoding/asn1: support fractions of a second when unmarshaling GeneralizedTime
A GeneralizedTime value may contain an optional fractional seconds
element (according to X.680 46.2, restricted by X.690 11.7.3). This
change adds support for this fractional part, up to nine digits, so that
Unmarshal won't fail when decoding a DER encoded GeneralizedTime value
with fractional digits.  Also, test cases related to this change have
been added.

X.680 and X.690 can be found at:
	https://www.itu.int/rec/T-REC-X.680
	https://www.itu.int/rec/T-REC-X.690

Fixes #15842

Change-Id: If217c007e01b686db508a940e9e2ed3bfb901879
Reviewed-on: https://go-review.googlesource.com/c/go/+/108355
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-28 16:22:36 +00:00
Daniel Martí 9768f736ea all: add a few links in package godocs
I noticed the one in path/filepath while reading the docs,
and the other ones were found via some quick grepping.

Change-Id: I386f2f74ef816a6d18aa2f58ee6b64dbd0147c9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/478795
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-25 10:16:23 +00:00
Ian Lance Taylor c994067e5b encoding/gob: update decgen to generate current dec_helpers
I edited dec_helpers.go without realizing that it is a generated file.
Fix the generator to generate the current version (which generates
a small comment change).

Change-Id: I70e3bc78eb0728d23c08972611218f288dc1d29c
Reviewed-on: https://go-review.googlesource.com/c/go/+/479117
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Andrew Ekstedt <andrew.ekstedt@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-03-24 19:28:46 +00:00
Johan Abildskov 3d5391ed87 encoding/gob: extend partially allocated string slices
Fixes #59172

Change-Id: I54d5e724f10117a40ec5dd58c810f6bbb2475933
GitHub-Last-Rev: d1a986698c
GitHub-Pull-Request: golang/go#59173
Reviewed-on: https://go-review.googlesource.com/c/go/+/478215
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-03-23 17:32:41 +00:00
Joe Tsai 2e51f6f25c encoding/json: make use of Buffer.AvailableBuffer
Use the explicit API for acquiring an empty available buffer,
rather than the hack that's implemented in terms of Bytes and Len.

Change-Id: If286ed42693acd61ffe28dc849ed4b76c3ae4434
Reviewed-on: https://go-review.googlesource.com/c/go/+/476337
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-15 15:41:55 +00:00
Joe Tsai b9b8cecbfc encoding/json: simplify folded name logic
The folded name logic (despite all attempts to optimize it)
was fundamentally an O(n) operation where every field in a struct
needed to be linearly scanned in order to find a match.
This made unmashaling of unknown fields always O(n).
Instead of optimizing the comparison for each field,
make it such that we can look up a name in O(1).

We accomplish this by maintaining a map keyed by pre-folded names,
which we can pre-calculate when processing the struct type.
Using a stack-allocated buffer, we can fold the input name and
look up its presence in the map.

Also, instead of mapping from names to indexes,
map directly to a pointer to the field information.
The memory cost of this is the same and avoids an extra slice index.

The new logic is both simpler and faster.

Performance:

	name                   old time/op    new time/op    delta
	CodeDecoder           2.47ms ± 4%    2.42ms ± 2%  -1.83%  (p=0.022 n=10+9)
	UnicodeDecoder         259ns ± 2%     248ns ± 1%  -4.32%  (p=0.000 n=10+10)
	DecoderStream          150ns ± 1%     149ns ± 1%    ~     (p=0.516 n=10+10)
	CodeUnmarshal         3.13ms ± 2%    3.09ms ± 2%  -1.37%  (p=0.022 n=10+9)
	CodeUnmarshalReuse    2.50ms ± 1%    2.45ms ± 1%  -1.96%  (p=0.001 n=8+9)
	UnmarshalString       67.1ns ± 5%    64.5ns ± 5%  -3.90%  (p=0.005 n=10+10)
	UnmarshalFloat64      60.1ns ± 4%    58.4ns ± 2%  -2.89%  (p=0.002 n=10+8)
	UnmarshalInt64        51.0ns ± 4%    49.2ns ± 1%  -3.53%  (p=0.001 n=10+8)
	Issue10335            80.7ns ± 2%    79.2ns ± 1%  -1.82%  (p=0.016 n=10+8)
	Issue34127            28.6ns ± 3%    28.8ns ± 3%    ~     (p=0.388 n=9+10)
	Unmapped               177ns ± 2%     177ns ± 2%    ~     (p=0.956 n=10+10)

Change-Id: I478b2b958f5a63a69c9a991a39cd5ffb43244a2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/471196
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2023-02-27 17:37:27 +00:00
Joe Tsai 2de406bb9e encoding/json: use append for HTMLEscape
Use append for HTMLEscape similar to Indent and Compact.
Move it to indent.go alongside Compact, as it shares similar logic.
In a future CL, we will modify appendCompact to be written in terms
of appendHTMLEscape, but we need to first move the JSON decoder logic
out of the main loop of appendCompact.

Change-Id: I131c64cd53d5d2b4ca798b37349aeefe17b418c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/471198
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
2023-02-27 17:33:26 +00:00
Joe Tsai cb3e170ae7 encoding/json: rely on reflect.Value.SetZero
v.SetZero() is faster than v.Set(reflect.Zero(v.Type()))
and was recently added in Go 1.20.

Benchmark numbers are largely unchanged since this mainly
affects the unmarshaling of large numbers of JSON nulls,
which our benchmarks do not heavily exercise.

Change-Id: I464f60f63c9027e63a99fd5da92e7ab782018329
Reviewed-on: https://go-review.googlesource.com/c/go/+/471195
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2023-02-27 17:29:42 +00:00
Joe Tsai 8367e2dfc7 encoding/json: remove legacy fuzz.go file
With native support for fuzzing in the Go toolchain,
rely instead on the fuzz tests declared in fuzz_test.go.

Change-Id: I601842cd0bc7e64ea3bfdafbbbc3534df11acf59
Reviewed-on: https://go-review.googlesource.com/c/go/+/471197
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-27 17:26:54 +00:00
Daniel Martí fcfbbf2ff6 encoding/gob: use reflect.Value.SetZero
Particularly helps when decoding map maps.

	goos: linux
	goarch: amd64
	pkg: encoding/gob
	cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
							│     old     │                new                 │
							│   sec/op    │   sec/op     vs base               │
	DecodeStringSlice-16      30.78µ ± 2%   30.68µ ± 1%        ~ (p=0.699 n=6)
	DecodeStringsSlice-16     61.36µ ± 6%   60.85µ ± 4%        ~ (p=0.589 n=6)
	DecodeBytesSlice-16       16.57µ ± 2%   16.51µ ± 4%        ~ (p=0.937 n=6)
	DecodeInterfaceSlice-16   124.3µ ± 0%   125.4µ ± 1%        ~ (p=0.065 n=6)
	DecodeMap-16              251.0µ ± 3%   216.2µ ± 2%  -13.86% (p=0.002 n=6)
	geomean                   62.80µ        60.87µ        -3.07%

							│     old      │                new                 │
							│     B/op     │     B/op      vs base              │
	DecodeStringSlice-16      37.98Ki ± 0%   37.98Ki ± 0%       ~ (p=1.000 n=6)
	DecodeStringsSlice-16     63.79Ki ± 0%   63.79Ki ± 0%       ~ (p=0.675 n=6)
	DecodeBytesSlice-16       22.40Ki ± 0%   22.40Ki ± 0%       ~ (p=0.364 n=6)
	DecodeInterfaceSlice-16   80.27Ki ± 0%   80.27Ki ± 0%       ~ (p=0.924 n=6)
	DecodeMap-16              52.67Ki ± 0%   52.67Ki ± 0%       ~ (p=0.145 n=6)
	geomean                   47.01Ki        47.01Ki       +0.00%

							│     old     │                 new                 │
							│  allocs/op  │  allocs/op   vs base                │
	DecodeStringSlice-16      1.169k ± 0%   1.169k ± 0%       ~ (p=1.000 n=6) ¹
	DecodeStringsSlice-16     2.178k ± 0%   2.178k ± 0%       ~ (p=1.000 n=6) ¹
	DecodeBytesSlice-16        169.0 ± 0%    169.0 ± 0%       ~ (p=1.000 n=6) ¹
	DecodeInterfaceSlice-16   3.178k ± 0%   3.178k ± 0%       ~ (p=1.000 n=6) ¹
	DecodeMap-16               181.0 ± 0%    181.0 ± 0%       ~ (p=1.000 n=6) ¹
	geomean                    756.3         756.3       +0.00%
	¹ all samples are equal

Change-Id: I5ae9268b1a86296494d8569f5158ef0e78128eb1
Reviewed-on: https://go-review.googlesource.com/c/go/+/471257
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-02-27 15:31:13 +00:00
Joe Tsai ac27b4dd1d encoding/json: rely on reflect.Value.Grow
The Grow method is generally a more efficient way to grow a slice.
The older approach of using reflect.MakeSlice has to
waste effort zeroing the elements overwritten by the older slice
and has to allocate the slice header on the heap.

Performance:

	name                old time/op    new time/op    delta
	CodeDecoder         2.41ms ± 2%    2.42ms ± 2%    ~
	CodeUnmarshal       3.12ms ± 3%    3.13ms ± 3%    ~
	CodeUnmarshalReuse  2.49ms ± 3%    2.52ms ± 3%    ~

	name                 old alloc/op  new alloc/op   delta
	CodeDecoder         2.00MB ± 1%    1.99MB ± 1%    ~
	CodeUnmarshal       3.05MB ± 0%    2.92MB ± 0%    -4.23%
	CodeUnmarshalReuse  1.68MB ± 0%    1.68MB ± 0%    -0.32%

	name                old allocs/op  new allocs/op  delta
	CodeDecoder         77.1k ± 0%     77.0k ± 0%     -0.09%
	CodeUnmarshal       92.7k ± 0%     91.3k ± 0%     -1.47%
	CodeUnmarshalReuse  77.1k ± 0%     77.0k ± 0%     -0.07%

The Code benchmarks (which are the only ones that uses slices)
are largely unaffected. There is a slight reduction in allocations.

A histogram of slice lengths from the Code testdata is as follows:

	   ≤1: 392
	   ≤2: 256
	   ≤4: 252
	   ≤8: 152
	  ≤16: 126
	  ≤32: 78
	  ≤64: 62
	 ≤128: 46
	 ≤256: 18
	 ≤512: 10
	≤1024: 8

A bulk majority of slice lengths are 8 elements or under.
Use of reflect.Value.Grow performs better for larger slices since
it can avoid the zeroing of memory and has a faster growth rate.
However, Grow grows starting from 1 element,
with a 2x growth rate until some threshold (currently 512),
Starting from 1 ensures better utilization of the heap,
but at the cost of more frequent regrowth early on.

In comparison, the previous logic always started
with a minimum of 4 elements, which leads to a wasted capacity
of 75% for the highly frequent case of a single element slice.
The older code always had a growth rate of 1.5x,
and so wastes less memory for number of elements below 512.

All in all, there are too many factors that hurt or help performance.
Rergardless, the simplicity of favoring reflect.Value.Grow
over manually managing growth rates is a welcome simplification.

Change-Id: I62868a7f112ece3c2da3b4f6bdf74d397110243c
Reviewed-on: https://go-review.googlesource.com/c/go/+/471175
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
2023-02-27 11:58:36 +00:00
Joe Tsai 8e5f56a2e3 encoding/json: use append-like operations for encoding
As part of the effort to rely less on bytes.Buffer,
switch most operations to use more natural append-like operations.
This makes it easier to swap bytes.Buffer out with a buffer type
that only needs to support a minimal subset of operations.

As a simplification, we can remove the use of the scratch buffer
and use the available capacity of the buffer itself as the scratch.
Also, declare an inlineable mayAppendQuote function to conditionally
append a double-quote if necessary.

Performance:

    name              old time/op    new time/op    delta
    CodeEncoder          405µs ± 2%     397µs ± 2%  -1.94%  (p=0.000 n=20+20)
    CodeEncoderError     453µs ± 1%     444µs ± 4%  -1.83%  (p=0.000 n=19+19)
    CodeMarshal          559µs ± 4%     548µs ± 2%  -2.02%  (p=0.001 n=19+17)
    CodeMarshalError     724µs ± 3%     716µs ± 2%  -1.13%  (p=0.030 n=19+20)
    EncodeMarshaler     24.9ns ±15%    22.9ns ± 5%    ~     (p=0.086 n=20+17)
    EncoderEncode       14.0ns ±27%    15.0ns ±20%    ~     (p=0.365 n=20+20)

There is a slight performance gain across the board due to
the elimination of the scratch buffer. Appends are done directly
into the unused capacity of the underlying buffer,
avoiding an additional copy. See #53685

Updates #27735

Change-Id: Icf6d612a7f7a51ecd10097af092762dd1225d49e
Reviewed-on: https://go-review.googlesource.com/c/go/+/469558
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
2023-02-24 19:16:21 +00:00
Joe Tsai e2f4134025 encoding/json: unify encodeState.string and encodeState.stringBytes
This is part of the effort to reduce direct reliance on bytes.Buffer
so that we can use a buffer with better pooling characteristics.

Unify these two methods as a single version that uses generics
to reduce duplicated logic. Unfortunately, we lack a generic
version of utf8.DecodeRune (see #56948), so we cast []byte to string.
The []byte variant is slightly slower for multi-byte unicode since
casting results in a stack-allocated copy operation.
Fortunately, this code path is used only for TextMarshalers.
We can also delete TestStringBytes, which exists to ensure
that the two duplicate implementations remain in sync.

Performance:

    name              old time/op    new time/op    delta
    CodeEncoder          399µs ± 2%     409µs ± 2%   +2.59%  (p=0.000 n=9+9)
    CodeEncoderError     450µs ± 1%     451µs ± 2%     ~     (p=0.684 n=10+10)
    CodeMarshal          553µs ± 2%     562µs ± 3%     ~     (p=0.075 n=10+10)
    CodeMarshalError     733µs ± 3%     737µs ± 2%     ~     (p=0.400 n=9+10)
    EncodeMarshaler     24.9ns ±12%    24.1ns ±13%     ~     (p=0.190 n=10+10)
    EncoderEncode       12.3ns ± 3%    14.7ns ±20%     ~     (p=0.315 n=8+10)

    name              old speed      new speed      delta
    CodeEncoder       4.87GB/s ± 2%  4.74GB/s ± 2%   -2.53%  (p=0.000 n=9+9)
    CodeEncoderError  4.31GB/s ± 1%  4.30GB/s ± 2%     ~     (p=0.684 n=10+10)
    CodeMarshal       3.51GB/s ± 2%  3.46GB/s ± 3%     ~     (p=0.075 n=10+10)
    CodeMarshalError  2.65GB/s ± 3%  2.63GB/s ± 2%     ~     (p=0.400 n=9+10)

    name              old alloc/op   new alloc/op   delta
    CodeEncoder          327B ±347%     447B ±232%  +36.93%  (p=0.034 n=9+10)
    CodeEncoderError      142B ± 1%      143B ± 0%     ~     (p=1.000 n=8+7)
    CodeMarshal         1.96MB ± 2%    1.96MB ± 2%     ~     (p=0.468 n=10+10)
    CodeMarshalError    2.04MB ± 3%    2.03MB ± 1%     ~     (p=0.971 n=10+10)
    EncodeMarshaler      4.00B ± 0%     4.00B ± 0%     ~     (all equal)
    EncoderEncode        0.00B          0.00B          ~     (all equal)

    name              old allocs/op  new allocs/op  delta
    CodeEncoder           0.00           0.00          ~     (all equal)
    CodeEncoderError      4.00 ± 0%      4.00 ± 0%     ~     (all equal)
    CodeMarshal           1.00 ± 0%      1.00 ± 0%     ~     (all equal)
    CodeMarshalError      6.00 ± 0%      6.00 ± 0%     ~     (all equal)
    EncodeMarshaler       1.00 ± 0%      1.00 ± 0%     ~     (all equal)
    EncoderEncode         0.00           0.00          ~     (all equal)

There is a very slight performance degradation for CodeEncoder
due to an increase in allocation sizes. However, the number of allocations
did not change. This is likely due to remote effects of the growth rate
differences between bytes.Buffer and the builtin append function.
We shouldn't overly rely on the growth rate of bytes.Buffer anyways
since that is subject to possibly change in #51462.
As the benchtime increases, the alloc/op goes down indicating
that the amortized memory cost is fixed.

Updates #27735

Change-Id: Ie35e480e292fe082d7986e0a4d81212c1d4202b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/469556
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
2023-02-24 19:00:16 +00:00
Joe Tsai 21ff6704bc encoding/json: use append for Compact and Indent
This is part of the effort to reduce direct reliance on bytes.Buffer
so that we can use a buffer with better pooling characteristics.

Avoid direct use of bytes.Buffer in Compact and Indent and
instead modify the logic to rely only on append.
This avoids reliance on the bytes.Buffer.Truncate method,
which makes switching to a custom buffer implementation easier.

Performance:

	name                old time/op    new time/op    delta
	EncodeMarshaler    25.5ns ± 8%    25.7ns ± 9%   ~     (p=0.724 n=10+10)

	name                old alloc/op   new alloc/op   delta
	EncodeMarshaler     4.00B ± 0%     4.00B ± 0%   ~     (all equal)

	name                old allocs/op  new allocs/op  delta
	EncodeMarshaler      1.00 ± 0%      1.00 ± 0%   ~     (all equal)

Updates #27735

Change-Id: I8cded03fab7651d43b5a238ee721f3472530868e
Reviewed-on: https://go-review.googlesource.com/c/go/+/469555
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-02-24 19:00:14 +00:00
Joe Tsai 81eda3a339 Revert "hex: fix panic in Decode when len(src) > 2*len(dst)"
This reverts CL 461958 and CL 465855.

Reason for revert: This introduced an irreconcilable inconsistency with Encode

Fixes #58391.

Change-Id: Ifd01a04d433b24c092b73e627b8149a5851c2bca
Reviewed-on: https://go-review.googlesource.com/c/go/+/469615
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-02-21 15:35:11 +00:00
Oleksandr Redko 3ad6393f86 src: rename unexported errors by adding prefix err
By convention, use `err` as prefix for variables of type `error`.

Change-Id: I9401d5d47e994a27be245b2c8b1edd55cdd52db1
Reviewed-on: https://go-review.googlesource.com/c/go/+/467536
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-02-16 23:09:19 +00:00
Joe Tsai f02cdba163 encoding/hex: fix Decode output check regression
CL 461958 fixed a potential panic,
but also introduced an observable regression where
invalid input could be detected before the panic occurs.
Adjust the check to preserve prior behavior,
while also preventing the panic.

Change-Id: I52819f88a6a64883fbc9fd512697c38c29ca0ccd
Reviewed-on: https://go-review.googlesource.com/c/go/+/465855
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-02-07 15:37:55 +00:00
Constantin Konstantinidis dfd2ddd671 encoding/xml: allow overriding by empty namespace
The namespace defined by xmlns="value" can be overridden in every included tag
by the empty namespace xmlns="" without a prefix.

Method to calculate indent of XML handles depth of tag and its associated namespace is
still active even when no indent is required.

An XMLName field in a struct means that namespace must be enforced even if empty.
This occurs only on an inner tag as an override of any non-empty namespace of its outer tag.
An attribute is added to have the required namespace display.

Fixes #7113

Change-Id: I57f2308e98c66f04108ab136d350bdc3a6091e98
Reviewed-on: https://go-review.googlesource.com/c/go/+/108796
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-01-31 18:23:50 +00:00
cuiweixie 848d658c45 encoding/binary: add String and GoString method to nativeEndian
Updates #57237

Change-Id: Ib626610130cae9c1d1aff5dd2a5035ffde0e127f
Reviewed-on: https://go-review.googlesource.com/c/go/+/463985
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: xie cui <523516579@qq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-30 18:51:09 +00:00
cui fliter dbfdc446fe all: fix problematic comments
Change-Id: If092ae7c72b66f172ae32fa6c7294a7ac250362e
Reviewed-on: https://go-review.googlesource.com/c/go/+/463995
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
2023-01-30 18:10:36 +00:00
cuiweixie 4b3726e99b encoding/binary: add var NativeEndian
Updates #57237

Change-Id: I149c8b7eeac91b87b5810250f96d48ca87135807
Reviewed-on: https://go-review.googlesource.com/c/go/+/463218
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: xie cui <523516579@qq.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-01-27 18:17:20 +00:00
Benjamin Prosnitz a106defdda hex: fix panic in Decode when len(src) > 2*len(dst)
hex.Decode never checks the length of dst and triggers a panic
if there are insufficient bytes in the slice.

There isn't document on what the behavior *should* be in this case.
Two possibilities:
1. Error dst has insufficient space (as done in this change)
2. Reduce the length of the decode to min(dst, src)

Option 1 was chosen because it seems the least surprising or
subtle.

Change-Id: I3bf029e3d928202de716830434285e3c165f26dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/461958
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Benjamin Prosnitz <bprosnitz@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2023-01-27 15:28:30 +00:00
Daniel Martí 43f9b826c3 encoding/gob: slightly simplify init code
https://go.dev/cl/460543 stopped using the "expect" parameter in
bootstrapType, but we forgot to actually remove it.

While here, staticcheck correctly points out that we can use the copy
builtin to fill builtinIdToTypeSlice, now that it and idToType are an
array and slice respectively.

Change-Id: I48078415ab9bdd5633cf41f33ab4dc78eb30b48a
Reviewed-on: https://go-review.googlesource.com/c/go/+/462301
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Rob Pike <r@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2023-01-24 20:07:49 +00:00
Daniel Martí 8259ac4986 encoding/gob: shave off some init time cost
Avoid unnecessary allocations when calling reflect.TypeOf;
we can use nil pointers, which fit into an interface without allocating.
This saves about 1% of CPU time.

The builtin types are limited to typeIds between 0 and firstUserId,
and since firstUserId is 64, builtinIdToType does not need to be a map.
We can simply use an array of length firstUserId, which is simpler.
This saves about 1% of CPU time.

idToType is similar to firstUserId in that it is a map keyed by typeIds.
The difference is that it can grow with the user's types.
However, each added type gets the next available typeId,
meaning that we can use a growing slice, similar to the case above.
nextId then becomes the current length of the slice.
This saves about 1% of CPU time.

typeInfoMap is stored globally as an atomic.Value,
where each modification loads the map, makes a whole copy,
adds the new element, and stores the modified copy.
This is perfectly fine when the user registers types,
as that can happen concurrently and at any point in the future.

However, during init time, we sequentially register many types,
and the overhead of copying maps adds up noticeably.
During init time, use a regular global map instead,
which gets replaced by the atomic.Value when our init work is done.
This saves about 2% of CPU time.

Finally, avoid calling checkId in bootstrapType;
we have just called setTypeId, whose logic for getting nextId is simple,
so the extra check doesn't gain us much.
This saves about 1% of CPU time.

Using benchinit, which transforms GODEBUG=inittrace=1 data into Go
benchmark compatible output, results in a nice improvement:

	name         old time/op    new time/op    delta
	EncodingGob     175µs ± 0%     162µs ± 0%  -7.45%  (p=0.016 n=5+4)

	name         old alloc/op   new alloc/op   delta
	EncodingGob    39.0kB ± 0%    36.1kB ± 0%  -7.35%  (p=0.016 n=5+4)

	name         old allocs/op  new allocs/op  delta
	EncodingGob       588 ± 0%       558 ± 0%  -5.10%  (p=0.000 n=5+4)

Updates #26775.

Change-Id: I28618e8b96ef440480e666ef2cd5c4a9a332ef21
Reviewed-on: https://go-review.googlesource.com/c/go/+/460543
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-20 09:34:41 +00:00
Russ Cox db4be8f510 encoding/csv: use proper doc comment for Deprecated notes
End-of-line comments are not doc comments,
so Deprecated notes in them are not recognized
as deprecation notices. Rewrite the comments.

Change-Id: I275fa9aec403132fda45853e52daa22bc06fcd36
Reviewed-on: https://go-review.googlesource.com/c/go/+/453617
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-12-02 16:30:23 +00:00
Ian Lance Taylor 483f3dc39f Revert "encoding/xml: disallow empty namespace when prefix is set"
This reverts CL 105636.

Reason for revert: Fails with existing XML data.  At this stage in the release cycle we should revert, and try again next time with some way to support existing XML.

For #8068

Change-Id: Ia84cbf3a84878ac7190f72998545dee22c36c45e
Reviewed-on: https://go-review.googlesource.com/c/go/+/453996
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2022-11-29 22:25:08 +00:00
cui fliter b2faff18ce all: add missing periods in comments
Change-Id: I69065f8adf101fdb28682c55997f503013a50e29
Reviewed-on: https://go-review.googlesource.com/c/go/+/449757
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joedian Reid <joedian@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-18 17:59:44 +00:00
Constantin Konstantinidis 89332e037a encoding/xml: error when more than one colon in qualified names
Add test.

Fixes #20396

Change-Id: I89e9013eb338f831e1908e390b284794df78fb6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/103875
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-11-09 22:56:44 +00:00
Constantin Konstantinidis bef5eca118 encoding/xml: disallow empty namespace when prefix is set
Non-regression tests are added.

Fixes #8068

Change-Id: Icb36c910bbf4955743b7aa8382002b2d9246fadc
Reviewed-on: https://go-review.googlesource.com/c/go/+/105636
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-11-09 22:45:43 +00:00
Constantin Konstantinidis f13849a7af encoding/xml: error when closing tag does not match opening tag
Comparing opening and closing tag is done using the prefix when available.
Documentation states that Token returns URI in the Space part of the Name.
Translation has been moved for the End tag before the namespace is removed
from the stack.

After closing a tag using a namespace, the valid namespace must be taken
from the opening tag. Tests added.

Fixes #20685

Change-Id: I4d90b19f7e21a76663f0ea1c1db6c6bf9fd2a389
Reviewed-on: https://go-review.googlesource.com/c/go/+/107255
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-11-09 03:01:28 +00:00
Constantin Konstantinidis b459166219 encoding/xml: add check of namespaces to detect field names conflicts
Test added.

Fixes #8535

Change-Id: Ic89c2781e81d963a653180812748b3fc95fb7fae
Reviewed-on: https://go-review.googlesource.com/c/go/+/106575
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-11-09 03:01:25 +00:00
Dan Kortschak 1c05968c9a all: fix comment typos
Change-Id: Ic16824482142d4de4d0b949459e36505ee944ff7
Reviewed-on: https://go-review.googlesource.com/c/go/+/448175
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Dan Kortschak <dan@kortschak.io>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dan Kortschak <dan@kortschak.io>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-05 21:02:45 +00:00
Russ Cox 6695cebeec encoding/xml: reduce depth limit on wasm
Wasm can't handle the recusion for XML nested to depth 10,000.
Cut it off at 5,000 instead. This fixes TestCVE202228131 on trybots
in certain conditions.

Also disable TestCVE202230633 to fix 'go test -v encoding/xml' on gomotes.

Also rename errExeceededMaxUnmarshalDepth [misspelled and unwieldy]
to errUnmarshalDepth.

For #56498.

Change-Id: I7cc337ccfee251bfd9771497be0e5272737114f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/446639
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-31 20:35:56 +00:00
Ian Lance Taylor 4c61e079c0 encoding/gob: support large slices in slice decode helpers
The slice decode helpers weren't aware of partially allocated slices.

Also add large slice support for []byte.

Change-Id: I5044587e917508887c7721f8059d364189831693
Reviewed-on: https://go-review.googlesource.com/c/go/+/443777
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-20 23:11:47 +00:00
Roland Shoemaker 36ca37f3a0 encoding/gob: note pacakge not covered by security policy
And add a link. Resolves a comment left on http://go.dev/cl/436096
after it was submitted.

Change-Id: I2847d29134ffb4fee2b0ea37842cdf57df55ec0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/442816
Reviewed-by: Julie Qiu <julieqiu@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-13 18:26:45 +00:00
cui fliter b314eea4cd all: remove redundant type conversion
Change-Id: I375233dc700adbc58a6d4af995d07b352bf85b11
GitHub-Last-Rev: ef12920523
GitHub-Pull-Request: golang/go#55994
Reviewed-on: https://go-review.googlesource.com/c/go/+/437715
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2022-10-06 04:37:00 +00:00
Tomasz Jezierski 9c2fd81ee1 encoding/asn1: remove allocation from init
asn1 allocates due to reflect.TypeOf(new(big.Int)) in init time.
We could replace it with (*big.Int)(nil).

Before:
init encoding/asn1 @1.0 ms, 0.009 ms clock, 224 bytes, 7 allocs

After:
init encoding/asn1 @0.70 ms, 0.002 ms clock, 192 bytes, 6 allocs

Fixes #55973

Change-Id: I7c3cc0f48631af73cf34ad3c731c380f46c72359
Reviewed-on: https://go-review.googlesource.com/c/go/+/435257
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: hopehook <hopehook@golangcn.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-10-04 23:29:48 +00:00
Daniel Martí cbd931c9c2 encoding/gob: prevent a decoder state overflow
When decoding a struct, if a positive delta is large enough to overflow
when added to fieldnum, we would panic due to the resulting negative index.

Instead, catch this problem and produce an error like we do with
negative delta integers. If fieldnum ends up being negative or smaller
than state.fieldnum, the addition overflowed.

While here, remove an unnecessary break after an error call,
since those error functions cause a panic.

Fixes #55337.

Change-Id: I7a0e4f43e5c81a703e79c1597e3bb3714cc832c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/432715
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-04 13:27:35 +00:00
cui fliter aeab76f0be encoding: fix a few function names on comments
Change-Id: I17a311afb94a056b3d35bfa241f5d0d206db602d
GitHub-Last-Rev: 42129464c9
GitHub-Pull-Request: golang/go#55962
Reviewed-on: https://go-review.googlesource.com/c/go/+/436882
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
2022-09-30 23:03:07 +00:00
cuiweixie 1e65fa58c1 encoding/json: return comparison directly
Change-Id: I4698d0fa78108d83ee91732e8d3878dbff7f9c90
Reviewed-on: https://go-review.googlesource.com/c/go/+/436711
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: xie cui <523516579@qq.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2022-09-30 20:10:56 +00:00
hopehook 972496ae6e all: use strings.Builder where appropriate
Change-Id: I164d350ca480640996055dedf38d962921c474a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/435975
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-29 22:56:00 +00:00
Roland Shoemaker 13d48bb6a1 encoding/gob: add top level security doc
Add a slightly expanded version of the Decoder type comment to the top
level package doc, which explains that this package is not designed
to be hardened against adversarial inputs.

Change-Id: I8b83433838c8235eb06ded99041fdf726c811ee5
Reviewed-on: https://go-review.googlesource.com/c/go/+/436096
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2022-09-29 00:30:21 +00:00
cuiweixie 09919ac339 encoding/xml: use bytes.Clone
Change-Id: I3218b1e3f8869f579facddb29471df13c835dc66
Reviewed-on: https://go-review.googlesource.com/c/go/+/435281
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2022-09-28 03:44:25 +00:00
Ian Lance Taylor a0441c7ae3 encoding/gob: use saferio.SliceCap when decoding a slice
This avoids allocating an overly large slice for corrupt input.

Change the saferio.SliceCap function to take a pointer to the element type,
so that we can handle slices of interface types. This revealed that a
couple of existing calls were actually incorrect, passing the slice type
rather than the element type.

No test case because the problem can only happen for invalid data. Let
the fuzzer find cases like this.

Fixes #55338

Change-Id: I3c1724183cc275d4981379773b0b8faa01a9cbd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/433296
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-25 01:18:43 +00:00
cuiweixie ab87db8b46 encoding: use strings.Builder
Change-Id: Ib07699df8ea41fd8d1bca8ad050859fac24623de
Reviewed-on: https://go-review.googlesource.com/c/go/+/428258
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-06 15:46:20 +00:00
RenKanai 202b7e7e76 encoding/asn1: fix doc for BitString.At's return value
Fixes #53287

Change-Id: If983ae34850d9b1b29764156a38628fa53897573
Reviewed-on: https://go-review.googlesource.com/c/go/+/411134
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: hopehook <hopehook@golangcn.org>
Reviewed-by: Subham <sarkar.subhams2@gmail.com>
2022-09-02 09:14:48 +00:00
Andy Pan 1a8dfadbfe encoding/json: move some misplaced benchmark tests to bench_test.go
Change-Id: I5987eed00ee825421abe62699a06e9b66499f35f
Reviewed-on: https://go-review.googlesource.com/c/go/+/425016
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-24 05:40:28 +00:00
Andy Pan e7f2e5697a encoding/json: give it a chance to put encodeState back in pool when error occurs
name                       old time/op    new time/op    delta
CodeEncoderError-10           688µs ± 8%     496µs ±15%   -27.92%  (p=0.000 n=10+9)
CodeMarshalError-10           747µs ± 6%     546µs ± 4%   -26.86%  (p=0.000 n=10+10)
MarshalBytesError/32-10       284µs ± 2%     273µs ± 1%    -3.84%  (p=0.000 n=10+10)
MarshalBytesError/256-10      281µs ± 2%     278µs ± 4%      ~     (p=0.053 n=9+10)
MarshalBytesError/4096-10     290µs ± 1%     279µs ± 3%    -3.52%  (p=0.000 n=10+10)

name                       old speed      new speed      delta
CodeEncoderError-10        2.83GB/s ± 8%  3.84GB/s ±20%   +36.03%  (p=0.000 n=10+10)
CodeMarshalError-10        2.60GB/s ± 5%  3.56GB/s ± 4%   +36.61%  (p=0.000 n=10+10)

name                       old alloc/op   new alloc/op   delta
CodeEncoderError-10          4.05MB ± 1%    0.00MB ± 1%  -100.00%  (p=0.000 n=10+9)
CodeMarshalError-10          6.05MB ± 0%    1.99MB ± 1%   -67.13%  (p=0.000 n=10+10)
MarshalBytesError/32-10      66.0kB ± 0%     0.2kB ± 0%   -99.67%  (p=0.000 n=9+8)
MarshalBytesError/256-10     50.1kB ± 0%     0.9kB ± 0%   -98.23%  (p=0.000 n=9+9)
MarshalBytesError/4096-10    87.4kB ± 0%     7.5kB ± 0%   -91.47%  (p=0.000 n=8+10)

name                       old allocs/op  new allocs/op  delta
CodeEncoderError-10            25.0 ± 0%       4.0 ± 0%   -84.00%  (p=0.000 n=9+10)
CodeMarshalError-10            27.0 ± 0%       6.0 ± 0%   -77.78%  (p=0.000 n=10+10)
MarshalBytesError/32-10        18.0 ± 0%       5.0 ± 0%   -72.22%  (p=0.000 n=10+10)
MarshalBytesError/256-10       17.0 ± 0%       6.0 ± 0%   -64.71%  (p=0.000 n=10+10)
MarshalBytesError/4096-10      16.0 ± 0%       6.0 ± 0%   -62.50%  (p=0.000 n=10+10)

Change-Id: I48070bb05f55707251c694e40d2570403bbf61f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/423694
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-23 20:23:19 +00:00
Axel Wagner 7f632f76db encoding/xml: add (*Encoder).Close
Flush can not check for unclosed elements, as more data might be encoded
after Flush is called. Close implicitly calls Flush and also checks that
all opened elements are closed as well.

Fixes #53346

Change-Id: I889b9f5ae54e5dfabb9e6948d96c5ed7bc1110f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/424777
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
2022-08-23 18:24:30 +00:00
Joe Tsai 0f42e35fee encoding/json: rely on utf8.AppendRune
Change-Id: I50e5609ff9c5f2b216b93cec7fb5214d196cae90
Reviewed-on: https://go-review.googlesource.com/c/go/+/412537
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-08-22 23:36:38 +00:00
Ian Lance Taylor dee9adc0f7 encoding/gob: use saferio to read large buffer
Avoid allocating large amounts of memory for corrupt input.

No test case because the problem can only happen for invalid data.
Let the fuzzer find cases like this.

Fixes #53369

Change-Id: I67c5e75bf181ad84988d6d6da12507df0e6df8e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/413979
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-19 16:03:42 +00:00
ErikPelli e1b62efaf3 encoding/base64: optimize decodemap memory set
The existing implementation has an execution time higher in the benchmark than this one.
This is an optimized implementation using the copy() function and a constant 256 bytes string with the values to be copied.

```
name           old time/op    new time/op    delta
NewEncoding-4     329ns ± 1%     231ns ± 1%  -29.72%  (p=0.008 n=5+5)

name           old speed      new speed      delta
NewEncoding-4   778MB/s ± 1%  1108MB/s ± 1%  +42.29%  (p=0.008 n=5+5)
```

Fixes #53211

Change-Id: I80fe62aa40623125ef81ae9164a8405eed30b71b
GitHub-Last-Rev: 55dce6f636
GitHub-Pull-Request: golang/go#53212
Reviewed-on: https://go-review.googlesource.com/c/go/+/410194
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-08-17 03:15:44 +00:00
Ludi Rehak d39b54171a encoding/gob: change typeInfo.encoder type to atomic.Pointer[T]
Replace loading and storing an atomic.Value of type pointer with
atomic.Pointer.

Change-Id: I018ac1e18eee4f203ebca65c2833daf991075371
Reviewed-on: https://go-review.googlesource.com/c/go/+/422174
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
2022-08-09 21:26:45 +00:00
Daniel Martí beebd537ca encoding/xml: use bytes.Buffer.WriteString
Change-Id: Icb0b917c48252ed8831e6c07109c1bc2d2438a42
Reviewed-on: https://go-review.googlesource.com/c/go/+/421234
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: hopehook <hopehook@qq.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dan Kortschak <dan@kortschak.io>
2022-08-09 18:32:01 +00:00
cuiweixie 9903ab5469 encoding/binary: ReadUvarint return io.ErrUnexpectedEOF when read at least 1 byte
Fixes #54139

Change-Id: Ifc73bd7f181b13970ee6a08968f9d8f6e55d7ff3
GitHub-Last-Rev: 1e0a79bd3e
GitHub-Pull-Request: golang/go#54143
Reviewed-on: https://go-review.googlesource.com/c/go/+/420274
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Keith Randall <khr@google.com>
2022-08-08 17:29:13 +00:00
Tomasz Jezierski 0f6ee42fe0 encoding/gob: replace runtime values with constants in init()
Current init() implementation in `encoding/gob/decode.go` checks int/uint/uintptr bit size with reflection in runtime. We could replace it with values available on compile stage. This should reduce time and allocations on binary start.
Results from GODEBUG=inittrace=1:
before:
init encoding/gob @4.4 ms, 0.21 ms clock, 43496 bytes, 652 allocs
after:
init encoding/gob @4.4 ms, 0.15 ms clock, 41672 bytes, 643 allocs

Updates #54184

Change-Id: I46dda2682fb92519da199415e29401d61edce697
Reviewed-on: https://go-review.googlesource.com/c/go/+/420455
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-08-08 06:16:59 +00:00
Ian Lance Taylor 4651ebf961 encoding/gob: s/TestIngoreDepthLimit/TestIgnoreDepthLimit/
For #53615

Change-Id: Ib85004d400931094fc1ea933cf73f4a5157aece1
Reviewed-on: https://go-review.googlesource.com/c/go/+/417559
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-07-15 14:20:24 +00:00
Roland Shoemaker 783ff7dfc4 encoding/xml: skip TestCVE202230633 for short tests
TestCVE202230633 uses a bunch of memory, and the input cannot be
feasibly reduced while maintaining the behavior hasn't regressed. This
test could be reasonably removed, but I'd rather keep it around if we
can.

Fixes #53814

Change-Id: Ie8b3f306efd20b2d9c0fb73122c26351a55694c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/417655
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-07-14 21:03:14 +00:00
Roland Shoemaker 6fa37e98ea encoding/gob: add a depth limit for ignored fields
Enforce a nesting limit of 10,000 for ignored fields during decoding
of messages. This prevents the possibility of triggering stack
exhaustion.

Fixes #53615
Fixes CVE-2022-30635

Change-Id: I05103d06dd5ca3945fcba3c1f5d3b5a645e8fb0f
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1484771
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417064
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tatiana Bradley <tatiana@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-07-12 15:05:49 +00:00
Roland Shoemaker 08c46ed43d encoding/xml: use iterative Skip, rather than recursive
Prevents exhausting the stack limit in _incredibly_ deeply nested
structures.

Fixes #53614
Fixes CVE-2022-28131

Change-Id: I47db4595ce10cecc29fbd06afce7b299868599e6
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1419912
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417062
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-07-12 15:05:39 +00:00
Roland Shoemaker c4c1993fd2 encoding/xml: limit depth of nesting in unmarshal
Prevent exhausting the stack limit when unmarshalling extremely deeply
nested structures into nested types.

Fixes #53611
Fixes CVE-2022-30633

Change-Id: Ic6c5d41674c93cfc9a316135a408db9156d39c59
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1421319
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417061
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-07-12 15:05:33 +00:00
shaoliming 606c6c371a encoding/xml: check nil pointer in DecodeElement
Fixes #53350

Change-Id: Id5e1f4016db5f1d4349ee1a76a9dfe3aeae83cee
GitHub-Last-Rev: 45add12161
GitHub-Pull-Request: golang/go#53407
Reviewed-on: https://go-review.googlesource.com/c/go/+/412634
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
2022-06-22 16:23:59 +00:00
Lukas Joisten 46ab7a5c4f encoding/json: mention SyntaxError in Unmarshal doc comment
Change-Id: I71c9d9ef9d21a7ae9466d8c7b283fdfbba01f5a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/390734
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-01 22:59:44 +00:00
John Bampton a6f3f8d973 all: fix spelling
Change-Id: I68538a50c22b02cdb5aa2a889f9440fed7b94c54
GitHub-Last-Rev: aaac9e7834
GitHub-Pull-Request: golang/go#52944
Reviewed-on: https://go-review.googlesource.com/c/go/+/406835
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2022-05-17 19:51:29 +00:00
hopehook 3474cd4eee encoding/csv: add Reader.InputOffset method
Fixes #43401.

Change-Id: I2498e77e41d845130d95012bc8623bfb29c0dda1
Reviewed-on: https://go-review.googlesource.com/c/go/+/405675
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-05-14 04:25:13 +00:00
teivah 8a5845e4e3 encoding/base32: decoder output depends on chunking of underlying reader
After an analysis, I figured that a way to do it could be to check, after
the call to readEncodedData whether the decoder already saw the end or not.

Fixes #38657

Change-Id: I06fd718ea4ee6ded2cb26c2866b28581ad86e271
GitHub-Last-Rev: d0b7bb38e4
GitHub-Pull-Request: golang/go#52631
Reviewed-on: https://go-review.googlesource.com/c/go/+/403315
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-03 18:30:15 +00:00
Joe Tsai 35a92f92bd encoding/binary: add AppendVarint AppendUvarint
This adds a straight-forward implementation of the functionality.
A more performant version could be added that unrolls the loop
as is done in google.golang.org/protobuf/encoding/protowire,
but usages that demand high performance can use that package instead.

Fixes #51644

Change-Id: I9d3b615a60cdff47e5200e7e5d2276adf4c93783
Reviewed-on: https://go-review.googlesource.com/c/go/+/400176
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-15 01:19:37 +00:00
Julie Qiu 45c3387d77 encoding/pem: fix stack overflow in Decode
Previously, Decode called decodeError, a recursive function that was
prone to stack overflows when given a large PEM file containing errors.

Credit to Juho Nurminen of Mattermost who reported the error.

Fixes CVE-2022-24675
Fixes #51853

Change-Id: Iffe768be53c8ddc0036fea0671d290f8f797692c
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1391157
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Filippo Valsorda <valsorda@google.com>
(cherry picked from commit 794ea5e828010e8b68493b2fc6d2963263195a02)
Reviewed-on: https://go-review.googlesource.com/c/go/+/399820
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-12 15:19:32 +00:00
Russ Cox 19309779ac all: gofmt main repo
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]

Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.

For #51082.

Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-11 16:34:30 +00:00
Russ Cox 9839668b56 all: separate doc comment from //go: directives
A future change to gofmt will rewrite

	// Doc comment.
	//go:foo

to

	// Doc comment.
	//
	//go:foo

Apply that change preemptively to all comments (not necessarily just doc comments).

For #51082.

Change-Id: Iffe0285418d1e79d34526af3520b415a12203ca9
Reviewed-on: https://go-review.googlesource.com/c/go/+/384260
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-05 17:54:15 +00:00
Russ Cox 81431c7aa7 all: replace `` and '' with “ (U+201C) and ” (U+201D) in doc comments
go/doc in all its forms applies this replacement when rendering
the comments. We are considering formatting doc comments,
including doing this replacement as part of the formatting.
Apply it to our source files ahead of time.

For #51082.

Change-Id: Ifcc1f5861abb57c5d14e7d8c2102dfb31b7a3a19
Reviewed-on: https://go-review.googlesource.com/c/go/+/384262
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-05 17:52:29 +00:00
Russ Cox 690ac4071f all: remove trailing blank doc comment lines
A future change to gofmt will rewrite

	// Doc comment.
	//
	func f()

to

	// Doc comment.
	func f()

Apply that change preemptively to all doc comments.

For #51082.

Change-Id: I4023e16cfb0729b64a8590f071cd92f17343081d
Reviewed-on: https://go-review.googlesource.com/c/go/+/384259
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01 18:18:07 +00:00
Patrick Gundlach 0eea25159f encoding/xml: expose decoder line and column
The existing implementation of the xml decoder uses the line number
only for reporting syntax errors. The line number of the last read
token and the column within the line is useful for the users even
in non-error conditions.

Fixes #45628

Change-Id: I37b5033ff5ff8411793d8f5180f96aa4537e83f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/311270
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2022-03-27 20:26:46 +00:00
eh-steve baf61e4a67 encoding/hex: implement Decode with a lookup table
Implement hex decode using a 256 byte lookup table instead of branching logic.

In happy flow, uses 3x 64 byte (or 5x 32 byte) cache lines.

	name             old time/op    new time/op    delta
	Decode/256-64       223ns ± 3%     135ns ± 2%  -39.64%  (p=0.000 n=8+8)
	Decode/1024-64      872ns ± 2%     512ns ± 2%  -41.25%  (p=0.000 n=8+8)
	Decode/4096-64     3.43µs ± 1%    2.01µs ± 2%  -41.31%  (p=0.001 n=7+7)
	Decode/16384-64    13.9µs ± 1%     8.0µs ± 1%  -42.69%  (p=0.000 n=8+7)

	name             old speed      new speed      delta
	Decode/256-64    1.15GB/s ± 3%  1.90GB/s ± 2%  +65.66%  (p=0.000 n=8+8)
	Decode/1024-64   1.17GB/s ± 2%  2.00GB/s ± 2%  +70.22%  (p=0.000 n=8+8)
	Decode/4096-64   1.20GB/s ± 1%  2.04GB/s ± 2%  +70.39%  (p=0.001 n=7+7)
	Decode/16384-64  1.18GB/s ± 1%  2.06GB/s ± 1%  +74.49%  (p=0.000 n=8+7)

Also reduces amd64 object size by 766 bytes, despite the extra RODATA due to removal of `fromHexChar()` and duplicated inlined versions of it and simplification of `Decode()`.

Change-Id: I0988c7a30562ec154eff11db6e27954e0ce2b611
GitHub-Last-Rev: 64818018af
GitHub-Pull-Request: golang/go#51432
Reviewed-on: https://go-review.googlesource.com/c/go/+/390037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-11 21:51:20 +00:00
Iskander Sharipov e79c39f004 encoding/xml: improve the test coverage, fix minor bugs
Improve the test coverage of encoding/xml package by adding
the test cases for the execution paths that were not covered before.

Since it reveals a couple of issues, fix them as well while we're at it.

As I used an `strings.EqualFold` instead of adding one more `strings.ToLower`,
our fix to `autoClose()` tends to run faster as well as a result.

	name             old time/op    new time/op    delta
	HTMLAutoClose-8    5.93µs ± 2%    5.75µs ± 3%  -3.16%  (p=0.000 n=10+10)
	name             old alloc/op   new alloc/op   delta
	HTMLAutoClose-8    2.60kB ± 0%    2.58kB ± 0%  -0.46%  (p=0.000 n=10+10)
	name             old allocs/op  new allocs/op  delta
	HTMLAutoClose-8      72.0 ± 0%      67.0 ± 0%  -6.94%  (p=0.000 n=10+10)

The overall `encoding/xml` test coverage increase is `88.1% -> 89.9%`;
although it may look insignificant, this CL covers some important corner cases,
like `autoClose()` functionality (that was not tested at all).

Fixes #49635
Fixes #49636

Change-Id: I50b2769896c197eb285672313b7148f4fe8bdb38
Reviewed-on: https://go-review.googlesource.com/c/go/+/364734
Trust: Bryan Mills <bcmills@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-04 20:29:47 +00:00
Joe Tsai 986b04c0f1 encoding/binary: add AppendByteOrder
AppendByteOrder specifies new methods for LittleEndian and BigEndian
for appending an unsigned integer to a byte slice.

The performance of AppendXXX methods are slower than PutXXX methods
since the former needs to do a few slice operations,
while the latter is essentially a single integer store.
In practice, existing usages of PutXXX performed slicing operations
around the call such that this cost was present, regardless.

name                           time/op
PutUint16-24                   0.48ns ± 2%
AppendUint16-24                1.54ns ± 1%
PutUint32-24                   0.46ns ± 2%
AppendUint32-24                0.89ns ± 1%
PutUint64-24                   0.46ns ± 2%
AppendUint64-24                0.89ns ± 1%
LittleEndianPutUint16-24       0.47ns ± 2%
LittleEndianAppendUint16-24    1.54ns ± 1%
LittleEndianPutUint32-24       0.45ns ± 3%
LittleEndianAppendUint32-24    0.92ns ± 2%
LittleEndianPutUint64-24       0.46ns ± 3%
LittleEndianAppendUint64-24    0.95ns ± 4%

Fixes #50601

Change-Id: I33d2bbc93a3ce01a9269feac33a2432bc1166ead
Reviewed-on: https://go-review.googlesource.com/c/go/+/386017
Trust: Joseph Tsai <joetsai@digital-static.net>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-02 18:21:38 +00:00
Joe Tsai 5a03cbd12a encoding/json: use reflect.Value.UnsafePointer over Pointer
The latter returns a uintptr, while the former returns a unsafe.Pointer.
A uintptr is unsafe if Go ever switches to a moving GC,
while a unsafe.Pointer will be properly tracked by the GC.

We do not use unsafe.Pointer for any unsafe type conversions,
and only use it for comparability purposes, which is relatively safe.

Updates #40592

Change-Id: I813e218668704b63a3043acda4331205a3835a66
Reviewed-on: https://go-review.googlesource.com/c/go/+/360855
Trust: Joseph Tsai <joetsai@digital-static.net>
Trust: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-02 17:42:53 +00:00
hopehook 851ecea4cc encoding/xml: embedded reference to substruct causes XML marshaller to panic on encoding
When encoding a xml attribute is zero value (IsValid == false), we need
a `continue` to jump over the attribute. If not, followed marshalAttr
function will panic.

Fixes: #50164
Change-Id: I42e064558e7becfbf47728b14cbf5c7afa1e8798
Reviewed-on: https://go-review.googlesource.com/c/go/+/385514
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-02-20 20:58:11 +00:00
Roland Shoemaker 4fa6e33f30 all: add a handful of fuzz targets
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip,
encoding/json, image/jpeg, image/gif, and image/png.

Second attempt, this time we don't use the archives in testdata when
fuzzing archive/tar, since those are rather memory intensive, and
were crashing a number of builders.

Change-Id: I4828d64fa4763c0d8c980392a6578e4dfd956e13
Reviewed-on: https://go-review.googlesource.com/c/go/+/378174
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-13 18:06:33 +00:00
Bryan Mills 6e8b7e4f42 Revert "all: add a handful of fuzz targets"
This reverts CL 352109.

Reason for revert: causing OOM failures on several builders, and may cause OOMs for end users with small machines as well.

Change-Id: I58308d09919969d5a6512ee5cee6aa5c4af6769b
Reviewed-on: https://go-review.googlesource.com/c/go/+/377934
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
2022-01-12 15:53:47 +00:00
Roland Shoemaker 9bce08999a all: add a handful of fuzz targets
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip,
encoding/json, image/jpeg, image/gif, and image/png.

Change-Id: Ide1a8de88a9421e786eeeaea3bb93f41e0bae347
Reviewed-on: https://go-review.googlesource.com/c/go/+/352109
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-12 00:12:10 +00:00
Russ Cox 2580d0e08d all: gofmt -w -r 'interface{} -> any' src
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.

Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.

A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.

Fixes #49884.

Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13 18:45:54 +00:00
Keith Randall b93220c9ca encoding/xml: add generic encoding test
Fixes #48521

Change-Id: Id8402bcff243c0ab19e4ec0b138b9af8c111f88d
Reviewed-on: https://go-review.googlesource.com/c/go/+/355492
Trust: Keith Randall <khr@golang.org>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2021-11-09 21:26:25 +00:00
Josh Bleecher Snyder 65f3478633 encoding: simplify some code
Change-Id: I63aac663da4ce7bdd198d385f87929c74f6a5cf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/360601
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-11-02 03:53:23 +00:00
Russ Cox f229e7031a all: go fix -fix=buildtag std cmd (except for bootstrap deps, vendor)
When these packages are released as part of Go 1.18,
Go 1.16 will no longer be supported, so we can remove
the +build tags in these files.

Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs
as defined in src/cmd/dist/buildtool.go, which need to continue
to build with Go 1.4 for now.

Also reverted src/vendor and src/cmd/vendor, which will need
to be updated in their own repos first.

Manual changes in runtime/pprof/mprof_test.go to adjust line numbers.

For #41184.

Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e
Reviewed-on: https://go-review.googlesource.com/c/go/+/344955
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 18:17:57 +00:00
Cuong Manh Le 283d8a3d53 all: use reflect.{Pointer,PointerTo}
Updates #47651
Updates #48665

Change-Id: I69a87b45a5cad7a07fbd855040cd9935cf874554
Reviewed-on: https://go-review.googlesource.com/c/go/+/358454
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-10-26 14:24:17 +00:00
180909 1e49210ccd encoding/base64: add examples for Encode/Decode
Fixes #37595

Change-Id: I83e5f6105748a0a9238322a4f7ec4b0bbf61a263
Reviewed-on: https://go-review.googlesource.com/c/go/+/348394
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Daniel Martí <mvdan@mvdan.cc>
Trust: Cherry Mui <cherryyz@google.com>
2021-10-19 08:44:22 +00:00
Daniel Martí d973bb107e encoding/gob: follow documented io.EOF semantics
The docs say:

	If the input is at EOF, Decode returns io.EOF and does not modify e.

However, the added test fails:

	--- FAIL: TestDecodePartial (0.00s)
		encoder_test.go:1263: 31/81: expected io.ErrUnexpectedEOF: EOF
		encoder_test.go:1263: 51/81: expected io.ErrUnexpectedEOF: EOF

In particular, the decoder would return io.EOF after reading a valid
message for a type specification, and then hit EOF before reading a data
item message.

Fix that by only allowing a Decode call to return io.EOF if the reader
hits EOF immediately, without successfully reading any message.
Otherwise, hitting EOF is an ErrUnexpectedEOF, like in other cases.

Also fix a net/rpc test that, coincidentally, expected an io.EOF
as an error when feeding bad non-zero data to a gob decoder.
An io.ErrUnexpectedEOF is clearly better in that scenario.

Fixes #48905.

Change-Id: Ied6a0d8ac8377f89646319a18c0380c4f2b09b85
Reviewed-on: https://go-review.googlesource.com/c/go/+/354972
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2021-10-11 21:58:33 +00:00
Russ Cox 4d8db00641 all: use bytes.Cut, strings.Cut
Many uses of Index/IndexByte/IndexRune/Split/SplitN
can be written more clearly using the new Cut functions.
Do that. Also rewrite to other functions if that's clearer.

For #46336.

Change-Id: I68d024716ace41a57a8bf74455c62279bde0f448
Reviewed-on: https://go-review.googlesource.com/c/go/+/351711
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-06 15:53:04 +00:00
jiahua wang f9a53b6b4d encoding/base32: Add examples for Encode/Decode
Updates golang/go#37595

Change-Id: I7568e7416d5504e9dc67061c79f66e3a0d597dee
Reviewed-on: https://go-review.googlesource.com/c/go/+/351470
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: David Chase <drchase@google.com>
2021-09-27 16:54:36 +00:00