Commit Graph

601 Commits

Author SHA1 Message Date
Joe Tsai f02cdba163 encoding/hex: fix Decode output check regression
CL 461958 fixed a potential panic,
but also introduced an observable regression where
invalid input could be detected before the panic occurs.
Adjust the check to preserve prior behavior,
while also preventing the panic.

Change-Id: I52819f88a6a64883fbc9fd512697c38c29ca0ccd
Reviewed-on: https://go-review.googlesource.com/c/go/+/465855
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-02-07 15:37:55 +00:00
Constantin Konstantinidis dfd2ddd671 encoding/xml: allow overriding by empty namespace
The namespace defined by xmlns="value" can be overridden in every included tag
by the empty namespace xmlns="" without a prefix.

Method to calculate indent of XML handles depth of tag and its associated namespace is
still active even when no indent is required.

An XMLName field in a struct means that namespace must be enforced even if empty.
This occurs only on an inner tag as an override of any non-empty namespace of its outer tag.
An attribute is added to have the required namespace display.

Fixes #7113

Change-Id: I57f2308e98c66f04108ab136d350bdc3a6091e98
Reviewed-on: https://go-review.googlesource.com/c/go/+/108796
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-01-31 18:23:50 +00:00
cuiweixie 848d658c45 encoding/binary: add String and GoString method to nativeEndian
Updates #57237

Change-Id: Ib626610130cae9c1d1aff5dd2a5035ffde0e127f
Reviewed-on: https://go-review.googlesource.com/c/go/+/463985
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: xie cui <523516579@qq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-30 18:51:09 +00:00
cui fliter dbfdc446fe all: fix problematic comments
Change-Id: If092ae7c72b66f172ae32fa6c7294a7ac250362e
Reviewed-on: https://go-review.googlesource.com/c/go/+/463995
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
2023-01-30 18:10:36 +00:00
cuiweixie 4b3726e99b encoding/binary: add var NativeEndian
Updates #57237

Change-Id: I149c8b7eeac91b87b5810250f96d48ca87135807
Reviewed-on: https://go-review.googlesource.com/c/go/+/463218
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: xie cui <523516579@qq.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-01-27 18:17:20 +00:00
Benjamin Prosnitz a106defdda hex: fix panic in Decode when len(src) > 2*len(dst)
hex.Decode never checks the length of dst and triggers a panic
if there are insufficient bytes in the slice.

There isn't document on what the behavior *should* be in this case.
Two possibilities:
1. Error dst has insufficient space (as done in this change)
2. Reduce the length of the decode to min(dst, src)

Option 1 was chosen because it seems the least surprising or
subtle.

Change-Id: I3bf029e3d928202de716830434285e3c165f26dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/461958
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Benjamin Prosnitz <bprosnitz@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2023-01-27 15:28:30 +00:00
Daniel Martí 43f9b826c3 encoding/gob: slightly simplify init code
https://go.dev/cl/460543 stopped using the "expect" parameter in
bootstrapType, but we forgot to actually remove it.

While here, staticcheck correctly points out that we can use the copy
builtin to fill builtinIdToTypeSlice, now that it and idToType are an
array and slice respectively.

Change-Id: I48078415ab9bdd5633cf41f33ab4dc78eb30b48a
Reviewed-on: https://go-review.googlesource.com/c/go/+/462301
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Rob Pike <r@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2023-01-24 20:07:49 +00:00
Daniel Martí 8259ac4986 encoding/gob: shave off some init time cost
Avoid unnecessary allocations when calling reflect.TypeOf;
we can use nil pointers, which fit into an interface without allocating.
This saves about 1% of CPU time.

The builtin types are limited to typeIds between 0 and firstUserId,
and since firstUserId is 64, builtinIdToType does not need to be a map.
We can simply use an array of length firstUserId, which is simpler.
This saves about 1% of CPU time.

idToType is similar to firstUserId in that it is a map keyed by typeIds.
The difference is that it can grow with the user's types.
However, each added type gets the next available typeId,
meaning that we can use a growing slice, similar to the case above.
nextId then becomes the current length of the slice.
This saves about 1% of CPU time.

typeInfoMap is stored globally as an atomic.Value,
where each modification loads the map, makes a whole copy,
adds the new element, and stores the modified copy.
This is perfectly fine when the user registers types,
as that can happen concurrently and at any point in the future.

However, during init time, we sequentially register many types,
and the overhead of copying maps adds up noticeably.
During init time, use a regular global map instead,
which gets replaced by the atomic.Value when our init work is done.
This saves about 2% of CPU time.

Finally, avoid calling checkId in bootstrapType;
we have just called setTypeId, whose logic for getting nextId is simple,
so the extra check doesn't gain us much.
This saves about 1% of CPU time.

Using benchinit, which transforms GODEBUG=inittrace=1 data into Go
benchmark compatible output, results in a nice improvement:

	name         old time/op    new time/op    delta
	EncodingGob     175µs ± 0%     162µs ± 0%  -7.45%  (p=0.016 n=5+4)

	name         old alloc/op   new alloc/op   delta
	EncodingGob    39.0kB ± 0%    36.1kB ± 0%  -7.35%  (p=0.016 n=5+4)

	name         old allocs/op  new allocs/op  delta
	EncodingGob       588 ± 0%       558 ± 0%  -5.10%  (p=0.000 n=5+4)

Updates #26775.

Change-Id: I28618e8b96ef440480e666ef2cd5c4a9a332ef21
Reviewed-on: https://go-review.googlesource.com/c/go/+/460543
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-20 09:34:41 +00:00
Russ Cox db4be8f510 encoding/csv: use proper doc comment for Deprecated notes
End-of-line comments are not doc comments,
so Deprecated notes in them are not recognized
as deprecation notices. Rewrite the comments.

Change-Id: I275fa9aec403132fda45853e52daa22bc06fcd36
Reviewed-on: https://go-review.googlesource.com/c/go/+/453617
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-12-02 16:30:23 +00:00
Ian Lance Taylor 483f3dc39f Revert "encoding/xml: disallow empty namespace when prefix is set"
This reverts CL 105636.

Reason for revert: Fails with existing XML data.  At this stage in the release cycle we should revert, and try again next time with some way to support existing XML.

For #8068

Change-Id: Ia84cbf3a84878ac7190f72998545dee22c36c45e
Reviewed-on: https://go-review.googlesource.com/c/go/+/453996
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2022-11-29 22:25:08 +00:00
cui fliter b2faff18ce all: add missing periods in comments
Change-Id: I69065f8adf101fdb28682c55997f503013a50e29
Reviewed-on: https://go-review.googlesource.com/c/go/+/449757
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joedian Reid <joedian@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-18 17:59:44 +00:00
Constantin Konstantinidis 89332e037a encoding/xml: error when more than one colon in qualified names
Add test.

Fixes #20396

Change-Id: I89e9013eb338f831e1908e390b284794df78fb6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/103875
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-11-09 22:56:44 +00:00
Constantin Konstantinidis bef5eca118 encoding/xml: disallow empty namespace when prefix is set
Non-regression tests are added.

Fixes #8068

Change-Id: Icb36c910bbf4955743b7aa8382002b2d9246fadc
Reviewed-on: https://go-review.googlesource.com/c/go/+/105636
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-11-09 22:45:43 +00:00
Constantin Konstantinidis f13849a7af encoding/xml: error when closing tag does not match opening tag
Comparing opening and closing tag is done using the prefix when available.
Documentation states that Token returns URI in the Space part of the Name.
Translation has been moved for the End tag before the namespace is removed
from the stack.

After closing a tag using a namespace, the valid namespace must be taken
from the opening tag. Tests added.

Fixes #20685

Change-Id: I4d90b19f7e21a76663f0ea1c1db6c6bf9fd2a389
Reviewed-on: https://go-review.googlesource.com/c/go/+/107255
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-11-09 03:01:28 +00:00
Constantin Konstantinidis b459166219 encoding/xml: add check of namespaces to detect field names conflicts
Test added.

Fixes #8535

Change-Id: Ic89c2781e81d963a653180812748b3fc95fb7fae
Reviewed-on: https://go-review.googlesource.com/c/go/+/106575
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-11-09 03:01:25 +00:00
Dan Kortschak 1c05968c9a all: fix comment typos
Change-Id: Ic16824482142d4de4d0b949459e36505ee944ff7
Reviewed-on: https://go-review.googlesource.com/c/go/+/448175
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Dan Kortschak <dan@kortschak.io>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dan Kortschak <dan@kortschak.io>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-05 21:02:45 +00:00
Russ Cox 6695cebeec encoding/xml: reduce depth limit on wasm
Wasm can't handle the recusion for XML nested to depth 10,000.
Cut it off at 5,000 instead. This fixes TestCVE202228131 on trybots
in certain conditions.

Also disable TestCVE202230633 to fix 'go test -v encoding/xml' on gomotes.

Also rename errExeceededMaxUnmarshalDepth [misspelled and unwieldy]
to errUnmarshalDepth.

For #56498.

Change-Id: I7cc337ccfee251bfd9771497be0e5272737114f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/446639
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-31 20:35:56 +00:00
Ian Lance Taylor 4c61e079c0 encoding/gob: support large slices in slice decode helpers
The slice decode helpers weren't aware of partially allocated slices.

Also add large slice support for []byte.

Change-Id: I5044587e917508887c7721f8059d364189831693
Reviewed-on: https://go-review.googlesource.com/c/go/+/443777
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-20 23:11:47 +00:00
Roland Shoemaker 36ca37f3a0 encoding/gob: note pacakge not covered by security policy
And add a link. Resolves a comment left on http://go.dev/cl/436096
after it was submitted.

Change-Id: I2847d29134ffb4fee2b0ea37842cdf57df55ec0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/442816
Reviewed-by: Julie Qiu <julieqiu@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-13 18:26:45 +00:00
cui fliter b314eea4cd all: remove redundant type conversion
Change-Id: I375233dc700adbc58a6d4af995d07b352bf85b11
GitHub-Last-Rev: ef12920523
GitHub-Pull-Request: golang/go#55994
Reviewed-on: https://go-review.googlesource.com/c/go/+/437715
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2022-10-06 04:37:00 +00:00
Tomasz Jezierski 9c2fd81ee1 encoding/asn1: remove allocation from init
asn1 allocates due to reflect.TypeOf(new(big.Int)) in init time.
We could replace it with (*big.Int)(nil).

Before:
init encoding/asn1 @1.0 ms, 0.009 ms clock, 224 bytes, 7 allocs

After:
init encoding/asn1 @0.70 ms, 0.002 ms clock, 192 bytes, 6 allocs

Fixes #55973

Change-Id: I7c3cc0f48631af73cf34ad3c731c380f46c72359
Reviewed-on: https://go-review.googlesource.com/c/go/+/435257
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: hopehook <hopehook@golangcn.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-10-04 23:29:48 +00:00
Daniel Martí cbd931c9c2 encoding/gob: prevent a decoder state overflow
When decoding a struct, if a positive delta is large enough to overflow
when added to fieldnum, we would panic due to the resulting negative index.

Instead, catch this problem and produce an error like we do with
negative delta integers. If fieldnum ends up being negative or smaller
than state.fieldnum, the addition overflowed.

While here, remove an unnecessary break after an error call,
since those error functions cause a panic.

Fixes #55337.

Change-Id: I7a0e4f43e5c81a703e79c1597e3bb3714cc832c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/432715
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-04 13:27:35 +00:00
cui fliter aeab76f0be encoding: fix a few function names on comments
Change-Id: I17a311afb94a056b3d35bfa241f5d0d206db602d
GitHub-Last-Rev: 42129464c9
GitHub-Pull-Request: golang/go#55962
Reviewed-on: https://go-review.googlesource.com/c/go/+/436882
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
2022-09-30 23:03:07 +00:00
cuiweixie 1e65fa58c1 encoding/json: return comparison directly
Change-Id: I4698d0fa78108d83ee91732e8d3878dbff7f9c90
Reviewed-on: https://go-review.googlesource.com/c/go/+/436711
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: xie cui <523516579@qq.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2022-09-30 20:10:56 +00:00
hopehook 972496ae6e all: use strings.Builder where appropriate
Change-Id: I164d350ca480640996055dedf38d962921c474a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/435975
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-29 22:56:00 +00:00
Roland Shoemaker 13d48bb6a1 encoding/gob: add top level security doc
Add a slightly expanded version of the Decoder type comment to the top
level package doc, which explains that this package is not designed
to be hardened against adversarial inputs.

Change-Id: I8b83433838c8235eb06ded99041fdf726c811ee5
Reviewed-on: https://go-review.googlesource.com/c/go/+/436096
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2022-09-29 00:30:21 +00:00
cuiweixie 09919ac339 encoding/xml: use bytes.Clone
Change-Id: I3218b1e3f8869f579facddb29471df13c835dc66
Reviewed-on: https://go-review.googlesource.com/c/go/+/435281
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2022-09-28 03:44:25 +00:00
Ian Lance Taylor a0441c7ae3 encoding/gob: use saferio.SliceCap when decoding a slice
This avoids allocating an overly large slice for corrupt input.

Change the saferio.SliceCap function to take a pointer to the element type,
so that we can handle slices of interface types. This revealed that a
couple of existing calls were actually incorrect, passing the slice type
rather than the element type.

No test case because the problem can only happen for invalid data. Let
the fuzzer find cases like this.

Fixes #55338

Change-Id: I3c1724183cc275d4981379773b0b8faa01a9cbd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/433296
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-25 01:18:43 +00:00
cuiweixie ab87db8b46 encoding: use strings.Builder
Change-Id: Ib07699df8ea41fd8d1bca8ad050859fac24623de
Reviewed-on: https://go-review.googlesource.com/c/go/+/428258
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-06 15:46:20 +00:00
RenKanai 202b7e7e76 encoding/asn1: fix doc for BitString.At's return value
Fixes #53287

Change-Id: If983ae34850d9b1b29764156a38628fa53897573
Reviewed-on: https://go-review.googlesource.com/c/go/+/411134
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: hopehook <hopehook@golangcn.org>
Reviewed-by: Subham <sarkar.subhams2@gmail.com>
2022-09-02 09:14:48 +00:00
Andy Pan 1a8dfadbfe encoding/json: move some misplaced benchmark tests to bench_test.go
Change-Id: I5987eed00ee825421abe62699a06e9b66499f35f
Reviewed-on: https://go-review.googlesource.com/c/go/+/425016
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-24 05:40:28 +00:00
Andy Pan e7f2e5697a encoding/json: give it a chance to put encodeState back in pool when error occurs
name                       old time/op    new time/op    delta
CodeEncoderError-10           688µs ± 8%     496µs ±15%   -27.92%  (p=0.000 n=10+9)
CodeMarshalError-10           747µs ± 6%     546µs ± 4%   -26.86%  (p=0.000 n=10+10)
MarshalBytesError/32-10       284µs ± 2%     273µs ± 1%    -3.84%  (p=0.000 n=10+10)
MarshalBytesError/256-10      281µs ± 2%     278µs ± 4%      ~     (p=0.053 n=9+10)
MarshalBytesError/4096-10     290µs ± 1%     279µs ± 3%    -3.52%  (p=0.000 n=10+10)

name                       old speed      new speed      delta
CodeEncoderError-10        2.83GB/s ± 8%  3.84GB/s ±20%   +36.03%  (p=0.000 n=10+10)
CodeMarshalError-10        2.60GB/s ± 5%  3.56GB/s ± 4%   +36.61%  (p=0.000 n=10+10)

name                       old alloc/op   new alloc/op   delta
CodeEncoderError-10          4.05MB ± 1%    0.00MB ± 1%  -100.00%  (p=0.000 n=10+9)
CodeMarshalError-10          6.05MB ± 0%    1.99MB ± 1%   -67.13%  (p=0.000 n=10+10)
MarshalBytesError/32-10      66.0kB ± 0%     0.2kB ± 0%   -99.67%  (p=0.000 n=9+8)
MarshalBytesError/256-10     50.1kB ± 0%     0.9kB ± 0%   -98.23%  (p=0.000 n=9+9)
MarshalBytesError/4096-10    87.4kB ± 0%     7.5kB ± 0%   -91.47%  (p=0.000 n=8+10)

name                       old allocs/op  new allocs/op  delta
CodeEncoderError-10            25.0 ± 0%       4.0 ± 0%   -84.00%  (p=0.000 n=9+10)
CodeMarshalError-10            27.0 ± 0%       6.0 ± 0%   -77.78%  (p=0.000 n=10+10)
MarshalBytesError/32-10        18.0 ± 0%       5.0 ± 0%   -72.22%  (p=0.000 n=10+10)
MarshalBytesError/256-10       17.0 ± 0%       6.0 ± 0%   -64.71%  (p=0.000 n=10+10)
MarshalBytesError/4096-10      16.0 ± 0%       6.0 ± 0%   -62.50%  (p=0.000 n=10+10)

Change-Id: I48070bb05f55707251c694e40d2570403bbf61f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/423694
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-23 20:23:19 +00:00
Axel Wagner 7f632f76db encoding/xml: add (*Encoder).Close
Flush can not check for unclosed elements, as more data might be encoded
after Flush is called. Close implicitly calls Flush and also checks that
all opened elements are closed as well.

Fixes #53346

Change-Id: I889b9f5ae54e5dfabb9e6948d96c5ed7bc1110f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/424777
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
2022-08-23 18:24:30 +00:00
Joe Tsai 0f42e35fee encoding/json: rely on utf8.AppendRune
Change-Id: I50e5609ff9c5f2b216b93cec7fb5214d196cae90
Reviewed-on: https://go-review.googlesource.com/c/go/+/412537
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-08-22 23:36:38 +00:00
Ian Lance Taylor dee9adc0f7 encoding/gob: use saferio to read large buffer
Avoid allocating large amounts of memory for corrupt input.

No test case because the problem can only happen for invalid data.
Let the fuzzer find cases like this.

Fixes #53369

Change-Id: I67c5e75bf181ad84988d6d6da12507df0e6df8e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/413979
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-19 16:03:42 +00:00
ErikPelli e1b62efaf3 encoding/base64: optimize decodemap memory set
The existing implementation has an execution time higher in the benchmark than this one.
This is an optimized implementation using the copy() function and a constant 256 bytes string with the values to be copied.

```
name           old time/op    new time/op    delta
NewEncoding-4     329ns ± 1%     231ns ± 1%  -29.72%  (p=0.008 n=5+5)

name           old speed      new speed      delta
NewEncoding-4   778MB/s ± 1%  1108MB/s ± 1%  +42.29%  (p=0.008 n=5+5)
```

Fixes #53211

Change-Id: I80fe62aa40623125ef81ae9164a8405eed30b71b
GitHub-Last-Rev: 55dce6f636
GitHub-Pull-Request: golang/go#53212
Reviewed-on: https://go-review.googlesource.com/c/go/+/410194
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-08-17 03:15:44 +00:00
Ludi Rehak d39b54171a encoding/gob: change typeInfo.encoder type to atomic.Pointer[T]
Replace loading and storing an atomic.Value of type pointer with
atomic.Pointer.

Change-Id: I018ac1e18eee4f203ebca65c2833daf991075371
Reviewed-on: https://go-review.googlesource.com/c/go/+/422174
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
2022-08-09 21:26:45 +00:00
Daniel Martí beebd537ca encoding/xml: use bytes.Buffer.WriteString
Change-Id: Icb0b917c48252ed8831e6c07109c1bc2d2438a42
Reviewed-on: https://go-review.googlesource.com/c/go/+/421234
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: hopehook <hopehook@qq.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dan Kortschak <dan@kortschak.io>
2022-08-09 18:32:01 +00:00
cuiweixie 9903ab5469 encoding/binary: ReadUvarint return io.ErrUnexpectedEOF when read at least 1 byte
Fixes #54139

Change-Id: Ifc73bd7f181b13970ee6a08968f9d8f6e55d7ff3
GitHub-Last-Rev: 1e0a79bd3e
GitHub-Pull-Request: golang/go#54143
Reviewed-on: https://go-review.googlesource.com/c/go/+/420274
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Keith Randall <khr@google.com>
2022-08-08 17:29:13 +00:00
Tomasz Jezierski 0f6ee42fe0 encoding/gob: replace runtime values with constants in init()
Current init() implementation in `encoding/gob/decode.go` checks int/uint/uintptr bit size with reflection in runtime. We could replace it with values available on compile stage. This should reduce time and allocations on binary start.
Results from GODEBUG=inittrace=1:
before:
init encoding/gob @4.4 ms, 0.21 ms clock, 43496 bytes, 652 allocs
after:
init encoding/gob @4.4 ms, 0.15 ms clock, 41672 bytes, 643 allocs

Updates #54184

Change-Id: I46dda2682fb92519da199415e29401d61edce697
Reviewed-on: https://go-review.googlesource.com/c/go/+/420455
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-08-08 06:16:59 +00:00
Ian Lance Taylor 4651ebf961 encoding/gob: s/TestIngoreDepthLimit/TestIgnoreDepthLimit/
For #53615

Change-Id: Ib85004d400931094fc1ea933cf73f4a5157aece1
Reviewed-on: https://go-review.googlesource.com/c/go/+/417559
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-07-15 14:20:24 +00:00
Roland Shoemaker 783ff7dfc4 encoding/xml: skip TestCVE202230633 for short tests
TestCVE202230633 uses a bunch of memory, and the input cannot be
feasibly reduced while maintaining the behavior hasn't regressed. This
test could be reasonably removed, but I'd rather keep it around if we
can.

Fixes #53814

Change-Id: Ie8b3f306efd20b2d9c0fb73122c26351a55694c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/417655
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-07-14 21:03:14 +00:00
Roland Shoemaker 6fa37e98ea encoding/gob: add a depth limit for ignored fields
Enforce a nesting limit of 10,000 for ignored fields during decoding
of messages. This prevents the possibility of triggering stack
exhaustion.

Fixes #53615
Fixes CVE-2022-30635

Change-Id: I05103d06dd5ca3945fcba3c1f5d3b5a645e8fb0f
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1484771
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417064
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tatiana Bradley <tatiana@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-07-12 15:05:49 +00:00
Roland Shoemaker 08c46ed43d encoding/xml: use iterative Skip, rather than recursive
Prevents exhausting the stack limit in _incredibly_ deeply nested
structures.

Fixes #53614
Fixes CVE-2022-28131

Change-Id: I47db4595ce10cecc29fbd06afce7b299868599e6
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1419912
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417062
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-07-12 15:05:39 +00:00
Roland Shoemaker c4c1993fd2 encoding/xml: limit depth of nesting in unmarshal
Prevent exhausting the stack limit when unmarshalling extremely deeply
nested structures into nested types.

Fixes #53611
Fixes CVE-2022-30633

Change-Id: Ic6c5d41674c93cfc9a316135a408db9156d39c59
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1421319
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417061
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-07-12 15:05:33 +00:00
shaoliming 606c6c371a encoding/xml: check nil pointer in DecodeElement
Fixes #53350

Change-Id: Id5e1f4016db5f1d4349ee1a76a9dfe3aeae83cee
GitHub-Last-Rev: 45add12161
GitHub-Pull-Request: golang/go#53407
Reviewed-on: https://go-review.googlesource.com/c/go/+/412634
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
2022-06-22 16:23:59 +00:00
Lukas Joisten 46ab7a5c4f encoding/json: mention SyntaxError in Unmarshal doc comment
Change-Id: I71c9d9ef9d21a7ae9466d8c7b283fdfbba01f5a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/390734
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-01 22:59:44 +00:00
John Bampton a6f3f8d973 all: fix spelling
Change-Id: I68538a50c22b02cdb5aa2a889f9440fed7b94c54
GitHub-Last-Rev: aaac9e7834
GitHub-Pull-Request: golang/go#52944
Reviewed-on: https://go-review.googlesource.com/c/go/+/406835
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2022-05-17 19:51:29 +00:00
hopehook 3474cd4eee encoding/csv: add Reader.InputOffset method
Fixes #43401.

Change-Id: I2498e77e41d845130d95012bc8623bfb29c0dda1
Reviewed-on: https://go-review.googlesource.com/c/go/+/405675
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-05-14 04:25:13 +00:00
teivah 8a5845e4e3 encoding/base32: decoder output depends on chunking of underlying reader
After an analysis, I figured that a way to do it could be to check, after
the call to readEncodedData whether the decoder already saw the end or not.

Fixes #38657

Change-Id: I06fd718ea4ee6ded2cb26c2866b28581ad86e271
GitHub-Last-Rev: d0b7bb38e4
GitHub-Pull-Request: golang/go#52631
Reviewed-on: https://go-review.googlesource.com/c/go/+/403315
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-03 18:30:15 +00:00