Commit Graph

582 Commits

Author SHA1 Message Date
cui fliter b314eea4cd all: remove redundant type conversion
Change-Id: I375233dc700adbc58a6d4af995d07b352bf85b11
GitHub-Last-Rev: ef12920523
GitHub-Pull-Request: golang/go#55994
Reviewed-on: https://go-review.googlesource.com/c/go/+/437715
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2022-10-06 04:37:00 +00:00
Tomasz Jezierski 9c2fd81ee1 encoding/asn1: remove allocation from init
asn1 allocates due to reflect.TypeOf(new(big.Int)) in init time.
We could replace it with (*big.Int)(nil).

Before:
init encoding/asn1 @1.0 ms, 0.009 ms clock, 224 bytes, 7 allocs

After:
init encoding/asn1 @0.70 ms, 0.002 ms clock, 192 bytes, 6 allocs

Fixes #55973

Change-Id: I7c3cc0f48631af73cf34ad3c731c380f46c72359
Reviewed-on: https://go-review.googlesource.com/c/go/+/435257
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: hopehook <hopehook@golangcn.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-10-04 23:29:48 +00:00
Daniel Martí cbd931c9c2 encoding/gob: prevent a decoder state overflow
When decoding a struct, if a positive delta is large enough to overflow
when added to fieldnum, we would panic due to the resulting negative index.

Instead, catch this problem and produce an error like we do with
negative delta integers. If fieldnum ends up being negative or smaller
than state.fieldnum, the addition overflowed.

While here, remove an unnecessary break after an error call,
since those error functions cause a panic.

Fixes #55337.

Change-Id: I7a0e4f43e5c81a703e79c1597e3bb3714cc832c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/432715
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-04 13:27:35 +00:00
cui fliter aeab76f0be encoding: fix a few function names on comments
Change-Id: I17a311afb94a056b3d35bfa241f5d0d206db602d
GitHub-Last-Rev: 42129464c9
GitHub-Pull-Request: golang/go#55962
Reviewed-on: https://go-review.googlesource.com/c/go/+/436882
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Rob Pike <r@golang.org>
2022-09-30 23:03:07 +00:00
cuiweixie 1e65fa58c1 encoding/json: return comparison directly
Change-Id: I4698d0fa78108d83ee91732e8d3878dbff7f9c90
Reviewed-on: https://go-review.googlesource.com/c/go/+/436711
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: xie cui <523516579@qq.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2022-09-30 20:10:56 +00:00
hopehook 972496ae6e all: use strings.Builder where appropriate
Change-Id: I164d350ca480640996055dedf38d962921c474a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/435975
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-29 22:56:00 +00:00
Roland Shoemaker 13d48bb6a1 encoding/gob: add top level security doc
Add a slightly expanded version of the Decoder type comment to the top
level package doc, which explains that this package is not designed
to be hardened against adversarial inputs.

Change-Id: I8b83433838c8235eb06ded99041fdf726c811ee5
Reviewed-on: https://go-review.googlesource.com/c/go/+/436096
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2022-09-29 00:30:21 +00:00
cuiweixie 09919ac339 encoding/xml: use bytes.Clone
Change-Id: I3218b1e3f8869f579facddb29471df13c835dc66
Reviewed-on: https://go-review.googlesource.com/c/go/+/435281
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2022-09-28 03:44:25 +00:00
Ian Lance Taylor a0441c7ae3 encoding/gob: use saferio.SliceCap when decoding a slice
This avoids allocating an overly large slice for corrupt input.

Change the saferio.SliceCap function to take a pointer to the element type,
so that we can handle slices of interface types. This revealed that a
couple of existing calls were actually incorrect, passing the slice type
rather than the element type.

No test case because the problem can only happen for invalid data. Let
the fuzzer find cases like this.

Fixes #55338

Change-Id: I3c1724183cc275d4981379773b0b8faa01a9cbd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/433296
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-25 01:18:43 +00:00
cuiweixie ab87db8b46 encoding: use strings.Builder
Change-Id: Ib07699df8ea41fd8d1bca8ad050859fac24623de
Reviewed-on: https://go-review.googlesource.com/c/go/+/428258
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-06 15:46:20 +00:00
RenKanai 202b7e7e76 encoding/asn1: fix doc for BitString.At's return value
Fixes #53287

Change-Id: If983ae34850d9b1b29764156a38628fa53897573
Reviewed-on: https://go-review.googlesource.com/c/go/+/411134
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: hopehook <hopehook@golangcn.org>
Reviewed-by: Subham <sarkar.subhams2@gmail.com>
2022-09-02 09:14:48 +00:00
Andy Pan 1a8dfadbfe encoding/json: move some misplaced benchmark tests to bench_test.go
Change-Id: I5987eed00ee825421abe62699a06e9b66499f35f
Reviewed-on: https://go-review.googlesource.com/c/go/+/425016
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-24 05:40:28 +00:00
Andy Pan e7f2e5697a encoding/json: give it a chance to put encodeState back in pool when error occurs
name                       old time/op    new time/op    delta
CodeEncoderError-10           688µs ± 8%     496µs ±15%   -27.92%  (p=0.000 n=10+9)
CodeMarshalError-10           747µs ± 6%     546µs ± 4%   -26.86%  (p=0.000 n=10+10)
MarshalBytesError/32-10       284µs ± 2%     273µs ± 1%    -3.84%  (p=0.000 n=10+10)
MarshalBytesError/256-10      281µs ± 2%     278µs ± 4%      ~     (p=0.053 n=9+10)
MarshalBytesError/4096-10     290µs ± 1%     279µs ± 3%    -3.52%  (p=0.000 n=10+10)

name                       old speed      new speed      delta
CodeEncoderError-10        2.83GB/s ± 8%  3.84GB/s ±20%   +36.03%  (p=0.000 n=10+10)
CodeMarshalError-10        2.60GB/s ± 5%  3.56GB/s ± 4%   +36.61%  (p=0.000 n=10+10)

name                       old alloc/op   new alloc/op   delta
CodeEncoderError-10          4.05MB ± 1%    0.00MB ± 1%  -100.00%  (p=0.000 n=10+9)
CodeMarshalError-10          6.05MB ± 0%    1.99MB ± 1%   -67.13%  (p=0.000 n=10+10)
MarshalBytesError/32-10      66.0kB ± 0%     0.2kB ± 0%   -99.67%  (p=0.000 n=9+8)
MarshalBytesError/256-10     50.1kB ± 0%     0.9kB ± 0%   -98.23%  (p=0.000 n=9+9)
MarshalBytesError/4096-10    87.4kB ± 0%     7.5kB ± 0%   -91.47%  (p=0.000 n=8+10)

name                       old allocs/op  new allocs/op  delta
CodeEncoderError-10            25.0 ± 0%       4.0 ± 0%   -84.00%  (p=0.000 n=9+10)
CodeMarshalError-10            27.0 ± 0%       6.0 ± 0%   -77.78%  (p=0.000 n=10+10)
MarshalBytesError/32-10        18.0 ± 0%       5.0 ± 0%   -72.22%  (p=0.000 n=10+10)
MarshalBytesError/256-10       17.0 ± 0%       6.0 ± 0%   -64.71%  (p=0.000 n=10+10)
MarshalBytesError/4096-10      16.0 ± 0%       6.0 ± 0%   -62.50%  (p=0.000 n=10+10)

Change-Id: I48070bb05f55707251c694e40d2570403bbf61f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/423694
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-23 20:23:19 +00:00
Axel Wagner 7f632f76db encoding/xml: add (*Encoder).Close
Flush can not check for unclosed elements, as more data might be encoded
after Flush is called. Close implicitly calls Flush and also checks that
all opened elements are closed as well.

Fixes #53346

Change-Id: I889b9f5ae54e5dfabb9e6948d96c5ed7bc1110f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/424777
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
2022-08-23 18:24:30 +00:00
Joe Tsai 0f42e35fee encoding/json: rely on utf8.AppendRune
Change-Id: I50e5609ff9c5f2b216b93cec7fb5214d196cae90
Reviewed-on: https://go-review.googlesource.com/c/go/+/412537
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-08-22 23:36:38 +00:00
Ian Lance Taylor dee9adc0f7 encoding/gob: use saferio to read large buffer
Avoid allocating large amounts of memory for corrupt input.

No test case because the problem can only happen for invalid data.
Let the fuzzer find cases like this.

Fixes #53369

Change-Id: I67c5e75bf181ad84988d6d6da12507df0e6df8e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/413979
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-19 16:03:42 +00:00
ErikPelli e1b62efaf3 encoding/base64: optimize decodemap memory set
The existing implementation has an execution time higher in the benchmark than this one.
This is an optimized implementation using the copy() function and a constant 256 bytes string with the values to be copied.

```
name           old time/op    new time/op    delta
NewEncoding-4     329ns ± 1%     231ns ± 1%  -29.72%  (p=0.008 n=5+5)

name           old speed      new speed      delta
NewEncoding-4   778MB/s ± 1%  1108MB/s ± 1%  +42.29%  (p=0.008 n=5+5)
```

Fixes #53211

Change-Id: I80fe62aa40623125ef81ae9164a8405eed30b71b
GitHub-Last-Rev: 55dce6f636
GitHub-Pull-Request: golang/go#53212
Reviewed-on: https://go-review.googlesource.com/c/go/+/410194
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-08-17 03:15:44 +00:00
Ludi Rehak d39b54171a encoding/gob: change typeInfo.encoder type to atomic.Pointer[T]
Replace loading and storing an atomic.Value of type pointer with
atomic.Pointer.

Change-Id: I018ac1e18eee4f203ebca65c2833daf991075371
Reviewed-on: https://go-review.googlesource.com/c/go/+/422174
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
2022-08-09 21:26:45 +00:00
Daniel Martí beebd537ca encoding/xml: use bytes.Buffer.WriteString
Change-Id: Icb0b917c48252ed8831e6c07109c1bc2d2438a42
Reviewed-on: https://go-review.googlesource.com/c/go/+/421234
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: hopehook <hopehook@qq.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Dan Kortschak <dan@kortschak.io>
2022-08-09 18:32:01 +00:00
cuiweixie 9903ab5469 encoding/binary: ReadUvarint return io.ErrUnexpectedEOF when read at least 1 byte
Fixes #54139

Change-Id: Ifc73bd7f181b13970ee6a08968f9d8f6e55d7ff3
GitHub-Last-Rev: 1e0a79bd3e
GitHub-Pull-Request: golang/go#54143
Reviewed-on: https://go-review.googlesource.com/c/go/+/420274
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Keith Randall <khr@google.com>
2022-08-08 17:29:13 +00:00
Tomasz Jezierski 0f6ee42fe0 encoding/gob: replace runtime values with constants in init()
Current init() implementation in `encoding/gob/decode.go` checks int/uint/uintptr bit size with reflection in runtime. We could replace it with values available on compile stage. This should reduce time and allocations on binary start.
Results from GODEBUG=inittrace=1:
before:
init encoding/gob @4.4 ms, 0.21 ms clock, 43496 bytes, 652 allocs
after:
init encoding/gob @4.4 ms, 0.15 ms clock, 41672 bytes, 643 allocs

Updates #54184

Change-Id: I46dda2682fb92519da199415e29401d61edce697
Reviewed-on: https://go-review.googlesource.com/c/go/+/420455
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-08-08 06:16:59 +00:00
Ian Lance Taylor 4651ebf961 encoding/gob: s/TestIngoreDepthLimit/TestIgnoreDepthLimit/
For #53615

Change-Id: Ib85004d400931094fc1ea933cf73f4a5157aece1
Reviewed-on: https://go-review.googlesource.com/c/go/+/417559
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-07-15 14:20:24 +00:00
Roland Shoemaker 783ff7dfc4 encoding/xml: skip TestCVE202230633 for short tests
TestCVE202230633 uses a bunch of memory, and the input cannot be
feasibly reduced while maintaining the behavior hasn't regressed. This
test could be reasonably removed, but I'd rather keep it around if we
can.

Fixes #53814

Change-Id: Ie8b3f306efd20b2d9c0fb73122c26351a55694c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/417655
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-07-14 21:03:14 +00:00
Roland Shoemaker 6fa37e98ea encoding/gob: add a depth limit for ignored fields
Enforce a nesting limit of 10,000 for ignored fields during decoding
of messages. This prevents the possibility of triggering stack
exhaustion.

Fixes #53615
Fixes CVE-2022-30635

Change-Id: I05103d06dd5ca3945fcba3c1f5d3b5a645e8fb0f
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1484771
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417064
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tatiana Bradley <tatiana@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2022-07-12 15:05:49 +00:00
Roland Shoemaker 08c46ed43d encoding/xml: use iterative Skip, rather than recursive
Prevents exhausting the stack limit in _incredibly_ deeply nested
structures.

Fixes #53614
Fixes CVE-2022-28131

Change-Id: I47db4595ce10cecc29fbd06afce7b299868599e6
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1419912
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417062
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-07-12 15:05:39 +00:00
Roland Shoemaker c4c1993fd2 encoding/xml: limit depth of nesting in unmarshal
Prevent exhausting the stack limit when unmarshalling extremely deeply
nested structures into nested types.

Fixes #53611
Fixes CVE-2022-30633

Change-Id: Ic6c5d41674c93cfc9a316135a408db9156d39c59
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1421319
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/417061
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-07-12 15:05:33 +00:00
shaoliming 606c6c371a encoding/xml: check nil pointer in DecodeElement
Fixes #53350

Change-Id: Id5e1f4016db5f1d4349ee1a76a9dfe3aeae83cee
GitHub-Last-Rev: 45add12161
GitHub-Pull-Request: golang/go#53407
Reviewed-on: https://go-review.googlesource.com/c/go/+/412634
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
2022-06-22 16:23:59 +00:00
Lukas Joisten 46ab7a5c4f encoding/json: mention SyntaxError in Unmarshal doc comment
Change-Id: I71c9d9ef9d21a7ae9466d8c7b283fdfbba01f5a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/390734
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-01 22:59:44 +00:00
John Bampton a6f3f8d973 all: fix spelling
Change-Id: I68538a50c22b02cdb5aa2a889f9440fed7b94c54
GitHub-Last-Rev: aaac9e7834
GitHub-Pull-Request: golang/go#52944
Reviewed-on: https://go-review.googlesource.com/c/go/+/406835
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2022-05-17 19:51:29 +00:00
hopehook 3474cd4eee encoding/csv: add Reader.InputOffset method
Fixes #43401.

Change-Id: I2498e77e41d845130d95012bc8623bfb29c0dda1
Reviewed-on: https://go-review.googlesource.com/c/go/+/405675
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-05-14 04:25:13 +00:00
teivah 8a5845e4e3 encoding/base32: decoder output depends on chunking of underlying reader
After an analysis, I figured that a way to do it could be to check, after
the call to readEncodedData whether the decoder already saw the end or not.

Fixes #38657

Change-Id: I06fd718ea4ee6ded2cb26c2866b28581ad86e271
GitHub-Last-Rev: d0b7bb38e4
GitHub-Pull-Request: golang/go#52631
Reviewed-on: https://go-review.googlesource.com/c/go/+/403315
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-03 18:30:15 +00:00
Joe Tsai 35a92f92bd encoding/binary: add AppendVarint AppendUvarint
This adds a straight-forward implementation of the functionality.
A more performant version could be added that unrolls the loop
as is done in google.golang.org/protobuf/encoding/protowire,
but usages that demand high performance can use that package instead.

Fixes #51644

Change-Id: I9d3b615a60cdff47e5200e7e5d2276adf4c93783
Reviewed-on: https://go-review.googlesource.com/c/go/+/400176
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-15 01:19:37 +00:00
Julie Qiu 45c3387d77 encoding/pem: fix stack overflow in Decode
Previously, Decode called decodeError, a recursive function that was
prone to stack overflows when given a large PEM file containing errors.

Credit to Juho Nurminen of Mattermost who reported the error.

Fixes CVE-2022-24675
Fixes #51853

Change-Id: Iffe768be53c8ddc0036fea0671d290f8f797692c
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1391157
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Filippo Valsorda <valsorda@google.com>
(cherry picked from commit 794ea5e828010e8b68493b2fc6d2963263195a02)
Reviewed-on: https://go-review.googlesource.com/c/go/+/399820
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-12 15:19:32 +00:00
Russ Cox 19309779ac all: gofmt main repo
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]

Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.

For #51082.

Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-11 16:34:30 +00:00
Russ Cox 9839668b56 all: separate doc comment from //go: directives
A future change to gofmt will rewrite

	// Doc comment.
	//go:foo

to

	// Doc comment.
	//
	//go:foo

Apply that change preemptively to all comments (not necessarily just doc comments).

For #51082.

Change-Id: Iffe0285418d1e79d34526af3520b415a12203ca9
Reviewed-on: https://go-review.googlesource.com/c/go/+/384260
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-05 17:54:15 +00:00
Russ Cox 81431c7aa7 all: replace `` and '' with “ (U+201C) and ” (U+201D) in doc comments
go/doc in all its forms applies this replacement when rendering
the comments. We are considering formatting doc comments,
including doing this replacement as part of the formatting.
Apply it to our source files ahead of time.

For #51082.

Change-Id: Ifcc1f5861abb57c5d14e7d8c2102dfb31b7a3a19
Reviewed-on: https://go-review.googlesource.com/c/go/+/384262
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-05 17:52:29 +00:00
Russ Cox 690ac4071f all: remove trailing blank doc comment lines
A future change to gofmt will rewrite

	// Doc comment.
	//
	func f()

to

	// Doc comment.
	func f()

Apply that change preemptively to all doc comments.

For #51082.

Change-Id: I4023e16cfb0729b64a8590f071cd92f17343081d
Reviewed-on: https://go-review.googlesource.com/c/go/+/384259
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01 18:18:07 +00:00
Patrick Gundlach 0eea25159f encoding/xml: expose decoder line and column
The existing implementation of the xml decoder uses the line number
only for reporting syntax errors. The line number of the last read
token and the column within the line is useful for the users even
in non-error conditions.

Fixes #45628

Change-Id: I37b5033ff5ff8411793d8f5180f96aa4537e83f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/311270
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2022-03-27 20:26:46 +00:00
eh-steve baf61e4a67 encoding/hex: implement Decode with a lookup table
Implement hex decode using a 256 byte lookup table instead of branching logic.

In happy flow, uses 3x 64 byte (or 5x 32 byte) cache lines.

	name             old time/op    new time/op    delta
	Decode/256-64       223ns ± 3%     135ns ± 2%  -39.64%  (p=0.000 n=8+8)
	Decode/1024-64      872ns ± 2%     512ns ± 2%  -41.25%  (p=0.000 n=8+8)
	Decode/4096-64     3.43µs ± 1%    2.01µs ± 2%  -41.31%  (p=0.001 n=7+7)
	Decode/16384-64    13.9µs ± 1%     8.0µs ± 1%  -42.69%  (p=0.000 n=8+7)

	name             old speed      new speed      delta
	Decode/256-64    1.15GB/s ± 3%  1.90GB/s ± 2%  +65.66%  (p=0.000 n=8+8)
	Decode/1024-64   1.17GB/s ± 2%  2.00GB/s ± 2%  +70.22%  (p=0.000 n=8+8)
	Decode/4096-64   1.20GB/s ± 1%  2.04GB/s ± 2%  +70.39%  (p=0.001 n=7+7)
	Decode/16384-64  1.18GB/s ± 1%  2.06GB/s ± 1%  +74.49%  (p=0.000 n=8+7)

Also reduces amd64 object size by 766 bytes, despite the extra RODATA due to removal of `fromHexChar()` and duplicated inlined versions of it and simplification of `Decode()`.

Change-Id: I0988c7a30562ec154eff11db6e27954e0ce2b611
GitHub-Last-Rev: 64818018af
GitHub-Pull-Request: golang/go#51432
Reviewed-on: https://go-review.googlesource.com/c/go/+/390037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-11 21:51:20 +00:00
Iskander Sharipov e79c39f004 encoding/xml: improve the test coverage, fix minor bugs
Improve the test coverage of encoding/xml package by adding
the test cases for the execution paths that were not covered before.

Since it reveals a couple of issues, fix them as well while we're at it.

As I used an `strings.EqualFold` instead of adding one more `strings.ToLower`,
our fix to `autoClose()` tends to run faster as well as a result.

	name             old time/op    new time/op    delta
	HTMLAutoClose-8    5.93µs ± 2%    5.75µs ± 3%  -3.16%  (p=0.000 n=10+10)
	name             old alloc/op   new alloc/op   delta
	HTMLAutoClose-8    2.60kB ± 0%    2.58kB ± 0%  -0.46%  (p=0.000 n=10+10)
	name             old allocs/op  new allocs/op  delta
	HTMLAutoClose-8      72.0 ± 0%      67.0 ± 0%  -6.94%  (p=0.000 n=10+10)

The overall `encoding/xml` test coverage increase is `88.1% -> 89.9%`;
although it may look insignificant, this CL covers some important corner cases,
like `autoClose()` functionality (that was not tested at all).

Fixes #49635
Fixes #49636

Change-Id: I50b2769896c197eb285672313b7148f4fe8bdb38
Reviewed-on: https://go-review.googlesource.com/c/go/+/364734
Trust: Bryan Mills <bcmills@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-04 20:29:47 +00:00
Joe Tsai 986b04c0f1 encoding/binary: add AppendByteOrder
AppendByteOrder specifies new methods for LittleEndian and BigEndian
for appending an unsigned integer to a byte slice.

The performance of AppendXXX methods are slower than PutXXX methods
since the former needs to do a few slice operations,
while the latter is essentially a single integer store.
In practice, existing usages of PutXXX performed slicing operations
around the call such that this cost was present, regardless.

name                           time/op
PutUint16-24                   0.48ns ± 2%
AppendUint16-24                1.54ns ± 1%
PutUint32-24                   0.46ns ± 2%
AppendUint32-24                0.89ns ± 1%
PutUint64-24                   0.46ns ± 2%
AppendUint64-24                0.89ns ± 1%
LittleEndianPutUint16-24       0.47ns ± 2%
LittleEndianAppendUint16-24    1.54ns ± 1%
LittleEndianPutUint32-24       0.45ns ± 3%
LittleEndianAppendUint32-24    0.92ns ± 2%
LittleEndianPutUint64-24       0.46ns ± 3%
LittleEndianAppendUint64-24    0.95ns ± 4%

Fixes #50601

Change-Id: I33d2bbc93a3ce01a9269feac33a2432bc1166ead
Reviewed-on: https://go-review.googlesource.com/c/go/+/386017
Trust: Joseph Tsai <joetsai@digital-static.net>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-02 18:21:38 +00:00
Joe Tsai 5a03cbd12a encoding/json: use reflect.Value.UnsafePointer over Pointer
The latter returns a uintptr, while the former returns a unsafe.Pointer.
A uintptr is unsafe if Go ever switches to a moving GC,
while a unsafe.Pointer will be properly tracked by the GC.

We do not use unsafe.Pointer for any unsafe type conversions,
and only use it for comparability purposes, which is relatively safe.

Updates #40592

Change-Id: I813e218668704b63a3043acda4331205a3835a66
Reviewed-on: https://go-review.googlesource.com/c/go/+/360855
Trust: Joseph Tsai <joetsai@digital-static.net>
Trust: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-02 17:42:53 +00:00
hopehook 851ecea4cc encoding/xml: embedded reference to substruct causes XML marshaller to panic on encoding
When encoding a xml attribute is zero value (IsValid == false), we need
a `continue` to jump over the attribute. If not, followed marshalAttr
function will panic.

Fixes: #50164
Change-Id: I42e064558e7becfbf47728b14cbf5c7afa1e8798
Reviewed-on: https://go-review.googlesource.com/c/go/+/385514
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-02-20 20:58:11 +00:00
Roland Shoemaker 4fa6e33f30 all: add a handful of fuzz targets
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip,
encoding/json, image/jpeg, image/gif, and image/png.

Second attempt, this time we don't use the archives in testdata when
fuzzing archive/tar, since those are rather memory intensive, and
were crashing a number of builders.

Change-Id: I4828d64fa4763c0d8c980392a6578e4dfd956e13
Reviewed-on: https://go-review.googlesource.com/c/go/+/378174
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-13 18:06:33 +00:00
Bryan Mills 6e8b7e4f42 Revert "all: add a handful of fuzz targets"
This reverts CL 352109.

Reason for revert: causing OOM failures on several builders, and may cause OOMs for end users with small machines as well.

Change-Id: I58308d09919969d5a6512ee5cee6aa5c4af6769b
Reviewed-on: https://go-review.googlesource.com/c/go/+/377934
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
2022-01-12 15:53:47 +00:00
Roland Shoemaker 9bce08999a all: add a handful of fuzz targets
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip,
encoding/json, image/jpeg, image/gif, and image/png.

Change-Id: Ide1a8de88a9421e786eeeaea3bb93f41e0bae347
Reviewed-on: https://go-review.googlesource.com/c/go/+/352109
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-12 00:12:10 +00:00
Russ Cox 2580d0e08d all: gofmt -w -r 'interface{} -> any' src
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.

Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.

A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.

Fixes #49884.

Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13 18:45:54 +00:00
Keith Randall b93220c9ca encoding/xml: add generic encoding test
Fixes #48521

Change-Id: Id8402bcff243c0ab19e4ec0b138b9af8c111f88d
Reviewed-on: https://go-review.googlesource.com/c/go/+/355492
Trust: Keith Randall <khr@golang.org>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2021-11-09 21:26:25 +00:00
Josh Bleecher Snyder 65f3478633 encoding: simplify some code
Change-Id: I63aac663da4ce7bdd198d385f87929c74f6a5cf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/360601
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-11-02 03:53:23 +00:00
Russ Cox f229e7031a all: go fix -fix=buildtag std cmd (except for bootstrap deps, vendor)
When these packages are released as part of Go 1.18,
Go 1.16 will no longer be supported, so we can remove
the +build tags in these files.

Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs
as defined in src/cmd/dist/buildtool.go, which need to continue
to build with Go 1.4 for now.

Also reverted src/vendor and src/cmd/vendor, which will need
to be updated in their own repos first.

Manual changes in runtime/pprof/mprof_test.go to adjust line numbers.

For #41184.

Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e
Reviewed-on: https://go-review.googlesource.com/c/go/+/344955
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 18:17:57 +00:00