Gerrit is complaining about pushes that affect these files
and forcing people to use -o nokeycheck, which defeats
the point of the check. Hide the keys from this kind of scan
by marking them explicitly as testing keys.
This is a little annoying but better than training everyone
who ever edits one of these test files to reflexively override
the Gerrit check.
The only remaining keys explicitly marked as private instead
of testing are in examples, and there's not much to do
about those. Hopefully they are not edited as much.
Change-Id: I4431592b5266cb39fe6a80b40e742d97da803a0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/178178
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Renaming the method makes clear, both to readers and to vet,
that this method is not the implementation of io.ByteWriter.
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.
For #31916.
Change-Id: I5b509eb7f0118d5f2d3c6e352ff2849cd5a3071e
Reviewed-on: https://go-review.googlesource.com/c/go/+/176110
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Most changes are removing redundant declaration of type when direct
instantiating value of map or slice, e.g. []T{T{}} become []T{{}}.
Small changes are removing the high order of subslice if its value
is the length of slice itself, e.g. T[:len(T)] become T[:].
The following file is excluded due to incompatibility with go1.4,
- src/cmd/compile/internal/gc/ssa.go
Change-Id: Id3abb09401795ce1e6da591a89749cba8502fb26
Reviewed-on: https://go-review.googlesource.com/c/go/+/166437
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Add Unwrap methods to types which wrap an underlying error:
"encodinc/csv".ParseError
"encoding/json".MarshalerError
"net/http".transportReadFromServerError
"net".OpError
"net".DNSConfigError
"net/url".Error
"os/exec".Error
"signal/internal/pty".PtyError
"text/template".ExecError
Add os.ErrTemporary. A case could be made for putting this error
value in package net, since no exported error types in package os
include a Temporary method. However, syscall errors returned from
the os package do include this method.
Add Is methods to error types with a Timeout or Temporary method,
making errors.Is(err, os.Err{Timeout,Temporary}) equivalent to
testing the corresponding method:
"context".DeadlineExceeded
"internal/poll".TimeoutError
"net".adrinfoErrno
"net".OpError
"net".DNSError
"net/http".httpError
"net/http".tlsHandshakeTimeoutError
"net/pipe".timeoutError
"net/url".Error
Updates #30322
Updates #29934
Change-Id: I409fb20c072ea39116ebfb8c7534d493483870dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/170037
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
It's slow & often times out randomly on longtest builders. Not useful.
Fixes#31517
Change-Id: Icedbb0c94fbe43d04e8b47d5785ac61c5e2d8750
Reviewed-on: https://go-review.googlesource.com/c/go/+/174522
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
indirect walks down v until it gets to a non-pointer. But it does not
handle the case when v is a pointer to itself, like in:
var v interface{}
v = &v
Unmarshal(b, v)
So just stop immediately if we see v is a pointer to itself.
Fixes#31740
Change-Id: Ie396264119e24d70284cd9bf76dcb2050babb069
Reviewed-on: https://go-review.googlesource.com/c/go/+/174337
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Make explicit that Compact does HTML escaping.
Fixes#30357.
Change-Id: I4648f8f3e907d659db977d07253f716df6e07d7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/173417
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In the common case, structs have a handful of fields and most inputs
match struct field names exactly.
The previous code would do a linear search over the fields, stopping at
the first exact match, and otherwise using the first case insensitive
match.
This is unfortunate, because it means that for the common case, we'd do
a linear search with bytes.Equal. Even for structs with only two or
three fields, that is pretty wasteful.
Worse even, up until the exact match was found via the linear search,
all previous fields would run their equalFold functions, which aren't
cheap even in the simple case.
Instead, cache a map along with the field list that indexes the fields
by their name. This way, a case sensitive field search doesn't involve a
linear search, nor does it involve any equalFold func calls.
This patch should also slightly speed up cases where there's a case
insensitive match but not a case sensitive one, as then we'd avoid
calling bytes.Equal on all the fields. Though that's not a common case,
and there are no benchmarks for it.
name old time/op new time/op delta
CodeDecoder-8 11.0ms ± 0% 10.6ms ± 1% -4.42% (p=0.000 n=9+10)
name old speed new speed delta
CodeDecoder-8 176MB/s ± 0% 184MB/s ± 1% +4.62% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
CodeDecoder-8 2.28MB ± 0% 2.28MB ± 0% ~ (p=0.725 n=10+10)
name old allocs/op new allocs/op delta
CodeDecoder-8 76.9k ± 0% 76.9k ± 0% ~ (all equal)
Updates #28923.
Change-Id: I9929c1f06c76505e5b96914199315dbdaae5dc76
Reviewed-on: https://go-review.googlesource.com/c/go/+/172918
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We can work out how many bytes can be unquoted trivially in
rescanLiteral, which already iterates over a string's bytes.
Removing the extra loop in unquoteBytes simplifies the function and
speeds it up, especially when decoding simple strings, which are common.
While at it, we can remove unnecessary checks like len(s)<2 and
s[0]=='"'. Add a comment explaining why.
name old time/op new time/op delta
CodeDecoder-8 11.2ms ± 0% 11.1ms ± 1% -1.63% (p=0.000 n=9+10)
name old speed new speed delta
CodeDecoder-8 173MB/s ± 0% 175MB/s ± 1% +1.66% (p=0.000 n=9+10)
Updates #28923.
Change-Id: I2436a3a7f8148a2f7a6a4cdbd7dec6b32ef5e20c
Reviewed-on: https://go-review.googlesource.com/c/go/+/151157
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
readValue is a hot function, clocking in at ~13% flat CPU use in
CodeDecoder. In particular, looping over the bytes is slow. That's
partially because the code contains a bounds check at the start of the
loop.
The source of the problem is that scanp is a signed integer, and comes
from a field, so the compiler doesn't know that it's non-negative. Help
it with a simple and comparatively cheap hint.
While at it, use scanp as the index variable directly, removing the need
for a duplicate index variable which is later added back into scanp.
name old time/op new time/op delta
CodeDecoder-8 11.3ms ± 1% 11.2ms ± 1% -0.98% (p=0.000 n=9+9)
name old speed new speed delta
CodeDecoder-8 172MB/s ± 1% 174MB/s ± 1% +0.99% (p=0.000 n=9+9)
Updates #28923.
Change-Id: I138f83babdf316fc97697cc18f595c3403c1ddb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/170939
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This was the only benchmark missing the SetBytes call, as spotted
earlier by Bryan.
It's not required to make the benchmark useful, but it can still be a
good way to see how its speed is affected by the reduced allocations:
name time/op
CodeUnmarshal-8 12.1ms ± 1%
CodeUnmarshalReuse-8 11.4ms ± 1%
name speed
CodeUnmarshal-8 161MB/s ± 1%
CodeUnmarshalReuse-8 171MB/s ± 1%
name alloc/op
CodeUnmarshal-8 3.28MB ± 0%
CodeUnmarshalReuse-8 1.94MB ± 0%
name allocs/op
CodeUnmarshal-8 92.7k ± 0%
CodeUnmarshalReuse-8 77.6k ± 0%
While at it, remove some unnecessary empty lines.
Change-Id: Ib2bd92d5b3237b8f3092e8c6f863dab548fee2f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/170938
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Decoder.Decode and Unmarshal actually scan the input bytes twice - the
first time to check for syntax errors and the length of the value, and
the second to perform the decoding.
It's in the second scan that we actually tokenize the bytes. Since
syntax errors aren't a possibility, we can take shortcuts.
In particular, literals such as quoted strings are very common in JSON,
so we can avoid a lot of work by special casing them.
name old time/op new time/op delta
CodeDecoder-8 10.3ms ± 1% 9.1ms ± 0% -11.89% (p=0.002 n=6+6)
UnicodeDecoder-8 342ns ± 0% 283ns ± 0% -17.25% (p=0.000 n=6+5)
DecoderStream-8 239ns ± 0% 230ns ± 0% -3.90% (p=0.000 n=6+5)
CodeUnmarshal-8 11.0ms ± 0% 9.8ms ± 0% -11.45% (p=0.002 n=6+6)
CodeUnmarshalReuse-8 10.3ms ± 0% 9.0ms ± 0% -12.72% (p=0.004 n=5+6)
UnmarshalString-8 104ns ± 0% 92ns ± 0% -11.35% (p=0.002 n=6+6)
UnmarshalFloat64-8 93.2ns ± 0% 87.6ns ± 0% -6.01% (p=0.010 n=6+4)
UnmarshalInt64-8 74.5ns ± 0% 71.5ns ± 0% -3.91% (p=0.000 n=5+6)
name old speed new speed delta
CodeDecoder-8 189MB/s ± 1% 214MB/s ± 0% +13.50% (p=0.002 n=6+6)
UnicodeDecoder-8 40.9MB/s ± 0% 49.5MB/s ± 0% +20.96% (p=0.002 n=6+6)
CodeUnmarshal-8 176MB/s ± 0% 199MB/s ± 0% +12.93% (p=0.002 n=6+6)
Updates #28923.
Change-Id: I7a5e2aef51bd4ddf2004aad24210f6f50e01eaeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/151042
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In golang.org/cl/145218, a feature was added where the JSON decoder
would keep track of the entire path to a field when reporting an
UnmarshalTypeError.
However, we all failed to check if this affected the benchmarks - myself
included, as a reviewer. Below are the numbers comparing the CL's parent
with itself, once it was merged:
name old time/op new time/op delta
CodeDecoder-8 12.9ms ± 1% 28.2ms ± 2% +119.33% (p=0.002 n=6+6)
name old speed new speed delta
CodeDecoder-8 151MB/s ± 1% 69MB/s ± 3% -54.40% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
CodeDecoder-8 2.74MB ± 0% 109.39MB ± 0% +3891.83% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
CodeDecoder-8 77.5k ± 0% 168.5k ± 0% +117.30% (p=0.004 n=6+5)
The reason why the decoder got twice as slow is because it now allocated
~40x as many objects, which puts a lot of pressure on the garbage
collector.
The reason is that the CL concatenated strings every time a nested field
was decoded. In other words, practically every field generated garbage
when decoded. This is hugely wasteful, especially considering that the
vast majority of JSON decoding inputs won't return UnmarshalTypeError.
Instead, use a stack of fields, and make sure to always use the same
backing array, to ensure we only need to grow the slice to the maximum
depth once.
The original CL also introduced a bug. The field stack string wasn't
reset to its original state when reaching "d.opcode == scanEndObject",
so the last field in a decoded struct could leak. For example, an added
test decodes a list of structs, and encoding/json before this CL would
fail:
got: cannot unmarshal string into Go struct field T.Ts.Y.Y.Y of type int
want: cannot unmarshal string into Go struct field T.Ts.Y of type int
To fix that, simply reset the stack after decoding every field, even if
it's the last.
Below is the original performance versus this CL. There's a tiny
performance hit, probably due to the append for every decoded field, but
at least we're back to the usual ~150MB/s.
name old time/op new time/op delta
CodeDecoder-8 12.9ms ± 1% 13.0ms ± 1% +1.25% (p=0.009 n=6+6)
name old speed new speed delta
CodeDecoder-8 151MB/s ± 1% 149MB/s ± 1% -1.24% (p=0.009 n=6+6)
name old alloc/op new alloc/op delta
CodeDecoder-8 2.74MB ± 0% 2.74MB ± 0% +0.00% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
CodeDecoder-8 77.5k ± 0% 77.5k ± 0% +0.00% (p=0.002 n=6+6)
Finally, make all of these benchmarks report allocs by default. The
decoder ones are pretty sensitive to generated garbage, so ReportAllocs
would have made the performance regression more obvious.
Change-Id: I67b50f86b2e72f55539429450c67bfb1a9464b67
Reviewed-on: https://go-review.googlesource.com/c/go/+/167978
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Most of the decoding time is spent in the first Decode loop, since the
rest of the function only deals with the few remaining bytes. Any
unnecessary work done in that loop body matters tremendously.
One such unnecessary bottleneck was the use of the enc.decodeMap table.
Since enc is a pointer receiver, and the field is used within the
non-inlineable function decode64, the decoder must perform a nil check
at every iteration.
To fix that, move the enc.decodeMap uses to the parent function, where
we can lift the nil check outside the loop. That gives roughly a 15%
speed-up. The function no longer performs decoding per se, so rename it.
While at it, remove the now unnecessary receivers.
An unfortunate side effect of this change is that the loop now contains
eight bounds checks on src instead of just one. However, not having to
slice src plus the nil check removal well outweigh the added cost.
The other piece that made decode64 slow was that it wasn't inlined, and
had multiple branches. Use a simple bitwise-or trick suggested by Roger
Peppe, and collapse the rest of the bitwise logic into a single
expression. Inlinability and the reduced branching give a further 10%
speed-up.
Finally, add these two functions to TestIntendedInlining, since we want
them to stay inlinable.
Apply the same refactor to decode32 for consistency, and to let 32-bit
architectures see a similar performance gain for large inputs.
name old time/op new time/op delta
DecodeString/2-8 47.3ns ± 1% 45.8ns ± 0% -3.28% (p=0.002 n=6+6)
DecodeString/4-8 55.8ns ± 2% 51.5ns ± 0% -7.71% (p=0.004 n=5+6)
DecodeString/8-8 64.9ns ± 0% 61.7ns ± 0% -4.99% (p=0.004 n=5+6)
DecodeString/64-8 238ns ± 0% 198ns ± 0% -16.54% (p=0.002 n=6+6)
DecodeString/8192-8 19.5µs ± 0% 14.6µs ± 0% -24.96% (p=0.004 n=6+5)
name old speed new speed delta
DecodeString/2-8 84.6MB/s ± 1% 87.4MB/s ± 0% +3.38% (p=0.002 n=6+6)
DecodeString/4-8 143MB/s ± 2% 155MB/s ± 0% +8.41% (p=0.004 n=5+6)
DecodeString/8-8 185MB/s ± 0% 195MB/s ± 0% +5.29% (p=0.004 n=5+6)
DecodeString/64-8 369MB/s ± 0% 442MB/s ± 0% +19.78% (p=0.002 n=6+6)
DecodeString/8192-8 560MB/s ± 0% 746MB/s ± 0% +33.27% (p=0.004 n=6+5)
Updates #19636.
Change-Id: Ib839577b0e3f5a2bb201f5cae580c61365d92894
Reviewed-on: https://go-review.googlesource.com/c/go/+/151177
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: roger peppe <rogpeppe@gmail.com>
If a for loop has a simple condition and begins with a simple
"if x { break; }"; we can simply add "!x" to the loop's condition.
While at it, simplify a few assignments to use the common pattern
"x := staticDefault; if cond { x = otherValue(); }".
Finally, simplify a couple of var declarations.
Change-Id: I413982c6abd32905adc85a9a666cb3819139c19f
Reviewed-on: https://go-review.googlesource.com/c/go/+/165342
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I had been finding these over a year or so, but none were big enough
changes to warrant CLs. They're a handful now, so clean them all up in a
single commit.
The smaller bodies get a bit simpler, but most importantly, the larger
bodies get unindented.
Change-Id: I5707a6fee27d4c9ff9efd3d363af575d7a4bf2aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/165340
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Base64-encoding 32 bytes results in a 44-byte string.
While in general a 44-byte string might decode to 33 bytes,
if you take a 44-byte string that actually only encodes 32 bytes,
and you try to decode it into 32 bytes, that should succeed.
Instead it fails trying to do a useless dst[33:] slice operation.
Delete that slice operation.
Noticed while preparing CL 156322.
Change-Id: I8024bf28a65e2638675b980732b2ff91c66c62cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/164628
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Two additions are faster than two multiplications and one addition. The
code seems simpler to me too, as it's more obvious that we advance two
destination bytes for each source byte.
name old time/op new time/op delta
Encode/256-8 374ns ± 0% 331ns ± 0% -11.44% (p=0.008 n=5+5)
Encode/1024-8 1.47µs ± 0% 1.29µs ± 0% -11.89% (p=0.004 n=6+5)
Encode/4096-8 5.85µs ± 1% 5.15µs ± 0% -11.89% (p=0.004 n=6+5)
Encode/16384-8 23.3µs ± 0% 20.6µs ± 0% -11.68% (p=0.004 n=6+5)
Change-Id: Iabc63616c1d9fded55fa668ff41dd49efeaa2ea4
Reviewed-on: https://go-review.googlesource.com/c/go/+/151198
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: roger peppe <rogpeppe@gmail.com>
When parsing nested object, UnmarshalTypeError does not contain actual
path to nested field in original JSON.
This commit change Field to contain the full path to that field. One
can get the Field name by stripping all the leading path elements.
Fixes#22369
Change-Id: I6969cc08abe8387a351e3fb2944adfaa0dccad2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/145218
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add documentation that individual Write calls are buffered and
copy documentation from bufio.Writer notifying the user to call
Flush and Error when all writes are complete. Remove reference
to "file" since the implementation is general and allows any
io.Writer.
Fixes#30045
Change-Id: I50165470e548f296494e764707fbabe36c665015
Reviewed-on: https://go-review.googlesource.com/c/160680
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Most of the encoding time is spent in the first Encode loop, since the
rest of the function only deals with the few remaining bytes. Any
unnecessary work done in that loop body matters tremendously.
One such unnecessary bottleneck was the use of the enc.encode table.
Since enc is a pointer receiver, and the field is first used within the
loop, the encoder must perform a nil check at every iteration.
Add a dummy use of the field before the start of the loop, to move the
nil check there. After that line, the compiler now knows that enc can't
be nil, and thus the hot loop is free of nil checks.
name old time/op new time/op delta
EncodeToString-4 14.7µs ± 0% 13.7µs ± 1% -6.53% (p=0.000 n=10+10)
name old speed new speed delta
EncodeToString-4 559MB/s ± 0% 598MB/s ± 1% +6.99% (p=0.000 n=10+10)
Updates #20206.
Change-Id: Icbb523a7bd9e470a8be0a448d1d78ade97ed4ff6
Reviewed-on: https://go-review.googlesource.com/c/151158
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
encoding/base64 already skips \r and \n when decoding, so this package
must only deal with spaces and tabs. Those aren't nearly as common, so
we can add a fast path with bytes.ContainsAny to skip the costly alloc
and filtering code.
name old time/op new time/op delta
Decode-8 279µs ± 0% 259µs ± 1% -7.07% (p=0.002 n=6+6)
name old speed new speed delta
Decode-8 319MB/s ± 0% 343MB/s ± 1% +7.61% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
Decode-8 164kB ± 0% 74kB ± 0% -54.90% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
Decode-8 12.0 ± 0% 11.0 ± 0% -8.33% (p=0.002 n=6+6)
Change-Id: Idfca8700c52f46eb70a4a7e0d2db3bf0124e4699
Reviewed-on: https://go-review.googlesource.com/c/155964
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Comparing errors using DeepEqual breaks if frame information
is added as proposed in Issue #29934.
Updates #29934.
Change-Id: Ib430c9ddbe588dd1dd51314c408c74c07285e1ff
Reviewed-on: https://go-review.googlesource.com/c/162179
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
First, we can lift the enc.decodeMap nil check out of the loop.
Second, we can make it clear to the compiler that 'in := src[0]' doesn't
need a bounds check, by making len(src)==0 a single if check that always
stops the loop. This is by far the largest speed-up.
Third, we can use a dst slice index instead of reslicing dst, which
removes work from the loop body.
While at it, we can merge the two 'switch dlen' pieces of code, which
simplifies the code and doesn't affect performance.
name old time/op new time/op delta
DecodeString-8 80.2µs ± 0% 67.5µs ± 0% -15.81% (p=0.002 n=6+6)
name old speed new speed delta
DecodeString-8 163MB/s ± 0% 194MB/s ± 0% +18.78% (p=0.002 n=6+6)
Change-Id: Iefeaae94c03453f8760452b1da706a77b3522718
Reviewed-on: https://go-review.googlesource.com/c/154422
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Both the encoding/xml and encoding/json packages support custom
marshalers for JSON and XML, as well as the basic encoding.TextMarshaler
and encoding.TextUnmarshaler interfaces, but the docs and examples for
these are missing.
There are docs for how to use encoding.TextMarshaler and
encoding.TextUnmarshaler in encoding/json, but not encoding/xml. There
are no examples for how to use them with either json or xml. This commit
includes docs for encoding/xml and examples for both encoding/json and
encoding/xml.
There is an example using custom marshalers MarshalJSON and
UnmarshalJSON in encoding/json, but not MarshalXML and UnmarshalXML in
encoding/json. These docs are more so necessary for encoding/xml because
the complexities of XML documents is significantly greater than JSON
documents which more often leads to the need for custom marshaling. The
encoding/json package includes an example of how to write a custom
marshaler, and this commit includes the same example for the xml
package.
All examples are mirrored off the existing custom marshaler example in
encoding/json.
Fixes#6859
Change-Id: Ic93abc27c0b4d5e48dea6ede4e20b1bedca4ab39
Reviewed-on: https://go-review.googlesource.com/c/76350
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I7a1046f5e0aedbbdd1106a616de410fe4e0cb7d8
Reviewed-on: https://go-review.googlesource.com/c/92295
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Go documentation style for boolean funcs is to say:
// Foo reports whether ...
func Foo() bool
(rather than "returns true if")
This CL also replaces 4 uses of "iff" with the same "reports whether"
wording, which doesn't lose any meaning, and will prevent people from
sending typo fixes when they don't realize it's "if and only if". In
the past I think we've had the typo CLs updated to just say "reports
whether". So do them all at once.
(Inspired by the addition of another "returns true if" in CL 146938
in fd_plan9.go)
Created with:
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor)
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor)
Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f
Reviewed-on: https://go-review.googlesource.com/c/147037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Now that the library allows much larger data, it can kill
machines with less memory.
Fixes#28321
Change-Id: I98e1a5fdf812fd75adfb22bf01542423de405fe2
Reviewed-on: https://go-review.googlesource.com/c/143817
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
A little shift magic makes it easy to adjust the maximum buffer
size on machines with larger integers.
Fixes#27635
Change-Id: I1f26b07a363fbb9730df2377052475fa88bbb781
Reviewed-on: https://go-review.googlesource.com/c/143678
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Calling .Interface on a struct field's reflect.Value isn't always safe.
For example, if that field is an unexported anonymous struct.
We only descended into this branch if the struct type had any methods,
so this bug had gone unnoticed for a few release cycles.
Add the check, and add a simple test case.
Fixes#28145.
Change-Id: I02f7e0ab9a4a0c18a5e2164211922fe9c3d30f64
Reviewed-on: https://go-review.googlesource.com/c/141537
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Given a program as follows:
data := []byte(`{"F": {
"a": 2,
"3": 4
}}`)
json.Unmarshal(data, &map[string]map[int]int{})
The JSON package should error, as "a" is not a valid integer. However,
we'd encounter a panic:
panic: JSON decoder out of sync - data changing underfoot?
The reason was that decodeState.object would return a nil error on
encountering the invalid map key string, while saving the key type error
for later. This broke if we were inside another object, as we would
abruptly end parsing the nested object, leaving the decoder in an
unexpected state.
To fix this, simply avoid storing the map element and continue decoding
the object, to leave the decoder state exactly as if we hadn't seen an
invalid key type.
This affected both signed and unsigned integer keys, so fix both and add
two test cases.
Updates #28189.
Change-Id: I8a6204cc3ff9fb04ed769df7a20a824c8b94faff
Reviewed-on: https://go-review.googlesource.com/c/142518
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change makes stateEndTop use isSpace instead of specifically
recreating the same functionality.
Change-Id: I81f8f51682e46e7f8e2b9fed423a968457200625
Reviewed-on: https://go-review.googlesource.com/c/121797
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)
Updates #27864
Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Having these panic-like errors used to be ok, since they were used in
the internal decoder state instead of passed around via return
parameters.
Recently, the decoder was rewritten to use explicit error returns
instead. This error is a terrible fit for error returns; a handful of
functions must return an error because of it, and their callers must
check for an error that should never happen.
This is precisely what panics are for, so use them. The test coverage of
the package goes up from 91.3% to 91.6%, and performance is unaffected.
We can also get rid of some unnecessary verbosity in the code.
name old time/op new time/op delta
CodeDecoder-4 27.5ms ± 1% 27.5ms ± 1% ~ (p=0.937 n=6+6)
Change-Id: I01033b3f5b7c0cf0985082fa272754f96bf6353c
Reviewed-on: https://go-review.googlesource.com/134835
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The overall coverage of the json package goes up from 90.8% to 91.3%.
While at it, apply two minor code simplifications found while inspecting
the HTML coverage report.
Change-Id: I0fba968afeedc813b1385e4bde72d93b878854d7
Reviewed-on: https://go-review.googlesource.com/134735
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Given the following types:
type S2 struct{ Field string }
type S struct{ *S2 }
Marshalling a value of type T1 should result in "{}", as there's no way
to access any value of T2.Field. This is how Go 1.10 and earlier
versions behave.
However, in the recent refactor golang.org/cl/125417 I broke this logic.
When the encoder found an anonymous struct pointer field that was nil,
it no longer skipped the embedded fields underneath it. This can be seen
in the added test:
--- FAIL: TestAnonymousFields/EmbeddedFieldBehindNilPointer (0.00s)
encode_test.go:430: Marshal() = "{\"Field\":\"\\u003c*json.S2 Value\\u003e\"}", want "{}"
The human error was a misplaced label, meaning we weren't actually
skipping the right loop iteration. Fix that.
Change-Id: Iba8a4a77d358dac73dcba4018498fe4f81afa263
Reviewed-on: https://go-review.googlesource.com/131376
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TestRawMessage now passes without the need for the RawMessage field to
be a pointer. The TODO dates all the way back to 2010, so I presume the
issue has since been fixed.
TestNullRawMessage tested the decoding of a JSON null into a
*RawMessage. The existing behavior was correct, but for the sake of
completeness a non-pointer RawMessage field has been added too. The
non-pointer field behaves differently, as one can read in the docs:
To unmarshal JSON into a value implementing the Unmarshaler
interface, Unmarshal calls that value's UnmarshalJSON method,
including when the input is a JSON null.
Change-Id: Iabaed75d4ed10ea427d135ee1b80c6e6b83b2e6e
Reviewed-on: https://go-review.googlesource.com/131377
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Encoders like map and array can use the much cheaper "i > 0" check to
see if we're not writing the first element. However, since struct fields
support omitempty, we need to keep track of that separately.
This is much more expensive - after calling the field encoder itself,
and retrieving the field via reflection, this branch was the third most
expensive piece of this field loop.
Instead, hoist the branch logic outside of the loop. The code doesn't
get much more complex, since we just delay the writing of each byte
until the next iteration. Yet the performance improvement is noticeable,
even when the struct types in CodeEncoder only have 2 and 7 fields,
respectively.
name old time/op new time/op delta
CodeEncoder-4 5.39ms ± 0% 5.31ms ± 0% -1.37% (p=0.010 n=4+6)
name old speed new speed delta
CodeEncoder-4 360MB/s ± 0% 365MB/s ± 0% +1.39% (p=0.010 n=4+6)
Updates #5683.
Change-Id: I2662cf459e0dfd68e56fa52bc898a417e84266c2
Reviewed-on: https://go-review.googlesource.com/131401
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A few encoder struct types, such as map and slice, only encapsulate
other prepared encoder funcs. Using pointer receivers has no advantage,
and makes calling these methods slightly more expensive.
Not a huge performance win, but certainly an easy one. The struct types
used in the benchmark below contain one slice field and one pointer
field.
name old time/op new time/op delta
CodeEncoder-4 5.48ms ± 0% 5.39ms ± 0% -1.66% (p=0.010 n=6+4)
name old speed new speed delta
CodeEncoder-4 354MB/s ± 0% 360MB/s ± 0% +1.69% (p=0.010 n=6+4)
Updates #5683.
Change-Id: I9f78dbe07fcc6fbf19a6d96c22f5d6970db9eca4
Reviewed-on: https://go-review.googlesource.com/131400
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
First, use a dummy slice access on decode64 and decode32 to ensure that
there is a single bounds check for src.
Second, move the PutUint64/PutUint32 calls out of these functions,
meaning that they are simpler and smaller. This may also open the door
to inlineability in the future, but for now, they both go past the
budget.
While at it, get rid of the ilen and olen variables, which have no
impact whatsoever on performance. At least, not measurable by any of the
benchmarks.
name old time/op new time/op delta
DecodeString/2-4 54.3ns ± 1% 55.2ns ± 2% +1.60% (p=0.017 n=5+6)
DecodeString/4-4 66.6ns ± 1% 66.8ns ± 2% ~ (p=0.903 n=6+6)
DecodeString/8-4 79.3ns ± 2% 79.6ns ± 1% ~ (p=0.448 n=6+6)
DecodeString/64-4 300ns ± 1% 281ns ± 3% -6.54% (p=0.002 n=6+6)
DecodeString/8192-4 27.4µs ± 1% 23.7µs ± 2% -13.47% (p=0.002 n=6+6)
name old speed new speed delta
DecodeString/2-4 73.7MB/s ± 1% 72.5MB/s ± 2% -1.55% (p=0.026 n=5+6)
DecodeString/4-4 120MB/s ± 1% 120MB/s ± 2% ~ (p=0.851 n=6+6)
DecodeString/8-4 151MB/s ± 2% 151MB/s ± 1% ~ (p=0.485 n=6+6)
DecodeString/64-4 292MB/s ± 1% 313MB/s ± 3% +7.03% (p=0.002 n=6+6)
DecodeString/8192-4 399MB/s ± 1% 461MB/s ± 2% +15.58% (p=0.002 n=6+6)
For #19636.
Change-Id: I0dfbdafa2a41dc4c582f63aef94b90b8e473731c
Reviewed-on: https://go-review.googlesource.com/113776
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Some WriteByte('\\') calls can be deduplicated.
fillField is used in two occasions, but it is unnecessary when adding
fields to the "next" stack, as those aren't used for the final encoding.
Inline the func with its only remaining call.
Finally, unindent a default-if block.
The performance of the encoder is unaffected:
name old time/op new time/op delta
CodeEncoder-4 6.65ms ± 1% 6.65ms ± 0% ~ (p=0.662 n=6+5)
Change-Id: Ie55baeab89abad9b9f13e9f6ca886a670c30dba9
Reviewed-on: https://go-review.googlesource.com/122461
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This function was only used in a single place - in the field encoding
loop within the struct encoder.
Inlining the function call manually lets us get rid of the call
overhead. But most importantly, it lets us simplify the logic afterward.
We no longer need to use reflect.Value{} and !fv.IsValid(), as we can
skip the field immediately.
The two factors combined (mostly just the latter) give a moderate speed
improvement to this hot loop.
name old time/op new time/op delta
CodeEncoder-4 6.01ms ± 1% 5.91ms ± 1% -1.66% (p=0.002 n=6+6)
name old speed new speed delta
CodeEncoder-4 323MB/s ± 1% 328MB/s ± 1% +1.69% (p=0.002 n=6+6)
Updates #5683.
Change-Id: I12757c325a68abb2856026cf719c122612a1f38e
Reviewed-on: https://go-review.googlesource.com/125417
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
structEncoder had two slices - the list of fields, and a list containing
the encoder for each field. structEncoder.encode then looped over the
fields, and indexed into the second slice to grab the field encoder.
However, this makes it very hard for the compiler to be able to prove
that the two slices always have the same length, and that the index
expression doesn't need a bounds check.
Merge the two slices into one to completely remove the need for bounds
checks in the hot loop.
While at it, don't copy the field elements when ranging, which greatly
speeds up the hot loop in structEncoder.
name old time/op new time/op delta
CodeEncoder-4 6.18ms ± 0% 5.56ms ± 0% -10.08% (p=0.002 n=6+6)
name old speed new speed delta
CodeEncoder-4 314MB/s ± 0% 349MB/s ± 0% +11.21% (p=0.002 n=6+6)
name old alloc/op new alloc/op delta
CodeEncoder-4 93.2kB ± 0% 62.1kB ± 0% -33.33% (p=0.002 n=6+6)
Updates #5683.
Change-Id: I0dd47783530f439b125e084aede09dda172eb1e8
Reviewed-on: https://go-review.googlesource.com/125416
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reuse v.Type() and cachedTypeFields(t) when decoding maps and structs.
Always use the same data slices when in hot loops, to ensure that the
compiler generates good code. "for i < len(data) { use(d.data[i]) }"
makes it harder for the compiler.
Finally, do other minor clean-ups, such as deduplicating switch cases,
and using a switch instead of three chained ifs.
The decoder sees a noticeable speed-up, in particular when decoding
structs.
name old time/op new time/op delta
CodeDecoder-4 29.8ms ± 1% 27.5ms ± 0% -7.83% (p=0.002 n=6+6)
name old speed new speed delta
CodeDecoder-4 65.0MB/s ± 1% 70.6MB/s ± 0% +8.49% (p=0.002 n=6+6)
Updates #5683.
Change-Id: I9d751e22502221962da696e48996ffdeb777277d
Reviewed-on: https://go-review.googlesource.com/122468
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Calling Name on a reflect.Type is somewhat expensive, as it involves a
number of nested calls and string handling.
This cost was showing up when decoding structs, as we were calling it to
set up an error context.
We can avoid the extra work unless we do encounter an error, which makes
decoding via struct types faster.
name old time/op new time/op delta
CodeDecoder-4 31.0ms ± 1% 29.9ms ± 1% -3.69% (p=0.002 n=6+6)
name old speed new speed delta
CodeDecoder-4 62.6MB/s ± 1% 65.0MB/s ± 1% +3.83% (p=0.002 n=6+6)
Updates #5683.
Change-Id: I48a3a85ef0ba96f524b7c3e9096cb2c4589e077a
Reviewed-on: https://go-review.googlesource.com/122467
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If the encoded bytes fit in the bootstrap array encodeState.scratch, use
that instead of allocating a new byte slice.
Also tweaked the Encoding vs Encoder heuristic to use the length of the
encoded bytes, not the length of the input bytes. Encoding is used for
allocations of up to 1024 bytes, as we measured 2048 to be the point
where it no longer provides a noticeable advantage.
Also added some benchmarks. Only the first case changes in behavior.
name old time/op new time/op delta
MarshalBytes/32-4 420ns ± 1% 383ns ± 1% -8.69% (p=0.002 n=6+6)
MarshalBytes/256-4 913ns ± 1% 915ns ± 0% ~ (p=0.580 n=5+6)
MarshalBytes/4096-4 7.72µs ± 0% 7.74µs ± 0% ~ (p=0.340 n=5+6)
name old alloc/op new alloc/op delta
MarshalBytes/32-4 112B ± 0% 64B ± 0% -42.86% (p=0.002 n=6+6)
MarshalBytes/256-4 736B ± 0% 736B ± 0% ~ (all equal)
MarshalBytes/4096-4 7.30kB ± 0% 7.30kB ± 0% ~ (all equal)
name old allocs/op new allocs/op delta
MarshalBytes/32-4 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.002 n=6+6)
MarshalBytes/256-4 2.00 ± 0% 2.00 ± 0% ~ (all equal)
MarshalBytes/4096-4 2.00 ± 0% 2.00 ± 0% ~ (all equal)
Updates #5683.
Change-Id: I5fa55c27bd7728338d770ae7c0756885ba9a5724
Reviewed-on: https://go-review.googlesource.com/122462
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Struct field names are static, so we can run HTMLEscape on them when
building each struct type encoder. Then, when running the struct
encoder, we can select either the original or the escaped field name to
write directly.
When the encoder is not escaping HTML, using the original string works
because neither Go struct field names nor JSON tags allow any characters
that would need to be escaped, like '"', '\\', or '\n'.
When the encoder is escaping HTML, the only difference is that '<', '>',
and '&' are allowed via JSON struct field tags, hence why we use
HTMLEscape to properly escape them.
All of the above lets us encode field names with a simple if/else and
WriteString calls, which are considerably simpler and faster than
encoding an arbitrary string.
While at it, also include the quotes and colon in these strings, to
avoid three WriteByte calls in the loop hot path.
Also added a few tests, to ensure that the behavior in these edge cases
is not broken. The output of the tests is the same if this optimization
is reverted.
name old time/op new time/op delta
CodeEncoder-4 7.12ms ± 0% 6.14ms ± 0% -13.85% (p=0.004 n=6+5)
name old speed new speed delta
CodeEncoder-4 272MB/s ± 0% 316MB/s ± 0% +16.08% (p=0.004 n=6+5)
name old alloc/op new alloc/op delta
CodeEncoder-4 91.9kB ± 0% 93.2kB ± 0% +1.43% (p=0.002 n=6+6)
name old allocs/op new allocs/op delta
CodeEncoder-4 0.00 0.00 ~ (all equal)
Updates #5683.
Change-Id: I6f6a340d0de4670799ce38cf95b2092822d2e3ef
Reviewed-on: https://go-review.googlesource.com/122460
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The existing Decoder.tokenError implementation creates its error messages by
concatenating "invalid character " + quoteChar(c) + " " + context. All context
values however already start with a space leading to error messages containing
two spaces.
This change removes " " from the concatenation expression.
Fixes#26587
Change-Id: I93d14319396636b2a40d55053bda88c98e94a81a
GitHub-Last-Rev: 6db7e1991b
GitHub-Pull-Request: golang/go#26588
Reviewed-on: https://go-review.googlesource.com/125775
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
They didn't even have public types, which made them pretty mysterious.
Give them types and reference the Decoder, which uses them.
Also, refer them qualified by their package name in the examples, as
we usually do in example*.go files, which usually use package foo_test
specifically so we can show the package names along with the symbols.
Change-Id: I50ebbbf43778c1627bfa526f8824f52c7953454f
Reviewed-on: https://go-review.googlesource.com/127663
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Each URL was manually verified to ensure it did not serve up incorrect
content.
Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Simplify the wording of both.
Make the DecodeString docs more accurate:
DecodeString returns a slice, not a string.
Change-Id: Iba7003f55fb0a37aafcbeee59a30492c0f68aa4e
Reviewed-on: https://go-review.googlesource.com/115615
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Unmarshal/Marshal/Unmarshal was not idempotent as the Object Identifier
type was not returned through the interface. The limit case OID = 0
returns an error. The zero OID is 0.0
A test is fixed to use the Object Identifier type.
Other related test are added.
Fixes#11130
Change-Id: I15483a3126066c9b99cf5bd9c4b0cc15ec1d61ca
Reviewed-on: https://go-review.googlesource.com/113837
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Immediately following the conditional block removed here is a loop
which checks exactly what the conditional already checked, so the
entire conditional is redundant.
Change-Id: I892fd9f2364d87e2c1cacb0407531daec6643183
Reviewed-on: https://go-review.googlesource.com/114000
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
ASN.1 has an private class, but current implementation does not support it.
Change-Id: I3ebf07a048831869572f75223cb17d4c115caef7
GitHub-Last-Rev: b3c69ad091
GitHub-Pull-Request: golang/go#25195
Reviewed-on: https://go-review.googlesource.com/110561
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
This change adds functionality to properly handle NoPadding in NewDecoder.
Removes the following expectations when using NoPadding:
* the input message length is a multiple of 8
* the input message length is 0, or longer than 7 characters
Fixes#25332
Change-Id: I7c38160df23f7e8da4f85a5629530016e7bf71f3
GitHub-Last-Rev: 68ab8d2291
GitHub-Pull-Request: golang/go#25394
Reviewed-on: https://go-review.googlesource.com/113215
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This changes makes encoder.Close aware of how many bytes to write if there
is any data left in the buffer.
Fixes#25295
Change-Id: I4138891359935509cb561c453b8059ba2b9e576b
GitHub-Last-Rev: f374096d2f
GitHub-Pull-Request: golang/go#25316
Reviewed-on: https://go-review.googlesource.com/112515
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This changes decoder.Read to always return io.ErrUnexpectedEOF if the input
contains surplus padding or unexpected content. Previously the error could
be io.EOF or io.ErrUnexpectedEOF depending on how the input was chunked.
Fixes#25296
Change-Id: I07c36c35e6c83e795c3991bfe45647a35aa58aa4
GitHub-Last-Rev: 818dfda90b
GitHub-Pull-Request: golang/go#25319
Reviewed-on: https://go-review.googlesource.com/112516
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Section 2.2 of the referenced spec http://www.xml.com/axml/testaxml.htm
defines 0xD7FF as a (sub)range boundary, not 0xDF77.
Fixes#25172
Change-Id: Ic5a3328cd46ef6474b8e93c4a343dcfba0e6511f
Reviewed-on: https://go-review.googlesource.com/109495
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
intDataSize should return length of bool slice, so functions
Read and Write can use the fast path to process bool slice.
Change-Id: I8cd275e3ffea82024850662d86caca64bd91bf70
Reviewed-on: https://go-review.googlesource.com/112135
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
The general policy for the current state of js/wasm is that it only
has to support tests that are also supported by nacl.
The test nilptr3.go makes assumptions about which nil checks can be
removed. Since WebAssembly does not signal on reading a null pointer,
all nil checks have to be explicit.
Updates #18892
Change-Id: I06a687860b8d22ae26b1c391499c0f5183e4c485
Reviewed-on: https://go-review.googlesource.com/110096
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes#18037
Change-Id: I20e27bcc013b00b726eb348daf5ca86b138ddcc2
Reviewed-on: https://go-review.googlesource.com/107598
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fixes golint warning about "if block ends with a return statement, so drop this else and outdent its block".
Change-Id: Id17ad0bf37ba939386b177b709e9e3c067d8ba21
Reviewed-on: https://go-review.googlesource.com/105736
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The go/printer (and thus gofmt) uses a heuristic to determine
whether to break alignment between elements of an expression
list which is spread across multiple lines. The heuristic only
kicked in if the entry sizes (character length) was above a
certain threshold (20) and the ratio between the previous and
current entry size was above a certain value (4).
This heuristic worked reasonably most of the time, but also
led to unfortunate breaks in many cases where a single entry
was suddenly much smaller (or larger) then the previous one.
The behavior of gofmt was sufficiently mysterious in some of
these situations that many issues were filed against it.
The simplest solution to address this problem is to remove
the heuristic altogether and have a programmer introduce
empty lines to force different alignments if it improves
readability. The problem with that approach is that the
places where it really matters, very long tables with many
(hundreds, or more) entries, may be machine-generated and
not "post-processed" by a human (e.g., unicode/utf8/tables.go).
If a single one of those entries is overlong, the result
would be that the alignment would force all comments or
values in key:value pairs to be adjusted to that overlong
value, making the table hard to read (e.g., that entry may
not even be visible on screen and all other entries seem
spaced out too wide).
Instead, we opted for a slightly improved heuristic that
behaves much better for "normal", human-written code.
1) The threshold is increased from 20 to 40. This disables
the heuristic for many common cases yet even if the alignment
is not "ideal", 40 is not that many characters per line with
todays screens, making it very likely that the entire line
remains "visible" in an editor.
2) Changed the heuristic to not simply look at the size ratio
between current and previous line, but instead considering the
geometric mean of the sizes of the previous (aligned) lines.
This emphasizes the "overall picture" of the previous lines,
rather than a single one (which might be an outlier).
3) Changed the ratio from 4 to 2.5. Now that we ignore sizes
below 40, a ratio of 4 would mean that a new entry would have
to be 4 times bigger (160) or smaller (10) before alignment
would be broken. A ratio of 2.5 seems more sensible.
Applied updated gofmt to all of src and misc. Also tested
against several former issues that complained about this
and verified that the output for the given examples is
satisfactory (added respective test cases).
Some of the files changed because they were not gofmt-ed
in the first place.
For #644.
For #7335.
For #10392.
(and probably more related issues)
Fixes#22852.
Change-Id: I5e48b3d3b157a5cf2d649833b7297b33f43a6f6e
As found by unparam. Picked the low-hanging fruit, consisting only of
errors that were always nil and results that were never used. Left out
those that were useful for consistency with other func signatures.
Change-Id: I06b52bbd3541f8a5d66659c909bd93cb3e172018
Reviewed-on: https://go-review.googlesource.com/102418
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
I found files to change with this command:
git grep 'DO NOT EDIT' | grep -v 'Code generated .* DO NOT'
There are more files that match that grep, but I do not intend on fixing
them.
Change-Id: I4b474f1c29ca3135560d414785b0dbe0d1a4e52c
GitHub-Last-Rev: 65804b0263
GitHub-Pull-Request: golang/go#24334
Reviewed-on: https://go-review.googlesource.com/99955
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The index 248 results in the decoder calling reflect.MakeMapWithSize
with a size of 14754407682 - just under 15GB - which ends up in a
runtime out of memory panic after some recent runtime changes on
machines with 8GB of memory.
Until that is fixed in either runtime or gob, skip the troublesome
index.
Updates #24308.
Change-Id: Ia450217271c983e7386ba2f3f88c9ba50aa346f4
Reviewed-on: https://go-review.googlesource.com/99655
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
'"' has special semantic meaning that conflicts with using it as Comma.
Change-Id: Ife25ba43ca25dba2ea184c1bb7579a230d376059
Reviewed-on: https://go-review.googlesource.com/99696
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In the situation where a quoted field is necessary, avoid processing
each UTF-8 rune one-by-one, which causes mangling of invalid sequences
into utf8.RuneError, causing a loss of information.
Instead, search only for the escaped characters, handle those specially
and copy everything else in between verbatim.
This symmetrically matches the behavior of Reader.
Fixes#24298
Change-Id: I9276f64891084ce8487678f663fad711b4095dbb
Reviewed-on: https://go-review.googlesource.com/99297
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
While running make.bash, over 5% of all pointer writes
come from encoding/binary doing struct reads.
This change replaces slicing during such reads with an offset.
This avoids updating the slice pointer with every
struct field read or write.
This has no impact when the write barrier is off.
Running the benchmarks with GOGC=1, however,
shows significant improvement:
name old time/op new time/op delta
ReadStruct-8 13.2µs ± 6% 10.1µs ± 5% -23.24% (p=0.000 n=10+10)
name old speed new speed delta
ReadStruct-8 5.69MB/s ± 6% 7.40MB/s ± 5% +30.18% (p=0.000 n=10+10)
Change-Id: I22904263196bfeddc38abe8989428e263aee5253
Reviewed-on: https://go-review.googlesource.com/98757
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The previous type cache is quadratic in time in the situation where
new types are continually encountered. Now that it is possible to dynamically
create new types with the reflect package, this can cause json to
perform very poorly.
Switch to sync.Map which does well when the cache has hit steady state,
but also handles occasional updates in better than quadratic time.
benchmark old ns/op new ns/op delta
BenchmarkTypeFieldsCache/MissTypes1-8 14817 16202 +9.35%
BenchmarkTypeFieldsCache/MissTypes10-8 70926 69144 -2.51%
BenchmarkTypeFieldsCache/MissTypes100-8 976467 208973 -78.60%
BenchmarkTypeFieldsCache/MissTypes1000-8 79520162 1750371 -97.80%
BenchmarkTypeFieldsCache/MissTypes10000-8 6873625837 16847806 -99.75%
BenchmarkTypeFieldsCache/HitTypes1000-8 7.51 8.80 +17.18%
BenchmarkTypeFieldsCache/HitTypes10000-8 7.58 8.68 +14.51%
The old implementation takes 12 minutes just to build a cache of size 1e5
due to the quadratic behavior. I did not bother benchmark sizes above that.
Change-Id: I5e6facc1eb8e1b80e5ca285e4dd2cc8815618dad
Reviewed-on: https://go-review.googlesource.com/76850
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Consider the following:
type child struct{ Field string }
type parent struct{ child }
p := new(parent)
v := reflect.ValueOf(p).Elem().Field(0)
v.Field(0).SetString("hello") // v.Field = "hello"
v = v.Addr().Elem() // v = *(&v)
v.Field(0).SetString("goodbye") // v.Field = "goodbye"
It would appear that v.Addr().Elem() should have the same value, and
that it would be safe to set "goodbye".
However, after CL 66331, any interspersed calls between Field calls
causes the RO flag to be set.
Thus, setting to "goodbye" actually causes a panic.
That CL affects decodeState.indirect which assumes that back-to-back
Value.Addr().Elem() is side-effect free. We fix that logic to keep
track of the Addr() and Elem() calls and set v back to the original
after a full round-trip has occured.
Fixes#24152
Updates #24153
Change-Id: Ie50f8fe963f00cef8515d89d1d5cbc43b76d9f9c
Reviewed-on: https://go-review.googlesource.com/97796
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It appears that old code (from 2009) in xml.(*Decoder).rawToken
replicates append's slice-growing functionality by allocating a new,
bigger backing array and then calling copy.
Simplifying the code by replacing it with a single append call does
not seem to hurt performance:
name old time/op new time/op delta
Marshal-4 11.2µs ± 1% 11.3µs ±10% ~ (p=0.069 n=19+17)
Unmarshal-4 28.6µs ± 1% 28.4µs ± 1% -0.60% (p=0.000 n=20+18)
name old alloc/op new alloc/op delta
Marshal-4 5.78kB ± 0% 5.78kB ± 0% ~ (all equal)
Unmarshal-4 8.61kB ± 0% 8.27kB ± 0% -3.90% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
Marshal-4 23.0 ± 0% 23.0 ± 0% ~ (all equal)
Unmarshal-4 189 ± 0% 190 ± 0% +0.53% (p=0.000 n=20+20)
Change-Id: Ie580d1216a44760e611e63dee2c339af5465aea5
Reviewed-on: https://go-review.googlesource.com/86655
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Rather than only ignoring runtime.Error panics, which are a very
narrow set of possible panic values, switch it such that the json
package only captures panic values that have been properly wrapped
in a jsonError struct. This ensures that only intentional panics
originating from the json package are captured.
Fixes#23012
Change-Id: I5e85200259edd2abb1b0512ce6cc288849151a6d
Reviewed-on: https://go-review.googlesource.com/94019
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL reverts CL 76851 and takes a different approach to #21357.
The changes in encode.go and encode_test.go are reverts that
rolls back the changed behavior in CL 76851 where
embedded pointers to unexported struct types were
unilaterally ignored in both marshal and unmarshal.
Instead, these fields are handled as before with the exception that
it returns an error when Unmarshal is unable to set an unexported field.
The behavior of Marshal is now unchanged with regards to #21357.
This policy maintains the greatest degree of backwards compatibility
and avoids silently discarding data the user may have expected to be present.
Fixes#21357
Change-Id: I7dc753280c99f786ac51acf7e6c0246618c8b2b1
Reviewed-on: https://go-review.googlesource.com/82135
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Originally these routines could not fail except by
returning errors from the underlying writer.
Then we realized that header keys containing colons
needed to be rejected, and we started returning an error
from Encode. But that only happens after writing a
partial PEM block to the underlying writer, which is
unfortunate, but at least it was undocumented.
CL 77790 then documented this unfortunate behavior.
Instead of documenting unfortunate behavior, fix it.
Change-Id: Ic7467a576c4cecd16a99138571a1269cc4f96204
Reviewed-on: https://go-review.googlesource.com/82076
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This fixes a regression where only CRLF was folded into LF at EOF.
Now, we also truncate trailing CR at EOF to preserve the old behavior.
Every one of the test cases added exactly matches the behavior
of Go1.9, even if the results are somewhat unexpected.
Fixes#22937
Change-Id: I1bc6550533163ae489ea77ec1e598163267b7eec
Reviewed-on: https://go-review.googlesource.com/81577
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
There are, unfortunately, intermediate CA ceritificates in circulation
that contain the invalid character '&' in some PrintableString fields,
notably Organization Name. This patch allows for ampersand
to be parsed as though it is valid in an ASN.1 PrintableString.
Fixes#22970
Change-Id: Ifab1a10bbff1cdac68e843c6b857ff1a031051aa
Reviewed-on: https://go-review.googlesource.com/81635
Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This reverts commit 08f19bbde1.
Reason for revert:
The changed transformation takes effect on a larger set
of code snippets than expected.
For example, this:
func foo() {
// Comment
bar()
}
becomes:
func foo() {
// Comment
bar()
}
This is an unintended consequence.
Change-Id: Ifca88d6267dab8a8170791f7205124712bf8ace8
Reviewed-on: https://go-review.googlesource.com/81335
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <joetsai@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It was added in CL 79995. It is unnecessarily confusing.
Change-Id: Ib8ff35b9f71b54ff99d2d6e0534c7128e1f4345a
Reviewed-on: https://go-review.googlesource.com/80035
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
ASN.1 has an specific string type, called NumericString (tag 18). The
value of this type can be numeric characters (0-9) and space.
Fixes#22396
Change-Id: Ia6d81ab7faa311ff22759bf76862626974d3013e
Reviewed-on: https://go-review.googlesource.com/78655
Run-TryBot: Adam Langley <agl@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 52810 changed Reader to interpret a quoted \r\n as a raw \r\n
when reading fields. This seems likely to break existing users, and
discussion on both #21201 (the original issue that triggered the change)
and #22746 (discussing whether to revert the change) failed to identify
a single motivating example for this change. To avoid breaking existing
users for no clear reason, revert the change.
The Reader has been rewritten in the interim so this is not a git revert
but instead and adjustment (and slight simplification) of the new Reader.
Fixes#22746.
Change-Id: Ie857b2f4b1359a207d085b6d3c3a6d440a997d12
Reviewed-on: https://go-review.googlesource.com/78295
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
CL 70210 added Decoder for #21590, and in doing so it changed
the existing func Decode to return partial results for decoding errors.
That seems like a good change to make to Decode, but it was
untested (except as used by Decoder), inconsistent with DecodeString
in all error cases, and inconsistent with Decoder in not returning
partial results for odd-length input strings.
This CL makes Decode, DecodeString, and Decoder all agree about
the handling of partial results (they are returned) and error
precedence (the error earliest in the input is reported),
and it documents and tests this.
Change-Id: Ifb7d1e100ecb66fe2ed5ba34a621084d480f16db
Reviewed-on: https://go-review.googlesource.com/78120
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 58210 introduced this constant for reasons I don't understand.
It should not be in the exported const block, which will pollute
godoc output with a "... unexported" notice.
Also since we already have a constant named xmlnsPrefix for "xmlns",
it is very confusing to also have xmlNamespacePrefix for "xml".
If we must have the constant at all, rename it to xmlPrefix.
Change-Id: I15f937454d730005816fcd32b1acca703acf1e51
Reviewed-on: https://go-review.googlesource.com/78121
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
A record can span multiple lines (the whole reason for the extra field),
so the important fact is that it's the _start_ of the record.
Make that clear in the name.
(This API was added during the Go 1.10 cycle so it can still be cleaned up.)
Change-Id: Id95b3ceb7cdfc4aa0ed5a053cb84da8945fa5496
Reviewed-on: https://go-review.googlesource.com/78119
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Mainly get rid of the weird zero-value struct literal,
but while we're here also group and order things a bit better:
first the reader, then the data, then the call (which takes reader then data).
Change-Id: I901b0661d85d8eaa0807e4482aac66500ca996c7
Reviewed-on: https://go-review.googlesource.com/78118
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
CL 60410 fixes a bug in reflect that allows assignments to an embedded
field of a pointer to an unexported struct type.
This breaks the json package because unmarshal is now unable to assign
a newly allocated struct to such fields.
In order to be consistent in the behavior for marshal and unmarshal,
this CL changes both marshal and unmarshal to always ignore
embedded pointers to unexported structs.
Fixes#21357
Change-Id: If62ea11155555e61115ebb9cfa5305caf101bde5
Reviewed-on: https://go-review.googlesource.com/76851
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The docs for xml.Marshal state that the XML elements name is derived
from one of five locations in a specific order of precedence, but does
not mention that if the field is a struct type and has its name defined
in a tag and in the types XMLName field that an error will occur. This
is documented in the structFieldInfo function but not in the function
documentation, and the existing docs in Marshal are misleading without
this behavior being discussed.
Fixes#18564
Change-Id: I29042f124a534bd1bc993f1baeddaa0af2e72fed
Reviewed-on: https://go-review.googlesource.com/76321
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The go repository contains a mix of github.com/golang/go/issues/xxxxx
and golang.org/issues/xxxxx URLs for references to issues in the issue
tracker. We should use one for consistency, and golang.org is preferred
in case the project moves the issue tracker in the future.
This reasoning is taken from a comment Sam Whited left on a CL I
recently opened: https://go-review.googlesource.com/c/go/+/73890.
In that CL I referenced an issue using its github.com URL, because other
tests in the file I was changing contained references to issues using
their github.com URL. Sam Whited left a comment on the CL stating I
should change it to the golang.org URL.
If new code is intended to reference issues via golang.org and not
github.com, existing code should be updated so that precedence exists
for contributors who are looking at the existing code as a guide for the
code they should write.
Change-Id: I3b9053fe38a1c56fc101a8b7fd7b8f310ba29724
Reviewed-on: https://go-review.googlesource.com/75673
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds new rules to recognize consecutive byte loads and
stores and lowers them to loads and stores such as lhz, lwz, ld,
sth, stw, std. This change only covers the little endian cases
on little endian machines, such as is found in encoding/binary
UintXX or PutUintXX for little endian. Big endian will be done
later.
Updates were also made to binary_test.go to allow the benchmark
for Uint and PutUint to actually use those functions because
the way they were written, those functions were being
optimized out.
Testcases were also added to cmd/compile/internal/gc/asm_test.go.
Updates #22496
The following improvement can be found in golang.org/x/crypto
poly1305:
Benchmark64-16 142 114 -19.72%
Benchmark1K-16 1717 1424 -17.06%
Benchmark64Unaligned-16 142 113 -20.42%
Benchmark1KUnaligned-16 1721 1428 -17.02%
chacha20poly1305:
BenchmarkChacha20Poly1305Open_64-16 1012 885 -12.55%
BenchmarkChacha20Poly1305Seal_64-16 971 836 -13.90%
BenchmarkChacha20Poly1305Open_1350-16 11113 9539 -14.16%
BenchmarkChacha20Poly1305Seal_1350-16 11013 9392 -14.72%
BenchmarkChacha20Poly1305Open_8K-16 61074 53431 -12.51%
BenchmarkChacha20Poly1305Seal_8K-16 61214 54806 -10.47%
Other improvements of around 10% found in crypto/tls.
Results after updating encoding/binary/binary_test.go:
BenchmarkLittleEndianPutUint64-16 1.87 0.93 -50.27%
BenchmarkLittleEndianPutUint32-16 1.19 0.93 -21.85%
BenchmarkLittleEndianPutUint16-16 1.16 1.03 -11.21%
Change-Id: I7bbe2fbcbd11362d58662fecd907a0c07e6ca2fb
Reviewed-on: https://go-review.googlesource.com/74410
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Munday <mike.munday@ibm.com>
To improve readability when exported fields are removed,
forbid the printer from emitting an empty line before the first comment
in a const, var, or type block.
Also, when printing the "Has filtered or unexported fields." message,
add an empty line before it to separate the message from the struct
or interfact contents.
Before the change:
<<<
type NamedArg struct {
// Name is the name of the parameter placeholder.
//
// If empty, the ordinal position in the argument list will be
// used.
//
// Name must omit any symbol prefix.
Name string
// Value is the value of the parameter.
// It may be assigned the same value types as the query
// arguments.
Value interface{}
// contains filtered or unexported fields
}
>>>
After the change:
<<<
type NamedArg struct {
// Name is the name of the parameter placeholder.
//
// If empty, the ordinal position in the argument list will be
// used.
//
// Name must omit any symbol prefix.
Name string
// Value is the value of the parameter.
// It may be assigned the same value types as the query
// arguments.
Value interface{}
// contains filtered or unexported fields
}
>>>
Fixes#18264
Change-Id: I9fe17ca39cf92fcdfea55064bd2eaa784ce48c88
Reviewed-on: https://go-review.googlesource.com/71990
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Whitespace is ignored in bool values and attrs. It is convenient and
relatively safe since whitespace around a bool value is often
unimportant. The same logic can be applied to numeric values of types
int, uint, and float.
Fixes#22146
Change-Id: Ie0462def90304af144b8e2e72d85b644857c27cc
Reviewed-on: https://go-review.googlesource.com/73891
Reviewed-by: Sam Whited <sam@samwhited.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Whitespace is ignored in bool values and attrs, but there are no tests
capturing this behavior.
Change-Id: I7a7249de4886f510869e91de937e69b83c3254c8
Reviewed-on: https://go-review.googlesource.com/73890
Reviewed-by: Sam Whited <sam@samwhited.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When a SyntaxError occurs, report the current offset within the stream.
The code already accounted for the offset within the current buffer
being scanned. By including how much data was already scanned, the
current offset can be computed.
Fixes#22478
Change-Id: I91ecd4cad0b85a5c1556bc597f3ee914e769af01
Reviewed-on: https://go-review.googlesource.com/74251
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add a DisallowUnknownFields flag to Decoder.
DisallowUnknownFields causes the Decoder to return an error when
the the decoding destination is a struct and the input contains
object keys which do not match any non-ignored, public field the
destination, including keys whose value is set to null.
Note: this fix has already been worked on in 27231, which seems
to be abandoned. This version is a slightly simpler implementation
and is up to date with the master branch.
Fixes#15314
Change-Id: I987a5857c52018df334f4d1a2360649c44a7175d
Reviewed-on: https://go-review.googlesource.com/74830
Reviewed-by: Joe Tsai <joetsai@google.com>
Run-TryBot: Joe Tsai <joetsai@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The record delimiter (not configurable by user) is "\r\n" or "\n".
It is insensible for the user to set Comma or Comment delimiters
to be some character that conflicts with the record delimiter.
Furthermore, it is insensible for Comma or Comment to be the same rune.
Allowing this leaks implementation details to the user in regards to
the evaluation order of which rune is checked for first.
Fixes#22404
Change-Id: I31e86abc9b3a8fb4584e090477795587740970ae
Reviewed-on: https://go-review.googlesource.com/72793
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The ErrQuote variable is only returned when a parsing error
occurs within a quoted string. Make that clear in the message.
Change-Id: I06ad5a9edb41afedde193c4f8b93551bb8342bbb
Reviewed-on: https://go-review.googlesource.com/72794
Reviewed-by: Avelino <t@avelino.xxx>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We should be referring to ParseError.Err, which is the underlying error,
not ParseError.Error, which is the error method.
Change-Id: Ic3cef5ecbe1ada5fa14b9573222f29da8fc9a8d5
Reviewed-on: https://go-review.googlesource.com/72450
Reviewed-by: Tim Cooper <tim.cooper@layeh.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 72150 fixes#22352 by reverting the problematic parts of that CL
where the line number and column number were inconsistent with each other.
This CL adds back functionality to address the issue that CL 72150
was trying to solve in the first place. That is, it reports the starting
line of the record, so that users have a frame of reference to start with
when debugging what went wrong.
In the event of gnarly CSV files with multiline quoted strings, a parse
failure likely occurs somewhere between the start of the record and
the point where the parser finally detected an error.
Since ParserError.{Line,Column} reports where the *error* occurs, we
add a RecordLine field to report where the record starts.
Also take this time to cleanup and modernize TestRead.
Fixes#19019Fixes#22352
Change-Id: I16cebf0b81922c35f75804c7073e9cddbfd11a04
Reviewed-on: https://go-review.googlesource.com/72310
Reviewed-by: Ian Lance Taylor <iant@golang.org>
NewEncoder returns an io.Writer that writes all incoming bytes as
hexadecimal characters to the underlying io.Writer. NewDecoder returns an
io.Reader that does the inverse.
Fixes#21590
Change-Id: Iebe0813faf365b42598f19a9aa41768f571dc0a8
Reviewed-on: https://go-review.googlesource.com/70210
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The Reader implementation is slow because it operates on a rune-by-rune
basis via bufio.Reader.ReadRune. We speed this up by operating on entire
lines that we read from bufio.Reader.ReadSlice.
In order to ensure that we read the full line, we augment ReadSlice
in our Reader.readLine method to automatically expand the slice if
bufio.ErrBufferFull is every hit.
This change happens to fix#19410 because it no longer relies on
rune-by-rune parsing and only searches for the relevant delimiter rune.
In order to keep column accounting simple and consistent, this change
reverts parts of CL 52830.
This CL is an alternative to CL 36270 and builds on some of the ideas
from that change by Diogo Pinela.
name old time/op new time/op delta
Read-8 3.12µs ± 1% 2.54µs ± 2% -18.76% (p=0.000 n=10+9)
ReadWithFieldsPerRecord-8 3.12µs ± 1% 2.53µs ± 1% -18.91% (p=0.000 n=9+9)
ReadWithoutFieldsPerRecord-8 3.13µs ± 0% 2.57µs ± 3% -18.07% (p=0.000 n=10+10)
ReadLargeFields-8 52.3µs ± 1% 5.3µs ± 2% -89.93% (p=0.000 n=10+9)
ReadReuseRecord-8 2.05µs ± 1% 1.40µs ± 1% -31.48% (p=0.000 n=10+9)
ReadReuseRecordWithFieldsPerRecord-8 2.05µs ± 1% 1.41µs ± 0% -31.03% (p=0.000 n=10+9)
ReadReuseRecordWithoutFieldsPerRecord-8 2.06µs ± 1% 1.40µs ± 1% -31.70% (p=0.000 n=9+10)
ReadReuseRecordLargeFields-8 50.9µs ± 0% 4.1µs ± 3% -92.01% (p=0.000 n=10+10)
name old alloc/op new alloc/op
Read-8 664B ± 0% 664B ± 0%
ReadWithFieldsPerRecord-8 664B ± 0% 664B ± 0%
ReadWithoutFieldsPerRecord-8 664B ± 0% 664B ± 0%
ReadLargeFields-8 3.94kB ± 0% 3.94kB ± 0%
ReadReuseRecord-8 24.0B ± 0% 24.0B ± 0%
ReadReuseRecordWithFieldsPerRecord-8 24.0B ± 0% 24.0B ± 0%
ReadReuseRecordWithoutFieldsPerRecord-8 24.0B ± 0% 24.0B ± 0%
ReadReuseRecordLargeFields-8 2.98kB ± 0% 2.98kB ± 0%
name old allocs/op new allocs/op
Read-8 18.0 ± 0% 18.0 ± 0%
ReadWithFieldsPerRecord-8 18.0 ± 0% 18.0 ± 0%
ReadWithoutFieldsPerRecord-8 18.0 ± 0% 18.0 ± 0%
ReadLargeFields-8 24.0 ± 0% 24.0 ± 0%
ReadReuseRecord-8 8.00 ± 0% 8.00 ± 0%
ReadReuseRecordWithFieldsPerRecord-8 8.00 ± 0% 8.00 ± 0%
ReadReuseRecordWithoutFieldsPerRecord-8 8.00 ± 0% 8.00 ± 0%
ReadReuseRecordLargeFields-8 12.0 ± 0% 12.0 ± 0%
Updates #22352
Updates #19019Fixes#16791Fixes#19410
Change-Id: I31c27cfcc56880e6abac262f36c947179b550bbf
Reviewed-on: https://go-review.googlesource.com/72150
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In #10909, it was decided that "Deprecated:" is a magic string for
tools (e.g., #17056 for godoc) to detect deprecated identifiers.
Use those convention instead of custom written prose.
Change-Id: Ia514fc3c88fc502e86c6e3de361c435f4cb80b22
Reviewed-on: https://go-review.googlesource.com/70110
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
The '*' character is not allowed in ASN.1 PrintableString. However, due
to wide-spread use, we permit it so that we can parse many certificates
with wildcards. However, that also meant that generic strings with
asterisks in would be encoded as PrintableString.
This change makes the default for such strings to be UTF8String. Thus,
while the code PrintableStrings containing '*', it will not generate
them unless the string type was specified in the struct field tag.
Change-Id: I2d458da36649427352eeaa50a1b6020108b2ccbd
Reviewed-on: https://go-review.googlesource.com/68990
Reviewed-by: Adam Langley <agl@golang.org>
Previously, any “explicit” and/or “tag” decorations on a RawValue would
be ignored when unmarshaling. The RawValue would swallow whatever
element was encountered.
This change causes these decorations to be respected. Thus a field like:
Foo asn1.RawValue `asn1:"explicit,tag:1,optional"`
will only match if an explicit tag with value one is encountered.
Otherwise the RawValue will get the default value and parsing will move
onto the next element.
Thanks to Martin Kreichgauer for reporting the issue.
Change-Id: If6c4488685b9bd039cb5e352d6d75744f98dbb1f
Reviewed-on: https://go-review.googlesource.com/34503
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
Remove an old comment introduced in golang.org/cl/9073.
Change-Id: I14be27ddfac987f44d839920bc4d02361a576f06
Reviewed-on: https://go-review.googlesource.com/66371
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
bytes.IndexByte can be used wherever the second argument to
strings.Index is exactly one byte long, so we do that with this change.
This avoids generating unnecessary string symbols/converison and saves
a few calls to bytes.Index.
Change-Id: If31c775790e01edfece1169e398ad6a754fb4428
Reviewed-on: https://go-review.googlesource.com/66373
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.LastIndexByte was introduced in go1.5 and it can be used
effectively wherever the second argument to strings.LastIndex is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.LastIndex.
Change-Id: I7b5679d616197b055cffe6882a8675d24a98b574
Reviewed-on: https://go-review.googlesource.com/66372
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.IndexByte was introduced in go1.2 and it can be used
effectively wherever the second argument to strings.Index is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.Index.
Change-Id: I1ab5edb7c4ee9058084cfa57cbcc267c2597e793
Reviewed-on: https://go-review.googlesource.com/65930
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 60410 fixes the compiler such that reflect.StructField.PkgPath
is non-empty if and only if the field is unexported.
Given that property, we can cleanup the logic in the json encoder
to avoid parsing the field name to detect export properties.
Updates #21122
Change-Id: Ic01b9c4ca76386774846b742b0c1b9b948f53e7c
Reviewed-on: https://go-review.googlesource.com/65550
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Make arguments semantics clear without the need to look for
json.Indent documentation.
Change-Id: If9adfe9f477a30d426ae83790b0f2578c0a809b7
Reviewed-on: https://go-review.googlesource.com/61670
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Found with mvdan.cc/unindent. It skipped the cases where parentheses
would need to be added, where comments would have to be moved elsewhere,
or where actions and simple logic would mix.
One of them was of the form "err != nil && err == io.EOF", so the first
part was removed.
Change-Id: Ie504c2b03a2c87d10ecbca1b9270069be1171b91
Reviewed-on: https://go-review.googlesource.com/57690
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Most of these are return values that were part of a receiving parameter,
so they're still accessible.
A few others are not, but those have never had a use.
Found with github.com/mvdan/unparam, after Kevin Burke's suggestion that
the tool should also warn about unused result parameters.
Change-Id: Id8b5ed89912a99db22027703a88bd94d0b292b8b
Reviewed-on: https://go-review.googlesource.com/55910
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Went mainly for the ones that make no sense, such as the ones
mid-sentence or after commas.
Change-Id: Ie245d2c19cc7428a06295635cf6a9482ade25ff0
Reviewed-on: https://go-review.googlesource.com/57293
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The destination slice does not need to be created at all. The source
slice itself can be used as the destination because the decode loop
increments by one and then the 'seen' byte is not used anymore. Therefore
the decoded byte can be stored in that index of the source slice itself.
This trick cannot be applied to EncodeString() because in that case,
the destination slice is large than the source. And for a single byte
in the source slice, two bytes in the destination slice is written.
func BenchmarkDecodeString(b *testing.B) {
for i := 0; i < b.N; i++ {
DecodeString("0123456789abcdef")
}
}
name old time/op new time/op delta
DecodeString 71.0ns ± 6% 58.0ns ± 0% -18.28% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
DecodeString 16.0B ± 0% 8.0B ± 0% -50.00% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
DecodeString 1.00 ± 0% 1.00 ± 0% ~ (all equal)
Change-Id: Id98db4e712444557a804155457a4dd8d1b8b416d
Reviewed-on: https://go-review.googlesource.com/55611
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The parser mistakenly assumed it could always fold \r\n into \n, which
is not true since a \r\n inside a quoted fields has no special meaning
and should be kept as is.
Fix this by not folding \r\n to \n inside quotes fields.
Fixes#21201
Change-Id: Ifebc302e49cf63e0a027ee90f088dbc050a2b7a6
Reviewed-on: https://go-review.googlesource.com/52810
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The tests for error scenarios were done by manually checking
error strings. Improved them by checking the actual error type
instead of just the string.
Printing the actual error in case of failure instead of a
generic string.
Also added a new scenario with both an invalid byte and an
invalid length string to verify that the length is checked first
before doing any computation.
Change-Id: Ic2a19a6d6058912632d597590186ee2d8348cb45
Reviewed-on: https://go-review.googlesource.com/55256
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Errors returned by Reader contain the line where the Reader originally
encountered the error. This can be suboptimal since that line does not
always correspond with the line the current record/field started at.
This can easily happen with LazyQuotes as seen in #19019, but also
happens for example when a quoted fields has no closing quote and
the parser hits EOF before it finds another quote.
When this happens finding the erroneous field can be somewhat
complicated and time consuming, and in most cases it would be better to
report the line where the record started.
This change updates Reader to keep track of the line on which a record
begins and uses it for errors instead of the current line, making it
easier to find errors.
Although a user-visible change, this should have no impact on existing
code, since most users don't explicitly work with the line in the error
and probably already expect the new behaviour.
Updates #19019
Change-Id: Ic9bc70fad2651c69435d614d537e7a9266819b05
Reviewed-on: https://go-review.googlesource.com/52830
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I27ff99aa7abb070f6ae79c8f964aa9bd6a83b89d
Reviewed-on: https://go-review.googlesource.com/53730
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change fixes the remaining examples where the raw strings had
suboptimal indentation (one level too many) when viewed in godoc.
Follows CL 48910.
Fixes#21026.
Change-Id: Ifc0dae3fa899a9fff8b1ff958414e2fe6852321d
Reviewed-on: https://go-review.googlesource.com/50990
Run-TryBot: Dmitri Shuralyov <shurcool@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
https://golang.org/cl/33773 fixes the JSON marshaler to avoid serializing
embedded fields on unexported types of non-struct types. However, Go allows
embedding pointer to types, so the check for whether the field is a non-struct
type must first dereference the pointer to get at the underlying type.
Furthermore, due to a edge-case in the behavior of StructField.PkgPath not
being a reliable indicator of whether the field is unexported (see #21122),
we use our own logic to determine whether the field is exported or not.
The logic in this CL may be simplified depending on what happens in #21122.
Fixes#21121
Updates #21122
Change-Id: I8dfd1cdfac8a87950df294a566fb96dfd04fd749
Reviewed-on: https://go-review.googlesource.com/50711
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The existing example for Decoder.Decode (Stream) had excessive
indentation in the godoc interface for the const jsonStream,
making it hard to read. This fixes the indentation in the
example_test.go to improve the readability in godoc.
Helps #21026.
Change-Id: I16f56b82182da1dcc73cca44e535a7f5695e975d
Reviewed-on: https://go-review.googlesource.com/48910
Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 47341 added support for decoding non-padded messages. But DecodedLen
still returned a multiple of 5 for messages without a padding, even
though it is possible to calculate the len exactly when using NoPadding.
This change makes DecodedLen return the exact number of bytes that
will be written. A change to the decoding logic is also made so that it
can handle this case.
DecodedLen now has the same behaviour as DecodedLen in encoding/base64.
Fixes#20854
Change-Id: I729e0b1c0946c866fb675c854f835f366dd4b5a4
Reviewed-on: https://go-review.googlesource.com/47710
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
And some double space after period cleanup while I'm here.
I guess my previous regexps missed these. My next cleaner should
probably use go/ast instead of perl.
Updates #20221
Change-Id: Idb051e7ac3a7fb1fb86e015f709e32139d065d92
Reviewed-on: https://go-review.googlesource.com/47094
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Use 2 slashes, space, then tab. This is more consistent, and removes
inadvertent leading space.
Change-Id: I383770ed4eb8ac17c78c7ae5675b553d4fb70b1e
Reviewed-on: https://go-review.googlesource.com/46726
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
ascii85_test.go contains a variable called bigtest that is used as
test data for TestDecoderBuffering and TestEncoderBuffering. The
variable is initialised to a copy of the last element of the pairs
slice. When the variable was first added the last element of this
slice contained a sizable test case, 342 encoded characters. However,
https://golang.org/cl/5970078 added a new element to the end of the pairs
slice without updating bigtest. As the new element contained only 1 byte
of encoded data bigtest became very small test. This commit fixes the
problem by resetting bigtest to its original value and making its
initialisation independent of the layout of pairs. All the unit tests
still pass.
Change-Id: If7fb609ced9da93a2321dfd8372986b2fa772fd5
Reviewed-on: https://go-review.googlesource.com/46475
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The ascii85, base32 and base64 packages all contain a test called
TestDecoderBuffering. Each of these tests contain a loop that ignores
the error returned from the Read method of their decoders. The result
being that the tests loop for ever if the decoders actually return an
error. This commit fixes the issue by terminating the loops if an error
occurs and failing the tests with a suitable error message.
Change-Id: Idb385673cf9f3f6f8befe4288b4be366ab0985fd
Reviewed-on: https://go-review.googlesource.com/46010
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some of the _test.go files in the encoding packages contain a private
function called testEqual that calls testing.Errorf if the arguments
passed to it are unequal. The line numbers output by such calls to
Errorf identify the failure as being in testEqual itself which is not
very useful. This commit fixes the problem by adding a call to the
new t.Helper method in each of the testEqual functions. The line
numbers output when errors do occur now identify the real source of
the error.
Change-Id: I582d1934f40ef2b788116c3811074c67ea882021
Reviewed-on: https://go-review.googlesource.com/45871
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Marshal must process unexported embedded fields of struct type,
looking for exported fields in those structs. However, it must
not process unexported embedded fields of non-struct type.
For example, consider:
type t1 struct {
X int
}
type t2 int
type T struct {
t1
t2
}
When considering T, Marshal must process t1 to find t1.X.
Marshal must not process t2, but it was. Fix that.
Fixes#18009
Change-Id: I62ba0b65ba30fd927990e101a26405a9998787a3
Reviewed-on: https://go-review.googlesource.com/33773
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
A number of issues in decoder.Read and newlineFilteringReader.Read were
preventing errors from the reader supplying the encoded data from being
propagated to the caller. Fixing these issues revealed some additional
problems in which valid decoded data was not always returned to the user
when errors were actually propagated.
This commit fixes both the error propagation and the lost decoded data
problems. It also adds some new unit tests to ensure errors are handled
correctly by decoder.Read. The new unit tests increase the test coverage
of this package from 96.2% to 97.9%.
Fixes#20044
Change-Id: I1a8632da20135906e2d191c2a8825b10e7ecc4c5
Reviewed-on: https://go-review.googlesource.com/42094
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In many cases the records returned by Reader.Read will only be used between calls
to Read and become garbage once a new record is read. In this case, instead of
allocating a new slice on each call to Read, we can reuse the last allocated slice
for successive calls to avoid unnecessary allocations.
This change adds a new field ReuseRecord to the Reader struct to enable this reuse.
ReuseRecord is false by default to avoid breaking existing code which dependss on
the current behaviour.
I also added 4 new benchmarks, corresponding to the existing Read benchmarks, which
set ReuseRecord to true.
Benchstat on my local machine (old is ReuseRecord = false, new is ReuseRecord = true)
name old time/op new time/op delta
Read-8 2.75µs ± 2% 1.88µs ± 1% -31.52% (p=0.000 n=14+15)
ReadWithFieldsPerRecord-8 2.75µs ± 0% 1.89µs ± 1% -31.43% (p=0.000 n=13+13)
ReadWithoutFieldsPerRecord-8 2.77µs ± 1% 1.88µs ± 1% -32.06% (p=0.000 n=15+15)
ReadLargeFields-8 55.4µs ± 1% 54.2µs ± 0% -2.07% (p=0.000 n=15+14)
name old alloc/op new alloc/op delta
Read-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadLargeFields-8 3.94kB ± 0% 2.98kB ± 0% -24.39% (p=0.000 n=15+15)
name old allocs/op new allocs/op delta
Read-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadLargeFields-8 24.0 ± 0% 12.0 ± 0% -50.00% (p=0.000 n=15+15)
Fixes#19721
Change-Id: I79b14128bb9bb3465f53f40f93b1b528a9da6f58
Reviewed-on: https://go-review.googlesource.com/41730
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Mostly unnecessary *testing.T arguments.
Found with github.com/mvdan/unparam.
Change-Id: Ifb955cb88f2ce8784ee4172f4f94d860fa36ae9a
Reviewed-on: https://go-review.googlesource.com/41691
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There were a number of places in crypto/x509 that used hardcoded
representations of the ASN.1 NULL type, in both byte slice and
RawValue struct forms. This change adds two new exported vars to
the asn1 package for working with ASN.1 NULL in both its forms, and
converts all usages from the x509 package.
In addition, tests were added to exercise Marshal and Unmarshal on
both vars.
See #19446 for discussion.
Change-Id: I63dbd0835841ccbc810bd6ec794360a84e933f1e
Reviewed-on: https://go-review.googlesource.com/38660
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
Returns at the end of func bodies where the funcs have no return values
are pointless.
Change-Id: I0da5ea78671503e41a9f56dd770df8c919310ce5
Reviewed-on: https://go-review.googlesource.com/41093
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 27254 changed hextable to a byte array for performance.
CL 28219 fixed the compiler so that that is no longer necessary.
As Kirill notes in #15808, a string is preferable
as the linker can easily de-dup it.
So go back. No performance changes.
Change-Id: Ibef7d21d0f2507968a0606602c5dd57ed4a85b1b
Reviewed-on: https://go-review.googlesource.com/40970
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The current implementation uses a max of 28 bits when decoding an
ObjectIdentifier. This change makes it so that an int64 is used to
accumulate up to 35 bits. If the resulting data would not overflow
an int32, it is used as an int. Thus up to 31 bits may be used to
represent each subidentifier of an ObjectIdentifier.
Fixes#19933
Change-Id: I95d74b64b24cdb1339ff13421055bce61c80243c
Reviewed-on: https://go-review.googlesource.com/40436
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
The header is literally
Key: Value
If the value or the key has leading or trailing spaces, those will
be lost by the round trip.
Found because testing/quick returns different values now.
Change-Id: I0f574bdbb5990689509c24309854d8f814b5efa0
Reviewed-on: https://go-review.googlesource.com/39211
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When unmarshaling, if an element is empty, eg. '<tag></tag>', and
destination type is int, uint, float or bool, do not attempt to parse
value (""). Set to its zero value instead.
Fixes#13417
Change-Id: I2d79f6d8f39192bb277b1a9129727d5abbb2dd1f
Reviewed-on: https://go-review.googlesource.com/38386
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I2d155c838935cd8427abd142a462ff4c56829715
Reviewed-on: https://go-review.googlesource.com/37948
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This paragraph has been added, as the notion was missing from the
documentation.
If a value is passed to Encode and the type is not a struct (or pointer to struct,
etc.), for simplicity of processing it is represented as a struct of one field.
The only visible effect of this is to encode a zero byte after the value, just as
after the last field of an encoded struct, so that the decode algorithm knows when
the top-level value is complete.
Fixes#16978
Change-Id: I5f008e792d1b6fe80d2e026a7ff716608889db32
Reviewed-on: https://go-review.googlesource.com/38414
Reviewed-by: Ian Lance Taylor <iant@golang.org>
JSON decoding performs poorly for unmapped and ignored fields. We noticed better
performance when unmarshalling unused fields. The loss comes mostly from calls
to scanner.error as described at #17914.
benchmark old ns/op new ns/op delta
BenchmarkIssue10335-8 431 408 -5.34%
BenchmarkUnmapped-8 1744 1314 -24.66%
benchmark old allocs new allocs delta
BenchmarkIssue10335-8 4 3 -25.00%
BenchmarkUnmapped-8 18 4 -77.78%
benchmark old bytes new bytes delta
BenchmarkIssue10335-8 320 312 -2.50%
BenchmarkUnmapped-8 568 344 -39.44%
Fixes#17914, improves #10335
Change-Id: I7d4258a94eb287c0fe49e7334795209b90434cd0
Reviewed-on: https://go-review.googlesource.com/33276
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Tinkering with the gob package shows that is currently possible to
*completely destroy* Int slices encoding without triggering a single
test failure.
The various encInt{8,16,32,64}Slice methods are only called during the
execution of the GobMapInterfaceEncode test, which only encodes a few
slices of length exactly 1 and then just checks that the error
returned by Encode is nil (without trying to Decode back the data).
This patch adds a few tests for signed integer slices encoding.
Change-Id: Ifaaee2f32132873118b241f79aa8203e4ad31416
Reviewed-on: https://go-review.googlesource.com/38066
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Replace 'does not contains' with 'does not contain' where it appears
in the source code.
Change-Id: Ie7266347c429512c8a41a7e19142afca7ead3922
Reviewed-on: https://go-review.googlesource.com/37887
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Document and check that the alphabet cannot contain '\n' or '\r'.
Document that the alphabet cannot contain the padding character.
Document that the padding character must be equal or bellow '\xff'.
Document that the padding character must not be '\n' or '\r'.
Fixes#19343Fixes#19318
Change-Id: I6de0034d347ffdf317d7ea55d6fe38b01c2c4c03
Reviewed-on: https://go-review.googlesource.com/37838
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix Decode to return the correct illegal data index from a corrupted
input that contains whitespaces.
Fixes#19406
Change-Id: Ib2b2b6ed7e41f024d0da2bd035caec4317c2869c
Reviewed-on: https://go-review.googlesource.com/37837
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Found by github.com/mvdan/unparam.
Change-Id: Ic97f05a2ecb5b17caa36aafe403e2266abea3e0e
Reviewed-on: https://go-review.googlesource.com/37836
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Found by github.com/mvdan/unparam.
Change-Id: I5a6664cceeba1cf1c2f3236ddf4db5ce7a64b02a
Reviewed-on: https://go-review.googlesource.com/37835
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously the code didn't check for extra data after the final five
dashes of the ending line of a PEM block.
Fixes#19147Fixes#7042
Change-Id: Idaab2390914a2bed8c2c12b14dfb6d68233fdfec
Reviewed-on: https://go-review.googlesource.com/37147
Reviewed-by: Adam Langley <agl@golang.org>
The new tests in this CL have been checked against Go 1.7 as well
and all pass in Go 1.7, with the one exception noted in a comment
(an intentional change to omitempty already present before this CL).
CL 15684 made the intentional change to omitempty.
This CL fixes bugs introduced along the way.
Most of these are corner cases that are arguably not that important,
but they've always worked all the way back to Go 1, and someone
cared enough to file #19063. The most significant problem found
while adding tests is that in the case of a nil *string field with
`xml:",chardata"`, the existing code silently stops processing not just
that field but the entire remainder of the struct.
Even if #19063 were not worth fixing, this chardata bug would be.
Fixes#19063.
Change-Id: I318cf8f9945e1a4615982d9904e109fde577ebf9
Reviewed-on: https://go-review.googlesource.com/36954
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
... and same for stores. This does for binary.BigEndian.Uint16() what
was already done for Uint32 and Uint64 with BSWAP in 10f75748 (CL 32222).
Here is how generated code changes e.g. for the following function
(omitting saying the same prologue/epilogue):
func get16(b [2]byte) uint16 {
return binary.BigEndian.Uint16(b[:])
}
"".get16 t=1 size=21 args=0x10 locals=0x0
// before
0x0000 00000 (x.go:15) MOVBLZX "".b+9(FP), AX
0x0005 00005 (x.go:15) MOVBLZX "".b+8(FP), CX
0x000a 00010 (x.go:15) SHLL $8, CX
0x000d 00013 (x.go:15) ORL CX, AX
// after
0x0000 00000 (x.go:15) MOVWLZX "".b+8(FP), AX
0x0005 00005 (x.go:15) ROLW $8, AX
encoding/binary is speedup overall a bit:
name old time/op new time/op delta
ReadSlice1000Int32s-4 4.83µs ± 0% 4.83µs ± 0% ~ (p=0.206 n=4+5)
ReadStruct-4 1.29µs ± 2% 1.28µs ± 1% -1.27% (p=0.032 n=4+5)
ReadInts-4 384ns ± 1% 385ns ± 1% ~ (p=0.968 n=4+5)
WriteInts-4 534ns ± 3% 526ns ± 0% -1.54% (p=0.048 n=4+5)
WriteSlice1000Int32s-4 5.02µs ± 0% 5.11µs ± 3% ~ (p=0.175 n=4+5)
PutUint16-4 0.59ns ± 0% 0.49ns ± 2% -16.95% (p=0.016 n=4+5)
PutUint32-4 0.52ns ± 0% 0.52ns ± 0% ~ (all equal)
PutUint64-4 0.53ns ± 0% 0.53ns ± 0% ~ (all equal)
PutUvarint32-4 19.9ns ± 0% 19.9ns ± 1% ~ (p=0.556 n=4+5)
PutUvarint64-4 54.5ns ± 1% 54.2ns ± 0% ~ (p=0.333 n=4+5)
name old speed new speed delta
ReadSlice1000Int32s-4 829MB/s ± 0% 828MB/s ± 0% ~ (p=0.190 n=4+5)
ReadStruct-4 58.0MB/s ± 2% 58.7MB/s ± 1% +1.30% (p=0.032 n=4+5)
ReadInts-4 78.0MB/s ± 1% 77.8MB/s ± 1% ~ (p=0.968 n=4+5)
WriteInts-4 56.1MB/s ± 3% 57.0MB/s ± 0% ~ (p=0.063 n=4+5)
WriteSlice1000Int32s-4 797MB/s ± 0% 783MB/s ± 3% ~ (p=0.190 n=4+5)
PutUint16-4 3.37GB/s ± 0% 4.07GB/s ± 2% +20.83% (p=0.016 n=4+5)
PutUint32-4 7.73GB/s ± 0% 7.72GB/s ± 0% ~ (p=0.556 n=4+5)
PutUint64-4 15.1GB/s ± 0% 15.1GB/s ± 0% ~ (p=0.905 n=4+5)
PutUvarint32-4 201MB/s ± 0% 201MB/s ± 0% ~ (p=0.905 n=4+5)
PutUvarint64-4 147MB/s ± 1% 147MB/s ± 0% ~ (p=0.286 n=4+5)
( "a bit" only because most of the time is spent in reflection-like things
there, not actual bytes decoding. Even for direct PutUint16 benchmark the
looping adds overhead and lowers visible benefit. For code-generated encoders /
decoders actual effect is more than 20% )
Adding Uint32 and Uint64 raw benchmarks too for completeness.
NOTE I had to adjust load-combining rule for bswap case to match first 2 bytes
loads as result of "2-bytes load+shift" -> "loadw + rorw 8" rewrite. Reason is:
for loads+shift, even e.g. into uint16 var
var b []byte
var v uin16
v = uint16(b[1]) | uint16(b[0])<<8
the compiler eventually generates L(ong) shift - SHLLconst [8], probably
because it is more straightforward / other reasons to work on the whole
register. This way 2 bytes rewriting rule is using SHLLconst (not SHLWconst) in
its pattern, and then it always gets matched first, even if 2-byte rule comes
syntactically after 4-byte rule in AMD64.rules because 4-bytes rule seemingly
needs more applyRewrite() cycles to trigger. If 2-bytes rule gets matched for
inner half of
var b []byte
var v uin32
v = uint32(b[3]) | uint32(b[2])<<8 | uint32(b[1])<<16 | uint32(b[0])<<24
and we keep 4-byte load rule unchanged, the result will be MOVW + RORW $8 and
then series of byte loads and shifts - not one MOVL + BSWAPL.
There is no such problem for stores: there compiler, since it probably knows
store destination is 2 bytes wide, uses SHRWconst 8 (not SHRLconst 8) and thus
2-byte store rule is not a subset of rule for 4-byte stores.
Fixes#17151 (int16 was last missing piece there)
Change-Id: Idc03ba965bfce2b94fef456b02ff6742194748f6
Reviewed-on: https://go-review.googlesource.com/34636
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Mention that it specifically returns x / 2, and do the same for
EncodedLen.
Change-Id: Ie334f5abecbc487caf4965abbcd14442591bef2a
Change-Id: Idfa413faad487e534489428451bf736b009293d6
Reviewed-on: https://go-review.googlesource.com/33191
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The tree is inconsistent about single l vs double l in those
words in documentation, test messages, and one error value text.
$ git grep -E '[Mm]arshall(|s|er|ers|ed|ing)' | wc -l
42
$ git grep -E '[Mm]arshal(|s|er|ers|ed|ing)' | wc -l
1694
Make it consistently a single l, per earlier decisions. This means
contributors won't be confused by misleading precedence, and it helps
consistency.
Change the spelling in one error value text in newRawAttributes of
crypto/x509 package to be consistent.
This change was generated with:
perl -i -npe 's,([Mm]arshal)l(|s|er|ers|ed|ing),$1$2,' $(git grep -l -E '[Mm]arshall' | grep -v AUTHORS | grep -v CONTRIBUTORS)
Updates #12431.
Follows https://golang.org/cl/14150.
Change-Id: I85d28a2d7692862ccb02d6a09f5d18538b6049a2
Reviewed-on: https://go-review.googlesource.com/33017
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We only support unmarshaling into a string or a []byte, but we
previously would try (and panic while) setting a slice of a different
type. The docs say ",innerxml" is ignored if the type is not string or
[]byte, so do that for other slices as well.
Fixes#15600.
Change-Id: Ia64815945a14c3d04a0a45ccf413e38b58a69416
Reviewed-on: https://go-review.googlesource.com/32919
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
I used the slowtests.go tool as described in
https://golang.org/cl/32684 on packages that stood out.
go test -short std drops from ~56 to ~52 seconds.
This isn't a huge win, but it was mostly an exercise.
Updates #17751
Change-Id: I9f3402e36a038d71e662d06ce2c1d52f6c4b674d
Reviewed-on: https://go-review.googlesource.com/32751
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL expands upon a change made in (http://golang.org/cl/21811)
to ensure that a nil RawMessage gets serialized as "null" instead of
being a nil slice.
The added check only triggers when the RawMessage is nil. We do not
handle the case when the RawMessage is non-nil, but empty.
Fixes#17704
Updates #14493
Change-Id: I0fbebcdd81f7466c5b78c94953afc897f162ceb4
Reviewed-on: https://go-review.googlesource.com/32472
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
- Like ",any" for elements, add ",any,attr" for attributes to allow
a mop-up field that gets any otherwise unmapped attributes.
- Map attributes to fields of type slice by extending the slice,
just like for elements.
- Allow storing an attribute into an xml.Attr directly, to provide
a way to record the name.
Combined, these three independent features allow
AllAttrs []Attr `xml:",any,attr"`
to collect all attributes not otherwise spoken for in a particular struct.
Tests based on CL 16292 by Charles Weill.
Fixes#3633.
Change-Id: I2d75817f17ca8752d7df188080a407836af92611
Reviewed-on: https://go-review.googlesource.com/30946
Reviewed-by: Quentin Smith <quentin@golang.org>
1. Define behavior for Unmarshal of JSON null into Unmarshaler and
TextUnmarshaler. Specifically, an Unmarshaler will be given the
literal null and can decide what to do (because otherwise
json.RawMessage is impossible to implement), and a TextUnmarshaler
will be skipped over (because there is no text to unmarshal), like
most other inappropriate types. Document this in Unmarshal, with a
reminder in UnmarshalJSON about handling null.
2. Test all this.
3. Fix the TextUnmarshaler case, which was returning an unmarshalling
error, to match the definition.
4. Fix the error that had been used for the TextUnmarshaler, since it
was claiming that there was a JSON string when in fact the problem was
NOT having a string.
5. Adjust time.Time and big.Int's UnmarshalJSON to ignore null, as is
conventional.
Fixes#9037.
Change-Id: If78350414eb8dda712867dc8f4ca35a9db041b0c
Reviewed-on: https://go-review.googlesource.com/30944
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
No functional changes here. Just makes next CL easier to read.
Change-Id: Icf7b2281b4da6cb59ff4edff05943b2ee288576a
Reviewed-on: https://go-review.googlesource.com/30945
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There was an inconsistency between the (json encoding + documentation)
and the xml encoding implementation. Pointer to an empty value was
not being serialized (i.e simply ignored). Which had the effect of making
impossible to have a struct with a string field for which we wanted to
serialize the value ""
Fixes#5452
Change-Id: Id858701801158409be01e962d2cda843424bd22a
Reviewed-on: https://go-review.googlesource.com/15684
Reviewed-by: Russ Cox <rsc@golang.org>
If strict option is enabled, when decoding, instead of skip the padding
bits, it will do strict check to enforce they are set to zero.
Fixes#15656
Change-Id: I869fb725a39cc9dde44dbc4ff0046446e7abc642
Reviewed-on: https://go-review.googlesource.com/24964
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Struct fields can be suppressed in JSON serialization by "-" tags, but
that doesn't preclude generation of "-" object keys.
Document and verify the mechanism for doing so.
Change-Id: I7f60e1759cfee15cb7b2447cd35fab91c5b004e6
Reviewed-on: https://go-review.googlesource.com/21204
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The UnmarshalTypeError has two new fields Struct and Field,
used when constructing the error message.
Fixes#6716.
Change-Id: I67da171480a9491960b3ae81893770644180f848
Reviewed-on: https://go-review.googlesource.com/18692
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change float32/float64 formatting to use non-exponential form
for a slightly wider range, to more closely match ES6 JSON.stringify
and other JSON generators.
Most notably:
1e20 now formats as 100000000000000000000 (previously 1e+20)
1e-6 now formats as 0.000001 (previously 1e-06)
1e-7 now formats as 1e-7 (previously 1e-07)
This also brings the int64 and float64 formatting in line with each other,
for all shared representable values. For example both int64(1234567)
and float64(1234567) now format as "1234567", where before the
float64 formatted as "1.234567e+06".
The only variation now compared to ES6 JSON.stringify is that
Go continues to encode negative zero as "-0", not "0", so that
the value continues to be preserved during JSON round trips.
Fixes#6384.
Fixes#14135.
Change-Id: Ib0e0e009cd9181d75edc0424a28fe776bcc5bbf8
Reviewed-on: https://go-review.googlesource.com/30371
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Russ Cox
Shulhan
Damien Neil
Brad Fitzpatrick
Romain Baugue
LE Manh Cuong
Romain Baugue
Daniel Martí
Rob Pike
Brian Kessler
Leon Klingele
Marcel van Lohuizen
Ketan Parmar
GuilhermeCaruso
Mostyn Bramley-Moore
Leigh McCulloch
Santhosh Kumar Tekuri
hearot
Mihai Todor
Gabriel Aszalos
Ian Davis
Josh Bleecher Snyder
Ivan Kutuzov
Taesu Pyo
Tim Cooper
Philip Børgesen
Ian Lance Taylor
Niek Sanders
Tobias Klauser
Constantin Konstantinidis
dchenk
HAMANO Tsukasa
Gustav Westling
Artyom Pervukhin
Joe Kyo
Richard Musiol
tengufromsky
Michael Fraenkel
Jeremy Jackins
bontequero
Robert Griesemer
Mark Rushakoff
Joe Tsai
Pascal S. de Kloe
Ryuma Yoshida
Alberto Donizetti
Igor Vashyst
christopher-henderson
Joe Tsai
rajender
Mansour Rahimi
Michael Schurter
Hiroshi Ioka
Lynn Boger
Ivan Bertona
Sam Whited
Adam Langley
Josselin Costanzi
Laurent Voisin
Marvin Stenger
Kunpei Sakai
tbunyk
Danny Rosseau
Karel Pazdera
Agniva De Sarker
Ilya Tocar
Justin Nuß
Mark Wolfe
Dmitri Shuralyov
Axel Wagner
Iccha Sethi
Ross Light
Mark Ryan
thoeni
Filip Gruszczyński
Gustav Westling
Bryan C. Mills
Andrew Benton
Monis Khan
Sarah Adams
Kevin Burke
Joe Shaw
Kirill Smelkov
Tuo Shan
Matt Layher
Thordur Bjornsson
Emmanuel Odeke
Quentin Smith
Victor Vrantchan
Allan Simon
Xuyang Kang
Richard Gibson
Jirka Daněk