CL 60410 fixes a bug in reflect that allows assignments to an embedded
field of a pointer to an unexported struct type.
This breaks the json package because unmarshal is now unable to assign
a newly allocated struct to such fields.
In order to be consistent in the behavior for marshal and unmarshal,
this CL changes both marshal and unmarshal to always ignore
embedded pointers to unexported structs.
Fixes#21357
Change-Id: If62ea11155555e61115ebb9cfa5305caf101bde5
Reviewed-on: https://go-review.googlesource.com/76851
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The go repository contains a mix of github.com/golang/go/issues/xxxxx
and golang.org/issues/xxxxx URLs for references to issues in the issue
tracker. We should use one for consistency, and golang.org is preferred
in case the project moves the issue tracker in the future.
This reasoning is taken from a comment Sam Whited left on a CL I
recently opened: https://go-review.googlesource.com/c/go/+/73890.
In that CL I referenced an issue using its github.com URL, because other
tests in the file I was changing contained references to issues using
their github.com URL. Sam Whited left a comment on the CL stating I
should change it to the golang.org URL.
If new code is intended to reference issues via golang.org and not
github.com, existing code should be updated so that precedence exists
for contributors who are looking at the existing code as a guide for the
code they should write.
Change-Id: I3b9053fe38a1c56fc101a8b7fd7b8f310ba29724
Reviewed-on: https://go-review.googlesource.com/75673
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When a SyntaxError occurs, report the current offset within the stream.
The code already accounted for the offset within the current buffer
being scanned. By including how much data was already scanned, the
current offset can be computed.
Fixes#22478
Change-Id: I91ecd4cad0b85a5c1556bc597f3ee914e769af01
Reviewed-on: https://go-review.googlesource.com/74251
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add a DisallowUnknownFields flag to Decoder.
DisallowUnknownFields causes the Decoder to return an error when
the the decoding destination is a struct and the input contains
object keys which do not match any non-ignored, public field the
destination, including keys whose value is set to null.
Note: this fix has already been worked on in 27231, which seems
to be abandoned. This version is a slightly simpler implementation
and is up to date with the master branch.
Fixes#15314
Change-Id: I987a5857c52018df334f4d1a2360649c44a7175d
Reviewed-on: https://go-review.googlesource.com/74830
Reviewed-by: Joe Tsai <joetsai@google.com>
Run-TryBot: Joe Tsai <joetsai@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In #10909, it was decided that "Deprecated:" is a magic string for
tools (e.g., #17056 for godoc) to detect deprecated identifiers.
Use those convention instead of custom written prose.
Change-Id: Ia514fc3c88fc502e86c6e3de361c435f4cb80b22
Reviewed-on: https://go-review.googlesource.com/70110
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
Remove an old comment introduced in golang.org/cl/9073.
Change-Id: I14be27ddfac987f44d839920bc4d02361a576f06
Reviewed-on: https://go-review.googlesource.com/66371
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
bytes.IndexByte can be used wherever the second argument to
strings.Index is exactly one byte long, so we do that with this change.
This avoids generating unnecessary string symbols/converison and saves
a few calls to bytes.Index.
Change-Id: If31c775790e01edfece1169e398ad6a754fb4428
Reviewed-on: https://go-review.googlesource.com/66373
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.IndexByte was introduced in go1.2 and it can be used
effectively wherever the second argument to strings.Index is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.Index.
Change-Id: I1ab5edb7c4ee9058084cfa57cbcc267c2597e793
Reviewed-on: https://go-review.googlesource.com/65930
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 60410 fixes the compiler such that reflect.StructField.PkgPath
is non-empty if and only if the field is unexported.
Given that property, we can cleanup the logic in the json encoder
to avoid parsing the field name to detect export properties.
Updates #21122
Change-Id: Ic01b9c4ca76386774846b742b0c1b9b948f53e7c
Reviewed-on: https://go-review.googlesource.com/65550
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Make arguments semantics clear without the need to look for
json.Indent documentation.
Change-Id: If9adfe9f477a30d426ae83790b0f2578c0a809b7
Reviewed-on: https://go-review.googlesource.com/61670
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Most of these are return values that were part of a receiving parameter,
so they're still accessible.
A few others are not, but those have never had a use.
Found with github.com/mvdan/unparam, after Kevin Burke's suggestion that
the tool should also warn about unused result parameters.
Change-Id: Id8b5ed89912a99db22027703a88bd94d0b292b8b
Reviewed-on: https://go-review.googlesource.com/55910
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Went mainly for the ones that make no sense, such as the ones
mid-sentence or after commas.
Change-Id: Ie245d2c19cc7428a06295635cf6a9482ade25ff0
Reviewed-on: https://go-review.googlesource.com/57293
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change fixes the remaining examples where the raw strings had
suboptimal indentation (one level too many) when viewed in godoc.
Follows CL 48910.
Fixes#21026.
Change-Id: Ifc0dae3fa899a9fff8b1ff958414e2fe6852321d
Reviewed-on: https://go-review.googlesource.com/50990
Run-TryBot: Dmitri Shuralyov <shurcool@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
https://golang.org/cl/33773 fixes the JSON marshaler to avoid serializing
embedded fields on unexported types of non-struct types. However, Go allows
embedding pointer to types, so the check for whether the field is a non-struct
type must first dereference the pointer to get at the underlying type.
Furthermore, due to a edge-case in the behavior of StructField.PkgPath not
being a reliable indicator of whether the field is unexported (see #21122),
we use our own logic to determine whether the field is exported or not.
The logic in this CL may be simplified depending on what happens in #21122.
Fixes#21121
Updates #21122
Change-Id: I8dfd1cdfac8a87950df294a566fb96dfd04fd749
Reviewed-on: https://go-review.googlesource.com/50711
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The existing example for Decoder.Decode (Stream) had excessive
indentation in the godoc interface for the const jsonStream,
making it hard to read. This fixes the indentation in the
example_test.go to improve the readability in godoc.
Helps #21026.
Change-Id: I16f56b82182da1dcc73cca44e535a7f5695e975d
Reviewed-on: https://go-review.googlesource.com/48910
Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Marshal must process unexported embedded fields of struct type,
looking for exported fields in those structs. However, it must
not process unexported embedded fields of non-struct type.
For example, consider:
type t1 struct {
X int
}
type t2 int
type T struct {
t1
t2
}
When considering T, Marshal must process t1 to find t1.X.
Marshal must not process t2, but it was. Fix that.
Fixes#18009
Change-Id: I62ba0b65ba30fd927990e101a26405a9998787a3
Reviewed-on: https://go-review.googlesource.com/33773
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
JSON decoding performs poorly for unmapped and ignored fields. We noticed better
performance when unmarshalling unused fields. The loss comes mostly from calls
to scanner.error as described at #17914.
benchmark old ns/op new ns/op delta
BenchmarkIssue10335-8 431 408 -5.34%
BenchmarkUnmapped-8 1744 1314 -24.66%
benchmark old allocs new allocs delta
BenchmarkIssue10335-8 4 3 -25.00%
BenchmarkUnmapped-8 18 4 -77.78%
benchmark old bytes new bytes delta
BenchmarkIssue10335-8 320 312 -2.50%
BenchmarkUnmapped-8 568 344 -39.44%
Fixes#17914, improves #10335
Change-Id: I7d4258a94eb287c0fe49e7334795209b90434cd0
Reviewed-on: https://go-review.googlesource.com/33276
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The tree is inconsistent about single l vs double l in those
words in documentation, test messages, and one error value text.
$ git grep -E '[Mm]arshall(|s|er|ers|ed|ing)' | wc -l
42
$ git grep -E '[Mm]arshal(|s|er|ers|ed|ing)' | wc -l
1694
Make it consistently a single l, per earlier decisions. This means
contributors won't be confused by misleading precedence, and it helps
consistency.
Change the spelling in one error value text in newRawAttributes of
crypto/x509 package to be consistent.
This change was generated with:
perl -i -npe 's,([Mm]arshal)l(|s|er|ers|ed|ing),$1$2,' $(git grep -l -E '[Mm]arshall' | grep -v AUTHORS | grep -v CONTRIBUTORS)
Updates #12431.
Follows https://golang.org/cl/14150.
Change-Id: I85d28a2d7692862ccb02d6a09f5d18538b6049a2
Reviewed-on: https://go-review.googlesource.com/33017
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
I used the slowtests.go tool as described in
https://golang.org/cl/32684 on packages that stood out.
go test -short std drops from ~56 to ~52 seconds.
This isn't a huge win, but it was mostly an exercise.
Updates #17751
Change-Id: I9f3402e36a038d71e662d06ce2c1d52f6c4b674d
Reviewed-on: https://go-review.googlesource.com/32751
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL expands upon a change made in (http://golang.org/cl/21811)
to ensure that a nil RawMessage gets serialized as "null" instead of
being a nil slice.
The added check only triggers when the RawMessage is nil. We do not
handle the case when the RawMessage is non-nil, but empty.
Fixes#17704
Updates #14493
Change-Id: I0fbebcdd81f7466c5b78c94953afc897f162ceb4
Reviewed-on: https://go-review.googlesource.com/32472
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
1. Define behavior for Unmarshal of JSON null into Unmarshaler and
TextUnmarshaler. Specifically, an Unmarshaler will be given the
literal null and can decide what to do (because otherwise
json.RawMessage is impossible to implement), and a TextUnmarshaler
will be skipped over (because there is no text to unmarshal), like
most other inappropriate types. Document this in Unmarshal, with a
reminder in UnmarshalJSON about handling null.
2. Test all this.
3. Fix the TextUnmarshaler case, which was returning an unmarshalling
error, to match the definition.
4. Fix the error that had been used for the TextUnmarshaler, since it
was claiming that there was a JSON string when in fact the problem was
NOT having a string.
5. Adjust time.Time and big.Int's UnmarshalJSON to ignore null, as is
conventional.
Fixes#9037.
Change-Id: If78350414eb8dda712867dc8f4ca35a9db041b0c
Reviewed-on: https://go-review.googlesource.com/30944
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Struct fields can be suppressed in JSON serialization by "-" tags, but
that doesn't preclude generation of "-" object keys.
Document and verify the mechanism for doing so.
Change-Id: I7f60e1759cfee15cb7b2447cd35fab91c5b004e6
Reviewed-on: https://go-review.googlesource.com/21204
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The UnmarshalTypeError has two new fields Struct and Field,
used when constructing the error message.
Fixes#6716.
Change-Id: I67da171480a9491960b3ae81893770644180f848
Reviewed-on: https://go-review.googlesource.com/18692
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change float32/float64 formatting to use non-exponential form
for a slightly wider range, to more closely match ES6 JSON.stringify
and other JSON generators.
Most notably:
1e20 now formats as 100000000000000000000 (previously 1e+20)
1e-6 now formats as 0.000001 (previously 1e-06)
1e-7 now formats as 1e-7 (previously 1e-07)
This also brings the int64 and float64 formatting in line with each other,
for all shared representable values. For example both int64(1234567)
and float64(1234567) now format as "1234567", where before the
float64 formatted as "1.234567e+06".
The only variation now compared to ES6 JSON.stringify is that
Go continues to encode negative zero as "-0", not "0", so that
the value continues to be preserved during JSON round trips.
Fixes#6384.
Fixes#14135.
Change-Id: Ib0e0e009cd9181d75edc0424a28fe776bcc5bbf8
Reviewed-on: https://go-review.googlesource.com/30371
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
I avoided anywhere in the compiler or things which might be used by
the compiler in the future, since they need to build with Go 1.4.
I also avoided anywhere where there was no benefit to changing it.
I probably missed some.
Updates #16721
Change-Id: Ib3c895ff475c6dec2d4322393faaf8cb6a6d4956
Reviewed-on: https://go-review.googlesource.com/30250
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Documentation made reference to an unknown entity "DisableHTMLEscaping,"
but I think it actually meant the method "Encoder.SetEscapeHTML."
Fixes#17255
Change-Id: I18fda76f8066110caef85fd33698de83d632e646
Reviewed-on: https://go-review.googlesource.com/29931
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The previous check for characters inside of a JSON string that needed
to be escaped performed seven different boolean comparisons before
determining that a ASCII character did not need to be escaped. Most
characters do not need to be escaped, so this check can be done in a
more performant way.
Use the same strategy as the unicode package for precomputing a range
of characters that need to be escaped, then do a single lookup into a
character array to determine whether the character needs escaping.
On an AWS c4.large node:
$ benchstat benchmarks/master-bench benchmarks/json-table-bench
name old time/op new time/op delta
CodeEncoder-2 19.0ms ± 0% 15.5ms ± 1% -18.16% (p=0.000 n=19+20)
CodeMarshal-2 20.1ms ± 1% 16.8ms ± 2% -16.35% (p=0.000 n=20+21)
CodeDecoder-2 49.3ms ± 1% 49.5ms ± 2% ~ (p=0.498 n=16+20)
DecoderStream-2 416ns ± 0% 416ns ± 1% ~ (p=0.978 n=19+19)
CodeUnmarshal-2 51.0ms ± 1% 50.9ms ± 1% ~ (p=0.490 n=19+17)
CodeUnmarshalReuse-2 48.5ms ± 2% 48.5ms ± 2% ~ (p=0.989 n=20+19)
UnmarshalString-2 541ns ± 1% 532ns ± 1% -1.75% (p=0.000 n=20+21)
UnmarshalFloat64-2 485ns ± 1% 481ns ± 1% -0.92% (p=0.000 n=20+21)
UnmarshalInt64-2 429ns ± 1% 427ns ± 1% -0.49% (p=0.000 n=19+20)
Issue10335-2 631ns ± 1% 619ns ± 1% -1.84% (p=0.000 n=20+20)
NumberIsValid-2 19.1ns ± 0% 19.1ns ± 0% ~ (all samples are equal)
NumberIsValidRegexp-2 689ns ± 1% 690ns ± 0% ~ (p=0.150 n=20+20)
SkipValue-2 14.0ms ± 0% 14.0ms ± 0% -0.05% (p=0.000 n=18+18)
EncoderEncode-2 525ns ± 2% 512ns ± 1% -2.33% (p=0.000 n=20+18)
name old speed new speed delta
CodeEncoder-2 102MB/s ± 0% 125MB/s ± 1% +22.20% (p=0.000 n=19+20)
CodeMarshal-2 96.6MB/s ± 1% 115.6MB/s ± 2% +19.56% (p=0.000 n=20+21)
CodeDecoder-2 39.3MB/s ± 1% 39.2MB/s ± 2% ~ (p=0.464 n=16+20)
CodeUnmarshal-2 38.1MB/s ± 1% 38.1MB/s ± 1% ~ (p=0.525 n=19+17)
SkipValue-2 143MB/s ± 0% 143MB/s ± 0% +0.05% (p=0.000 n=18+18)
I also took the data set reported in #5683 (browser
telemetry data from Mozilla), added named structs for
the data set, and turned it into a proper benchmark:
https://github.com/kevinburke/jsonbench/blob/master/go/bench_test.go
The results from that test are similarly encouraging. On a 64-bit
Mac:
$ benchstat benchmarks/master-benchmark benchmarks/json-table-benchmark
name old time/op new time/op delta
CodeMarshal-4 1.19ms ± 2% 1.08ms ± 2% -9.33% (p=0.000 n=21+17)
Unmarshal-4 3.09ms ± 3% 3.06ms ± 1% -0.83% (p=0.027 n=22+17)
UnmarshalReuse-4 3.04ms ± 1% 3.04ms ± 1% ~ (p=0.169 n=20+15)
name old speed new speed delta
CodeMarshal-4 80.3MB/s ± 1% 88.5MB/s ± 1% +10.29% (p=0.000 n=21+17)
Unmarshal-4 31.0MB/s ± 2% 31.2MB/s ± 1% +0.83% (p=0.025 n=22+17)
On the c4.large:
$ benchstat benchmarks/master-bench benchmarks/json-table-bench
name old time/op new time/op delta
CodeMarshal-2 1.10ms ± 1% 0.98ms ± 1% -10.12% (p=0.000 n=20+54)
Unmarshal-2 2.82ms ± 1% 2.79ms ± 0% -1.09% (p=0.000 n=20+51)
UnmarshalReuse-2 2.80ms ± 0% 2.77ms ± 0% -1.03% (p=0.000 n=20+52)
name old speed new speed delta
CodeMarshal-2 87.3MB/s ± 1% 97.1MB/s ± 1% +11.27% (p=0.000 n=20+54)
Unmarshal-2 33.9MB/s ± 1% 34.2MB/s ± 0% +1.10% (p=0.000 n=20+51)
For what it's worth, I tried other heuristics - short circuiting the
conditional for common ASCII characters, for example:
if (b >= 63 && b != 92) || (b >= 39 && b <= 59) || (rest of the conditional)
This offered a speedup around 7-9%, not as large as the submitted
change.
Change-Id: Idcf88f7b93bfcd1164cdd6a585160b7e407a0d9b
Reviewed-on: https://go-review.googlesource.com/24466
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Swtich from a sync.RWMutex to atomic.Value for cacheTypeFields.
On GOARCH=386, this recovers most of the remaining performance
difference from the 1.6 release. Compared with tip on linux/386:
name old time/op new time/op delta
CodeDecoder-40 92.8ms ± 1% 87.7ms ± 1% -5.50% (p=0.000 n=10+10)
name old speed new speed delta
CodeDecoder-40 20.9MB/s ± 1% 22.1MB/s ± 1% +5.83% (p=0.000 n=10+10)
With more time and care, I believe more of the JSON decoder's work
could be shifted so it is done before decoding, and independent of
the number of bytes processed. Maybe someone could explore that for
Go 1.8.
For #16117.
Change-Id: I049655b2e5b76384a0d5f4b90e3ec7cc8d8c4340
Reviewed-on: https://go-review.googlesource.com/24472
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This has been inaccurate since https://golang.org/cl/6048047.
Fixes#15317.
Change-Id: If93d2161f51ccb91912cb94a35318cf33f4d526a
Reviewed-on: https://go-review.googlesource.com/23691
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Decoding a JSON message does not touch unspecified or null fields;
always use a new underlying struct to prevent old field values from
sticking around.
Fixes: #14640
Change-Id: Ica78c208ce104e2cdee1d4e92bf58596ea5587c8
Reviewed-on: https://go-review.googlesource.com/23483
Reviewed-by: Andrew Gerrand <adg@golang.org>
CL 21057 added this method during the Go 1.7 cycle
(so it is not yet released and still possible to revise).
This makes it clearer that the method is not doing something
(like func Indent does), but just changing a setting about doing
something later.
Also document that this is in some sense irreversible.
I think that's probably a mistake but the original CL discussion
claimed it as a feature, so I'll leave it alone.
For #6492.
Change-Id: If4415c869a9196501056c143811a308822d5a420
Reviewed-on: https://go-review.googlesource.com/23295
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
DisableHTMLEscaping is now SetEscapeHTML, allowing the escaping
to be toggled, not just disabled. This API is new for Go 1.7,
so there are no compatibility concerns (quite the opposite,
the point is to fix the API before we commit to it in Go 1.7).
Change-Id: I96b9f8f169a9c44995b8a157a626eb62d0b6dea7
Reviewed-on: https://go-review.googlesource.com/23293
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
CL 19725 changed the encoding of []typedByte to look for
typedByte.MarshalJSON and typedByte.MarshalText.
Previously it was handled like []byte, producing a base64 encoding of the underlying byte data.
CL 19725 forgot to look for (*typedByte).MarshalJSON and (*typedByte).MarshalText,
as the marshaling of other slices would. Add test and fix for those.
This CL also adds tests that the decoder can handle both the old and new encodings.
(This was true even in Go 1.6, which is the only reason we can consider this
not an incompatible change.)
For #13783.
Change-Id: I7cab8b6c0154a7f2d09335b7fa23173bcf856c37
Reviewed-on: https://go-review.googlesource.com/23294
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
This change makes encoding and decoding support integer types in map
keys, converting to/from JSON string keys.
JSON object keys are still sorted lexically, even though the keys may be
integer strings.
For backwards-compatibility, the existing Text(Un)Marshaler support for
map keys (added in CL 20356) does not take precedence over the default
encoding for string types. There is no such concern for integer types,
so integer map key encoding is only used as a fallback if the map key
type is not a Text(Un)Marshaler.
Fixes#12529.
Change-Id: I7e68c34f9cd19704b1d233a9862da15fabf0908a
Reviewed-on: https://go-review.googlesource.com/22060
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This provides a way to disable the escaping of <, >, and & in JSON
strings.
Fixes#14749.
Change-Id: I1afeb0244455fc8b06c6cce920444532f229555b
Reviewed-on: https://go-review.googlesource.com/21796
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In JSON terminology, "object" is a collect of key/value pairs. But a
JSON object is only one type of JSON value (others are string, number,
array, true, false, null).
This updates the Go docs (at least the public godoc) to not use
"object" when we mean any JSON value.
Change-Id: Ieb1c456c703693714d63d9d09d306f4d9e8f4597
Reviewed-on: https://go-review.googlesource.com/22003
Reviewed-by: Andrew Gerrand <adg@golang.org>
This CL allows JSON-encoding & -decoding maps whose keys are types that
implement encoding.TextMarshaler / TextUnmarshaler.
During encode, the map keys are marshaled upfront so that they can be
sorted.
Fixes#12146
Change-Id: I43809750a7ad82a3603662f095c7baf75fd172da
Reviewed-on: https://go-review.googlesource.com/20356
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a subset of https://golang.org/cl/20022 with only the copyright
header lines, so the next CL will be smaller and more reviewable.
Go policy has been single space after periods in comments for some time.
The copyright header template at:
https://golang.org/doc/contribute.html#copyright
also uses a single space.
Make them all consistent.
Change-Id: Icc26c6b8495c3820da6b171ca96a74701b4a01b0
Reviewed-on: https://go-review.googlesource.com/20111
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Followup to CL 12250.
For #10281.
Change-Id: If25d9cac92f10327bb355f2d11b00c625b464661
Reviewed-on: https://go-review.googlesource.com/17199
Reviewed-by: Ian Lance Taylor <iant@golang.org>
json.Number is a special case which didn't have any checks and could result in invalid JSON.
Fixes#10281
Change-Id: Ie3e726e4d6bf6a6aba535d36f6107013ceac913a
Reviewed-on: https://go-review.googlesource.com/12250
Reviewed-by: Russ Cox <rsc@golang.org>
This allows slices of custom types with byte as underlying type to be
decoded, fixing a regression introduced in CL 9371.
Fixes#12921.
Change-Id: I62a715eaeaaa912b6bc599e94f9981a9ba5cb242
Reviewed-on: https://go-review.googlesource.com/16303
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Addresses issue #12367.
Must be checked in before CL 14010.
Change-Id: I7233c3a62d4f55d0ac7e8a87df5fc4ee7beb7207
Reviewed-on: https://go-review.googlesource.com/14011
Reviewed-by: Russ Cox <rsc@golang.org>
As correctly mentioned in #11883, encodeState.string and
encodeState.stringBytes never return an error.
This CL removes the error from the function signatures and somewhat
simplifies call sites.
Fixes#11883
Change-Id: I1d1853d09631c545b68b5eea86ff7daa2e0ca10b
Reviewed-on: https://go-review.googlesource.com/15836
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The fields step and redoState of struct scanner are now defined as
`func(s *scanner, c byte) int` instead of
`func(s *scanner, c int) int`, since bytes are sufficient.
Further changes improve the consistency in the scanner.go file.
Change-Id: Ifb85f2130d728d2b936d79914d87a1f0b5c6ee7d
Reviewed-on: https://go-review.googlesource.com/14801
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Rename test name from Http to HTTP, and fix some style nits.
Change-Id: I00fe1cecd69ca2f50be86a76ec90031c2f921707
Reviewed-on: https://go-review.googlesource.com/12760
Reviewed-by: Andrew Gerrand <adg@golang.org>
The new Token API is meant to sit on the side of the Decoder,
so that you only get the new code (and any latent bugs in it)
if you are actively using the Token API.
The unconditional use of dec.peek in dec.tokenPrepareForDecode
violates that intention.
Change tokenPrepareForDecode not to call dec.peek unless needed
(because the Token API has advanced the state).
This restores the old code path behavior, no peeking allowed.
I checked by patching in the new tests from CL 12726 that
this change suffices to "fix" the error handling bug in dec.peek.
Obviously that bug should be fixed too, but the point is that
with this CL, bugs in dec.peek do not affect plain use of Decode
or Unmarshal.
I also checked by putting a panic in dec.peek that the only
tests that now invoke peek are:
TestDecodeInStream
ExampleDecoder_Token
ExampleDecoder_Decode_stream
and those tests all invoke dec.Token directly.
Change-Id: I0b242d0cb54a9c830548644670dc5ab5ccef69f2
Reviewed-on: https://go-review.googlesource.com/12740
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Peter Waldschmidt <peter@waldschmidt.com>
This change adds new methods to Decoder.
* Decoder.Token steps through a JSON document, returning a value for each token.
* Decoder.Decode unmarshals the entire value at the token stream's current
position (in addition to its existing function in a stream of JSON values)
Fixes#6050.
Fixes#6499.
Change-Id: Iff283e0e7b537221ae256392aca6529f06ebe211
Reviewed-on: https://go-review.googlesource.com/9073
Reviewed-by: Russ Cox <rsc@golang.org>
JSON decoding currently fails for null values bound to any type
which does implement the JSON Unmarshaler interface without checking
for null values (such as time.Time).
It also fails for types implementing the TextUnmarshaler interface.
The expected behavior of the JSON decoding engine in such case is
to process null by keeping the value unchanged without producing
any error.
Make sure null values are handled by the decoding engine itself,
and never passed to the UnmarshalText or UnmarshalJSON methods.
Fixes#9037
Change-Id: I261d85587ba543ef6f1815555b2af9311034d5bb
Reviewed-on: https://go-review.googlesource.com/9376
Reviewed-by: Russ Cox <rsc@golang.org>
The "string" option only applies for strings, floats, integers, and
booleans as per the documentation. So when decoding ignore the "string"
option if the value is not of one of the types mentioned. This matches
the Marshal step which also ignores the "string" option for invalid
types.
Fixes#9812
Change-Id: I0fb2b43d0668bc0e2985886d989abbf2252070e2
Reviewed-on: https://go-review.googlesource.com/10183
Reviewed-by: Russ Cox <rsc@golang.org>
The previous commit (git 2ae77376) just did golang.org. This one
includes golang.org subdomains like blog, play, and build.
Change-Id: I4469f7b307ae2a12ea89323422044e604c5133ae
Reviewed-on: https://go-review.googlesource.com/12071
Reviewed-by: Rob Pike <r@golang.org>
The one in misc/makerelease/makerelease.go is particularly bad and
probably warrants rotating our keys.
I didn't update old weekly notes, and reverted some changes involving
test code for now, since we're late in the Go 1.5 freeze. Otherwise,
the rest are all auto-generated changes, and all manually reviewed.
Change-Id: Ia2753576ab5d64826a167d259f48a2f50508792d
Reviewed-on: https://go-review.googlesource.com/12048
Reviewed-by: Rob Pike <r@golang.org>
When the scanner receives a non-whitespace character in stateEndTop,
it creates an error message and caches it to return on the next
transition. nextValue() uses the scanner to sub-scan for a value
inside a larger JSON structure. Since stateEndTop is triggered
*after* the ending byte, whatever character immediately follows the
sub-value gets pulled into the scanner's state machine as well.
Even though it is not used and doesn't cause an error, it does
cause the state machine to allocate an error that will never be used.
The fix is to probe the state machine with whitespace after
scanEndObject or scanEndArray to see if the next character would
result in a scanEnd state transition. If so, we can return right
away without processing the next character and avoid triggering
an allocation.
benchmark old ns/op new ns/op delta
BenchmarkCodeEncoder 17022194 16611336 -2.41%
BenchmarkCodeMarshal 18443250 18090144 -1.91%
BenchmarkCodeDecoder 61502053 61010936 -0.80%
BenchmarkCodeUnmarshal 61410829 60363605 -1.71%
BenchmarkCodeUnmarshalReuse 59124836 58361772 -1.29%
BenchmarkUnmarshalString 602 603 +0.17%
BenchmarkUnmarshalFloat64 535 537 +0.37%
BenchmarkUnmarshalInt64 482 482 +0.00%
BenchmarkIssue10335 1206 799 -33.75%
BenchmarkSkipValue 17605751 18355391 +4.26%
BenchmarkEncoderEncode 612 604 -1.31%
benchmark old MB/s new MB/s speedup
BenchmarkCodeEncoder 114.00 116.82 1.02x
BenchmarkCodeMarshal 105.21 107.27 1.02x
BenchmarkCodeDecoder 31.55 31.81 1.01x
BenchmarkCodeUnmarshal 31.60 32.15 1.02x
BenchmarkSkipValue 111.63 107.07 0.96x
benchmark old allocs new allocs delta
BenchmarkIssue10335 11 4 -63.64%
BenchmarkEncoderEncode 2 2 +0.00%
benchmark old bytes new bytes delta
BenchmarkIssue10335 376 272 -27.66%
BenchmarkEncoderEncode 40 40 +0.00%
Fixes#10335
Change-Id: I3d4f2b67f7a038adfb33ba48bb6b680f528baf18
Reviewed-on: https://go-review.googlesource.com/9074
Reviewed-by: Russ Cox <rsc@golang.org>
All slice types which have elements of kind reflect.Uint8 are marshalled
into base64 for compactness. When decoding such data into a custom type
based on []byte the decoder checked the slice kind instead of the slice
element kind, so no appropriate decoder was found.
Fixed by letting the decoder check slice element kind like the encoder.
This guarantees that already encoded data can still be successfully
decoded.
Fixes#8962.
Change-Id: Ia320d4dc2c6e9e5fe6d8dc15788c81da23d20c4f
Reviewed-on: https://go-review.googlesource.com/9371
Reviewed-by: Peter Waldschmidt <peter@waldschmidt.com>
Reviewed-by: Russ Cox <rsc@golang.org>
BenchmarkSkipValue was sensitive to the value of
b.N due to its significant startup cost.
Two adjacent runs before this CL:
BenchmarkSkipValue 50 21047499 ns/op 93.37 MB/s
BenchmarkSkipValue 100 17260554 ns/op 118.05 MB/s
After this CL, using benchtime to recreate the
difference in b.N:
BenchmarkSkipValue 50 15204797 ns/op 131.67 MB/s
BenchmarkSkipValue 100 15332319 ns/op 130.58 MB/s
Change-Id: Iac86f86dd774d535302fa5e4c08f89f8da00be9e
Reviewed-on: https://go-review.googlesource.com/10053
Reviewed-by: Andrew Gerrand <adg@golang.org>
The comment previously was reversed in sense (it appeared to be
describing unmarshaling). I've fixed that, and added the caveat that map
keys are subject to UTF-8 coercion like other strings.
Change-Id: Id08082aa71401a6e7530a42f979fbb50bd1f4e6a
Reviewed-on: https://go-review.googlesource.com/5221
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
The error message for decoding a unquoted value into a struct field with
the ,string option specified has two arguments when one is needed.
Make the error message take one argument and add a test in order to cover
the case when a unquoted value is specified.
Also add error value as the missing argument for Fatalf call in test.
Fixes the following go vet reports:
decode.go:602: wrong number of args for format in Errorf call: 1 needed but 2 args
decode_test.go:1088: missing argument for Fatalf("%v"): format reads arg 1, have only 0 args
Change-Id: Id036e10c54c4a7c1ee9952f6910858ecc2b84134
Reviewed-on: https://go-review.googlesource.com/2109
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Joe Tsai
Ian Lance Taylor
Leigh McCulloch
Michael Fraenkel
Ivan Bertona
Marvin Stenger
tbunyk
Daniel Martí
Dmitri Shuralyov
Iccha Sethi
Brad Fitzpatrick
thoeni
Alberto Donizetti
Bryan C. Mills
Pascal S. de Kloe
Tuo Shan
Matt Layher
Russ Cox
Emmanuel Odeke
Richard Gibson
Jirka Daněk
mike andrews
Kevin Burke
Carlos C
David Crawshaw
David Glasser
Quentin Smith
Scott Bell
Caleb Spare
Håvard Haugen
Augusto Roman
Dominik Honnef
Martin Möhrmann
Erik Dubbelboer
Marcel van Lohuizen
Nodir Turakulov
Robert Griesemer
Andrew Gerrand
Peter Waldschmidt
Didier Spezia
Larz Conwell
Josh Bleecher Snyder
Aaron Jacobs
Alex Plugaru
Emil Hessman