CL 58210 introduced this constant for reasons I don't understand.
It should not be in the exported const block, which will pollute
godoc output with a "... unexported" notice.
Also since we already have a constant named xmlnsPrefix for "xmlns",
it is very confusing to also have xmlNamespacePrefix for "xml".
If we must have the constant at all, rename it to xmlPrefix.
Change-Id: I15f937454d730005816fcd32b1acca703acf1e51
Reviewed-on: https://go-review.googlesource.com/78121
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
A record can span multiple lines (the whole reason for the extra field),
so the important fact is that it's the _start_ of the record.
Make that clear in the name.
(This API was added during the Go 1.10 cycle so it can still be cleaned up.)
Change-Id: Id95b3ceb7cdfc4aa0ed5a053cb84da8945fa5496
Reviewed-on: https://go-review.googlesource.com/78119
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Mainly get rid of the weird zero-value struct literal,
but while we're here also group and order things a bit better:
first the reader, then the data, then the call (which takes reader then data).
Change-Id: I901b0661d85d8eaa0807e4482aac66500ca996c7
Reviewed-on: https://go-review.googlesource.com/78118
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
CL 60410 fixes a bug in reflect that allows assignments to an embedded
field of a pointer to an unexported struct type.
This breaks the json package because unmarshal is now unable to assign
a newly allocated struct to such fields.
In order to be consistent in the behavior for marshal and unmarshal,
this CL changes both marshal and unmarshal to always ignore
embedded pointers to unexported structs.
Fixes#21357
Change-Id: If62ea11155555e61115ebb9cfa5305caf101bde5
Reviewed-on: https://go-review.googlesource.com/76851
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The docs for xml.Marshal state that the XML elements name is derived
from one of five locations in a specific order of precedence, but does
not mention that if the field is a struct type and has its name defined
in a tag and in the types XMLName field that an error will occur. This
is documented in the structFieldInfo function but not in the function
documentation, and the existing docs in Marshal are misleading without
this behavior being discussed.
Fixes#18564
Change-Id: I29042f124a534bd1bc993f1baeddaa0af2e72fed
Reviewed-on: https://go-review.googlesource.com/76321
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The go repository contains a mix of github.com/golang/go/issues/xxxxx
and golang.org/issues/xxxxx URLs for references to issues in the issue
tracker. We should use one for consistency, and golang.org is preferred
in case the project moves the issue tracker in the future.
This reasoning is taken from a comment Sam Whited left on a CL I
recently opened: https://go-review.googlesource.com/c/go/+/73890.
In that CL I referenced an issue using its github.com URL, because other
tests in the file I was changing contained references to issues using
their github.com URL. Sam Whited left a comment on the CL stating I
should change it to the golang.org URL.
If new code is intended to reference issues via golang.org and not
github.com, existing code should be updated so that precedence exists
for contributors who are looking at the existing code as a guide for the
code they should write.
Change-Id: I3b9053fe38a1c56fc101a8b7fd7b8f310ba29724
Reviewed-on: https://go-review.googlesource.com/75673
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds new rules to recognize consecutive byte loads and
stores and lowers them to loads and stores such as lhz, lwz, ld,
sth, stw, std. This change only covers the little endian cases
on little endian machines, such as is found in encoding/binary
UintXX or PutUintXX for little endian. Big endian will be done
later.
Updates were also made to binary_test.go to allow the benchmark
for Uint and PutUint to actually use those functions because
the way they were written, those functions were being
optimized out.
Testcases were also added to cmd/compile/internal/gc/asm_test.go.
Updates #22496
The following improvement can be found in golang.org/x/crypto
poly1305:
Benchmark64-16 142 114 -19.72%
Benchmark1K-16 1717 1424 -17.06%
Benchmark64Unaligned-16 142 113 -20.42%
Benchmark1KUnaligned-16 1721 1428 -17.02%
chacha20poly1305:
BenchmarkChacha20Poly1305Open_64-16 1012 885 -12.55%
BenchmarkChacha20Poly1305Seal_64-16 971 836 -13.90%
BenchmarkChacha20Poly1305Open_1350-16 11113 9539 -14.16%
BenchmarkChacha20Poly1305Seal_1350-16 11013 9392 -14.72%
BenchmarkChacha20Poly1305Open_8K-16 61074 53431 -12.51%
BenchmarkChacha20Poly1305Seal_8K-16 61214 54806 -10.47%
Other improvements of around 10% found in crypto/tls.
Results after updating encoding/binary/binary_test.go:
BenchmarkLittleEndianPutUint64-16 1.87 0.93 -50.27%
BenchmarkLittleEndianPutUint32-16 1.19 0.93 -21.85%
BenchmarkLittleEndianPutUint16-16 1.16 1.03 -11.21%
Change-Id: I7bbe2fbcbd11362d58662fecd907a0c07e6ca2fb
Reviewed-on: https://go-review.googlesource.com/74410
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Munday <mike.munday@ibm.com>
To improve readability when exported fields are removed,
forbid the printer from emitting an empty line before the first comment
in a const, var, or type block.
Also, when printing the "Has filtered or unexported fields." message,
add an empty line before it to separate the message from the struct
or interfact contents.
Before the change:
<<<
type NamedArg struct {
// Name is the name of the parameter placeholder.
//
// If empty, the ordinal position in the argument list will be
// used.
//
// Name must omit any symbol prefix.
Name string
// Value is the value of the parameter.
// It may be assigned the same value types as the query
// arguments.
Value interface{}
// contains filtered or unexported fields
}
>>>
After the change:
<<<
type NamedArg struct {
// Name is the name of the parameter placeholder.
//
// If empty, the ordinal position in the argument list will be
// used.
//
// Name must omit any symbol prefix.
Name string
// Value is the value of the parameter.
// It may be assigned the same value types as the query
// arguments.
Value interface{}
// contains filtered or unexported fields
}
>>>
Fixes#18264
Change-Id: I9fe17ca39cf92fcdfea55064bd2eaa784ce48c88
Reviewed-on: https://go-review.googlesource.com/71990
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Whitespace is ignored in bool values and attrs. It is convenient and
relatively safe since whitespace around a bool value is often
unimportant. The same logic can be applied to numeric values of types
int, uint, and float.
Fixes#22146
Change-Id: Ie0462def90304af144b8e2e72d85b644857c27cc
Reviewed-on: https://go-review.googlesource.com/73891
Reviewed-by: Sam Whited <sam@samwhited.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Whitespace is ignored in bool values and attrs, but there are no tests
capturing this behavior.
Change-Id: I7a7249de4886f510869e91de937e69b83c3254c8
Reviewed-on: https://go-review.googlesource.com/73890
Reviewed-by: Sam Whited <sam@samwhited.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When a SyntaxError occurs, report the current offset within the stream.
The code already accounted for the offset within the current buffer
being scanned. By including how much data was already scanned, the
current offset can be computed.
Fixes#22478
Change-Id: I91ecd4cad0b85a5c1556bc597f3ee914e769af01
Reviewed-on: https://go-review.googlesource.com/74251
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add a DisallowUnknownFields flag to Decoder.
DisallowUnknownFields causes the Decoder to return an error when
the the decoding destination is a struct and the input contains
object keys which do not match any non-ignored, public field the
destination, including keys whose value is set to null.
Note: this fix has already been worked on in 27231, which seems
to be abandoned. This version is a slightly simpler implementation
and is up to date with the master branch.
Fixes#15314
Change-Id: I987a5857c52018df334f4d1a2360649c44a7175d
Reviewed-on: https://go-review.googlesource.com/74830
Reviewed-by: Joe Tsai <joetsai@google.com>
Run-TryBot: Joe Tsai <joetsai@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The record delimiter (not configurable by user) is "\r\n" or "\n".
It is insensible for the user to set Comma or Comment delimiters
to be some character that conflicts with the record delimiter.
Furthermore, it is insensible for Comma or Comment to be the same rune.
Allowing this leaks implementation details to the user in regards to
the evaluation order of which rune is checked for first.
Fixes#22404
Change-Id: I31e86abc9b3a8fb4584e090477795587740970ae
Reviewed-on: https://go-review.googlesource.com/72793
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The ErrQuote variable is only returned when a parsing error
occurs within a quoted string. Make that clear in the message.
Change-Id: I06ad5a9edb41afedde193c4f8b93551bb8342bbb
Reviewed-on: https://go-review.googlesource.com/72794
Reviewed-by: Avelino <t@avelino.xxx>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We should be referring to ParseError.Err, which is the underlying error,
not ParseError.Error, which is the error method.
Change-Id: Ic3cef5ecbe1ada5fa14b9573222f29da8fc9a8d5
Reviewed-on: https://go-review.googlesource.com/72450
Reviewed-by: Tim Cooper <tim.cooper@layeh.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 72150 fixes#22352 by reverting the problematic parts of that CL
where the line number and column number were inconsistent with each other.
This CL adds back functionality to address the issue that CL 72150
was trying to solve in the first place. That is, it reports the starting
line of the record, so that users have a frame of reference to start with
when debugging what went wrong.
In the event of gnarly CSV files with multiline quoted strings, a parse
failure likely occurs somewhere between the start of the record and
the point where the parser finally detected an error.
Since ParserError.{Line,Column} reports where the *error* occurs, we
add a RecordLine field to report where the record starts.
Also take this time to cleanup and modernize TestRead.
Fixes#19019Fixes#22352
Change-Id: I16cebf0b81922c35f75804c7073e9cddbfd11a04
Reviewed-on: https://go-review.googlesource.com/72310
Reviewed-by: Ian Lance Taylor <iant@golang.org>
NewEncoder returns an io.Writer that writes all incoming bytes as
hexadecimal characters to the underlying io.Writer. NewDecoder returns an
io.Reader that does the inverse.
Fixes#21590
Change-Id: Iebe0813faf365b42598f19a9aa41768f571dc0a8
Reviewed-on: https://go-review.googlesource.com/70210
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The Reader implementation is slow because it operates on a rune-by-rune
basis via bufio.Reader.ReadRune. We speed this up by operating on entire
lines that we read from bufio.Reader.ReadSlice.
In order to ensure that we read the full line, we augment ReadSlice
in our Reader.readLine method to automatically expand the slice if
bufio.ErrBufferFull is every hit.
This change happens to fix#19410 because it no longer relies on
rune-by-rune parsing and only searches for the relevant delimiter rune.
In order to keep column accounting simple and consistent, this change
reverts parts of CL 52830.
This CL is an alternative to CL 36270 and builds on some of the ideas
from that change by Diogo Pinela.
name old time/op new time/op delta
Read-8 3.12µs ± 1% 2.54µs ± 2% -18.76% (p=0.000 n=10+9)
ReadWithFieldsPerRecord-8 3.12µs ± 1% 2.53µs ± 1% -18.91% (p=0.000 n=9+9)
ReadWithoutFieldsPerRecord-8 3.13µs ± 0% 2.57µs ± 3% -18.07% (p=0.000 n=10+10)
ReadLargeFields-8 52.3µs ± 1% 5.3µs ± 2% -89.93% (p=0.000 n=10+9)
ReadReuseRecord-8 2.05µs ± 1% 1.40µs ± 1% -31.48% (p=0.000 n=10+9)
ReadReuseRecordWithFieldsPerRecord-8 2.05µs ± 1% 1.41µs ± 0% -31.03% (p=0.000 n=10+9)
ReadReuseRecordWithoutFieldsPerRecord-8 2.06µs ± 1% 1.40µs ± 1% -31.70% (p=0.000 n=9+10)
ReadReuseRecordLargeFields-8 50.9µs ± 0% 4.1µs ± 3% -92.01% (p=0.000 n=10+10)
name old alloc/op new alloc/op
Read-8 664B ± 0% 664B ± 0%
ReadWithFieldsPerRecord-8 664B ± 0% 664B ± 0%
ReadWithoutFieldsPerRecord-8 664B ± 0% 664B ± 0%
ReadLargeFields-8 3.94kB ± 0% 3.94kB ± 0%
ReadReuseRecord-8 24.0B ± 0% 24.0B ± 0%
ReadReuseRecordWithFieldsPerRecord-8 24.0B ± 0% 24.0B ± 0%
ReadReuseRecordWithoutFieldsPerRecord-8 24.0B ± 0% 24.0B ± 0%
ReadReuseRecordLargeFields-8 2.98kB ± 0% 2.98kB ± 0%
name old allocs/op new allocs/op
Read-8 18.0 ± 0% 18.0 ± 0%
ReadWithFieldsPerRecord-8 18.0 ± 0% 18.0 ± 0%
ReadWithoutFieldsPerRecord-8 18.0 ± 0% 18.0 ± 0%
ReadLargeFields-8 24.0 ± 0% 24.0 ± 0%
ReadReuseRecord-8 8.00 ± 0% 8.00 ± 0%
ReadReuseRecordWithFieldsPerRecord-8 8.00 ± 0% 8.00 ± 0%
ReadReuseRecordWithoutFieldsPerRecord-8 8.00 ± 0% 8.00 ± 0%
ReadReuseRecordLargeFields-8 12.0 ± 0% 12.0 ± 0%
Updates #22352
Updates #19019Fixes#16791Fixes#19410
Change-Id: I31c27cfcc56880e6abac262f36c947179b550bbf
Reviewed-on: https://go-review.googlesource.com/72150
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In #10909, it was decided that "Deprecated:" is a magic string for
tools (e.g., #17056 for godoc) to detect deprecated identifiers.
Use those convention instead of custom written prose.
Change-Id: Ia514fc3c88fc502e86c6e3de361c435f4cb80b22
Reviewed-on: https://go-review.googlesource.com/70110
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
The '*' character is not allowed in ASN.1 PrintableString. However, due
to wide-spread use, we permit it so that we can parse many certificates
with wildcards. However, that also meant that generic strings with
asterisks in would be encoded as PrintableString.
This change makes the default for such strings to be UTF8String. Thus,
while the code PrintableStrings containing '*', it will not generate
them unless the string type was specified in the struct field tag.
Change-Id: I2d458da36649427352eeaa50a1b6020108b2ccbd
Reviewed-on: https://go-review.googlesource.com/68990
Reviewed-by: Adam Langley <agl@golang.org>
Previously, any “explicit” and/or “tag” decorations on a RawValue would
be ignored when unmarshaling. The RawValue would swallow whatever
element was encountered.
This change causes these decorations to be respected. Thus a field like:
Foo asn1.RawValue `asn1:"explicit,tag:1,optional"`
will only match if an explicit tag with value one is encountered.
Otherwise the RawValue will get the default value and parsing will move
onto the next element.
Thanks to Martin Kreichgauer for reporting the issue.
Change-Id: If6c4488685b9bd039cb5e352d6d75744f98dbb1f
Reviewed-on: https://go-review.googlesource.com/34503
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
Remove an old comment introduced in golang.org/cl/9073.
Change-Id: I14be27ddfac987f44d839920bc4d02361a576f06
Reviewed-on: https://go-review.googlesource.com/66371
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
bytes.IndexByte can be used wherever the second argument to
strings.Index is exactly one byte long, so we do that with this change.
This avoids generating unnecessary string symbols/converison and saves
a few calls to bytes.Index.
Change-Id: If31c775790e01edfece1169e398ad6a754fb4428
Reviewed-on: https://go-review.googlesource.com/66373
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.LastIndexByte was introduced in go1.5 and it can be used
effectively wherever the second argument to strings.LastIndex is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.LastIndex.
Change-Id: I7b5679d616197b055cffe6882a8675d24a98b574
Reviewed-on: https://go-review.googlesource.com/66372
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.IndexByte was introduced in go1.2 and it can be used
effectively wherever the second argument to strings.Index is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.Index.
Change-Id: I1ab5edb7c4ee9058084cfa57cbcc267c2597e793
Reviewed-on: https://go-review.googlesource.com/65930
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 60410 fixes the compiler such that reflect.StructField.PkgPath
is non-empty if and only if the field is unexported.
Given that property, we can cleanup the logic in the json encoder
to avoid parsing the field name to detect export properties.
Updates #21122
Change-Id: Ic01b9c4ca76386774846b742b0c1b9b948f53e7c
Reviewed-on: https://go-review.googlesource.com/65550
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Make arguments semantics clear without the need to look for
json.Indent documentation.
Change-Id: If9adfe9f477a30d426ae83790b0f2578c0a809b7
Reviewed-on: https://go-review.googlesource.com/61670
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Found with mvdan.cc/unindent. It skipped the cases where parentheses
would need to be added, where comments would have to be moved elsewhere,
or where actions and simple logic would mix.
One of them was of the form "err != nil && err == io.EOF", so the first
part was removed.
Change-Id: Ie504c2b03a2c87d10ecbca1b9270069be1171b91
Reviewed-on: https://go-review.googlesource.com/57690
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Most of these are return values that were part of a receiving parameter,
so they're still accessible.
A few others are not, but those have never had a use.
Found with github.com/mvdan/unparam, after Kevin Burke's suggestion that
the tool should also warn about unused result parameters.
Change-Id: Id8b5ed89912a99db22027703a88bd94d0b292b8b
Reviewed-on: https://go-review.googlesource.com/55910
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Went mainly for the ones that make no sense, such as the ones
mid-sentence or after commas.
Change-Id: Ie245d2c19cc7428a06295635cf6a9482ade25ff0
Reviewed-on: https://go-review.googlesource.com/57293
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The destination slice does not need to be created at all. The source
slice itself can be used as the destination because the decode loop
increments by one and then the 'seen' byte is not used anymore. Therefore
the decoded byte can be stored in that index of the source slice itself.
This trick cannot be applied to EncodeString() because in that case,
the destination slice is large than the source. And for a single byte
in the source slice, two bytes in the destination slice is written.
func BenchmarkDecodeString(b *testing.B) {
for i := 0; i < b.N; i++ {
DecodeString("0123456789abcdef")
}
}
name old time/op new time/op delta
DecodeString 71.0ns ± 6% 58.0ns ± 0% -18.28% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
DecodeString 16.0B ± 0% 8.0B ± 0% -50.00% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
DecodeString 1.00 ± 0% 1.00 ± 0% ~ (all equal)
Change-Id: Id98db4e712444557a804155457a4dd8d1b8b416d
Reviewed-on: https://go-review.googlesource.com/55611
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The parser mistakenly assumed it could always fold \r\n into \n, which
is not true since a \r\n inside a quoted fields has no special meaning
and should be kept as is.
Fix this by not folding \r\n to \n inside quotes fields.
Fixes#21201
Change-Id: Ifebc302e49cf63e0a027ee90f088dbc050a2b7a6
Reviewed-on: https://go-review.googlesource.com/52810
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The tests for error scenarios were done by manually checking
error strings. Improved them by checking the actual error type
instead of just the string.
Printing the actual error in case of failure instead of a
generic string.
Also added a new scenario with both an invalid byte and an
invalid length string to verify that the length is checked first
before doing any computation.
Change-Id: Ic2a19a6d6058912632d597590186ee2d8348cb45
Reviewed-on: https://go-review.googlesource.com/55256
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Errors returned by Reader contain the line where the Reader originally
encountered the error. This can be suboptimal since that line does not
always correspond with the line the current record/field started at.
This can easily happen with LazyQuotes as seen in #19019, but also
happens for example when a quoted fields has no closing quote and
the parser hits EOF before it finds another quote.
When this happens finding the erroneous field can be somewhat
complicated and time consuming, and in most cases it would be better to
report the line where the record started.
This change updates Reader to keep track of the line on which a record
begins and uses it for errors instead of the current line, making it
easier to find errors.
Although a user-visible change, this should have no impact on existing
code, since most users don't explicitly work with the line in the error
and probably already expect the new behaviour.
Updates #19019
Change-Id: Ic9bc70fad2651c69435d614d537e7a9266819b05
Reviewed-on: https://go-review.googlesource.com/52830
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I27ff99aa7abb070f6ae79c8f964aa9bd6a83b89d
Reviewed-on: https://go-review.googlesource.com/53730
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change fixes the remaining examples where the raw strings had
suboptimal indentation (one level too many) when viewed in godoc.
Follows CL 48910.
Fixes#21026.
Change-Id: Ifc0dae3fa899a9fff8b1ff958414e2fe6852321d
Reviewed-on: https://go-review.googlesource.com/50990
Run-TryBot: Dmitri Shuralyov <shurcool@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
https://golang.org/cl/33773 fixes the JSON marshaler to avoid serializing
embedded fields on unexported types of non-struct types. However, Go allows
embedding pointer to types, so the check for whether the field is a non-struct
type must first dereference the pointer to get at the underlying type.
Furthermore, due to a edge-case in the behavior of StructField.PkgPath not
being a reliable indicator of whether the field is unexported (see #21122),
we use our own logic to determine whether the field is exported or not.
The logic in this CL may be simplified depending on what happens in #21122.
Fixes#21121
Updates #21122
Change-Id: I8dfd1cdfac8a87950df294a566fb96dfd04fd749
Reviewed-on: https://go-review.googlesource.com/50711
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The existing example for Decoder.Decode (Stream) had excessive
indentation in the godoc interface for the const jsonStream,
making it hard to read. This fixes the indentation in the
example_test.go to improve the readability in godoc.
Helps #21026.
Change-Id: I16f56b82182da1dcc73cca44e535a7f5695e975d
Reviewed-on: https://go-review.googlesource.com/48910
Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 47341 added support for decoding non-padded messages. But DecodedLen
still returned a multiple of 5 for messages without a padding, even
though it is possible to calculate the len exactly when using NoPadding.
This change makes DecodedLen return the exact number of bytes that
will be written. A change to the decoding logic is also made so that it
can handle this case.
DecodedLen now has the same behaviour as DecodedLen in encoding/base64.
Fixes#20854
Change-Id: I729e0b1c0946c866fb675c854f835f366dd4b5a4
Reviewed-on: https://go-review.googlesource.com/47710
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
And some double space after period cleanup while I'm here.
I guess my previous regexps missed these. My next cleaner should
probably use go/ast instead of perl.
Updates #20221
Change-Id: Idb051e7ac3a7fb1fb86e015f709e32139d065d92
Reviewed-on: https://go-review.googlesource.com/47094
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Use 2 slashes, space, then tab. This is more consistent, and removes
inadvertent leading space.
Change-Id: I383770ed4eb8ac17c78c7ae5675b553d4fb70b1e
Reviewed-on: https://go-review.googlesource.com/46726
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
ascii85_test.go contains a variable called bigtest that is used as
test data for TestDecoderBuffering and TestEncoderBuffering. The
variable is initialised to a copy of the last element of the pairs
slice. When the variable was first added the last element of this
slice contained a sizable test case, 342 encoded characters. However,
https://golang.org/cl/5970078 added a new element to the end of the pairs
slice without updating bigtest. As the new element contained only 1 byte
of encoded data bigtest became very small test. This commit fixes the
problem by resetting bigtest to its original value and making its
initialisation independent of the layout of pairs. All the unit tests
still pass.
Change-Id: If7fb609ced9da93a2321dfd8372986b2fa772fd5
Reviewed-on: https://go-review.googlesource.com/46475
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The ascii85, base32 and base64 packages all contain a test called
TestDecoderBuffering. Each of these tests contain a loop that ignores
the error returned from the Read method of their decoders. The result
being that the tests loop for ever if the decoders actually return an
error. This commit fixes the issue by terminating the loops if an error
occurs and failing the tests with a suitable error message.
Change-Id: Idb385673cf9f3f6f8befe4288b4be366ab0985fd
Reviewed-on: https://go-review.googlesource.com/46010
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some of the _test.go files in the encoding packages contain a private
function called testEqual that calls testing.Errorf if the arguments
passed to it are unequal. The line numbers output by such calls to
Errorf identify the failure as being in testEqual itself which is not
very useful. This commit fixes the problem by adding a call to the
new t.Helper method in each of the testEqual functions. The line
numbers output when errors do occur now identify the real source of
the error.
Change-Id: I582d1934f40ef2b788116c3811074c67ea882021
Reviewed-on: https://go-review.googlesource.com/45871
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Marshal must process unexported embedded fields of struct type,
looking for exported fields in those structs. However, it must
not process unexported embedded fields of non-struct type.
For example, consider:
type t1 struct {
X int
}
type t2 int
type T struct {
t1
t2
}
When considering T, Marshal must process t1 to find t1.X.
Marshal must not process t2, but it was. Fix that.
Fixes#18009
Change-Id: I62ba0b65ba30fd927990e101a26405a9998787a3
Reviewed-on: https://go-review.googlesource.com/33773
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
A number of issues in decoder.Read and newlineFilteringReader.Read were
preventing errors from the reader supplying the encoded data from being
propagated to the caller. Fixing these issues revealed some additional
problems in which valid decoded data was not always returned to the user
when errors were actually propagated.
This commit fixes both the error propagation and the lost decoded data
problems. It also adds some new unit tests to ensure errors are handled
correctly by decoder.Read. The new unit tests increase the test coverage
of this package from 96.2% to 97.9%.
Fixes#20044
Change-Id: I1a8632da20135906e2d191c2a8825b10e7ecc4c5
Reviewed-on: https://go-review.googlesource.com/42094
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In many cases the records returned by Reader.Read will only be used between calls
to Read and become garbage once a new record is read. In this case, instead of
allocating a new slice on each call to Read, we can reuse the last allocated slice
for successive calls to avoid unnecessary allocations.
This change adds a new field ReuseRecord to the Reader struct to enable this reuse.
ReuseRecord is false by default to avoid breaking existing code which dependss on
the current behaviour.
I also added 4 new benchmarks, corresponding to the existing Read benchmarks, which
set ReuseRecord to true.
Benchstat on my local machine (old is ReuseRecord = false, new is ReuseRecord = true)
name old time/op new time/op delta
Read-8 2.75µs ± 2% 1.88µs ± 1% -31.52% (p=0.000 n=14+15)
ReadWithFieldsPerRecord-8 2.75µs ± 0% 1.89µs ± 1% -31.43% (p=0.000 n=13+13)
ReadWithoutFieldsPerRecord-8 2.77µs ± 1% 1.88µs ± 1% -32.06% (p=0.000 n=15+15)
ReadLargeFields-8 55.4µs ± 1% 54.2µs ± 0% -2.07% (p=0.000 n=15+14)
name old alloc/op new alloc/op delta
Read-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadLargeFields-8 3.94kB ± 0% 2.98kB ± 0% -24.39% (p=0.000 n=15+15)
name old allocs/op new allocs/op delta
Read-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadLargeFields-8 24.0 ± 0% 12.0 ± 0% -50.00% (p=0.000 n=15+15)
Fixes#19721
Change-Id: I79b14128bb9bb3465f53f40f93b1b528a9da6f58
Reviewed-on: https://go-review.googlesource.com/41730
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Mostly unnecessary *testing.T arguments.
Found with github.com/mvdan/unparam.
Change-Id: Ifb955cb88f2ce8784ee4172f4f94d860fa36ae9a
Reviewed-on: https://go-review.googlesource.com/41691
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There were a number of places in crypto/x509 that used hardcoded
representations of the ASN.1 NULL type, in both byte slice and
RawValue struct forms. This change adds two new exported vars to
the asn1 package for working with ASN.1 NULL in both its forms, and
converts all usages from the x509 package.
In addition, tests were added to exercise Marshal and Unmarshal on
both vars.
See #19446 for discussion.
Change-Id: I63dbd0835841ccbc810bd6ec794360a84e933f1e
Reviewed-on: https://go-review.googlesource.com/38660
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
Returns at the end of func bodies where the funcs have no return values
are pointless.
Change-Id: I0da5ea78671503e41a9f56dd770df8c919310ce5
Reviewed-on: https://go-review.googlesource.com/41093
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 27254 changed hextable to a byte array for performance.
CL 28219 fixed the compiler so that that is no longer necessary.
As Kirill notes in #15808, a string is preferable
as the linker can easily de-dup it.
So go back. No performance changes.
Change-Id: Ibef7d21d0f2507968a0606602c5dd57ed4a85b1b
Reviewed-on: https://go-review.googlesource.com/40970
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The current implementation uses a max of 28 bits when decoding an
ObjectIdentifier. This change makes it so that an int64 is used to
accumulate up to 35 bits. If the resulting data would not overflow
an int32, it is used as an int. Thus up to 31 bits may be used to
represent each subidentifier of an ObjectIdentifier.
Fixes#19933
Change-Id: I95d74b64b24cdb1339ff13421055bce61c80243c
Reviewed-on: https://go-review.googlesource.com/40436
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
The header is literally
Key: Value
If the value or the key has leading or trailing spaces, those will
be lost by the round trip.
Found because testing/quick returns different values now.
Change-Id: I0f574bdbb5990689509c24309854d8f814b5efa0
Reviewed-on: https://go-review.googlesource.com/39211
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When unmarshaling, if an element is empty, eg. '<tag></tag>', and
destination type is int, uint, float or bool, do not attempt to parse
value (""). Set to its zero value instead.
Fixes#13417
Change-Id: I2d79f6d8f39192bb277b1a9129727d5abbb2dd1f
Reviewed-on: https://go-review.googlesource.com/38386
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I2d155c838935cd8427abd142a462ff4c56829715
Reviewed-on: https://go-review.googlesource.com/37948
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This paragraph has been added, as the notion was missing from the
documentation.
If a value is passed to Encode and the type is not a struct (or pointer to struct,
etc.), for simplicity of processing it is represented as a struct of one field.
The only visible effect of this is to encode a zero byte after the value, just as
after the last field of an encoded struct, so that the decode algorithm knows when
the top-level value is complete.
Fixes#16978
Change-Id: I5f008e792d1b6fe80d2e026a7ff716608889db32
Reviewed-on: https://go-review.googlesource.com/38414
Reviewed-by: Ian Lance Taylor <iant@golang.org>
JSON decoding performs poorly for unmapped and ignored fields. We noticed better
performance when unmarshalling unused fields. The loss comes mostly from calls
to scanner.error as described at #17914.
benchmark old ns/op new ns/op delta
BenchmarkIssue10335-8 431 408 -5.34%
BenchmarkUnmapped-8 1744 1314 -24.66%
benchmark old allocs new allocs delta
BenchmarkIssue10335-8 4 3 -25.00%
BenchmarkUnmapped-8 18 4 -77.78%
benchmark old bytes new bytes delta
BenchmarkIssue10335-8 320 312 -2.50%
BenchmarkUnmapped-8 568 344 -39.44%
Fixes#17914, improves #10335
Change-Id: I7d4258a94eb287c0fe49e7334795209b90434cd0
Reviewed-on: https://go-review.googlesource.com/33276
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Tinkering with the gob package shows that is currently possible to
*completely destroy* Int slices encoding without triggering a single
test failure.
The various encInt{8,16,32,64}Slice methods are only called during the
execution of the GobMapInterfaceEncode test, which only encodes a few
slices of length exactly 1 and then just checks that the error
returned by Encode is nil (without trying to Decode back the data).
This patch adds a few tests for signed integer slices encoding.
Change-Id: Ifaaee2f32132873118b241f79aa8203e4ad31416
Reviewed-on: https://go-review.googlesource.com/38066
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Replace 'does not contains' with 'does not contain' where it appears
in the source code.
Change-Id: Ie7266347c429512c8a41a7e19142afca7ead3922
Reviewed-on: https://go-review.googlesource.com/37887
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Document and check that the alphabet cannot contain '\n' or '\r'.
Document that the alphabet cannot contain the padding character.
Document that the padding character must be equal or bellow '\xff'.
Document that the padding character must not be '\n' or '\r'.
Fixes#19343Fixes#19318
Change-Id: I6de0034d347ffdf317d7ea55d6fe38b01c2c4c03
Reviewed-on: https://go-review.googlesource.com/37838
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix Decode to return the correct illegal data index from a corrupted
input that contains whitespaces.
Fixes#19406
Change-Id: Ib2b2b6ed7e41f024d0da2bd035caec4317c2869c
Reviewed-on: https://go-review.googlesource.com/37837
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Found by github.com/mvdan/unparam.
Change-Id: Ic97f05a2ecb5b17caa36aafe403e2266abea3e0e
Reviewed-on: https://go-review.googlesource.com/37836
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Hiroshi Ioka
Russ Cox
Tim Cooper
Joe Tsai
Ian Lance Taylor
Leigh McCulloch
Lynn Boger
Michael Fraenkel
Ivan Bertona
Sam Whited
Adam Langley
Josselin Costanzi
Laurent Voisin
Marvin Stenger
Kunpei Sakai
tbunyk
Daniel Martí
Danny Rosseau
Karel Pazdera
Agniva De Sarker
Ilya Tocar
Justin Nuß
Mark Wolfe
Dmitri Shuralyov
Axel Wagner
Iccha Sethi
Ross Light
Gustav Westling
Brad Fitzpatrick
Mark Ryan
thoeni
Alberto Donizetti
Filip Gruszczyński
Gustav Westling
Bryan C. Mills
Andrew Benton
Josh Bleecher Snyder
Monis Khan
Sarah Adams
Rob Pike
Pascal S. de Kloe
Kevin Burke