Fixes#53350
Change-Id: Id5e1f4016db5f1d4349ee1a76a9dfe3aeae83cee
GitHub-Last-Rev: 45add12161
GitHub-Pull-Request: golang/go#53407
Reviewed-on: https://go-review.googlesource.com/c/go/+/412634
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Rakoczy <alex@golang.org>
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]
Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.
For #51082.
Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The existing implementation of the xml decoder uses the line number
only for reporting syntax errors. The line number of the last read
token and the column within the line is useful for the users even
in non-error conditions.
Fixes#45628
Change-Id: I37b5033ff5ff8411793d8f5180f96aa4537e83f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/311270
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Improve the test coverage of encoding/xml package by adding
the test cases for the execution paths that were not covered before.
Since it reveals a couple of issues, fix them as well while we're at it.
As I used an `strings.EqualFold` instead of adding one more `strings.ToLower`,
our fix to `autoClose()` tends to run faster as well as a result.
name old time/op new time/op delta
HTMLAutoClose-8 5.93µs ± 2% 5.75µs ± 3% -3.16% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
HTMLAutoClose-8 2.60kB ± 0% 2.58kB ± 0% -0.46% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
HTMLAutoClose-8 72.0 ± 0% 67.0 ± 0% -6.94% (p=0.000 n=10+10)
The overall `encoding/xml` test coverage increase is `88.1% -> 89.9%`;
although it may look insignificant, this CL covers some important corner cases,
like `autoClose()` functionality (that was not tested at all).
Fixes#49635Fixes#49636
Change-Id: I50b2769896c197eb285672313b7148f4fe8bdb38
Reviewed-on: https://go-review.googlesource.com/c/go/+/364734
Trust: Bryan Mills <bcmills@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
When encoding a xml attribute is zero value (IsValid == false), we need
a `continue` to jump over the attribute. If not, followed marshalAttr
function will panic.
Fixes: #50164
Change-Id: I42e064558e7becfbf47728b14cbf5c7afa1e8798
Reviewed-on: https://go-review.googlesource.com/c/go/+/385514
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.
Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.
A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.
Fixes#49884.
Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Fixes#48521
Change-Id: Id8402bcff243c0ab19e4ec0b138b9af8c111f88d
Reviewed-on: https://go-review.googlesource.com/c/go/+/355492
Trust: Keith Randall <khr@golang.org>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
Many uses of Index/IndexByte/IndexRune/Split/SplitN
can be written more clearly using the new Cut functions.
Do that. Also rewrite to other functions if that's clearer.
For #46336.
Change-Id: I68d024716ace41a57a8bf74455c62279bde0f448
Reviewed-on: https://go-review.googlesource.com/c/go/+/351711
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
xml names can't have any of '[],' in them, which might appear in
generic type names. Truncate at the first '[' so the names are still valid.
Fixes#48318
Change-Id: I110ff4269f763089467e7cf84b0f0c5075fb44b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/349349
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
A Directive (like <!ENTITY xxx []>) can't have other nodes nested inside
it (in our data structure representation), so there is no way to
preserve comments. The previous behavior was to just elide them, which
however might change the semantic meaning of the surrounding markup.
Instead, replace them with a space which hopefully has the same semantic
effect of the comment.
Directives are not actually a node type in the XML spec, which instead
specifies each of them separately (<!ENTITY, <!DOCTYPE, etc.), each with
its own grammar. The rules for where and when the comments are allowed
are not straightforward, and can't be implemented without implementing
custom logic for each of the directives.
Simply preserving the comments in the body of the directive would be
problematic, as there can be unmatched quotes inside the comment.
Whether those quotes are considered meaningful semantically or not,
other parsers might disagree and interpret the output differently.
This issue was reported by Juho Nurminen of Mattermost as it leads to
round-trip mismatches. See #43168. It's not being fixed in a security
release because round-trip stability is not a currently supported
security property of encoding/xml, and we don't believe these fixes
would be sufficient to reliably guarantee it in the future.
Fixes CVE-2020-29510
Updates #43168
Change-Id: Icd86c75beff3e1e0689543efebdad10ed5178ce3
Reviewed-on: https://go-review.googlesource.com/c/go/+/277893
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Before this change, <:name> would parse as <name>, which could cause
issues in applications that rely on the parse-encode cycle to
round-trip. Similarly, <x name:=""> would parse as expected but then
have the attribute dropped when serializing because its name was empty.
Finally, <a🅱️c> would parse and get serialized incorrectly. All these
values are invalid XML, but to minimize the impact of this change, we
parse them whole into Name.Local.
This issue was reported by Juho Nurminen of Mattermost as it leads to
round-trip mismatches. See #43168. It's not being fixed in a security
release because round-trip stability is not a currently supported
security property of encoding/xml, and we don't believe these fixes
would be sufficient to reliably guarantee it in the future.
Fixes CVE-2020-29509
Fixes CVE-2020-29511
Updates #43168
Change-Id: I68321c4d867305046f664347192948a889af3c7f
Reviewed-on: https://go-review.googlesource.com/c/go/+/277892
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
This change properly handles a TokenReader which
returns an EOF in the middle of an open XML
element.
Thanks to Sam Whited for reporting this.
Fixes CVE-2021-27918
Fixes#44913
Change-Id: Id02a3f3def4a1b415fa2d9a8e3b373eb6cb0f433
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004594
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/300391
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
The IsExported method is a more intuitive helper for checking whether
the method or field is exported than checking whether PkgPath is empty.
In the same CL, modify the standard library to make use of this helper.
Fixes#41563
Change-Id: Iaacfb3b74449501f98e2707aa32095a32bd3c3c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/266197
Trust: Joe Tsai <joetsai@digital-static.net>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fixes the check for the reserved namespace prefix
"xml" to be case insensitive, so as to match all variants of:
(('X'|'x')('M'|'m')('L'|'l'))
as mandated by Section 2.3 of https://www.w3.org/TR/REC-xml/
This is a roll forward of CL 203417, which was rolled back by CL 240179.
We've decided that the roll back was incorrect, and any broken tests
should be fixed.
The original CL 203417 was by Tamás Gulácsi.
Fixes#35151
For #39876
Change-Id: I2e6daa7aeb252531fba0b8a56086613e13059528
Reviewed-on: https://go-review.googlesource.com/c/go/+/264024
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This reverts CL 203417.
Reason for revert: This change changes uses of tags like "XMLSchema-instance" without any recourse.
For #35151Fixes#39876
Change-Id: I4c85c8267a46b3748664b5078794dafffb42aa26
Reviewed-on: https://go-review.googlesource.com/c/go/+/240179
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
fieldInfo.value used to initialize nil anonymous struct fields if they
were encountered. This behavior is wanted when decoding, but not when
encoding. When encoding, the value should never be modified, and these
nil fields should be skipped entirely.
To fix the bug, add a bool argument to the function which tells the
code whether we are encoding or decoding.
Finally, add a couple of tests to cover the edge cases pointed out in
the original issue.
Fixes#27240.
Change-Id: Ic97ae4bfe5f2062c8518e03d1dec07c3875e18f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/196809
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Fixes the check for the reserved namespace prefix
"xml" to be case insensitive, so as to match all variants of:
(('X'|'x')('M'|'m')('L'|'l'))
as mandated by Section 2.3 of https://www.w3.org/TR/REC-xml/Fixes#35151.
Change-Id: Id5a98e5f9d69d3741dc16f567c4320f1ad0b3c70
Reviewed-on: https://go-review.googlesource.com/c/go/+/203417
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Instead use string(r) where r has type rune.
This is in preparation for a vet warning for string(i).
Updates #32479
Change-Id: Ic205269bba1bd41723950219ecfb67ce17a7aa79
Reviewed-on: https://go-review.googlesource.com/c/go/+/220844
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Akhil Indurti <aindurti@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com>
The documentation for TokenReader suggests that implementations of the
interface may return a token and io.EOF together, indicating that it is
the last token in the stream. This is similar to io.Reader. However, if
you wrap such a TokenReader in a Decoder it complained about the EOF.
A test was added to ensure this behavior on Decoder's.
Change-Id: I9083c91d9626180d3bcf5c069a017050f3c7c4a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/130556
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Some were never used, and some haven't been used for years.
One exception is net/http's readerAndCloser, which was only used in a
test. Move it to a test file.
While at it, remove a check in regexp that could never fire; the field
is an uint32, so it can never be negative.
Change-Id: Ia2200f6afa106bae4034045ea8233b452f38747b
Reviewed-on: https://go-review.googlesource.com/c/go/+/192621
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Per the code review guidelines: "Words in names that are
initialisms or acronyms have a consistent case."
Change-Id: I347b02d2f48455f2cbbc040191ba197e3e8f23fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/191970
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Both the encoding/xml and encoding/json packages support custom
marshalers for JSON and XML, as well as the basic encoding.TextMarshaler
and encoding.TextUnmarshaler interfaces, but the docs and examples for
these are missing.
There are docs for how to use encoding.TextMarshaler and
encoding.TextUnmarshaler in encoding/json, but not encoding/xml. There
are no examples for how to use them with either json or xml. This commit
includes docs for encoding/xml and examples for both encoding/json and
encoding/xml.
There is an example using custom marshalers MarshalJSON and
UnmarshalJSON in encoding/json, but not MarshalXML and UnmarshalXML in
encoding/json. These docs are more so necessary for encoding/xml because
the complexities of XML documents is significantly greater than JSON
documents which more often leads to the need for custom marshaling. The
encoding/json package includes an example of how to write a custom
marshaler, and this commit includes the same example for the xml
package.
All examples are mirrored off the existing custom marshaler example in
encoding/json.
Fixes#6859
Change-Id: Ic93abc27c0b4d5e48dea6ede4e20b1bedca4ab39
Reviewed-on: https://go-review.googlesource.com/c/76350
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
They didn't even have public types, which made them pretty mysterious.
Give them types and reference the Decoder, which uses them.
Also, refer them qualified by their package name in the examples, as
we usually do in example*.go files, which usually use package foo_test
specifically so we can show the package names along with the symbols.
Change-Id: I50ebbbf43778c1627bfa526f8824f52c7953454f
Reviewed-on: https://go-review.googlesource.com/127663
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Each URL was manually verified to ensure it did not serve up incorrect
content.
Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Section 2.2 of the referenced spec http://www.xml.com/axml/testaxml.htm
defines 0xD7FF as a (sub)range boundary, not 0xDF77.
Fixes#25172
Change-Id: Ic5a3328cd46ef6474b8e93c4a343dcfba0e6511f
Reviewed-on: https://go-review.googlesource.com/109495
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It appears that old code (from 2009) in xml.(*Decoder).rawToken
replicates append's slice-growing functionality by allocating a new,
bigger backing array and then calling copy.
Simplifying the code by replacing it with a single append call does
not seem to hurt performance:
name old time/op new time/op delta
Marshal-4 11.2µs ± 1% 11.3µs ±10% ~ (p=0.069 n=19+17)
Unmarshal-4 28.6µs ± 1% 28.4µs ± 1% -0.60% (p=0.000 n=20+18)
name old alloc/op new alloc/op delta
Marshal-4 5.78kB ± 0% 5.78kB ± 0% ~ (all equal)
Unmarshal-4 8.61kB ± 0% 8.27kB ± 0% -3.90% (p=0.000 n=20+20)
name old allocs/op new allocs/op delta
Marshal-4 23.0 ± 0% 23.0 ± 0% ~ (all equal)
Unmarshal-4 189 ± 0% 190 ± 0% +0.53% (p=0.000 n=20+20)
Change-Id: Ie580d1216a44760e611e63dee2c339af5465aea5
Reviewed-on: https://go-review.googlesource.com/86655
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 58210 introduced this constant for reasons I don't understand.
It should not be in the exported const block, which will pollute
godoc output with a "... unexported" notice.
Also since we already have a constant named xmlnsPrefix for "xmlns",
it is very confusing to also have xmlNamespacePrefix for "xml".
If we must have the constant at all, rename it to xmlPrefix.
Change-Id: I15f937454d730005816fcd32b1acca703acf1e51
Reviewed-on: https://go-review.googlesource.com/78121
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The docs for xml.Marshal state that the XML elements name is derived
from one of five locations in a specific order of precedence, but does
not mention that if the field is a struct type and has its name defined
in a tag and in the types XMLName field that an error will occur. This
is documented in the structFieldInfo function but not in the function
documentation, and the existing docs in Marshal are misleading without
this behavior being discussed.
Fixes#18564
Change-Id: I29042f124a534bd1bc993f1baeddaa0af2e72fed
Reviewed-on: https://go-review.googlesource.com/76321
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The go repository contains a mix of github.com/golang/go/issues/xxxxx
and golang.org/issues/xxxxx URLs for references to issues in the issue
tracker. We should use one for consistency, and golang.org is preferred
in case the project moves the issue tracker in the future.
This reasoning is taken from a comment Sam Whited left on a CL I
recently opened: https://go-review.googlesource.com/c/go/+/73890.
In that CL I referenced an issue using its github.com URL, because other
tests in the file I was changing contained references to issues using
their github.com URL. Sam Whited left a comment on the CL stating I
should change it to the golang.org URL.
If new code is intended to reference issues via golang.org and not
github.com, existing code should be updated so that precedence exists
for contributors who are looking at the existing code as a guide for the
code they should write.
Change-Id: I3b9053fe38a1c56fc101a8b7fd7b8f310ba29724
Reviewed-on: https://go-review.googlesource.com/75673
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Whitespace is ignored in bool values and attrs. It is convenient and
relatively safe since whitespace around a bool value is often
unimportant. The same logic can be applied to numeric values of types
int, uint, and float.
Fixes#22146
Change-Id: Ie0462def90304af144b8e2e72d85b644857c27cc
Reviewed-on: https://go-review.googlesource.com/73891
Reviewed-by: Sam Whited <sam@samwhited.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Whitespace is ignored in bool values and attrs, but there are no tests
capturing this behavior.
Change-Id: I7a7249de4886f510869e91de937e69b83c3254c8
Reviewed-on: https://go-review.googlesource.com/73890
Reviewed-by: Sam Whited <sam@samwhited.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
strings.LastIndexByte was introduced in go1.5 and it can be used
effectively wherever the second argument to strings.LastIndex is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.LastIndex.
Change-Id: I7b5679d616197b055cffe6882a8675d24a98b574
Reviewed-on: https://go-review.googlesource.com/66372
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.IndexByte was introduced in go1.2 and it can be used
effectively wherever the second argument to strings.Index is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.Index.
Change-Id: I1ab5edb7c4ee9058084cfa57cbcc267c2597e793
Reviewed-on: https://go-review.googlesource.com/65930
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Went mainly for the ones that make no sense, such as the ones
mid-sentence or after commas.
Change-Id: Ie245d2c19cc7428a06295635cf6a9482ade25ff0
Reviewed-on: https://go-review.googlesource.com/57293
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Roland Shoemaker
shaoliming
John Bampton
Russ Cox
Patrick Gundlach
Iskander Sharipov
hopehook
Keith Randall
Cuong Manh Le
Filippo Valsorda
Katie Hockman
Joe Tsai
Ian Lance Taylor
Daniel Martí
Tamás Gulácsi
Sam Whited
Eddie Scholtz
Leigh McCulloch
Brad Fitzpatrick
Tim Cooper
Artyom Pervukhin
tengufromsky
Ryuma Yoshida
Alberto Donizetti
Igor Vashyst
Marvin Stenger
Karel Pazdera
Bryan C. Mills