Semicolons are no longer valid separators, so
net/url.ParseQuery will now return an error
if any part of the query contains a semicolon.
net/http.(*Request).ParseMultipartForm has been
changed to fall through and continue parsing
even if the call to (*Request).ParseForm fails.
This change also includes a few minor refactors
to existing tests.
Fixes#25192
Change-Id: Iba3f108950fb99b9288e402c41fe71ca3a2ababd
Reviewed-on: https://go-review.googlesource.com/c/go/+/325697
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Current MaxBytesReader behaviour differs from its documentation. It's
not similar enough to io.LimitReader. It panics when limit (n) < -1 and
returns [-1, <nil>] when limit (n) = -1. To fix that, we treat all
negative limits as equivalent to 0.
It would be possible to make MaxBytesReader analogically identical in
behaviour to io.LimitReader, but that would require to stop
maxBytesReader's Read from reading past the limit. Read always reads one
more byte (if possible) for non-negative limits and returns a non-EOF
error. This behaviour will now apply to all limits.
Fixes#45101
Change-Id: I25d1877dbff1eb4b195c8741fe5e4a025d01ebc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/303171
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Damien Neil <dneil@google.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)
Update the Go tree to use the preferred names.
As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.
ReadDir changes are in a separate CL, because they are not a
simple search and replace.
For #42026.
Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Returning an error about integer overflow is needlessly pedantic.
The meaning of ReadForm(MaxInt64) is easily understood
(accept a lot of data) and can be implemented.
Fixes#40430.
Change-Id: I8a522033dd9a2f9ad31dd2ad82cf08d553736ab9
Reviewed-on: https://go-review.googlesource.com/c/go/+/275112
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Tests that if the combination of:
* HTTP multipart file payload size
* ParseMultipartForm's maxMemory parameter
* the internal leeway buffer size of 10MiB
overflows, then we'll report an overflow instead of silently
passing.
Reapplies and fixes CL 254977, which was reverted in CL 263658.
The prior test lacked a res.Body.Close(), so fixed that and
added a leaked Transport check to verify correctness.
Updates 40430.
Change-Id: I3c0f7ef43d621f6eb00f07755f04f9f36c51f98f
Reviewed-on: https://go-review.googlesource.com/c/go/+/263817
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Damien Neil <dneil@google.com>
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.
For #41190.
Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
This reverts CL 254977.
Reason for revert: introduced test failures on longtest builders.
Change-Id: I75e868245f980189ad85dd4103d9178989e06ecf
Reviewed-on: https://go-review.googlesource.com/c/go/+/263658
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
ParseMultipartForm has been changed to return an error if maxMemory
parameter + 10MB causes int overflows. This adds a test for the new
behaviour.
For #40430
Change-Id: I4f66ce8a9382940182011d22a84ee52b1d1364cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/254977
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Use names to better communicate when a test case fails.
Change-Id: Id882783cb5e444b705443fbcdf612713f8a3b032
Reviewed-on: https://go-review.googlesource.com/c/go/+/187823
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
For Go 1.13 we introduced Header.Clone and it returns
nil if a nil Header is cloned. Unfortunately, though,
this exported Header.Clone nil behavior differed from
the old Go 1.12 and earlier internal header clone
behavior which always returned non-nil Headers.
This CL fixes the places where that distinction mattered.
Fixes#34878
Change-Id: Id19dea2272948c8dd10883b18ea7f7b8b33ea8eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/200977
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds benchmarks to test file uploads using PUT requests.
It's designed to complement changes https://golang.org/cl/163599 and
https://golang.org/cl/163737, allowing an easy comparison of
performance before and after these changes are applied.
Updates #30377.
Co-authored-by: Emmanuel Odeke <emm.odeke@gmail.com>
Change-Id: Ib8e692c61e1f7957d88c7101669d4f7fb8110c65
GitHub-Last-Rev: 242622b4fc
GitHub-Pull-Request: golang/go#30424
Reviewed-on: https://go-review.googlesource.com/c/go/+/163862
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)
Updates #27864
Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Replace references to the obsoleted RFC 2616 with references to RFC
7230 through 7235, to avoid unnecessary confusion.
Obvious inconsistencies are marked with todo comments.
Updates #21974
Change-Id: I8fb4fcdd1333fc5193b93a2f09598f18c45e7a00
Reviewed-on: https://go-review.googlesource.com/94095
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Despite the previously known behavior of Request.WithContext
shallow copying a request, usage of the request inside server.ServeHTTP
mutates the request's URL. This CL implements deep copying of the URL.
Fixes#20068
Change-Id: I86857d7259e23ac624d196401bf12dde401c42af
Reviewed-on: https://go-review.googlesource.com/41308
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The introduction of NoBody and related body-peeking bug fixes also
added a "cleanup" of sorts to make NewRequest set the returned
Requests's ContentLength to -1 when it didn't know it.
Using -1 to mean unknown is what the documentation says, but then
people apparently(?) depended on it being zero so they could do this:
req, _ := http.NewRequest("POST", url, someNonNilReaderWithUnkownSize)
req.Body = nil
res, err := http.DefaultClient.Do(req)
... and expect it to work.
After https://golang.org/cl/31445 the contrived(?) code above stopped
working, since Body was nil and ContentLength was -1, which has been
disallowed since Go 1.0.
So this restores the old behavior of NewRequest, not setting it to -1.
That part of the fix isn't required as of https://golang.org/cl/31726
(which added NoBody)
I still don't know whether this bug is hypothetical or actually
affected people in practice.
Let's assume it's real for now.
Fixes#18117
Change-Id: I42400856ee92a1a4999b5b4668bef97d885fbb53
Reviewed-on: https://go-review.googlesource.com/33801
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
NoBody is new in Go 1.8.
Found while investigating #18117
Change-Id: I6bda030f358e2270f090d108cb3a89c8a2665fcb
Reviewed-on: https://go-review.googlesource.com/33714
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is an alternate solution to https://golang.org/cl/31445
Instead of making NewRequest return a request with Request.Body == nil
to signal a zero byte body, add a well-known variable that means
explicitly zero.
Too many tests inside Google (and presumably the outside world)
broke.
Change-Id: I78f6ecca8e8aa1e12179c234ccfb6bcf0ee29ba8
Reviewed-on: https://go-review.googlesource.com/31726
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
This CL makes NewRequest set Body nil for known-zero bodies, and makes
the http1 Transport not peek-Read a byte to determine whether there's
a body.
Background:
Many fields of the Request struct have different meanings for whether
they're outgoing (via the Transport) or incoming (via the Server).
For outgoing requests, ContentLength and Body are documented as:
// Body is the request's body.
//
// For client requests a nil body means the request has no
// body, such as a GET request. The HTTP Client's Transport
// is responsible for calling the Close method.
Body io.ReadCloser
// ContentLength records the length of the associated content.
// The value -1 indicates that the length is unknown.
// Values >= 0 indicate that the given number of bytes may
// be read from Body.
// For client requests, a value of 0 with a non-nil Body is
// also treated as unknown.
ContentLength int64
Because of the ambiguity of what ContentLength==0 means, the http1 and
http2 Transports previously Read the first byte of a non-nil Body when
the ContentLength was 0 to determine whether there was an actual body
(with a non-zero length) and ContentLength just wasn't populated, or
it was actually empty.
That byte-sniff has been problematic and gross (see #17480, #17071)
and was removed for http2 in a previous commit.
That means, however, that users doing:
req, _ := http.NewRequest("POST", url, strings.NewReader(""))
... would not send a Content-Length header in their http2 request,
because the size of the reader (even though it was known, being one of
the three common recognized types from NewRequest) was zero, and so
the HTTP Transport thought it was simply unset.
To signal explicitly-zero vs unset-zero, this CL changes NewRequest to
signal explicitly-zero by setting the Body to nil, instead of the
strings.NewReader("") or other zero-byte reader.
This CL also removes the byte sniff from the http1 Transport, like
https://golang.org/cl/31326 did for http2.
Updates #17480
Updates #17071
Change-Id: I329f02f124659bf7d8bc01e2c9951ebdd236b52a
Reviewed-on: https://go-review.googlesource.com/31445
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Renamed TestPOSTQuery to TestParseFormQuery and added testing
for the ";" delimiter, an empty key, an empty value and an
empty key + value.
Also added TestParseFormQueryMethods to make sure forms sent in
PATCH and PUT (and no others) request are parsed correctly in
ParseForm.
Fixes#17368
Change-Id: I445aad324ffc7b38d179ea41953bffbac0cddffe
Reviewed-on: https://go-review.googlesource.com/30555
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This reverts commit 59320c396e.
Reasons:
This CL was causing failures on a large regression test that we run
within Google. The issues arises from two bugs in the CL:
* The CL dropped support for ';' as a delimiter (see https://golang.org/issue/2210)
* The handling of an empty string caused an empty record to be added when
no record was added (see https://golang.org/cl/30454 for my attempted fix)
The logic being added is essentially a variation of url.ParseQuery,
but altered to accept an io.Reader instead of a string.
Since it is duplicated (but modified) logic, there needs to be good
tests to ensure that it's implementation doesn't drift in functionality
from url.ParseQuery. Fixing the above issues and adding the associated
regression tests leads to >100 lines of codes.
For a 4% reduction in CPU time, I think this complexity and duplicated
logic is not worth the effort.
As such, I am abandoning my efforts to fix the existing issues and
believe that reverting CL/20301 is the better course of action.
Updates #14655
Change-Id: Ibb5be0a5b48a16c46337e213b79467fcafee69df
Reviewed-on: https://go-review.googlesource.com/30470
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Remove the use of io.ReadAll in http.parsePostForm to avoid converting
the whole input from []byte to string and not performing well
space-allocated-wise.
Instead a new function called parsePostFormURLEncoded is used and is
fed directly an io.Reader that is parsed using a bufio.Reader.
Benchmark:
name old time/op new time/op delta
PostQuery-4 2.90µs ± 6% 2.82µs ± 4% ~ (p=0.094 n=9+9)
name old alloc/op new alloc/op delta
PostQuery-4 1.05kB ± 0% 0.90kB ± 0% -14.49% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
PostQuery-4 6.00 ± 0% 7.00 ± 0% +16.67% (p=0.000 n=10+10)
Fixes#14655
Change-Id: I112c263d4221d959ed6153cfe88bc57a2aa8ea73
Reviewed-on: https://go-review.googlesource.com/20301
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
- Ensures that the empty port and preceeding ":"
in a URL.Host are stripped.
Normalize the empty port in a URL.Host's ":port" as
mandated by RFC 3986 Section 6.2.3 which states that:
`Likewise an explicit ":port", for which the port is empty or
the default for the scheme, is equivalent to one where the port
and its ":" delimiter are elided and thus should be
removed by scheme-based normalization.`
- Moves function `hasPort` from client.go (where it was defined but
not used directly), to http.go the common area.
Fixes#14836
Change-Id: I2067410377be9c71106b1717abddc2f8b1da1c03
Reviewed-on: https://go-review.googlesource.com/22140
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ensures that after request.ParseMultipartForm has been invoked,
Request.PostForm and Request.Form are both populated with the
same formValues read in, instead of only populating Request.Form.
Fixes#9305
Change-Id: I3d4a11b006fc7dffaa35360014fe15b8c74d00a3
Reviewed-on: https://go-review.googlesource.com/19986
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Host names in URLs must not use %-escaping for ASCII bytes, per RFC 3986.
url.Parse has historically allowed spaces and < > " in the URL host.
In Go 1.5, URL's String method started escaping those,
but then Parse would rejects the escaped form.
This CL is an attempt at some consistency between Parse and String
as far as the accepted host characters and the encoding of host characters,
so that if Parse succeeds, then Parse -> String -> Parse also succeeds.
Allowing space seems like a mistake, so reject that in Parse.
(Similarly, reject \t, \x01, and so on, all of which were being allowed.)
Allowing < > " doesn't seem awful, so continue to do that,
and go back to the Go 1.4 behavior of not escaping them in String.
Fixes#11302.
Change-Id: I0bf65b874cd936598f20694574364352a5abbe5f
Reviewed-on: https://go-review.googlesource.com/17387
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Until recently, we always permitted an empty string to NewRequest.
Keep that property, since it broke tests within in Google when trying
out Go 1.6, and probably would've broken others too.
Change-Id: Idddab1ae7b9423d5caac00af2c897fe1065b600b
Reviewed-on: https://go-review.googlesource.com/17699
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A malformed Host header can result in a malformed HTTP request.
Clean them to avoid this.
Updates #11206. We may come back and make this stricter for 1.6.
Change-Id: I23c7d821cd9dbf66c3c15d26750f305e3672d984
Reviewed-on: https://go-review.googlesource.com/11241
Reviewed-by: Russ Cox <rsc@golang.org>
Includes a new net/http test too.
Fixes#11202
Change-Id: I61edc594f4de8eb6780b8dfa221269dd482e8f35
Reviewed-on: https://go-review.googlesource.com/11492
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
When making a request to an IPv6 address with a zone identifier, for
exmaple [fe80::1%en0], RFC 6874 says HTTP clients must remove the zone
identifier "%en0" before writing the request for security reason.
This change removes any IPv6 zone identifer attached to URI in the Host
header field in requests.
Fixes#9544.
Change-Id: I7406bd0aa961d260d96f1f887c2e45854e921452
Reviewed-on: https://go-review.googlesource.com/3111
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Katie Hockman
ian woolf
Artur M. Wolff
Russ Cox
Emmanuel T Odeke
Bryan C. Mills
avivklas
dqu123
David Ndungu
Leon Klingele
Chris Marchesi
Brad Fitzpatrick
OneOfOne
David Url
Emmanuel Odeke
Quentin Renard
Joe Tsai
Martin Möhrmann
Jeff R. Allen
Mikio Hara
Jongmin Kim