Commit Graph

1119 Commits

Author SHA1 Message Date
Brad Fitzpatrick 0ba0ea172d net/http: document that Request.Host includes HTTP/2 :authority
Fixes #34640

Change-Id: I4a6c9414fe369cd5e9915472331c4bd8a21d8b0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/198457
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2019-10-02 16:47:29 +00:00
Kyohei Kadota 148ec3e3bc net/http/cgi: skip tests if not functional perl
TestEnvOverride sets PATH to /wibble before executing a CGI.
So customized Perl that is starting with '#!/usr/bin/env bash' will fail
because /usr/bin/env can't lookup bash.

Fixes #27790

Change-Id: I25e433061a7ff9da8c86429e934418fc15f12f90
Reviewed-on: https://go-review.googlesource.com/c/go/+/196845
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-02 05:52:16 +00:00
Nuno Cruces a49067aab6 net/http: avoid sending unspecified time for directories
Change applies to sendFile.
This is already done for sendContent.

Change-Id: If43d9ab99e6e66a1363b08e0bdcceb57df1f855c
GitHub-Last-Rev: 1c47620a09
GitHub-Pull-Request: golang/go#34631
Reviewed-on: https://go-review.googlesource.com/c/go/+/198139
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-01 15:27:28 +00:00
Emmanuel T Odeke 55738850c4 net/http: remove TestTimeoutHandlerAndFlusher due to flakes
Removes TestTimeoutHandlerAndFlusher due to flakes on
one of the builders due to timing issues.

Perhaps later, we might need to bring it back when we've
figured out the timing issues.

Fixes #34573.

Change-Id: Ia88d4da31fb228296144dc31f9a4288167fb4a53
Reviewed-on: https://go-review.googlesource.com/c/go/+/197757
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-27 15:38:16 +00:00
Emmanuel T Odeke 4faf8a8dc4 net/http, doc/go1.13.html: revert TimeoutHandler.Flush
Also added a test to ensure that any interactions
between TimeoutHandler and Flusher result in the
correct status code and body, but also that we don't
get superfluous logs from stray writes as was seen
in the bug report.

Fixes #34439.

Change-Id: I4af62db256742326f9353f98a2fcb5f71d2a5fd9
Reviewed-on: https://go-review.googlesource.com/c/go/+/197659
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 21:12:34 +00:00
Filippo Valsorda 41b1f88efa net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like

X-Answer : 42

being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.

On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.

For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.

This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn

net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.

Fixes CVE-2019-16276
Fixes #34540

Change-Id: I6d272de827e0870da85d93df770d6a0e161bbcf1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/549719
Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/197503
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-09-26 16:43:06 +00:00
Brad Fitzpatrick 7fc2625ef1 net/http: propagate Client.Timeout down into Request's context deadline
Fixes #31657

Change-Id: I85e9595d3ea30d410f1f4b787925a6879a72bdf2
Reviewed-on: https://go-review.googlesource.com/c/go/+/175857
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-25 04:18:18 +00:00
Michael Fraenkel eb4e5defb4 net/http: remove http2 connections when no longer cached
When the http2 transport returns a NoCachedConnError, the connection
must be removed from the idle list as well as the connections per host.

Fixes #34387

Change-Id: I7875c9c95e694a37a339bb04385243b49f9b20d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/196665
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-24 15:21:08 +00:00
Tom Thorogood 582d5194fa net/http: fix HTTP/2 idle pool tracing
CL 140357 caused HTTP/2 connections to be put in the idle pool, but
failed to properly guard the trace.GotConn call in getConn. dialConn
returns a minimal persistConn with conn == nil for HTTP/2 connections.
This persistConn was then returned from queueForIdleConn and caused the
httptrace.GotConnInfo passed into GotConn to have a nil Conn field.

HTTP/2 connections call GotConn themselves so leave it for HTTP/2 to call
GotConn as is done directly below.

Fixes #34282

Change-Id: If54bfaf6edb14f5391463f908efbef5bb8a5d78e
GitHub-Last-Rev: 2b7d66a1ce
GitHub-Pull-Request: golang/go#34283
Reviewed-on: https://go-review.googlesource.com/c/go/+/195237
Reviewed-by: Michael Fraenkel <michael.fraenkel@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-18 18:52:45 +00:00
Ainar Garipov 0efbd10157 all: fix typos
Use the following (suboptimal) script to obtain a list of possible
typos:

  #!/usr/bin/env sh

  set -x

  git ls-files |\
    grep -e '\.\(c\|cc\|go\)$' |\
    xargs -n 1\
    awk\
    '/\/\// { gsub(/.*\/\//, ""); print; } /\/\*/, /\*\// { gsub(/.*\/\*/, ""); gsub(/\*\/.*/, ""); }' |\
    hunspell -d en_US -l |\
    grep '^[[:upper:]]\{0,1\}[[:lower:]]\{1,\}$' |\
    grep -v -e '^.\{1,4\}$' -e '^.\{16,\}$' |\
    sort -f |\
    uniq -c |\
    awk '$1 == 1 { print $2; }'

Then, go through the results manually and fix the most obvious typos in
the non-vendored code.

Change-Id: I3cb5830a176850e1a0584b8a40b47bde7b260eae
Reviewed-on: https://go-review.googlesource.com/c/go/+/193848
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-09-08 17:28:20 +00:00
smasher164 141b09726d net/http: make copyValues append instead of calling Add
This results in a performance boost:

name          old time/op    new time/op    delta
CopyValues-4    3.46µs ± 3%    1.53µs ± 3%  -55.85%  (p=0.000 n=18+19)

name          old alloc/op   new alloc/op   delta
CopyValues-4    1.52kB ± 0%    0.74kB ± 0%  -51.58%  (p=0.000 n=20+20)

name          old allocs/op  new allocs/op  delta
CopyValues-4      24.0 ± 0%      11.0 ± 0%  -54.17%  (p=0.000 n=20+20)

Fixes #33744.

Change-Id: Ibc653fb076a9a6aaa775fcc9ca720fb90e68cf96
Reviewed-on: https://go-review.googlesource.com/c/go/+/191057
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-09-07 22:21:30 +00:00
Daniel Martí 03ac39ce5e std: remove unused bits of code all over the place
Some were never used, and some haven't been used for years.

One exception is net/http's readerAndCloser, which was only used in a
test. Move it to a test file.

While at it, remove a check in regexp that could never fire; the field
is an uint32, so it can never be negative.

Change-Id: Ia2200f6afa106bae4034045ea8233b452f38747b
Reviewed-on: https://go-review.googlesource.com/c/go/+/192621
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-02 12:57:37 +00:00
spacewander 4a457e3edc net/http: merge IsDir checks in fs.go's serveFile function
Fixes #33385

Change-Id: I497ccd868d408a9c5648c72aa5ce41221368daf4
GitHub-Last-Rev: 3bf483808e
GitHub-Pull-Request: golang/go#33423
Reviewed-on: https://go-review.googlesource.com/c/go/+/188677
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-02 11:13:03 +00:00
Derek Phan 9c295bdeab net/http: remove redundant port validation
The URL port is already checked in net/url, so we can remove the redundant validation in net/http.

Fixes #33600

Change-Id: I62511a452df6262d4b66180933758d34627ff9df
GitHub-Last-Rev: c19afa3102
GitHub-Pull-Request: golang/go#33611
Reviewed-on: https://go-review.googlesource.com/c/go/+/190057
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-02 09:45:08 +00:00
Brad Fitzpatrick 7ed973b4d9 net/http: don't panic serving dir in ServeFile with empty Request.URL.Path
Updates #30165
Updates #31622

Change-Id: I7a4b91aa7c5c3af8c0b1273cbb42046feddf7d78
Reviewed-on: https://go-review.googlesource.com/c/go/+/180499
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-28 18:38:52 +00:00
Daniel Martí 81fde0c941 net/http: fix a long test after CL 185117
The net/url error in question now quotes the URL, so update the expected
output string.

While at it, also update a comment in httputil, though that doesn't
affect any test.

Fixes #33910.

Change-Id: I0981f528b24337c2952ef60c0db3b7ff72d72110
Reviewed-on: https://go-review.googlesource.com/c/go/+/192078
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-28 14:02:41 +00:00
Stefan Baebler 64cfe9fe22 net/url: improve url parsing error messages by quoting
Current implementation doesn't always make it obvious what the exact
problem with the URL is, so this makes it clearer by consistently quoting
the invalid URL, as is the norm in other parsing implementations, eg.:
strconv.Atoi(" 123") returns an error: parsing " 123": invalid syntax

Updates #29261

Change-Id: Icc6bff8b4a4584677c0f769992823e6e1e0d397d
GitHub-Last-Rev: 648b9d93fe
GitHub-Pull-Request: golang/go#29384
Reviewed-on: https://go-review.googlesource.com/c/go/+/185117
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-28 12:47:06 +00:00
Agniva De Sarker e7a4ab427d net/http/httputil: fix goroutine leak for DumpRequestOut
When an invalid URL was passed to DumpRequestOut, it would directly return
without gracefully shutting down the reader goroutine.

So we create a channel and signal the reader goroutine to exit
if an error occurs during roundtrip.

Fixes #32571

Change-Id: I8c2970f1601e599f3d1ebfed298faf5f5716fc2c
Reviewed-on: https://go-review.googlesource.com/c/go/+/182037
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-28 04:44:32 +00:00
Anderson Queiroz 2b59894425 net/http: enhance documentation for Server.Addr
Fixes golang/go#31249

Change-Id: I3280f8ab170ed31d4efb71106533e016d430d44c
Reviewed-on: https://go-review.googlesource.com/c/go/+/191557
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-08-27 20:52:49 +00:00
Bryan C. Mills 94bf9a8d4a net/http: fix wantConnQueue memory leaks in Transport
I'm trying to keep the code changes minimal for backporting to Go 1.13,
so it is still possible for a handful of entries to leak,
but the leaks are now O(1) instead of O(N) in the steady state.

Longer-term, I think it would be a good idea to coalesce idleMu with
connsPerHostMu and clear entries out of both queues as soon as their
goroutines are done waiting.

Fixes #33849
Fixes #33850

Change-Id: Ia66bc64671eb1014369f2d3a01debfc023b44281
Reviewed-on: https://go-review.googlesource.com/c/go/+/191964
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-08-27 18:23:13 +00:00
Javier Revillas cd33d271b4 net/http: fix a typo in comments
HTTP is an initialism, not an acronym, where you pronounce each letter as a
word. It's "an H", not "a H".

Running `find src/net/http -type f | xargs grep -n 'an HTTP' | wc -l` shows
that the "an HTTP" form is used 67 times across the `net/http` package.
Furthermore, `find src/net/http -type f | xargs grep -n 'a HTTP' | wc -l`
yields only 4 results.

Change-Id: I219c292a9e2c9bf7a009dbfe82ea8b15874685e9
GitHub-Last-Rev: 6ebd095023
GitHub-Pull-Request: golang/go#33810
Reviewed-on: https://go-review.googlesource.com/c/go/+/191700
Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-08-27 16:50:35 +00:00
Kevin Gillette 8b03a3992b net/http: make docs refer to Context.Value as a getter instead of context.WithValue
The doc comments of both ServerContextKey and LocalAddrContextKey both suggest that context.WithValue can be used to access (get) properties of the server or connection. This PR fixes those comments to refer to Context.Value instead.

Change-Id: I4ed383ef97ba1951f90c555243007469cfc18d4d
GitHub-Last-Rev: 05bc3acf82
GitHub-Pull-Request: golang/go#33833
Reviewed-on: https://go-review.googlesource.com/c/go/+/191838
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-08-26 16:03:18 +00:00
Filippo Valsorda 145e193131 net/http: update bundled golang.org/x/net/http2 to import security fix
Update golang.org/x/net to v0.0.0-20190813141303-74dc4d7220e7 to import
the following security fix.

    commit 74dc4d7220e7acc4e100824340f3e66577424772
    Author: Filippo Valsorda <filippo@golang.org>
    Date:   Sun Aug 11 02:12:18 2019 -0400

    http2: limit number of control frames in server send queue

    An attacker could cause servers to queue an unlimited number of PING
    ACKs or RST_STREAM frames by soliciting them and not reading them, until
    the program runs out of memory.

    Limit control frames in the queue to a few thousands (matching the limit
    imposed by other vendors) by counting as they enter and exit the scheduler,
    so the protection will work with any WriteScheduler.

    Once the limit is exceeded, close the connection, as we have no way to
    communicate with the peer.

    Change-Id: I842968fc6ed3eac654b497ade8cea86f7267886b
    Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/525552
    Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>

This change was generated with cmd/go and cmd/bundle:

$ go get -u golang.org/x/net
$ go mod tidy
$ go mod vendor
$ go generate net/http

Fixes CVE-2019-9512 and CVE-2019-9514
Fixes #33606

Change-Id: I464baf96175006aa101d65d3b0f6494f28a626ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/190137
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-23 17:03:17 +00:00
Jeff Hodges f3e3b71a50 net/http: change TimeoutHandler's docs to match its new interfaces
As of Go 1.13rc1, TimeoutHandler supports the Flusher and Pusher interfaces and
this change corrects its documentation to say that.

Fixes #33769
Updates #29193

Change-Id: Ia0523f7f2e3dc1f8f0b68950b85a7bf81c4abe60
GitHub-Last-Rev: 5310d2c960
GitHub-Pull-Request: golang/go#33770
Reviewed-on: https://go-review.googlesource.com/c/go/+/191237
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-22 17:04:05 +00:00
Filippo Valsorda 61bb56ad63 net/url: make Hostname and Port predictable for invalid Host values
When Host is not valid per RFC 3986, the behavior of Hostname and Port
was wildly unpredictable, to the point that Host could have a suffix
that didn't appear in neither Hostname nor Port.

This is a security issue when applications are applying checks to Host
and expecting them to be meaningful for the contents of Hostname.

To reduce disruption, this change only aims to guarantee the following
two security-relevant invariants.

* Host is either Hostname or [Hostname] with Port empty, or
  Hostname:Port or [Hostname]:Port.

* Port is only decimals.

The second invariant is the one that's most likely to cause disruption,
but I believe it's important, as it's conceivable an application might
do a suffix check on Host and expect it to be meaningful for the
contents of Hostname (if the suffix is not a valid port).

There are three ways to ensure it.

1) Reject invalid ports in Parse. Note that non-numeric ports are
   already rejected if and only if the host starts with "[".

2) Consider non-numeric ports as part of Hostname, not Port.

3) Allow non-numeric ports, and hope they only flow down to net/http,
   which will reject them (#14353).

This change adopts both 1 and 2. We could do only the latter, but then
these invalid hosts would flow past port checks, like in
http_test.TestTransportRejectsAlphaPort. Non-numeric ports weren't fully
supported anyway, because they were rejected after IPv6 literals, so
this restores consistency. We could do only the former, but at this
point 2) is free and might help with manually constructed Host values
(or if we get something wrong in Parse).

Note that net.SplitHostPort and net.Dial explicitly accept service names
in place of port numbers, but this is an URL package, and RFC 3986,
Section 3.2.3, clearly specifies ports as a number in decimal.

net/http uses a mix of net.SplitHostPort and url.Parse that would
deserve looking into, but in general it seems that it will still accept
service names in Addr fields as they are passed to net.Listen, while
rejecting them in URLs, which feels correct.

This leaves a number of invalid URLs to reject, which however are not
security relevant once the two invariants above hold, so can be done in
Go 1.14: IPv6 literals without brackets (#31024), invalid IPv6 literals,
hostnames with invalid characters, and more.

Tested with 200M executions of go-fuzz and the following Fuzz function.

	u, err := url.Parse(string(data))
	if err != nil {
		return 0
	}
	h := u.Hostname()
	p := u.Port()

	switch u.Host {
	case h + ":" + p:
		return 1
	case "[" + h + "]:" + p:
		return 1
	case h:
		fallthrough
	case "[" + h + "]":
		if p != "" {
			panic("unexpected Port()")
		}
		return 1
	}
	panic("Host is not a variant of [Hostname]:Port")

Fixes CVE-2019-14809
Updates #29098

Change-Id: I7ef40823dab28f29511329fa2d5a7fb10c3ec895
Reviewed-on: https://go-review.googlesource.com/c/go/+/189258
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-08-12 23:12:29 +00:00
Filippo Valsorda 45504066d7 src/go.mod: sync golang.org/x/net with h2_bundle.go
The bundle included changes from a commit after the one referred to by
the go.mod, probably due to cmd/bundle using the GOPATH source.

Identified with the new go/packages based cmd/bundle from CL 189818.

$ go get golang.org/x/net@461777fb6f
$ go mod tidy
$ go mod vendor
$ go generate net/http # with CL 189818

Also, updated the socks_bundle.go generate command to drop obsolete
options and match h2_bundle.go. It caused no output changes.

Updates #32031

Change-Id: I0322d4e842dbfdad749455111072ca4872a62ad4
Reviewed-on: https://go-review.googlesource.com/c/go/+/189897
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-08-12 20:14:38 +00:00
Shivani Singhal 09c9d3a58a net/http: fix typo in Response.Body's doc
Fixes #33453.

Change-Id: If0aa48f953633690884ddcff033b3be6d1d66443
Reviewed-on: https://go-review.googlesource.com/c/go/+/188541
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-05 05:44:06 +00:00
Damien Neil 0e54d28ff7 all: remove os.ErrTimeout
It is unclear whether the current definition of os.IsTimeout is
desirable or not. Drop ErrTimeout for now so we can consider adding it
(or some other error) in a future release with a corrected definition.

Fixes #33411

Change-Id: I8b880da7d22afc343a08339eb5f0efd1075ecafe
Reviewed-on: https://go-review.googlesource.com/c/go/+/188758
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-02 17:57:18 +00:00
Darren McCleary e7c51cad71 net/http: fix a typo in spelling of MultiPartForm
Change-Id: I33a5313ef10e8c88d9c12507573b385fa0843afe
GitHub-Last-Rev: 844d435158
GitHub-Pull-Request: golang/go#33412
Reviewed-on: https://go-review.googlesource.com/c/go/+/188498
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-08-01 20:35:26 +00:00
Damien Neil ea8b0acdac all: remove os.ErrTemporary
As discussed in
https://github.com/golang/go/issues/32463#issuecomment-506833421
the classification of deadline-based timeouts as "temporary" errors is a
historical accident. I/O timeouts used to be duration-based, so they
really were temporary--retrying a timed-out operation could succeed. Now
that they're deadline-based, timeouts aren't temporary unless you reset
the deadline.

Drop ErrTemporary from Go 1.13, since its definition is wrong. We'll
consider putting it back in Go 1.14 with a clear definition and
deprecate net.OpError.Temporary.

Fixes #32463

Change-Id: I70cda664590d8872541e17409a5780da76920891
Reviewed-on: https://go-review.googlesource.com/c/go/+/188398
Reviewed-by: Jonathan Amsterdam <jba@google.com>
2019-08-01 15:39:45 +00:00
Andrew Bonventre 8dddf7556e net/http: return nil from Header.Clone if the receiver is nil
Fixes #33141

Change-Id: I84a8b3496fc9396fd1c09ba9505697c34bdf7105
Reviewed-on: https://go-review.googlesource.com/c/go/+/188022
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-07-31 16:15:56 +00:00
David Ndungu 1ad64fafc8 net/http: improve Request.Form and Request.PostForm documentation
Request.PostForm gets populated with form data for PATCH, POST, or PUT
http verbs.

Change-Id: I33065aa78a8470c4e9490aac830aa6f5963c61cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/187821
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-07-30 20:49:54 +00:00
Joe Tsai f93234ad62 net/http/httputil: fix regression in ReverseProxy.ServeHTTP
In Go1.12 and below, the logic in ReverseProxy.ServeHTTP would always
allocate request.Header even if it were not present in the incoming request.
CL 174324 added http.Request.Clone and re-factors ReverseProxy.ServeHTTP
to use the new Clone method. However, the new Clone logic is not equivalent
to the former logic. We preserve former semantics by explicitly allocating
the Header map if nil.

Fixes #33142

Change-Id: I356f94a915dd9779584ce3fe31e56e5474b9ad37
Reviewed-on: https://go-review.googlesource.com/c/go/+/186437
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-07-17 21:52:45 +00:00
Russ Cox fbaf881cc6 net/http: fix Transport.MaxConnsPerHost limits & idle pool races
There were at least three races in the implementation of the pool of
idle HTTP connections before this CL.

The first race is that HTTP/2 connections can be shared for many
requests, but each requesting goroutine would take the connection out
of the pool and then immediately return it before using it; this
created unnecessary, tiny little race windows during which another
goroutine might dial a second connection instead of reusing the first.
This CL changes the idle pool to just leave the HTTP/2 connection in
the pool permanently (until there is reason to close it), instead of
doing the take-it-out-put-it-back dance race.

The second race is that “is there an idle connection?” and
“register to wait for an idle connection” were implemented as two
separate steps, in different critical sections. So a client could end
up registered to wait for an idle connection and be waiting or perhaps
dialing, not having noticed the idle connection sitting in the pool
that arrived between the two steps.

The third race is that t.getIdleConnCh assumes that the inability to
send on the channel means the client doesn't need the result, when it
could mean that the client has not yet entered the select.
That is, the main dial does:

	idleConnCh := t.getIdleConnCh(cm)
	select {
	case v := <-dialc:
		...
	case pc := <-idleConnCh
		...
	...
	}

But then tryPutIdleConn does:

	waitingDialer := t.idleConnCh[key] // what getIdleConnCh(cm) returned
	select {
	case waitingDialer <- pconn:
		// We're done ...
		return nil
	default:
		if waitingDialer != nil {
			// They had populated this, but their dial won
			// first, so we can clean up this map entry.
			delete(t.idleConnCh, key)
		}
	}

If the client has returned from getIdleConnCh but not yet reached the
select, tryPutIdleConn will be unable to do the send, incorrectly
conclude that the client does not care anymore, and put the connection
in the idle pool instead, again leaving the client dialing unnecessarily
while a connection sits in the idle pool.

(It's also odd that the success case does not clean up the map entry,
and also that the map has room for only a single waiting goroutine for
a given host.)

None of these races mattered too much before Go 1.11: at most they
meant that connections were not reused quite as promptly as possible,
or a few more than necessary would be created. But Go 1.11 added
Transport.MaxConnsPerHost, which limited the number of connections
created for a given host. The default is 0 (unlimited), but if a user
did explicitly impose a low limit (2 is common), all these misplaced
conns could easily add up to the entire limit, causing a deadlock.
This was causing intermittent timeouts in TestTransportMaxConnsPerHost.

The addition of the MaxConnsPerHost support added its own races.

For example, here t.incHostConnCount could increment the count
and return a channel ready for receiving, and then the client would
not receive from it nor ever issue the decrement, because the select
need not evaluate these two cases in order:

	select {
	case <-t.incHostConnCount(cmKey):
		// count below conn per host limit; proceed
	case pc := <-t.getIdleConnCh(cm):
		if trace != nil && trace.GotConn != nil {
			trace.GotConn(httptrace.GotConnInfo{Conn: pc.conn, Reused: pc.isReused()})
		}
		return pc, nil
	...
	}

Obviously, unmatched increments are another way to get to a deadlock.
TestTransportMaxConnsPerHost deadlocked approximately 100% of
the time with a small random sleep added between incHostConnCount
and the select:

	ch := t.incHostConnCount(cmKey):
	time.Sleep(time.Duration(rand.Intn(10))*time.Millisecond)
	select {
	case <-ch
		// count below conn per host limit; proceed
	case pc := <-t.getIdleConnCh(cm):
		...
	}

The limit also did not properly apply to HTTP/2, because of the
decrement being attached to the underlying net.Conn.Close
and net/http not having access to the underlying HTTP/2 conn.
The alternate decrements for HTTP/2 may also have introduced
spurious decrements (discussion in #29889). Perhaps those
spurious decrements or other races caused the other intermittent
non-deadlock failures in TestTransportMaxConnsPerHost,
in which the HTTP/2 phase created too many connections (#31982).

This CL replaces the buggy, racy code with new code that is hopefully
neither buggy nor racy.

Fixes #29889.
Fixes #31982.
Fixes #32336.

Change-Id: I0dfac3a6fe8a6cdf5f0853722781fe2ec071ac97
Reviewed-on: https://go-review.googlesource.com/c/go/+/184262
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-07-08 14:31:42 +00:00
Patrik Lundin 04e2e81ecb net/http: stop ExampleServer_Shutdown from hanging on error
Running the example code when not having permissions
to bind to port 80 will cause the program to hang after
printing the error message.

Change-Id: I2433ba2629b362fc8f1731e40cab5eea72ec354f
GitHub-Last-Rev: 0bb3dc08b6
GitHub-Pull-Request: golang/go#32947
Reviewed-on: https://go-review.googlesource.com/c/go/+/185157
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-07-05 00:26:14 +00:00
Raghavendra Nagaraj 1962dc88eb net/http: fix Server.IdleTimeout and Server.ReadHeaderTimeout docs
CL 46434 changed the doc for Server.IdleTimeout to include
falling back to Server.ReadHeaderTimeout if both
Server.IdleTimeout and Server.ReadTimeout are zero.
However, we explicitly set the ReadDeadlines firstly based
off Server.IdleTimeout or Server.ReadTimeout before attempting
to read the next request, thus the current doc is incorrect.

This CL reverts CL 46434 and also updates the doc for
Server.ReadHeaderTimeout to documenting falling back
to Server.ReadTimeout, if the former is zero, otherwise
there is no timeout.

Fixes #32053

Change-Id: I43dd0252d1bcee6c29a8529abd84c84a49b2fba9
GitHub-Last-Rev: e1cdb59977
GitHub-Pull-Request: golang/go#32164
Reviewed-on: https://go-review.googlesource.com/c/go/+/178337
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-06-19 05:46:46 +00:00
Andrew Bonventre 5ce1819cca doc, net/http: add release notes for net/http and fix doc comments
Change-Id: I684e3522e387b2d96d5cfb2878d2f77bf4558443
Reviewed-on: https://go-review.googlesource.com/c/go/+/181545
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-10 20:32:36 +00:00
Brad Fitzpatrick 4c84d87813 net/http: support BaseContext & ConnContext for http2 Server
This is the net/http half of #32476. This supplies the method needed
by the other half in x/net/http2 in the already-submitted CL 181259,
which this CL also bundles in h2_bundle.go.

Thanks to Tom Thorogood (@tmthrgd) for the bug report and test.

Fixes #32476
Updates #30694

Change-Id: I79d2a280e486fbf75d116f6695fd3abb61278765
Reviewed-on: https://go-review.googlesource.com/c/go/+/181260
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-06-07 18:48:19 +00:00
Dmitri Shuralyov 003dbc4cda net/http: roll back "clean the path of the stripped URL by StripPrefix"
Roll back CL 161738. That fix changed StripPrefix behavior in the
general case, not just in the situation where where stripping the
prefix from path resulted in the empty string, causing issue #31622.

That kind of change to StripPrefix behavior is not backwards compatible,
and there can be a smaller, more targeted fix for the original issue.

Fixes #31622
Updates #30165

Change-Id: Ie2fcfe6787a32e44f71d564d8f9c9d580fc6f704
Reviewed-on: https://go-review.googlesource.com/c/go/+/180498
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-06-04 22:36:51 +00:00
Brad Fitzpatrick ba66d89d78 net/http: prevent Transport from spamming stderr on server 408 reply
HTTP 408 responses now exist and are seen in the wild (e.g. from
Google's GFE), so make Go's HTTP client not spam about them when seen.
They're normal (now).

Fixes #32310

Change-Id: I558eb4654960c74cf20db1902ccaae13d03310f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/179457
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-30 22:29:29 +00:00
Johan Brandhorst 220552f662 net/http: enable WASM fetch where supported
The existing check was introduced to allow tests to pass
on WASM without an environment where the fetch RoundTripper
could run. However, the check now prohibits the use of the
Fetch RoundTripper in all WASM tests, even where the
RoundTripper could run. The new change should only disable
the RoundTripper when used in an environment without fetch.

Fixes #32289

Change-Id: I30d2e0dbcb0e64d4b1a46b583f7e984c2a57d5e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/179118
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-30 17:59:57 +00:00
Brad Fitzpatrick 770746af1c net/http: quiet some log spam in tests
One of these tests creates a bunch of connections concurrently, then
discovers it doesn't need them all, which then makes the server log
that the client went away midway through the TLS handshake. Perhaps
the server should recognize that as a case not worthy of logging
about, but this is a safer way to eliminate the stderr spam during go
test for now.

The other test's client gives up on its connection and closes it,
similarly confusing the server.

Change-Id: I49ce442c9a63fc437e58ca79f044aa76e8c317b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/179179
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-28 18:17:57 +00:00
Brad Fitzpatrick 3970667d95 net/http: fix TestTransportServerClosingUnexpectedly flake
Fixes #32119

Change-Id: I8cf2e2e69737e2485568af91ab75149f3cf66781
Reviewed-on: https://go-review.googlesource.com/c/go/+/178918
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-28 00:54:10 +00:00
Richard Musiol c468ad0417 syscall/js: replace TypedArrayOf with CopyBytesToGo/CopyBytesToJS
The typed arrays returned by TypedArrayOf were backed by WebAssembly
memory. They became invalid each time we grow the WebAssembly memory.
This made them very error prone and hard to use correctly.

This change removes TypedArrayOf completely and instead introduces
CopyBytesToGo and CopyBytesToJS for copying bytes between a byte
slice and an Uint8Array. This breaking change is still allowed for
the syscall/js package.

Fixes #31980.
Fixes #31812.

Change-Id: I14c76fdd60b48dd517c1593972a56d04965cb272
Reviewed-on: https://go-review.googlesource.com/c/go/+/177537
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-24 09:38:37 +00:00
Russ Cox 06b0babf31 all: shorten some tests
Shorten some of the longest tests that run during all.bash.
Removes 7r 50u 21s from all.bash.

After this change, all.bash is under 5 minutes again on my laptop.

For #26473.

Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79
Reviewed-on: https://go-review.googlesource.com/c/go/+/177559
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-22 12:54:00 +00:00
Russ Cox 02fe6ba958 all: remove PEM-encoded private keys from tests
Gerrit is complaining about pushes that affect these files
and forcing people to use -o nokeycheck, which defeats
the point of the check. Hide the keys from this kind of scan
by marking them explicitly as testing keys.

This is a little annoying but better than training everyone
who ever edits one of these test files to reflexively override
the Gerrit check.

The only remaining keys explicitly marked as private instead
of testing are in examples, and there's not much to do
about those. Hopefully they are not edited as much.

Change-Id: I4431592b5266cb39fe6a80b40e742d97da803a0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/178178
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-21 20:03:55 +00:00
Brad Fitzpatrick c6f9321b5a net/http/httptest: update docs, remove old inaccurate sentence
The "After it is called, changing rw.Header will not affect
rw.HeaderMap" claim predates the Result method which changed how the
Recorder should be used.

Fixes #32144
Fixes #32136

Change-Id: I95bdfa5ac489ce7b0202824bb5663f4da188e8a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/178058
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-20 04:22:40 +00:00
Dmitri Shuralyov 3f7c560829 net/http/httputil: remove os.Stderr claim in ReverseProxy.ErrorLog docs
The motivation for doing so is to avoid making inaccurate claims.
Logging may not go to os.Stderr if anyone overrides the log package's
default output via https://godoc.org/log#SetOutput. Saying that
the standard logger is used should be sufficient to explain the
behavior, and users can infer that os.Stderr is used by default,
unless it's changed.

This change is the same as what was applied to http.Server.ErrorLog
documentation in CL 53950.

Change-Id: I32873fc548ceee573f8616b4d49b8a8b98881803
Reviewed-on: https://go-review.googlesource.com/c/go/+/176817
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-17 17:33:16 +00:00
Brad Fitzpatrick cb5c82bc3d net/http: update vendored, bundled x/net/http2
For:

    http2: track reused connections
    https://golang.org/cl/176720 (updates golang/go#31982)

Some x/sys/unix updates come along for the ride too.

I filed #32031 for making the bundling process less difficult and
error-prone in the future.

Change-Id: Ic822080991ffa2d50352c5f613e45648a327cf16
Reviewed-on: https://go-review.googlesource.com/c/go/+/177037
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-16 19:56:27 +00:00
Jonathon Lacher a5cea062b3 net/http/httputil: remove all fields in Connection header
In the reverseproxy, replace use (Header).Get, which returns only one value
of a multiple value header, with using the Header map directly. Also fixes
corresponding tests which hid the bug, and adds more tests.

Fixes #30303

Change-Id: Ic9094b5983043460697748759f6dfd95fc111db7
GitHub-Last-Rev: b41038143f
GitHub-Pull-Request: golang/go#30687
Reviewed-on: https://go-review.googlesource.com/c/go/+/166298
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-14 15:37:14 +00:00
Vivek Sekhar e64241216d net/http: add support for SameSite=None
Section 4.2 of the Internet-Draft for SameSite includes the possible
SameSite value of "None".

https://tools.ietf.org/html/draft-ietf-httpbis-cookie-same-site-00

Change-Id: I44f246024429ec175db13ff6b36bee465f3d233d
GitHub-Last-Rev: 170d24aaca
GitHub-Pull-Request: golang/go#31842
Reviewed-on: https://go-review.googlesource.com/c/go/+/175337
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-06 18:01:25 +00:00
Brad Fitzpatrick 79f79c34de net/http: update bundled x/net/http2
Updates x/net/http2 to git rev 1da14a5a36f220ea3f03470682b737b1dfd5de22 for:

    http2: make empty method mean GET
    https://golang.org/cl/169557 (Fixes golang/go#31061)

    http2: don't hang a stream if trailers values are not provided
    https://golang.org/cl/161958

Change-Id: I628af8c6d07d19e8f19ee37637243f6c242ef3a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/174677
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-06 15:55:25 +00:00
Damien Neil 170b8b4b12 all: add Unwrap and Is methods to various error types
Add Unwrap methods to types which wrap an underlying error:

  "encodinc/csv".ParseError
  "encoding/json".MarshalerError
  "net/http".transportReadFromServerError
  "net".OpError
  "net".DNSConfigError
  "net/url".Error
  "os/exec".Error
  "signal/internal/pty".PtyError
  "text/template".ExecError

Add os.ErrTemporary. A case could be made for putting this error
value in package net, since no exported error types in package os
include a Temporary method. However, syscall errors returned from
the os package do include this method.

Add Is methods to error types with a Timeout or Temporary method,
making errors.Is(err, os.Err{Timeout,Temporary}) equivalent to
testing the corresponding method:

  "context".DeadlineExceeded
  "internal/poll".TimeoutError
  "net".adrinfoErrno
  "net".OpError
  "net".DNSError
  "net/http".httpError
  "net/http".tlsHandshakeTimeoutError
  "net/pipe".timeoutError
  "net/url".Error

Updates #30322
Updates #29934

Change-Id: I409fb20c072ea39116ebfb8c7534d493483870dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/170037
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2019-05-04 16:14:12 +00:00
Michael Fraenkel 0a4d352992 net/http: fix TestTransportMaxConnsPerHost flakes
The testcase created a race between the close of the current connection
and the client grabbing a connection for the next request. The client
may receive the current connection which may be closed during its use.
We can have the trasnport close all idle connections thereby forcing the
client to receive a new connection.

Closing idle connections did not handle cleaning up host connection
counts for http/2. We will now decrement the host connection count for
http/2 connections.

Fixes #31784

Change-Id: Iefc0d0d7ed9fa3acd8b4f42004f1579fc1de63fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174950
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-03 15:58:50 +00:00
Thanabodee Charoenpiriyakij 2c67cdf7cf net/http: strip escaped password from error
Using password that returns from User.Password() won't work in this case
because password in Userinfo already unescaped. The solution is uses
User.String() to escape password back again and then stringify it to error.

Fixes #31808

Change-Id: I723aafd5a57a5b69f2dd7d3a21b82ebbd4174451
Reviewed-on: https://go-review.googlesource.com/c/go/+/175018
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-03 15:20:41 +00:00
Brad Fitzpatrick f5c43b9194 net/http: add func NewRequestWithContext, Request.Clone
Fixes #23544

Change-Id: Iaa31d76c4cda8ce22412d73c9025fc57e4fb1967
Reviewed-on: https://go-review.googlesource.com/c/go/+/174324
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-05-03 15:20:15 +00:00
Brad Fitzpatrick 5e404b3620 net/http: add Transport.Clone
Fixes #26013

Change-Id: I2c82bd90ea7ce6f7a8e5b6c460d3982dca681a93
Reviewed-on: https://go-review.googlesource.com/c/go/+/174597
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-05-03 15:17:54 +00:00
Brad Fitzpatrick bf35b7c8b1 net/http: skip flaky TestTransportMaxConnsPerHost for now
Updates #31784

Change-Id: Iee056c850c03939606b227a12715c76b0339d268
Reviewed-on: https://go-review.googlesource.com/c/go/+/175097
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-02 21:46:14 +00:00
Michael Fraenkel 43b9fcf6fe net/http: make Transport.MaxConnsPerHost work for HTTP/2
Treat HTTP/2 connections as an ongoing persistent connection. When we
are told there is no cached connections, cleanup the associated
connection and host connection count.

Fixes #27753

Change-Id: I6b7bd915fc7819617cb5d3b35e46e225c75eda29
Reviewed-on: https://go-review.googlesource.com/c/go/+/140357
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-30 23:25:55 +00:00
Nir Soffer c706d42289 net/http: add Transport.ReadBufferSize and WriteBufferSize
Previously transport was using the hardcoded bufio.defaultBufSize
(4096), limiting throughput and increasing cpu usage when uploading or
downloading large files.

Add options to allow users to configure the buffer sizes as needed.

I tested the maximum benefit of this change by uploading data from
/dev/zero to a server discarding the bytes. Here is an example upload
using the default buffer size:

$ time ./upload 10 https://localhost:8000/
Uploaded 10.00g in 25.13 seconds (407.49m/s)

real	0m25.135s
user	0m5.167s
sys	0m11.643s

With this change, using 128k buffer size:

$ time ./upload 10 https://localhost:8000/
Uploaded 10.00g in 7.93 seconds (1291.51m/s)

real	0m7.935s
user	0m4.517s
sys	0m2.603s

In real world usage the difference will be smaller, depending on the
local and remote storage and the network.

See https://github.com/nirs/http-bench for more info.

Fixes #22618

Change-Id: Iac99ed839c7b95d6dc66602ba8fe1fc5b500c47c
Reviewed-on: https://go-review.googlesource.com/c/go/+/76410
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-30 22:56:01 +00:00
Emmanuel T Odeke 88548d0211 net/http: make Server return 501 for unsupported transfer-encodings
Ensures that our HTTP/1.X Server properly responds
with a 501 Unimplemented as mandated by the spec at
RFC 7230 Section 3.3.1, which says:
    A server that receives a request message with a
    transfer coding it does not understand SHOULD
    respond with 501 (Unimplemented).

Fixes #30710

Change-Id: I096904e6df053cd1e4b551774cc27523ff3d09f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/167017
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-30 19:23:58 +00:00
BigMikes cbf90b0028 net: correct docs of KeepAlive field in Dialer type
KeepAlive field used to report the wording "keep-alive period"
which may be misleading. This field does not represent the whole
TCP keepalive time, that is the inactivity period upon which one
endpoint starts probing the other end. But it acctually specifies
the keepalive interval, that is the time between two keepalive
probes.

Fixes #29089

Change-Id: If99b38ba108830d0e5fe527171a2f5c96a3bcde7
Reviewed-on: https://go-review.googlesource.com/c/go/+/155960
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-29 16:49:52 +00:00
Dmitri Shuralyov db1514cbf2 net/http: remove "number:" from Response.Status string
The behavior of Value.String method on non-string JavaScript types has
changed after CL 169757.

Update the implementation of Transport.RoundTrip method to construct the
Response.Status string without relying on result.Get("status").String(),
since that now returns strings like "<number: 200>" instead of "200".

Fixes #31736

Change-Id: I27b3e6cc95aa65fd1918b1400e88478a154aad12
Reviewed-on: https://go-review.googlesource.com/c/go/+/174218
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Richard Musiol <neelance@gmail.com>
2019-04-29 15:23:10 +00:00
Brad Fitzpatrick 13d0af4e70 net/http: export Header.Clone, reduce its allocations, use it everywhere
Fixes #29915

Change-Id: I6e6edf4f9a0e062211f74d120ae1a242bce1b274
Reviewed-on: https://go-review.googlesource.com/c/go/+/173658
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ingo Oeser <nightlyone@googlemail.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2019-04-24 18:43:11 +00:00
Benoit Sigoure 415da714fa net/http: document that Basic Auth may require URL encoding
Explicitly warn callers that no URL encoding is performed and
that they might need to do it.

Fixes #31577

Change-Id: I52dc3fd2798ba8c3652d4a967b1c5c48eb69f43b
Reviewed-on: https://go-review.googlesource.com/c/go/+/173319
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-22 22:54:41 +00:00
Josh Bleecher Snyder 5781df421e all: s/cancelation/cancellation/
Though there is variation in the spelling of canceled,
cancellation is always spelled with a double l.

Reference: https://www.grammarly.com/blog/canceled-vs-cancelled/

Change-Id: I240f1a297776c8e27e74f3eca566d2bc4c856f2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/170060
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-16 20:27:15 +00:00
Kunpei Sakai 2a931bad4e net/http: rename DialerAndTLSConfigSupportsHTTP2 to ForceAttemptHTTP2
Transport.DialerAndTLSConfigSupportsHTTP2 was added just earlier
in CL 130256 but we thought of a better name moments after submitting.
ForceAttemptHTTP2 is shorter, more direct, and doesn't constrain what
we can use it with in the future.

Updates #14391
Updates #27011

Change-Id: Ie5fc71bafcbcaa1941b5d49f748b6d710503d477
Reviewed-on: https://go-review.googlesource.com/c/go/+/172299
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-16 17:15:01 +00:00
Kunpei Sakai 94e720059f net/http: introduce DialerAndTLSConfigSupportsHTTP2 in Transport
Even when a custom TLS config or custom dialer is specified,
enables HTTP/2 if DialerAndTLSConfigSupportsHTTP2 is true.
By this change, avoid automatically enabling HTTP/2 if DialContext is set.

This change also ensures that DefaultTransport still automatically
enable HTTP/2 as discussed in #14391.

Updates #14391
Fixes #27011

Change-Id: Icc46416810bee61dbd65ebc96468335030b80573
Reviewed-on: https://go-review.googlesource.com/c/go/+/130256
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Kunpei Sakai <namusyaka@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-16 16:12:22 +00:00
Daniel Martí 9b968df177 all: clean up code with token.IsExported
A handful of packages were reimplementing IsExported, so use
token.IsExported instead. This caused the deps test to fail for net/rpc.
However, net/rpc deals with Go types, and go/token is light and fairly
low-level in terms of Go tooling packages, so that's okay.

While at it, replace all uses of ast.IsExported with token.IsExported.
This is more consistent, and also means that the import graphs are
leaner. A couple of files no longer need to import go/ast, for example.

We can't get rid of cmd/compile/internal/types.IsExported, as the
compiler can only depend on go/token as of Go 1.4. However, gc used
different implementations in a couple of places, so consolidate the use
of types.IsExported there.

Finally, we can't get rid of the copied IsExported implementation in
encoding/gob, as go/token depends on it as part of a test. That test
can't be an external test either, so there's no easy way to break the
import cycle.

Overall, this removes about forty lines of unnecessary code.

Change-Id: I86a475b7614261e6a7b0b153d5ca02b9f64a7b2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/172037
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-16 14:43:48 +00:00
bronze1man 09b2b6e9dd net/http: remove unnecessary string replace operation in Cookie.String
Fixes #29135

Change-Id: I4c10b0395047775e8488b8b0f00f74a7fa01b86c
GitHub-Last-Rev: 1209770405
GitHub-Pull-Request: golang/go#29728
Reviewed-on: https://go-review.googlesource.com/c/go/+/157777
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-15 19:51:36 +00:00
sergey 6b7114b9e5 net/http: speed up parsing of Cookie headers
Parse the headers without splitting them upfront to reduce
memory allocations.

For non-pathological Cookie headers we can make
a good estimate of the number of cookies in there and preallocate
the slice of cookies

name              old time/op    new time/op    delta
CookieString-4      1.73µs ± 2%    1.70µs ± 5%     ~     (p=0.841 n=5+5)
ReadSetCookies-4    6.09µs ± 3%    5.93µs ± 3%     ~     (p=0.095 n=5+5)
ReadCookies-4       7.63µs ± 1%    6.41µs ± 4%  -15.99%  (p=0.008 n=5+5)

name              old alloc/op   new alloc/op   delta
CookieString-4        360B ± 0%      360B ± 0%     ~     (all equal)
ReadSetCookies-4      976B ± 0%      976B ± 0%     ~     (all equal)
ReadCookies-4       2.17kB ± 0%    1.84kB ± 0%  -15.13%  (p=0.008 n=5+5)

name              old allocs/op  new allocs/op  delta
CookieString-4        5.00 ± 0%      5.00 ± 0%     ~     (all equal)
ReadSetCookies-4      15.0 ± 0%      15.0 ± 0%     ~     (all equal)
ReadCookies-4         16.0 ± 0%      11.0 ± 0%  -31.25%  (p=0.008 n=5+5)

Change-Id: Ica1ca0d40c0d8d275134d1dfafb73f1082115826
Reviewed-on: https://go-review.googlesource.com/c/go/+/163617
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-15 19:30:44 +00:00
Brad Fitzpatrick 2c802e9980 net/http: add Server BaseContext & ConnContext fields to control early context
Fixes golang/go#30694

Change-Id: I12a0a870e4aee6576e879d88a4868666ef448298
Reviewed-on: https://go-review.googlesource.com/c/go/+/167681
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JP Sugarbroad <jpsugar@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-15 19:15:11 +00:00
Leo Antunes 1abf3aa55b net: add KeepAlive field to ListenConfig
This commit adds a KeepAlive field to ListenConfig and uses it
analogously to Dialer.KeepAlive to set TCP KeepAlives per default on
Accept()

Fixes #23378

Change-Id: I57eaf9508c979e7f0e2b8c5dd8e8901f6eb27fd6
GitHub-Last-Rev: e9e035d53e
GitHub-Pull-Request: golang/go#31242
Reviewed-on: https://go-review.googlesource.com/c/go/+/170678
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-04 14:51:33 +00:00
Jordan Liggitt 2cc347382f net/http/httputil: make ReverseProxy flush headers on FlushInterval
A regression was introduced in CL 137335 (5440bfc) that caused FlushInterval
to not be honored until the first Write() call was encountered. This change
starts the flush timer as part of setting up the maxLatencyWriter.

Fixes #31125
Fixes #31126

Change-Id: I75325bd926652922219bd1457b2b00ac6d0d41b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/170066
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-29 15:56:57 +00:00
Brad Fitzpatrick ea9eddb3e7 net/http: fix typo in Response.Body field docs
Fixes #31096

Change-Id: I5b36bfc6d18eb8c1bbf15abcd92b0e6559cda3c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/169683
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-03-28 02:38:28 +00:00
sergey cf99535e21 net/http: reduce allocs on write cookie
Pregrow buffer to reduce allocs on building cookie string.
We calc cookie name value and domain size (most distributed by value) and
add 110 extra characters allows to write most cookie attributes
without additional allocations in most cases.

name              old time/op    new time/op    delta
CookieString-4      1.65µs ± 3%    1.27µs ± 2%  -23.01%  (p=0.008 n=5+5)
ReadSetCookies-4    5.78µs ± 4%    5.67µs ± 3%     ~     (p=0.310 n=5+5)
ReadCookies-4       7.12µs ± 4%    7.04µs ± 3%     ~     (p=0.690 n=5+5)

name              old alloc/op   new alloc/op   delta
CookieString-4        360B ± 0%      144B ± 0%  -60.00%  (p=0.008 n=5+5)
ReadSetCookies-4      976B ± 0%      976B ± 0%     ~     (all equal)
ReadCookies-4       2.17kB ± 0%    2.17kB ± 0%     ~     (all equal)

name              old allocs/op  new allocs/op  delta
CookieString-4        5.00 ± 0%      1.00 ± 0%  -80.00%  (p=0.008 n=5+5)
ReadSetCookies-4      15.0 ± 0%      15.0 ± 0%     ~     (all equal)
ReadCookies-4         16.0 ± 0%      16.0 ± 0%     ~     (all equal)

Change-Id: I50defac954a135b785b3551342b00878429b3d7f
Reviewed-on: https://go-review.googlesource.com/c/go/+/163017
Reviewed-by: Volker Dobler <dr.volker.dobler@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-25 15:21:06 +00:00
LE Manh Cuong 349e7df2c3 net/http: fix wrong mime rar signature
MIME sniffing standard defines the RAR signature as 52 61 72 20 1A 07 00.

But this signature is wrong, the RARlab spec defines the 4th byte must
be 0x21 or "!", not 0x20 or " ". Checking a rar file also indicates that:

	$ file abc.rar
	abc.rar: RAR archive data, v1d, os: Win32
	$ head -c 7 abc.rar | od -v -t x1
	0000000 52 61 72 21 1a 07 00
	0000007

There is also an issue to fix this problem in MIME standard.

See:
 - https://www.rarlab.com/technote.htm#rarsign
 - https://github.com/whatwg/mimesniff/issues/63

Fixes #30926

Change-Id: Id2e2de7ecbf7f44d37ebaf280efd05e4972c5078
Reviewed-on: https://go-review.googlesource.com/c/go/+/167781
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-19 15:54:05 +00:00
bakape 27e444d5e6 net/http: Detect MIME type of v5 RAR archives
Change-Id: Id9dedc861523e2dafe0e67f70862973124fa07b3
GitHub-Last-Rev: b662561f19
GitHub-Pull-Request: golang/go#30909
Reviewed-on: https://go-review.googlesource.com/c/go/+/168039
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-18 20:32:57 +00:00
Leon Klingele 62bfa69e6e net/http: add missing error checks in tests
Change-Id: I73441ba2eb349f0e0f25068e6b24c74dd33f1456
GitHub-Last-Rev: b9e6705962
GitHub-Pull-Request: golang/go#30017
Reviewed-on: https://go-review.googlesource.com/c/go/+/160441
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-11 21:40:21 +00:00
Bryan C. Mills c5cf662407 all: move internal/x to vendor/golang.org/x and revendor using 'go mod vendor'
This also updates the vendored-in versions of several packages: 'go
mod vendor' selects a consistent version of each module, but we had
previously vendored an ad-hoc selection of packages.

Notably, x/crypto/hkdf was previously vendored in at a much newer
commit than the rest of x/crypto. Bringing the rest of x/crypto up to
that commit introduced an import of golang.org/x/sys/cpu, which broke
the js/wasm build, requiring an upgrade of x/sys to pick up CL 165749.

Updates #30228
Updates #30241
Updates #25822

Change-Id: I5b3dbc232b7e6a048a158cbd8d36137af1efb711
Reviewed-on: https://go-review.googlesource.com/c/go/+/164623
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2019-03-11 20:28:54 +00:00
Chris Marchesi 6ebfbbaadf net/http: let Transport request body writes use sendfile
net.TCPConn has the ability to send data out using system calls such as
sendfile when the source data comes from an *os.File. However, the way
that I/O has been laid out in the transport means that the File is
actually wrapped behind two outer io.Readers, and as such the TCP stack
cannot properly type-assert the reader, ensuring that it falls back to
genericReadFrom.

This commit does the following:

* Removes transferBodyReader and moves its functionality to a new
doBodyCopy helper. This is not an io.Reader implementation, but no
functionality is lost this way, and it allows us to unwrap one layer
from the body.

* The second layer of the body is unwrapped if the original reader
was wrapped with ioutil.NopCloser, which is what NewRequest wraps the
body in if it's not a ReadCloser on its own. The unwrap operation
passes through the existing body if there's no nopCloser.

Note that this depends on change https://golang.org/cl/163737 to
properly function, as the lack of ReaderFrom implementation otherwise
means that this functionality is essentially walled off.

Benchmarks between this commit and https://golang.org/cl/163862,
incorporating https://golang.org/cl/163737:

linux/amd64:
name                        old time/op    new time/op    delta
FileAndServer_1KB/NoTLS-4     53.2µs ± 0%    53.3µs ± 0%      ~     (p=0.075 n=10+9)
FileAndServer_1KB/TLS-4       61.2µs ± 0%    60.7µs ± 0%    -0.77%  (p=0.000 n=10+9)
FileAndServer_16MB/NoTLS-4    25.3ms ± 5%     3.8ms ± 6%   -84.95%  (p=0.000 n=10+10)
FileAndServer_16MB/TLS-4      33.2ms ± 2%    13.4ms ± 2%   -59.57%  (p=0.000 n=10+10)
FileAndServer_64MB/NoTLS-4     106ms ± 4%      16ms ± 2%   -84.45%  (p=0.000 n=10+10)
FileAndServer_64MB/TLS-4       129ms ± 1%      54ms ± 3%   -58.32%  (p=0.000 n=8+10)

name                        old speed      new speed      delta
FileAndServer_1KB/NoTLS-4   19.2MB/s ± 0%  19.2MB/s ± 0%      ~     (p=0.095 n=10+9)
FileAndServer_1KB/TLS-4     16.7MB/s ± 0%  16.9MB/s ± 0%    +0.78%  (p=0.000 n=10+9)
FileAndServer_16MB/NoTLS-4   664MB/s ± 5%  4415MB/s ± 6%  +565.27%  (p=0.000 n=10+10)
FileAndServer_16MB/TLS-4     505MB/s ± 2%  1250MB/s ± 2%  +147.32%  (p=0.000 n=10+10)
FileAndServer_64MB/NoTLS-4   636MB/s ± 4%  4090MB/s ± 2%  +542.81%  (p=0.000 n=10+10)
FileAndServer_64MB/TLS-4     522MB/s ± 1%  1251MB/s ± 3%  +139.95%  (p=0.000 n=8+10)

darwin/amd64:
name                        old time/op    new time/op     delta
FileAndServer_1KB/NoTLS-8     93.0µs ± 5%     96.6µs ±11%      ~     (p=0.190 n=10+10)
FileAndServer_1KB/TLS-8        105µs ± 7%      100µs ± 5%    -5.14%  (p=0.002 n=10+9)
FileAndServer_16MB/NoTLS-8    87.5ms ±19%     10.0ms ± 6%   -88.57%  (p=0.000 n=10+10)
FileAndServer_16MB/TLS-8      52.7ms ±11%     17.4ms ± 5%   -66.92%  (p=0.000 n=10+10)
FileAndServer_64MB/NoTLS-8     363ms ±54%       39ms ± 7%   -89.24%  (p=0.000 n=10+10)
FileAndServer_64MB/TLS-8       209ms ±13%       73ms ± 5%   -65.37%  (p=0.000 n=9+10)

name                        old speed      new speed       delta
FileAndServer_1KB/NoTLS-8   11.0MB/s ± 5%   10.6MB/s ±10%      ~     (p=0.184 n=10+10)
FileAndServer_1KB/TLS-8     9.75MB/s ± 7%  10.27MB/s ± 5%    +5.26%  (p=0.003 n=10+9)
FileAndServer_16MB/NoTLS-8   194MB/s ±16%   1680MB/s ± 6%  +767.83%  (p=0.000 n=10+10)
FileAndServer_16MB/TLS-8     319MB/s ±10%    963MB/s ± 4%  +201.36%  (p=0.000 n=10+10)
FileAndServer_64MB/NoTLS-8   180MB/s ±31%   1719MB/s ± 7%  +853.61%  (p=0.000 n=9+10)
FileAndServer_64MB/TLS-8     321MB/s ±12%    926MB/s ± 5%  +188.24%  (p=0.000 n=9+10)

Updates #30377.

Change-Id: I631a73cea75371dfbb418c9cd487c4aa35e73fcd
GitHub-Last-Rev: 4a77dd1b80
GitHub-Pull-Request: golang/go#30378
Reviewed-on: https://go-review.googlesource.com/c/go/+/163599
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-03-07 20:11:21 +00:00
Chris Marchesi a2ace8ec18 net/http: unfurl persistConnWriter's underlying writer
Make persistConnWriter implement io.ReaderFrom, via an io.Copy on the
underlying net.Conn. This in turn enables it to use OS level
optimizations such as sendfile.

This has been observed giving performance gains even in the absence
of ReaderFrom, more than likely due to the difference in io's default
buffer (32 KB) versus bufio's (4 KB).

Speedups on linux/amd64:
benchmark                               old MB/s     new MB/s     speedup
BenchmarkFileAndServer_16MB/NoTLS-4     662.96       2703.74      4.08x
BenchmarkFileAndServer_16MB/TLS-4       552.76       1420.72      2.57x

Speedups on darwin/amd64:
benchmark                               old MB/s     new MB/s     speedup
BenchmarkFileAndServer_16MB/NoTLS-8     357.58       1972.86      5.52x
BenchmarkFileAndServer_16MB/TLS-8       346.20       1067.41      3.08x

Updates #30377.

Change-Id: Ic88d4ac254f665223536fcba4d551fc32ae105b6
GitHub-Last-Rev: a6f67cda2e
GitHub-Pull-Request: golang/go#30390
Reviewed-on: https://go-review.googlesource.com/c/go/+/163737
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-03-07 07:48:16 +00:00
Kshitij Saraogi 583fddf3bc net/http: remove discrepancies between the MIME Sniffing Spec and its implementation
The change fixes the following deviations between the existing implementation and the Spec:
1. Using pattern instead of "mask" for assertion and iteration in the Pattern Matching Algorithm.
2. Rename "image/vnd.microsoft.icon" to "image/x-icon" and add another signature for the same.
3. Using named strings instead of hexadecimal representation in "application/zip" and "application/x-rar-compressed".
4. Reordering "sniffSignatures" in accordance with the Spec section "Identifying a resource with an unknown MIME type".

In addition to the above fixes, unit tests for Image MIME type group are added.

Fixes #30570

Change-Id: I97d2ae22b426c3c57bf8efd2ed9396c0be983688
Reviewed-on: https://go-review.googlesource.com/c/go/+/165277
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-06 07:31:18 +00:00
Chris Marchesi 312bfc5d55 net/http: add request file upload benchmarks
This adds benchmarks to test file uploads using PUT requests.

It's designed to complement changes https://golang.org/cl/163599 and
https://golang.org/cl/163737, allowing an easy comparison of
performance before and after these changes are applied.

Updates #30377.

Co-authored-by: Emmanuel Odeke <emm.odeke@gmail.com>

Change-Id: Ib8e692c61e1f7957d88c7101669d4f7fb8110c65
GitHub-Last-Rev: 242622b4fc
GitHub-Pull-Request: golang/go#30424
Reviewed-on: https://go-review.googlesource.com/c/go/+/163862
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-03-06 04:29:41 +00:00
sergey 4b142806fa net/http: add corner cases for readCookiesTests
The following corner cases for readCookiesTests are tested now:
 - An extra cookie delimiter ";" at the end of a Cookie header
 - An empty Cookie header

Change-Id: Id8028b448e4182613fb261bf0903efc98cbf4997
Reviewed-on: https://go-review.googlesource.com/c/go/+/164702
Reviewed-by: Volker Dobler <dr.volker.dobler@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-05 22:52:48 +00:00
Daniel Martí 340129e4c8 all: join a few chained ifs
I had been finding these over a year or so, but none were big enough
changes to warrant CLs. They're a handful now, so clean them all up in a
single commit.

The smaller bodies get a bit simpler, but most importantly, the larger
bodies get unindented.

Change-Id: I5707a6fee27d4c9ff9efd3d363af575d7a4bf2aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/165340
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-05 20:31:32 +00:00
Johan Brandhorst 94cbfc2f7f net/http: support configuring redirect fetch option
Adds a magic header value that is translated to the
Fetch API redirect option, following existing practices.

Updates #26769

Change-Id: Iaf1c9f710de63ea941a360b73f1b4bb725331a35
Reviewed-on: https://go-review.googlesource.com/c/go/+/164666
Reviewed-by: Richard Musiol <neelance@gmail.com>
Reviewed-by: Agniva De Sarker <agniva.quicksilver@gmail.com>
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-05 14:19:59 +00:00
LE Manh Cuong 2889332edf net/http: make TimeoutHandler's ResponseWriter implement Pusher
Fixes #29193

Change-Id: I03088205e51036abbc861ab5b7d141327b0429ae
Reviewed-on: https://go-review.googlesource.com/c/154383
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-02 05:42:50 +00:00
Samuel Kelemen c05f2b4869 net/http: update net/http package to replace a broken link with an archive link
replaces broken link with a web.archive.org link.

Change-Id: I438536a6ac51d837c30be5df7d3d0caadf65bb95
GitHub-Last-Rev: 0601e4d6b2
GitHub-Pull-Request: golang/go#30523
Reviewed-on: https://go-review.googlesource.com/c/164761
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-03-02 01:34:38 +00:00
Vladimir Kuzmin c04e47f821 net/http: add StatusEarlyHints (103)
HTTP status code 103 (Early Hints) from RFC 8297.

Fixes #29655

Change-Id: Ia1edbb561ee46f42d7fa1aae3ab9586497fcdb6c
Reviewed-on: https://go-review.googlesource.com/c/157339
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-01 16:10:59 +00:00
Marcel van Lohuizen b34c5f0cb4 net/http: remove use of DeepEqual for testing errors
Comparing errors using DeepEqual breaks if frame information
is added as proposed in Issue #29934.

Updates #29934.

Change-Id: I4ef076e262109a9d6f5b18846129df2535611d71
Reviewed-on: https://go-review.googlesource.com/c/162178
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
2019-02-27 18:24:06 +00:00
Dmitry Mottl b68624464d net/http: add godoc for Dir.Open function
This commit adds godoc for Dir.Open function.

Change-Id: Ibc3b22f38660a082802e1f868c5cf9d880fc2801
GitHub-Last-Rev: 774cfd7d8c
GitHub-Pull-Request: golang/go#30353
Reviewed-on: https://go-review.googlesource.com/c/163437
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-26 23:03:32 +00:00
Ggicci ea65d015b8 net/http: clean the path of the stripped URL by StripPrefix
The path of the new stripped URL should also be cleaned. Since an empty path
may cause unexpected errors in some HTTP handlers, e.g. http.ServeFile.

Fixes #30165

Change-Id: Ib44fdce6388b5d62ffbcab5266925ef8f13f26e2
Reviewed-on: https://go-review.googlesource.com/c/161738
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-02-26 22:55:59 +00:00
Emmanuel T Odeke b690d7e513 net/http/httputil: make TestDumpRequest idempotent
TestDumpRequest was failing with -count=2 or more
because for test cases that involved mustReadRequest,
the body was created as a *bufio.Reader. DumpRequest
and DumpRequestOut would then read the body until EOF
and would close it after use.
However, on re-runs of the test, the body would
be terminally exhausted and result in an unexpected
error "http: invalid Read on closed Body".

The update to the test cases adds an extra field "GetReq"
which allows us to construct requests per run of the tests
and hence make the test indefinitely re-runnable/idempotent.
"Req" or "GetReq" are mutually exclusive: either one of them
can be set or nil, but not both.

Fixes #26858

Change-Id: Ice3083dac1aa3249da4afc7075cd984eb159530d
Reviewed-on: https://go-review.googlesource.com/c/153377
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-13 16:58:53 +00:00
berkant ipek 1edd2a34c1 net/http/httptrace: fix typo
Change-Id: I15279e4aa9306bde925929907a7b5e7ef5d8b642
GitHub-Last-Rev: 6bc2d66aec
GitHub-Pull-Request: golang/go#30193
Reviewed-on: https://go-review.googlesource.com/c/162018
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-12 14:47:52 +00:00
Brad Fitzpatrick f1d662f347 net/url, net/http: relax CTL-in-URL validation to only ASCII CTLs
CL 159157 was doing UTF-8 decoding of URLs. URLs aren't really UTF-8,
even if sometimes they are in some contexts.

Instead, only reject ASCII CTLs.

Updates #27302
Updates #22907

Change-Id: Ibd64efa5d3a93263d175aadf1c9f87deb4670c62
Reviewed-on: https://go-review.googlesource.com/c/160178
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-29 20:42:54 +00:00
Brad Fitzpatrick 1e450aa2f2 net/http: update bundled http2
Updates bundled http2 to x/net git rev ed066c81e7 for:

    http2: Revert a closed stream cannot receive data
    https://golang.org/cl/153977

Updates golang/go#28204

Change-Id: I0a489e4e8a581a107970199f64f0fa9281982efe
Reviewed-on: https://go-review.googlesource.com/c/159179
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-01-23 21:33:28 +00:00
Brad Fitzpatrick 829c5df586 net/url, net/http: reject control characters in URLs
This is a more conservative version of the reverted CL 99135 (which
was reverted in CL 137716)

The net/url part rejects URLs with ASCII CTLs from being parsed and
the net/http part rejects writing them if a bogus url.URL is
constructed otherwise.

Updates #27302
Updates #22907

Change-Id: I09a2212eb74c63db575223277aec363c55421ed8
Reviewed-on: https://go-review.googlesource.com/c/159157
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2019-01-23 19:31:46 +00:00
Brad Fitzpatrick 246e6ceb3b net/http: clarify Transport connection reuse docs a bit
Updates #26095 (or fixes it)

Change-Id: I92488dabe823b82e1ba534648fe6d63d25d0ae9f
Reviewed-on: https://go-review.googlesource.com/c/158417
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-18 02:38:08 +00:00
Brad Fitzpatrick a14ed2a82a net/http/httputil: run the ReverseProxy.ModifyResponse hook for upgrades
Fixes #29627

Change-Id: I08a5b45151a11b5a4f3b5a2d984c0322cf904697
Reviewed-on: https://go-review.googlesource.com/c/157098
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-09 15:51:59 +00:00
Tobias Klauser 99ea99ec4c net/http/httputil: fix typo in ReverseProxy godoc
Change-Id: Iea33fe64403ca2e6f87a4e070af5e97d96506e41
Reviewed-on: https://go-review.googlesource.com/c/157118
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-01-09 14:49:46 +00:00
Julien Salleyron 2175177497 net/http/httputil: fix missing previous headers in response when switching protocol in ReverseProxy
When using switching protocol, previous headers set before the reverse proxy are lost.

Fixes #29407

Change-Id: Ia2b9784022d9bccef8625519ccbabbe8a276dfc0
GitHub-Last-Rev: 79bb493dcb
GitHub-Pull-Request: golang/go#29408
Reviewed-on: https://go-review.googlesource.com/c/155741
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-01-02 17:00:30 +00:00
Taufiq Rahman 58a17b43d1 net/http: fix typographical error in transport.go
Change-Id: I5f9de0daa3c18ecd7d6cd30ea13d147e227b3550
GitHub-Last-Rev: 5eabcbd91f
GitHub-Pull-Request: golang/go#29454
Reviewed-on: https://go-review.googlesource.com/c/155920
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-29 17:24:26 +00:00
catatsuy 7e1ec1e9cc net/http: fix the old url about deflate
Change-Id: Iaa1468296fbc98389165a152cf8b591216c22489
Reviewed-on: https://go-review.googlesource.com/c/155217
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-20 05:42:07 +00:00
Brad Fitzpatrick b875991a49 net/http/httptest: don't register a flag unless it looks like it's in use
We shouldn't pollute people's flags with this debugging flag that was
never really meant to be public. It's certainly not documented.

So keep it for now, but don't register it unless it looks like it's in
use (by looking at os.Args). Kinda gross, but less gross than before.

Fixes #28619

Change-Id: I47498948a26a71ff36f9658a6d9dac73fd0a3016
Reviewed-on: https://go-review.googlesource.com/c/154217
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2018-12-14 19:56:15 +00:00
Richard Musiol 7d9649bf0d syscall/js: rename js.Callback to js.Func
The name "Callback" does not fit to all use cases of js.Callback.
This commit changes its name to Func. Accordingly NewCallback
gets renamed to FuncOf, which matches ValueOf and TypedArrayOf.

The package syscall/js is currently exempt from Go's compatibility
promise and js.Callback is already affected by a breaking change in
this release cycle. See #28711 for details.

Fixes #28711

Change-Id: I2c380970c3822bed6a3893909672c15d0cbe9da3
Reviewed-on: https://go-review.googlesource.com/c/153559
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-13 18:34:01 +00:00
Bobby DeSimone ea509c95e9 net/http/httputil: add tests for singleJoiningSlash.
These changes add tests for the unexported function singleJoiningSlash.

Change-Id: I107905aac4a3c2544be309098b67e970ea5b542c
GitHub-Last-Rev: ed6f86f619
GitHub-Pull-Request: golang/go#29088
Reviewed-on: https://go-review.googlesource.com/c/152337
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-12-04 05:49:46 +00:00
SALLEYRON Julien 48399cae9f net/http/httputil: fix unannounced trailers when body is empty
Fix unannounced trailers when body is empty and without announced trailers.

Fixes #29031

Change-Id: If49951a42fe56d4be4436a999627db4c2678659d
GitHub-Last-Rev: 3469adc8f5
GitHub-Pull-Request: golang/go#29032
Reviewed-on: https://go-review.googlesource.com/c/151898
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-03 21:45:17 +00:00
Brad Fitzpatrick f98081e515 net/http: document CanonicalHeaderKey from Header
And remove some unnecessary textproto references. (The net/http
package's CanonicalHeaderKey just calls textproto's
CanonicalMIMEHeaderKey)

Fixes #28894

Change-Id: Ibd277893a168368c593147a2677ad6130870cb88
Reviewed-on: https://go-review.googlesource.com/c/152157
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-12-03 20:09:50 +00:00
Leon Klingele 1adbb2bb9b net/http: add StatusTooEarly (425)
StatusTooEarly can be returned to indicate that a server is unwilling
to accept early data as introduced in TLS 1.3.
The status code was specified in RFC 8470, section 5.2.

Major supported browsers are:
- Firefox as of version 58
  https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/425#Browser_compatibility
- Chromium as of version 73.0.3628.1
  58097ec382

Change-Id: I3f62f4193bae198994d08fde7e92e0ccd080e59a
GitHub-Last-Rev: fa885040ea
GitHub-Pull-Request: golang/go#29073
Reviewed-on: https://go-review.googlesource.com/c/152118
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-03 06:01:35 +00:00
Tobias Klauser 9e277f7d55 all: use "reports whether" consistently instead of "returns whether"
Follow-up for CL 147037 and after Brad noticed the "returns whether"
pattern during the review of CL 150621.

Go documentation style for boolean funcs is to say:

    // Foo reports whether ...
    func Foo() bool

(rather than "returns whether")

Created with:

    $ perl -i -npe 's/returns whether/reports whether/' $(git grep -l "returns whether" | grep -v vendor)

Change-Id: I15fe9ff99180ad97750cd05a10eceafdb12dc0b4
Reviewed-on: https://go-review.googlesource.com/c/150918
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-02 15:12:26 +00:00
Brad Fitzpatrick 6d6c8582df net/http: update bundled x/net/http2
This updates x/net/http2 to x/net git rev 351d144f for:

   http2: revert Transport's strict interpretation of MAX_CONCURRENT_STREAMS
   https://golang.org/cl/151857

   http2: don't leak streams on broken body
   https://golang.org/cl/132715

   http2: remove support for Go 1.8 and earlier
   https://golang.org/cl/145677

   http2: reduce init-time work & allocations
   https://golang.org/cl/127664

And some CLs fixing typos.

Fixes #27044
Fixes #27208

Change-Id: I11cc32576c690199ceb4c0bd1448d01e3cab3097
Reviewed-on: https://go-review.googlesource.com/c/152080
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-12-01 03:56:48 +00:00
Bryan C. Mills 2012227b01 vendor/golang_org/x: move to internal/x
Packages in vendor/ directories have a "vendor/" path prefix in GOPATH
mode, but intentionally do not in module mode. Since the import path
is embedded in the compiled output, changing that path invalidates
cache entries and causes cmd/go to try to rebuild (and reinstall) the
vendored libraries, which will fail if the directory containing those
libraries is read-only.

If I understood correctly, this is the approach Russ suggested as an
alternative to https://golang.org/cl/136138.

Fixes #27285
Fixes #26988

Change-Id: I8a2507fa892b84cde0a803aaa79e460723da572b
Reviewed-on: https://go-review.googlesource.com/c/147443
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2018-11-29 15:42:16 +00:00
Meng Zhuo 9aadbf5755 net/http: prevent transport sends two "Connection: close" headers
There are three functions that do Connection header write:
1. transport.go/ persistConn.roundTrip
2. transfer.go/ transferWriter.writeHeader
3. request.go/ Request.write

The root cause is roundTrip didn't lookup into request.Close and
transferWriter
didn't take care of extraHeaders.

Fixes #28886

Change-Id: I1d131019c7cd42eb1bcc972c631b7df7511c1f39
Reviewed-on: https://go-review.googlesource.com/c/150722
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-28 21:33:21 +00:00
haormj 4a8ce14af9 net/http: fix spelling mistake in a comment
Fixes #28904

Change-Id: I8d416c47479a266735a39c926fd2f0f2bb25d57b
GitHub-Last-Rev: 3a7865a5be
GitHub-Pull-Request: golang/go#28907
Reviewed-on: https://go-review.googlesource.com/c/150737
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-21 12:31:39 +00:00
mbj36 b1e8846dfb net/http: fix typo in the SameSite docs
Fixes #28244

Change-Id: I3ca36fd513f5543af0c8af254d267254c7d5e803
GitHub-Last-Rev: 83b16fac4e
GitHub-Pull-Request: golang/go#28302
Reviewed-on: https://go-review.googlesource.com/c/143480
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-16 16:39:22 +00:00
Brad Fitzpatrick bfd9b94069 net/http: make Transport respect {X-,}Idempotency-Key header
Fixes #19943

Change-Id: I5e0fefe44791d7b3556095d726c2a753ec551ef2
Reviewed-on: https://go-review.googlesource.com/c/147457
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-11-14 21:07:02 +00:00
Brad Fitzpatrick ee55f0856a net/http/httputil: make ReverseProxy automatically proxy WebSocket requests
Fixes #26937

Change-Id: I6cdc1bad4cf476cd2ea1462b53444eccd8841e14
Reviewed-on: https://go-review.googlesource.com/c/146437
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-11-13 01:42:47 +00:00
Filippo Valsorda 30cc978085 crypto/tls: enable TLS 1.3 and update tests
To disable TLS 1.3, simply remove VersionTLS13 from supportedVersions,
as tested by TestEscapeRoute, and amend documentation. To make it
opt-in, revert the change to (*Config).supportedVersions from this CL.

I did not have the heart to implement the early data skipping feature
when I realized that it did not offer a choice between two
abstraction-breaking options, but demanded them both (look for handshake
type in case of HelloRetryRequest, trial decryption otherwise). It's a
lot of complexity for an apparently small gain, but if anyone has strong
opinions about it let me know.

Note that in TLS 1.3 alerts are encrypted, so the close_notify peeking
to return (n > 0, io.EOF) from Read doesn't work. If we are lucky, those
servers that unexpectedly close connections after serving a single
request will have stopped (maybe thanks to H/2) before they got updated
to TLS 1.3.

Relatedly, session tickets are now provisioned on the client first Read
instead of at Handshake time, because they are, well, post-handshake
messages. If this proves to be a problem we might try to peek at them.

Doubled the tests that cover logic that's different in TLS 1.3.

The benchmarks for TLS 1.2 compared to be0f3c286b (before TLS 1.3 and
its refactors, after CL 142817 changed them to use real connections)
show little movement.

name                                       old time/op   new time/op   delta
HandshakeServer/RSA-8                        795µs ± 1%    798µs ± 1%    ~     (p=0.057 n=10+18)
HandshakeServer/ECDHE-P256-RSA-8             903µs ± 0%    909µs ± 1%  +0.68%  (p=0.000 n=8+17)
HandshakeServer/ECDHE-P256-ECDSA-P256-8      198µs ± 0%    204µs ± 1%  +3.24%  (p=0.000 n=9+18)
HandshakeServer/ECDHE-X25519-ECDSA-P256-8    202µs ± 3%    208µs ± 1%  +2.98%  (p=0.000 n=9+20)
HandshakeServer/ECDHE-P521-ECDSA-P521-8     15.5ms ± 1%   15.9ms ± 2%  +2.49%  (p=0.000 n=10+20)
Throughput/MaxPacket/1MB-8                  5.81ms ±23%   6.14ms ±44%    ~     (p=0.605 n=8+18)
Throughput/MaxPacket/2MB-8                  8.91ms ±22%   8.74ms ±33%    ~     (p=0.498 n=9+19)
Throughput/MaxPacket/4MB-8                  12.8ms ± 3%   14.0ms ±10%  +9.74%  (p=0.000 n=10+17)
Throughput/MaxPacket/8MB-8                  25.1ms ± 7%   24.6ms ±16%    ~     (p=0.129 n=9+19)
Throughput/MaxPacket/16MB-8                 46.3ms ± 4%   45.9ms ±12%    ~     (p=0.340 n=9+20)
Throughput/MaxPacket/32MB-8                 88.5ms ± 4%   86.0ms ± 4%  -2.82%  (p=0.004 n=10+20)
Throughput/MaxPacket/64MB-8                  173ms ± 2%    167ms ± 7%  -3.42%  (p=0.001 n=10+19)
Throughput/DynamicPacket/1MB-8              5.88ms ± 4%   6.59ms ±64%    ~     (p=0.232 n=9+18)
Throughput/DynamicPacket/2MB-8              9.08ms ±12%   8.73ms ±21%    ~     (p=0.408 n=10+18)
Throughput/DynamicPacket/4MB-8              14.2ms ± 5%   14.0ms ±11%    ~     (p=0.188 n=9+19)
Throughput/DynamicPacket/8MB-8              25.1ms ± 6%   24.0ms ± 7%  -4.39%  (p=0.000 n=10+18)
Throughput/DynamicPacket/16MB-8             45.6ms ± 3%   43.3ms ± 1%  -5.22%  (p=0.000 n=10+8)
Throughput/DynamicPacket/32MB-8             88.4ms ± 3%   84.8ms ± 2%  -4.06%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8              175ms ± 3%    167ms ± 2%  -4.63%  (p=0.000 n=10+10)
Latency/MaxPacket/200kbps-8                  694ms ± 0%    694ms ± 0%  -0.02%  (p=0.000 n=9+9)
Latency/MaxPacket/500kbps-8                  279ms ± 0%    279ms ± 0%  -0.09%  (p=0.000 n=10+10)
Latency/MaxPacket/1000kbps-8                 140ms ± 0%    140ms ± 0%  -0.15%  (p=0.000 n=10+9)
Latency/MaxPacket/2000kbps-8                71.1ms ± 0%   71.0ms ± 0%  -0.09%  (p=0.001 n=8+9)
Latency/MaxPacket/5000kbps-8                30.5ms ± 6%   30.1ms ± 6%    ~     (p=0.905 n=10+9)
Latency/DynamicPacket/200kbps-8              134ms ± 0%    134ms ± 0%    ~     (p=0.796 n=9+9)
Latency/DynamicPacket/500kbps-8             54.8ms ± 0%   54.7ms ± 0%  -0.18%  (p=0.000 n=8+10)
Latency/DynamicPacket/1000kbps-8            28.5ms ± 0%   29.1ms ± 8%    ~     (p=0.173 n=8+10)
Latency/DynamicPacket/2000kbps-8            15.3ms ± 6%   15.9ms ±10%    ~     (p=0.905 n=9+10)
Latency/DynamicPacket/5000kbps-8            9.14ms ±21%   9.65ms ±82%    ~     (p=0.529 n=10+10)

name                                       old speed     new speed     delta
Throughput/MaxPacket/1MB-8                 175MB/s ±13%  167MB/s ±64%    ~     (p=0.646 n=7+20)
Throughput/MaxPacket/2MB-8                 241MB/s ±25%  241MB/s ±40%    ~     (p=0.660 n=9+20)
Throughput/MaxPacket/4MB-8                 328MB/s ± 3%  300MB/s ± 9%  -8.70%  (p=0.000 n=10+17)
Throughput/MaxPacket/8MB-8                 335MB/s ± 7%  340MB/s ±17%    ~     (p=0.212 n=9+20)
Throughput/MaxPacket/16MB-8                363MB/s ± 4%  367MB/s ±11%    ~     (p=0.340 n=9+20)
Throughput/MaxPacket/32MB-8                379MB/s ± 4%  390MB/s ± 4%  +2.93%  (p=0.004 n=10+20)
Throughput/MaxPacket/64MB-8                388MB/s ± 2%  401MB/s ± 7%  +3.25%  (p=0.004 n=10+20)
Throughput/DynamicPacket/1MB-8             178MB/s ± 4%  157MB/s ±73%    ~     (p=0.127 n=9+20)
Throughput/DynamicPacket/2MB-8             232MB/s ±11%  243MB/s ±18%    ~     (p=0.415 n=10+18)
Throughput/DynamicPacket/4MB-8             296MB/s ± 5%  299MB/s ±15%    ~     (p=0.295 n=9+20)
Throughput/DynamicPacket/8MB-8             334MB/s ± 6%  350MB/s ± 7%  +4.58%  (p=0.000 n=10+18)
Throughput/DynamicPacket/16MB-8            368MB/s ± 3%  388MB/s ± 1%  +5.48%  (p=0.000 n=10+8)
Throughput/DynamicPacket/32MB-8            380MB/s ± 3%  396MB/s ± 2%  +4.20%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8            384MB/s ± 3%  403MB/s ± 2%  +4.83%  (p=0.000 n=10+10)

Comparing TLS 1.2 and TLS 1.3 at tip shows a slight (~5-10%) slowdown of
handshakes, which might be worth looking at next cycle, but the latency
improvements are expected to overshadow that.

name                                       old time/op   new time/op   delta
HandshakeServer/ECDHE-P256-RSA-8             909µs ± 1%    963µs ± 0%   +5.87%  (p=0.000 n=17+18)
HandshakeServer/ECDHE-P256-ECDSA-P256-8      204µs ± 1%    225µs ± 2%  +10.20%  (p=0.000 n=18+20)
HandshakeServer/ECDHE-X25519-ECDSA-P256-8    208µs ± 1%    230µs ± 2%  +10.35%  (p=0.000 n=20+18)
HandshakeServer/ECDHE-P521-ECDSA-P521-8     15.9ms ± 2%   15.9ms ± 1%     ~     (p=0.444 n=20+19)
Throughput/MaxPacket/1MB-8                  6.14ms ±44%   7.07ms ±46%     ~     (p=0.057 n=18+19)
Throughput/MaxPacket/2MB-8                  8.74ms ±33%   8.61ms ± 9%     ~     (p=0.552 n=19+17)
Throughput/MaxPacket/4MB-8                  14.0ms ±10%   14.1ms ±12%     ~     (p=0.707 n=17+20)
Throughput/MaxPacket/8MB-8                  24.6ms ±16%   25.6ms ±14%     ~     (p=0.107 n=19+20)
Throughput/MaxPacket/16MB-8                 45.9ms ±12%   44.7ms ± 6%     ~     (p=0.607 n=20+19)
Throughput/MaxPacket/32MB-8                 86.0ms ± 4%   87.9ms ± 8%     ~     (p=0.113 n=20+19)
Throughput/MaxPacket/64MB-8                  167ms ± 7%    169ms ± 2%   +1.26%  (p=0.011 n=19+19)
Throughput/DynamicPacket/1MB-8              6.59ms ±64%   6.79ms ±43%     ~     (p=0.480 n=18+19)
Throughput/DynamicPacket/2MB-8              8.73ms ±21%   9.58ms ±13%   +9.71%  (p=0.006 n=18+20)
Throughput/DynamicPacket/4MB-8              14.0ms ±11%   13.9ms ±10%     ~     (p=0.687 n=19+20)
Throughput/DynamicPacket/8MB-8              24.0ms ± 7%   24.6ms ± 8%   +2.36%  (p=0.045 n=18+17)
Throughput/DynamicPacket/16MB-8             43.3ms ± 1%   44.3ms ± 2%   +2.48%  (p=0.001 n=8+9)
Throughput/DynamicPacket/32MB-8             84.8ms ± 2%   86.7ms ± 2%   +2.27%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8              167ms ± 2%    170ms ± 3%   +1.89%  (p=0.005 n=10+10)
Latency/MaxPacket/200kbps-8                  694ms ± 0%    699ms ± 0%   +0.65%  (p=0.000 n=9+10)
Latency/MaxPacket/500kbps-8                  279ms ± 0%    280ms ± 0%   +0.68%  (p=0.000 n=10+10)
Latency/MaxPacket/1000kbps-8                 140ms ± 0%    141ms ± 0%   +0.59%  (p=0.000 n=9+9)
Latency/MaxPacket/2000kbps-8                71.0ms ± 0%   71.3ms ± 0%   +0.42%  (p=0.000 n=9+9)
Latency/MaxPacket/5000kbps-8                30.1ms ± 6%   30.7ms ±10%   +1.93%  (p=0.019 n=9+9)
Latency/DynamicPacket/200kbps-8              134ms ± 0%    138ms ± 0%   +3.22%  (p=0.000 n=9+10)
Latency/DynamicPacket/500kbps-8             54.7ms ± 0%   56.3ms ± 0%   +3.03%  (p=0.000 n=10+8)
Latency/DynamicPacket/1000kbps-8            29.1ms ± 8%   29.1ms ± 0%     ~     (p=0.173 n=10+8)
Latency/DynamicPacket/2000kbps-8            15.9ms ±10%   16.4ms ±36%     ~     (p=0.633 n=10+8)
Latency/DynamicPacket/5000kbps-8            9.65ms ±82%   8.32ms ± 8%     ~     (p=0.573 n=10+8)

name                                       old speed     new speed     delta
Throughput/MaxPacket/1MB-8                 167MB/s ±64%  155MB/s ±55%     ~     (p=0.224 n=20+19)
Throughput/MaxPacket/2MB-8                 241MB/s ±40%  244MB/s ± 9%     ~     (p=0.407 n=20+17)
Throughput/MaxPacket/4MB-8                 300MB/s ± 9%  298MB/s ±11%     ~     (p=0.707 n=17+20)
Throughput/MaxPacket/8MB-8                 340MB/s ±17%  330MB/s ±13%     ~     (p=0.201 n=20+20)
Throughput/MaxPacket/16MB-8                367MB/s ±11%  375MB/s ± 5%     ~     (p=0.607 n=20+19)
Throughput/MaxPacket/32MB-8                390MB/s ± 4%  382MB/s ± 8%     ~     (p=0.113 n=20+19)
Throughput/MaxPacket/64MB-8                401MB/s ± 7%  397MB/s ± 2%   -0.96%  (p=0.030 n=20+19)
Throughput/DynamicPacket/1MB-8             157MB/s ±73%  156MB/s ±39%     ~     (p=0.738 n=20+20)
Throughput/DynamicPacket/2MB-8             243MB/s ±18%  220MB/s ±14%   -9.65%  (p=0.006 n=18+20)
Throughput/DynamicPacket/4MB-8             299MB/s ±15%  303MB/s ± 9%     ~     (p=0.512 n=20+20)
Throughput/DynamicPacket/8MB-8             350MB/s ± 7%  342MB/s ± 8%   -2.27%  (p=0.045 n=18+17)
Throughput/DynamicPacket/16MB-8            388MB/s ± 1%  378MB/s ± 2%   -2.41%  (p=0.001 n=8+9)
Throughput/DynamicPacket/32MB-8            396MB/s ± 2%  387MB/s ± 2%   -2.21%  (p=0.000 n=10+10)
Throughput/DynamicPacket/64MB-8            403MB/s ± 2%  396MB/s ± 3%   -1.84%  (p=0.005 n=10+10)

Fixes #9671

Change-Id: Ieb57c5140eb2c083b8be0d42b240cd2eeec0dcf6
Reviewed-on: https://go-review.googlesource.com/c/147638
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
2018-11-12 20:44:39 +00:00
Richard Musiol 6dd70fc5e3 all: add support for synchronous callbacks to js/wasm
With this change, callbacks returned by syscall/js.NewCallback
get executed synchronously. This is necessary for the APIs of
many JavaScript libraries.

A callback triggered during a call from Go to JavaScript gets executed
on the same goroutine. A callback triggered by JavaScript's event loop
gets executed on an extra goroutine.

Fixes #26045
Fixes #27441

Change-Id: I591b9e85ab851cef0c746c18eba95fb02ea9e85b
Reviewed-on: https://go-review.googlesource.com/c/142004
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-10 11:57:17 +00:00
Mikio Hara 510eea2dfc net/http: update bundled SOCKS client
Updates socks_bundle.go to git rev 26e67e7 for:
- 26e67e7 internal/socks: fix socket descriptor leakage in Dialer.Dial

Change-Id: I9ab27a85504d77f1ca2e97cb005f5e37fd3c3ff4
Reviewed-on: https://go-review.googlesource.com/c/147717
Run-TryBot: Mikio Hara <mikioh.public.networking@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-06 06:57:39 +00:00
Carl Mastrangelo 1645dfa23f net/http: speed up ServeMux matching
Scanning through all path patterns is not necessary, since the
paths do not change frequently.  Instead, maintain a sorted list
of path prefixes and return the first match.

name            old time/op  new time/op  delta
ServerMatch-12   134ns ± 3%    17ns ± 4%  -86.95%  (p=0.000 n=19+20)

Change-Id: I15b4483dc30db413321435ee6815fc9bf2bcc546
Reviewed-on: https://go-review.googlesource.com/c/144937
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-02 23:22:57 +00:00
BigMikes 7b0fa529f3 net/http: in Transport, don't error on non-chunked response with Trailer header
There are cases where HTTP message specifies the Trailer header
but not the Transfer-Encoding = chunked. The existing
implementation would return an error in those cases, without
returning also the message itself.
Instead, it would be preferable to let the library user decide when
the message is valid or not.
This change makes the fixTrailer() function not to return an error
and to keep the Trailer value in the Response.Header map but not
populate Response.Trailer.

Fixes #27197

Change-Id: Ic1e96791fde97f31dc5ecb8de05c8e4f49465c2c
Reviewed-on: https://go-review.googlesource.com/c/145398
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-30 01:42:59 +00:00
Eugene Kalinin c659be4dc8 net: make cgo resolver work more accurately with network parameter
Unlike the go resolver, the existing cgo resolver exchanges both DNS A
and AAAA RR queries unconditionally and causes unreasonable connection
setup latencies to applications using the cgo resolver.

This change adds new argument (`network`) in all functions through the
series of calls: from Resolver.internetAddrList to cgoLookupIPCNAME.

Benefit: no redundant DNS calls if certain IP version is used IPv4/IPv6
(no `AAAA` DNS requests if used tcp4, udp4, ip4 network. And vice
versa: no `A` DNS requests if used tcp6, udp6, ip6 network)

Fixes #25947

Change-Id: I39edbd726d82d6133fdada4d06cd90d401e7e669
Reviewed-on: https://go-review.googlesource.com/c/120215
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-25 03:14:03 +00:00
Brad Fitzpatrick fc4f2e5692 net/http: fix comment change omitted between versions of CL 143177
Updates #23689

Change-Id: Icddec2fcc39802cacd651a9c94290e86cf1e48d1
Reviewed-on: https://go-review.googlesource.com/c/144517
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-25 03:03:15 +00:00
Brad Fitzpatrick c942191c20 crypto/tls, net/http: reject HTTP requests to HTTPS server
This adds a crypto/tls.RecordHeaderError.Conn field containing the TLS
underlying net.Conn for non-TLS handshake errors, and then uses it in
the net/http Server to return plaintext HTTP 400 errors when a client
mistakenly sends a plaintext HTTP request to an HTTPS server. This is the
same behavior as Apache.

Also in crypto/tls: swap two error paths to not use a value before
it's valid, and don't send a alert record when a handshake contains a
bogus TLS record (a TLS record in response won't help a non-TLS
client).

Fixes #23689

Change-Id: Ife774b1e3886beb66f25ae4587c62123ccefe847
Reviewed-on: https://go-review.googlesource.com/c/143177
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2018-10-24 22:49:50 +00:00
Brad Fitzpatrick da6c168378 net/http: flesh out Transport's HTTP/1 CONNECT+bidi support to match HTTP/2
Fixes #17227

Change-Id: I0f8964593d69623b85d5759f6276063ee62b2915
Reviewed-on: https://go-review.googlesource.com/c/123156
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-12 15:00:32 +00:00
Brad Fitzpatrick c870d56f98 net/http/httputil: fix race/crash in previous ReverseProxy change
The previous ReverseProxy change, CL 137335, introduced a bug which could cause
a race and/or a crash.

This reliably crashed before:

$ go test -short -race -v -run=TestReverseProxyFlushInterval -count=20 net/http/httputil

The problem was a goroutine was running http.ResponseWriter.Flush
after the http.Handler's ServeHTTP completed. There was code to
prevent that (a deferred stop call) but the stop call didn't consider
the case where time.AfterFunc had already fired off a new goroutine
but that goroutine hadn't yet scheduled.

Change-Id: I06357908465a3b953efc33e63c70dec19a501adf
Reviewed-on: https://go-review.googlesource.com/c/140977
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-10 02:08:36 +00:00
Brad Fitzpatrick 5440bfc2ea net/http/httputil: rewrite flushing code, disable on Server-Sent Events
* Rewrite the flushing code to not use a persistent goroutine, which
  also simplifies testing.
* Define the meaning of a negative flush interval. Its meaning doesn't
  change, but now it's locked in, and then we can use it to optimize
  the performance of the non-buffered case to avoid use of an AfterFunc.
* Support (internal-only) special casing of FlushInterval values per
  request/response.
* For now, treat Server-Sent Event responses as unbuffered. (or rather,
  immediately flushed from the buffer per-write)

Fixes #27816

Change-Id: Ie0f975c997daa3db539504137c741a96d7022665
Reviewed-on: https://go-review.googlesource.com/c/137335
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-09 15:26:06 +00:00
Igor Zhilianin 04dc1b2443 all: fix a bunch of misspellings
Change-Id: I94cebca86706e072fbe3be782d3edbe0e22b9432
GitHub-Last-Rev: 8e15a40545
GitHub-Pull-Request: golang/go#28067
Reviewed-on: https://go-review.googlesource.com/c/140437
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-08 03:12:03 +00:00
Igor Zhilianin f90e89e675 all: fix a bunch of misspellings
Change-Id: If2954bdfc551515403706b2cd0dde94e45936e08
GitHub-Last-Rev: d4cfc41a55
GitHub-Pull-Request: golang/go#28049
Reviewed-on: https://go-review.googlesource.com/c/140299
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-06 15:40:03 +00:00
Daniel Martí 112f28defc io: export StringWriter
And start using it elsewhere in the standard library, removing the
copies in the process.

While at it, rewrite the io.WriteString godoc to be more clear, since it
can now make reference to the defined interface.

Fixes #27946.

Change-Id: Id5ba223c09c19e5fb49815bd3b1bd3254fc786f3
Reviewed-on: https://go-review.googlesource.com/c/139457
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-03 20:13:35 +00:00
David Url 0df9fa2ebe net/http: log call site which causes multiple header writes
If an illegal header write is detected, find the first caller outside of
net/http using runtime.CallersFrames and include the call site in the log
message.

Fixes #18761

Change-Id: I92be00ac206c6ebdd60344ad7bf40a7c4c188547
Reviewed-on: https://go-review.googlesource.com/c/130997
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-03 15:29:15 +00:00
Eduard Urbach 8919fe9e4d net/http: explain more how Pusher.Push works
This will clarify that the resources are not completely pushed yet when `Push` returns and that it starts a separate goroutine. This might be implementation dependant but as I believe there is currently only one implementation it should be added to the documentation of the interface which most people will look up first.

Change-Id: Id151c5563fd0c4e611eb1d93b4f64bf747ddf6d4
GitHub-Last-Rev: 1f46eb9a08
GitHub-Pull-Request: golang/go#25025
Reviewed-on: https://go-review.googlesource.com/c/108939
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-03 02:32:28 +00:00
Michael Henderson 06ff4772ab net/http: add missing words to SameSite doc/comments
Change-Id: Ie7f6592d726eba73a5e6b3203e083bff55ff8e4e
GitHub-Last-Rev: 4e317b39dd
GitHub-Pull-Request: golang/go#27977
Reviewed-on: https://go-review.googlesource.com/c/139100
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-03 02:09:38 +00:00
Brad Fitzpatrick a73d8f5a86 net/http: make Transport send WebSocket upgrade requests over HTTP/1
WebSockets requires HTTP/1 in practice (no spec or implementations
work over HTTP/2), so if we get an HTTP request that looks like it's
trying to initiate WebSockets, use HTTP/1, like browsers do.

This is part of a series of commits to make WebSockets work over
httputil.ReverseProxy. See #26937.

Updates #26937

Change-Id: I6ad3df9b0a21fddf62fa7d9cacef48e7d5d9585b
Reviewed-on: https://go-review.googlesource.com/c/137437
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-02 23:33:23 +00:00
Aleksandr Razumov 3aa3c052e3 net/http: rewind request body unconditionally
When http2 fails with ErrNoCachedConn the request is retried with body
that has already been read.

Fixes #25009

Change-Id: I51ed5c8cf469dd8b17c73fff6140ab80162bf267
Reviewed-on: https://go-review.googlesource.com/c/131755
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-02 21:11:23 +00:00
Muhammad Falak R Wani 9ec5d9c15b net/http: document Header.Set canonicalizes the header key
Fixes #27923

Change-Id: Ia902a1966beeae56e43265fc5ed987555fa834b6
Reviewed-on: https://go-review.googlesource.com/138677
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-02 17:07:42 +00:00
esell 3ee9672955 net/http: add Handle example
Currently there is no example for http.Handle in the
documentation. This adds an example.

Change-Id: I66ee9983bea1f5237757e1ef4956eae9a056e963
Reviewed-on: https://go-review.googlesource.com/137715
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-02 16:59:25 +00:00
Ian Davis d3dcd89130 all: remove repeated "the" from comments
A simple grep over the codebase for "the the" which is often
missed by humans.

Change-Id: Ie4b4f07abfc24c73dcd51c8ef1edf4f73514a21c
Reviewed-on: https://go-review.googlesource.com/138335
Reviewed-by: Dave Cheney <dave@cheney.net>
2018-09-28 08:46:55 +00:00
esell 4ba4c5ae79 net/http: add http.NotFoundHandler example
Change-Id: I6a69c7a5b829a967d75e1c79210a4906c0d8f505
Reviewed-on: https://go-review.googlesource.com/132276
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-09-27 07:40:12 +00:00
Brad Fitzpatrick da0d1a44ba all: use strings.ReplaceAll and bytes.ReplaceAll where applicable
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)

Updates #27864

Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-09-26 22:14:25 +00:00
Ingo Oeser 1058aecf61 net/http: configure http2 transport only once
it looks like we should abort trying to configure the http2 transport
again, once it has been configured already.

Otherwise there will be no effect of these checks and changes, as they
will be overridden later again and the disable logic below will have no
effect, too.

So it really looks like we just forgot a return statement here.

Change-Id: Ic99b3bbc662a4e1e1bdbde77681bd1ae597255ad
Reviewed-on: https://go-review.googlesource.com/134795
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-09-26 14:39:02 +00:00
Johan Brandhorst ce536837d8 net/http: ensure null body in Fetch response is not read
The Fetch API returns a null body if there is no response body,
on browsers that support streaming the response body. This
change ensures we check for both undefined and null bodies
before attempting to read the body.

Fixes #27196

Change-Id: I0da86b61284fe394418b4b431495e715a037f335
Reviewed-on: https://go-review.googlesource.com/131236
Reviewed-by: Richard Musiol <neelance@gmail.com>
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-09-23 19:59:57 +00:00
Vladimir Varankin fdefabadf0 net/http: use Header.clone rather then duplicating functionality
cloneHeader duplicates what Header.clone() method is doing. It's
used in a single place, which can be replaced with the use of the
method.

Change-Id: I6e8bbd6c95063f31ca3695f13fa7478873230525
GitHub-Last-Rev: eb08aeae9b
GitHub-Pull-Request: golang/go#27817
Reviewed-on: https://go-review.googlesource.com/136762
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-09-23 06:53:10 +00:00
Ivy Evans c381ba8657 net/http: fix minor typos in Request godoc
Fixes missing commas where it wasn't immediately apparent whether
"requests" was being used as a verb or a noun.

Change-Id: Ic8c99b4f46475f40a6160d26a3cd11c215940dd5
GitHub-Last-Rev: 1becf6fabe
GitHub-Pull-Request: golang/go#27649
Reviewed-on: https://go-review.googlesource.com/135135
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-09-18 08:48:52 +00:00
David Timm 0dac1e2e87 net/http: add example for http.HandleFunc
Change-Id: Id0e2fb2abad5b776ac0ed76e55e36c6b774b5b7a
Reviewed-on: https://go-review.googlesource.com/132278
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-08-30 19:02:30 +00:00
Brad Fitzpatrick 541620409d net/http: make Transport return Writable Response.Body on protocol switch
Updates #26937
Updates #17227

Change-Id: I79865938b05c219e1947822e60e4f52bb2604b70
Reviewed-on: https://go-review.googlesource.com/131279
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-25 22:46:42 +00:00
Marcus Willock 187a41dbf7 net/http: add an example of creating a custom FileSystem
The existing documentation of http.Dir is clear in that, if you want to hide
your files and directories that start with a period, you must create
a custom FileSystem. However, there are currently no example on how
to create a custom FileSystem. This commit provides an example.

Fixes #20759

Change-Id: I5a350675536f81412af384d1a316fd6cd6241563
GitHub-Last-Rev: 8b0b644cd0
GitHub-Pull-Request: golang/go#26768
Reviewed-on: https://go-review.googlesource.com/127576
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-08-21 04:55:04 +00:00
Brad Fitzpatrick d58f3e6fea net/http: add test showing that ReverseProxy HTTP/2 bidi streaming works
Change-Id: I8361ae33c785e45e3ccc7e9bc2732c887eeb41c4
Reviewed-on: https://go-review.googlesource.com/127015
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matt Layher <mdlayher@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-21 03:26:03 +00:00