See big comment in code.
Fixes#23921
Change-Id: I2dbd1acc2e9da07a71f9e0640aafe0c59a335627
Reviewed-on: https://go-review.googlesource.com/123875
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This permits specifying an ErrorHandler to customize the RoundTrip
error handling if the backend fails to return a response.
Fixes#22700Fixes#21255
Change-Id: I8879f0956e2472a07f584660afa10105ef23bf11
Reviewed-on: https://go-review.googlesource.com/77410
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
syscall/js does not allow []byte to be used in direct inputs to
its JavaScript manipulation methods since
bafe466a95.
Unfortunately, this use of a byte slice was missed, so any
uses of the WASM Roundtripper with a body will panic.
This ensures the byte slice is appropriately converted
before being passed to syscall.
Fixes#26349
Change-Id: I83847645d71ce310c1eee3decddbac990fae166b
GitHub-Last-Rev: 3914bda2ff
GitHub-Pull-Request: golang/go#26350
Reviewed-on: https://go-review.googlesource.com/123537
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Richard Musiol <neelance@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
According to RFC-3986, the sub-delims chars should not be escaped in
fragment.
So this change fixes current behavior a bit.
Fixes#19917
Change-Id: I1a8deb93255d979532f75bae183c3fb53a05d395
Reviewed-on: https://go-review.googlesource.com/61650
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Updates bundled x/net/http2 to git rev d0887baf81f4 for:
http2: ignore unknown 1xx responses like HTTP/1
https://golang.org/cl/123615
http2: fix bug in earlier CL 123615
https://golang.org/cl/123675
Also along for the ride, but without any effect:
http2: export a field of an internal type for use by net/http
https://golang.org/cl/123656Fixes#26189
Updates #17739
Change-Id: I1955d844d74113efbcbbdaa7d7a7faebb2225b45
Reviewed-on: https://go-review.googlesource.com/123676
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Previously Transport.CloseIdleConnections only closed the HTTP/2
Transport's idle connections if the HTTP/2 transport was configured
automatically via the bundled copy (in h2_bundle.go).
This makes it also work if the user called http2.ConfigureTransport
themselves using golang.org/x/net/http2 instead of the bundled copy.
No tests because we have no current way to run such cross-repo tests,
at least in any efficient or non-flaky way.
Tested by hand that:
package main
import (
"net/http"
"golang.org/x/net/http2"
)
func main() {
tr := &http.Transport{}
http2.ConfigureTransport(tr)
tr.CloseIdleConnections()
}
... now works and calls the x/net/http2.Transport.CloseIdleConnections
code. (I threw in a print statement locally)
Fixes#22891 once CL 123656 is also in.
Change-Id: Id697fd3e7877c3a988bc3c3368b88940ba56cfd0
Reviewed-on: https://go-review.googlesource.com/123657
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Updates bundled x/net/http2 to git rev cffdcf67 for:
http2: use GetBody unconditionally on Transport retry, when available
https://golang.org/cl/123476
http2: a closed stream cannot receive data
https://golang.org/cl/111676
Updates #25009
Updates #25023
Change-Id: I84f50cc50c0fa5a3c34f0037a9cb1ef468e5f0d9
Reviewed-on: https://go-review.googlesource.com/123515
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Also, remove some test code that was trying to work on XP and fix up
some comments referencing XP.
Fixes#26191
Updates #23380
Change-Id: I0b7319fe1954afddb22d396e5ec91d8c960268d8
Reviewed-on: https://go-review.googlesource.com/123415
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Remove some incorrect code that was present after since I added
support for idle timeouts in CL 22670.
This code actually caused a bug (a rare goroutine leak) rather than
prevent a bogus connection reuse.
The t.idleMu mutex already protects most the invariants, including an
explicit Stop call. There's only one Stop call on that timer, and it's
guarded by t.idleMu. What idleMu doesn't protect against is the timer
firing on its own. But we don't need code to protect against that case
because the goroutine that is created via AfterFunc when the timer
fires already checks the invariants:
// closeConnIfStillIdle closes the connection if it's still sitting idle.
// This is what's called by the persistConn's idleTimer, and is run in its
// own goroutine.
func (pc *persistConn) closeConnIfStillIdle() {
t := pc.t
t.idleMu.Lock()
defer t.idleMu.Unlock()
if _, ok := t.idleLRU.m[pc]; !ok {
// Not idle.
return
}
(note the "Not idle." part).
Tested by hand with the repro code from #25621. No more leaks.
Fixes#25621
Change-Id: Idf011a4cb1fcd01f55a5a6269e4c0ee5f4446786
Reviewed-on: https://go-review.googlesource.com/123315
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Adds tests for #122590 and updates x/net/http2 to git rev 6a8eb5e2b1 for:
http2: call httptrace.ClientTrace.GetConn in Transport when needed
https://golang.org/cl/122590
http2: fire httptrace.ClientTrace.WroteHeaderField if set
https://golang.org/cl/122816
http2: compare Connection header value case-insensitively
https://golang.org/cl/122588
This also includes the code for graceful shutdown, but it has no
public API surface via net/http, and should not affect any existing
code paths, as it's purely new stuff:
http2: implement client initiated graceful shutdown
https://golang.org/cl/30076Fixes#19761Fixes#23041
Change-Id: I5558a84591014554cad15ee3852a349ed717530f
Reviewed-on: https://go-review.googlesource.com/122591
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Prior to the fix to #23643, the ReverseProxy didn't panic with
ErrAbortHandler when the copy to a client failed.
During Go 1.11 beta testing, we found plenty of code using
ReverseProxy in tests that were unprepared for a panic.
Change the behavior to only panic when running under the http.Server
that'll handle the panic.
Updates #23643
Change-Id: Ic1fa8405fd54c858ce8c797cec79d006833a9f7d
Reviewed-on: https://go-review.googlesource.com/122819
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The same-site cookie attribute prevents a cookie from being sent along with
cross-site requests. The main goal is mitigate the risk of cross-origin
information leakage and provides some protection against cross-site request
forgery attacks.
This change adds the option to http.Cookie so it can be stored and
passed to HTTP clients.
Spec: https://tools.ietf.org/html/draft-ietf-httpbis-cookie-same-site-00Fixes#15867
Based on
eb31a0f063
by Reed Loden <reed@hackerone.com>
Change-Id: I98c8a9a92358b2f632990576879759e3aff38cff
Reviewed-on: https://go-review.googlesource.com/79919
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Add field to http.Transport which limits connections per host,
including dial-in-progress, in-use and idle (keep-alive) connections.
For HTTP/2, this field only controls the number of dials in progress.
Fixes#13957
Change-Id: I7a5e045b4d4793c6b5b1a7191e1342cd7df78e6c
Reviewed-on: https://go-review.googlesource.com/71272
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This function tests that calling Shutdown on a Server that has a "new"
connection yet to write any bytes, in which case it should wait for five
seconds until considering the connection as "idle".
However, the test was flaky. If Shutdown happened to run before the
server accepted the connection, the connection would immediately be
rejected as the server is already closed, as opposed to being accepted
in the "new" state. Then, Shutdown would return almost immediately, as
it had no connections to wait for:
--- FAIL: TestServerShutdownStateNew (2.00s)
serve_test.go:5603: shutdown too soon after 49.41µs
serve_test.go:5617: timeout waiting for Read to unblock
Fix this by making sure that the connection has been accepted before
calling Shutdown. Verified that the flake is gone after 50k concurrent
runs of the test with no failures, whereas the test used to fail around
10% of the time on my laptop:
go test -c && stress -p 256 ./http.test -test.run TestServerShutdownStateNew
Fixes#26233.
Change-Id: I819d7eedb67c48839313427675facb39d9c17257
Reviewed-on: https://go-review.googlesource.com/122355
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Before CL 116855, Transport would only skip over 100 (expect-continue)
responses automatically and treat all other 1xx responses as if they
were the final status. CL 116855 made the Transport more spec
compliant (ignoring unknown 1xx responses), but broke "101 Switching
Protocols" in the process. Since 101 is already in use and defined to
not have a following message, treat it as terminal.
Note that because the Client/Transport don't support hijacking the
underlying Conn, most clients doing a WebSocket or other protocol
upgrade are probably using net.Dial + http.ReadResponse instead, which
remained unaffected (before & after this CL).
The main affect of this CL is to fix tests that were using the
Client/Transport to test that a server returns 101, presumably without
actually switching to another protocol.
Fixes#26161
Change-Id: Ie3cd3a465f948c4d6f7ddf2a6a78a7fb935d0672
Reviewed-on: https://go-review.googlesource.com/121860
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Be super explicit that HTTP keep-alives != TCP keep-alives.
Fixes#26128
Change-Id: I77d74a6fe077259d996543f901a58aa3e49c1093
Reviewed-on: https://go-review.googlesource.com/121616
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I thought I removed this but failed to amend it to my commit before
submitting.
Change-Id: I2d687d91f4de72251548faa700006af0fea503af
Reviewed-on: https://go-review.googlesource.com/121615
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
The Server distinguishes "new" vs "idle" connections. A TCP connection
from which no bytes have yet been written is "new". A connection that
has previously served a request and is in "keep-alive" state while
waiting for a second or further request is "idle".
The graceful Server.Shutdown historically only shut down "idle"
connections, with the assumption that a "new" connection was about to
read its request and would then shut down on its own afterwards.
But apparently some clients spin up connections and don't end up using
them, so we have something that's "new" to us, but browsers or other
clients are treating as "idle" to them.
This CL tweaks our heuristic to treat a StateNew connection as
StateIdle if it's been stuck in StateNew for over 5 seconds.
Fixes#22682
Change-Id: I01ba59a6ab67755ca5ab567041b1f54aa7b7da6f
Reviewed-on: https://go-review.googlesource.com/121419
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This makes Callback more in line with TypedArray. The name "Release" is
better than "Close" because the function does not implement io.Closer.
Change-Id: I23829a14b1c969ceb04608afd9505fd5b4b0df2e
Reviewed-on: https://go-review.googlesource.com/121216
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Allow a zone to be included with the ip address that is parsed when
using DefaultResolver's LookupHost or LookupIPAddr
Fixes#20790Fixes#20767
Change-Id: I4e0baf9ade6a095af10a1b85ca6216788ba680ae
Reviewed-on: https://go-review.googlesource.com/79935
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The new function js.TypedArrayOf returns a JavaScript typed array for
a given slice.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Typed_arrays
This change also changes js.ValueOf to not accept a []byte any more.
Fixes#25532.
Change-Id: I8c7bc98ca4e21c3514d19eee7a1f92388d74ab2a
Reviewed-on: https://go-review.googlesource.com/121215
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The current resolver uses a global lookupGroup which merges LookupIPAddr
calls together for lookups for the same hostname if used concurrently.
As a result only one of the resolvers is actually used to perform the
DNS lookup but the result is shared by all the resolvers.
This commit limits the scope of the lookupGroup to the resolver itself
allowing each resolver to make its own requests without sharing the
result with other resolvers.
Fixes#22908
Change-Id: Ibba896eebb05e59f18ce4132564ea1f2b4b6c6d9
Reviewed-on: https://go-review.googlesource.com/80775
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
At least on Darwin, if getaddrinfo can't open a file descriptor it
returns EAI_NONAME ("no such host") rather than a meaningful error.
Limit the number of concurrent getaddrinfo calls to the number of file
descriptors we can open, to make that meaningless error less likely.
We don't apply the same limit to Go lookups, because for that we will
return a meaningful "too many open files" error.
Fixes#25694
Change-Id: I601857190aeb64f11e22b4a834c1c6a722a0788d
Reviewed-on: https://go-review.googlesource.com/121176
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Some headers, which are set or modified by the http library,
are not written to the standard http.Request.Header and are
not included as part of http.Response.Request.Header.
Exposing all headers alleviates this problem.
This is not a complete solution to 19761 since it does not have http/2 support.
Updates #19761
Change-Id: Ie8d4f702f4f671666b120b332378644f094e288b
Reviewed-on: https://go-review.googlesource.com/67430
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Also updates comment on isConnected field of netFD for clarification.
Change-Id: Icb1b0332e3b4c7802eae00ddc26cd5ba54c82dc2
Reviewed-on: https://go-review.googlesource.com/120955
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Beginning on Go 1.9, ServeMux has been dropping the port number from the Host
header and in the path pattern. This commit explicitly mentions the change in
behavior and adds a simple test case to ensure consistency.
Fixes#23351.
Change-Id: I0270c8bd96cda92c13ac6437cdf66c2807b3042d
Reviewed-on: https://go-review.googlesource.com/120557
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Logf doesn't make the test fail, so the test was always OK.
Change-Id: I7c10ee74ff7e5d28cbd3a35e185093cb9f349470
Reviewed-on: https://go-review.googlesource.com/120496
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The previous code acquired a read lock on src and a write lock on
dst for the entire duration of Splice. This resulted in deadlock,
in a situation akin to the following:
Splice is blocking, waiting to read from src.
The caller tries to close dst from another goroutine, but Close on
dst blocks in runtime.semacquire, because Splice is still holding a
write lock on it, and the Close didn't unblock any I/O.
The caller cannot unblock the read side of Splice through other means,
because they are stuck waiting in dst.Close().
Use more fine-grained locking instead: acquire the read lock on src
just before trying to splice from the source socket to the pipe,
and acquire the write lock on dst just before trying to splice from
the pipe to the destination socket.
Fixes#25985
Change-Id: I264c91c7a69bb6c5e28610e2bd801244804cf86d
Reviewed-on: https://go-review.googlesource.com/120317
Run-TryBot: Aram Hăvărneanu <aram@mgk.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 96575 introduced concurrency protection for
ServeMux.shouldRedirect with a read lock and deferred unlock.
However, the change produced a noticeable regression.
Instead add the suffix "RLocked" to the function name to
declare that we should hold the read lock as a pre-requisite
before calling it, hence avoiding the defer altogether.
Benchmarks:
name old time/op new time/op delta
ServeMux-8 63.3µs ± 0% 54.6µs ± 0% -13.74% (p=0.000 n=9+9)
ServeMux_SkipServe-8 41.4µs ± 2% 32.7µs ± 1% -21.05% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
ServeMux-8 17.3kB ± 0% 17.3kB ± 0% ~ (all equal)
ServeMux_SkipServe-8 0.00B 0.00B ~ (all equal)
name old allocs/op new allocs/op delta
ServeMux-8 360 ± 0% 360 ± 0% ~ (all equal)
ServeMux_SkipServe-8 0.00 0.00 ~ (all equal)
Updates #25383
Updates #25482
Change-Id: I2ffa4eafe165faa961ce23bd29b5653a89facbc2
Reviewed-on: https://go-review.googlesource.com/113996
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Second try. The previous version (CL 115039 in git rev 3988863) wasn't
accurate.
Fixes#22347
Change-Id: I473165f308c730f50b14ba787cb215f7cb9ea364
Reviewed-on: https://go-review.googlesource.com/119235
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Previously the Transport had good support for 100 Continue responses,
but other 1xx informational responses were returned as-is.
But per https://tools.ietf.org/html/rfc7231#section-6.2:
> A client MUST be able to parse one or more 1xx responses received
> prior to a final response, even if the client does not expect one. A
> user agent MAY ignore unexpected 1xx responses.
We weren't doing that. Instead, we were returning any 1xx that wasn't
100 as the final result.
With this change we instead loop over up to 5 (arbitrary) 1xx
responses until we find the final one, returning an error if there's
more than 5. The limit is just there to guard against malicious
servers and to have _some_ limit.
By default we ignore the 1xx responses, unless the user defines the
new httptrace.ClientTrace.Got1xxResponse hook, which is an expanded
version of the previous ClientTrace.Got100Continue.
Still remaining:
* httputil.ReverseProxy work. (From rfc7231#section-6.2: "A proxy MUST
forward 1xx responses unless the proxy itself requested the
generation of the 1xx response."). Which would require:
* Support for an http.Handler to generate 1xx informational responses.
Those can happen later. Fixing the Transport to be resilient to others
using 1xx in the future without negotiation (as is being discussed
with HTTP status 103) is most important for now.
Updates #17739
Change-Id: I55aae8cd978164643fccb9862cd60a230e430486
Reviewed-on: https://go-review.googlesource.com/116855
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
On dragonfly, freebsd and solaris the sendfile syscall does not update
the read position of the source fd. Update it after sendfile so
successive calls start at the correct position.
Fixes#25809
Change-Id: Iaac79f89704b75b8038d4bb60eaf793a262cdd8f
Reviewed-on: https://go-review.googlesource.com/117895
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Skip it like on freebsd until there is proper a fix for #25809
Updates #25809
Change-Id: Id53c433aee75f2a992ab6a8d58d98fd1f8a6c1c6
Reviewed-on: https://go-review.googlesource.com/117698
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Add test for freebsd issue #25809.
This test also fails on my Windows 10 Version 1803.
My hope is that adding new test will break one of our builders.
Updates #25722
Updates #25809
Change-Id: Ia103bc708b8fa3b9af57613acc44893f90b3fa18
Reviewed-on: https://go-review.googlesource.com/117775
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Brad Fitzpatrick
Julien Salleyron
Johan Brandhorst
Kunpei Sakai
Michael Fraenkel
Stephan Renatus
Mark Fischer
Daniel Martí
Caleb Martinez
Richard Musiol
Grégoire Delattre
Ian Lance Taylor
Mikio Hara
Guilherme Goncalves
Thomas Bruyelle
Andrei Tudor Călin
Emmanuel T Odeke
Tobias Klauser
Alex Brainman