Similarly to Authentication entries, Proxy-Authentication entries should be stripped to ensure sensitive information is not leaked on redirects outside of the original domain.
https://fetch.spec.whatwg.org/#authentication-entries
Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this issue.
Updates golang/go#73816Fixesgolang/go#73906
Fixes CVE-2025-4673
Change-Id: I8a0f30d5d6bff6c71689bba6efa0b747947e7eb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/679256
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
When an HTTP redirect changes the host of a request, we drop
sensitive headers such as Authorization from the redirected request.
Fix a bug where a chain of redirects could result in sensitive
headers being sent to the wrong host:
1. request to a.tld with Authorization header
2. a.tld redirects to b.tld
3. request to b.tld with no Authorization header
4. b.tld redirects to b.tld
3. request to b.tld with Authorization header restored
Thanks to Kyle Seely for reporting this issue.
For #70530Fixes#71212
Fixes CVE-2024-45336
Change-Id: Ia58a2e10d33d6b0cc7220935e771450e5c34de72
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1641
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Commit-Queue: Roland Shoemaker <bracewell@google.com>
(cherry picked from commit 2889169b87a61f1218a02994feb80fd3d8bfa87c)
Reviewed-on: https://go-internal-review.googlesource.com/c/go/+/1766
Reviewed-on: https://go-review.googlesource.com/c/go/+/643100
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
On a 301 redirect, the HTTP client changes the request to be
a GET with no body.
On a 308 redirect, the client leaves the request method and
body unchanged.
A 308 following a 301 should preserve the rewritten request
from the first redirect: GET with no body. We were preserving
the method, but sending the original body. Fix this.
Fixes#70180
Change-Id: Ie20027a6058a82bfdffc7197d07ac6c7f98099e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/626055
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
All changes are related to the code, except for the comments in src/regexp/syntax/parse.go and src/slices/slices.go.
Change-Id: I73c5d3c54099749b62210aa7f3182c5eb84bb6a6
GitHub-Last-Rev: 794aa9b053
GitHub-Pull-Request: golang/go#69170
Reviewed-on: https://go-review.googlesource.com/c/go/+/609678
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Currently, when a Transport creates a new connection for a request,
it uses the request's Context to make the Dial. If a request
times out or is canceled before a Dial completes, the Dial is
canceled.
Change this so that the lifetime of a Dial call is not bound
by the request that originated it.
This change avoids a scenario where a Transport can start and
then cancel many Dial calls in rapid succession:
- Request starts a Dial.
- A previous request completes, making its connection available.
- The new request uses the now-idle connection, and completes.
- The request Context is canceled, and the Dial is aborted.
Fixes#59017
Change-Id: I996ffabc56d3b1b43129cbfd9b3e9ea7d53d263c
Reviewed-on: https://go-review.googlesource.com/c/go/+/576555
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
When deciding whether to forward cookies or sensitive headers
across a redirect, do not attempt to interpret an IPv6 address
as a domain name.
Avoids a case where a maliciously-crafted redirect to an
IPv6 address with a scoped addressing zone could be
misinterpreted as a within-domain redirect. For example,
we could interpret "::1%.www.example.com" as a subdomain
of "www.example.com".
Thanks to Juho Nurminen of Mattermost for reporting this issue.
Fixes CVE-2023-45289
Fixes#65065
Change-Id: I8f463f59f0e700c8a18733d2b264a8bcb3a19599
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2131938
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/569340
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
When returning an error which implements net.Error and reports
itself as a timeout, also report it as matching context.DeadlineExceeded.
This matches the behavior of timeout errors in the net package
and elsewhere.
Fixes#50856
Change-Id: I2ca911e3677a699af27ba89b1200401baa8b3b1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/567537
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
This reverts CL 533119.
Reason for revert: the test fails frequently, see #65287.
Fixes#65287.
Change-Id: I5bf2ee2b7ce435608ff76b892da261c0a4a189bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/558916
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Cherry Mui <cherryyz@google.com>
Removes the RSA KEX based ciphers from the default list. This can be
reverted using the tlsrsakex GODEBUG.
Fixes#63413
Change-Id: Id221be3eb2f6c24b91039d380313f0c87d339f98
Reviewed-on: https://go-review.googlesource.com/c/go/+/541517
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
This also allows us to remove the chanWriter helper from the test,
using a simpler strings.Builder instead, relying on
clientServerTest.close for synchronization.
(I don't think this runs afoul of #38370, because the handler
functions themselves in these tests should never be executed,
let alone result in an asynchronous write to the error log.)
Fixes#57599.
Change-Id: I45c6cefca0bb218f6f9a9659de6bde454547f704
Reviewed-on: https://go-review.googlesource.com/c/go/+/539436
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Currently, net/http replaces the Referer header with the URL of the
previous request, regardless of its status. This CL changes this
behavior, respecting the Referer header for secure connections, if it is
set.
Fixes#44160
Change-Id: I2d7fe37dd681549136329e832188294691584870
Reviewed-on: https://go-review.googlesource.com/c/go/+/291636
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Nick Craig-Wood <nickcw@gmail.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Preserve sensitive headers on a redirect to a different port of the same host.
Fixes#35104
Change-Id: I5ab57c414ce92a70e688ee684b9ff02fb062b3c6
GitHub-Last-Rev: 8d53e71e22
GitHub-Pull-Request: golang/go#54539
Reviewed-on: https://go-review.googlesource.com/c/go/+/424935
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Replace the ad-hoc approach to running tests in HTTP/1 and HTTP/2
modes with a 'run' function that executes a test in various modes.
By default, these modes are HTTP/1 and HTTP/2, but tests can
opt-in to HTTPS/1 as well.
The 'run' function also takes care of post-test cleanup (running the
afterTest function).
The 'run' function runs tests in parallel by default. Tests which
can't run in parallel (generally because they use global test hooks)
pass a testNotParallel option to disable parallelism.
Update clientServerTest to use t.Cleanup to clean up after itself,
rather than leaving this up to tests to handle.
Drop an unnecessary mutex in SetReadLoopBeforeNextReadHook.
Test hooks can't be set in parallel, and we want the race detector
to notify us if two simultaneous tests try to set a hook.
Fixes#56032
Change-Id: I16be64913c426fc93d84abc6ad85dbd3bc191224
Reviewed-on: https://go-review.googlesource.com/c/go/+/438137
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David Chase <drchase@google.com>
There is already a skip in case of a later failure in the same test on
these platforms. Skip the failure if it occurs earlier too.
For #43120.
Change-Id: Id530370caa6a7df8cae593f6fdcb66871b86b125
Reviewed-on: https://go-review.googlesource.com/c/go/+/425096
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
RFC 7231 does not require that a 3xx response contain a Location header.
When receiving such a response, just return it to the caller rather than
treating it as an error.
Fixes#49281.
Change-Id: I66c06d81b0922016384a0f4ff32bf52e3a3d5983
Reviewed-on: https://go-review.googlesource.com/c/go/+/375354
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
This extends the skip added in CL 375635 to the "_Headers" variant of
the test, since we have observed similar failures in that variant on
the builders.
For #43120
Change-Id: Ib1c97fbb776b576271629272f3194da77913a941
Reviewed-on: https://go-review.googlesource.com/c/go/+/379156
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
These tests are empirically flaky on the windows/arm and windows/arm64
builders, with a consistent (but rare) failure mode.
This change skips the test if that particular failure mode is
encountered on those platforms; the skip can be removed if and when
someone has the time to pin down the root cause.
For #43120
Change-Id: Ie3a9a06bf47e3a907c7b07441acc1494a4631135
Reviewed-on: https://go-review.googlesource.com/c/go/+/375635
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.
Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.
A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.
Fixes#49884.
Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
When sending a Request with a non-context deadline, we create a
context with a timeout. This context is canceled when closing the
response body, and also if a read from the response body returns
an error (including io.EOF).
Cancelling the context in Response.Body.Read interferes with the
HTTP/2 client cleaning up after a request is completed, and is
unnecessary: The user should always close the body, the impact
from not canceling the context is minor (the context timer leaks
until it fires).
Fixes#49366.
Change-Id: Ieaed866116916261d9079f71d8fea7a7b303b8fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/361919
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The test had been setting an arbitrary 200ms timeout to allow the
server's handler to set up before timing out. That is not only
potentially flaky on slow machines, but also typically much longer
than necessary. Replace the hard-coded timeout with a much shorter
initial timeout, and use exponential backoff to lengthen it if needed.
This allows the test to be run about 20x faster in the typical case,
which may make it easier to reproduce rare failure modes by running
with a higher -count flag.
For #43120
Change-Id: I1e0d0ec99d5a107fff56e3bcc7174d686ec582d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/361275
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Many uses of Index/IndexByte/IndexRune/Split/SplitN
can be written more clearly using the new Cut functions.
Do that. Also rewrite to other functions if that's clearer.
For #46336.
Change-Id: I68d024716ace41a57a8bf74455c62279bde0f448
Reviewed-on: https://go-review.googlesource.com/c/go/+/351711
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fix a hang that occurs when making a request and all of the following apply:
* The request method is one of GET, HEAD, DELETE, OPTIONS, PROPFIND, or SEARCH.
* The Request.Body is non-nil.
* The content length is not set, or is set to -1.
* Transfer-Encoding: chunked is not set.
* The request body does not respond to a read within 200ms.
In this case, we give up on probing for a zero-length body and send the
request while the probe completes in the background. Fix a bug in the
io.Reader wrapping the in-flight probe: It should return io.EOF after
the probe completes, but does not.
Fixes#47568.
Change-Id: I7f9188c96e1210055df68424081af927006e4816
Reviewed-on: https://go-review.googlesource.com/c/go/+/340256
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ingo Oeser <nightlyone@googlemail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This follows the spelling choices that the Go project has made for English words.
https://github.com/golang/go/wiki/Spelling
Change-Id: Ie7c586d2cf23020cb492cfff58c0831d2d8d3a78
GitHub-Last-Rev: e16a32cd22
GitHub-Pull-Request: golang/go#45442
Reviewed-on: https://go-review.googlesource.com/c/go/+/308291
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Change-Id: Iff5074713e1a4484c04b8628fb2611b6d4e154c7
GitHub-Last-Rev: bb0861bbbe
GitHub-Pull-Request: golang/go#45294
Reviewed-on: https://go-review.googlesource.com/c/go/+/305772
Reviewed-by: Ben Shi <powerman1st@163.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Go Bot <gobot@golang.org>
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.
For #41190.
Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Makes *Request.write always close the body, so that callers no longer
have to close the body on returned errors, which was the trigger for
double-close behavior.
Fixes#40382
Change-Id: I128f7ec70415f240d82154cfca134b3f692191e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/257819
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Damien Neil <dneil@google.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Fixes the docs to correctly match the implementation
and also adds a test locking-in the behavior to prevent
any accidental future regressions on the docs.
Fixes#33545
Change-Id: I6fdac6048cce8ac99beaa2db0dfc81d0dccc10f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/200798
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Current implementation doesn't always make it obvious what the exact
problem with the URL is, so this makes it clearer by consistently quoting
the invalid URL, as is the norm in other parsing implementations, eg.:
strconv.Atoi(" 123") returns an error: parsing " 123": invalid syntax
Updates #29261
Change-Id: Icc6bff8b4a4584677c0f769992823e6e1e0d397d
GitHub-Last-Rev: 648b9d93fe
GitHub-Pull-Request: golang/go#29384
Reviewed-on: https://go-review.googlesource.com/c/go/+/185117
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
One of these tests creates a bunch of connections concurrently, then
discovers it doesn't need them all, which then makes the server log
that the client went away midway through the TLS handshake. Perhaps
the server should recognize that as a case not worthy of logging
about, but this is a safer way to eliminate the stderr spam during go
test for now.
The other test's client gives up on its connection and closes it,
similarly confusing the server.
Change-Id: I49ce442c9a63fc437e58ca79f044aa76e8c317b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/179179
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Using password that returns from User.Password() won't work in this case
because password in Userinfo already unescaped. The solution is uses
User.String() to escape password back again and then stringify it to error.
Fixes#31808
Change-Id: I723aafd5a57a5b69f2dd7d3a21b82ebbd4174451
Reviewed-on: https://go-review.googlesource.com/c/go/+/175018
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If the client sends a request with a custom Host header and receives
a relative redirect in response, the second request should use the
same Host header as the first request. However, if the response is
an abolute redirect, the Host header should not be preserved. See
further discussion on the issue tracker.
Fixes#22233
Change-Id: I8796e2fbc1c89b3445e651f739d5d0c82e727c14
Reviewed-on: https://go-review.googlesource.com/70792
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: Ic3f7f575d3640706adb7d64545ed8027add6c58f
Reviewed-on: https://go-review.googlesource.com/61350
Run-TryBot: Tom Bergan <tombergan@google.com>
Reviewed-by: Tom Bergan <tombergan@google.com>
This is another attempt at the change attempted in
https://golang.org/cl/27117 and rolled back in https://golang.org/cl/34134
The difference between this and the previous attempt is that this version only
retries if the new field GetBody is set on the Request.
Additionally, this allows retries of requests with idempotent methods even if
they have bodies, as long as GetBody is defined.
This also fixes an existing bug where readLoop could make a redundant call to
setReqCanceler for DELETE/POST/PUT/etc requests with no body with zero bytes
written.
This clarifies the existing TestRetryIdempotentRequestsOnError test (and changes
it into a test with 4 subtests). When that test was written, it was in fact
testing "retry idempotent requests" logic, but the logic had changed since then,
and it was actually testing "retry requests with no body when no bytes have been
written". (You can confirm this by changing the existing test from a GET to a
DELETE; it passes without the changes in this CL.) We now test for the no-Body
and GetBody cases for both idempotent and nothing-written-non-idempotent
requests.
Fixes#18241Fixes#17844
Change-Id: I69a48691796f6dc08c31f7aa7887b7dfd67e278a
Reviewed-on: https://go-review.googlesource.com/42142
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
After merging https://go-review.googlesource.com/c/34639/,
it was pointed out to me that a lot of tests under net/http
could use the new functionality to simplify and unify testing.
Using the httptest.Server provided Client removes the need to
call CloseIdleConnections() on all Transports created, as it
is automatically called on the Transport associated with the
client when Server.Close() is called.
Change the transport used by the non-TLS
httptest.Server to a new *http.Transport rather than using
http.DefaultTransport implicitly. The TLS version already
used its own *http.Transport. This change is to prevent
concurrency problems with using DefaultTransport implicitly
across several httptest.Server's.
Add tests to ensure the httptest.Server.Client().Transport
RoundTripper interface is implemented by a *http.Transport,
as is now assumed across large parts of net/http tests.
Change-Id: I9f9d15f59d72893deead5678d314388718c91821
Reviewed-on: https://go-review.googlesource.com/37771
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If I put a 10 millisecond sleep at testHookWaitResLoop, before the big
select in (*persistConn).roundTrip, two flakes immediately started
happening, TestTransportBodyReadError (#19231) and
TestTransportPersistConnReadLoopEOF.
The problem was that there are many ways for a RoundTrip call to fail
(errors reading from Request.Body while writing the response, errors
writing the response, errors reading the response due to server
closes, errors due to servers sending malformed responses,
cancelations, timeouts, etc.), and many of those failures then tear
down the TCP connection, causing more failures, since there are always
at least three goroutines involved (reading, writing, RoundTripping).
Because the errors were communicated over buffered channels to a giant
select, the error returned to the caller was a function of which
random select case was called, which was why a 10ms delay before the
select brought out so many bugs. (several fixed in my previous CLs the past
few days).
Instead, track the error explicitly in the transportRequest, guarded
by a mutex.
In addition, this CL now:
* differentiates between the two ways writing a request can fail: the
io.Copy reading from the Request.Body or the io.Copy writing to the
network. A new io.Reader type notes read errors from the
Request.Body. The read-from-body vs write-to-network errors are now
prioritized differently.
* unifies the two mapRoundTripErrorFromXXX methods into one
mapRoundTripError method since their logic is now the same.
* adds a (*Request).WithT(*testing.T) method in export_test.go, usable
by tests, to call t.Logf at points during RoundTrip. This is disabled
behind a constant except when debugging.
* documents and deflakes TestClientRedirectContext
I've tested this CL with high -count values, with/without -race,
with/without delays before the select, etc. So far it seems robust.
Fixes#19231 (TestTransportBodyReadError flake)
Updates #14203 (source of errors unclear; they're now tracked more)
Updates #15935 (document Transport errors more; at least understood more now)
Change-Id: I3cccc3607f369724b5344763e35ad2b7ea415738
Reviewed-on: https://go-review.googlesource.com/37495
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
In an unrelated CL I found a way to increase the likelihood of latent
flaky tests and found this one.
This is just like yesterday's https://golang.org/cl/37624 and dozens
before it (all remnants from the great net/http test parallelization
of Nov 2016 in https://golang.org/cl/32684).
Change-Id: I3fe61d1645062e5109206ff27d74f573ef6ebb2e
Reviewed-on: https://go-review.googlesource.com/37627
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This was a t.Parallel test but it was using the global DefaultTransport
via the global Get func.
Use a private Transport that won't have its CloseIdleConnections etc
methods called by other tests.
(I hit this flake myself while testing a different change.)
Change-Id: If0665e3e8580ee198f8e5f3079bfaea55f036eca
Reviewed-on: https://go-review.googlesource.com/37624
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
The presence of Request.GetBody being set on a request was causing all
redirected requests to have a body, even if the redirect status didn't
warrant one.
This bug came from 307/308 support (https://golang.org/cl/29852) which
removed the line that set req.Body to nil after POST/PUT redirects.
Change-Id: I2a4dd5320f810ae25cfd8ea8ca7c9700e5dbd369
Reviewed-on: https://go-review.googlesource.com/35633
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
In Go1.7, a 301, 302, or 303 redirect on a HEAD method, would still
cause the following redirects to still use a HEAD.
In CL/29852 this behavior was changed such that those codes always
caused a redirect with the GET method. Fix this such that both
GET and HEAD will preserve the method.
Fixes#18570
Change-Id: I4bfe69872a30799419e3fad9178f907fe439b449
Reviewed-on: https://go-review.googlesource.com/34981
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This rolls back https://golang.org/cl/27117 partly, softening it so it
only retries POST/PUT/DELETE etc requests where there's no Body (nil
or NoBody). This is a little useless, since most idempotent requests
have a body (except maybe DELETE), but it's late in the Go 1.8 release
cycle and I want to do the proper fix.
The proper fix will look like what we did for http2 and only retrying
the request if Request.GetBody is defined, and then creating a new request
for the next attempt. See https://golang.org/cl/33971 for the http2 fix.
Updates #15723Fixes#18239
Updates #18241
Change-Id: I6ebaa1fd9b19b5ccb23c8d9e7b3b236e71cf57f3
Reviewed-on: https://go-review.googlesource.com/34134
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tom Bergan <tombergan@google.com>
Neal Patel
Damien Neil
nlwkobe30
Filippo Valsorda
Cherry Mui
kayos@tcp.direct
Roland Shoemaker
Bryan C. Mills
Oleksandr Redko
Paschalis Tsilias
Gustavo Falco
Russ Cox
Naman Gera
徐志伟
Ross Light
Emmanuel T Odeke
Brad Fitzpatrick
Stefan Baebler
Thanabodee Charoenpiriyakij
Gregory Man
Tom Bergan
Guilherme Rezende
David Glasser
Johan Brandhorst
Joe Tsai