Commit Graph

80 Commits

Author SHA1 Message Date
Damien Neil 960654be0c net/http/httputil: avoid ReverseProxy data race on 1xx response and error
ReverseProxy uses a httptrace.ClientTrace.Got1xxResponse trace hook
to capture 1xx response headers for proxying. This hook can be called
asynchrnously after RoundTrip returns. (This should only happen when
RoundTrip has failed for some reason.) Add synchronization so we don't
attempt to modifying the ResponseWriter headers map from the hook
after another goroutine has begun making use of it.

Fixes #65123

Change-Id: I8b7ecb1a140f7ba7e37b9d27b8a20bca41a118b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/567216
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
2024-02-27 23:37:42 +00:00
cui fliter 1d45a7ef56 net: add available godoc link
Change-Id: Ib7c4baf0247c421954aedabfbb6a6af8a08a8936
Reviewed-on: https://go-review.googlesource.com/c/go/+/540021
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-10 03:29:50 +00:00
Jes Cok 3bc28402fa all: use built-in clear to clear maps
Change-Id: I7f4ac72fe3230d8b7486fab0c925015cefcbe355
GitHub-Last-Rev: 54455839b6
GitHub-Pull-Request: golang/go#61544
Reviewed-on: https://go-review.googlesource.com/c/go/+/512376
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-07-27 22:07:13 +00:00
Shang Ding 2449bbb5e6 net/http/httputil: use response controller in reverse proxy
Previously, the reverse proxy is unable to detect
the support for hijack or flush if those things
are residing in the response writer in a wrapped
manner.

The reverse proxy now makes use of the new http
response controller as the means to discover
the underlying flusher and hijacker associated
with the response writer, allowing wrapped flusher
and hijacker become discoverable.

Change-Id: I53acbb12315c3897be068e8c00598ef42fc74649
Reviewed-on: https://go-review.googlesource.com/c/go/+/468755
Run-TryBot: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-17 15:23:38 +00:00
Ethan Lowman 32284526c8 net/http/httputil: fix syntax in ReverseProxy example comment
Change-Id: I7e71626246af94047fbd1abb9bb77f2cd9b281fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/465195
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
2023-02-04 04:11:32 +00:00
Damien Neil 458241f981 net/http/httputil: don't add X-Forwarded-{Host,Proto} after invoking Director funcs
This reverts CL 407414.

When forwarding an inbound request that contains an existing
X-Forwarded-Host or X-Forwarded-Proto header, a proxy might want
to preserve the header from the inbound request, replace it with
its own header, or not include any header at all.

CL 407414 replaces inbound X-Forwarded-{Host,Proto} headers by default,
and allows a Director func to disable sending these headers at all.
However, the Director hook API isn't sufficiently flexible to permit the
previous behavior of preserving inbound values unchanged.

The new Rewrite API does have this flexibility; users of Rewrite can
easily pick the exact behavior they want.

Revert the change to ReverseProxy when using a Director func.
Users who want a convenient way to set X-Forwarded-* headers to
reasonable values can migrate to Rewrite at their convenience,
and users depending on the current behavior will be unaffected.

For #50465.
Fixes #57132.

Change-Id: Ic42449c1bb525d6c9920bf721efbc519697f4f20
Reviewed-on: https://go-review.googlesource.com/c/go/+/457595
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-12-21 18:56:32 +00:00
Damien Neil 69ca0a859c Revert "net/url, net/http/httputil: accept invalid percent encodings"
This reverts CL 450375.

Reason for revert: This change causes test failures (and possibly other
problems) for users depending on the existing validation behavior.
Rolling back the change for now to give us more time to consider its
impact. This landed late in the cycle and isn't urgent; it can wait
for 1.21 if we do want to make the change.

Fixes #56884
For #56732

Change-Id: I082023c67f1bbb933a617453ab92b67abba876ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/452795
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-11-22 18:25:01 +00:00
Ian Lance Taylor e6ebbefaf8 net/url, net/http/httputil: accept invalid percent encodings
Per https://url.spec.whatwg.org/#percent-encoded-bytes an invalid
percent encoding should be handled as ordinary text.

Fixes #56732

Change-Id: Ib0259dfd704922905289eebaacbf722e28f6d636
Reviewed-on: https://go-review.googlesource.com/c/go/+/450375
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-15 00:02:58 +00:00
Damien Neil 7c84234142 net/http/httputil: avoid query parameter smuggling
Query parameter smuggling occurs when a proxy's interpretation
of query parameters differs from that of a downstream server.
Change ReverseProxy to avoid forwarding ignored query parameters.

Remove unparsable query parameters from the outbound request

   * if req.Form != nil after calling ReverseProxy.Director; and
   * before calling ReverseProxy.Rewrite.

This change preserves the existing behavior of forwarding the
raw query untouched if a Director hook does not parse the query
by calling Request.ParseForm (possibly indirectly).

Fixes #54663
Fixes CVE-2022-2880

Change-Id: If1621f6b0e73a49d79059dae9e6b256e0ff18ca9
Reviewed-on: https://go-review.googlesource.com/c/go/+/432976
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
2022-09-23 21:06:17 +00:00
Damien Neil a84f46a28a net/http/httputil: remove duplicated code block
Remove a harmless but redundant section of code introduced
in CL 407214.

Change-Id: Id6522e6ff13a283d726b3b97dfc72f101884f733
Reviewed-on: https://go-review.googlesource.com/c/go/+/431395
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tatiana Bradley <tatiana@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
2022-09-16 17:59:31 +00:00
Kévin Dunglas 972870da11 net/http/httputil: forward 1xx responses in ReverseProxy
Support for 1xx responses has recently been merged in
net/http (CL 269997).

As discussed in this CL
(https://go-review.googlesource.com/c/go/+/269997/comments/1ff70bef_c25a829a),
support for forwarding 1xx responses in ReverseProxy has been extracted
in this separate patch.

According to RFC 7231, "a proxy MUST forward 1xx responses unless the
proxy itself requested the generation of the 1xx response".
Consequently, all received 1xx responses are automatically forwarded as long as the
underlying transport supports ClientTrace.Got1xxResponse.

Fixes #26088
Fixes #51914

Change-Id: I3a35ea023b798bfe56b7fb8696d5a49695229cfd
GitHub-Last-Rev: dab8a461fb
GitHub-Pull-Request: golang/go#53164
Reviewed-on: https://go-review.googlesource.com/c/go/+/409536
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Rhys Hiltner <rhys@justin.tv>
Run-TryBot: hopehook <hopehook@golangcn.org>
2022-09-14 23:59:32 +00:00
Abirdcfly d8f90ce0f8 all: remove duplicate "the" words in comments
Following CL 424454, using command

   rg --multiline " the\s{1,}the " *
   rg --multiline " the\s{1,}//\s{1,}the " *

all the words "the" that are repeated in comments are found.

Change-Id: I60b769b98f04c927b4c228e10f37faf190964069
Reviewed-on: https://go-review.googlesource.com/c/go/+/423836
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2022-08-18 13:54:47 +00:00
Damien Neil a55793835f net/http/httputil: add ReverseProxy.Rewrite
Add a new Rewrite hook to ReverseProxy, superseding the Director hook.

Director does not distinguish between the inbound and outbound request,
which makes it possible for headers added by Director to be inadvertently
removed before forwarding if they are listed in the inbound request's
Connection header. Rewrite accepts a value containing the inbound
and outbound requests, with hop-by-hop headers already removed from
the outbound request, avoiding this problem.

ReverseProxy's appends the client IP to the inbound X-Forwarded-For
header by default. Users must manually delete untrusted X-Forwarded-For
values. When used with a Rewrite hook, ReverseProxy now strips
X-Forwarded-* headers by default.

NewSingleHostReverseProxy creates a proxy that does not rewrite the
Host header of inbound requests. Changing this behavior is
cumbersome, as it requires wrapping the Director function created
by NewSingleHostReverseProxy. The Rewrite hook's ProxyRequest
parameter provides a SetURL method that provides equivalent
functionality to NewSingleHostReverseProxy, rewrites the Host
header by default, and can be more easily extended with additional
customizations.

Fixes #28168.
Fixes #50580.
Fixes #53002.

Change-Id: Ib84e2fdd1d52c610e3887af66f517d4a74e594d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/407214
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
2022-08-16 20:01:36 +00:00
Damien Neil f001df540b net/http/httputil: don't insert default User-Agent header in proxied requests
When ReverseProxy forwards a request with no User-Agent header, leave
the header in the forwarded request blank rather than inserting the
default Go HTTP clent User-Agent.

We already did this for NewSingleHostReverseProxy; generalize it to
every ReverseProxy.

Change-Id: Id81a230cb8d384acdfae190b78a4265d80720388
Reviewed-on: https://go-review.googlesource.com/c/go/+/407375
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-08-12 16:38:52 +00:00
Damien Neil 1513e57b70 net/http/httputil: add X-Forwarded-{Host,Proto} headers in ReverseProxy
X-Forwarded-Host contains the original request's host.

X-Forwarded-Proto contains "http" or "https", depending on whether the
original request was made on a TLS-secured connection.

Setting either header to nil in Director disables adding the header,
same as for X-Forwarded-For.

Fixes #50465.

Change-Id: If8ed1f48d83f8ea0389c53519bc7994cb53891db
Reviewed-on: https://go-review.googlesource.com/c/go/+/407414
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2022-08-12 16:38:28 +00:00
John Bampton b2116f748a all: fix spelling
Change-Id: Iee18987c495d1d4bde9da888d454eea8079d3ebc
GitHub-Last-Rev: ff5e01599d
GitHub-Pull-Request: golang/go#52949
Reviewed-on: https://go-review.googlesource.com/c/go/+/406915
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-05-17 21:46:33 +00:00
Ryan Leung dd8d425fed all: fix some lint issues
Make some code more simple.

Change-Id: I801adf0dba5f6c515681345c732dbb907f945419
GitHub-Last-Rev: a505146bac
GitHub-Pull-Request: golang/go#49626
Reviewed-on: https://go-review.googlesource.com/c/go/+/364634
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2022-05-08 17:27:54 +00:00
Damien Neil a10a209b23 net/http/httputil: ignore CloseNotify when a non-background context is present
If the http.Request passed to ReverseProxy.ServeHTTP has a context
with a non-nil Done channel, don't watch the ResponseWriter's
CloseNotify channel.

Avoids starting an extra background goroutine in the common case.

Change-Id: I1328f3e02d3025caa0f446a2f20dfc14ef604c64
Reviewed-on: https://go-review.googlesource.com/c/go/+/376415
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Damien Neil <dneil@google.com>
2022-03-24 21:51:49 +00:00
Russ Cox 2580d0e08d all: gofmt -w -r 'interface{} -> any' src
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.

Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.

A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.

Fixes #49884.

Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13 18:45:54 +00:00
James Harris 4a3daeee63 net/http/httputil: allow MIME parameters when detecting SSE in ReverseProxy
This change allows httputil.ReverseProxy to detect SSE (server-sent events)
content when the response's Content-Type header includes MIME parameters,
such as "text/event-stream;charset=utf-8".

Prior to this change the value of the Content-Type header was compared
directly to the literal "text/event-stream". This caused a false-negative
which failed to set the FlushInterval correctly when MIME parameters were
present.

Change-Id: If8bb43efb78787b6519d7fe7599ca018a0da0023
GitHub-Last-Rev: 224518c5eb
GitHub-Pull-Request: golang/go#48427
Reviewed-on: https://go-review.googlesource.com/c/go/+/350509
Trust: Alexander Rakoczy <alex@golang.org>
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Alexander Rakoczy <alex@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2021-10-13 17:16:12 +00:00
Damien Neil b7a85e0003 net/http/httputil: close incoming ReverseProxy request body
Reading from an incoming request body after the request handler aborts
with a panic can cause a panic, becuse http.Server does not (contrary
to its documentation) close the request body in this case.

Always close the incoming request body in ReverseProxy.ServeHTTP to
ensure that any in-flight outgoing requests using the body do not
read from it.

Updates #46866
Fixes CVE-2021-36221

Change-Id: I310df269200ad8732c5d9f1a2b00de68725831df
Reviewed-on: https://go-review.googlesource.com/c/go/+/333191
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-07-30 14:01:30 +00:00
Filippo Valsorda 950fa11c4c net/http/httputil: always remove hop-by-hop headers
Previously, we'd fail to remove the Connection header from a request
like this:

    Connection:
    Connection: x-header

Fixes #46313
Fixes CVE-2021-33197

Change-Id: Ie3009e926ceecfa86dfa6bcc6fe14ff01086be7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/321929
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-27 15:00:58 +00:00
Roberto Clapis 5c489514bc net/http: switch HTTP1 to ASCII equivalents of string functions
The current implementation uses UTF-aware functions
like strings.EqualFold and strings.ToLower.

This could, in some cases, cause http smuggling.

Change-Id: I0e76a993470a1e1b1b472f4b2859ea0a2b22ada0
Reviewed-on: https://go-review.googlesource.com/c/go/+/308009
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Roberto Clapis <roberto@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-05-10 23:42:56 +00:00
Naman Gera 52bf14e0e8 all: fix spellings
This follows the spelling choices that the Go project has made for English words.
https://github.com/golang/go/wiki/Spelling

Change-Id: Ie7c586d2cf23020cb492cfff58c0831d2d8d3a78
GitHub-Last-Rev: e16a32cd22
GitHub-Pull-Request: golang/go#45442
Reviewed-on: https://go-review.googlesource.com/c/go/+/308291
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-04-10 01:46:41 +00:00
Aofei Sheng 9c017ff30d net/http/httputil: copy response header back to http.ResponseWriter in ReverseProxy.handleUpgradeResponse
Fixes: #41634

Change-Id: Ib78cc37a4d2ca0753d567eafb616238e4103484e
Reviewed-on: https://go-review.googlesource.com/c/go/+/257777
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-19 17:17:28 +00:00
Ignacio Hagopian ca3c0df1f8 net/http/httputil: flush ReverseProxy immediately if Content-Length is -1
Finish up a prior TODO by making ReverseProxy flush immediately
if Content-Length is -1, which is a case that can occur if for
example we have a streamed response, or chunked encoding, or when
the body's length wasn't known.

Fixes #41642

Change-Id: I30babaaf3e14837b99e3ecdc562a0a0e50c579bf
GitHub-Last-Rev: efc019a9fe
GitHub-Pull-Request: golang/go#41858
Reviewed-on: https://go-review.googlesource.com/c/go/+/260637
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-12 22:23:38 +00:00
Filippo Valsorda 21898524f6 net/http: use ASCII space trimming throughout
Security hardening against HTTP request smuggling. Thank you to ZeddYu
for reporting this issue.

Change-Id: I98bd9f8ffe58360fc3bca9dc5d9a106773e55373
Reviewed-on: https://go-review.googlesource.com/c/go/+/231419
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-05-06 16:25:52 +00:00
Daniel Kumor 2d323f900d net/http/httputil: handle escaped paths in SingleHostReverseProxy
When forwarding a request, a SingleHostReverseProxy appends the
request's path to the target URL's path. However, if certain path
elements are encoded, (such as %2F for slash in either the request or
target path), simply joining the URL.Path elements is not sufficient,
since the field holds the decoded path.

Since 87a605, the RawPath field was added which holds a decoding
hint for the URL. When joining URL paths, this decoding hint needs
to be taken into consideration.

As an example, if the target URL.Path is /a/b, and URL.RawPath
is /a%2Fb, joining the path with /c should result in /a/b/c
in URL.Path, and /a%2Fb/c in RawPath.

The added joinURLPath function combines the two URL's Paths,
while taking into account escaping, and replaces the previously used
singleJoiningSlash in NewSingleHostReverseProxy.

Fixes #35908

Change-Id: I45886aee548431fe4031883ab1629a41e35f1727
GitHub-Last-Rev: 7be6b8d421
GitHub-Pull-Request: golang/go#36378
Reviewed-on: https://go-review.googlesource.com/c/go/+/213257
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-05-02 20:20:16 +00:00
Brad Fitzpatrick ecdbffd4ec net/http/httputil: don't append to X-Forwarded-For in ReverseProxy when nil
Fixes #38079

Change-Id: Iac02d7f9574061bb26d1d9a41bb6ee6cc38934e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/230937
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-04-30 14:41:10 +00:00
Pierre Carru 8bcf2834af net/http/httputil: make Switching Protocol requests (e.g. Websockets) cancelable
Ensures that a canceled client request for Switching Protocols
(e.g. h2c, Websockets) will cause the underlying connection to
be terminated.

Adds a goroutine in handleUpgradeResponse in order to select on
the incoming client request's context and appropriately cancel it.

Fixes #35559

Change-Id: I1238e18fd4cce457f034f78d9cdce0e7f93b8bf6
GitHub-Last-Rev: 3629c78493
GitHub-Pull-Request: golang/go#38021
Reviewed-on: https://go-review.googlesource.com/c/go/+/224897
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-04-26 09:26:10 +00:00
Kévin Dunglas 88ed322afc net/http/httputil: add docs about X-Forwarded-For in ReverseProxy
ReverseProxy automatically sets the X-Forwarded-For header, if the request
already contains a X-Forwarded-For header, the value of the client IP is
appended to the existing header value.
This behavior isn't documented anywhere, and can lead to IP spoofing
security issues is the client is untrusted (the most common situation).
This PR documents this behavior.

For future versions, I proposed #36678 that implements a more secure
default behavior and adds support for other forwarded headers.

Change-Id: Ief14f5063caebfccb87714f54cffa927c714e5fd
GitHub-Last-Rev: fd0bd29a18
GitHub-Pull-Request: golang/go#36672
Reviewed-on: https://go-review.googlesource.com/c/go/+/215617
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-21 21:51:46 +00:00
Joe Tsai f93234ad62 net/http/httputil: fix regression in ReverseProxy.ServeHTTP
In Go1.12 and below, the logic in ReverseProxy.ServeHTTP would always
allocate request.Header even if it were not present in the incoming request.
CL 174324 added http.Request.Clone and re-factors ReverseProxy.ServeHTTP
to use the new Clone method. However, the new Clone logic is not equivalent
to the former logic. We preserve former semantics by explicitly allocating
the Header map if nil.

Fixes #33142

Change-Id: I356f94a915dd9779584ce3fe31e56e5474b9ad37
Reviewed-on: https://go-review.googlesource.com/c/go/+/186437
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-07-17 21:52:45 +00:00
Dmitri Shuralyov 3f7c560829 net/http/httputil: remove os.Stderr claim in ReverseProxy.ErrorLog docs
The motivation for doing so is to avoid making inaccurate claims.
Logging may not go to os.Stderr if anyone overrides the log package's
default output via https://godoc.org/log#SetOutput. Saying that
the standard logger is used should be sufficient to explain the
behavior, and users can infer that os.Stderr is used by default,
unless it's changed.

This change is the same as what was applied to http.Server.ErrorLog
documentation in CL 53950.

Change-Id: I32873fc548ceee573f8616b4d49b8a8b98881803
Reviewed-on: https://go-review.googlesource.com/c/go/+/176817
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-17 17:33:16 +00:00
Jonathon Lacher a5cea062b3 net/http/httputil: remove all fields in Connection header
In the reverseproxy, replace use (Header).Get, which returns only one value
of a multiple value header, with using the Header map directly. Also fixes
corresponding tests which hid the bug, and adds more tests.

Fixes #30303

Change-Id: Ic9094b5983043460697748759f6dfd95fc111db7
GitHub-Last-Rev: b41038143f
GitHub-Pull-Request: golang/go#30687
Reviewed-on: https://go-review.googlesource.com/c/go/+/166298
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-14 15:37:14 +00:00
Brad Fitzpatrick f5c43b9194 net/http: add func NewRequestWithContext, Request.Clone
Fixes #23544

Change-Id: Iaa31d76c4cda8ce22412d73c9025fc57e4fb1967
Reviewed-on: https://go-review.googlesource.com/c/go/+/174324
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2019-05-03 15:20:15 +00:00
Brad Fitzpatrick 13d0af4e70 net/http: export Header.Clone, reduce its allocations, use it everywhere
Fixes #29915

Change-Id: I6e6edf4f9a0e062211f74d120ae1a242bce1b274
Reviewed-on: https://go-review.googlesource.com/c/go/+/173658
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ingo Oeser <nightlyone@googlemail.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2019-04-24 18:43:11 +00:00
Jordan Liggitt 2cc347382f net/http/httputil: make ReverseProxy flush headers on FlushInterval
A regression was introduced in CL 137335 (5440bfc) that caused FlushInterval
to not be honored until the first Write() call was encountered. This change
starts the flush timer as part of setting up the maxLatencyWriter.

Fixes #31125
Fixes #31126

Change-Id: I75325bd926652922219bd1457b2b00ac6d0d41b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/170066
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-29 15:56:57 +00:00
Bryan C. Mills c5cf662407 all: move internal/x to vendor/golang.org/x and revendor using 'go mod vendor'
This also updates the vendored-in versions of several packages: 'go
mod vendor' selects a consistent version of each module, but we had
previously vendored an ad-hoc selection of packages.

Notably, x/crypto/hkdf was previously vendored in at a much newer
commit than the rest of x/crypto. Bringing the rest of x/crypto up to
that commit introduced an import of golang.org/x/sys/cpu, which broke
the js/wasm build, requiring an upgrade of x/sys to pick up CL 165749.

Updates #30228
Updates #30241
Updates #25822

Change-Id: I5b3dbc232b7e6a048a158cbd8d36137af1efb711
Reviewed-on: https://go-review.googlesource.com/c/go/+/164623
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2019-03-11 20:28:54 +00:00
Brad Fitzpatrick a14ed2a82a net/http/httputil: run the ReverseProxy.ModifyResponse hook for upgrades
Fixes #29627

Change-Id: I08a5b45151a11b5a4f3b5a2d984c0322cf904697
Reviewed-on: https://go-review.googlesource.com/c/157098
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-09 15:51:59 +00:00
Tobias Klauser 99ea99ec4c net/http/httputil: fix typo in ReverseProxy godoc
Change-Id: Iea33fe64403ca2e6f87a4e070af5e97d96506e41
Reviewed-on: https://go-review.googlesource.com/c/157118
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-01-09 14:49:46 +00:00
Julien Salleyron 2175177497 net/http/httputil: fix missing previous headers in response when switching protocol in ReverseProxy
When using switching protocol, previous headers set before the reverse proxy are lost.

Fixes #29407

Change-Id: Ia2b9784022d9bccef8625519ccbabbe8a276dfc0
GitHub-Last-Rev: 79bb493dcb
GitHub-Pull-Request: golang/go#29408
Reviewed-on: https://go-review.googlesource.com/c/155741
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-01-02 17:00:30 +00:00
SALLEYRON Julien 48399cae9f net/http/httputil: fix unannounced trailers when body is empty
Fix unannounced trailers when body is empty and without announced trailers.

Fixes #29031

Change-Id: If49951a42fe56d4be4436a999627db4c2678659d
GitHub-Last-Rev: 3469adc8f5
GitHub-Pull-Request: golang/go#29032
Reviewed-on: https://go-review.googlesource.com/c/151898
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-03 21:45:17 +00:00
Bryan C. Mills 2012227b01 vendor/golang_org/x: move to internal/x
Packages in vendor/ directories have a "vendor/" path prefix in GOPATH
mode, but intentionally do not in module mode. Since the import path
is embedded in the compiled output, changing that path invalidates
cache entries and causes cmd/go to try to rebuild (and reinstall) the
vendored libraries, which will fail if the directory containing those
libraries is read-only.

If I understood correctly, this is the approach Russ suggested as an
alternative to https://golang.org/cl/136138.

Fixes #27285
Fixes #26988

Change-Id: I8a2507fa892b84cde0a803aaa79e460723da572b
Reviewed-on: https://go-review.googlesource.com/c/147443
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2018-11-29 15:42:16 +00:00
Brad Fitzpatrick ee55f0856a net/http/httputil: make ReverseProxy automatically proxy WebSocket requests
Fixes #26937

Change-Id: I6cdc1bad4cf476cd2ea1462b53444eccd8841e14
Reviewed-on: https://go-review.googlesource.com/c/146437
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-11-13 01:42:47 +00:00
Brad Fitzpatrick c870d56f98 net/http/httputil: fix race/crash in previous ReverseProxy change
The previous ReverseProxy change, CL 137335, introduced a bug which could cause
a race and/or a crash.

This reliably crashed before:

$ go test -short -race -v -run=TestReverseProxyFlushInterval -count=20 net/http/httputil

The problem was a goroutine was running http.ResponseWriter.Flush
after the http.Handler's ServeHTTP completed. There was code to
prevent that (a deferred stop call) but the stop call didn't consider
the case where time.AfterFunc had already fired off a new goroutine
but that goroutine hadn't yet scheduled.

Change-Id: I06357908465a3b953efc33e63c70dec19a501adf
Reviewed-on: https://go-review.googlesource.com/c/140977
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-10 02:08:36 +00:00
Brad Fitzpatrick 5440bfc2ea net/http/httputil: rewrite flushing code, disable on Server-Sent Events
* Rewrite the flushing code to not use a persistent goroutine, which
  also simplifies testing.
* Define the meaning of a negative flush interval. Its meaning doesn't
  change, but now it's locked in, and then we can use it to optimize
  the performance of the non-buffered case to avoid use of an AfterFunc.
* Support (internal-only) special casing of FlushInterval values per
  request/response.
* For now, treat Server-Sent Event responses as unbuffered. (or rather,
  immediately flushed from the buffer per-write)

Fixes #27816

Change-Id: Ie0f975c997daa3db539504137c741a96d7022665
Reviewed-on: https://go-review.googlesource.com/c/137335
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-10-09 15:26:06 +00:00
Julien Salleyron 5201b1ad22 http/http/httputil: add ReverseProxy.ErrorHandler
This permits specifying an ErrorHandler to customize the RoundTrip
error handling if the backend fails to return a response.

Fixes #22700
Fixes #21255

Change-Id: I8879f0956e2472a07f584660afa10105ef23bf11
Reviewed-on: https://go-review.googlesource.com/77410
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-07-13 16:45:34 +00:00
Brad Fitzpatrick eab57b27f5 net/http/httputil: don't panic in ReverseProxy unless running under a Server
Prior to the fix to #23643, the ReverseProxy didn't panic with
ErrAbortHandler when the copy to a client failed.

During Go 1.11 beta testing, we found plenty of code using
ReverseProxy in tests that were unprepared for a panic.

Change the behavior to only panic when running under the http.Server
that'll handle the panic.

Updates #23643

Change-Id: Ic1fa8405fd54c858ce8c797cec79d006833a9f7d
Reviewed-on: https://go-review.googlesource.com/122819
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-07-09 23:40:02 +00:00
Tim Cooper 161874da2a all: update comment URLs from HTTP to HTTPS, where possible
Each URL was manually verified to ensure it did not serve up incorrect
content.

Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2018-06-01 21:52:00 +00:00
Brad Fitzpatrick 9e56156ade net/http/httputil: pass through any "TE: trailers" header to backend
Fixes #21096

Change-Id: I2a4688a79bdaa25b4e8ef38e3390d93d3d0bce04
Reviewed-on: https://go-review.googlesource.com/115135
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-05-29 22:33:14 +00:00