Commit Graph

2768 Commits

Author SHA1 Message Date
Jes Cok 59ecebf47f net/http: clarify the precedence order for Request.FormValue
Fixes #64575

Change-Id: I0eaec642a9dc8ae3b273a6d41131cc7cb8332947
GitHub-Last-Rev: 17aa5170cb
GitHub-Pull-Request: golang/go#64578
Reviewed-on: https://go-review.googlesource.com/c/go/+/547855
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-09 19:42:18 +00:00
Mauri de Souza Meneguzzo 8eaa7935db net: clarify maxListenerBacklog windows implementation
The previous TODO comments were somewhat ambiguous. This aims to
provide a clearer understanding of the behavior on Windows.

Windows does not offer a way to peek at the current backlog length, this
is explicitly stated in the winapi for `listen`.

When set to `syscall.SOMAXCONN`, the OS dynamically adjusts the
backlog to a maximum reasonable value. It goes as far as the dotnet
runtime itself introducing a new version of `listen` that does not accept a
backlog parameter to help eliminate the confusion when comparing the
behavior with UNIXes.

The docs also mention that `SOMAXCONN_HINT(N)` can be used, and that
it clips the final computed value between (200, 65535), which suggests
windows might use a `uint16` to back this number. Either way it does not
matter since windows will adjust this value anyway, so I removed the
wrapping TODO as well.

See https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-listen

Change-Id: I7b2e7cb547467c4bfc572ef0477a58de8c772521
GitHub-Last-Rev: 34e74abffe
GitHub-Pull-Request: golang/go#63549
Reviewed-on: https://go-review.googlesource.com/c/go/+/535475
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-01-08 18:52:56 +00:00
Andy Pan ead47b0ab3 net/http: respond with 400 Bad Request for empty hex number of chunk length
Fixes #64517

Change-Id: I78b8a6a83301deee05c3ff052a6adcd1f965aef2
Reviewed-on: https://go-review.googlesource.com/c/go/+/553835
Auto-Submit: Damien Neil <dneil@google.com>
Commit-Queue: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2024-01-04 20:45:19 +00:00
Bryan C. Mills 1e07c144c3 net/http/cgi: in TestCopyError, check for a Handler.ServeHTTP goroutine instead of a running PID
Previously, the test could fail spuriously if the CGI process's PID
happened to be reused in between checks. That sort of reuse is highly
unlikely on platforms that cycle through the PID space sequentially
(such as Linux), but plausible on platforms that use randomized PIDs
(such as OpenBSD).

Also unskip the test on Windows, since it no longer relies on being
able to send signal 0 to an arbitrary PID.

Also change the expected failure mode of the test to a timeout instead
of a call to t.Fatalf, so that on failure we get a useful goroutine
dump for debugging instead of a non-actionable failure message.

Fixes #57369 (maybe).

Change-Id: Ib7e3fff556450b48cb5e6ea120fdf4d53547479b
Reviewed-on: https://go-review.googlesource.com/c/go/+/554075
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-01-04 20:40:52 +00:00
Jes Cok 7d1b82dbf1 net/http: make Request.Clone create fresh copies for matches and otherValues
This change fixes Request.Clone to correctly work with SetPathValue
by creating fresh copies for matches and otherValues so that
SetPathValue for cloned requests doesn't pollute the original request.

While here, also added a doc for Request.SetPathValue.

Fixes #64911

Change-Id: I2831b38e135935dfaea2b939bb9db554c75b65ef
GitHub-Last-Rev: 1981db1647
GitHub-Pull-Request: golang/go#64913
Reviewed-on: https://go-review.googlesource.com/c/go/+/553375
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Jes Cok <xigua67damn@gmail.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-01-03 17:43:26 +00:00
Russ Cox d73b4322ed net/netip: remove Prefix.Compare for Go 1.22
API questions remain, so we decided to back it out for Go 1.22.
Code still lives in the repo, just unexported.

For #61642.

Change-Id: Iccd91b0da48ae72dec9f660476826a220c7ca4be
Reviewed-on: https://go-review.googlesource.com/c/go/+/549615
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Anderson <dave@natulte.net>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2023-12-14 22:24:40 +00:00
Danil Timerbulatov 527829a7cb all: remove newline characters after return statements
This commit is aimed at improving the readability and consistency
of the code base. Extraneous newline characters were present after
some return statements, creating unnecessary separation in the code.

Fixes #64610

Change-Id: Ic1b05bf11761c4dff22691c2f1c3755f66d341f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/548316
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-12-14 17:22:18 +00:00
Bryan C. Mills f4ff0e2505 net: remove an arbitrary timeout in TestAcceptTimeoutMustReturn
This probably fixes the failure mode seen in
https://build.golang.org/log/e73acfd930cbe82302505cac0041d9883e2360c5.

If not, allowing the test to deadlock and dump goroutines
should produce better debugging information than the existing
"didn't return in an expected time" failure message.

For #58901.

Change-Id: Ie0bb1887a4329d2f6b0e7348a4820af71385494e
Reviewed-on: https://go-review.googlesource.com/c/go/+/548881
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-12-11 23:01:52 +00:00
Michael Pratt e1c0349a7c internal/profile: fully decode proto even if there are no samples
This is a partial revert of CL 483137.

CL 483137 started checking errors in postDecode, which is good. Now we
can catch more malformed pprof protos. However this made
TestEmptyProfile fail, so an early return was added when the profile was
"empty" (no samples).

Unfortunately, this was problematic. Profiles with no samples can still
be valid, but skipping postDecode meant that the resulting Profile was
missing values from the string table. In particular, net/http/pprof
needs to parse empty profiles in order to pass through the sample and
period types to a final output proto. CL 483137 broke this behavior.

internal/profile.Parse is only used in two places: in cmd/compile to
parse PGO pprof profiles, and in net/http/pprof to parse before/after
pprof profiles for delta profiles. In both cases, the input is never
literally empty (0 bytes). Even a pprof proto with no samples still
contains some header fields, such as sample and period type. Upstream
github.com/google/pprof/profile even has an explicit error on 0 byte
input, so `go tool pprof` will not support such an input.

Thus TestEmptyProfile was misleading; this profile doesn't need to
support empty input at all.

Resolve this by removing TestEmptyProfile and replacing it with an
explicit error on empty input, as upstream
github.com/google/pprof/profile has. For non-empty input, always run
postDecode to ensure the string table is processed.

TestConvertCPUProfileEmpty is reverted back to assert the values from
before CL 483137. Note that in this case "Empty" means no samples, not a
0 byte input.

Continue to allow empty files for PGO in order to minimize the chance of
last minute breakage if some users have empty files.

Fixes #64566.

Change-Id: I83a1f0200ae225ac6da0009d4b2431fe215b283f
Reviewed-on: https://go-review.googlesource.com/c/go/+/547996
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-12-07 19:52:28 +00:00
Russ Cox c29444ef39 math/rand, math/rand/v2: use ChaCha8 for global rand
Move ChaCha8 code into internal/chacha8rand and use it to implement
runtime.rand, which is used for the unseeded global source for
both math/rand and math/rand/v2. This also affects the calculation of
the start point for iteration over very very large maps (when the
32-bit fastrand is not big enough).

The benefit is that misuse of the global random number generators
in math/rand and math/rand/v2 in contexts where non-predictable
randomness is important for security reasons is no longer a
security problem, removing a common mistake among programmers
who are unaware of the different kinds of randomness.

The cost is an extra 304 bytes per thread stored in the m struct
plus 2-3ns more per random uint64 due to the more sophisticated
algorithm. Using PCG looks like it would cost about the same,
although I haven't benchmarked that.

Before this, the math/rand and math/rand/v2 global generator
was wyrand (https://github.com/wangyi-fudan/wyhash).
For math/rand, using wyrand instead of the Mitchell/Reeds/Thompson
ALFG was justifiable, since the latter was not any better.
But for math/rand/v2, the global generator really should be
at least as good as one of the well-studied, specific algorithms
provided directly by the package, and it's not.

(Wyrand is still reasonable for scheduling and cache decisions.)

Good randomness does have a cost: about twice wyrand.

Also rationalize the various runtime rand references.

goos: linux
goarch: amd64
pkg: math/rand/v2
cpu: AMD Ryzen 9 7950X 16-Core Processor
                        │ bbb48afeb7.amd64 │           5cf807d1ea.amd64           │
                        │      sec/op      │    sec/op     vs base                │
ChaCha8-32                     1.862n ± 2%    1.861n ± 2%        ~ (p=0.825 n=20)
PCG_DXSM-32                    1.471n ± 1%    1.460n ± 2%        ~ (p=0.153 n=20)
SourceUint64-32                1.636n ± 2%    1.582n ± 1%   -3.30% (p=0.000 n=20)
GlobalInt64-32                 2.087n ± 1%    3.663n ± 1%  +75.54% (p=0.000 n=20)
GlobalInt64Parallel-32        0.1042n ± 1%   0.2026n ± 1%  +94.48% (p=0.000 n=20)
GlobalUint64-32                2.263n ± 2%    3.724n ± 1%  +64.57% (p=0.000 n=20)
GlobalUint64Parallel-32       0.1019n ± 1%   0.1973n ± 1%  +93.67% (p=0.000 n=20)
Int64-32                       1.771n ± 1%    1.774n ± 1%        ~ (p=0.449 n=20)
Uint64-32                      1.863n ± 2%    1.866n ± 1%        ~ (p=0.364 n=20)
GlobalIntN1000-32              3.134n ± 3%    4.730n ± 2%  +50.95% (p=0.000 n=20)
IntN1000-32                    2.489n ± 1%    2.489n ± 1%        ~ (p=0.683 n=20)
Int64N1000-32                  2.521n ± 1%    2.516n ± 1%        ~ (p=0.394 n=20)
Int64N1e8-32                   2.479n ± 1%    2.478n ± 2%        ~ (p=0.743 n=20)
Int64N1e9-32                   2.530n ± 2%    2.514n ± 2%        ~ (p=0.193 n=20)
Int64N2e9-32                   2.501n ± 1%    2.494n ± 1%        ~ (p=0.616 n=20)
Int64N1e18-32                  3.227n ± 1%    3.205n ± 1%        ~ (p=0.101 n=20)
Int64N2e18-32                  3.647n ± 1%    3.599n ± 1%        ~ (p=0.019 n=20)
Int64N4e18-32                  5.135n ± 1%    5.069n ± 2%        ~ (p=0.034 n=20)
Int32N1000-32                  2.657n ± 1%    2.637n ± 1%        ~ (p=0.180 n=20)
Int32N1e8-32                   2.636n ± 1%    2.636n ± 1%        ~ (p=0.763 n=20)
Int32N1e9-32                   2.660n ± 2%    2.638n ± 1%        ~ (p=0.358 n=20)
Int32N2e9-32                   2.662n ± 2%    2.618n ± 2%        ~ (p=0.064 n=20)
Float32-32                     2.272n ± 2%    2.239n ± 2%        ~ (p=0.194 n=20)
Float64-32                     2.272n ± 1%    2.286n ± 2%        ~ (p=0.763 n=20)
ExpFloat64-32                  3.762n ± 1%    3.744n ± 1%        ~ (p=0.171 n=20)
NormFloat64-32                 3.706n ± 1%    3.655n ± 2%        ~ (p=0.066 n=20)
Perm3-32                       32.93n ± 3%    34.62n ± 1%   +5.13% (p=0.000 n=20)
Perm30-32                      202.9n ± 1%    204.0n ± 1%        ~ (p=0.482 n=20)
Perm30ViaShuffle-32            115.0n ± 1%    114.9n ± 1%        ~ (p=0.358 n=20)
ShuffleOverhead-32             112.8n ± 1%    112.7n ± 1%        ~ (p=0.692 n=20)
Concurrent-32                  2.107n ± 0%    3.725n ± 1%  +76.75% (p=0.000 n=20)

goos: darwin
goarch: arm64
pkg: math/rand/v2
                       │ bbb48afeb7.arm64 │           5cf807d1ea.arm64            │
                       │      sec/op      │    sec/op     vs base                 │
ChaCha8-8                     2.480n ± 0%    2.429n ± 0%    -2.04% (p=0.000 n=20)
PCG_DXSM-8                    2.531n ± 0%    2.530n ± 0%         ~ (p=0.877 n=20)
SourceUint64-8                2.534n ± 0%    2.533n ± 0%         ~ (p=0.732 n=20)
GlobalInt64-8                 2.172n ± 1%    4.794n ± 0%  +120.67% (p=0.000 n=20)
GlobalInt64Parallel-8        0.4320n ± 0%   0.9605n ± 0%  +122.32% (p=0.000 n=20)
GlobalUint64-8                2.182n ± 0%    4.770n ± 0%  +118.58% (p=0.000 n=20)
GlobalUint64Parallel-8       0.4307n ± 0%   0.9583n ± 0%  +122.51% (p=0.000 n=20)
Int64-8                       4.107n ± 0%    4.104n ± 0%         ~ (p=0.416 n=20)
Uint64-8                      4.080n ± 0%    4.080n ± 0%         ~ (p=0.052 n=20)
GlobalIntN1000-8              2.814n ± 2%    5.643n ± 0%  +100.50% (p=0.000 n=20)
IntN1000-8                    4.141n ± 0%    4.139n ± 0%         ~ (p=0.140 n=20)
Int64N1000-8                  4.140n ± 0%    4.140n ± 0%         ~ (p=0.313 n=20)
Int64N1e8-8                   4.140n ± 0%    4.139n ± 0%         ~ (p=0.103 n=20)
Int64N1e9-8                   4.139n ± 0%    4.140n ± 0%         ~ (p=0.761 n=20)
Int64N2e9-8                   4.140n ± 0%    4.140n ± 0%         ~ (p=0.636 n=20)
Int64N1e18-8                  5.266n ± 0%    5.326n ± 1%    +1.14% (p=0.001 n=20)
Int64N2e18-8                  6.052n ± 0%    6.167n ± 0%    +1.90% (p=0.000 n=20)
Int64N4e18-8                  8.826n ± 0%    9.051n ± 0%    +2.55% (p=0.000 n=20)
Int32N1000-8                  4.127n ± 0%    4.132n ± 0%    +0.12% (p=0.000 n=20)
Int32N1e8-8                   4.126n ± 0%    4.131n ± 0%    +0.12% (p=0.000 n=20)
Int32N1e9-8                   4.127n ± 0%    4.132n ± 0%    +0.12% (p=0.000 n=20)
Int32N2e9-8                   4.132n ± 0%    4.131n ± 0%         ~ (p=0.017 n=20)
Float32-8                     4.109n ± 0%    4.105n ± 0%         ~ (p=0.379 n=20)
Float64-8                     4.107n ± 0%    4.106n ± 0%         ~ (p=0.867 n=20)
ExpFloat64-8                  5.339n ± 0%    5.383n ± 0%    +0.82% (p=0.000 n=20)
NormFloat64-8                 5.735n ± 0%    5.737n ± 1%         ~ (p=0.856 n=20)
Perm3-8                       26.65n ± 0%    26.80n ± 1%    +0.58% (p=0.000 n=20)
Perm30-8                      194.8n ± 1%    197.0n ± 0%    +1.18% (p=0.000 n=20)
Perm30ViaShuffle-8            156.6n ± 0%    157.6n ± 1%    +0.61% (p=0.000 n=20)
ShuffleOverhead-8             124.9n ± 0%    125.5n ± 0%    +0.52% (p=0.000 n=20)
Concurrent-8                  2.434n ± 3%    5.066n ± 0%  +108.09% (p=0.000 n=20)

goos: linux
goarch: 386
pkg: math/rand/v2
cpu: AMD Ryzen 9 7950X 16-Core Processor
                        │ bbb48afeb7.386 │            5cf807d1ea.386             │
                        │     sec/op     │    sec/op     vs base                 │
ChaCha8-32                  11.295n ± 1%    4.748n ± 2%   -57.96% (p=0.000 n=20)
PCG_DXSM-32                  7.693n ± 1%    7.738n ± 2%         ~ (p=0.542 n=20)
SourceUint64-32              7.658n ± 2%    7.622n ± 2%         ~ (p=0.344 n=20)
GlobalInt64-32               3.473n ± 2%    7.526n ± 2%  +116.73% (p=0.000 n=20)
GlobalInt64Parallel-32      0.3198n ± 0%   0.5444n ± 0%   +70.22% (p=0.000 n=20)
GlobalUint64-32              3.612n ± 0%    7.575n ± 1%  +109.69% (p=0.000 n=20)
GlobalUint64Parallel-32     0.3168n ± 0%   0.5403n ± 0%   +70.51% (p=0.000 n=20)
Int64-32                     7.673n ± 2%    7.789n ± 1%         ~ (p=0.122 n=20)
Uint64-32                    7.773n ± 1%    7.827n ± 2%         ~ (p=0.920 n=20)
GlobalIntN1000-32            6.268n ± 1%    9.581n ± 1%   +52.87% (p=0.000 n=20)
IntN1000-32                  10.33n ± 2%    10.45n ± 1%         ~ (p=0.233 n=20)
Int64N1000-32                10.98n ± 2%    11.01n ± 1%         ~ (p=0.401 n=20)
Int64N1e8-32                 11.19n ± 2%    10.97n ± 1%         ~ (p=0.033 n=20)
Int64N1e9-32                 11.06n ± 1%    11.08n ± 1%         ~ (p=0.498 n=20)
Int64N2e9-32                 11.10n ± 1%    11.01n ± 2%         ~ (p=0.995 n=20)
Int64N1e18-32                15.23n ± 2%    15.04n ± 1%         ~ (p=0.973 n=20)
Int64N2e18-32                15.89n ± 1%    15.85n ± 1%         ~ (p=0.409 n=20)
Int64N4e18-32                18.96n ± 2%    19.34n ± 2%         ~ (p=0.048 n=20)
Int32N1000-32                10.46n ± 2%    10.44n ± 2%         ~ (p=0.480 n=20)
Int32N1e8-32                 10.46n ± 2%    10.49n ± 2%         ~ (p=0.951 n=20)
Int32N1e9-32                 10.28n ± 2%    10.26n ± 1%         ~ (p=0.431 n=20)
Int32N2e9-32                 10.50n ± 2%    10.44n ± 2%         ~ (p=0.249 n=20)
Float32-32                   13.80n ± 2%    13.80n ± 2%         ~ (p=0.751 n=20)
Float64-32                   23.55n ± 2%    23.87n ± 0%         ~ (p=0.408 n=20)
ExpFloat64-32                15.36n ± 1%    15.29n ± 2%         ~ (p=0.316 n=20)
NormFloat64-32               13.57n ± 1%    13.79n ± 1%    +1.66% (p=0.005 n=20)
Perm3-32                     45.70n ± 2%    46.99n ± 2%    +2.81% (p=0.001 n=20)
Perm30-32                    399.0n ± 1%    403.8n ± 1%    +1.19% (p=0.006 n=20)
Perm30ViaShuffle-32          349.0n ± 1%    350.4n ± 1%         ~ (p=0.909 n=20)
ShuffleOverhead-32           322.3n ± 1%    323.8n ± 1%         ~ (p=0.410 n=20)
Concurrent-32                3.331n ± 1%    7.312n ± 1%  +119.50% (p=0.000 n=20)

For #61716.

Change-Id: Ibdddeed85c34d9ae397289dc899e04d4845f9ed2
Reviewed-on: https://go-review.googlesource.com/c/go/+/516860
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 20:34:30 +00:00
Damien Neil 285ef16b38 net/http: limit chunked data overhead
The chunked transfer encoding adds some overhead to
the content transferred. When writing one byte per
chunk, for example, there are five bytes of overhead
per byte of data transferred: "1\r\nX\r\n" to send "X".

Chunks may include "chunk extensions",
which we skip over and do not use.
For example: "1;chunk extension here\r\nX\r\n".

A malicious sender can use chunk extensions to add
about 4k of overhead per byte of data.
(The maximum chunk header line size we will accept.)

Track the amount of overhead read in chunked data,
and produce an error if it seems excessive.

Fixes #64433
Fixes CVE-2023-39326

Change-Id: I40f8d70eb6f9575fb43f506eb19132ccedafcf39
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2076135
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/547335
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 17:17:59 +00:00
Bryan C. Mills 58bfef82fc net/http: avoid leaking io.Copy goroutines (and hijacked connections) in TestTransportNoReuseAfterEarlyResponse
Fixes #64252 (maybe).

Change-Id: Iba2a403a9347be4206f14acb11591dc2eb7f9fb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/546616
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-12-01 22:07:19 +00:00
Dmitri Shuralyov 83da21f052 all: update vendored dependencies
The Go 1.22 code freeze has recently started. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.

Generated with:

go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master

For #36905.

Change-Id: I76525261b9a954ed21a3bd3cb6c4a12e6c031d80
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-linux-amd64-longtest,gotip-linux-386-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/546055
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-11-29 21:28:33 +00:00
Jorropo 195c88b202 net/http: use pointers to array for copyBufPool
This is inspired by CL 539915, I'm only submitting now that
CL 456435 has been merged.

This divide the number of objects kept alive by the heap by two
and remove the slice header allocation in New and in the put back.

Change-Id: Ibcd5166bac5a37f365a533e09a28f3b79f81ad58
Reviewed-on: https://go-review.googlesource.com/c/go/+/543515
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-11-21 00:52:06 +00:00
Ville Skyttä 1d187fd0be src: a/an grammar fixes
Change-Id: I179b50ae8e73677d4d408b83424afbbfe6aa17a1
GitHub-Last-Rev: 2e2d9c1e45
GitHub-Pull-Request: golang/go#63478
Reviewed-on: https://go-review.googlesource.com/c/go/+/534015
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-11-20 15:56:08 +00:00
Jes Cok 631a6c2abf all: add missing copyright header
Change-Id: Ic61fb181923159e80a86a41582e83ec466ab9bc4
GitHub-Last-Rev: 9246984566
GitHub-Pull-Request: golang/go#64080
Reviewed-on: https://go-review.googlesource.com/c/go/+/541741
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Jes Cok <xigua67damn@gmail.com>
2023-11-17 23:34:11 +00:00
Andy Pan f664031bc1 net,os: arrange zero-copy of os.File and net.TCPConn to net.UnixConn
Fixes #58808

goos: linux
goarch: amd64
pkg: net
cpu: DO-Premium-Intel
                             │      old      │                 new                  │
                             │    sec/op     │    sec/op     vs base                │
Splice/tcp-to-unix/1024-4       3.783µ ± 10%   3.201µ ±  7%  -15.40% (p=0.001 n=10)
Splice/tcp-to-unix/2048-4       3.967µ ± 13%   3.818µ ± 16%        ~ (p=0.971 n=10)
Splice/tcp-to-unix/4096-4       4.988µ ± 16%   4.590µ ± 11%        ~ (p=0.089 n=10)
Splice/tcp-to-unix/8192-4       6.981µ ± 13%   5.236µ ±  9%  -25.00% (p=0.000 n=10)
Splice/tcp-to-unix/16384-4     10.192µ ±  9%   7.350µ ±  7%  -27.89% (p=0.000 n=10)
Splice/tcp-to-unix/32768-4      19.65µ ± 13%   10.28µ ± 16%  -47.69% (p=0.000 n=10)
Splice/tcp-to-unix/65536-4      41.89µ ± 18%   15.70µ ± 13%  -62.52% (p=0.000 n=10)
Splice/tcp-to-unix/131072-4     90.05µ ± 11%   29.55µ ± 10%  -67.18% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    170.24µ ± 15%   52.66µ ±  4%  -69.06% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4     326.4µ ± 13%   109.3µ ± 11%  -66.52% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4    651.4µ ±  9%   228.3µ ± 14%  -64.95% (p=0.000 n=10)
geomean                         29.42µ         15.62µ        -46.90%

                             │      old      │                  new                   │
                             │      B/s      │      B/s       vs base                 │
Splice/tcp-to-unix/1024-4      258.2Mi ± 11%   305.2Mi ±  8%   +18.21% (p=0.001 n=10)
Splice/tcp-to-unix/2048-4      492.5Mi ± 15%   511.7Mi ± 13%         ~ (p=0.971 n=10)
Splice/tcp-to-unix/4096-4      783.5Mi ± 14%   851.2Mi ± 12%         ~ (p=0.089 n=10)
Splice/tcp-to-unix/8192-4      1.093Gi ± 11%   1.458Gi ±  8%   +33.36% (p=0.000 n=10)
Splice/tcp-to-unix/16384-4     1.497Gi ±  9%   2.076Gi ±  7%   +38.67% (p=0.000 n=10)
Splice/tcp-to-unix/32768-4     1.553Gi ± 11%   2.969Gi ± 14%   +91.17% (p=0.000 n=10)
Splice/tcp-to-unix/65536-4     1.458Gi ± 23%   3.888Gi ± 11%  +166.69% (p=0.000 n=10)
Splice/tcp-to-unix/131072-4    1.356Gi ± 10%   4.131Gi ±  9%  +204.72% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    1.434Gi ± 13%   4.637Gi ±  4%  +223.32% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4    1.497Gi ± 15%   4.468Gi ± 10%  +198.47% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4   1.501Gi ± 10%   4.277Gi ± 16%  +184.88% (p=0.000 n=10)
geomean                        1.038Gi         1.954Gi         +88.28%

                             │      old      │                   new                   │
                             │     B/op      │    B/op     vs base                     │
Splice/tcp-to-unix/1024-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4     0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4     0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4     1.000 ±   ?     0.000 ± 0%  -100.00% (p=0.001 n=10)
Splice/tcp-to-unix/131072-4    2.000 ±  0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    4.000 ± 25%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4    7.500 ± 33%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4   17.00 ± 12%      0.00 ± 0%  -100.00% (p=0.000 n=10)
geomean                                    ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                             │     old      │                 new                 │
                             │  allocs/op   │ allocs/op   vs base                 │
Splice/tcp-to-unix/1024-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/131072-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/262144-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/524288-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/1048576-4   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                   ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Change-Id: I829061b009a0929a8ef1a15c183793c0b9104dde
Reviewed-on: https://go-review.googlesource.com/c/go/+/472475
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-17 23:16:28 +00:00
Mateusz Poliwczak a9722bf918 net: document IsNotFound field
After CL 534937, the IsNotFound field is set to true on
nodata and NXDOMAIN conditions consistently across systems
and all lookup methods.

Change-Id: I419d1c818f17e915cee4959c3a9ac92c2e4bb92c
GitHub-Last-Rev: 59c537fc06
GitHub-Pull-Request: golang/go#63654
Reviewed-on: https://go-review.googlesource.com/c/go/+/536775
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-17 18:12:04 +00:00
Tobias Klauser 1cecbb38f4 net/netip: optimize AddrPort.String for IPv4 addresses
Use Addr.appendTo4 like in Addr.String.

name                          old time/op    new time/op    delta
AddrPortString/v4-8             47.5ns ± 0%    27.7ns ± 1%  -41.81%  (p=0.000 n=8+10)
AddrPortString/v6-8             95.2ns ± 0%    94.9ns ± 1%   -0.35%  (p=0.034 n=10+10)
AddrPortString/v6_ellipsis-8    96.8ns ± 0%    96.6ns ± 0%   -0.15%  (p=0.008 n=9+9)
AddrPortString/v6_v4-8          70.9ns ± 0%    70.9ns ± 0%     ~     (p=0.425 n=10+10)
AddrPortString/v6_zone-8        97.0ns ± 0%    97.0ns ± 0%     ~     (p=0.838 n=10+10)

name                          old alloc/op   new alloc/op   delta
AddrPortString/v4-8              24.0B ± 0%     24.0B ± 0%     ~     (all equal)
AddrPortString/v6-8              96.0B ± 0%     96.0B ± 0%     ~     (all equal)
AddrPortString/v6_ellipsis-8     56.0B ± 0%     56.0B ± 0%     ~     (all equal)
AddrPortString/v6_v4-8           56.0B ± 0%     56.0B ± 0%     ~     (all equal)
AddrPortString/v6_zone-8         56.0B ± 0%     56.0B ± 0%     ~     (all equal)

name                          old allocs/op  new allocs/op  delta
AddrPortString/v4-8               1.00 ± 0%      1.00 ± 0%     ~     (all equal)
AddrPortString/v6-8               2.00 ± 0%      2.00 ± 0%     ~     (all equal)
AddrPortString/v6_ellipsis-8      2.00 ± 0%      2.00 ± 0%     ~     (all equal)
AddrPortString/v6_v4-8            2.00 ± 0%      2.00 ± 0%     ~     (all equal)
AddrPortString/v6_zone-8          2.00 ± 0%      2.00 ± 0%     ~     (all equal)

Change-Id: I3d3fcee807ca33d1e8d6dafb03ab844ea0c76bea
Reviewed-on: https://go-review.googlesource.com/c/go/+/542396
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
2023-11-17 16:50:42 +00:00
Tobias Klauser 128d96b3ca net/netip: optimize AddrPort.String for IPv6 addresses
Inline the IPv6-case of joinHostPort and avoid the check for a colon in
the address. The address is already known to be an IPv6 address. Also
use iota.Uitoa to convert the uin16 port.

name                          old time/op    new time/op    delta
AddrPortString/v4-8             45.7ns ± 0%    46.6ns ± 1%   +1.97%  (p=0.000 n=10+10)
AddrPortString/v6-8              106ns ± 3%      95ns ± 0%  -10.79%  (p=0.000 n=9+10)
AddrPortString/v6_ellipsis-8     110ns ± 0%      96ns ± 0%  -12.37%  (p=0.000 n=9+10)
AddrPortString/v6_v4-8          85.1ns ± 0%    70.8ns ± 0%  -16.80%  (p=0.000 n=9+10)
AddrPortString/v6_zone-8         110ns ± 0%      97ns ± 0%  -12.29%  (p=0.000 n=10+10)

name                          old alloc/op   new alloc/op   delta
AddrPortString/v4-8              24.0B ± 0%     24.0B ± 0%     ~     (all equal)
AddrPortString/v6-8               101B ± 0%       96B ± 0%   -4.95%  (p=0.000 n=10+10)
AddrPortString/v6_ellipsis-8     61.0B ± 0%     56.0B ± 0%   -8.20%  (p=0.000 n=10+10)
AddrPortString/v6_v4-8           61.0B ± 0%     56.0B ± 0%   -8.20%  (p=0.000 n=10+10)
AddrPortString/v6_zone-8         61.0B ± 0%     56.0B ± 0%   -8.20%  (p=0.000 n=10+10)

name                          old allocs/op  new allocs/op  delta
AddrPortString/v4-8               1.00 ± 0%      1.00 ± 0%     ~     (all equal)
AddrPortString/v6-8               3.00 ± 0%      2.00 ± 0%  -33.33%  (p=0.000 n=10+10)
AddrPortString/v6_ellipsis-8      3.00 ± 0%      2.00 ± 0%  -33.33%  (p=0.000 n=10+10)
AddrPortString/v6_v4-8            3.00 ± 0%      2.00 ± 0%  -33.33%  (p=0.000 n=10+10)
AddrPortString/v6_zone-8          3.00 ± 0%      2.00 ± 0%  -33.33%  (p=0.000 n=10+10)

Change-Id: I96a7f01c4e33bfa157a98558c62e2e0a8147726d
Reviewed-on: https://go-review.googlesource.com/c/go/+/542395
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-11-17 16:45:56 +00:00
Damien Neil fdc21f3eaf net/http: don't set length for non-range encoded content requests
Historically, serveContent has not set Content-Length
when the user provides Content-Encoding.

This causes broken responses when the user sets both Content-Length
and Content-Encoding, and the request is a range request,
because the returned data doesn't match the declared length.

CL 381956 fixed this case by changing serveContent to always set
a Content-Length header.

Unfortunately, I've discovered multiple cases in the wild of
users setting Content-Encoding: gzip and passing serveContent
a ResponseWriter wrapper that gzips the data written to it.

This breaks serveContent in a number of ways. In particular,
there's no way for it to respond to Range requests properly,
because it doesn't know the recipient's view of the content.

What the user should be doing in this case is just using
io.Copy to send the gzipped data to the response.
Or possibly setting Transfer-Encoding: gzip.
But whatever they should be doing, what they are doing has
mostly worked for non-Range requests, and setting
Content-Length makes it stop working because the length
of the file being served doesn't match the number of bytes
being sent.

So in the interests of not breaking users (even if they're
misusing serveContent in ways that are already broken),
partially revert CL 381956.

For non-Range requests, don't set Content-Length when
the user has set Content-Encoding. This matches our previous
behavior and causes minimal harm in cases where we could
have set Content-Length. (We will send using chunked
encoding rather than identity, but that's fine.)

For Range requests, set Content-Length unconditionally.
Either the user isn't mangling the data in the ResponseWriter,
in which case the length is correct, or they are, in which
case the response isn't going to contain the right bytes anyway.
(Note that a Range request for a Content-Length: gzip file
is requesting a range of *gzipped* bytes, not a range from
the uncompressed file.)

Change-Id: I5e788e6756f34cee520aa7c456826f462a59f7eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/542595
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
2023-11-15 18:59:12 +00:00
Roland Shoemaker 67d555e089 crypto/tls: remove RSA KEX ciphers from the default list
Removes the RSA KEX based ciphers from the default list. This can be
reverted using the tlsrsakex GODEBUG.

Fixes #63413

Change-Id: Id221be3eb2f6c24b91039d380313f0c87d339f98
Reviewed-on: https://go-review.googlesource.com/c/go/+/541517
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-11-14 18:03:29 +00:00
Michał Matczuk 96eeb4512b net/http: use copyBufPool in transferWriter.doBodyCopy()
This is a followup to CL 14177. It applies copyBufPool optimization to
transferWriter.doBodyCopy(). The function is used every time Request or
Response is written.

Without this patch for every Request and Response processed, if there is
a body, we need to allocate and GC a 32k buffer. This is quickly causing
GC pressure.

Fixes #57202

Change-Id: I4c30e1737726ac8d9937846106efd02effbae300
GitHub-Last-Rev: 908573cdbe
GitHub-Pull-Request: golang/go#57205
Reviewed-on: https://go-review.googlesource.com/c/go/+/456435
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
2023-11-09 18:51:38 +00:00
Mikhail Mazurskiy ac85f2bedd net/url: use quick path in URL.Encode() on empty map
Make url.Values.Encode() slightly more efficient when url.Values
is an empty but non-nil map.

Change-Id: I7f205cc7e67526a1fa0035eab4773cec5e0f2c99
GitHub-Last-Rev: 0530b439db
GitHub-Pull-Request: golang/go#63836
Reviewed-on: https://go-review.googlesource.com/c/go/+/538637
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2023-11-07 22:01:35 +00:00
wulianglongrd 995ec5c85b net/http/cookiejar: remove unused variable
The errNoHostname variable is not used, delete it.

Change-Id: I62ca6390fd026e6a8cb1e8147f3fbfc3078c2249
Reviewed-on: https://go-review.googlesource.com/c/go/+/538455
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-11-07 22:00:28 +00:00
aimuz ff19f8e763 net/http/cgi: eliminate use of Perl in tests
Previously, a Perl script was used to test the net/http/cgi package.
This sometimes led to hidden failures as these tests were not run
on builders without Perl.
Also, this approach posed maintenance difficulties for those
unfamiliar with Perl.

We have now replaced Perl-based tests with a Go handler to simplify
maintenance and ensure consistent testing environments.
It's part of our ongoing effort to reduce reliance on Perl throughout
the Go codebase (see #20032,#25586,#25669,#27779),
thus improving reliability and ease of maintenance.

Fixes #63800
Fixes #63828

Change-Id: I8d554af93d4070036cf0cc3aaa9c9b256affbd17
GitHub-Last-Rev: a8034083d8
GitHub-Pull-Request: golang/go#63869
Reviewed-on: https://go-review.googlesource.com/c/go/+/538861
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: qiulaidongfeng <2645477756@qq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Commit-Queue: Bryan Mills <bcmills@google.com>
2023-11-07 18:42:44 +00:00
aimuz 6458c8e45f net/http/cgi: the PATH_INFO should be empty or start with a slash
fixed PATH_INFO not starting with a slash as described in RFC 3875
for PATH_INFO.

Fixes #63925

Change-Id: I1ead98dff190c53eb7a50546569ef6ded3199a0a
GitHub-Last-Rev: 1c532e330b
GitHub-Pull-Request: golang/go#63926
Reviewed-on: https://go-review.googlesource.com/c/go/+/539615
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-06 22:04:34 +00:00
carl.tao 42b20297d3 net/http: remove Content-Encoding header in roundtrip_js
The fetch api will decode the gzip, but Content-Encoding not be deleted.
To ensure that the behavior of roundtrip_js is consistent with native. delete the Content-Encoding header when the response body is decompressed by js fetch api.

Fixes #63139

Change-Id: Ie35b3aa050786e2ef865f9ffa992e30ab060506e
Reviewed-on: https://go-review.googlesource.com/c/go/+/530155
Commit-Queue: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2023-11-06 21:44:04 +00:00
Mauri de Souza Meneguzzo f83bbaf3af net/netip: allow only valid prefix digits in ParsePrefix
The prefix bits for a call to ParsePrefix are passed raw to
strconv.Atoi, this means that it can accept +- signs as well as leading
zeroes, which are not allowed prefix values following RFC 4632 Section
3.1 and RFC 4291 Section 2.3.

Validate non-digit characters as well as leading zeroes and return an
error accordingly.

Fixes #63850

Change-Id: I412a7e1cecc6ee9ea1582d4b04cb40d79ee714f1
GitHub-Last-Rev: 462d97fc5f
GitHub-Pull-Request: golang/go#63859
Reviewed-on: https://go-review.googlesource.com/c/go/+/538860
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-06 21:18:11 +00:00
Mitar 8a360d68c4 net/http: set/override Content-Length for encoded range requests
Currently, http.ServeContent returns invalid Content-Length header if:

* Request is a range request.
* Content is encoded (e.g., gzip compressed).
* Content-Length of the encoded content has been set before calling
  http.ServeContent, as suggested in https://github.com/golang/go/issues/19420.

Example:

	w.Header().Set("Content-Type", "application/json")
	w.Header().Set("Content-Length", strconv.Itoa(len(compressedJsonBody)))
	w.Header().Set("Content-Encoding", "gzip")
	w.Header().Set("Etag", etag)
	http.ServeContent(
		w, req, "", time.Time{},
		bytes.NewReader(compressedJsonBody),
	)

The issue is that http.ServeContent currently sees Content-Length as
something optional when Content-Encoding is set, but that is a problem
with range request which can send a payload of different size. So this
reverts https://go.dev/cl/4538111 and makes Content-Length be set
always to the number of bytes which will actually be send (both for
range and non-range requests).

Without this fix, this is an example response:

	HTTP/1.1 206 Partial Content
	Accept-Ranges: bytes
	Content-Encoding: gzip
	Content-Length: 351
	Content-Range: bytes 100-350/351
	Content-Type: application/json; charset=UTF-8
	Etag: "amCTP_vgT5PQt5OsAEI7NFJ6Hx1UfEpR5nIaYEInfOA"
	Date: Sat, 29 Jan 2022 14:42:15 GMT

As you see, Content-Length is invalid and should be 251.

Change-Id: I4d2ea3a8489a115f92ef1f7e98250d555b47a94e
GitHub-Last-Rev: 3aff9126f5
GitHub-Pull-Request: golang/go#50904
Reviewed-on: https://go-review.googlesource.com/c/go/+/381956
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: t hepudds <thepudds1460@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-11-06 21:17:29 +00:00
Zeke Lu f5ec2e46d9 net/http/httptest: remove unnecessary creation of http.Transport
In (*Server).StartTLS, it's unnecessary to create an http.Client
with a Transport, because a new one will be created with the
TLSClientConfig later.

Change-Id: I086e28717e9739787529006c3f0296c8224cd790
GitHub-Last-Rev: 33724596bd
GitHub-Pull-Request: golang/go#60124
Reviewed-on: https://go-review.googlesource.com/c/go/+/494355
Run-TryBot: t hepudds <thepudds1460@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
2023-11-06 21:16:01 +00:00
Bryan C. Mills 547eb8d699 net/http: remove arbitrary timeouts in tests of Server.ErrorLog
This also allows us to remove the chanWriter helper from the test,
using a simpler strings.Builder instead, relying on
clientServerTest.close for synchronization.
(I don't think this runs afoul of #38370, because the handler
functions themselves in these tests should never be executed,
let alone result in an asynchronous write to the error log.)

Fixes #57599.

Change-Id: I45c6cefca0bb218f6f9a9659de6bde454547f704
Reviewed-on: https://go-review.googlesource.com/c/go/+/539436
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-02 20:15:19 +00:00
Mauri de Souza Meneguzzo 0046c1414c net/http: pull http2 underflow fix from x/net/http2
After CL 534295 was merged to fix a CVE it introduced
an underflow when we try to decrement sc.curHandlers
in handlerDone.

Pull in a fix from x/net/http2:
http2: fix underflow in http2 server push
https://go-review.googlesource.com/c/net/+/535595

Fixes #63511

Change-Id: I5c678ce7dcc53635f3ad5e4999857cb120dfc1ab
GitHub-Last-Rev: 587ffa3caf
GitHub-Pull-Request: golang/go#63561
Reviewed-on: https://go-review.googlesource.com/c/go/+/535575
Run-TryBot: Mauri de Souza Meneguzzo <mauri870@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-10-26 15:35:15 +00:00
Keiichi Hirobe f9c54f9cd4 net/http: remove outdated comment about a support of CONNECT method
The net/http.Transport already supports CONNECT after
https://go-review.googlesource.com/c/go/+/123156 was merged, which
deleted comments in transport.go.

Change-Id: I784fdb9b044bc8a4a29bf252328c80a11aaf6901
Reviewed-on: https://go-review.googlesource.com/c/go/+/536057
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2023-10-23 20:47:30 +00:00
Andy Pan c65f74d339 net,internal/poll: mark it as handled even if sendfile(2) succeeded with 0 bytes sent
CL 415834 fixed #53658 and somehow it only fixed it on Linux,
sendfile can also succeed with 0 bytes sent on other platforms
according to their manuals, this CL will finish the work that
CL 415834 left out on other platforms.

goos: darwin
goarch: arm64
pkg: net
                     │     old     │                new                 │
                     │   sec/op    │   sec/op     vs base               │
SendfileZeroBytes-10   7.563µ ± 5%   7.184µ ± 6%  -5.01% (p=0.009 n=10)

                     │     old     │                new                 │
                     │    B/op     │    B/op     vs base                │
SendfileZeroBytes-10   3562.5 ± 7%   590.0 ± 2%  -83.44% (p=0.000 n=10)

                     │    old    │             new              │
                     │ allocs/op │ allocs/op   vs base          │
SendfileZeroBytes-10   0.00 ± 0%   11.00 ± 0%  ? (p=0.000 n=10)

[1] https://man.freebsd.org/cgi/man.cgi?sendfile(2)
[2] https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/sendfile.2.html
[3] https://man.dragonflybsd.org/?command=sendfile&section=2
[4] https://docs.oracle.com/cd/E88353_01/html/E37843/sendfile-3c.html

Change-Id: I55832487595ee8e0f44f367cf2a3a1d827ba590d
Reviewed-on: https://go-review.googlesource.com/c/go/+/536455
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-10-23 18:44:26 +00:00
Mateusz Poliwczak 95974b379f net: set IsNotFound on windows and plan9 DNS queries
Change-Id: I2a12acb3e4f31dd561d49f47a3b1ae3ac47ab894
GitHub-Last-Rev: 4733964f69
GitHub-Pull-Request: golang/go#63542
Reviewed-on: https://go-review.googlesource.com/c/go/+/534937
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Run-TryBot: Mateusz Poliwczak <mpoliwczak34@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
2023-10-23 09:03:09 +00:00
Dmitri Shuralyov c729dc187a all: update vendored dependencies
One of the remaining uses of the old +build syntax was in the bundled
copy of golang.org/x/net/http2 in net/http. Pull in a newer version of
bundle with CL 536075 that drops said +build syntax line. Also pull in
newer x/sys and other golang.org/x modules where old +build lines were
recently dropped.

Generated with:

go install golang.org/x/build/cmd/updatestd@latest
go install golang.org/x/tools/cmd/bundle@latest
updatestd -goroot=$(pwd) -branch=master

For #36905.
For #41184.
For #60268.

Change-Id: Ia18d1ce9eadce85b38176058ad1fe38562b004e9
Cq-Include-Trybots: luci.golang.try:gotip-linux-386-longtest,gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/536575
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-10-19 23:33:33 +00:00
Dmitri Shuralyov bf97e724b5 all: drop old +build lines
Running 'go fix' on the cmd+std packages handled much of this change.

Also update code generators to use only the new go:build lines,
not the old +build ones.

For #41184.
For #60268.

Change-Id: If35532abe3012e7357b02c79d5992ff5ac37ca23
Cq-Include-Trybots: luci.golang.try:gotip-linux-386-longtest,gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/536237
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-19 23:33:27 +00:00
cui fliter bc9dc8d415 all: fix function names
Change-Id: Iba8878420c59d705066d1d9955e91a5c2eb4faf5
Reviewed-on: https://go-review.googlesource.com/c/go/+/507615
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-10-14 22:30:23 +00:00
Ian Lance Taylor 3b0fc5a51f net: clear malloc'ed memory in cgoResSearch
For #61666

Change-Id: I7a0a849fba0abebe28804bdd6d364b154456e399
Reviewed-on: https://go-review.googlesource.com/c/go/+/534516
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-10-11 22:40:01 +00:00
Dmitri Shuralyov f34964a5cf all: pull in x/net v0.17.0 and its dependencies
Pull in a security fix from x/net/http2:
http2: limit maximum handler goroutines to MaxConcurrentStreams

Fixes #63417.
Fixes CVE-2023-39325.

Change-Id: I01e7774912e81007a7cf70f33e5989fb50a0b708
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/534295
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-10 16:32:44 +00:00
Jonathan Amsterdam 62bdbd2596 net/http: fix ServeMux pattern registration
When the httpmuxgo121 GODEBUG setting was active, we were registering
patterns in the old and the new way. Fix to register only in the old
way.

Change-Id: Ibc1fd41e7f4d162ee5bc34575df409e1db5657cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/533095
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Olena Synenka <olenasynenka@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-10-05 19:03:37 +00:00
edef 3a69dcdc9f net/http/cgi: set SERVER_PORT to 443 when req.TLS != nil
A hostname without a port leaves the port implied by the protocol.
For HTTPS, the implied port is 443, not 80.

Change-Id: I873a076068f84c8041abf10a435d9499635730a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/454975
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-04 16:50:58 +00:00
Jonathan Amsterdam 90276c268b net/http: document new ServeMux patterns
Updates #61410.

Change-Id: Ib9dd8ebca43cec6e27c6fdfcf01ee6a1539c2fa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/530481
Reviewed-by: Eli Bendersky <eliben@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-10-03 18:06:38 +00:00
Mateusz Poliwczak ef3171c5eb net: handle the network parameter properly in LookupPort
The cgo version (unix) is populating the GetAddrInfo hints
based on the network parameter, but windows not quite.

This change populates the hints the same way as the
cgo unix version does now.

This bug was spotted by Bryan in CL 530415.
https://go-review.googlesource.com/c/go/+/530415/comment/76640dc7_ed0409ca/

Change-Id: I6fc29b1e4cdc879123ab0f5a624b6f37c68c00ba
GitHub-Last-Rev: eaa616378b
GitHub-Pull-Request: golang/go#63284
Reviewed-on: https://go-review.googlesource.com/c/go/+/531635
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Mateusz Poliwczak <mpoliwczak34@gmail.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-10-02 23:13:53 +00:00
Jonathan Amsterdam 32b6d2d9a8 net/http: add a test for an empty ServeMux
Make sure a ServeMux with no patterns is well-behaved.

Updates #61410.

Change-Id: Ib3eb85b384e1309e785663902d2c45ae01e64807
Reviewed-on: https://go-review.googlesource.com/c/go/+/530479
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-10-02 20:28:47 +00:00
Jonathan Amsterdam eb070d7483 net/http: add GODEBUG setting for old ServeMux behavior
Add the GODEBUG setting httpmuxgo121.
When set to "1", ServeMux behaves exactly like it did in Go 1.21.

Implemented by defining a new, unexported type, serveMux121, that
uses the original code.

Updates #61410.

Change-Id: I0a9d0fe2a2286e442d680393e62895ab50683cea
Reviewed-on: https://go-review.googlesource.com/c/go/+/530461
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2023-10-02 20:28:30 +00:00
Mateusz Poliwczak 0ba8ef4702 net: set IsNotFound for unknown services in LookupPort
Change-Id: I9d5f0ea5edd2c121179e3d2f8d4a890fa25a3fa9
GitHub-Last-Rev: 48a13fe5f5
GitHub-Pull-Request: golang/go#63160
Reviewed-on: https://go-review.googlesource.com/c/go/+/530415
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-09-28 13:41:21 +00:00
Bryan C. Mills 6cf6067d4e net/http: add extra synchronization for a Logf call in TestTransportAndServerSharedBodyRace
This race was reported in
https://build.golang.org/log/6f043170946b665edb85b50804a62db68348c52f.

As best as I can tell, it is another instance of #38370. The deferred
call to backend.close() ought to be enough to ensure that the t.Logf
happens before the end of the test, but in practice it is not, and
with enough scheduling delay we can manage to trip the race detector
on a call to Logf after the test function has returned.

Updates #38370.

Change-Id: I5ee45df45c6bfad3239d665df65a138f1c4573a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/531195
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2023-09-27 15:58:11 +00:00
Jonathan Amsterdam 70e04706d8 net/http: remove unused function
Change-Id: I4364d94663282249e632d12026a810147844ad2e
Reviewed-on: https://go-review.googlesource.com/c/go/+/530615
Run-TryBot: Jonathan Amsterdam <jba@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2023-09-25 16:46:57 +00:00