mirror
/
go
mirror of https://github.com/golang/go.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
59,235 Commits 63 Branches 451 Tags 660 MiB
Commit Graph

13 Commits

Author SHA1 Message Date
Andy Pan f664031bc1 net,os: arrange zero-copy of os.File and net.TCPConn to net.UnixConn 
Fixes #58808

goos: linux
goarch: amd64
pkg: net
cpu: DO-Premium-Intel
                             │      old      │                 new                  │
                             │    sec/op     │    sec/op     vs base                │
Splice/tcp-to-unix/1024-4       3.783µ ± 10%   3.201µ ±  7%  -15.40% (p=0.001 n=10)
Splice/tcp-to-unix/2048-4       3.967µ ± 13%   3.818µ ± 16%        ~ (p=0.971 n=10)
Splice/tcp-to-unix/4096-4       4.988µ ± 16%   4.590µ ± 11%        ~ (p=0.089 n=10)
Splice/tcp-to-unix/8192-4       6.981µ ± 13%   5.236µ ±  9%  -25.00% (p=0.000 n=10)
Splice/tcp-to-unix/16384-4     10.192µ ±  9%   7.350µ ±  7%  -27.89% (p=0.000 n=10)
Splice/tcp-to-unix/32768-4      19.65µ ± 13%   10.28µ ± 16%  -47.69% (p=0.000 n=10)
Splice/tcp-to-unix/65536-4      41.89µ ± 18%   15.70µ ± 13%  -62.52% (p=0.000 n=10)
Splice/tcp-to-unix/131072-4     90.05µ ± 11%   29.55µ ± 10%  -67.18% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    170.24µ ± 15%   52.66µ ±  4%  -69.06% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4     326.4µ ± 13%   109.3µ ± 11%  -66.52% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4    651.4µ ±  9%   228.3µ ± 14%  -64.95% (p=0.000 n=10)
geomean                         29.42µ         15.62µ        -46.90%

                             │      old      │                  new                   │
                             │      B/s      │      B/s       vs base                 │
Splice/tcp-to-unix/1024-4      258.2Mi ± 11%   305.2Mi ±  8%   +18.21% (p=0.001 n=10)
Splice/tcp-to-unix/2048-4      492.5Mi ± 15%   511.7Mi ± 13%         ~ (p=0.971 n=10)
Splice/tcp-to-unix/4096-4      783.5Mi ± 14%   851.2Mi ± 12%         ~ (p=0.089 n=10)
Splice/tcp-to-unix/8192-4      1.093Gi ± 11%   1.458Gi ±  8%   +33.36% (p=0.000 n=10)
Splice/tcp-to-unix/16384-4     1.497Gi ±  9%   2.076Gi ±  7%   +38.67% (p=0.000 n=10)
Splice/tcp-to-unix/32768-4     1.553Gi ± 11%   2.969Gi ± 14%   +91.17% (p=0.000 n=10)
Splice/tcp-to-unix/65536-4     1.458Gi ± 23%   3.888Gi ± 11%  +166.69% (p=0.000 n=10)
Splice/tcp-to-unix/131072-4    1.356Gi ± 10%   4.131Gi ±  9%  +204.72% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    1.434Gi ± 13%   4.637Gi ±  4%  +223.32% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4    1.497Gi ± 15%   4.468Gi ± 10%  +198.47% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4   1.501Gi ± 10%   4.277Gi ± 16%  +184.88% (p=0.000 n=10)
geomean                        1.038Gi         1.954Gi         +88.28%

                             │      old      │                   new                   │
                             │     B/op      │    B/op     vs base                     │
Splice/tcp-to-unix/1024-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4     0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4     0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4     1.000 ±   ?     0.000 ± 0%  -100.00% (p=0.001 n=10)
Splice/tcp-to-unix/131072-4    2.000 ±  0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    4.000 ± 25%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4    7.500 ± 33%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4   17.00 ± 12%      0.00 ± 0%  -100.00% (p=0.000 n=10)
geomean                                    ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                             │     old      │                 new                 │
                             │  allocs/op   │ allocs/op   vs base                 │
Splice/tcp-to-unix/1024-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/131072-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/262144-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/524288-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/1048576-4   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                   ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Change-Id: I829061b009a0929a8ef1a15c183793c0b9104dde
Reviewed-on: https://go-review.googlesource.com/c/go/+/472475
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
 2023-11-17 23:16:28 +00:00
Ian Lance Taylor b441eb3f97 os: remove mixed dot- and non-dot- imports in tests 
Some test files imported "os" twice, once with a dot and once without.
Consolidate on importing with a dot.

Change-Id: I1db31053dff9dee19a6bdfc263c7e7ef0c15ee42
Reviewed-on: https://go-review.googlesource.com/c/go/+/479995
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
 2023-03-29 16:21:13 +00:00
Ian Lance Taylor be27fcfd2b os, internal/poll: don't use splice with tty 
Also don't try to wait for a non-pollable FD.

Fixes #59041

Change-Id: Ife469d8738f2cc27c0beba223bdc8f8bc757b2a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/476335
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
 2023-03-15 22:01:36 +00:00
Cuong Manh Le 0d52401e23 os: checking for testable network before creating local listener 
Fixes #58745

Change-Id: Id6666477b2c25f081d6f86047cea12bf8b3cb679
Reviewed-on: https://go-review.googlesource.com/c/go/+/471495
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Andy Pan <panjf2000@gmail.com>
 2023-02-27 11:56:55 +00:00
Andy Pan af6bb3f9b2 os: support zero-copy from TCP/Unix socket to file 
Go currently supports the cases of zero-copy: from TCP/Unix socket to TCP socket, from file to TCP socket, from file to file.

Now implementing the new support of zero-copy: from TCP/Unix socket to file.

goos: linux
goarch: amd64
pkg: net
cpu: DO-Premium-Intel
                                  │      old      │                 new                  │
                                  │    sec/op     │    sec/op     vs base                │
SpliceFile/tcp-to-file/1024-4        5.910µ ±  9%   4.116µ ± 13%  -30.35% (p=0.000 n=10)
SpliceFile/tcp-to-file/2048-4        6.150µ ± 10%   4.077µ ± 13%  -33.72% (p=0.002 n=10)
SpliceFile/tcp-to-file/4096-4        4.837µ ± 28%   4.447µ ± 23%        ~ (p=0.353 n=10)
SpliceFile/tcp-to-file/8192-4        9.309µ ±  7%   6.293µ ±  9%  -32.40% (p=0.000 n=10)
SpliceFile/tcp-to-file/16384-4       19.43µ ± 12%   12.48µ ±  9%  -35.76% (p=0.000 n=10)
SpliceFile/tcp-to-file/32768-4       42.73µ ± 10%   25.32µ ±  8%  -40.76% (p=0.000 n=10)
SpliceFile/tcp-to-file/65536-4       70.37µ ± 11%   48.60µ ±  4%  -30.93% (p=0.000 n=10)
SpliceFile/tcp-to-file/131072-4     141.91µ ±  6%   96.24µ ±  4%  -32.18% (p=0.000 n=10)
SpliceFile/tcp-to-file/262144-4      329.7µ ±  8%   246.7µ ± 13%  -25.19% (p=0.000 n=10)
SpliceFile/tcp-to-file/524288-4      653.5µ ±  7%   441.6µ ±  7%  -32.43% (p=0.000 n=10)
SpliceFile/tcp-to-file/1048576-4    1184.4µ ±  9%   851.8µ ± 14%  -28.09% (p=0.000 n=10)
SpliceFile/unix-to-file/1024-4       1.734µ ± 10%   1.524µ ± 25%  -12.06% (p=0.035 n=10)
SpliceFile/unix-to-file/2048-4       2.614µ ±  7%   2.231µ ±  8%  -14.65% (p=0.000 n=10)
SpliceFile/unix-to-file/4096-4       5.081µ ±  7%   3.947µ ± 11%  -22.33% (p=0.000 n=10)
SpliceFile/unix-to-file/8192-4       8.560µ ±  5%   8.531µ ± 17%        ~ (p=0.796 n=10)
SpliceFile/unix-to-file/16384-4      18.09µ ± 12%   12.92µ ± 25%  -28.59% (p=0.000 n=10)
SpliceFile/unix-to-file/32768-4      35.50µ ±  5%   24.50µ ±  6%  -31.00% (p=0.000 n=10)
SpliceFile/unix-to-file/65536-4      69.99µ ±  7%   51.22µ ± 23%  -26.82% (p=0.000 n=10)
SpliceFile/unix-to-file/131072-4     133.7µ ± 17%   119.7µ ±  6%  -10.43% (p=0.000 n=10)
SpliceFile/unix-to-file/262144-4     246.5µ ±  5%   207.3µ ± 19%  -15.90% (p=0.007 n=10)
SpliceFile/unix-to-file/524288-4     484.8µ ± 20%   382.9µ ± 10%  -21.02% (p=0.000 n=10)
SpliceFile/unix-to-file/1048576-4   1188.4µ ± 27%   781.8µ ± 11%  -34.21% (p=0.000 n=10)
geomean                              42.24µ         31.45µ        -25.53%

                                  │      old       │                  new                   │
                                  │      B/s       │      B/s        vs base                │
SpliceFile/tcp-to-file/1024-4        165.4Mi ± 10%    237.3Mi ± 11%  +43.47% (p=0.000 n=10)
SpliceFile/tcp-to-file/2048-4        317.6Mi ± 12%    479.7Mi ± 14%  +51.02% (p=0.002 n=10)
SpliceFile/tcp-to-file/4096-4        808.2Mi ± 22%    886.8Mi ± 19%        ~ (p=0.353 n=10)
SpliceFile/tcp-to-file/8192-4        839.3Mi ±  6%   1241.5Mi ±  8%  +47.91% (p=0.000 n=10)
SpliceFile/tcp-to-file/16384-4       804.7Mi ± 13%   1252.2Mi ± 10%  +55.61% (p=0.000 n=10)
SpliceFile/tcp-to-file/32768-4       731.3Mi ± 11%   1234.3Mi ±  7%  +68.78% (p=0.000 n=10)
SpliceFile/tcp-to-file/65536-4       888.7Mi ± 10%   1286.2Mi ±  4%  +44.73% (p=0.000 n=10)
SpliceFile/tcp-to-file/131072-4      880.9Mi ±  6%   1299.0Mi ±  4%  +47.47% (p=0.000 n=10)
SpliceFile/tcp-to-file/262144-4      758.2Mi ±  7%   1014.4Mi ± 15%  +33.78% (p=0.000 n=10)
SpliceFile/tcp-to-file/524288-4      765.3Mi ±  7%   1132.5Mi ±  7%  +47.99% (p=0.000 n=10)
SpliceFile/tcp-to-file/1048576-4     845.0Mi ±  8%   1174.0Mi ± 16%  +38.94% (p=0.000 n=10)
SpliceFile/unix-to-file/1024-4       564.2Mi ± 11%    640.5Mi ± 20%  +13.53% (p=0.035 n=10)
SpliceFile/unix-to-file/2048-4       747.4Mi ±  7%    875.7Mi ±  8%  +17.17% (p=0.000 n=10)
SpliceFile/unix-to-file/4096-4       768.8Mi ±  6%    989.8Mi ± 10%  +28.74% (p=0.000 n=10)
SpliceFile/unix-to-file/8192-4       912.9Mi ±  5%    915.8Mi ± 15%        ~ (p=0.796 n=10)
SpliceFile/unix-to-file/16384-4      863.6Mi ± 10%   1209.7Mi ± 20%  +40.06% (p=0.000 n=10)
SpliceFile/unix-to-file/32768-4      880.2Mi ±  6%   1275.7Mi ±  6%  +44.93% (p=0.000 n=10)
SpliceFile/unix-to-file/65536-4      893.0Mi ±  7%   1220.3Mi ± 19%  +36.66% (p=0.000 n=10)
SpliceFile/unix-to-file/131072-4     935.1Mi ± 14%   1043.9Mi ±  7%  +11.64% (p=0.000 n=10)
SpliceFile/unix-to-file/262144-4    1014.2Mi ±  6%   1205.9Mi ± 16%  +18.91% (p=0.007 n=10)
SpliceFile/unix-to-file/524288-4     1.007Gi ± 17%    1.275Gi ±  9%  +26.61% (p=0.000 n=10)
SpliceFile/unix-to-file/1048576-4    841.8Mi ± 21%   1279.0Mi ± 10%  +51.94% (p=0.000 n=10)
geomean                              740.1Mi          994.2Mi        +34.33%

                                  │      old       │                   new                   │
                                  │      B/op      │    B/op     vs base                     │
SpliceFile/tcp-to-file/1024-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/2048-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/4096-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/8192-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/16384-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/32768-4      1.000 ±   0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/65536-4      1.000 ± 100%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/131072-4     3.000 ±  33%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/262144-4     8.500 ±  18%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/524288-4     16.50 ±  21%      0.00 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/1048576-4    30.50 ±  15%      0.00 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/1024-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/2048-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/4096-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/8192-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/16384-4     0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/32768-4     0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/65536-4     1.000 ± 100%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/131072-4    3.000 ±  33%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/262144-4    6.000 ±  17%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/524288-4    12.00 ±   8%      0.00 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/1048576-4   33.50 ±  10%      0.00 ± 0%  -100.00% (p=0.000 n=10)
geomean                                          ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                                  │     old      │                 new                 │
                                  │  allocs/op   │ allocs/op   vs base                 │
SpliceFile/tcp-to-file/1024-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/2048-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/4096-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/8192-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/16384-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/32768-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/65536-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/131072-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/262144-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/524288-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/1048576-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/1024-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/2048-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/4096-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/8192-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/16384-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/32768-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/65536-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/131072-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/262144-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/524288-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/1048576-4   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                        ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Change-Id: Ie7f7d4d7b6b373d9ee7ce6da8f6a4cd157632486
Reviewed-on: https://go-review.googlesource.com/c/go/+/466015
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
 2023-02-27 00:12:08 +00:00
Bryan C. Mills e49cb0208b os: clean up tests 
- Use testenv.Command instead of exec.Command to try to get more
  useful timeout behavior.

- Parallelize tests that appear not to require global state.
  (And add explanatory comments for a few that are not
  parallelizable for subtle reasons.)

- Consolidate some “Helper” tests with their parent tests.

- Use t.TempDir instead of os.MkdirTemp when appropriate.

- Factor out subtests for repeated test helpers.

For #36107.
Updates #22315.

Change-Id: Ic24b6957094dcd40908a59f48e44c8993729222b
Reviewed-on: https://go-review.googlesource.com/c/go/+/458015
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
 2023-01-19 20:45:37 +00:00
Andy Pan 86477e507f os: add a test case of copying a file itself via io.Copy 
Change-Id: Ib9746cb4f27625cb22620271b280d2da242b2fba
Reviewed-on: https://go-review.googlesource.com/c/go/+/428437
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
 2022-09-09 15:34:46 +00:00
John Bampton 6ba4a300d8 docs: fix spelling 
Change-Id: Ib689e5793d9cb372e759c4f34af71f004010c822
GitHub-Last-Rev: d63798388e
GitHub-Pull-Request: golang/go#44259
Reviewed-on: https://go-review.googlesource.com/c/go/+/291949
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
 2021-02-24 04:11:43 +00:00
Ian Lance Taylor 30641e36aa internal/poll: if copy_file_range returns 0, assume it failed 
On current Linux kernels copy_file_range does not correctly handle
files in certain special file systems, such as /proc. For those file
systems it fails to copy any data and returns zero. This breaks Go's
io.Copy for those files.

Fix the problem by assuming that if copy_file_range returns 0 the
first time it is called on a file, that that file is not supported.
In that case fall back to just using read. This will force an extra
system call when using io.Copy to copy a zero-sized normal file,
but at least it will work correctly.

For #36817
Fixes #44272

Change-Id: I02e81872cb70fda0ce5485e2ea712f219132e614
Reviewed-on: https://go-review.googlesource.com/c/go/+/291989
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
 2021-02-16 00:46:04 +00:00
Russ Cox 4f1b0a44cb all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTemp 
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)

Update the Go tree to use the preferred names.

As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.

ReadDir changes are in a separate CL, because they are not a
simple search and replace.

For #42026.

Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
 2020-12-09 19:12:23 +00:00
Joe Tsai 0844ff8eef os: fix regression with handling of nil *File 
Use of a nil *File as an argument should not result in a panic,
but result in the ErrInvalid error being returned.
Fix the copy_file_range implementation to preserve this semantic.

Fixes #40115

Change-Id: Iad5ac39664a3efb7964cf55685be636940a8db13
Reviewed-on: https://go-review.googlesource.com/c/go/+/241417
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
 2020-07-08 18:29:38 +00:00
Bryan C. Mills 13617380ca testing: clean up remaining TempDir issues from CL 231958 
Updates #38850

Change-Id: I33f48762f5520eb0c0a841d8ca1ccdd65ecc20c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/234583
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
 2020-05-19 19:55:14 +00:00
Andrei Tudor Călin 7be3f09deb os, internal/poll, internal/syscall/unix: use copy_file_range on Linux 
Linux 4.5 introduced (and Linux 5.3 refined) the copy_file_range
system call, which allows file systems the opportunity to implement
copy acceleration techniques. This commit adds support for
copy_file_range(2) to the os package.

Introduce a new ReadFrom method on *os.File, which makes *os.File
implement the io.ReaderFrom interface. If dst and src are both files,
this enables io.Copy(dst, src) to call dst.ReadFrom(src), which, in
turn, will call copy_file_range(2) if possible. If copy_file_range(2)
is not supported by the host kernel, or if either of dst or src
refers to a non-regular file, ReadFrom falls back to the regular
io.Copy code path.

Add internal/poll.CopyFileRange, which acquires locks on the
appropriate poll.FDs and performs the actual work, as well as
internal/syscall/unix.CopyFileRange, which wraps the copy_file_range
system call itself at the lowest level.

Rework file layout in internal/syscall/unix to accomodate the
additional system call numbers needed for copy_file_range.
Merge these definitions with the ones used by getrandom(2) into
sysnum_linux_$GOARCH.go files.

A note on additional optimizations: if dst and src both refer to pipes
in the invocation dst.ReadFrom(src), we could, in theory, use the
existing splice(2) code in package internal/poll to splice directly
from src to dst. Attempting this runs into trouble with the poller,
however. If we call splice(src, dst) and see EAGAIN, we cannot know
if it came from src not being ready for reading or dst not being
ready for writing. The write end of src and the read end of dst are
not under our control, so we cannot reliably use the poller to wait
for readiness. Therefore, it seems infeasible to use the new ReadFrom
method to splice between pipes directly. In conclusion, for now, the
only optimization enabled by the new ReadFrom method on *os.File is
the copy_file_range optimization.

Fixes #36817.

Change-Id: I696372639fa0cdf704e3f65414f7321fc7d30adb
Reviewed-on: https://go-review.googlesource.com/c/go/+/229101
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
 2020-04-28 00:59:36 +00:00