Commit Graph

881 Commits

Author SHA1 Message Date
Takeshi Yoneda 0835701224 os: fix TestChdirAndGetwd/ReaddirnamesOneAtATime on wasip1 to run on Windows hosts
TestReaddirnamesOneAtATime and TestChdirAndGetwd assumes the underlying file system
has /usr/bin but it is not the case when running it on WASI runtime hosted on Windows.

This change adds wasip1 in the special cased switch case to make them host OS agonstic.

Change-Id: Idb667021b565f939c814b9cd9e637cd75f9a610d
Reviewed-on: https://go-review.googlesource.com/c/go/+/489575
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2023-04-28 17:26:10 +00:00
Bryan C. Mills 9027e5d2b4 os/user: skip tests that invoke Current if it returns an expected error
Today Current may fail if the binary is not built with cgo
and USER and/or HOME is not set in the environment.
That should not cause the test to fail.

After this change,

	GOCACHE=$(go env GOCACHE) CGO_ENABLED=0 USER= HOME= go test os/user

now passes on linux/amd64.

For #59583.

Change-Id: Id290cd1088051e930d73f0dd554177124796e8f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/487015
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-21 15:59:43 +00:00
Alan Donovan 9d35ebba06 std: fix various nilness findings
Found by running
$ go run golang.org/x/tools/go/analysis/passes/nilness/cmd/nilness@latest std

No actual bugs--other than one panic(nil)--but a
few places where error nilness was unclear.

Change-Id: Ia916ba30f46f29c1bcf928cc62280169b922463a
Reviewed-on: https://go-review.googlesource.com/c/go/+/486675
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Alan Donovan <adonovan@google.com>
2023-04-20 01:07:29 +00:00
Bryan C. Mills 8c2ff88c61 os: check for ErrNotExist instead of ErrExist in TestNonPollable
An apparent typo in CL 484837 caused the test to check for ErrExist
instead of ErrNotExist when opening /dev/net/tun for read. That causes
the test to fail on platforms where /dev/net/ton does not exist,
such as on the darwin-amd64-longtest builder.

Updates #59545.

Change-Id: I9402ce0dba11ab459674e8358ae9a8b97eabc8d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/486255
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Commit-Queue: Bryan Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-19 14:12:49 +00:00
Ian Lance Taylor d5514013b6 runtime: add and use pollDesc fd sequence field
It is possible for a netpoll file to be closed and for the pollDesc
to be reused while a netpoll is running. This normally only causes
spurious wakeups, but if there is an error on the old file then the
new file can be incorrectly marked as having an error.

Fix this problem on most systems by introducing an fd sequence field
and using that as a tag in a taggedPointer. The taggedPointer is
stored in epoll or kqueue or whatever is being used. If the taggedPointer
returned by the kernel has a tag that does not match the fd
sequence field, the notification is for a closed file, and we
can ignore it. We check the tag stored in the pollDesc, and we also
check the tag stored in the pollDesc.atomicInfo.

This approach does not work on 32-bit systems where the kernel
only provides a 32-bit field to hold a user value. On those systems
we continue to use the older method without the sequence protection.
This is not ideal, but it is not an issue on Linux because the kernel
provides a 64-bit field, and it is not an issue on Windows because
there are no poller errors on Windows. It is potentially an issue
on *BSD systems, but on those systems we already call fstat in newFile
in os/file_unix.go to avoid adding non-pollable files to kqueue.
So we currently don't know of any cases that will fail.

Fixes #59545

Change-Id: I9a61e20dc39b4266a7a2978fc16446567fe683ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/484837
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Orlando Labao <orlando.labao43@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
2023-04-18 20:22:52 +00:00
Russ Cox 14ab998f95 cmd/go: add check for unknown godebug setting
A //go:debug line mentioning an unknown or retired setting
should be diagnosed as making the program invalid. Do that.
We agreed on this in the proposal but I forgot to implement it.

Change-Id: Ie69072a1682d4eeb6866c02adbbb426f608567c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/476280
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-18 13:19:19 +00:00
qmuntal de475e8a66 syscall: implement Fchdir on Windows
This CL adds support for os.File.Chdir() on Windows by implementing
syscall.Fchdir, which is internally used by Chdir.

Windows does not provide a function that sets the working directory
using a file handle, so we have to fallback to retrieving the file
handle path and then use it in SetCurrentDirectory.

Change-Id: I2ae93575e50411e5a9426ea531541958d7c9e812
Reviewed-on: https://go-review.googlesource.com/c/go/+/480135
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-04-12 09:13:54 +00:00
Tobias Klauser ef82b4c94e os: adjust wait6/waitid comment for netbsd
CL 431855 changed (*Process).blockUntilWaitable on netbsd to use wait6
again.

Update #48789

Change-Id: I948f5445a44ab2e82c02560480a2a244d2b5f473
Reviewed-on: https://go-review.googlesource.com/c/go/+/483396
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-04-11 13:16:52 +00:00
Johan Brandhorst-Satzkorn 66cac9e1e4 os: add wasip1 support
For #58141

Co-authored-by: Richard Musiol <neelance@gmail.com>
Co-authored-by: Achille Roussel <achille.roussel@gmail.com>
Co-authored-by: Julien Fabre <ju.pryz@gmail.com>
Co-authored-by: Evan Phoenix <evan@phx.io>
Change-Id: I52e3e161f81dcbe8605570e47d732992979c4d34
Reviewed-on: https://go-review.googlesource.com/c/go/+/479623
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-04-07 23:34:21 +00:00
Tobias Klauser 2f2b874b0a os/exec: skip remount in TestFindExecutableVsNoexec on EROFS
To allow using testenv.SyscallIsNotSupported, rewrite the test to use
the exported API only. Given that path is an absolute path,
exec.LookPath is equivalent to exec.findExecutable on linux.

Fixes #59087

Change-Id: Ia01b84d4e9d5a65a88dd995f9e3c8a81c4ccd19f
Reviewed-on: https://go-review.googlesource.com/c/go/+/481620
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Bypass: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-04-04 21:20:20 +00:00
Stefan Dombrowski fde1b139be os: don't check for IsExist in MkdirAll example
If a directory already exists, then MkdirAll returns nil. Therefore the
check with IsExist is not necessary.

Change-Id: Idf83c056f64bb56f49eb2b649af7827b759bcd7c
GitHub-Last-Rev: 1f29873d0c
GitHub-Pull-Request: golang/go#53242
Reviewed-on: https://go-review.googlesource.com/c/go/+/410434
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-29 22:44:59 +00:00
Ian Lance Taylor b441eb3f97 os: remove mixed dot- and non-dot- imports in tests
Some test files imported "os" twice, once with a dot and once without.
Consolidate on importing with a dot.

Change-Id: I1db31053dff9dee19a6bdfc263c7e7ef0c15ee42
Reviewed-on: https://go-review.googlesource.com/c/go/+/479995
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-29 16:21:13 +00:00
Daniel Martí 9768f736ea all: add a few links in package godocs
I noticed the one in path/filepath while reading the docs,
and the other ones were found via some quick grepping.

Change-Id: I386f2f74ef816a6d18aa2f58ee6b64dbd0147c9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/478795
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-25 10:16:23 +00:00
Heschi Kreinick e0c69587c4 Revert "os/user: lookup Linux users and groups via systemd userdb"
This reverts CL 459455.

Reason for revert: breaks tests on various platforms, see https://go-review.googlesource.com/c/go/+/459455/74#message-3d9462b24872f6e0b12b4abf5ea3983e1588f91a

Change-Id: I4c79b28f750c2369909688f86616d76d7eaf0ab4
Reviewed-on: https://go-review.googlesource.com/c/go/+/479135
Run-TryBot: Heschi Kreinick <heschi@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-23 22:09:58 +00:00
Ananth Bhaskararaman 1596d71255 os/user: lookup Linux users and groups via systemd userdb
Fetch usernames and groups via systemd userdb if available.
Otherwise fall back to parsing /etc/passwd, etc.

Fixes #38810

Co-authored-by: Michael Stapelberg <stapelberg@google.com>

Change-Id: Iff6ffc54feec6b6cec241b89e362c2285c8c0454
GitHub-Last-Rev: 1a627cc9a1
GitHub-Pull-Request: golang/go#57458
Reviewed-on: https://go-review.googlesource.com/c/go/+/459455
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-03-23 17:32:05 +00:00
Nont Thanonchai ed2442a0e1 os: avoid creating a new file in Truncate on Windows
Truncate() a non existent file on Windows currently creates a new blank
file. This behavior is not consistent with other OSes where a file not
found error would instead be returned. This change makes Truncate on
Windows return a file-not-found error when the specified file doesn't
exist, bringing the behavior consistent.

New test cases have been added to prevent a regression.

Fixes #58977

Change-Id: Iaf7b41fc4ea86a2b2ccc59f8be81be42ed211b5c
GitHub-Last-Rev: 636b6c37c1
GitHub-Pull-Request: golang/go#59085
Reviewed-on: https://go-review.googlesource.com/c/go/+/477215
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-03-22 07:02:33 +00:00
Ian Lance Taylor 4f5859c046 os: don't check for TTY before calling splice
I think I confused myself in CL 476335. The TTY check did fix the
problem with os.Stdout, but it was still possible to get the same
problem in other ways. I fixed that by making the splice call blocking,
but it turns out that doing that is enough to fix the TTY problem also.
So we can just remove the TTY check.

Fixes #59041

Change-Id: I4d7ca9dad8361001edb4cfa96bb29b1badb54df0
Reviewed-on: https://go-review.googlesource.com/c/go/+/477035
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-16 19:34:54 +00:00
Ian Lance Taylor be27fcfd2b os, internal/poll: don't use splice with tty
Also don't try to wait for a non-pollable FD.

Fixes #59041

Change-Id: Ife469d8738f2cc27c0beba223bdc8f8bc757b2a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/476335
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-15 22:01:36 +00:00
Ian Lance Taylor 491153a71a os, syscall: move rlimit code to syscall
In CL 393354 the os package was changed to raise the open file rlimit
at program start. That code is not inherently tied to the os package.
This CL moves it into the syscall package.

This is in preparation for future changes to restore the original
soft rlimit when exec'ing a new program.

For #46279

Change-Id: I981401b0345d017fd39fdd3dfbb58069be36c272
Reviewed-on: https://go-review.googlesource.com/c/go/+/476096
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-15 17:18:32 +00:00
Ian Lance Taylor 9ae97f8b8b os: don't hide all methods in recursive call to io.Copy
In order to avoid a recursive call to ReadFrom, we were converting
a *File to an io.Writer. But all we really need to do is hide
the ReadFrom method. In particular, this gives us the option of
adding a WriteTo method.

For #58808

Change-Id: I20d3a45749d528c93c23267c467e607fc17dc83f
Reviewed-on: https://go-review.googlesource.com/c/go/+/475535
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-15 00:12:37 +00:00
Tobias Klauser 93b8995594 os: correct func name in WriteFile godoc comment
Change-Id: Ideb70ce04f49ff676c20c2e1f0b43f1d7a6665dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/474355
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
2023-03-13 13:35:04 +00:00
Bryan C. Mills 53c91f9205 os: skip TestExecutableDeleted earlier when 'go build' is not supported
The test unconditionally calls testenv.GoToolPath, which will skip the
test anyway. Moving the skip earlier gets this test out of goroutine
dumps if the test process fails or times out, making it easier to
diagnose failures in the remaining tests.

Change-Id: Ibd39546708a83b6f15616b2c4ae7af420e2401f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/475455
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-10 21:27:06 +00:00
Bryan C. Mills 3f8f929d60 cmd/link/internal/ld: move more of mustLinkExternal into internal/platform
internal/platform.MustLinkExternal is used in various places to
determine whether external linking is required. It should always
match what the linker actually requires, but today does not match
because the linker imposes additional constraints.

Updates #31544.

Change-Id: I0cc6ad587e95c607329dea5d60d29a5fb2a9e722
Reviewed-on: https://go-review.googlesource.com/c/go/+/472515
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-03-02 16:34:21 +00:00
Andy Pan af2bc6de62 net,os: set the theoretical unlimited remaining bytes to max int64
Based on https://go-review.googlesource.com/c/go/+/466015/comment/073a63fa_7a9e485f

Change-Id: I3e1b035de6b8217c5fa5695e436f164b3058e33c
Reviewed-on: https://go-review.googlesource.com/c/go/+/471439
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-02-28 04:49:22 +00:00
pgxiaolianzi 81cd9ff7db all: fix typos
Change-Id: Ica8d5e5799a4de532764ae86cdb623508d3a8e18
GitHub-Last-Rev: 3e97cca9de
GitHub-Pull-Request: golang/go#58689
Reviewed-on: https://go-review.googlesource.com/c/go/+/471021
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-02-28 03:43:42 +00:00
Ian Lance Taylor 52af6550c9 os: don't try to put directory into non-blocking mode
Fixes #56843

Change-Id: I3cb3e8397499cd8c57a3edddd45f38c510519b36
Reviewed-on: https://go-review.googlesource.com/c/go/+/451997
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-27 23:05:46 +00:00
Andy Pan 08a68b73a4 os: skip zero-copy attempts with copy_file_range(2)/splice(2) for target files with O_APPEND flag
Change-Id: I6cccac9295ab4a9bf7f7a33382a34f31b1c4a000
Reviewed-on: https://go-review.googlesource.com/c/go/+/471496
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
2023-02-27 22:20:11 +00:00
Cuong Manh Le 0d52401e23 os: checking for testable network before creating local listener
Fixes #58745

Change-Id: Id6666477b2c25f081d6f86047cea12bf8b3cb679
Reviewed-on: https://go-review.googlesource.com/c/go/+/471495
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Andy Pan <panjf2000@gmail.com>
2023-02-27 11:56:55 +00:00
Andy Pan 31881f8787 os: ensure File.ReadFrom returns ErrClosed instead of the internal poll.ErrFileClosing
Fixes #58622

Change-Id: Ibb80296c39614478c75cb6bb04b6d0695cb990d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/469795
Run-TryBot: Andy Pan <panjf2000@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-02-27 06:21:27 +00:00
Andy Pan af6bb3f9b2 os: support zero-copy from TCP/Unix socket to file
Go currently supports the cases of zero-copy: from TCP/Unix socket to TCP socket, from file to TCP socket, from file to file.

Now implementing the new support of zero-copy: from TCP/Unix socket to file.

goos: linux
goarch: amd64
pkg: net
cpu: DO-Premium-Intel
                                  │      old      │                 new                  │
                                  │    sec/op     │    sec/op     vs base                │
SpliceFile/tcp-to-file/1024-4        5.910µ ±  9%   4.116µ ± 13%  -30.35% (p=0.000 n=10)
SpliceFile/tcp-to-file/2048-4        6.150µ ± 10%   4.077µ ± 13%  -33.72% (p=0.002 n=10)
SpliceFile/tcp-to-file/4096-4        4.837µ ± 28%   4.447µ ± 23%        ~ (p=0.353 n=10)
SpliceFile/tcp-to-file/8192-4        9.309µ ±  7%   6.293µ ±  9%  -32.40% (p=0.000 n=10)
SpliceFile/tcp-to-file/16384-4       19.43µ ± 12%   12.48µ ±  9%  -35.76% (p=0.000 n=10)
SpliceFile/tcp-to-file/32768-4       42.73µ ± 10%   25.32µ ±  8%  -40.76% (p=0.000 n=10)
SpliceFile/tcp-to-file/65536-4       70.37µ ± 11%   48.60µ ±  4%  -30.93% (p=0.000 n=10)
SpliceFile/tcp-to-file/131072-4     141.91µ ±  6%   96.24µ ±  4%  -32.18% (p=0.000 n=10)
SpliceFile/tcp-to-file/262144-4      329.7µ ±  8%   246.7µ ± 13%  -25.19% (p=0.000 n=10)
SpliceFile/tcp-to-file/524288-4      653.5µ ±  7%   441.6µ ±  7%  -32.43% (p=0.000 n=10)
SpliceFile/tcp-to-file/1048576-4    1184.4µ ±  9%   851.8µ ± 14%  -28.09% (p=0.000 n=10)
SpliceFile/unix-to-file/1024-4       1.734µ ± 10%   1.524µ ± 25%  -12.06% (p=0.035 n=10)
SpliceFile/unix-to-file/2048-4       2.614µ ±  7%   2.231µ ±  8%  -14.65% (p=0.000 n=10)
SpliceFile/unix-to-file/4096-4       5.081µ ±  7%   3.947µ ± 11%  -22.33% (p=0.000 n=10)
SpliceFile/unix-to-file/8192-4       8.560µ ±  5%   8.531µ ± 17%        ~ (p=0.796 n=10)
SpliceFile/unix-to-file/16384-4      18.09µ ± 12%   12.92µ ± 25%  -28.59% (p=0.000 n=10)
SpliceFile/unix-to-file/32768-4      35.50µ ±  5%   24.50µ ±  6%  -31.00% (p=0.000 n=10)
SpliceFile/unix-to-file/65536-4      69.99µ ±  7%   51.22µ ± 23%  -26.82% (p=0.000 n=10)
SpliceFile/unix-to-file/131072-4     133.7µ ± 17%   119.7µ ±  6%  -10.43% (p=0.000 n=10)
SpliceFile/unix-to-file/262144-4     246.5µ ±  5%   207.3µ ± 19%  -15.90% (p=0.007 n=10)
SpliceFile/unix-to-file/524288-4     484.8µ ± 20%   382.9µ ± 10%  -21.02% (p=0.000 n=10)
SpliceFile/unix-to-file/1048576-4   1188.4µ ± 27%   781.8µ ± 11%  -34.21% (p=0.000 n=10)
geomean                              42.24µ         31.45µ        -25.53%

                                  │      old       │                  new                   │
                                  │      B/s       │      B/s        vs base                │
SpliceFile/tcp-to-file/1024-4        165.4Mi ± 10%    237.3Mi ± 11%  +43.47% (p=0.000 n=10)
SpliceFile/tcp-to-file/2048-4        317.6Mi ± 12%    479.7Mi ± 14%  +51.02% (p=0.002 n=10)
SpliceFile/tcp-to-file/4096-4        808.2Mi ± 22%    886.8Mi ± 19%        ~ (p=0.353 n=10)
SpliceFile/tcp-to-file/8192-4        839.3Mi ±  6%   1241.5Mi ±  8%  +47.91% (p=0.000 n=10)
SpliceFile/tcp-to-file/16384-4       804.7Mi ± 13%   1252.2Mi ± 10%  +55.61% (p=0.000 n=10)
SpliceFile/tcp-to-file/32768-4       731.3Mi ± 11%   1234.3Mi ±  7%  +68.78% (p=0.000 n=10)
SpliceFile/tcp-to-file/65536-4       888.7Mi ± 10%   1286.2Mi ±  4%  +44.73% (p=0.000 n=10)
SpliceFile/tcp-to-file/131072-4      880.9Mi ±  6%   1299.0Mi ±  4%  +47.47% (p=0.000 n=10)
SpliceFile/tcp-to-file/262144-4      758.2Mi ±  7%   1014.4Mi ± 15%  +33.78% (p=0.000 n=10)
SpliceFile/tcp-to-file/524288-4      765.3Mi ±  7%   1132.5Mi ±  7%  +47.99% (p=0.000 n=10)
SpliceFile/tcp-to-file/1048576-4     845.0Mi ±  8%   1174.0Mi ± 16%  +38.94% (p=0.000 n=10)
SpliceFile/unix-to-file/1024-4       564.2Mi ± 11%    640.5Mi ± 20%  +13.53% (p=0.035 n=10)
SpliceFile/unix-to-file/2048-4       747.4Mi ±  7%    875.7Mi ±  8%  +17.17% (p=0.000 n=10)
SpliceFile/unix-to-file/4096-4       768.8Mi ±  6%    989.8Mi ± 10%  +28.74% (p=0.000 n=10)
SpliceFile/unix-to-file/8192-4       912.9Mi ±  5%    915.8Mi ± 15%        ~ (p=0.796 n=10)
SpliceFile/unix-to-file/16384-4      863.6Mi ± 10%   1209.7Mi ± 20%  +40.06% (p=0.000 n=10)
SpliceFile/unix-to-file/32768-4      880.2Mi ±  6%   1275.7Mi ±  6%  +44.93% (p=0.000 n=10)
SpliceFile/unix-to-file/65536-4      893.0Mi ±  7%   1220.3Mi ± 19%  +36.66% (p=0.000 n=10)
SpliceFile/unix-to-file/131072-4     935.1Mi ± 14%   1043.9Mi ±  7%  +11.64% (p=0.000 n=10)
SpliceFile/unix-to-file/262144-4    1014.2Mi ±  6%   1205.9Mi ± 16%  +18.91% (p=0.007 n=10)
SpliceFile/unix-to-file/524288-4     1.007Gi ± 17%    1.275Gi ±  9%  +26.61% (p=0.000 n=10)
SpliceFile/unix-to-file/1048576-4    841.8Mi ± 21%   1279.0Mi ± 10%  +51.94% (p=0.000 n=10)
geomean                              740.1Mi          994.2Mi        +34.33%

                                  │      old       │                   new                   │
                                  │      B/op      │    B/op     vs base                     │
SpliceFile/tcp-to-file/1024-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/2048-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/4096-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/8192-4       0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/16384-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/32768-4      1.000 ±   0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/65536-4      1.000 ± 100%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/131072-4     3.000 ±  33%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/262144-4     8.500 ±  18%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/524288-4     16.50 ±  21%      0.00 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/tcp-to-file/1048576-4    30.50 ±  15%      0.00 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/1024-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/2048-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/4096-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/8192-4      0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/16384-4     0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/32768-4     0.000 ±   0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/65536-4     1.000 ± 100%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/131072-4    3.000 ±  33%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/262144-4    6.000 ±  17%     0.000 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/524288-4    12.00 ±   8%      0.00 ± 0%  -100.00% (p=0.000 n=10)
SpliceFile/unix-to-file/1048576-4   33.50 ±  10%      0.00 ± 0%  -100.00% (p=0.000 n=10)
geomean                                          ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                                  │     old      │                 new                 │
                                  │  allocs/op   │ allocs/op   vs base                 │
SpliceFile/tcp-to-file/1024-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/2048-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/4096-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/8192-4       0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/16384-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/32768-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/65536-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/131072-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/262144-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/524288-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/tcp-to-file/1048576-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/1024-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/2048-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/4096-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/8192-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/16384-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/32768-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/65536-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/131072-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/262144-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/524288-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
SpliceFile/unix-to-file/1048576-4   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                        ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Change-Id: Ie7f7d4d7b6b373d9ee7ce6da8f6a4cd157632486
Reviewed-on: https://go-review.googlesource.com/c/go/+/466015
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-02-27 00:12:08 +00:00
cui fliter dd87e33d88 os: use testenv.Command instead of exec.Command in tests
testenv.Command sets a default timeout based on the test's deadline
and sends SIGQUIT (where supported) in case of a hang.

Change-Id: I84d469d12ca1f39175b713ca41e7cd2202d11d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/451895
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-02-21 23:07:55 +00:00
cui fliter 117fe2e08d all: fix some comments
Change-Id: I16ec916b47de2f417b681c8abff5a1375ddf491b
Reviewed-on: https://go-review.googlesource.com/c/go/+/468055
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-02-14 16:27:39 +00:00
Bryan C. Mills bb4ea80bef os: allow $HOME to not exist in TestUserHomeDir
This seems like a cleaner fix for the situation Debian has patched
around with
https://sources.debian.org/patches/golang-1.19/1.19.3-1/0001-Disable-test-for-UserHomeDir.patch/

Tested with 'GOCACHE=$(go env GOCACHE) HOME=/home/nobody go test os'.

Change-Id: I9fd00da38b86df2edfcd8cb87629e2875573903e
Reviewed-on: https://go-review.googlesource.com/c/go/+/456126
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-31 18:47:06 +00:00
Thanonchai W 26fd1fba12 os: use GetTempPath2 on Windows if available
This generates GetTempPath2. Go now tries to determine if the windows it runs on has GetTempPath2 by finding it only once at the loading time. If GetTempPath2 exists, it sets the flag so that any calls to tempDir will use it. If it doesn't exist, Go then uses GetTempPath.

GetTempPath2 was generated into internal/syscall/windows since syscall is locked down.

Fixes #56899

Change-Id: Iff08502aebc787fde802ee9496c070c982fbdc08
GitHub-Last-Rev: b779389534
GitHub-Pull-Request: golang/go#57980
Reviewed-on: https://go-review.googlesource.com/c/go/+/463219
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-01-28 19:27:04 +00:00
Daniela Petruzalek 1eb37facdd os: fix findOneDriveDir to expand strings that contain environment variables
On Windows the registry data type REG_EXPAND_SZ indicates that the string requires expansion
of environment variables. The existing implementation doesn't take that into consideration
and just returns the unexpanded string, ignoring the registry type. This implementation ensures
that environment variables are properly expanded when needed.

Fixes #57576

Change-Id: Ia02c1b05a4cf6eaaffb3be88ce1c9ee100db250f
Reviewed-on: https://go-review.googlesource.com/c/go/+/460535
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-28 18:50:46 +00:00
Bryan C. Mills 6e82febaf0 os: eliminate arbitrary timeout in testClosewithBlockingRead
The 1-second timeout on execution of this test is empirically too
short on some platforms. Rather than trying to tune the timeout, allow
the test to time out on its own (and dump goroutines) if it deadlocks.

Fixes #57993.
Fixes #57994.

Change-Id: I69ee86c75034469e4b4cd391b8dc5616b93468b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/463180
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-01-25 20:45:53 +00:00
Bryan C. Mills 1d3088effd os: make Lstat for symlinks on Windows consistent with POSIX
This also makes path/filepath.Walk more consistent between
Windows and POSIX platforms.

According to
https://pubs.opengroup.org/onlinepubs/9699919799.2013edition/basedefs/V1_chap04.html#tag_04_12
symlinks in a path that includes a trailing slash must be resolved
before a function acts on that path.

POSIX defines an lstat function, whereas the Win32 API does not, so
Go's os.Lstat should follow the (defined) POSIX semantics instead of
doing something arbitrarily different.

CL 134195 added a test for the correct POSIX behavior when os.Lstat is
called on a symlink. However, the test turned out to be broken on Windows,
and when it was fixed (in CL 143578) it was fixed with different Lstat
behavior on Windows than on all other platforms that support symlinks.

In #50807 we are attempting to provide consistent symlink behavior for
cmd/go. This unnecessary platform difference, if left uncorrected,
will make that fix much more difficult.

CL 460595 reworked the implementation of Stat and Lstat on Windows,
and with the new implementation this fix is straightforward.

For #50807.
Updates #27225.

Change-Id: Ia28821aa4aab6cefa021da2d9b803506cdb2621b
Reviewed-on: https://go-review.googlesource.com/c/go/+/463177
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-01-25 16:38:21 +00:00
Nigel Tao a11b8e3765 os: have RemoveAll loop on EINTR
Fixes #57966

Change-Id: Ia732d499ff9bd6e70030daab8fac42d1e204be37
Reviewed-on: https://go-review.googlesource.com/c/go/+/463076
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Nigel Tao <nigeltao@golang.org>
Reviewed-by: Nigel Tao (INACTIVE; USE @golang.org INSTEAD) <nigeltao@google.com>
2023-01-25 07:07:01 +00:00
qmuntal 1951857ec0 os: use handle based APIs to read directories on windows
This CL updates File.readdir() on windows so it uses
GetFileInformationByHandleEx with FILE_ID_BOTH_DIR_INFO
instead of Find* APIs. The former is more performant because
it allows us to buffer IO calls and reduces the number of system calls,
passing from 1 per file to 1 every ~100 files
(depending on the size of the file name and the size of the buffer).

This change improve performance of File.ReadDir by 20-30%.

name        old time/op    new time/op    delta
ReadDir-12     562µs ±14%     385µs ± 9%  -31.60%  (p=0.000 n=9+9)

name        old alloc/op   new alloc/op   delta
ReadDir-12    29.7kB ± 0%    29.5kB ± 0%   -0.88%  (p=0.000 n=8+10)

name        old allocs/op  new allocs/op  delta
ReadDir-12       399 ± 0%       397 ± 0%   -0.50%  (p=0.000 n=10+10)

This change also speeds up calls to os.SameFile when using FileStats
returned from File.readdir(), as their file ID can be inferred while
reading the directory.

Change-Id: Id56a338ee66c39656b564105cac131099218fb5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/452995
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-01-24 13:26:00 +00:00
Bryan C. Mills 3e44b7d07a os: treat non-symlink reparse points as irregular files
Prior to this change (as of CL 143578), our stat function attempted to
resolve all reparse points as if they were symlinks.

This results in an additional call to CreateFile when statting a
symlink file: we use CreateFile once to obtain the reparse tag and
check whether the file is actually a symlink, and if it is we call
CreateFile again without FILE_FLAG_OPEN_REPARSE_POINT to stat the link
target. Fortunately, since symlinks are rare on Windows that overhead
shouldn't be a big deal in practice.

Fixes #42919.

Change-Id: If453930c6e98040cd6525ac4aea60a84498c9579
Reviewed-on: https://go-review.googlesource.com/c/go/+/460595
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-01-23 22:46:55 +00:00
Russ Cox 213495a4a6 internal/godebug: export non-default-behavior counters in runtime/metrics
Allow GODEBUG users to report how many times a setting
resulted in non-default behavior.

Record non-default-behaviors for all existing GODEBUGs.

Also rework tests to ensure that runtime is in sync with runtime/metrics.All,
and generate docs mechanically from metrics.All.

For #56986.

Change-Id: Iefa1213e2a5c3f19ea16cd53298c487952ef05a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/453618
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-19 22:26:43 +00:00
Bryan C. Mills e49cb0208b os: clean up tests
- Use testenv.Command instead of exec.Command to try to get more
  useful timeout behavior.

- Parallelize tests that appear not to require global state.
  (And add explanatory comments for a few that are not
  parallelizable for subtle reasons.)

- Consolidate some “Helper” tests with their parent tests.

- Use t.TempDir instead of os.MkdirTemp when appropriate.

- Factor out subtests for repeated test helpers.

For #36107.
Updates #22315.

Change-Id: Ic24b6957094dcd40908a59f48e44c8993729222b
Reviewed-on: https://go-review.googlesource.com/c/go/+/458015
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-01-19 20:45:37 +00:00
Bryan C. Mills f2884bf423 os: deflake TestPipeEOF and TestFifoEOF
- Consolidate the two test bodies as one helper function.
- Eliminate arbitrary timeout.
- Shorten arbitrary sleeps in short mode.
- Simplify goroutines.
- Mark the tests as parallel.

Fixes #36107.
Updates #24164.

Change-Id: I14fe4395963a7256cb6d2d743d348a1ade077d5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/457336
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-01-19 20:45:34 +00:00
Alan Donovan c0799f7015 os: document that Rename is not atomic on non-Unix platforms
Windows provides no reliable way to rename files atomically.

The Plan 9 implementation of os.Rename performs a deletion
if the target exists.

Change-Id: Ife5f9c97b21f48c11e300cd76d8c7f715db09fd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/462395
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Alan Donovan <adonovan@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-01-17 18:12:07 +00:00
Bryan C. Mills 82f09b75ca os/exec: avoid leaking an exec.Cmd in TestWaitInterrupt
In CL 436655 I added a GODEBUG setting to this test process to verify
that Wait is eventually called for every exec.Cmd before it becomes
unreachable. However, the cmdHang test helpers in
TestWaitInterrupt/Exit-hang and TestWaitInterrupt/SIGKILL-hang
intentially leak a subprocess in order to simulate a leaky third-party
program, as Go users might encounter in practical use.

To avoid tripping over the leak check, we call Wait on the leaked
subprocess in a background goroutine. Since we expect the process
running cmdHang to exit before its subprocess does, the call to Wait
should have no effect beyond suppressing the leak check.

Fixes #57596.
Updates #52580.
Updates #50436.

Change-Id: Ia4b88ea47fc6b605c27ca6d9d7669c874867a900
Reviewed-on: https://go-review.googlesource.com/c/go/+/460998
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-10 17:59:34 +00:00
qmuntal a5a4744250 os: reenable TestReaddirSmallSeek on windows
TestReaddirSmallSeek should have been reenabled as part of
CL 405275, but didn't. Do it now.

Updates #36019

Change-Id: I5676eee4e63675d30e9d48ac708e72bd036b6aee
Reviewed-on: https://go-review.googlesource.com/c/go/+/458336
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2022-12-19 21:17:00 +00:00
Than McIntosh 8bcc490667 os/user,net: add -fno-stack-protector to CFLAGS
Some compilers default to having -fstack-protector on, which breaks
when using internal linking because the linker doesn't know how to
find the support functions.

Updates #52919.
Updates #54313.
Fixes #57261.

Change-Id: Iaae731851407af4521fff2dfefc5b7e3e92cf284
Reviewed-on: https://go-review.googlesource.com/c/go/+/456855
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-12-16 17:45:51 +00:00
Bryan C. Mills 3323dab1f4 os/exec: retry ETXTBSY errors in TestFindExecutableVsNoexec
I made this test parallel in CL 439196, which exposed it to the
fork/exec race condition described in #22315. The ETXTBSY errors from
that race should resolve on their own, so we can simply retry the call
to get past them.

Fixes #56811.
Updates #22315.

Change-Id: I2c6aa405bf3a1769d69cf08bf661a9e7f86440b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/458016
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-12-16 03:04:44 +00:00
Ian Lance Taylor 9b8750f53e os: skip size test in TestLstat if the file is a symlink
Tested by temporarily changing sysdir to use a directory where
the expected files were all symlinks. We should consider using
a different approach that doesn't rely on sysdir, but for now
do a minimal fix.

Fixes #57210

Change-Id: Ifb1becef03e014ceb48290ce13527b3e103c0e07
Reviewed-on: https://go-review.googlesource.com/c/go/+/456557
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-12-10 00:35:24 +00:00
Bryan C. Mills 80f7484af7 os/user: zero-initialize C structs returned to Go
In the wrappers for getgrnam_r and similar, the structs to be returned
are allocated on the C stack and may be uninitialized. If the call to
the wrapped C function returns an error (such as ERANGE), it may leave
the struct uninitialized, expecting that the caller will not read it.

However, when that struct is returned to Go, it may be read by the Go
garbage collector. If the uninitialized struct fields happen to
contain wild pointers, the Go garbage collector will throw an error.
(Prior to CL 449335, the Go runtime would not scan the struct fields
because they did not reside in Go memory.)

Fix this by always zeroing the struct before the C call.

Fixes #57170.

Change-Id: I241ae8e4added6f9a406dac37a7f6452341aa0cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/456121
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2022-12-09 04:05:17 +00:00