Commit Graph

990 Commits

Author SHA1 Message Date
cui fliter 1e12eab870 all: fix a large number of comments
Partial typo corrections, following https://go.dev/wiki/Spelling

Change-Id: I2357906ff2ea04305c6357418e4e9556e20375d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/573776
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-26 19:58:28 +00:00
qmuntal 051e99292f os: reuse buffer pool more aggressively in readdir
We can reuse the buffer pool more aggressively when reading a directory
by returning the buffer to the pool as soon as we get to the end of the
directory, rather than waiting until the the os.File is closed.

This yields a significant memory usage reduction when traversing
nested directories recursively via os.File#ReadDir (and friends),
as the file pointers tends to be closed only after the entire
traversal is done. For example, this pattern is used in os.RemoveAll.
These are the improvements observed in BenchmarkRemoveAll:

goos: linux
goarch: amd64
pkg: os
cpu: AMD EPYC 7763 64-Core Processor
            │   old.txt   │            new.txt            │
            │   sec/op    │   sec/op     vs base          │
RemoveAll-4   3.847m ± 2%   3.823m ± 1%  ~ (p=0.143 n=10)

            │   old.txt    │               new.txt                │
            │     B/op     │     B/op      vs base                │
RemoveAll-4   39.77Ki ± 2%   17.63Ki ± 1%  -55.68% (p=0.000 n=10)

            │  old.txt   │              new.txt              │
            │ allocs/op  │ allocs/op   vs base               │
RemoveAll-4   510.0 ± 0%   503.0 ± 0%  -1.37% (p=0.000 n=10)

Change-Id: I70e1037378a02f1d670ccb7b275ee55f0caa6d0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/573358
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com>
2024-03-25 19:21:29 +00:00
guoguangwu 25aa45afcf os: fix typo in test
Change-Id: Ib445940f0f24385dde1b62277c8083e6369d0645
GitHub-Last-Rev: 02c18c2777
GitHub-Pull-Request: golang/go#66503
Reviewed-on: https://go-review.googlesource.com/c/go/+/574016
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Auto-Submit: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2024-03-25 15:26:11 +00:00
Joel Sing fba54f6345 os/signal: avoid calling ioctl via syscall.Syscall on BSDs
Provide appropriate implementations of internal/syscall/unix.Tcsetpgrp
and use this for runSessionLeader in os/signal/signal_cgo_test.go.
This avoids calling syscall.Syscall with SYS_IOCTL on BSDs.

Updates #59667
Updates #63900

Change-Id: Ifa4696bba9f1eb68e81e7103f030bc254adaf0af
Reviewed-on: https://go-review.googlesource.com/c/go/+/540020
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Joel Sing <joel@sing.id.au>
2024-03-20 10:09:15 +00:00
Andy Pan b750841906 os: kick FIFOs with O_NONBLOCK out of the kqueue on Darwin/iOS
Fixes #66239

Change-Id: I8210682c0cf4285b950e9fabe687b7ad2369835c
Reviewed-on: https://go-review.googlesource.com/c/go/+/570397
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2024-03-19 11:47:23 +00:00
Andy Pan 207511a0d4 os: use ignoringEINTR in openFileNolog and openDirNolog
Change-Id: Ie8fa25d5e326efd7d3c9b72203783110d9e22ce8
Reviewed-on: https://go-review.googlesource.com/c/go/+/572215
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-19 11:34:18 +00:00
qmuntal 2e8d84f148 os: support UNC paths and .. segments in fixLongPath
This CL reimplements fixLongPath using syscall.GetFullPathName instead
of a custom implementation that was not handling UNC paths and ..
segments correctly. It also fixes a bug here multiple trailing \
were removed instead of replaced by a single one.

The new implementation is slower than the previous one, as it does a
syscall and needs to convert UTF-8 to UTF-16 (and back), but it is
correct and should be fast enough for most use cases.

goos: windows
goarch: amd64
pkg: os
cpu: Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
            │   old.txt    │                new.txt                 │
            │    sec/op    │    sec/op      vs base                 │
LongPath-12   1.007µ ± 53%   4.093µ ± 109%  +306.41% (p=0.000 n=10)

            │  old.txt   │               new.txt                │
            │    B/op    │    B/op      vs base                 │
LongPath-12   576.0 ± 0%   1376.0 ± 0%  +138.89% (p=0.000 n=10)

            │  old.txt   │              new.txt               │
            │ allocs/op  │ allocs/op   vs base                │
LongPath-12   2.000 ± 0%   3.000 ± 0%  +50.00% (p=0.000 n=10)

Fixes #41734.

Change-Id: Iced5cf47f56f6ab0ca74a6e2374c31a75100902d
Reviewed-on: https://go-review.googlesource.com/c/go/+/570995
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-18 18:31:42 +00:00
Peter Collingbourne b822f098c5 os: don't try to make the directory FD non-blocking in os.ReadDir
This will fail because epoll_ctl() fails on directory FDs, so we
end up issuing unnecessary syscalls. My test program that calls
filepath.WalkDir on a large directory tree runs 1.23 ± 0.04 times
faster than with the original implementation.

Change-Id: Ie33d798c48057a7b2d0bacac80fcdde5b5a8bb1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/570877
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-16 00:34:52 +00:00
qmuntal 418e6d559e os,internal/godebugs: add missing IncNonDefault calls
Fixes #66215

Change-Id: Id7de15feabe08f66c048dc114c09494813c9febc
Reviewed-on: https://go-review.googlesource.com/c/go/+/570695
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-13 19:10:34 +00:00
guoguangwu 3e1a6925a1 os: close pipe in test
Change-Id: Ic8b06c6fd9fc6a30b26f4e4614aa40b5cad3a5e7
GitHub-Last-Rev: 8397a8b30c
GitHub-Pull-Request: golang/go#66240
Reviewed-on: https://go-review.googlesource.com/c/go/+/570515
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-03-11 17:08:40 +00:00
Tobias Klauser 80a91d34f4 os: use goarch.BigEndian
Change-Id: I83c23ae0933f6abe4c07144f69c3d9c18aece6e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/569175
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-03-08 01:59:11 +00:00
qmuntal 5e387b2fee os: fix 63703.md release notes
63703.md contains a paragraph that shouldn't be there,
remove it.

While here, fix a test error message related to the #63703
implementation.

Updates #63703.

Change-Id: I82a8b0b7dfa8f96530fb9a3a3aa971e03970f168
Reviewed-on: https://go-review.googlesource.com/c/go/+/569195
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-05 16:20:15 +00:00
qmuntal b09ac10bad os: don't normalize volumes to drive letters in os.Readlink
This CL updates os.Readlink so it no longer tries to normalize volumes
to drive letters, which was not always even possible.

This behavior is controlled by the `winreadlinkvolume` setting.
For Go 1.23, it defaults to `winreadlinkvolume=1`.
Previous versions default to `winreadlinkvolume=0`.

Fixes #63703.

Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: Icd6fabbc8f0b78e23a82eef8db89940e89e9222d
Reviewed-on: https://go-review.googlesource.com/c/go/+/567735
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-04 20:38:54 +00:00
qmuntal 7986e26a39 os: don't treat mount points as symbolic links
This CL changes the behavior of os.Lstat to stop setting the
os.ModeSymlink type mode bit for mount points on Windows. As a result,
filepath.EvalSymlinks no longer evaluates mount points, which was the
cause of many inconsistencies and bugs.

Additionally, os.Lstat starts setting the os.ModeIrregular type mode bit
for all reparse tags on Windows, except for those that are explicitly
supported by the os package, which, since this CL, doesn't include mount
points. This helps to identify files that need special handling outside
of the os package.

This behavior is controlled by the `winsymlink` GODEBUG setting.
For Go 1.23, it defaults to `winsymlink=1`.
Previous versions default to `winsymlink=0`.

Fixes #39786
Fixes #40176
Fixes #61893
Updates #63703
Updates #40180
Updates #63429

Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-windows-arm64
Change-Id: I2e7372ab8862f5062667d30db6958d972bce5407
Reviewed-on: https://go-review.googlesource.com/c/go/+/565136
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-03-04 18:59:50 +00:00
guoguangwu 1f1fcc3473 os/exec: remove unnecessary fmt.Sprintf call
Change-Id: Ic0ac97a15dadd756d727fd8abe23359b0347af19
GitHub-Last-Rev: a96a3f5fe7
GitHub-Pull-Request: golang/go#66052
Reviewed-on: https://go-review.googlesource.com/c/go/+/568317
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-04 15:53:53 +00:00
qiulaidongfeng b414d044ff os: remove if nil!=nil in openFileNolog
Change-Id: I21cecc13570f3b61f3d6c4fede18dc63ddca1b69
GitHub-Last-Rev: 3c351e4aa8
GitHub-Pull-Request: golang/go#65958
Reviewed-on: https://go-review.googlesource.com/c/go/+/567355
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
2024-03-04 13:37:12 +00:00
qmuntal ec8833ecc1 os: factor out newFileStatFromWin32FileAttributeData
The stat function is quite long on Windows. Simplify it a bit by
factoring out the creation of a fileStat from a Win32FileAttributeData.

This also makes it more consistent with the creation of fileStats
from other sources, which all have their own dedicated functions.

Change-Id: I0443f96d892b70ce7f3b5e92c5049e4e4a240c6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/566435
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-28 20:27:13 +00:00
qmuntal 49319ed5c7 os: avoid GetFileInformationByHandleEx call when stat'ing files on Windows
os.Stat and os.Lstat on Windows use GetFileInformationByHandleEx to
retrieve file information for reparse points and files that
GetFileAttributesEx does not handle.

However, GetFileInformationByHandleEx is only necessary for
reparse points, so we can avoid the call for regular files.

With this change we can drop the FAT hack that was added in CL 154377,
as files won't have the FILE_ATTRIBUTE_REPARSE_POINT attribute set
on that file system.

Change-Id: Id18639067a6c3fa1bb2c6706d5b79358c224fe37
Reviewed-on: https://go-review.googlesource.com/c/go/+/566397
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2024-02-27 16:41:24 +00:00
Alan Donovan f8b4653500 std: fix more nilness findings
(found with x/tools/go/analysis/passes/nilness)

Change-Id: I1bdc7811efbecea95608e634f894cb6c656e3a5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/564221
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-26 22:01:11 +00:00
cui fliter 73847aa41b os: add available godoc link
Change-Id: I430c9a7c4936d7a8c8c787aa63de9a796d20fdf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/539597
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-02-26 21:33:12 +00:00
Damien Neil e596e88318 path/filepath: add Localize
Add the Localize function, which takes an io/fs slash-separated path
and returns an operating system path.

Localize returns an error if the path cannot be represented on
the current platform.

Replace internal/safefile.FromFS with Localize,
which serves the same purpose as this function.

The internal/safefile package remains separate from path/filepath
to avoid a dependency cycle with the os package.

Fixes #57151

Change-Id: I75c88047ddea17808276761da07bf79172c4f6fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/531677
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-02-26 18:08:14 +00:00
Michael Pratt e58486e599 Revert "os: make use of pidfd on linux"
This reverts CL 528438.

Reason for revert: Implicated in "bad FD" test failures. Full extent of
issue still unclear.

For #62654.
Fixes #65857.

Change-Id: I066e38040544c506917e90255bd0e330964a0276
Reviewed-on: https://go-review.googlesource.com/c/go/+/566477
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-23 18:31:19 +00:00
Michael Pratt c4e4afc90e Revert "os: make FindProcess use pidfd on Linux"
This reverts CL 542699.

Reason for revert: Some applications assume FindProcess does not return
errors.

For #62654.
Fixes #65866.

Change-Id: Ic185a6253c8e508b08150b618c39a9905f6cdd60
Reviewed-on: https://go-review.googlesource.com/c/go/+/566476
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
2024-02-23 18:29:45 +00:00
Andy Pan 27e104ba09 os: avoid writing within the tree during testing CopyFS
Change-Id: I1d3e7c100ddfbd2ea9a7bb9e8efae89893c8365a
Reviewed-on: https://go-review.googlesource.com/c/go/+/566335
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Andy Pan <panjf2000@gmail.com>
2024-02-23 05:09:01 +00:00
Andy Pan d9be60974b os: implement CopyFS
Fixes #62484

Change-Id: I5d8950dedf86af48f42a641940b34e62aa2cddcb
Reviewed-on: https://go-review.googlesource.com/c/go/+/558995
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-23 00:19:54 +00:00
Kir Kolyshkin cdf3249d74 os: make FindProcess use pidfd on Linux
Amend FindProcess to use pidfdFind, and make it return ErrProcessDone
if pidfdFind is used and the process is not found.

Since this is a change in API, introduce GODEBUG osfinderr=0 setting
to disable the feature.

Change-Id: I724c6f622f0c99f21a70b864cf7cf2b8836869ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/542699
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2024-02-21 21:27:03 +00:00
Kir Kolyshkin 750738b5d1 os: make use of pidfd on linux
Use Process.handle field to store pidfd, and make use of it. Only use
pidfd functionality if all the needed syscalls are available.

1. StartProcess: obtain the pidfd from the kernel, if available,
   using the functionality added by CL 520266. Note we could not modify
   syscall.StartProcess to return pidfd directly because it is a public
   API and its callers do not expect it, so we have to use ensurePidfd
   and getPidfd.

2. (*Process).Kill: use pidfdSendSignal, if the syscall is available
   and pidfd is known. This is slightly more complicated than it should
   be, since the syscall can be blocked by e.g. seccomp security policy,
   therefore the need for a function to check if it's actually working,
   and a soft fallback to kill. Perhaps this precaution is not really
   needed.

3. (*Process).Wait: use pidfdWait, if available, otherwise fall back to
   using waitid/wait4. This is also more complicated than expected due
   to struct siginfo_t idiosyncrasy.

NOTE pidfdSendSignal and pidfdWait are used without a race workaround
(blockUntilWaitable and sigMu, added by CL 23967) because with pidfd,
PID recycle issue doesn't exist (IOW, pidfd, unlike PID, is guaranteed
to refer to one particular process) and thus the race doesn't exist
either.

For #62654.
Updates #13987.

Change-Id: I22ebcc7142b16a3a94c422d2f32504d1a80e8a8f
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/528438
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-21 21:26:00 +00:00
Kir Kolyshkin 4cd743e27e os: use atomic.Uintptr for Process.handle
Suggested-by: Michael Knyszek <mknyszek@google.com>
Change-Id: I116731b6c3738aae8ff1d3be227f8f51fa3320c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/544795
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Kirill Kolyshkin <kolyshkin@gmail.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-02-21 21:23:31 +00:00
qmuntal 628b1015b9 os: add os.ModeSocket support to os.Stat on windows
Unix sockets are identified by the IO_REPARSE_TAG_AF_UNIX reparse tag.

Teach fileStat.Mode() to recognize this tag and set the os.ModeSocket
bit in such case.

Note that there is a bug starting in Windows 19H1 until 20H1 that
makes the IO_REPARSE_TAG_AF_UNIX tag not being set for unix sockets.
This CL doesn't provide a workaround for this bug.

Fixes #33357.

Change-Id: Iea8f24b20672c8d4b03f55ef298d128431dc3fac
Reviewed-on: https://go-review.googlesource.com/c/go/+/561937
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2024-02-12 07:15:06 +00:00
qmuntal c098400548 runtime,internal/syscall/windows: remove long path support check
The runtime currently enables long path support process-wide by updating
the process environment block (PEB). It then tries to create a file
using a long path to check if the PEB update made any difference.

There hasn't been any report that the PEB update was not effective,
and the check itself is quite tricky, so it's time to remove it.

While here, linkname `runtime.canUseLongPaths` to a variable in
internal/syscall/windows instead of the os package so it is easier to
consume from other packages.

Change-Id: I549380b7f2c242dc4db20d5be603840282de69b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/536495
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: David Chase <drchase@google.com>
2024-02-07 20:54:38 +00:00
qmuntal 28b8851671 os: support Stat and LStat for CON device on Windows
\\.\con and CON need to be opened with GENERIC_READ access, else
CreateFile will fail with ERROR_INVALID_PARAMETER.

Special-case ERROR_INVALID_PARAMETER in os.[L]Stat so it retries with
GENERIC_READ access.

Fixes #34900.

Change-Id: I5010e736d0189c8ada4fc0eca98d71a438c41426
Reviewed-on: https://go-review.googlesource.com/c/go/+/560755
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: David Chase <drchase@google.com>
2024-02-07 20:53:05 +00:00
Bryan C. Mills e9b3ff15f4 os: relax tests and add examples for UserCacheDir and UserConfigDir
Per https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html:
“If, when attempting to write a file, the destination directory is
non-existent an attempt should be made to create it with permission
0700. […] The application should be prepared to handle the case where
the file could not be written […]. In such case it may choose to
present an error message to the user.”

In certain CI environments, these directories have well-defined
locations but do not exist and cannot be created. In that case,
we now choose to log and return from the test without failing it.

To prevent the functions from falling back to being entirely untested,
we still fail the test (and “present an error message to the user”) if
either function returns an empty string without an error, or returns a
path that refers to a non-directory or results in an error other than
ErrNotExist.

In addition, since the tests themselves no longer create subdirectories,
we add examples illustrating the suggested pattern of usage.

Fixes #64990.

Change-Id: Ie72106424f5ebe36eaf9288c22710d74bb14a462
Reviewed-on: https://go-review.googlesource.com/c/go/+/554815
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
2024-01-10 17:35:49 +00:00
Danil Timerbulatov 527829a7cb all: remove newline characters after return statements
This commit is aimed at improving the readability and consistency
of the code base. Extraneous newline characters were present after
some return statements, creating unnecessary separation in the code.

Fixes #64610

Change-Id: Ic1b05bf11761c4dff22691c2f1c3755f66d341f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/548316
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-12-14 17:22:18 +00:00
Russ Cox ba7c4e47b3 os: limit temp file randomness to uint32
CL 516860 accidentally changed the randomness
used in TempFile from 32 to 64 bits on 64-bit platforms,
meaning from 10 to 20 decimal bytes.
This is enough to cause problems in a few tests
because it makes temporary directory names just
a little bit longer.

Limit back down to 32 bits of randomness, which is fine,
and add a test to avoid repeating the mistake.

Fixes #64605.

Change-Id: I17b8c063d11d5c0a96a68b5e5f83c889a13bca77
Reviewed-on: https://go-review.googlesource.com/c/go/+/548635
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2023-12-11 15:36:54 +00:00
Bryan C. Mills bb34112d4d os: document Readlink behavior for relative links
Also provide a runnable example to illustrate that behavior.

This should help users to avoid the common mistake of expecting
os.Readlink to return an absolute path.

Fixes #57766.

Change-Id: I8f60aa111ebda0cae985758615019aaf26d5cb41
Reviewed-on: https://go-review.googlesource.com/c/go/+/546995
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2023-12-07 17:27:54 +00:00
Russ Cox c29444ef39 math/rand, math/rand/v2: use ChaCha8 for global rand
Move ChaCha8 code into internal/chacha8rand and use it to implement
runtime.rand, which is used for the unseeded global source for
both math/rand and math/rand/v2. This also affects the calculation of
the start point for iteration over very very large maps (when the
32-bit fastrand is not big enough).

The benefit is that misuse of the global random number generators
in math/rand and math/rand/v2 in contexts where non-predictable
randomness is important for security reasons is no longer a
security problem, removing a common mistake among programmers
who are unaware of the different kinds of randomness.

The cost is an extra 304 bytes per thread stored in the m struct
plus 2-3ns more per random uint64 due to the more sophisticated
algorithm. Using PCG looks like it would cost about the same,
although I haven't benchmarked that.

Before this, the math/rand and math/rand/v2 global generator
was wyrand (https://github.com/wangyi-fudan/wyhash).
For math/rand, using wyrand instead of the Mitchell/Reeds/Thompson
ALFG was justifiable, since the latter was not any better.
But for math/rand/v2, the global generator really should be
at least as good as one of the well-studied, specific algorithms
provided directly by the package, and it's not.

(Wyrand is still reasonable for scheduling and cache decisions.)

Good randomness does have a cost: about twice wyrand.

Also rationalize the various runtime rand references.

goos: linux
goarch: amd64
pkg: math/rand/v2
cpu: AMD Ryzen 9 7950X 16-Core Processor
                        │ bbb48afeb7.amd64 │           5cf807d1ea.amd64           │
                        │      sec/op      │    sec/op     vs base                │
ChaCha8-32                     1.862n ± 2%    1.861n ± 2%        ~ (p=0.825 n=20)
PCG_DXSM-32                    1.471n ± 1%    1.460n ± 2%        ~ (p=0.153 n=20)
SourceUint64-32                1.636n ± 2%    1.582n ± 1%   -3.30% (p=0.000 n=20)
GlobalInt64-32                 2.087n ± 1%    3.663n ± 1%  +75.54% (p=0.000 n=20)
GlobalInt64Parallel-32        0.1042n ± 1%   0.2026n ± 1%  +94.48% (p=0.000 n=20)
GlobalUint64-32                2.263n ± 2%    3.724n ± 1%  +64.57% (p=0.000 n=20)
GlobalUint64Parallel-32       0.1019n ± 1%   0.1973n ± 1%  +93.67% (p=0.000 n=20)
Int64-32                       1.771n ± 1%    1.774n ± 1%        ~ (p=0.449 n=20)
Uint64-32                      1.863n ± 2%    1.866n ± 1%        ~ (p=0.364 n=20)
GlobalIntN1000-32              3.134n ± 3%    4.730n ± 2%  +50.95% (p=0.000 n=20)
IntN1000-32                    2.489n ± 1%    2.489n ± 1%        ~ (p=0.683 n=20)
Int64N1000-32                  2.521n ± 1%    2.516n ± 1%        ~ (p=0.394 n=20)
Int64N1e8-32                   2.479n ± 1%    2.478n ± 2%        ~ (p=0.743 n=20)
Int64N1e9-32                   2.530n ± 2%    2.514n ± 2%        ~ (p=0.193 n=20)
Int64N2e9-32                   2.501n ± 1%    2.494n ± 1%        ~ (p=0.616 n=20)
Int64N1e18-32                  3.227n ± 1%    3.205n ± 1%        ~ (p=0.101 n=20)
Int64N2e18-32                  3.647n ± 1%    3.599n ± 1%        ~ (p=0.019 n=20)
Int64N4e18-32                  5.135n ± 1%    5.069n ± 2%        ~ (p=0.034 n=20)
Int32N1000-32                  2.657n ± 1%    2.637n ± 1%        ~ (p=0.180 n=20)
Int32N1e8-32                   2.636n ± 1%    2.636n ± 1%        ~ (p=0.763 n=20)
Int32N1e9-32                   2.660n ± 2%    2.638n ± 1%        ~ (p=0.358 n=20)
Int32N2e9-32                   2.662n ± 2%    2.618n ± 2%        ~ (p=0.064 n=20)
Float32-32                     2.272n ± 2%    2.239n ± 2%        ~ (p=0.194 n=20)
Float64-32                     2.272n ± 1%    2.286n ± 2%        ~ (p=0.763 n=20)
ExpFloat64-32                  3.762n ± 1%    3.744n ± 1%        ~ (p=0.171 n=20)
NormFloat64-32                 3.706n ± 1%    3.655n ± 2%        ~ (p=0.066 n=20)
Perm3-32                       32.93n ± 3%    34.62n ± 1%   +5.13% (p=0.000 n=20)
Perm30-32                      202.9n ± 1%    204.0n ± 1%        ~ (p=0.482 n=20)
Perm30ViaShuffle-32            115.0n ± 1%    114.9n ± 1%        ~ (p=0.358 n=20)
ShuffleOverhead-32             112.8n ± 1%    112.7n ± 1%        ~ (p=0.692 n=20)
Concurrent-32                  2.107n ± 0%    3.725n ± 1%  +76.75% (p=0.000 n=20)

goos: darwin
goarch: arm64
pkg: math/rand/v2
                       │ bbb48afeb7.arm64 │           5cf807d1ea.arm64            │
                       │      sec/op      │    sec/op     vs base                 │
ChaCha8-8                     2.480n ± 0%    2.429n ± 0%    -2.04% (p=0.000 n=20)
PCG_DXSM-8                    2.531n ± 0%    2.530n ± 0%         ~ (p=0.877 n=20)
SourceUint64-8                2.534n ± 0%    2.533n ± 0%         ~ (p=0.732 n=20)
GlobalInt64-8                 2.172n ± 1%    4.794n ± 0%  +120.67% (p=0.000 n=20)
GlobalInt64Parallel-8        0.4320n ± 0%   0.9605n ± 0%  +122.32% (p=0.000 n=20)
GlobalUint64-8                2.182n ± 0%    4.770n ± 0%  +118.58% (p=0.000 n=20)
GlobalUint64Parallel-8       0.4307n ± 0%   0.9583n ± 0%  +122.51% (p=0.000 n=20)
Int64-8                       4.107n ± 0%    4.104n ± 0%         ~ (p=0.416 n=20)
Uint64-8                      4.080n ± 0%    4.080n ± 0%         ~ (p=0.052 n=20)
GlobalIntN1000-8              2.814n ± 2%    5.643n ± 0%  +100.50% (p=0.000 n=20)
IntN1000-8                    4.141n ± 0%    4.139n ± 0%         ~ (p=0.140 n=20)
Int64N1000-8                  4.140n ± 0%    4.140n ± 0%         ~ (p=0.313 n=20)
Int64N1e8-8                   4.140n ± 0%    4.139n ± 0%         ~ (p=0.103 n=20)
Int64N1e9-8                   4.139n ± 0%    4.140n ± 0%         ~ (p=0.761 n=20)
Int64N2e9-8                   4.140n ± 0%    4.140n ± 0%         ~ (p=0.636 n=20)
Int64N1e18-8                  5.266n ± 0%    5.326n ± 1%    +1.14% (p=0.001 n=20)
Int64N2e18-8                  6.052n ± 0%    6.167n ± 0%    +1.90% (p=0.000 n=20)
Int64N4e18-8                  8.826n ± 0%    9.051n ± 0%    +2.55% (p=0.000 n=20)
Int32N1000-8                  4.127n ± 0%    4.132n ± 0%    +0.12% (p=0.000 n=20)
Int32N1e8-8                   4.126n ± 0%    4.131n ± 0%    +0.12% (p=0.000 n=20)
Int32N1e9-8                   4.127n ± 0%    4.132n ± 0%    +0.12% (p=0.000 n=20)
Int32N2e9-8                   4.132n ± 0%    4.131n ± 0%         ~ (p=0.017 n=20)
Float32-8                     4.109n ± 0%    4.105n ± 0%         ~ (p=0.379 n=20)
Float64-8                     4.107n ± 0%    4.106n ± 0%         ~ (p=0.867 n=20)
ExpFloat64-8                  5.339n ± 0%    5.383n ± 0%    +0.82% (p=0.000 n=20)
NormFloat64-8                 5.735n ± 0%    5.737n ± 1%         ~ (p=0.856 n=20)
Perm3-8                       26.65n ± 0%    26.80n ± 1%    +0.58% (p=0.000 n=20)
Perm30-8                      194.8n ± 1%    197.0n ± 0%    +1.18% (p=0.000 n=20)
Perm30ViaShuffle-8            156.6n ± 0%    157.6n ± 1%    +0.61% (p=0.000 n=20)
ShuffleOverhead-8             124.9n ± 0%    125.5n ± 0%    +0.52% (p=0.000 n=20)
Concurrent-8                  2.434n ± 3%    5.066n ± 0%  +108.09% (p=0.000 n=20)

goos: linux
goarch: 386
pkg: math/rand/v2
cpu: AMD Ryzen 9 7950X 16-Core Processor
                        │ bbb48afeb7.386 │            5cf807d1ea.386             │
                        │     sec/op     │    sec/op     vs base                 │
ChaCha8-32                  11.295n ± 1%    4.748n ± 2%   -57.96% (p=0.000 n=20)
PCG_DXSM-32                  7.693n ± 1%    7.738n ± 2%         ~ (p=0.542 n=20)
SourceUint64-32              7.658n ± 2%    7.622n ± 2%         ~ (p=0.344 n=20)
GlobalInt64-32               3.473n ± 2%    7.526n ± 2%  +116.73% (p=0.000 n=20)
GlobalInt64Parallel-32      0.3198n ± 0%   0.5444n ± 0%   +70.22% (p=0.000 n=20)
GlobalUint64-32              3.612n ± 0%    7.575n ± 1%  +109.69% (p=0.000 n=20)
GlobalUint64Parallel-32     0.3168n ± 0%   0.5403n ± 0%   +70.51% (p=0.000 n=20)
Int64-32                     7.673n ± 2%    7.789n ± 1%         ~ (p=0.122 n=20)
Uint64-32                    7.773n ± 1%    7.827n ± 2%         ~ (p=0.920 n=20)
GlobalIntN1000-32            6.268n ± 1%    9.581n ± 1%   +52.87% (p=0.000 n=20)
IntN1000-32                  10.33n ± 2%    10.45n ± 1%         ~ (p=0.233 n=20)
Int64N1000-32                10.98n ± 2%    11.01n ± 1%         ~ (p=0.401 n=20)
Int64N1e8-32                 11.19n ± 2%    10.97n ± 1%         ~ (p=0.033 n=20)
Int64N1e9-32                 11.06n ± 1%    11.08n ± 1%         ~ (p=0.498 n=20)
Int64N2e9-32                 11.10n ± 1%    11.01n ± 2%         ~ (p=0.995 n=20)
Int64N1e18-32                15.23n ± 2%    15.04n ± 1%         ~ (p=0.973 n=20)
Int64N2e18-32                15.89n ± 1%    15.85n ± 1%         ~ (p=0.409 n=20)
Int64N4e18-32                18.96n ± 2%    19.34n ± 2%         ~ (p=0.048 n=20)
Int32N1000-32                10.46n ± 2%    10.44n ± 2%         ~ (p=0.480 n=20)
Int32N1e8-32                 10.46n ± 2%    10.49n ± 2%         ~ (p=0.951 n=20)
Int32N1e9-32                 10.28n ± 2%    10.26n ± 1%         ~ (p=0.431 n=20)
Int32N2e9-32                 10.50n ± 2%    10.44n ± 2%         ~ (p=0.249 n=20)
Float32-32                   13.80n ± 2%    13.80n ± 2%         ~ (p=0.751 n=20)
Float64-32                   23.55n ± 2%    23.87n ± 0%         ~ (p=0.408 n=20)
ExpFloat64-32                15.36n ± 1%    15.29n ± 2%         ~ (p=0.316 n=20)
NormFloat64-32               13.57n ± 1%    13.79n ± 1%    +1.66% (p=0.005 n=20)
Perm3-32                     45.70n ± 2%    46.99n ± 2%    +2.81% (p=0.001 n=20)
Perm30-32                    399.0n ± 1%    403.8n ± 1%    +1.19% (p=0.006 n=20)
Perm30ViaShuffle-32          349.0n ± 1%    350.4n ± 1%         ~ (p=0.909 n=20)
ShuffleOverhead-32           322.3n ± 1%    323.8n ± 1%         ~ (p=0.410 n=20)
Concurrent-32                3.331n ± 1%    7.312n ± 1%  +119.50% (p=0.000 n=20)

For #61716.

Change-Id: Ibdddeed85c34d9ae397289dc899e04d4845f9ed2
Reviewed-on: https://go-review.googlesource.com/c/go/+/516860
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-12-05 20:34:30 +00:00
Jes Cok d961b12be9 io/fs, os: unify PathError.Path for dirFS.{ReadFile, ReadDir}
Fixes #64366

Change-Id: Ie78ab2cb9e11c0766665cd37fd7a26d36a1c24fa
GitHub-Last-Rev: 3cb3bb84a0
GitHub-Pull-Request: golang/go#64372
Reviewed-on: https://go-review.googlesource.com/c/go/+/544835
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Jes Cok <xigua67damn@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-11-27 16:25:41 +00:00
Kir Kolyshkin 3548f9a01f os: remove useless if from Wait on unix
Back in the day, Wait used to accept options argument.

CL 4962042 fixed the issue of setting process.done flag when WNOHANG
option was used.

Later, CL 5688046 removed options argument from Wait, but did not remove
pid1 != 0 check which was meant to be used with WNOHANG only.

Remove the check, which is useless and also confusing.

Change-Id: I73b9ef4a0dbe35466e659ca58b896d515ba86d02
Reviewed-on: https://go-review.googlesource.com/c/go/+/543736
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-11-21 20:39:17 +00:00
qiulaidongfeng 78d037b0e0 os: avoid TestFileChdir fail when GOROOT is a symbolic link
If GOROOT is a symbolic link,
the paths obtained from the
first and second Getwd of TestFileChdir are different,
and this CL fixes the test failure in this situation.

Fixes #64281

Change-Id: I53026b6c54a54be08833396e2c7081ca3ab8c282
GitHub-Last-Rev: 5cc418e625
GitHub-Pull-Request: golang/go#64001
Reviewed-on: https://go-review.googlesource.com/c/go/+/540521
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-21 16:29:21 +00:00
Ville Skyttä 1d187fd0be src: a/an grammar fixes
Change-Id: I179b50ae8e73677d4d408b83424afbbfe6aa17a1
GitHub-Last-Rev: 2e2d9c1e45
GitHub-Pull-Request: golang/go#63478
Reviewed-on: https://go-review.googlesource.com/c/go/+/534015
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-11-20 15:56:08 +00:00
Andy Pan f664031bc1 net,os: arrange zero-copy of os.File and net.TCPConn to net.UnixConn
Fixes #58808

goos: linux
goarch: amd64
pkg: net
cpu: DO-Premium-Intel
                             │      old      │                 new                  │
                             │    sec/op     │    sec/op     vs base                │
Splice/tcp-to-unix/1024-4       3.783µ ± 10%   3.201µ ±  7%  -15.40% (p=0.001 n=10)
Splice/tcp-to-unix/2048-4       3.967µ ± 13%   3.818µ ± 16%        ~ (p=0.971 n=10)
Splice/tcp-to-unix/4096-4       4.988µ ± 16%   4.590µ ± 11%        ~ (p=0.089 n=10)
Splice/tcp-to-unix/8192-4       6.981µ ± 13%   5.236µ ±  9%  -25.00% (p=0.000 n=10)
Splice/tcp-to-unix/16384-4     10.192µ ±  9%   7.350µ ±  7%  -27.89% (p=0.000 n=10)
Splice/tcp-to-unix/32768-4      19.65µ ± 13%   10.28µ ± 16%  -47.69% (p=0.000 n=10)
Splice/tcp-to-unix/65536-4      41.89µ ± 18%   15.70µ ± 13%  -62.52% (p=0.000 n=10)
Splice/tcp-to-unix/131072-4     90.05µ ± 11%   29.55µ ± 10%  -67.18% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    170.24µ ± 15%   52.66µ ±  4%  -69.06% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4     326.4µ ± 13%   109.3µ ± 11%  -66.52% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4    651.4µ ±  9%   228.3µ ± 14%  -64.95% (p=0.000 n=10)
geomean                         29.42µ         15.62µ        -46.90%

                             │      old      │                  new                   │
                             │      B/s      │      B/s       vs base                 │
Splice/tcp-to-unix/1024-4      258.2Mi ± 11%   305.2Mi ±  8%   +18.21% (p=0.001 n=10)
Splice/tcp-to-unix/2048-4      492.5Mi ± 15%   511.7Mi ± 13%         ~ (p=0.971 n=10)
Splice/tcp-to-unix/4096-4      783.5Mi ± 14%   851.2Mi ± 12%         ~ (p=0.089 n=10)
Splice/tcp-to-unix/8192-4      1.093Gi ± 11%   1.458Gi ±  8%   +33.36% (p=0.000 n=10)
Splice/tcp-to-unix/16384-4     1.497Gi ±  9%   2.076Gi ±  7%   +38.67% (p=0.000 n=10)
Splice/tcp-to-unix/32768-4     1.553Gi ± 11%   2.969Gi ± 14%   +91.17% (p=0.000 n=10)
Splice/tcp-to-unix/65536-4     1.458Gi ± 23%   3.888Gi ± 11%  +166.69% (p=0.000 n=10)
Splice/tcp-to-unix/131072-4    1.356Gi ± 10%   4.131Gi ±  9%  +204.72% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    1.434Gi ± 13%   4.637Gi ±  4%  +223.32% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4    1.497Gi ± 15%   4.468Gi ± 10%  +198.47% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4   1.501Gi ± 10%   4.277Gi ± 16%  +184.88% (p=0.000 n=10)
geomean                        1.038Gi         1.954Gi         +88.28%

                             │      old      │                   new                   │
                             │     B/op      │    B/op     vs base                     │
Splice/tcp-to-unix/1024-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4      0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4     0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4     0.000 ±  0%     0.000 ± 0%         ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4     1.000 ±   ?     0.000 ± 0%  -100.00% (p=0.001 n=10)
Splice/tcp-to-unix/131072-4    2.000 ±  0%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/262144-4    4.000 ± 25%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/524288-4    7.500 ± 33%     0.000 ± 0%  -100.00% (p=0.000 n=10)
Splice/tcp-to-unix/1048576-4   17.00 ± 12%      0.00 ± 0%  -100.00% (p=0.000 n=10)
geomean                                    ²               ?                       ² ³
¹ all samples are equal
² summaries must be >0 to compute geomean
³ ratios must be >0 to compute geomean

                             │     old      │                 new                 │
                             │  allocs/op   │ allocs/op   vs base                 │
Splice/tcp-to-unix/1024-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/2048-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/4096-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/8192-4      0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/16384-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/32768-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/65536-4     0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/131072-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/262144-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/524288-4    0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
Splice/tcp-to-unix/1048576-4   0.000 ± 0%     0.000 ± 0%       ~ (p=1.000 n=10) ¹
geomean                                   ²               +0.00%                ²
¹ all samples are equal
² summaries must be >0 to compute geomean

Change-Id: I829061b009a0929a8ef1a15c183793c0b9104dde
Reviewed-on: https://go-review.googlesource.com/c/go/+/472475
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-17 23:16:28 +00:00
Bryan C. Mills 1b03ec8a25 os/signal: remove some arbitrary timeouts in tests
This should fix the test flake found in
https://build.golang.org/log/48ffb18e85dda480b7a67e8305dd03ee8337f170.

For #58901.

Change-Id: I1fcdd713a78e6b7c81e38133ce5f42f7f448a1a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/541115
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-11-09 15:34:56 +00:00
Michael Pratt a334c45273 os/signal: skip nohup tests on darwin builders
The new LUCI builders have a temporary limitation that breaks nohup.
Skip nohup tests there.

For #63875.

Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_13
Change-Id: Ia9ffecea7310f84a21f6138d8f8cdfc5e1392307
Reviewed-on: https://go-review.googlesource.com/c/go/+/538698
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-11-02 17:31:41 +00:00
Michael Pratt 5622a4b205 os/signal: remove go t.Run from TestNohup
Since CL 226138, TestNohup has a bit of a strange construction: it wants
to run the "uncaught" subtests in parallel with each other, and the
"nohup" subtests in parallel with each other, but also needs join
between "uncaught" and "nohop" so it can Stop notifying for SIGHUP.

It achieves this by doing `go t.Run` with a WaitGroup rather than using
`t.Parallel` in the subtest (which would make `t.Run` return immediately).

However, this makes things more difficult to understand than necessary.
As noted on https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks,
a second layer of subtest can be used to join parallel subtests.

Switch to this form, which makes the test simpler to follow
(particularly the cleanup that goes with "uncaught").

For #63799.

Change-Id: Ibfce0f439508a7cfca848c7ccfd136c9c453ad8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/538899
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-11-02 16:31:19 +00:00
Joe Tsai 08b2f1f761 os: fix PathError.Op for dirFS.Open
This appears to be a copy-paste error from CL 455362.

The operation name used to be "open"
but seems to have been accidentally changed to "stat".
This CL reverts back to "open".

Change-Id: I3fc5168095e2d9eee3efa3cc091b10bcf4e3ecde
Reviewed-on: https://go-review.googlesource.com/c/go/+/539056
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-01 23:58:46 +00:00
Bryan C. Mills 0330aad038 os: report IO_REPARSE_TAG_DEDUP files as regular in Stat and Lstat
Prior to CL 460595, Lstat reported most reparse points as regular
files. However, reparse points can in general implement unusual
behaviors (consider IO_REPARSE_TAG_AF_UNIX or IO_REPARSE_TAG_LX_CHR),
and Windows allows arbitrary user-defined reparse points, so in
general we must not assume that an unrecognized reparse tag represents
a regular file; in CL 460595, we began marking them as irregular.

As it turns out, the Data Deduplication service on Windows Server runs
an Optimization job that turns regular files into reparse files with
the tag IO_REPARSE_TAG_DEDUP. Those files still behave more-or-less
like regular files, in that they have well-defined sizes and support
random-access reads and writes, so most programs can treat them as
regular files without difficulty. However, they are still reparse
files: as a result, on servers with the Data Deduplication service
enabled, files could arbitrarily change from “regular” to “irregular”
without explicit user intervention.

Since dedup files are converted in the background and otherwise behave
like regular files, this change adds a special case to report DEDUP
reparse points as regular.

Fixes #63429.

No test because to my knowledge we don't have any Windows builders
that have the deduplication service enabled, nor do we have a way to
reliably guarantee the existence of an IO_REPARSE_TAG_DEDUP file.

(In theory we could add a builder with the service enabled on a
specific volume, write a test that encodes knowledge of that volume,
and use the GO_BUILDER_NAME environment variable to run that test only
on the specially-configured builders. However, I don't currently have
the bandwidth to reconfigure the builders in this way, and given the
simplicity of the change I think it is unlikely to regress
accidentally.)

Change-Id: I649e7ef0b67e3939a980339ce7ec6a20b31b23a1
Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/537915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-11-01 19:01:53 +00:00
Joel Sing 0aa2197279 os/signal: use syscall.Wait4 directly in tests
Rather than using syscall.Syscall6 with SYS_WAIT4, use syscall.Wait4
directly.

Updates #59667

Change-Id: I50fea3b7d10003dbc632aafd5e170a9fe96d6f42
Reviewed-on: https://go-review.googlesource.com/c/go/+/538459
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-11-01 07:10:32 +00:00
cui fliter 51bdd3bdcc os: use strings.EqualFold in os_test.go
strings.EqualFold has no memory overhead and has better performance than strings.ToLower.

This is a performance test:

package bench

import (
	"strings"
	"testing"
)

func BenchmarkToLower(b *testing.B) {

	str1 := "Windows"
	str2 := "windows"

	for i := 0; i < b.N; i++ {
		if strings.ToLower(str1) == strings.ToLower(str2) {
		}
	}
}

func BenchmarkEqualFold(b *testing.B) {

	str1 := "Windows"
	str2 := "windows"

	for i := 0; i < b.N; i++ {
		if strings.EqualFold(str1, str2) {
		}
	}
}

The result:

goos: darwin
goarch: arm64
BenchmarkToLower-8      31404808                36.99 ns/op            8 B/op          1 allocs/op
BenchmarkEqualFold-8    194780793                5.989 ns/op           0 B/op          0 allocs/op
PASS

Change-Id: Id3d92534942d3eb0bdc1d01359324030ad0e434f
Reviewed-on: https://go-review.googlesource.com/c/go/+/533635
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-10-09 22:06:42 +00:00
Jes Cok 81c5d92f52 all: use the indefinite article an in comments
Change-Id: I8787458f9ccd3b5cdcdda820d8a45deb4f77eade
GitHub-Last-Rev: be865d67ef
GitHub-Pull-Request: golang/go#63165
Reviewed-on: https://go-review.googlesource.com/c/go/+/530120
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-09-25 14:29:30 +00:00
Bryan C. Mills 15e5e35be5 os: in TestMkdirStickyUmask, create a non-sticky directory as a control
Fixes #62684.

Change-Id: If7afa811526973671d83e21440cbbc1a7b2120d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/529115
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-09-18 17:32:43 +00:00