Commit Graph

83 Commits

Author SHA1 Message Date
Peter Collingbourne b822f098c5 os: don't try to make the directory FD non-blocking in os.ReadDir
This will fail because epoll_ctl() fails on directory FDs, so we
end up issuing unnecessary syscalls. My test program that calls
filepath.WalkDir on a large directory tree runs 1.23 ± 0.04 times
faster than with the original implementation.

Change-Id: Ie33d798c48057a7b2d0bacac80fcdde5b5a8bb1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/570877
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-16 00:34:52 +00:00
cui fliter 73847aa41b os: add available godoc link
Change-Id: I430c9a7c4936d7a8c8c787aa63de9a796d20fdf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/539597
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-02-26 21:33:12 +00:00
Bryan C. Mills bb34112d4d os: document Readlink behavior for relative links
Also provide a runnable example to illustrate that behavior.

This should help users to avoid the common mistake of expecting
os.Readlink to return an absolute path.

Fixes #57766.

Change-Id: I8f60aa111ebda0cae985758615019aaf26d5cb41
Reviewed-on: https://go-review.googlesource.com/c/go/+/546995
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2023-12-07 17:27:54 +00:00
Michael Pratt a6fb97b6a7 os: explicitly check for invalid FD in NewFile
CL 497075 refactored NewFile to unconditionally dereference the file
returned by newFile. However, newFile can return nil if passed a
negative FD, which now causes a crash.

Resolve this by moving the invalid check earlier in NewFile, which also
lets us avoid a useless fcntl syscall on a negative FD.

Since we convert to int to check sign, adjust newFile to take an int
rather than uintptr, which cleans up a lot of conversions.

Fixes #60406

Change-Id: I382a74e22f1cc01f7a2dcf1ff4efca6a79c4dd57
Reviewed-on: https://go-review.googlesource.com/c/go/+/497877
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-05-24 20:19:46 +00:00
Tobias Klauser 3c3a883667 os: avoid second fcntl syscall in NewFile on unix
CL 494915 introduced an additional fcntl(F_GETFL) syscall to determine
whether the file is in append-only mode. The existing unix.IsNonblock
call also issues an fcntl(F_GETFL) syscall. The two can be combined and
both the append-only mode and the non-blocking flags can be determined
from that syscall's result.

Change-Id: I915589ed94e079f6abaa2fd0032ef01f78698f7f
Reviewed-on: https://go-review.googlesource.com/c/go/+/497075
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
2023-05-23 19:42:05 +00:00
Ian Lance Taylor b950cc8f11 net, os: net.Conn.File.Fd should return a blocking descriptor
Historically net.Conn.File.Fd has returned a descriptor in blocking mode.
That was broken by CL 495079, which changed the behavior for os.OpenFile
and os.NewFile without intending to affect net.Conn.File.Fd.
Use a hidden os entry point to preserve the historical behavior,
to ensure backward compatibility.

Change-Id: I8d14b9296070ddd52bb8940cb88c6a8b2dc28c27
Reviewed-on: https://go-review.googlesource.com/c/go/+/496080
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-05-20 21:53:52 +00:00
Tobias Klauser 774f60223f os: set File.appendMode in NewFile if file was opened with O_APPEND
To allow skipping the use of the copy_file_range syscall on Linux which
isn't supported for destination files opened with O_APPEND, see comment
in (*File).readFrom and
https://man7.org/linux/man-pages/man2/copy_file_range.2.html#ERRORS

Fixes #60181

Change-Id: Ie0b0050faab16858412928a3d1f96442619581eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/494915
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
2023-05-18 09:16:07 +00:00
Ian Lance Taylor f777726ff0 os: if descriptor is non-blocking, retain that in Fd method
For #58408
Fixes #60211

Change-Id: I30f5678b46e15121865b19d1c0f82698493fad4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/495079
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-05-17 21:12:04 +00:00
Ian Lance Taylor 719e2b6f09 os, runtime: remove unused implementations of os.sigpipe
Clean up instances that are unused since CL 6450058.

Change-Id: I0e9ae28cfa83fcc8abda8f5eca9c7dfc2c1c4ad1
Reviewed-on: https://go-review.googlesource.com/c/go/+/477396
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
2023-05-12 21:33:26 +00:00
Constantin Konstantinidis 5a9b6432ec os: make Chtimes accept empty time values to skip file time modification
Empty time value time.Time{} leaves the corresponding time of the file
unchanged.

Fixes #32558

Change-Id: I1aff42f30668ff505ecec2e9509d8f2b8e4b1b6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/219638
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-05-11 18:19:17 +00:00
Ian Lance Taylor 1596aeec8e all: add String for fs.{FileInfo,DirEntry} implementations
The new String methods use the new FormatFileInfo and
FormatDirEntry functions.

Fixes #54451

Change-Id: I414cdfc212ec3c316fb2734756d2117842a23631
Reviewed-on: https://go-review.googlesource.com/c/go/+/491175
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-04 16:27:35 +00:00
Johan Brandhorst-Satzkorn 66cac9e1e4 os: add wasip1 support
For #58141

Co-authored-by: Richard Musiol <neelance@gmail.com>
Co-authored-by: Achille Roussel <achille.roussel@gmail.com>
Co-authored-by: Julien Fabre <ju.pryz@gmail.com>
Co-authored-by: Evan Phoenix <evan@phx.io>
Change-Id: I52e3e161f81dcbe8605570e47d732992979c4d34
Reviewed-on: https://go-review.googlesource.com/c/go/+/479623
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-04-07 23:34:21 +00:00
Ian Lance Taylor 52af6550c9 os: don't try to put directory into non-blocking mode
Fixes #56843

Change-Id: I3cb3e8397499cd8c57a3edddd45f38c510519b36
Reviewed-on: https://go-review.googlesource.com/c/go/+/451997
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Nicolas Hillegeer <aktau@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Rob Pike <r@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-27 23:05:46 +00:00
Yuval Pavel Zholkover b15c399a36 os: only add file descriptors which are set to non-blocking mode to the netpoller
Either ones where kind == kindNonBlock or those we've successfully called syscall.SetNonblock() on.
Restore blocking behavior if we detect an error registering with the netpoller and our flow was
successful in setting the inital syscall.SetNonblock().

Update #54100

Change-Id: I08934e4107c7fb36c15a7ca23ac880490b4df235
Reviewed-on: https://go-review.googlesource.com/c/go/+/420334
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Goutnik <dgoutnik@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Yuval Pavel Zholkover <paulzhol@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
2022-08-18 03:12:27 +00:00
Russ Cox 81431c7aa7 all: replace `` and '' with “ (U+201C) and ” (U+201D) in doc comments
go/doc in all its forms applies this replacement when rendering
the comments. We are considering formatting doc comments,
including doing this replacement as part of the formatting.
Apply it to our source files ahead of time.

For #51082.

Change-Id: Ifcc1f5861abb57c5d14e7d8c2102dfb31b7a3a19
Reviewed-on: https://go-review.googlesource.com/c/go/+/384262
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-05 17:52:29 +00:00
Ian Lance Taylor a2baae6851 all: use new "unix" build tag where appropriate
For #20322
For #51572

Change-Id: Id0b4799d097d01128e98ba4cc0092298357bca45
Reviewed-on: https://go-review.googlesource.com/c/go/+/389935
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2022-03-29 16:24:51 +00:00
Russ Cox f229e7031a all: go fix -fix=buildtag std cmd (except for bootstrap deps, vendor)
When these packages are released as part of Go 1.18,
Go 1.16 will no longer be supported, so we can remove
the +build tags in these files.

Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs
as defined in src/cmd/dist/buildtool.go, which need to continue
to build with Go 1.4 for now.

Also reverted src/vendor and src/cmd/vendor, which will need
to be updated in their own repos first.

Manual changes in runtime/pprof/mprof_test.go to adjust line numbers.

For #41184.

Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e
Reviewed-on: https://go-review.googlesource.com/c/go/+/344955
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 18:17:57 +00:00
Ian Lance Taylor 0d32d9e8a8 os: document that Windows Symlink to missing target creates file symlink
Fixes #39183

Change-Id: Iec4a5a561182ade57dc7dc24247710005d6b9f21
Reviewed-on: https://go-review.googlesource.com/c/go/+/314275
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-05-02 18:22:19 +00:00
Brad Fitzpatrick 13a4e8c41c all: simplify the spelling of Linux
The prefix didn't help clarify anything.

Change-Id: I897fd4022ce9df42a548b15714e4b592618ca547
Reviewed-on: https://go-review.googlesource.com/c/go/+/309573
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-04-13 13:56:21 +00:00
Tom Thorogood 9e7bc80b31 os: reuse readdir buffers on unix with a sync.Pool
(*File).readdir allocates a fixed-size 8KiB buffer on unix systems that
cannot be reused. While it accounts for just a single allocation, it's
more than large enough to show up in profiles and make things quite a
bit slower.

Instead of allocating so often, use a sync.Pool to allow these buffers to
be reused. This has a large impact on readdir heavy workloads.

Package os benchmarks:

name            old time/op    new time/op    delta
Readdirname-12    35.6µs ± 5%    18.1µs ± 4%  -49.00%  (p=0.000 n=10+10)
Readdir-12         142µs ± 1%     121µs ± 0%  -14.87%  (p=0.000 n=10+9)
ReadDir-12        44.0µs ± 6%    28.4µs ± 8%  -35.58%  (p=0.000 n=9+10)

name            old alloc/op   new alloc/op   delta
Readdirname-12    14.4kB ± 0%     6.2kB ± 0%  -57.08%  (p=0.000 n=10+10)
Readdir-12        41.6kB ± 0%    33.4kB ± 0%  -19.77%  (p=0.000 n=10+9)
ReadDir-12        21.9kB ± 0%    13.7kB ± 0%  -37.39%  (p=0.000 n=10+10)

name            old allocs/op  new allocs/op  delta
Readdirname-12       131 ± 0%       130 ± 0%   -0.76%  (p=0.000 n=10+10)
Readdir-12           367 ± 0%       366 ± 0%   -0.27%  (p=0.000 n=10+10)
ReadDir-12           249 ± 0%       248 ± 0%   -0.40%  (p=0.000 n=10+10)

A clunky benchmark I threw together that calls filepath.WalkDir on $GOMODCACHE:

name        old time/op    new time/op    delta
WalkDir-12    91.2ms ±19%    48.7ms ± 0%  -46.54%  (p=0.000 n=10+10)

name        old alloc/op   new alloc/op   delta
WalkDir-12    54.0MB ± 0%     7.6MB ± 0%  -85.92%  (p=0.000 n=8+9)

name        old allocs/op  new allocs/op  delta
WalkDir-12      136k ± 0%      130k ± 0%   -4.15%  (p=0.000 n=8+8)

Change-Id: I00e4d48726da0e46c528ab205409afd03127b844
Reviewed-on: https://go-review.googlesource.com/c/go/+/302169
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-04-03 21:06:18 +00:00
Russ Cox d4b2638234 all: go fmt std cmd (but revert vendor)
Make all our package sources use Go 1.17 gofmt format
(adding //go:build lines).

Part of //go:build change (#41184).
See https://golang.org/design/draft-gobuild

Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/294430
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-02-20 03:54:50 +00:00
Victor Michel 1d5e14632e os: further document limitations around naked file descriptors
NewFile requires the file descriptor to be either closed
through the returned File instance, or to stay valid at least
until the finalizer runs during garbage collection.

These requirements are easily violated when file descriptors
are closed via unix.Close, or when the *File returned by
NewFile is garbage collected while the underlying file descriptor is
still in use.

This commit adds further documentation for NewFile and Fd, making it
explicit that using naked file descriptors is subject to constraints
due to garbage collection of File objects.

Fixes #43863

Change-Id: I49ea1f0054eb2d2a72b616450c8e83476f4d07fb
GitHub-Last-Rev: 180d0130ae
GitHub-Pull-Request: golang/go#43867
Reviewed-on: https://go-review.googlesource.com/c/go/+/286032
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2021-01-26 00:30:18 +00:00
Russ Cox 2291cae2af os: use keyed literals for PathError
Necessary to move PathError to io/fs.

For #41190.

Change-Id: I05e87675f38a22f0570d4366b751b6169f7a1b13
Reviewed-on: https://go-review.googlesource.com/c/go/+/243900
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-20 00:59:22 +00:00
Russ Cox a4ede9f9a6 os: add File.ReadDir method and DirEntry type
ReadDir provides a portable, efficient way to read a directory
and discover the type of directory entries.
This enables a more efficient file system walk, yet to be added.

See #41467 for the proposal review for the API.

Fixes #41467.

Change-Id: I461a526793ae46df48821aa448b04f1705546739
Reviewed-on: https://go-review.googlesource.com/c/go/+/261540
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 00:59:20 +00:00
Changkun Ou 05b626e490 os: fix SyscallConn typos in the File.Fd comments
This CL fixes two typos introduced in CL 256899.

Change-Id: I47f0a3097deeeec8d6e9bbe7073fcf7a28c5dff9
Reviewed-on: https://go-review.googlesource.com/c/go/+/257997
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-28 08:16:57 +00:00
Changkun Ou 7bb6fed9b5 os: document and emphasize a potential misuse of File.Fd
This CL revises the document of File.Fd that explicitly points
its user to runtime.SetFinalizer where contains the information
that a file descriptor could be closed in a finalizer and therefore
causes a failure in syscall.Write if runtime.KeepAlive is not invoked.

The CL also suggests an alternative of File.Fd towards File.SyscallConn.

Fixes #41505

Change-Id: I6816f0157add48b649bf1fb793cf19dcea6894b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/256899
Reviewed-by: Rob Pike <r@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
2020-09-27 02:23:55 +00:00
Cherry Zhang a413908dd0 all: add GOOS=ios
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin"
build tag, like GOOS=android matches "linux" and GOOS=illumos
matches "solaris". Only ios/arm64 is supported (ios/amd64 is
not).

GOOS=ios and GOOS=darwin remain essentially the same at this
point. They will diverge at later time, to differentiate macOS
and iOS.

Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"),
except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"),
it remains GOOS=="darwin".

Updates #38485.

Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c
Reviewed-on: https://go-review.googlesource.com/c/go/+/254740
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-09-23 18:12:59 +00:00
Ian Lance Taylor 6b420169d7 os, internal/poll: loop on EINTR for all file syscalls
When using a FUSE file system, any system call that touches the file
system can return EINTR.

Fixes #40846

Change-Id: I25d32da22cec08dea81ab297291a85ad72db2df7
Reviewed-on: https://go-review.googlesource.com/c/go/+/249178
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-08-19 21:49:56 +00:00
Ian Lance Taylor e64675a79f os: always check for EINTR in calls to open
For #11180
For #20400
For #39237

Change-Id: I8de97517c8f92c08f0c8a51f651a17e31617979b
Reviewed-on: https://go-review.googlesource.com/c/go/+/236997
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-06-08 22:06:39 +00:00
Ian Lance Taylor 7f3ca5dfeb os: merge common Unix/Windows methods
Several method implementations were identical in file_unix.go and
file_windows.go. Merge them into file_posix.go.

Change-Id: I8bcfad468829530f81f52fe426b3a8c042e7bbd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/224138
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-20 00:26:13 +00:00
Keith Randall 529988d62c os: seek should invalidate any cached directory reads
When we seek on the underlying FD, discard any directory entries
we've already read and cached. This makes sure we won't return
the same entry twice.

We already fixed this for Darwin in CL 209961.

Fixes #37161

Change-Id: I20e1ac8d751443135e67fb4c43c18d69befb643b
Reviewed-on: https://go-review.googlesource.com/c/go/+/219143
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-03-01 22:26:33 +00:00
Keith Randall e3c7ffcd95 os: reset dirinfo when seeking on Darwin
The first Readdirnames calls opendir and caches the result.
The behavior of that cached opendir result isn't specified on a seek
of the underlying fd. Free the opendir result on a seek so that
we'll allocate a new one the next time around.

Also fix wasm behavior in this regard, so that a seek to the
file start resets the Readdirnames position, regardless of platform.

p.s. I hate the Readdirnames API.

Fixes #35767.

Change-Id: Ieffb61b3c5cdd42591f69ab13f932003966f2297
Reviewed-on: https://go-review.googlesource.com/c/go/+/209961
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-12-05 20:40:38 +00:00
Audrius Butkevicius 4a09a9b054 os: allow case only renames on case-insensitive filesystems
Fixes #35222

Change-Id: I8be45092ac4079d21ff54661637a3aa8ec4eb9bc
GitHub-Last-Rev: 954a016c9b
GitHub-Pull-Request: golang/go#35298
Reviewed-on: https://go-review.googlesource.com/c/go/+/204601
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-01 00:04:51 +00:00
Brad Fitzpatrick a38a917aee all: remove the nacl port (part 1)
You were a useful port and you've served your purpose.
Thanks for all the play.

A subsequent CL will remove amd64p32 (including assembly files and
toolchain bits) and remaining bits. The amd64p32 removal will be
separated into its own CL in case we want to support the Linux x32 ABI
in the future and want our old amd64p32 support as a starting point.

Updates #30439

Change-Id: Ia3a0c7d49804adc87bf52a4dea7e3d3007f2b1cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/199499
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-09 06:14:44 +00:00
Alex Myasoedov fc705275c3 doc: clarify safety of multiple and concurent os.(*File).Close() calls
Fixes #32427

Change-Id: I4b863bd3836067dcc2eb3a9c3a7169656763d003
Reviewed-on: https://go-review.googlesource.com/c/go/+/180438
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2019-06-05 21:21:59 +00:00
Yuval Pavel Zholkover 059f2d4a46 os: disable the use of netpoll on directories as well on *BSDs
Follow up CL 156379.

Updates #19093

Change-Id: I5ea3177fc5911d3af71cbb32584249e419e9d4a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/172937
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-04-19 19:24:47 +00:00
LE Manh Cuong c7f7f59368 os: reject WriteAt if file opened in append mode
WriteAt use pwrite syscall on *nix or WriteFile on Windows.

On Linux/Windows, these system calls always write to end of file in
append mode, regardless of offset parameter.

It is hard (maybe impossible) to make WriteAt work portably.

Making WriteAt returns an error if file is opened in append mode, we
guarantee to get consistent behavior between platforms, also prevent
user from accidently corrupting their data.

Fixes #30716

Change-Id: If83d935a22a29eed2ff8fe53d13d0b4798aa2b81
Reviewed-on: https://go-review.googlesource.com/c/go/+/166578
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-27 17:04:57 +00:00
Tobias Klauser 21a634e2e9 os: drop special case for FreeBSD 10.4 in newFile
Support for FreeBSD 10 will be dropped with Go 1.13, so revert the
workaround introduced in CL 157099.

Updates #29633
Updates #27619

Change-Id: I1a2e50d3f807a411389f3db07c0f4535a590da02
Reviewed-on: https://go-review.googlesource.com/c/go/+/165801
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-12 06:33:01 +00:00
Alex Brainman 2edd559223 os: make Readlink work with symlinks with target like \??\Volume{ABCD}\
windows-arm TMP directory live inside such link (see
https://github.com/golang/go/issues/29746#issuecomment-456526811 for
details), so symlinks like that will be common at least on windows-arm.

This CL builds on current syscall.Readlink implementation. Main
difference between the two is how new code handles symlink targets,
like \??\Volume{ABCD}\.

New implementation uses Windows CreateFile API with
FILE_FLAG_OPEN_REPARSE_POINT flag to get \??\Volume{ABCD}\ file handle.
And then it uses Windows GetFinalPathNameByHandle with VOLUME_NAME_DOS
flag to convert that handle into standard Windows path.
FILE_FLAG_OPEN_REPARSE_POINT flag ensures that symlink is not followed
when CreateFile opens the file.

Fixes #30463

Change-Id: I33b18227ce36144caed694169ef2e429fd995fb4
Reviewed-on: https://go-review.googlesource.com/c/164201
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-01 07:44:37 +00:00
Ian Lance Taylor eb72a30f87 os: make openFdAt act like openFileNolog
- add EINTR loop on Darwin
- return PathError on error
- call newFile rather than NewFile

This tries to minimize the possibility of any future changes.
It would be nice to put openFdAt in the same file as openFileNolog,
but build tags forbid.

Updates #29983

Change-Id: I866002416d6473fbfd80ff6ef09b2bc4607f2934
Reviewed-on: https://go-review.googlesource.com/c/160181
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2019-01-30 01:07:50 +00:00
Ian Lance Taylor a5318bf5d6 os: always treat files as unpollable on FreeBSD
Fixes #29633
Updates #27619

Change-Id: I1e38569ea2a02423b028331f2ed987d3ae47fd2e
Reviewed-on: https://go-review.googlesource.com/c/157099
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-01-09 21:50:11 +00:00
Yuval Pavel Zholkover 79c50c4d57 os: disable the use of netpoll on regular files on *BSDs.
The kqueue based netpoller always registers file descriptors with EVFILT_READ and EVFILT_WRITE.
However only EVFILT_READ notification is supported for regular files.
On FreeBSD a regular file is always reported as ready for writing, resulting in a busy wait.
On Darwin, Dragonfly, NetBSD and OpenBSD, a regular file is reported as ready for both reading and writing only once.

Updates #19093

Change-Id: If284341f60c6c2332fb5499637d4cfa7a4e26b7b
Reviewed-on: https://go-review.googlesource.com/c/156379
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-09 05:04:26 +00:00
Elias Naur 9eb383e8f0 runtime,os,syscall,internal/poll: replace getdirentries on iOS
The getdirentries syscall is considered private API on iOS and is
rejected by the App Store submission checks. Replace it with the
fdopendir/readdir_r/closedir syscalls.

Fixes #28984

Change-Id: I73341b124310e9cb34834a95f946769f337ec5b7
Reviewed-on: https://go-review.googlesource.com/c/153338
Reviewed-by: Keith Randall <khr@golang.org>
2018-12-13 17:10:32 +00:00
Clément Chigot 835f983da6 os: add AIX operating system
This commit adds AIX operating system to os package for ppc64
architecture.

Updates: #25893

Change-Id: Ieb9a2b3ac5b9abd3b5ab68eb732c17b6256d624d
Reviewed-on: https://go-review.googlesource.com/c/138725
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-11 15:41:04 +00:00
Ian Lance Taylor 0bad63437e os: document that Remove removes only empty directories
Fixes #26507

Change-Id: I967e4f897ca891c70ca8bb0e66b984530c240815
Reviewed-on: https://go-review.googlesource.com/127835
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2018-08-03 23:09:31 +00:00
Ian Lance Taylor e86bbc2e27 os: clarify that Close cancels pending I/O
Change-Id: I6be6818d951a999f916c2266a6753a5ce5144ee7
Reviewed-on: https://go-review.googlesource.com/122955
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2018-07-11 00:06:19 +00:00
Ian Lance Taylor 4a778cdf33 os: don't poll fifos on Darwin
The Darwin kqueue implementation doesn't report any event when the
last writer for a fifo is closed.

Fixes #24164

Change-Id: Ic2c47018ef1284bf2e26379f8dd7646edaad4d05
Reviewed-on: https://go-review.googlesource.com/118566
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-06-13 20:27:55 +00:00
Tim Cooper 161874da2a all: update comment URLs from HTTP to HTTPS, where possible
Each URL was manually verified to ensure it did not serve up incorrect
content.

Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2018-06-01 21:52:00 +00:00
Richard Musiol 063f97a611 os: add js/wasm architecture
This commit adds the js/wasm architecture to the os package.

Access to the actual file system is supported through Node.js.

Updates #18892

Change-Id: I6fa642fb294ca020b2c545649d4324d981aa0408
Reviewed-on: https://go-review.googlesource.com/109977
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-06-01 05:18:38 +00:00
Nick Patavalis ea5825b0b6 os: use poller when NewFile is called with a blocking descriptor.
If NewFile is called with a file descriptor that is already set to
non-blocking mode, it tries to return a pollable file (one for which
SetDeadline methods work) by adding the filedes to the poll/netpoll
mechanism. If called with a filedes in blocking mode, it returns a
non-pollable file, as it always did.

Fixes #22939
Updates #24331

Change-Id: Id54c8be1b83e6d35e14e76d7df0e57a9fd64e176
Reviewed-on: https://go-review.googlesource.com/100077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-04-11 17:39:11 +00:00