Go 1.5 worked with Unicode console input but not ^Z.
Go 1.6 did not work with Unicode console input but did handle one ^Z case.
Go 1.7 did not work with Unicode console input but did handle one ^Z case.
The intent of this CL is for Go 1.8 to work with Unicode console input
and also handle all ^Z cases.
Here's a simple test program for reading from the console.
It prints a "> " prompt, calls read, prints what it gets, and repeats.
package main
import (
"fmt"
"os"
)
func main() {
p := make([]byte, 100)
fmt.Printf("> ")
for {
n, err := os.Stdin.Read(p)
fmt.Printf("[%d %q %v]\n> ", n, p[:n], err)
}
}
On Unix, typing a ^D produces a break in the input stream.
If the ^D is at the beginning of a line, then the 0 bytes returned
appear as an io.EOF:
$ go run /tmp/x.go
> hello
[6 "hello\n" <nil>]
> hello^D[5 "hello" <nil>]
> ^D[0 "" EOF]
> ^D[0 "" EOF]
> hello^Dworld
[5 "hello" <nil>]
> [6 "world\n" <nil>]
>
On Windows, the EOF character is ^Z, not ^D, and there has
been a long-standing problem that in Go programs, ^Z on Windows
does not behave in the expected way, namely like ^D on Unix.
Instead, the ^Z come through as literal ^Z characters:
C:\>c:\go1.5.4\bin\go run x.go
> ^Z
[3 "\x1a\r\n" <nil>]
> hello^Zworld
[13 "hello\x1aworld\r\n" <nil>]
>
CL 4310 attempted to fix this bug, then known as #6303,
by changing the use of ReadConsole to ReadFile.
This CL was released as part of Go 1.6 and did fix the case
of a ^Z by itself, but not as part of a larger input:
C:\>c:\go1.6.3\bin\go run x.go
> ^Z
[0 "" EOF]
> hello^Zworld
[13 "hello\x1aworld\r\n" <nil>]
>
So the fix was incomplete.
Worse, the fix broke Unicode console input.
ReadFile does not handle Unicode console input correctly.
To handle Unicode correctly, programs must use ReadConsole.
Early versions of Go used ReadFile to read the console,
leading to incorrect Unicode handling, which was filed as #4760
and fixed in CL 7312053, which switched to ReadConsole
and was released as part of Go 1.1 and still worked as of Go 1.5:
C:\>c:\go1.5.4\bin\go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[16 "hello world™\r\n" <nil>]
>
But in Go 1.6:
C:\>c:\go1.6.3\bin\go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[0 "" EOF]
>
That is, changing back to ReadFile in Go 1.6 reintroduced #4760,
which has been refiled as #17097. (We have no automated test
for this because we don't know how to simulate console input
in a test: it appears that one must actually type at a keyboard
to use the real APIs. This CL at least adds a comment warning
not to reintroduce ReadFile again.)
CL 29493 attempted to fix#17097, but it was not a complete fix:
the hello world™ example above still fails, as does Shift-JIS input,
which was filed as #17939.
CL 29493 also broke ^Z handling, which was filed as #17427.
This CL attempts the never before successfully performed trick
of simultaneously fixing Unicode console input and ^Z handling.
It changes the console input to use ReadConsole again,
as in Go 1.5, which seemed to work for all known Unicode input.
Then it adds explicit handling of ^Z in the input stream.
(In the case where standard input is a redirected file, ^Z processing
should not happen, and it does not, because this code path is only
invoked when standard input is the console.)
With this CL:
C:\>go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[16 "hello world™\r\n" <nil>]
> ^Z
[0 "" EOF]
> [2 "\r\n" <nil>]
> hello^Zworld
[5 "hello" <nil>]
> [0 "" EOF]
> [7 "world\r\n" <nil>]
This almost matches Unix:
$ go run /tmp/x.go
> hello
[6 "hello\n" <nil>]
> hello world™
[15 "hello world™\n" <nil>]
> ^D
[0 "" EOF]
> [1 "\n" <nil>]
> hello^Dworld
[5 "hello" <nil>]
> [6 "world\n" <nil>]
>
The difference is in the handling of hello^Dworld / hello^Zworld.
On Unix, hello^Dworld terminates the read of hello but does not
result in a zero-length read between reading hello and world.
This is dictated by the tty driver, not any special Go code.
On Windows, in this CL, hello^Zworld inserts a zero length read
result between hello and world, which is treated as an interior EOF.
This is implemented by the Go code in this CL, but it matches the
handling of ^Z on the console in other programs:
C:\>copy con x.txt
hello^Zworld
1 file(s) copied.
C:\>type x.txt
hello
C:\>
A natural question is how to test all this. As noted above, we don't
know how to write automated tests using the actual Windows console.
CL 29493 introduced the idea of substituting a different syscall.ReadFile
implementation for testing; this CL continues that idea but substituting
for syscall.ReadConsole instead. To avoid the regression of putting
ReadFile back, this CL adds a comment warning against that.
Fixes#17427.
Fixes#17939.
Change-Id: Ibaabd0ceb2d7af501d44ac66d53f64aba3944142
Reviewed-on: https://go-review.googlesource.com/33451
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Quentin Smith <quentin@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes the os test on the Android builder.
Change-Id: Ibb9db712156a620fcccf515e035475c5e2f535a5
Reviewed-on: https://go-review.googlesource.com/33650
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Before this CL, Go programs in c-archive or c-shared buildmodes
would not handle SIGPIPE. That leads to surprising behaviour where
writes on a closed pipe or socket would raise SIGPIPE and terminate
the program. This CL changes the Go runtime to handle
SIGPIPE regardless of buildmode. In addition, SIGPIPE from non-Go
code is forwarded.
Fixes#17393
Updates #16760
Change-Id: I155e82020a03a5cdc627a147c27da395662c3fe8
Reviewed-on: https://go-review.googlesource.com/32796
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TestReadStdin always fill up buffer provided by ReadFile caller full.
But we do not know if real ReadFile does the same. Add tests where
buffer is only filled with limited data.
Change-Id: I0fc776325c2b1fe60511126c439f4b0560e9d653
Reviewed-on: https://go-review.googlesource.com/33030
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add an explicit WriteString method to closeOnce that acquires the
writers lock. This overrides the one promoted from the
embedded *os.File field. The promoted one naturally does not acquire
the lock, and can therefore race with the Close method.
Fixes#17647.
Change-Id: I3460f2a0d503449481cfb2fd4628b4855ab0ecdf
Reviewed-on: https://go-review.googlesource.com/33298
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change-Id: I9a42cb55544185ade20b2a4a9de5d39a6cfc6fc6
Reviewed-on: https://go-review.googlesource.com/33172
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously, `os.Clearenv()` (by way of `syscall.Clearenv`) would simply
set all environment variables' values to `""` rather than actually
unsetting them causing subsequent `os.LookupEnv` calls to return that
they were still set.
Fixes#17902
Change-Id: I54081b4b98665e9a39f55ea7582c8d40bb8a2a22
Reviewed-on: https://go-review.googlesource.com/33168
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Add tmpdir as a parameter to the closure otherwise the subsequent
modifications to tmpdir causes only the last subdirectory to be
removed.
Additionally, add the missing argument for the t.Fatalf call.
Change-Id: I3df53f9051f7ea40cf3f846d47d9cefe445e9b9d
Reviewed-on: https://go-review.googlesource.com/32892
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
// Executable returns the path name for the executable that started
// the current process. There is no guarantee that the path is still
// pointing to the correct executable. If a symlink was used to start
// the process, depending on the operating system, the result might
// be the symlink or the path it pointed to. If a stable result is
// needed, path/filepath.EvalSymlinks might help.
//
// Executable returns an absolute path unless an error occurred.
//
// The main use case is finding resources located relative to an
// executable.
//
// Executable is not supported on nacl or OpenBSD (unless procfs is
// mounted.)
func Executable() (string, error) {
return executable()
}
Fixes#12773.
Change-Id: I469738d905b12f0b633ea4d88954f8859227a88c
Reviewed-on: https://go-review.googlesource.com/16551
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Windows has a limit of 260 characters on normal paths, but it's possible
to use longer paths by using "extended-length paths" that begin with
`\\?\`. This commit attempts to transparently convert an absolute path
to an extended-length path, following the subtly different rules those
paths require. It does not attempt to handle relative paths, which
continue to be passed to the operating system unmodified.
This adds a new test, TestLongPath, to the os package. This test makes
sure that it is possible to write a path at least 400 characters long
and runs on every platform. It also tests symlinks and hardlinks, though
symlinks are not testable with our builder configuration.
HasLink is moved to internal/testenv so it can be used by multiple tests.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
has Microsoft's documentation on extended-length paths.
Fixes#3358.
Fixes#10577.
Fixes#17500.
Change-Id: I4ff6bb2ef9c9a4468d383d98379f65cf9c448218
Reviewed-on: https://go-review.googlesource.com/32451
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TestRemoveDevNull was added in CL 31657. However, this test
was failing on Plan 9, because /dev/null was considered as
a regular file.
On Plan 9, there is no special mode to distinguish between
device files and regular files.
However, files are served by different servers. For example,
/dev/null is served by #c (devcons), while /bin/cat is served
by #M (devmnt).
We chose to consider only the files served by #M as regular
files. All files served by different servers will be considered
as device files.
Fixes#17598.
Change-Id: Ibb1c3357d742cf2a7de15fc78c9e436dc31982bb
Reviewed-on: https://go-review.googlesource.com/32152
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Current implementation of syscall.Readlink mistakenly calculates
the end offset of the PrintName field.
Also, there are some cases that the PrintName field is empty.
Instead, the CL uses SubstituteName with correct calculation.
Fixes#15978Fixes#16145
Change-Id: If3257137141129ac1c552d003726d5b9c08bb754
Reviewed-on: https://go-review.googlesource.com/31118
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Unix rejects this when new is a non-empty directory.
Other systems reject this when new is a directory, empty or not.
Make Unix reject empty directory too.
Fixes#14527.
Change-Id: Ice24b8065264c91c22cba24aa73e142386c29c87
Reviewed-on: https://go-review.googlesource.com/31358
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd.StdinPipe returns an io.WriteCloser.
It's reasonable to expect the caller not to call Write and Close simultaneously,
but there is an implicit Close in cmd.Wait that's not obvious.
We already synchronize the implicit Close in cmd.Wait against
any explicit Close from the caller. Also synchronize that implicit
Close against any explicit Write from the caller.
Fixes#9307.
Change-Id: I8561e9369d6e5ac88dfbca1175549f6dfa04b8ac
Reviewed-on: https://go-review.googlesource.com/31148
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL introduces first test for readConsole. And new test
discovered couple of problems with readConsole.
Console characters consist of multiple bytes each, but byte blocks
returned by syscall.ReadFile have no character boundaries. Some
multi-byte characters might start at the end of one block, and end
at the start of next block. readConsole feeds these blocks to
syscall.MultiByteToWideChar to convert them into utf16, but if some
multi-byte characters have no ending or starting bytes, the
syscall.MultiByteToWideChar might get confused. Current version of
syscall.MultiByteToWideChar call will make
syscall.MultiByteToWideChar ignore all these not complete
multi-byte characters.
The CL solves this issue by changing processing from "randomly
sized block of bytes at a time" to "one multi-byte character at a
time". New readConsole code calls syscall.ReadFile to get 1 byte
first. Then it feeds this byte to syscall.MultiByteToWideChar.
The new syscall.MultiByteToWideChar call uses MB_ERR_INVALID_CHARS
flag to make syscall.MultiByteToWideChar return error if input is
not complete character. If syscall.MultiByteToWideChar returns
correspondent error, we read another byte and pass 2 byte buffer
into syscall.MultiByteToWideChar, and so on until success.
Old readConsole code would also sometimes return no data if user
buffer was smaller then uint16 size, which would confuse callers
that supply 1 byte buffer. This CL fixes that problem too.
Fixes#17097
Change-Id: I88136cdf6a7bf3aed5fbb9ad2c759b6c0304ce30
Reviewed-on: https://go-review.googlesource.com/29493
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
It is possible (and common) for Windows systems to use a different codepage
for console applications from that used on normal windowed application
(called ANSI codepage); for instance, most of the western Europe uses
CP850 for console (for backward compatibility with MS-DOS), while
windowed applications use a different codepage depending on the country
(eg: CP1252 aka Latin-1). The usage being changed with this commit is
specifically related to decoding input coming from the console, so the
previous usage of the ANSI codepage was wrong.
Also fixes an issue that previous did convert bytes as NFD. Go is
designed to handle single Unicode code point. This fix change behaivor
to NFC.
Fixes#16857.
Change-Id: I4f41ae83ece47321b6e9a79a2087ecbb8ac066dd
Reviewed-on: https://go-review.googlesource.com/27575
Reviewed-by: Hiroshi Ioka <hirochachacha@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
This change makes sure that tests are run with the correct
version of the go tool. The correct version is the one that
we invoked with "go test", not the one that is first in our path.
Fixes#16577
Change-Id: If22c8f8c3ec9e7c35d094362873819f2fbb8559b
Reviewed-on: https://go-review.googlesource.com/28089
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In the case of a file being deleted while Readdir was running, it was
possible for File.Readdir to return an empty slice and a nil error,
counter to its documentation.
Fixes#16919
Change-Id: If0e42882eea52fbf5530317a1895f3829ea8e67b
Reviewed-on: https://go-review.googlesource.com/28056
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
New beginners are not familiar with open(2)-style masking of the
flags. Add an example demonstrates the flag or'ing.
Change-Id: Ifa8009c55173ba0dc6642c1d3b3124c766b1ebbb
Reviewed-on: https://go-review.googlesource.com/27996
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
https://go-review.googlesource.com/#/c/27580 added the test.
However the test use syscall.ELOOP which is not defined on plan9.
Move test code from "os_test.go" to "os_windows_test.go" to prevent
build error.
Change-Id: Ie7f05bfb9ab229e06a8e82a4b3b8a7ca82d4663b
Reviewed-on: https://go-review.googlesource.com/27833
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
The Windows version of Stat calls Readlink iteratively until
reaching a non-symlink file.
If the given file is a circular symlink, It never stops.
This CL defines the maximum number of symlink loop count.
If the loop count will exceed that number, Stat will return error.
Fixes#16538
Change-Id: Ia9f3f2259a8d32801461c5041cc24a34f9f81009
Reviewed-on: https://go-review.googlesource.com/27580
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
The os package sets a finalizer on *Process. I looked through all the
uses of *Process in the package, looking for each case where a *Process
was passed as an argument and the final reference to the argument was
not a function or method call. I added a call to runtime.KeepAlive after
each such final reference (there were only three).
The code is safe today without the KeepAlive calls because the compiler
keeps arguments alive for the duration of the function. However, that is
not a language requirement, so adding the KeepAlive calls ensures that
this code remains safe even if the compiler changes in the future.
I also removed an existing unnecessry call to runtime.KeepAlive. The
syscall.Syscall function is handled specially by the compiler to keep
its arguments alive.
Change-Id: Ibd2ff20b31ed3de4f6a59dd1633c1b44001d91d9
Reviewed-on: https://go-review.googlesource.com/27637
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
'-' is one of shell special parameters.
The existing implementation of isShellSpecialVar missed '-'
from the list, causing "$-" and "${-}" expand differently.
Fixes#16554
Change-Id: Iafc7984692cc83cff58f7c1e01267bf78b3a20a9
Reviewed-on: https://go-review.googlesource.com/25352
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Code movement only.
If someone finds function 'foo' in "foo_linux.go",
they will expect that the Window version of 'foo' exists in "foo_windows.go".
Current code doesn't follow this manner.
For example, 'sameFile' exists in "file_unix.go",
"stat_plan9.go" and "types_windows.go".
The CL address that problem by following rules:
* readdir family => dir.go, dir_$GOOS.go
* stat family => stat.go, stat_$GOOS.go
* path-functions => path_$GOOS.go
* sameFile => types.go, types_$GOOS.go
* process-functions => exec.go, exec_$GOOS.go
* hostname => sys.go, sys_$GOOS.go
Change-Id: Ic3c64663ce0b2a364d7a414351cd3c772e70187b
Reviewed-on: https://go-review.googlesource.com/27035
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reportedly waitid is not available for Ubuntu on Windows.
Fixes#16610.
Change-Id: Ia724f45a85c6d3467b847da06d8c65d280781dcd
Reviewed-on: https://go-review.googlesource.com/25507
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previously we started checking for context cancelation in Wait, but
that meant that when using StdoutPipe context cancelation never took
effect.
Fixes#16222.
Change-Id: I89cd26d3499a6080bf1a07718ce38d825561899e
Reviewed-on: https://go-review.googlesource.com/24650
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The previous fix was wrong because it had two misunderstandings on
freebsd32 calling convention like the following:
- 32-bit id1 implies that it is the upper half of 64-bit id, indeed it
depends on machine endianness.
- 32-bit ARM calling convension doesn't conform to freebsd32_args,
indeed it does.
This change fixes the bugs and makes blockUntilWaitable work correctly
on freebsd/{386,arm}.
Fixes#16064.
Change-Id: I820c6d01d59a43ac4f2ab381f757c03b14bca75e
Reviewed-on: https://go-review.googlesource.com/24064
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Russ Cox
David Crawshaw
Elias Naur
Alex Brainman
Ian Lance Taylor
Mikio Hara
Jesse Szwedko
Josh Bleecher Snyder
Brad Fitzpatrick
Mohit Agarwal
Shenghou Ma
Quentin Smith
David du Colombier
Dan Caddigan
Hiroshi Ioka
Jean-Nicolas Moal
Sina Siadat
Alexander Döring
Kale Blankenship
Yasuhiro Matsumoto
Emmanuel Odeke
bogem
Keith Randall
Jonathan Boulle
Jaana Burcu Dogan
LE Manh Cuong