The file name suffix arm64 already limits the file to be build only on arm64.
Change-Id: I33db713041b6dec9eb00889bac3b54c727e90743
Reviewed-on: https://go-review.googlesource.com/108986
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
"Span" is a commonly used term in many distributed tracing systems
(Dapper, OpenCensus, OpenTracing, ...). They use it to refer to a
period of time, not necessarily tied into execution of underlying
processor, thread, or goroutine, unlike the "Span" of runtime/trace
package.
Since distributed tracing and go runtime execution tracing are
already similar enough to cause confusion, this CL attempts to avoid
using the same word if possible.
"Region" is being used in a certain tracing system to refer to a code
region which is pretty close to what runtime/trace.Span currently
refers to. So, replace that.
https://software.intel.com/en-us/itc-user-and-reference-guide-defining-and-recording-functions-or-regions
This CL also tweaks APIs a bit based on jbd and heschi's comments:
NewContext -> NewTask
and it now returns a Task object that exports End method.
StartSpan -> StartRegion
and it now returns a Region object that exports End method.
Also, changed WithSpan to WithRegion and it now takes func() with no
context. Another thought is to get rid of WithRegion. It is a nice
concept but in practice, it seems problematic (a lot of code churn,
and polluting stack trace). Already, the tracing concept is very low
level, and we hope this API to be used with great care.
Recommended usage will be
defer trace.StartRegion(ctx, "someRegion").End()
Left old APIs untouched in this CL. Once the usage of them are cleaned
up, they will be removed in a separate CL.
Change-Id: I73880635e437f3aad51314331a035dd1459b9f3a
Reviewed-on: https://go-review.googlesource.com/108296
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JBD <jbd@google.com>
This change adds support for the splice system call on Linux,
for the purpose of optimizing (*TCPConn).ReadFrom by reducing
copies of data from and to userspace. It does so by creating a
temporary pipe and splicing data from the source connection to the
pipe, then from the pipe to the destination connection. The pipe
serves as an in-kernel buffer for the data transfer.
No new API is added to package net, but a new Splice function is
added to package internal/poll, because using splice requires help
from the network poller. Users of the net package should benefit
from the change transparently.
This change only enables the optimization if the Reader in ReadFrom
is a TCP connection. Since splice is a more general interface, it
could, in theory, also be enabled if the Reader were a unix socket,
or the read half of a pipe.
However, benchmarks show that enabling it for unix sockets is most
likely not a net performance gain. The tcp <- unix case is also
fairly unlikely to be used very much by users of package net.
Enabling the optimization for pipes is also problematic from an
implementation perspective, since package net cannot easily get at
the *poll.FD of an *os.File. A possible solution to this would be
to dup the pipe file descriptor, register the duped descriptor with
the network poller, and work on that *poll.FD instead of the original.
However, this seems too intrusive, so it has not been done. If there
was a clean way to do it, it would probably be worth doing, since
splicing from a pipe to a socket can be done directly.
Therefore, this patch only enables the optimization for what is likely
the most common use case: tcp <- tcp.
The following benchmark compares the performance of the previous
userspace genericReadFrom code path to the new optimized code path.
The sub-benchmarks represent chunk sizes used by the writer on the
other end of the Reader passed to ReadFrom.
benchmark old ns/op new ns/op delta
BenchmarkTCPReadFrom/1024-4 4727 4954 +4.80%
BenchmarkTCPReadFrom/2048-4 4389 4301 -2.01%
BenchmarkTCPReadFrom/4096-4 4606 4534 -1.56%
BenchmarkTCPReadFrom/8192-4 5219 4779 -8.43%
BenchmarkTCPReadFrom/16384-4 8708 8008 -8.04%
BenchmarkTCPReadFrom/32768-4 16349 14973 -8.42%
BenchmarkTCPReadFrom/65536-4 35246 27406 -22.24%
BenchmarkTCPReadFrom/131072-4 72920 52382 -28.17%
BenchmarkTCPReadFrom/262144-4 149311 95094 -36.31%
BenchmarkTCPReadFrom/524288-4 306704 181856 -40.71%
BenchmarkTCPReadFrom/1048576-4 674174 357406 -46.99%
benchmark old MB/s new MB/s speedup
BenchmarkTCPReadFrom/1024-4 216.62 206.69 0.95x
BenchmarkTCPReadFrom/2048-4 466.61 476.08 1.02x
BenchmarkTCPReadFrom/4096-4 889.09 903.31 1.02x
BenchmarkTCPReadFrom/8192-4 1569.40 1714.06 1.09x
BenchmarkTCPReadFrom/16384-4 1881.42 2045.84 1.09x
BenchmarkTCPReadFrom/32768-4 2004.18 2188.41 1.09x
BenchmarkTCPReadFrom/65536-4 1859.38 2391.25 1.29x
BenchmarkTCPReadFrom/131072-4 1797.46 2502.21 1.39x
BenchmarkTCPReadFrom/262144-4 1755.69 2756.68 1.57x
BenchmarkTCPReadFrom/524288-4 1709.42 2882.98 1.69x
BenchmarkTCPReadFrom/1048576-4 1555.35 2933.84 1.89x
Fixes#10948
Change-Id: I3ce27f21f7adda8b696afdc48a91149998ae16a5
Reviewed-on: https://go-review.googlesource.com/107715
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I6e1fa67dc9d4d151c90eb19a6f736e4daa7d4fb3
Reviewed-on: https://go-review.googlesource.com/107615
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 99337 introduced a file with CRLF line endings. Convert them to LF
line endings as in all other Go files.
Change-Id: I68b28fd443f05bebbbd9280d1821c4ccd33a4a8a
Reviewed-on: https://go-review.googlesource.com/108075
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
RawRead assumes the callback will perform either (a) a blocking read
and always return true, (b) a blocking read with a SO_RCVTIMEO set
returning false on WSAETIMEDOUT, or (c) a non-blocking read
returning false on WSAEWOULDBLOCK. In the latter two cases, it uses
a 0-byte overlapped read for notifications from the IOCP runtime
when the socket becomes readable before trying again.
RawWrite assumes the callback will perform blocking write and will
always return true, and makes no effort to tie into the runtime loop.
Change-Id: Ib10074e9d502c040294f41a260e561e84208652f
Reviewed-on: https://go-review.googlesource.com/76391
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
CL 92475 added WSAEMSGSIZE const to syscall package. But there
is already copy of WSAEMSGSIZE in internal/syscall/windows.
So delete syscall.WSAEMSGSIZE
Change-Id: I0b81fa5dcf846887a0cb27d8bbd7e250860627b5
Reviewed-on: https://go-review.googlesource.com/106956
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If NewFile is called with a file descriptor that is already set to
non-blocking mode, it tries to return a pollable file (one for which
SetDeadline methods work) by adding the filedes to the poll/netpoll
mechanism. If called with a filedes in blocking mode, it returns a
non-pollable file, as it always did.
Fixes#22939
Updates #24331
Change-Id: Id54c8be1b83e6d35e14e76d7df0e57a9fd64e176
Reviewed-on: https://go-review.googlesource.com/100077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Use constant masks and align the definition of isSet with
arm64 and ppc64x.
Change-Id: I0c6eae30da5e3ce797cde0dab4a39855d4d245d9
Reviewed-on: https://go-review.googlesource.com/94759
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime.alginit needs runtime/support_{aes,ssse3,sse41} feature flag
to init aeshash function but internal/cpu.init not be called yet.
This CL will call internal/cpu.initialize before runtime.alginit, so
that we can move all cpu features related code to internal/cpu.
Change-Id: I00b8e403ace3553f8c707563d95f27dade0bc853
Reviewed-on: https://go-review.googlesource.com/104636
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add the following helpers in lookup_windows.go:
1) lookupGroupName() is used to obtain the SID of a group based
on name.
2) listGroupsForUsernameAndDomain() uses NetUserGetLocalGroups()
as a WINAPI backend to obtain the list of local groups for this
user.
3) lookupUserPrimaryGroup() is now used to populate the User.Gid
field when looking up a user by name.
Implement listGroups(), lookupGroupId(), lookupGroup() and no longer
return unimplemented errors.
Do not skip Windows User.Gid tests in user_test.go.
Change-Id: I81fd41b406da51f9a4cb24e50d392a333df81141
GitHub-Last-Rev: d1448fd55d
GitHub-Pull-Request: golang/go#24222
Reviewed-on: https://go-review.googlesource.com/98137
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When snapshotting the execution time stats of a goroutine
we take into account the intermediate values kepts in the GDesc's
gdesc field. At the end of goroutine analysis, we go through all
goroutines and replace the GExecutionStat with the new snapshot.
Here the mistake was that we replaced the GExecutionStat with
the value that reflects the intermediate values, but did clear
the intermediate values. So, when the following finalizeActiveSpans
runs, it trieds to add the intermediate values again (double-counting)
when taking a snapshot.
Fix the issue by using the finalized GExecutionStat instead of
recomputing the snapshot. While we are here, refactor the finalization
logic so it can be used when processing GoEnd, GoStop events.
Change-Id: Ibdb342214c29b65d4ea37e52b1f8b83f1fb20143
Reviewed-on: https://go-review.googlesource.com/103157
Reviewed-by: Peter Weinberger <pjw@google.com>
Move part of UserSpan event processing from cmd/trace.analyzeAnnotations
to internal/trace.GoroutineStats that returns analyzed per-goroutine
execution information. Now the execution information includes list of
spans and their execution information.
cmd/trace.analyzeAnnotations utilizes the span execution information
from internal/trace.GoroutineStats and connects them with task
information.
Change-Id: Ib7f79a3ba652a4ae55cd81ea17565bcc7e241c5c
Reviewed-on: https://go-review.googlesource.com/101917
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
As found by unparam. Picked the low-hanging fruit, consisting only of
errors that were always nil and results that were never used. Left out
those that were useful for consistency with other func signatures.
Change-Id: I06b52bbd3541f8a5d66659c909bd93cb3e172018
Reviewed-on: https://go-review.googlesource.com/102418
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reverts commit bfa8b6f8ff.
Reason for revert: This depends on another CL which is not yet submitted.
Change-Id: I50e7594f1473c911a2079fe910849a6694ac6c07
Reviewed-on: https://go-review.googlesource.com/101496
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Goroutine analysis reports the sum of all overlapping GC intervals as
the GCTime of a goroutine. The computation is done by adding the length
of a completed GC interval to 'active' goroutines when processing the
corresponding EvGCDone event. This change fixes the two corner cases
the current implementation ignores:
1) Goroutine that ends during GC. Previously, this goroutine was ignored
and GC time was undercounted. We handle this case by setting the
gcStartTime only when GC is active and handling non-zero gcStartTime
when processing EvGoStop and EvGoStart.
2) Goroutine that starts during GC. Previously, the entire GC interval
length was added to the Goroutine's GCTime which resulted in overcount
of GC time. We handle this case by computing the length of overlapped
period precisely.
Change-Id: Ifa8e82672ec341b5ff87837209f4311fa7262b7f
Reviewed-on: https://go-review.googlesource.com/100842
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is an updated version of golang.org/cl/96395, with the fix to
TestUserSpan.
This reverts commit 7b6f6267e90a8e4eab37a3f2164ba882e6222adb.
Change-Id: I31eec8ba0997f9178dffef8dac608e731ab70872
Reviewed-on: https://go-review.googlesource.com/98236
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Missed removing the argument loading from the indexbody function.
Change-Id: Ia1391231fc99771d00410a09fe80a09f08ceed02
Reviewed-on: https://go-review.googlesource.com/98575
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Make ReadSubKeyNames work even if key is opened with only
ENUMERATE_SUB_KEYs access rights mask.
Fixes#23869
Change-Id: I138bd51715fdbc3bda05607c64bde1150f4fe6b2
Reviewed-on: https://go-review.googlesource.com/97435
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Move the IndexByte function from the runtime to a new bytealg package.
The new package will eventually hold all the optimized assembly for
groveling through byte slices and strings. It seems a better home for
this code than randomly keeping it in runtime.
Once this is in, the next step is to move the other functions
(Compare, Equal, ...).
Update #19792
This change seems complicated enough that we might just declare
"not worth it" and abandon. Opinions welcome.
The core assembly is all unchanged, except minor modifications where
the code reads cpu feature bits.
The wrapper functions have been cleaned up as they are now actually
checked by vet.
Change-Id: I9fa75bee5d85db3a65b3fd3b7997e60367523796
Reviewed-on: https://go-review.googlesource.com/98016
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Even though undocumented, the assumption is the Event's link field
points to the following event in the future. The new span/task event
processing breaks the assumption.
Change-Id: I4ce2f30c67c4f525ec0a121a7e43d8bdd2ec3f77
Reviewed-on: https://go-review.googlesource.com/96395
Reviewed-by: Heschi Kreinick <heschi@google.com>
The SOCK_CLOEXEC and SOCK_NONBLOCK flags to the socket syscall and the
accept4 syscall are supported since OpenBSD 5.7.
Follows CL 40895 and CL 94295
Change-Id: Icaf35ace2ef5e73279a70d4f1a9fbf3be9371e6c
Reviewed-on: https://go-review.googlesource.com/97196
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
newUserFromSid() is extended so that the retriaval of the user home
path based on a user SID becomes possible.
(1) The primary method it uses is to lookup the Windows registry for
the following key:
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList\[SID]
If the key does not exist the user might not have logged in yet.
If (1) fails it falls back to (2)
(2) The second method the function uses is to look at the default home
path for users (e.g. WINAPI's GetProfilesDirectory()) and append
the username to that. The procedure is in the lines of:
c:\Users + \ + <username>
The function newUser() now requires the following arguments:
uid, gid, dir, username, domain
This is done to avoid multiple calls to usid.String() and
usid.LookupAccount("") in the case of a newUserFromSid()
call stack.
The functions current() and newUserFromSid() both call newUser()
supplying the arguments in question. The helpers
lookupUsernameAndDomain() and findHomeDirInRegistry() are
added.
This commit also updates:
- go/build/deps_test.go, so that the test now includes the
"internal/syscall/windows/registry" import.
- os/user/user_test.go, so that User.HomeDir is tested on Windows.
GitHub-Last-Rev: 25423e2a38
GitHub-Pull-Request: golang/go#23822
Change-Id: I6c3ad1c4ce3e7bc0d1add024951711f615b84ee5
Reviewed-on: https://go-review.googlesource.com/93935
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Calling UDPConn readers (Read, ReadFrom, ReadMsgUDP) to read part of
datagram returns error (in Windows), mentioning there is more data
available, and 0 as size of read data, even though part of data is
already read.
This fix makes UDPConn readers to return truncated payload size,
even there is error due more data available to read.
Fixes#14074
Updates #18056
Change-Id: Id7eec7f544dd759b2d970fa2561eef2937ec4662
Reviewed-on: https://go-review.googlesource.com/92475
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Also add testdata for version 1.11 including UserTaskSpan test trace.
Change-Id: I673fb29bb3aee96a14fadc0ab860d4f5832143f5
Reviewed-on: https://go-review.googlesource.com/93795
Reviewed-by: Heschi Kreinick <heschi@google.com>
This implements the annotation API proposed in golang.org/cl/63274.
traceString is updated to protect the string map with trace.stringsLock
because the assumption that traceString is called by a single goroutine
(either at the beginning of tracing and at the end of tracing when
dumping all the symbols and function names) is no longer true.
traceString is used by the annotation apis (NewContext, StartSpan, Log)
to register frequently appearing strings (task and span names, and log
keys) after this change.
NewContext -> one or two records (EvString, EvUserTaskCreate)
end function -> one record (EvUserTaskEnd)
StartSpan -> one or two records (EvString, EvUserSpan)
span end function -> one or two records (EvString, EvUserSpan)
Log -> one or two records (EvString, EvUserLog)
EvUserLog record is of the typical record format written by traceEvent
except that it is followed by bytes that represents the value string.
In addition to runtime/trace change, this change includes
corresponding changes in internal/trace to parse the new record types.
Future work to improve efficiency:
More efficient unique task id generation instead of atomic. (per-P
counter).
Instead of a centralized trace.stringsLock, consider using per-P
string cache or something more efficient.
R=go1.11
Change-Id: Iec9276c6c51e5be441ccd52dec270f1e3b153970
Reviewed-on: https://go-review.googlesource.com/71690
Reviewed-by: Austin Clements <austin@google.com>
This CL presents the proposed user annotation API skeleton.
This CL bumps up the trace version to 1.11.
Design doc https://goo.gl/iqJfJ3
Implementation CLs are followed.
The API introduces three basic building blocks. Log, Span, and Task.
Log is for basic logging. When called, the message will be recorded
to the trace along with timestamp, goroutine id, and stack info.
trace.Log(ctx, messageType message)
Span can be thought as an extension of log to record interesting
time interval during a goroutine's execution. A span is local to a
goroutine by definition.
trace.WithSpan(ctx, "doVeryExpensiveOp", func(ctx context) {
/* do something very expensive */
})
Task is higher-level concept that aids tracing of complex operations
that encompass multiple goroutines or are asynchronous.
For example, an RPC request, a HTTP request, a file write, or a
batch job can be traced with a Task.
Note we chose to design the API around context.Context so it allows
easier integration with other tracing tools, often designed around
context.Context as well. Log and WithSpan APIs recognize the task
information embedded in the context and record it in the trace as
well. That allows the Go execution tracer to associate and group
the spans and log messages based on the task information.
In order to create a Task,
ctx, end := trace.NewContext(ctx, "myTask")
defer end()
The Go execution tracer measures the time between the task created
and the task ended for the task latency.
More discussion history in golang.org/cl/59572.
Update #16619
R=go1.11
Change-Id: I59a937048294dafd23a75cf1723c6db461b193cd
Reviewed-on: https://go-review.googlesource.com/63274
Reviewed-by: Austin Clements <austin@google.com>
Per the notice in the Go 1.10 release notes, this change drops the
support for Windows Vista or below (including Windows XP) and
simplifies the code for the sake of maintenance.
There is one exception to the above. The code related to DLL and
system calls still remains in the runtime package. The remaining code
will be refined and used for supporting upcoming Windows versions in
future.
Updates #17245Fixes#23072
Change-Id: I9e2821721f25ef9b83dfbf85be2b7ee5d9023aa5
Reviewed-on: https://go-review.googlesource.com/94255
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
NetBSD supports the SOCK_CLOEXEC and SOCK_NONBLOCK flags to the socket
syscall since version 6.0. The same version also introduced the paccept
syscall which can be used to implement syscall.Accept4.
Follows CL 40895
Change-Id: I9e4e1829b0382744c7799f4e58929a53b4e193f7
Reviewed-on: https://go-review.googlesource.com/94295
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This makes the constant names less verbose and aligns them more
with the Linux kernel which uses HWCAP_XXX for the constant names.
Change-Id: Ia7d079b59b57978adc045945951eaa1d99b41fac
Reviewed-on: https://go-review.googlesource.com/91738
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
golang.org/cl/81315 attempted to distinguish system goroutines
by examining the function name in the goroutine stack. It assumes that
the information would be available when GoSysBlock or GoInSyscall
events are processed, but it turned out the stack information is
set too late (when the goroutine gets a chance to run).
This change initializes the goroutine information entry when
processing GoCreate event which should be one of the very first
events for the every goroutine in trace.
Fixes#22574
Change-Id: I1ed37087ce2e78ed27c9b419b7d942eb4140cc69
Reviewed-on: https://go-review.googlesource.com/83595
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
os.NewFile doesn't put the fd into non-blocking mode.
In most cases, an *os.File returned by os.NewFile is in blocking mode.
Updates #7970
Updates #21856
Updates #23111
Change-Id: Iab08432e41f7ac1b5e25aaa8855d478adb7f98ed
Reviewed-on: https://go-review.googlesource.com/83995
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When we write a cached test result, we now also write a log of the
environment variables and files inspected by the test run,
along with a hash of their content. Before reusing a cached test result,
we recompute the hash of the content specified by the log, and only
use the result if that content has not changed.
This makes test caching behave correctly for tests that consult
environment variables or stat or read files or directories.
Fixes#22593.
Change-Id: I8608798e73c90e0c1911a38bf7e03e1232d784dc
Reviewed-on: https://go-review.googlesource.com/81895
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The DNS code can start goroutines and not wait for them to complete.
This does no harm, but in tests this can cause a race condition with
the test hooks that are installed and unintalled around the tests.
Add a WaitGroup that tests of DNS can use to avoid the race.
Fixes#21090
Change-Id: I6c1443a9c2378e8b89d0ab1d6390c0e3e726b0ce
Reviewed-on: https://go-review.googlesource.com/82795
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It appears that TransmitFile Windows API does not work with Windows
pipes. So just copy data from pipe and into TCP connection manually.
Fixes#22278
Change-Id: I4810caca5345eac5bffb3176956689b8ae993256
Reviewed-on: https://go-review.googlesource.com/79775
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Test is in os/signal package because the problem is signal related.
Fixes#22838.
Change-Id: I223eeebb5fbc972910737eddef8ab9784cb984a6
Reviewed-on: https://go-review.googlesource.com/79215
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
change hash/crc32 package to use cpu package instead of using
runtime internal variables to check crc32 instruction
Change-Id: I8f88d2351bde8ed4e256f9adf822a08b9a00f532
Reviewed-on: https://go-review.googlesource.com/76490
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
I submitted two CLs which broke the build. Add temporary placeholder
with false bools to fix the build and restore old behavior.
Updates golang/go#22718 (details of why it broke)
Change-Id: I1f30624e14f631a95f4eff5aae462f1091f723a2
Reviewed-on: https://go-review.googlesource.com/77590
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This means {Read,Write}Msg{UDP,IP} now work on windows.
Fixes#9252
Change-Id: Ifb105f9ad18d61289b22d7358a95faabe73d2d02
Reviewed-on: https://go-review.googlesource.com/76393
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
CL 75253 introduced new SysProcAttr.Token field as Handle.
But we already have exact type for it - Token. Use Token
instead of Handle everywhere - it saves few type conversions
and provides better documentation for new API.
Change-Id: Ibc5407a234a1f49804de15a24b27c8e6a6eba7e0
Reviewed-on: https://go-review.googlesource.com/76314
Reviewed-by: Ian Lance Taylor <iant@golang.org>
WSASocket (unlike socket call) allows to create sockets that
will not be inherited by child process. So call WSASocket to
save on using syscall.ForkLock and calling syscall.CloseOnExec.
Some very old versions of Windows do not have that functionality.
Call socket, if WSASocket failed, to support these.
Change-Id: I2dab9fa00d1a8609dd6feae1c9cc31d4e55b8cb5
Reviewed-on: https://go-review.googlesource.com/72590
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This makes the output they print refer to the code that called them.
For example, instead of
=== RUN TestWindowsStackMemoryCgo
--- SKIP: TestWindowsStackMemoryCgo (0.00s)
testenv.go:213: skipping known flaky test ...
PASS
we see
=== RUN TestWindowsStackMemoryCgo
--- SKIP: TestWindowsStackMemoryCgo (0.00s)
crash_cgo_test.go:471: skipping known flaky test ...
PASS
Change-Id: I5f4c77c3aeab5c0e43c6dde2f15db70a6df24603
Reviewed-on: https://go-review.googlesource.com/76031
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add support for ADX cpuid bit detection and all instructions,
implied by that bit (ADOX/ADCX). They are useful for rsa and math/big in
general.
Change-Id: Idaa93303ead48fd18b9b3da09b3e79de2f7e2193
Reviewed-on: https://go-review.googlesource.com/74850
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
If the go install doesn't use the same flags as the main build
it can overwrite the installed standard library, leading to
flakiness and slow future tests.
Force uses of 'go install' etc to propagate $GO_GCFLAGS
or disable them entirely, to avoid problems.
As I understand it, the main place this happens is the ssacheck builder.
If there are other uses that need to run some of the now-disabled
tests we can reenable fixed tests in followup CLs.
Change-Id: Ib860a253539f402f8a96a3c00ec34f0bbf137c9a
Reviewed-on: https://go-review.googlesource.com/74470
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Add SetDeadline, SetReadDeadline, and SetWriteDeadline methods to os.File,
just as they exist today for the net package.
Fixes#22114
Change-Id: I4d390d739169b991175baba676010897dc8568fa
Reviewed-on: https://go-review.googlesource.com/71770
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
On systems that use kqueue, we always register descriptors for both
EVFILT_READ and EVFILT_WRITE. On at least FreeBSD and OpenBSD, when
the write end of a pipe is registered for EVFILT_READ and EVFILT_WRITE
events, and the read end of the pipe is closed, kqueue reports an
EVFILT_READ event with EV_EOF set, but does not report an EVFILT_WRITE
event. Since the write to the pipe is waiting for an EVFILT_WRITE
event, closing the read end of a pipe can cause the write end to hang
rather than attempt another write which will fail with EPIPE.
Fix this by treating EVFILT_READ with EV_EOF set as making both reads
and writes ready to proceed.
The real test for this is in CL 71770, which tests using various
timeouts with pipes.
Updates #22114
Change-Id: Ib23fbaaddbccd8eee77bdf18f27a7f0aa50e2742
Reviewed-on: https://go-review.googlesource.com/71973
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
No test because at present it is never called in a way that fails.
When #22114 is implemented, failure will be possible. Not including this
change in that work because this change is separable and clearly correct.
Updates #22114
Change-Id: I81eb9eec8800e8082d918c0e5fb71282f538267e
Reviewed-on: https://go-review.googlesource.com/71751
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Joe Tsai <joetsai@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 36799 made SetFileCompletionNotificationModes to be called for
file handles. I don't think it is correct. Revert that change.
Fixes#22024Fixes#22207
Change-Id: I26260e8a727131cffbf60958d79eca2457495554
Reviewed-on: https://go-review.googlesource.com/69871
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Current code assumes that SetFileCompletionNotificationModes
is safe to call even if we know that it is not safe to use
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS flag. It appears (see issue #22149),
SetFileCompletionNotificationModes crashes when we call it without
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS flag.
Do not call SetFileCompletionNotificationModes in that situation.
We are allowed to do that, because SetFileCompletionNotificationModes
is just an optimisation.
Fixes#22149
Change-Id: I0ad3aff4eabd8c27739417a62c286b1819ae166a
Reviewed-on: https://go-review.googlesource.com/69870
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.LastIndexByte was introduced in go1.5 and it can be used
effectively wherever the second argument to strings.LastIndex is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.LastIndex.
Change-Id: I7b5679d616197b055cffe6882a8675d24a98b574
Reviewed-on: https://go-review.googlesource.com/66372
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This permits the program to reliably know that when the Close method
returns, the descriptor has definitely been closed. This matters at
least for listeners.
Fixes#21856
Updates #7970
Change-Id: I1fd0cfd2333649e6e67c6ae956e19fdff3a35a83
Reviewed-on: https://go-review.googlesource.com/66150
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <joetsai@google.com>
internal/poll package assumes that only net sockets use runtime
netpoller on windows. We get memory corruption if other file
handles are passed into runtime poller. Make FD.Init receive
and use useNetpoller argument, so FD.Init caller is explicit
about using runtime netpoller.
Fixes#21172
Change-Id: I60e2bfedf9dda9b341eb7a3e5221035db29f5739
Reviewed-on: https://go-review.googlesource.com/65810
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In general, there are no guarantee that `go` command exist on $PATH.
This CL tries to get `go` command from $GOROOT/bin instead.
There are three kinds of code we should handle:
For normal code, the CL implements goCmd() or goCmdName().
For unit tests, the CL uses testenv.GoTool() or testenv.GoToolPath().
For integration tests, the CL sets PATH=$GOROOT/bin:$PATH in cmd/dist.
Note that make.bash sets PATH=$GOROOT/bin:$PATH in the build process.
So this change is only useful when we use toolchain manually.
Updates #21875
Change-Id: I963b9f22ea732dd735363ececde4cf94a5db5ca2
Reviewed-on: https://go-review.googlesource.com/64650
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I16475e9bb055b934302870ccb5136174dc3bc817
Reviewed-on: https://go-review.googlesource.com/64670
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Move it from the runtime package, as we will soon add more packages and
functions for it to check.
The test used the testEnv func, which cleaned certain environment
variables from a command, so it was moved to internal/testenv under a
more descriptive (and less ambiguous) name. Add a simple godoc to it
too.
For #21851.
Change-Id: I6f39c1f23b45377718355fafe66ffd87047d8ab6
Reviewed-on: https://go-review.googlesource.com/63550
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
Also add HasCGO() to internal/testenv for tests.
Updates #21706
Change-Id: I938188047024052bdb42b3ac1a77708f3c2a6dbb
Reviewed-on: https://go-review.googlesource.com/62591
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Right now we only kind of sort of trace GC STW events. We emit events
around mark termination, but those start well after stopping the world
and end before starting it again, and we don't emit any events for
sweep termination.
Fix this by generalizing EvGCScanStart/EvGCScanDone. These were
already re-purposed to indicate mark termination (despite the names).
This commit renames them to EvGCSTWStart/EvGCSTWDone, adds an argument
to indicate the STW reason, and shuffles the runtime to generate them
right before stopping the world and right after starting the world,
respectively.
These events will make it possible to generate precise minimum mutator
utilization (MMU) graphs and could be useful in detecting
non-preemptible goroutines (e.g., #20792).
Change-Id: If95783f370781d8ef66addd94886028103a7c26f
Reviewed-on: https://go-review.googlesource.com/55411
Reviewed-by: Rick Hudson <rlh@golang.org>
This change replaces the current runtime capabilities check for ppc64x with the
new internal/cpu package. It also adds support for the new POWER9 ISA and
capabilities.
Updates #15403
Change-Id: I5b64a79e782f8da3603e5529600434f602986292
Reviewed-on: https://go-review.googlesource.com/53830
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
I also wanted to test net sockets, but I do not know how to
access their file handles. So I did not implement socket tests.
Updates #21172
Change-Id: I5062c0e65a817571d755397d60762c175f9791ce
Reviewed-on: https://go-review.googlesource.com/53530
Reviewed-by: Ian Lance Taylor <iant@golang.org>
According to http://infocenter.arm.com:
* ARM Cortex-A53 (Raspberry Pi 3, Pine A64)
* ARM Cortex-A57 (Opteron A1100, Tegra X1)
* ARM Cortex-A72
all have a cache line size of 64 bytes.
Change-Id: I4b333e930792fb1a221b3ca6f395bfa1b7762afa
Reviewed-on: https://go-review.googlesource.com/43250
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
If we get an EAGAIN error on an unpollable file, don't try to wait for
it to be ready; just return EAGAIN.
It's possible that we should instead ensure that when Stdin is a pipe
in non-blocking mode, we wait for data to appear. For now take the
conservative approach of doing what we did in previous releases.
Based on https://golang.org/cl/47555 by Totoro W.
Fixes#20915
Change-Id: Icc9e97a5a877b0a3583ec056c35412d1afab62d1
Reviewed-on: https://go-review.googlesource.com/48490
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If a retryable error such as EAGAIN/EINTR is encountered during a call
to sendfile(), we should not assume that a partial write occurred.
Instead, just like any other platform, we should always try again even
if 0 bytes were written.
Fixes#20857
Change-Id: I9aa48d193c27c6794c550320da4d3f7544041f37
Reviewed-on: https://go-review.googlesource.com/47351
Run-TryBot: Shawn Walker-Salas <shawn.walker@oracle.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change adds RawControl, RawRead and RawWrite methods to type FD
to make the runtime-integrated network poller work together with a
user-defined function. The methods are used via the net package from
external packages and type FD is considered as an implementation of
syscall.Conn and syscall.RawConn interfaces.
Updates #19435.
Change-Id: I4ad04b10ffddb2b54fa8d70587440960d73c0a2d
Reviewed-on: https://go-review.googlesource.com/37038
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Implements detection of x86 cpu features that
are used in the go standard library.
Changes all standard library packages to use the new cpu package
instead of using runtime internal variables to check x86 cpu features.
Updates: #15403
Change-Id: I2999a10cb4d9ec4863ffbed72f4e021a1dbc4bb9
Reviewed-on: https://go-review.googlesource.com/41476
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In the past we returned "use of closed network connection" when using
a closed network descriptor in some way. In CL 36799 that was changed
to return "use of closed file or network connection". Because programs
have no access to a value of this error type (see issue #4373) they
resort to doing direct string comparisons (see issue #19252). This CL
restores the old error string so that we don't break programs
unnecessarily with the 1.9 release.
This adds a test to the net package for the expected string.
For symmetry check that the os package returns the expected error,
which for os already exists as os.ErrClosed.
Updates #4373.
Fixed#19252.
Change-Id: I5b83fd12cfa03501a077cad9336499b819f4a38b
Reviewed-on: https://go-review.googlesource.com/39997
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Since Pread/Pwrite specify a file offset, using incref is sufficient.
This permits multiple Pread/Pwrite calls in parallel.
Since Pread/Pwrite specify a file offset, it doesn't seem to make
sense to use the poller for them, so don't.
Updates #19586
Change-Id: I676be16bf519b9a45f8e6b1d991c44f10848bc11
Reviewed-on: https://go-review.googlesource.com/41670
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This extends the GCSweepDone event with counts of swept and reclaimed
bytes. These are useful for understanding the duration and
effectiveness of sweep events.
Change-Id: I3c97a4f0f3aad3adbd188adb264859775f54e2df
Reviewed-on: https://go-review.googlesource.com/40811
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
The iOS test harness only includes the current test directory in its
app bundles, but the tests need access to all source code.
Change-Id: I8a902b183bc2745b4fbfffef867002d573abb1f5
Reviewed-on: https://go-review.googlesource.com/37961
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We don't need to start the goroutines if the program isn't going to do
any I/O.
Fixes#19390.
Change-Id: I47eef992d3ad05ed5f3150f4d6e5b3e0cb16a551
Reviewed-on: https://go-review.googlesource.com/37762
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Fixes#19114.
Change-Id: I352add53d6ee8bf78792564225099f8537ac6b46
Reviewed-on: https://go-review.googlesource.com/37106
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
This changes the os package to use the runtime poller for file I/O
where possible. When a system call blocks on a pollable descriptor,
the goroutine will be blocked on the poller but the thread will be
released to run other goroutines. When using a non-pollable
descriptor, the os package will continue to use thread-blocking system
calls as before.
For example, on GNU/Linux, the runtime poller uses epoll. epoll does
not support ordinary disk files, so they will continue to use blocking
I/O as before. The poller will be used for pipes.
Since this means that the poller is used for many more programs, this
modifies the runtime to only block waiting for the poller if there is
some goroutine that is waiting on the poller. Otherwise, there is no
point, as the poller will never make any goroutine ready. This
preserves the runtime's current simple deadlock detection.
This seems to crash FreeBSD systems, so it is disabled on FreeBSD.
This is issue 19093.
Using the poller on Windows requires opening the file with
FILE_FLAG_OVERLAPPED. We should only do that if we can remove that
flag if the program calls the Fd method. This is issue 19098.
Update #6817.
Update #7903.
Update #15021.
Update #18507.
Update #19093.
Update #19098.
Change-Id: Ia5197dcefa7c6fbcca97d19a6f8621b2abcbb1fe
Reviewed-on: https://go-review.googlesource.com/36800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
We did not create it. We should not delete it.
Change-Id: If98454ab233ce25367e11a7c68d31b49074537dd
Reviewed-on: https://go-review.googlesource.com/37030
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Alternatively the contents of str.go could be moved into fd_io_plan9.go
Change-Id: I9d7ec85bbb376f4244eeca732f25c0b77cadc6a6
Reviewed-on: https://go-review.googlesource.com/36971
Run-TryBot: Dave Cheney <dave@cheney.net>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This will make it possible to use the poller with the os package.
This is a lot of code movement but the behavior is intended to be
unchanged.
Update #6817.
Update #7903.
Update #15021.
Update #18507.
Change-Id: I1413685928017c32df5654ded73a2643820977ae
Reviewed-on: https://go-review.googlesource.com/36799
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
During the mark phase of garbage collection, goroutines that allocate
may be recruited to assist. This change creates trace events for mark
assists and displays them similarly to sweep assists in the trace
viewer.
Mark assists are different than sweeps in that they can be preempted, so
displaying them in the trace viewer is a little tricky -- we may need to
synthesize multiple slices for one mark assist. This could have been
done in the parser instead, but I thought it might be preferable to keep
the parser as true to the event stream as possible.
Change-Id: I381dcb1027a187a354b1858537851fa68a620ea7
Reviewed-on: https://go-review.googlesource.com/36015
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Skip lines if they are empty or starting with "#" which are valid
legacy pprof output format.
Fixes#18025
Change-Id: I7aee439171496932637b8ae3188700911f569b16
Reviewed-on: https://go-review.googlesource.com/33454
Reviewed-by: Peter Weinberger <pjw@google.com>
The pprof code discards all heap allocations made by runtime
routines. This caused it to discard heap allocations made by functions
called by reflect.Call, as the calls are made via the functions
`runtime.call32`, `runtime.call64`, etc. Fix the profiler to retain
these heap allocations.
Fixes#18077.
Change-Id: I8962d552f1d0b70fc7e6f7b2dbae8d5bdefb0735
Reviewed-on: https://go-review.googlesource.com/33635
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Windows has a limit of 260 characters on normal paths, but it's possible
to use longer paths by using "extended-length paths" that begin with
`\\?\`. This commit attempts to transparently convert an absolute path
to an extended-length path, following the subtly different rules those
paths require. It does not attempt to handle relative paths, which
continue to be passed to the operating system unmodified.
This adds a new test, TestLongPath, to the os package. This test makes
sure that it is possible to write a path at least 400 characters long
and runs on every platform. It also tests symlinks and hardlinks, though
symlinks are not testable with our builder configuration.
HasLink is moved to internal/testenv so it can be used by multiple tests.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
has Microsoft's documentation on extended-length paths.
Fixes#3358.
Fixes#10577.
Fixes#17500.
Change-Id: I4ff6bb2ef9c9a4468d383d98379f65cf9c448218
Reviewed-on: https://go-review.googlesource.com/32451
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This allows both the runtime and the cmd/pprof code to use the package,
just like we do for internal/trace.
Change-Id: I7606977284e1def36c9647354c58e7c1e93dba6b
Reviewed-on: https://go-review.googlesource.com/32452
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds support to the runtime/trace test for saving traces
collected by its tests to disk and a script in internal/trace that
uses this to collect canned traces for the trace test suite. This can
be used to add to the test suite when we introduce a new trace format
version.
Change-Id: Id9ac1ff312235bf02f982fdfff8a827f54035758
Reviewed-on: https://go-review.googlesource.com/32290
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently when a goroutine blocks on a GC assist, it emits a generic
EvGoBlock event. Since assist blocking events and, in particular, the
length of the blocked assist queue, are important for diagnosing GC
behavior, this commit adds a new EvGoBlockGC event for blocking on a
GC assist. The trace viewer uses this event to report a "waiting on
GC" count in the "Goroutines" row. This makes sense because, unlike
other blocked goroutines, these goroutines do have work to do, so
being blocked on a GC assist is quite similar to being in the
"runnable" state, which we also report in the trace viewer.
Change-Id: Ic21a326992606b121ea3d3d00110d8d1fdc7a5ef
Reviewed-on: https://go-review.googlesource.com/30704
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently mark workers are shown in the trace as regular goroutines
labeled "runtime.gcBgMarkWorker". That's somewhat unhelpful to an end
user because of the opaque label and particularly unhelpful to runtime
developers because it doesn't distinguish the different types of mark
workers.
Fix this by introducing a variant of the GoStart event called
GoStartLabel that lets the runtime indicate a label for a goroutine
execution span and using this to label mark worker executions as "GC
(<mode>)" in the trace viewer.
Since this bumps the trace version to 1.8, we also add test data for
1.7 traces.
Change-Id: Id7b9c0536508430c661ffb9e40e436f3901ca121
Reviewed-on: https://go-review.googlesource.com/30702
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
So errnoErr can be used in other packages.
This is something I missed when I sent CL 28990.
Fixes#17539
Change-Id: I8ee3b79c4d70ca1e5b29e5b40024f7ae9a86061e
Reviewed-on: https://go-review.googlesource.com/29690
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Current implementation of syscall.Readlink mistakenly calculates
the end offset of the PrintName field.
Also, there are some cases that the PrintName field is empty.
Instead, the CL uses SubstituteName with correct calculation.
Fixes#15978Fixes#16145
Change-Id: If3257137141129ac1c552d003726d5b9c08bb754
Reviewed-on: https://go-review.googlesource.com/31118
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL introduces first test for readConsole. And new test
discovered couple of problems with readConsole.
Console characters consist of multiple bytes each, but byte blocks
returned by syscall.ReadFile have no character boundaries. Some
multi-byte characters might start at the end of one block, and end
at the start of next block. readConsole feeds these blocks to
syscall.MultiByteToWideChar to convert them into utf16, but if some
multi-byte characters have no ending or starting bytes, the
syscall.MultiByteToWideChar might get confused. Current version of
syscall.MultiByteToWideChar call will make
syscall.MultiByteToWideChar ignore all these not complete
multi-byte characters.
The CL solves this issue by changing processing from "randomly
sized block of bytes at a time" to "one multi-byte character at a
time". New readConsole code calls syscall.ReadFile to get 1 byte
first. Then it feeds this byte to syscall.MultiByteToWideChar.
The new syscall.MultiByteToWideChar call uses MB_ERR_INVALID_CHARS
flag to make syscall.MultiByteToWideChar return error if input is
not complete character. If syscall.MultiByteToWideChar returns
correspondent error, we read another byte and pass 2 byte buffer
into syscall.MultiByteToWideChar, and so on until success.
Old readConsole code would also sometimes return no data if user
buffer was smaller then uint16 size, which would confuse callers
that supply 1 byte buffer. This CL fixes that problem too.
Fixes#17097
Change-Id: I88136cdf6a7bf3aed5fbb9ad2c759b6c0304ce30
Reviewed-on: https://go-review.googlesource.com/29493
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
When starting tracing, EvGoCreate events are added for existing
goroutines that may have been blocking in syscall. EvGoCreate
increments the runnable goroutine count. This change makes the
following EvGoInSyscall event decrement the runnable goroutine count
because we now know that goroutine is in syscall, and not runnable.
Made generateTrace return an error, at any given time, the number
of runnable/running/insyscall goroutines becomes non-negative.
Added a basic test that checks the number of runnable/running
goroutines don't include the goroutines in syscall - the test failed
before this change.
Change-Id: Ib732c382e7bd17158a437576f9d589ab89097ce6
Reviewed-on: https://go-review.googlesource.com/25552
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently these are labeled "MARK", which was accurate in the STW
collector, but these really indicate mark termination now, since
marking happens for the full duration of the concurrent GC. Re-label
them as "MARK TERMINATION" to clarify this.
Change-Id: Ie98bd961195acde49598b4fa3f9e7d90d757c0a6
Reviewed-on: https://go-review.googlesource.com/30018
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently, the process-wide GC state is attributed to the P that
happened to perform the allocation that exceeded the GC trigger. This
is pretty arbitrary and makes it hard to see when GC is running since
the GC spans are intermingled with a lot of other trace noise.
The current display is particularly confusing because it creates three
sub-rows in the P row that can overlap each other. Usually a P has
just two sub-rows: one showing the current G and another showing that
G's activity. However, because GC is attributed to a proc, it winds up
as a third row that neither subsumes nor is subsumed by any other row.
This in turn screws up the trace's layout and results in overlapping
events.
Fix these problems by creating a new dedicated row like the existing
"Network" and "Timer" rows and displaying process-wide GC events in
this row. Mark termination and sweep events still appear in their
respective P rows because these are meaningfully attributed.
Change-Id: Ie1a1c6cf8c446e4b043f10f3968f91ff1b546d15
Reviewed-on: https://go-review.googlesource.com/30017
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
It is possible (and common) for Windows systems to use a different codepage
for console applications from that used on normal windowed application
(called ANSI codepage); for instance, most of the western Europe uses
CP850 for console (for backward compatibility with MS-DOS), while
windowed applications use a different codepage depending on the country
(eg: CP1252 aka Latin-1). The usage being changed with this commit is
specifically related to decoding input coming from the console, so the
previous usage of the ANSI codepage was wrong.
Also fixes an issue that previous did convert bytes as NFD. Go is
designed to handle single Unicode code point. This fix change behaivor
to NFC.
Fixes#16857.
Change-Id: I4f41ae83ece47321b6e9a79a2087ecbb8ac066dd
Reviewed-on: https://go-review.googlesource.com/27575
Reviewed-by: Hiroshi Ioka <hirochachacha@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
GoToolPath requires a *testing.T to handle errors.
GoTool provides a variant that returns errors
for clients without a *testing.T,
such as that found in CL 27811.
Change-Id: I7ac8b7ec9d472894c37223c5f7b121ec823e7f61
Reviewed-on: https://go-review.googlesource.com/28787
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
os package and path/filepath package have duplicated code for
checking symlink supports in test code.
This CL tries to simplify such test code.
Change-Id: I0371488337f5e951eca699852daab9ccb16ddd62
Reviewed-on: https://go-review.googlesource.com/27331
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When tracing is started in the middle of program execution,
we already have a number of runnable goroutines and a number
of blocked/in syscall goroutines. In order to reflect these
goroutines in the trace, we emit EvGoCreate for all existing
goroutines. Then for blocked/in syscall goroutines we additionally
emit EvGoWaiting/EvGoInSyscall events. These events don't reset g.ev
during trace analysis. So next EvGoStart finds g.ev set to the
previous EvGoCreate. As the result time between EvGoCreate and
EvGoStart is accounted as scheduler latency. While in reality
it is blocking/syscall time.
Properly reset g.ev for EvGoWaiting/EvGoInSyscall events.
Change-Id: I0615ba31ed7567600a0667ebb27458481da73adb
Reviewed-on: https://go-review.googlesource.com/25572
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
The httptrace.ConnectStart and ConnectDone hooks are just about the
post-DNS connection to the host. We were accidentally also firing on
the UDP dials to DNS. Exclude those for now. We can add them back
later as separate hooks if desired. (but they'd only work for pure Go
DNS)
This wasn't noticed earlier because I was developing on a Mac at the
time, which always uses cgo for DNS. When running other tests on
Linux, I started seeing UDP dials.
Updates #12580
Change-Id: I2b2403f2483e227308fe008019f1100f6300250b
Reviewed-on: https://go-review.googlesource.com/23069
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We run the external network tests on builders, but some of our
builders have less-than-ideal DNS connectivity. This change continues
to run the tests on all builders, but marks certain builders as flaky
(network-wise), and only validates their DNS results if they got DNS
results.
Change-Id: I826dc2a6f6da55add89ae9c6db892b3b2f7b526b
Reviewed-on: https://go-review.googlesource.com/22852
Reviewed-by: Ian Lance Taylor <iant@golang.org>
On some processors cputicks (used to generate trace timestamps)
produce non-monotonic timestamps. It is important that the parser
distinguishes logically inconsistent traces (e.g. missing, excessive
or misordered events) from broken timestamps. The former is a bug
in tracer, the latter is a machine issue.
Test that (1) parser does not return a logical error in case of
broken timestamps and (2) broken timestamps are eventually detected
and reported.
Change-Id: Ib4b1eb43ce128b268e754400ed8b5e8def04bd78
Reviewed-on: https://go-review.googlesource.com/21608
Reviewed-by: Austin Clements <austin@google.com>
Martin Möhrmann
erifan01
Hana Kim
Andrei Tudor Călin
Aman Gupta
Tobias Klauser
Felix Kollmann
Alex Brainman
Nick Patavalis
fanzha02
Meng Zhuo
Lubomir I. Ivanov (VMware)
Ilya Tocar
Daniel Martí
Cherry Zhang
Ian Lance Taylor
Keith Randall
Shamil Garatuev
Keith Randall
Mansour Rahimi
Mikio Hara
Wèi Cōngruì
Russ Cox
Fangming.Fang
Brad Fitzpatrick
Paul Querna
Marvin Stenger
Hiroshi Ioka
Aliaksandr Valialkin
Austin Clements
Agniva De Sarker
Carlos Eduardo Seo
Shawn Walker-Salas
Josh Bleecher Snyder
Elias Naur
Dave Cheney
Heschi Kreinick
Hana Kim
Shenghou Ma
Quentin Smith
Michael Munday
Vladimir Stefanovic
Yasuhiro Matsumoto
Dmitry Vyukov
Tom Bergan