I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)
Updates #27864
Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
For sweep events, we used to modify the ViewerEvent returned from
ctx.emitSlice later in order to embed more details about the sweep
operation. The trick no longer works after the change
https://golang.org/cl/92375 and caused a regression.
ctx.emit method encodes the ViewerEvent, so any modification to the
ViewerEvent object after ctx.emit returns will not be reflected.
Refactor ctx.emitSlice, so ctx.makeSlice can be used when producing
slices for SWEEP. ctx.emit* methods are meant to truely emit
ViewerEvents.
Fixes#27711
Change-Id: I0b733ebbbfd4facd8714db0535809ec3cab0833d
Reviewed-on: https://go-review.googlesource.com/135775
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This commit adds support for JavaScript callbacks back into
WebAssembly. This is experimental API, just like the rest of the
syscall/js package. The time package now also uses this mechanism
to properly support timers without resorting to a busy loop.
JavaScript code can call into the same entry point multiple times.
The new RUN register is used to keep track of the program's
run state. Possible values are: starting, running, paused and exited.
If no goroutine is ready any more, the scheduler can put the
program into the "paused" state and the WebAssembly code will
stop running. When a callback occurs, the JavaScript code puts
the callback data into a queue and then calls into WebAssembly
to allow the Go code to continue running.
Updates #18892
Updates #25506
Change-Id: Ib8701cfa0536d10d69bd541c85b0e2a754eb54fb
Reviewed-on: https://go-review.googlesource.com/114197
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
First, the regions sort was buggy, as its last comparison was
ineffective.
Second, the insyscall and insyscallRuntime fields were unsigned, so the
check for them being negative was pointless. Make them signed instead,
to also prevent the possibility of underflows when decreasing numbers
that might realistically be 0.
Third, the color constants were all untyped strings except the first
one. Be consistent with their typing.
Change-Id: I4eb8d08028ed92589493c2a4b9cc5a88d83f769b
Reviewed-on: https://go-review.googlesource.com/113895
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I1cb7c8b70a5ae16386f6abb577c23d821f7ff7f0
Reviewed-on: https://go-review.googlesource.com/112197
Reviewed-by: Peter Weinberger <pjw@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Currently isSystemGoroutine has a hard-coded list of known entry
points into system goroutines. This list is annoying to maintain. For
example, it's missing the ensureSigM goroutine.
Replace it with a check that simply looks for any goroutine with
runtime function as its entry point, with a few exceptions. This also
matches the definition recently added to the trace viewer (CL 81315).
Change-Id: Iaed723d4a6e8c2ffb7c0c48fbac1688b00b30f01
Reviewed-on: https://go-review.googlesource.com/81655
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
and assign the same colors for spans belong to the tasks
(sadly, the trace viewer will change the saturation/ligthness
for asynchronous slices so exact color mapping is impossible.
But I hope they are not too far from each other)
Change-Id: Idaaf0828a1e0dac8012d336dcefa1c6572ddca2e
Reviewed-on: https://go-review.googlesource.com/109338
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Also change tasks to be represented as "slices" instead of
asynchronous events which are more efficiently represented in trace
viewer data model. This change allows to utilize the flow events
(arrows) to represent task hierarchies.
Introduced RegionArgs and TaskArgs where the task id infomation and
goroutine id informations are stored for information-purpose.
Change-Id: I11bec7dd716fdfc5f94ea39661b2e51344367a6f
Reviewed-on: https://go-review.googlesource.com/109337
Reviewed-by: Heschi Kreinick <heschi@google.com>
A task may have other user annotation events after the task ends.
So far, task.lastTimestamp returned the task end event if the
event available. This change introduces task.endTimestamp for that
and makes task.lastTimestamp returns the "last" seen event's timestamp
if the task is ended.
If the task is not ended, both returns the last timestamp of the entire
trace assuming the task is still active.
This fixes the task-oriented trace view mode not to drop user
annotation instances when they appear outside a task's lifespan.
Adds a test.
Change-Id: Iba1062914f224edd521b9ee55c6cd5e180e55359
Reviewed-on: https://go-review.googlesource.com/109175
Reviewed-by: Heschi Kreinick <heschi@google.com>
"Span" is a commonly used term in many distributed tracing systems
(Dapper, OpenCensus, OpenTracing, ...). They use it to refer to a
period of time, not necessarily tied into execution of underlying
processor, thread, or goroutine, unlike the "Span" of runtime/trace
package.
Since distributed tracing and go runtime execution tracing are
already similar enough to cause confusion, this CL attempts to avoid
using the same word if possible.
"Region" is being used in a certain tracing system to refer to a code
region which is pretty close to what runtime/trace.Span currently
refers to. So, replace that.
https://software.intel.com/en-us/itc-user-and-reference-guide-defining-and-recording-functions-or-regions
This CL also tweaks APIs a bit based on jbd and heschi's comments:
NewContext -> NewTask
and it now returns a Task object that exports End method.
StartSpan -> StartRegion
and it now returns a Region object that exports End method.
Also, changed WithSpan to WithRegion and it now takes func() with no
context. Another thought is to get rid of WithRegion. It is a nice
concept but in practice, it seems problematic (a lot of code churn,
and polluting stack trace). Already, the tracing concept is very low
level, and we hope this API to be used with great care.
Recommended usage will be
defer trace.StartRegion(ctx, "someRegion").End()
Left old APIs untouched in this CL. Once the usage of them are cleaned
up, they will be removed in a separate CL.
Change-Id: I73880635e437f3aad51314331a035dd1459b9f3a
Reviewed-on: https://go-review.googlesource.com/108296
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JBD <jbd@google.com>
golang.org/cl/102697 attempted to fix the span presentation by utilizing
the position of the span in the span slices of a task. But it is
not complete either.
First, id=0 is omitted in json encoding and the trace viewer silently
drops entries with the missing id field, so we must avoid zero-value id.
Second, it is possible that a goroutine handles multiple tasks. Then,
id collisions will happen.
This takes a simpler approach - have a counter that increments for every
emitSpan call, and use the value as the id value.
Change-Id: Idaa9505634acf6d327c6f00af32d8260955b85e1
Reviewed-on: https://go-review.googlesource.com/106755
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
/spanio, /spanblock, /spansched, /spansyscall provide
the pprof-style summary of span execution's
io, block, scheduling, syscall latency distributions
respectively.
The computation logic for /io, /block, /sched, /syscall
analysis was refactored and extended for reuse in these
new types of analysis. Upon the analysis query, we create
a map of goroutine id to time intervals based on the query
parameter, that represents the interesting time intervals
of matching goroutines. Only the events from the matching
goroutines that fall into the intervals are considered
in the pprof computation.
The new endpoints are not yet hooked into other span
analysis page (e.g. /userspan) yet.
Change-Id: I80c3396e45a2d6631758710de67d132e5832c7ce
Reviewed-on: https://go-review.googlesource.com/105822
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
All tests involving trace collection and parsing
still need handling of failures caused by #16755
(Timestamp issue)
Fixes#24738
Change-Id: I6cd0f9c6f49854a22fad6fce1a00964c168aa614
Reviewed-on: https://go-review.googlesource.com/105821
Reviewed-by: Peter Weinberger <pjw@google.com>
The trace viewer interprets the slice as a non-terminating
time interval which is quite opposit to what trace records indicate
(i.e., almostly immediately terminating time interval).
As observed in the issue #24663 this can result in quite misleading
visualization of the trace.
Work around the trace viewer's issue by setting a small value
(0.0001usec) as the duration if the time interval is not positive.
Change-Id: I1c2aac135c194d0717f5c01a98ca60ffb14ef45c
Reviewed-on: https://go-review.googlesource.com/104716
Reviewed-by: Heschi Kreinick <heschi@google.com>
This mode is similar to the default traceview mode where the execution
trace is presented in P-oriented way. Each row represents a P, and each
slice represents the time interval of a goroutine's execution on the P.
The difference is that, in this mode, only the execution of goroutines
involved in the specified task is highlighted, and other goroutine
execution or events are greyed out. So, users can focus on how a task is
executed while considering other affecting conditions such as other
goroutines, network events, or process scheduling.
Example: https://user-images.githubusercontent.com/4999471/38116793-a6f995f0-337f-11e8-8de9-88eec2f2c497.png
Here, for a while the program remained idle after the first burst of
activity related to the task because all other goroutines were also
being blocked or waiting for events, or no incoming network traffic
(indicated by the lack of any network activity). This is a bit hard to
discover when the usual task-oriented view (/trace?taskid=<taskid>)
mode.
Also, it simplifies the traceview generation mode logic.
/trace ---> 0
/trace?goid ---> modeGoroutineOriented
/trace?taskid ---> modeGoroutineOriented|modeTaskOriented
/trace?focustask ---> modeTaskOriented
Change-Id: Idcc0ae31b708ddfd19766f4e26ee7efdafecd3a5
Reviewed-on: https://go-review.googlesource.com/103555
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Mostly same as golang.org/cl/102156, except the parts that
deal with different data types.
Change-Id: I061b858b73898725e3bf175ed022c2e3e55fc485
Reviewed-on: https://go-review.googlesource.com/103158
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Spans are represented using Async Event types of chrome trace viewer.
According to the doc, the 'id' should be unique within category, scope.
https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview#heading=h.jh64i9l3vwa1
Use the index in the task's span slice as the slice id, so it
can be unique within the task. The scope is the task id which
is unique.
This fixes a visualization bug that caused incorrect or missing
presentation of nested spans.
Change-Id: If1537ee00247f71fa967abfe45569a9e7dbcdce7
Reviewed-on: https://go-review.googlesource.com/102697
Reviewed-by: Heschi Kreinick <heschi@google.com>
Move part of UserSpan event processing from cmd/trace.analyzeAnnotations
to internal/trace.GoroutineStats that returns analyzed per-goroutine
execution information. Now the execution information includes list of
spans and their execution information.
cmd/trace.analyzeAnnotations utilizes the span execution information
from internal/trace.GoroutineStats and connects them with task
information.
Change-Id: Ib7f79a3ba652a4ae55cd81ea17565bcc7e241c5c
Reviewed-on: https://go-review.googlesource.com/101917
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
- Summary: also includes links to pprof data.
- Sortable table: sorting is done on server-side. The intention is
that later, I want to add pagination feature and limit the page
size the browser has to handle.
- Stacked horizontal bar graph to present total time breakdown.
- Human-friendly time representation.
- No dependency on external fancy javascript libraries to allow
it to function without an internet connection.
Change-Id: I91e5c26746e59ad0329dfb61e096e11f768c7b73
Reviewed-on: https://go-review.googlesource.com/102156
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Unused variables in closures are currently not diagnosed by the
compiler (this is Issue #3059), while go/types catches them.
One unused variable in the cmd/trace tests is causing the go/types
test that typechecks the whole standard library to fail:
FAIL: TestStdlib (8.05s)
stdlib_test.go:223: cmd/trace/annotations_test.go:241:6: gcTime
declared but not used
FAIL
Remove it.
Updates #24464
Change-Id: I0f1b9db6ae1f0130616ee649bdbfdc91e38d2184
Reviewed-on: https://go-review.googlesource.com/101815
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Add a search box to the top of the user task views that only displays
tasks containing a particular log message.
Change-Id: I92f4aa113f930954e8811416901e37824f0eb884
Reviewed-on: https://go-review.googlesource.com/100843
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Also grey out instants that represent events occurred outside the
task's span. Furthermore, if the unrelated instants represent user
annotation events but not for the task of the interest, skip rendering
completely.
This helps users to focus on the task-related events better.
UI screen shot:
https://gist.github.com/hyangah/1df5d2c8f429fd933c481e9636b89b55#file-golang-org_cl_99035
Change-Id: I2b5aef41584c827f8c1e915d0d8e5c95fe2b4b65
Reviewed-on: https://go-review.googlesource.com/99035
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
This is an updated version of golang.org/cl/96395, with the fix to
TestUserSpan.
This reverts commit 7b6f6267e90a8e4eab37a3f2164ba882e6222adb.
Change-Id: I31eec8ba0997f9178dffef8dac608e731ab70872
Reviewed-on: https://go-review.googlesource.com/98236
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Even though undocumented, the assumption is the Event's link field
points to the following event in the future. The new span/task event
processing breaks the assumption.
Change-Id: I4ce2f30c67c4f525ec0a121a7e43d8bdd2ec3f77
Reviewed-on: https://go-review.googlesource.com/96395
Reviewed-by: Heschi Kreinick <heschi@google.com>
runtime/trace test already skips tests in case of the timestamp
error.
Moreover, relax TestAnalyzeAnnotationGC test condition to
deal with the inaccuracy caused from use of cputicks in tracing.
Fixes#24081
Updates #16755
Change-Id: I708ecc6da202eaec07e431085a75d3dbfbf4cc06
Reviewed-on: https://go-review.googlesource.com/97757
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is for debugging the reported flaky tests.
Update #24081
Change-Id: Ica046928f675d69e38251a47a6f225efedce920c
Reviewed-on: https://go-review.googlesource.com/96855
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Enabled when the tool runs with DEBUG_MEMORY_USAGE=1 env var.
After reporting the usage, it waits until user enters input
(helpful when checking top or other memory monitor)
Also adds net/http/pprof to export debug endpoints.
From the trace included in #21870
$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out
2018/02/21 16:04:49 Parsing trace...
after parsing trace
Alloc: 3385747848 Bytes
Sys: 3661654648 Bytes
HeapReleased: 0 Bytes
HeapSys: 3488907264 Bytes
HeapInUse: 3426377728 Bytes
HeapAlloc: 3385747848 Bytes
Enter to continue...
2018/02/21 16:05:09 Serializing trace...
after generating trace
Alloc: 4908929616 Bytes
Sys: 5319063640 Bytes
HeapReleased: 0 Bytes
HeapSys: 5032411136 Bytes
HeapInUse: 4982865920 Bytes
HeapAlloc: 4908929616 Bytes
Enter to continue...
2018/02/21 16:05:18 Splitting trace...
after spliting trace
Alloc: 4909026200 Bytes
Sys: 5319063640 Bytes
HeapReleased: 0 Bytes
HeapSys: 5032411136 Bytes
HeapInUse: 4983046144 Bytes
HeapAlloc: 4909026200 Bytes
Enter to continue...
2018/02/21 16:05:39 Opening browser. Trace viewer is listening on http://127.0.0.1:33661
after httpJsonTrace
Alloc: 5288336048 Bytes
Sys: 7790245896 Bytes
HeapReleased: 0 Bytes
HeapSys: 7381123072 Bytes
HeapInUse: 5324120064 Bytes
HeapAlloc: 5288336048 Bytes
Enter to continue...
Change-Id: I88bb3cb1af3cb62e4643a8cbafd5823672b2e464
Reviewed-on: https://go-review.googlesource.com/92355
Reviewed-by: Peter Weinberger <pjw@google.com>
The task-oriented trace view presents the execution trace organized
based on goroutines. Often, which P a goroutine was running on is
useful, so this CL includes the P ids in the goroutine execution slices.
R=go1.11
Change-Id: I96539bf8215e5c1cd8cc997a90204f57347c48c8
Reviewed-on: https://go-review.googlesource.com/90221
Reviewed-by: Heschi Kreinick <heschi@google.com>
The GC time for a task is defined by the sum of GC duration
overlapping with the task's duration.
Also, grey out non-overlapping slices in the task-oriented
trace view.
R=go1.11
Change-Id: I42def0eb520f5d9bd07edd265e558706f6fab552
Reviewed-on: https://go-review.googlesource.com/90219
Reviewed-by: Heschi Kreinick <heschi@google.com>
This adds the ability to add a #x:y anchor to the trace view URL that
causes the viewer to initially select from x ms to y ms.
Change-Id: I4a980d8128ecc85dbe41f224e8ae336707a4eaab
Reviewed-on: https://go-review.googlesource.com/60794
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
golang.org/cl/81315 attempted to distinguish system goroutines
by examining the function name in the goroutine stack. It assumes that
the information would be available when GoSysBlock or GoInSyscall
events are processed, but it turned out the stack information is
set too late (when the goroutine gets a chance to run).
This change initializes the goroutine information entry when
processing GoCreate event which should be one of the very first
events for the every goroutine in trace.
Fixes#22574
Change-Id: I1ed37087ce2e78ed27c9b419b7d942eb4140cc69
Reviewed-on: https://go-review.googlesource.com/83595
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The number of threads in syscall presented by execution tracer's
trace view includes not only the threads calling system calls on behalf
of user created goroutines, but also those running on behalf of system
goroutines.
When the number of such system goroutines was small, the graph was
useful when examining where a program was saturating the CPU.
But as more and more system goroutines are invloved the graph became
less useful for the purpose - for example, after golang.org/cl/34784,
the timer goroutines dominate in the graph with large P
because the runtime creates per-P timer goroutines.
This change excludes the threads in syscall on behalf of runtime (system
goroutines) from the visualization. Alternatively, I could visualize the
count of such threads in a separate counter but in the same graph.
Given that many other debug endpoints (e.g. /debug/pprof/goroutine) hide
the system goroutines, including them in the same graph can confuse users.
Update #22574
Change-Id: If758cd6b9ed0596fde9a471e846b93246580b9d5
Reviewed-on: https://go-review.googlesource.com/81315
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The trace command computes IO, Schedule, Block, and Syscall profiles
by following the unblocking links in the execution trace and summing
up the duration. This change offers variations of those profiles
that include only selected goroutine types. The id parameter takes the
goroutine type - i.e. pc of the goroutine.
The output is available from the /goroutine view. So, users can see
where the goroutines of interest typically block.
Also, these profiles are available for download so users can use
pprof or other tools to interpret the output. This change adds links
for download of global profile in the main page.
Change-Id: I35699252056d164e60de282b0406caf96d629c85
Reviewed-on: https://go-review.googlesource.com/75710
Reviewed-by: Sameer Ajmani <sameer@golang.org>
CL 56950 correctly identified code with checks that were impossible.
But instead of correcting the checks it deleted them.
This CL corrects the code to check what was meant.
Change-Id: Ic89222184ee4fa5cacccae12d750601a9438ac8d
Reviewed-on: https://go-review.googlesource.com/78113
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When traceviewer encounters a failure of json trace import
due to data error, onImportFail tried to access an error variable
which was not yet defined.
Change-Id: I431be03f179aafacaf1fd3c62a6337e8b5bd18fb
Reviewed-on: https://go-review.googlesource.com/71970
Reviewed-by: Austin Clements <austin@google.com>
In general, there are no guarantee that `go` command exist on $PATH.
This CL tries to get `go` command from $GOROOT/bin instead.
There are three kinds of code we should handle:
For normal code, the CL implements goCmd() or goCmdName().
For unit tests, the CL uses testenv.GoTool() or testenv.GoToolPath().
For integration tests, the CL sets PATH=$GOROOT/bin:$PATH in cmd/dist.
Note that make.bash sets PATH=$GOROOT/bin:$PATH in the build process.
So this change is only useful when we use toolchain manually.
Updates #21875
Change-Id: I963b9f22ea732dd735363ececde4cf94a5db5ca2
Reviewed-on: https://go-review.googlesource.com/64650
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Brad Fitzpatrick
Hana Kim
Alex Kohler
Kazuhiro Sera
Richard Musiol
Daniel Martí
Austin Clements
Hana Kim
Alberto Donizetti
Heschi Kreinick
Russ Cox
Hiroshi Ioka
