Commit Graph

67 Commits

Author SHA1 Message Date
Michael Anthony Knyszek 3eaf75c13a runtime: move next_gc and last_next_gc into gcControllerState
This change moves next_gc and last_next_gc into gcControllerState under
the names heapGoal and lastHeapGoal respectively. These are
fundamentally GC pacer related values, and so it makes sense for them to
live here.

Partially generated by

rf '
    ex . {
	memstats.next_gc -> gcController.heapGoal
	memstats.last_next_gc -> gcController.lastHeapGoal
    }
'

except for updates to comments and gcControllerState methods, where
they're accessed through the receiver, and trace-related renames of
NextGC -> HeapGoal, while we're here.

For #44167.

Change-Id: I1e871ad78a57b01be8d9f71bd662530c84853bed
Reviewed-on: https://go-review.googlesource.com/c/go/+/306603
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-04-14 14:03:30 +00:00
Michael Matloob cc700bdc26 cmd/trace: move viewer data structs into cmd/internal/traceviewer
The ViewerEvent, ViewerData and ViewerFrame structs are moved into
cmd/internal/traceviewer, and renamed Event, Data, and Frame.

The structs are the same, except for the following: A definition
for the JSON "bp" field that's defined in the trace format, but
missing in the structs has been added. Also, the Tid and Pid fields
on Event have been renamed TID and PID to better match Go style.
Finally, the footer field on ViewerData, which hasn't been used
for a while, has been removed.

This CL is in preparation for the usage of these structs by cmd/go's
tracing functionality.

Updates #38714

Change-Id: I345f23617b96d4629b876ae717f89d56a67e05a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/239098
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-08-12 18:34:48 +00:00
Hana (Hyang-Ah) Kim 75ea964b3f cmd/trace: update to use WebComponents V0 polyfill
Old trace viewer stopped working with Chrome M80+ because the
old trace viewer heavily depended on WebComponents V0 which are deprecated.
Trace viewer recently migrated to use WebComponents V0 polyfill
(crbug.com/1036492). This CL brings in the newly updated trace_viewer_full.html
(sync'd @ 9508452e)
and updates the javascript snippet included in the /trace endpoint
to use the polyfill.

This brings in webcomponents.min.js copied from
https://chromium.googlesource.com/catapult/+/9508452e18f130c98499cb4c4f1e1efaedee8962/third_party/polymer/components/webcomponentsjs/webcomponents.min.js

That is necessary because the /trace endpoint needs to import
the vulcanized trace_viewer_full.html.

It's possible that some features are not working correctly with
this polyfill. In that case, report the issue to crbug.com/1036492.
There will be a warning message in the UI (yellow banner above the timeline)
which can be hidden by clicking the 'hide' button.

This allows to render the trace in browsers other than chrome in theory,
but I observed some buttons and functions still don't work outside
chrome.

Fixes #34374.

Change-Id: Ib575f756f5e6b22ad904ede6e4d224a995ebe259
Reviewed-on: https://go-review.googlesource.com/c/go/+/219997
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-02-20 19:12:11 +00:00
Hana Kim 6d5caf38e3 cmd/trace: revert internal/traceparser
The performance improvement is not as big as we hoped.
Until the API is feature complete, we postpone the release
and avoid added complexity.

This change was prepared by reverting all the changes affected
src/cmd/trace and src/internal/traceparser packages after
golang.org/cl/137635, and then bringing back MMU computation
APIs (originally in src/internal/traceparser) to the
src/internal/trace package.

Revert "cmd/trace: use new traceparser to parse the raw trace files"
This reverts https://golang.org/cl/145457
  (commit 08816cb8d7).

Revert "internal/traceparser: provide parser that uses less space and parses segments of runtime trace files"
This reverts https://golang.org/cl/137635
  (commit daaf361f74).

Change-Id: Ic2a068a7dbaf4053cd9674ca7bde9c58e74385b4
Reviewed-on: https://go-review.googlesource.com/c/150517
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2018-11-22 02:59:55 +00:00
Austin Clements 603af813d6 cmd/trace: list and link to worst mutator utilization windows
This adds the ability to click a point on the MMU graph to show a list
of the worst 10 mutator utilization windows of the selected size. This
list in turn links to the trace viewer to drill down on specifically
what happened in each specific window.

Change-Id: Ic1b72d8b37fbf2212211c513cf36b34788b30133
Reviewed-on: https://go-review.googlesource.com/c/60795
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-05 19:10:18 +00:00
Peter Weinberger 08816cb8d7 cmd/trace: use new traceparser to parse the raw trace files
Change-Id: I8b224ae48a2f8acd5a64c9ff283e97821479a9a8
Reviewed-on: https://go-review.googlesource.com/c/145457
Run-TryBot: Peter Weinberger <pjw@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-10-30 16:43:38 +00:00
Brad Fitzpatrick da0d1a44ba all: use strings.ReplaceAll and bytes.ReplaceAll where applicable
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)

Updates #27864

Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-09-26 22:14:25 +00:00
Hana Kim e57f24ab39 cmd/trace: don't drop sweep slice details
For sweep events, we used to modify the ViewerEvent returned from
ctx.emitSlice later in order to embed more details about the sweep
operation. The trick no longer works after the change
https://golang.org/cl/92375 and caused a regression.

ctx.emit method encodes the ViewerEvent, so any modification to the
ViewerEvent object after ctx.emit returns will not be reflected.

Refactor ctx.emitSlice, so ctx.makeSlice can be used when producing
slices for SWEEP. ctx.emit* methods are meant to truely emit
ViewerEvents.

Fixes #27711

Change-Id: I0b733ebbbfd4facd8714db0535809ec3cab0833d
Reviewed-on: https://go-review.googlesource.com/135775
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-09-17 19:38:10 +00:00
Alex Kohler 31389254de all: fix typos
Change-Id: Icded6c786b7b185d5aff055f34e0cfe9e521826a
Reviewed-on: https://go-review.googlesource.com/132176
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-30 15:41:29 +00:00
Daniel Martí 64a832debe cmd/trace: fix a few bugs found by staticcheck
First, the regions sort was buggy, as its last comparison was
ineffective.

Second, the insyscall and insyscallRuntime fields were unsigned, so the
check for them being negative was pointless. Make them signed instead,
to also prevent the possibility of underflows when decreasing numbers
that might realistically be 0.

Third, the color constants were all untyped strings except the first
one. Be consistent with their typing.

Change-Id: I4eb8d08028ed92589493c2a4b9cc5a88d83f769b
Reviewed-on: https://go-review.googlesource.com/113895
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-05-21 16:18:29 +00:00
Hana (Hyang-Ah) Kim 43b18f4241 cmd/trace: handle invalid goid para in /trace
Change-Id: I1cb7c8b70a5ae16386f6abb577c23d821f7ff7f0
Reviewed-on: https://go-review.googlesource.com/112197
Reviewed-by: Peter Weinberger <pjw@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-05-08 19:25:01 +00:00
Austin Clements 44286b17c5 runtime: replace system goroutine whitelist with symbol test
Currently isSystemGoroutine has a hard-coded list of known entry
points into system goroutines. This list is annoying to maintain. For
example, it's missing the ensureSigM goroutine.

Replace it with a check that simply looks for any goroutine with
runtime function as its entry point, with a few exceptions. This also
matches the definition recently added to the trace viewer (CL 81315).

Change-Id: Iaed723d4a6e8c2ffb7c0c48fbac1688b00b30f01
Reviewed-on: https://go-review.googlesource.com/81655
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-05-07 21:38:40 +00:00
Hana Kim af5143e384 cmd/trace: use different colors for tasks
and assign the same colors for spans belong to the tasks
(sadly, the trace viewer will change the saturation/ligthness
for asynchronous slices so exact color mapping is impossible.
But I hope they are not too far from each other)

Change-Id: Idaaf0828a1e0dac8012d336dcefa1c6572ddca2e
Reviewed-on: https://go-review.googlesource.com/109338
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-29 18:14:22 +00:00
Hana Kim e357ece0af cmd/trace: have tasks in a separate section (process group)
Also change tasks to be represented as "slices" instead of
asynchronous events which are more efficiently represented in trace
viewer data model. This change allows to utilize the flow events
(arrows) to represent task hierarchies.

Introduced RegionArgs and TaskArgs where the task id infomation and
goroutine id informations are stored for information-purpose.

Change-Id: I11bec7dd716fdfc5f94ea39661b2e51344367a6f
Reviewed-on: https://go-review.googlesource.com/109337
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-26 20:56:44 +00:00
Hana Kim 011f6c5fa0 cmd/trace: distinguish task endTimestamp and lastTimestamp
A task may have other user annotation events after the task ends.
So far, task.lastTimestamp returned the task end event if the
event available. This change introduces task.endTimestamp for that
and makes task.lastTimestamp returns the "last" seen event's timestamp
if the task is ended.

If the task is not ended, both returns the last timestamp of the entire
trace assuming the task is still active.

This fixes the task-oriented trace view mode not to drop user
annotation instances when they appear outside a task's lifespan.
Adds a test.

Change-Id: Iba1062914f224edd521b9ee55c6cd5e180e55359
Reviewed-on: https://go-review.googlesource.com/109175
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-24 21:49:40 +00:00
Hana Kim c2d1024368 runtime/trace: rename "Span" with "Region"
"Span" is a commonly used term in many distributed tracing systems
(Dapper, OpenCensus, OpenTracing, ...). They use it to refer to a
period of time, not necessarily tied into execution of underlying
processor, thread, or goroutine, unlike the "Span" of runtime/trace
package.

Since distributed tracing and go runtime execution tracing are
already similar enough to cause confusion, this CL attempts to avoid
using the same word if possible.

"Region" is being used in a certain tracing system to refer to a code
region which is pretty close to what runtime/trace.Span currently
refers to. So, replace that.
https://software.intel.com/en-us/itc-user-and-reference-guide-defining-and-recording-functions-or-regions

This CL also tweaks APIs a bit based on jbd and heschi's comments:

  NewContext -> NewTask
    and it now returns a Task object that exports End method.

  StartSpan -> StartRegion
    and it now returns a Region object that exports End method.

Also, changed WithSpan to WithRegion and it now takes func() with no
context. Another thought is to get rid of WithRegion. It is a nice
concept but in practice, it seems problematic (a lot of code churn,
and polluting stack trace). Already, the tracing concept is very low
level, and we hope this API to be used with great care.

Recommended usage will be
   defer trace.StartRegion(ctx, "someRegion").End()

Left old APIs untouched in this CL. Once the usage of them are cleaned
up, they will be removed in a separate CL.

Change-Id: I73880635e437f3aad51314331a035dd1459b9f3a
Reviewed-on: https://go-review.googlesource.com/108296
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JBD <jbd@google.com>
2018-04-24 16:33:15 +00:00
Hana Kim 1d8fc211f9 cmd/trace: change span id computation for trace view use
golang.org/cl/102697 attempted to fix the span presentation by utilizing
the position of the span in the span slices of a task. But it is
not complete either.

First, id=0 is omitted in json encoding and the trace viewer silently
drops entries with the missing id field, so we must avoid zero-value id.
Second, it is possible that a goroutine handles multiple tasks. Then,
id collisions will happen.

This takes a simpler approach - have a counter that increments for every
emitSpan call, and use the value as the id value.

Change-Id: Idaa9505634acf6d327c6f00af32d8260955b85e1
Reviewed-on: https://go-review.googlesource.com/106755
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-04-13 16:18:30 +00:00
Hana Kim e6ab614fda cmd/trace: avoid emitting traceview slice with 0 duration
The trace viewer interprets the slice as a non-terminating
time interval which is quite opposit to what trace records indicate
(i.e., almostly immediately terminating time interval).
As observed in the issue #24663 this can result in quite misleading
visualization of the trace.

Work around the trace viewer's issue by setting a small value
(0.0001usec) as the duration if the time interval is not positive.

Change-Id: I1c2aac135c194d0717f5c01a98ca60ffb14ef45c
Reviewed-on: https://go-review.googlesource.com/104716
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-04 18:15:20 +00:00
Hana Kim c4874aa2a2 cmd/trace: implement /trace?focustask=<taskid> mode
This mode is similar to the default traceview mode where the execution
trace is presented in P-oriented way. Each row represents a P, and each
slice represents the time interval of a goroutine's execution on the P.

The difference is that, in this mode, only the execution of goroutines
involved in the specified task is highlighted, and other goroutine
execution or events are greyed out. So, users can focus on how a task is
executed while considering other affecting conditions such as other
goroutines, network events, or process scheduling.

Example: https://user-images.githubusercontent.com/4999471/38116793-a6f995f0-337f-11e8-8de9-88eec2f2c497.png

Here, for a while the program remained idle after the first burst of
activity related to the task because all other goroutines were also
being blocked or waiting for events, or no incoming network traffic
(indicated by the lack of any network activity). This is a bit hard to
discover when the usual task-oriented view (/trace?taskid=<taskid>)
mode.

Also, it simplifies the traceview generation mode logic.
  /trace ---> 0
  /trace?goid ---> modeGoroutineOriented
  /trace?taskid ---> modeGoroutineOriented|modeTaskOriented
  /trace?focustask ---> modeTaskOriented

Change-Id: Idcc0ae31b708ddfd19766f4e26ee7efdafecd3a5
Reviewed-on: https://go-review.googlesource.com/103555
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-04-04 18:14:40 +00:00
Hana Kim aaeaad6870 cmd/trace: assign a unique span id for slice representation
Spans are represented using Async Event types of chrome trace viewer.
According to the doc, the 'id' should be unique within category, scope.

https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview#heading=h.jh64i9l3vwa1

Use the index in the task's span slice as the slice id, so it
can be unique within the task. The scope is the task id which
is unique.

This fixes a visualization bug that caused incorrect or missing
presentation of nested spans.

Change-Id: If1537ee00247f71fa967abfe45569a9e7dbcdce7
Reviewed-on: https://go-review.googlesource.com/102697
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-27 19:03:54 +00:00
Hana Kim f0eca373be cmd/trace: add /userspans, /userspan pages
Change-Id: Ifbefb659a8df3b079d69679871af444b179deaeb
Reviewed-on: https://go-review.googlesource.com/102599
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-26 22:30:52 +00:00
Hana Kim 68a1c9c400 internal/trace: compute span stats as computing goroutine stats
Move part of UserSpan event processing from cmd/trace.analyzeAnnotations
to internal/trace.GoroutineStats that returns analyzed per-goroutine
execution information. Now the execution information includes list of
spans and their execution information.

cmd/trace.analyzeAnnotations utilizes the span execution information
from internal/trace.GoroutineStats and connects them with task
information.

Change-Id: Ib7f79a3ba652a4ae55cd81ea17565bcc7e241c5c
Reviewed-on: https://go-review.googlesource.com/101917
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-03-26 16:59:01 +00:00
Hana Kim 6b5a0b5c16 cmd/trace: set cname for span slices
Define a set of color names available in trace viewer

https://user-images.githubusercontent.com/4999471/37063995-5d0bad48-2169-11e8-92be-9cb363e21c38.png

Change-Id: I312fcbc5430d7512b4c39ddc79a769259bad8c22
Reviewed-on: https://go-review.googlesource.com/99055
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-09 01:58:41 +00:00
Hana Kim 476f71c958 cmd/trace: remove unrelated arrows in task-oriented traceview
Also grey out instants that represent events occurred outside the
task's span. Furthermore, if the unrelated instants represent user
annotation events but not for the task of the interest, skip rendering
completely.

This helps users to focus on the task-related events better.

UI screen shot:
https://gist.github.com/hyangah/1df5d2c8f429fd933c481e9636b89b55#file-golang-org_cl_99035

Change-Id: I2b5aef41584c827f8c1e915d0d8e5c95fe2b4b65
Reviewed-on: https://go-review.googlesource.com/99035
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-03-09 01:34:12 +00:00
Hana Kim 93b0261d0a cmd/trace: force GC occassionally
to return memory to the OS after completing potentially
large operations.

Update #21870

Sys went down to 3.7G

$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out

2018/03/07 09:35:52 Parsing trace...
after parsing trace
 Alloc:	3385754360 Bytes
 Sys:	3662047864 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488907264 Bytes
 HeapInUse:	3426549760 Bytes
 HeapAlloc:	3385754360 Bytes
Enter to continue...
2018/03/07 09:36:09 Splitting trace...
after spliting trace
 Alloc:	3238309424 Bytes
 Sys:	3684410168 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488874496 Bytes
 HeapInUse:	3266461696 Bytes
 HeapAlloc:	3238309424 Bytes
Enter to continue...
2018/03/07 09:36:39 Opening browser. Trace viewer is listening on http://100.101.224.241:12345

after httpJsonTrace
 Alloc:	3000633872 Bytes
 Sys:	3693978424 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488743424 Bytes
 HeapInUse:	3030966272 Bytes
 HeapAlloc:	3000633872 Bytes
Enter to continue...

Change-Id: I56f64cae66c809cbfbad03fba7bd0d35494c1d04
Reviewed-on: https://go-review.googlesource.com/92376
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-03-07 14:39:25 +00:00
Hana Kim ee465831ec cmd/trace: generate jsontrace data in a streaming fashion
Update #21870

The Sys went down to 4.25G from 6.2G.

$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out
2018/03/07 08:49:01 Parsing trace...
after parsing trace
 Alloc:	3385757184 Bytes
 Sys:	3661195896 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488841728 Bytes
 HeapInUse:	3426516992 Bytes
 HeapAlloc:	3385757184 Bytes
Enter to continue...
2018/03/07 08:49:18 Splitting trace...
after spliting trace
 Alloc:	2352071904 Bytes
 Sys:	4243825464 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	4025712640 Bytes
 HeapInUse:	2377703424 Bytes
 HeapAlloc:	2352071904 Bytes
Enter to continue...
after httpJsonTrace
 Alloc:	3228697832 Bytes
 Sys:	4250379064 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	4025647104 Bytes
 HeapInUse:	3260014592 Bytes
 HeapAlloc:	3228697832 Bytes

Change-Id: I546f26bdbc68b1e58f1af1235a0e299dc0ff115e
Reviewed-on: https://go-review.googlesource.com/92375
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-03-07 14:33:54 +00:00
Hana Kim a66af7280d cmd/trace: add memory usage reporting
Enabled when the tool runs with DEBUG_MEMORY_USAGE=1 env var.
After reporting the usage, it waits until user enters input
(helpful when checking top or other memory monitor)

Also adds net/http/pprof to export debug endpoints.

From the trace included in #21870

$ DEBUG_MEMORY_USAGE=1 go tool trace trace.out
2018/02/21 16:04:49 Parsing trace...
after parsing trace
 Alloc:	3385747848 Bytes
 Sys:	3661654648 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	3488907264 Bytes
 HeapInUse:	3426377728 Bytes
 HeapAlloc:	3385747848 Bytes
Enter to continue...
2018/02/21 16:05:09 Serializing trace...
after generating trace
 Alloc:	4908929616 Bytes
 Sys:	5319063640 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	5032411136 Bytes
 HeapInUse:	4982865920 Bytes
 HeapAlloc:	4908929616 Bytes
Enter to continue...
2018/02/21 16:05:18 Splitting trace...
after spliting trace
 Alloc:	4909026200 Bytes
 Sys:	5319063640 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	5032411136 Bytes
 HeapInUse:	4983046144 Bytes
 HeapAlloc:	4909026200 Bytes
Enter to continue...
2018/02/21 16:05:39 Opening browser. Trace viewer is listening on http://127.0.0.1:33661
after httpJsonTrace
 Alloc:	5288336048 Bytes
 Sys:	7790245896 Bytes
 HeapReleased:	0 Bytes
 HeapSys:	7381123072 Bytes
 HeapInUse:	5324120064 Bytes
 HeapAlloc:	5288336048 Bytes
Enter to continue...

Change-Id: I88bb3cb1af3cb62e4643a8cbafd5823672b2e464
Reviewed-on: https://go-review.googlesource.com/92355
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-02-21 21:23:08 +00:00
Hana Kim 3e1ac1b017 cmd/trace: include P info in goroutine slices
The task-oriented trace view presents the execution trace organized
based on goroutines. Often, which P a goroutine was running on is
useful, so this CL includes the P ids in the goroutine execution slices.

R=go1.11

Change-Id: I96539bf8215e5c1cd8cc997a90204f57347c48c8
Reviewed-on: https://go-review.googlesource.com/90221
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-21 20:14:30 +00:00
Hana Kim f42418b25d cmd/trace: add user log event in the task-oriented trace view
Also append stack traces to task create/end slices.

R=go1.11

Change-Id: I2adb342e92b36d30bee2860393618eb4064450cf
Reviewed-on: https://go-review.googlesource.com/90220
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-21 20:13:47 +00:00
Hana Kim cacf8127e3 cmd/trace: present the GC time in the usertask view
The GC time for a task is defined by the sum of GC duration
overlapping with the task's duration.

Also, grey out non-overlapping slices in the task-oriented
trace view.

R=go1.11

Change-Id: I42def0eb520f5d9bd07edd265e558706f6fab552
Reviewed-on: https://go-review.googlesource.com/90219
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-21 19:48:35 +00:00
Hana Kim fdcf4f712b cmd/trace: task-oriented view includes child tasks
R=go1.11

Change-Id: Ibb09e309c745eba811a0b53000c063bc10a055e1
Reviewed-on: https://go-review.googlesource.com/90218
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-02-20 19:44:53 +00:00
Hana Kim d6856036bf cmd/trace: extend trace view (/trace) for task-oriented view
R=go1.11

Change-Id: I2d2db148fed96d0fcb228bee414b050fe4e46e2c
Reviewed-on: https://go-review.googlesource.com/90217
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-20 19:28:38 +00:00
Hana Kim d07581f28e cmd/trace: add analyzeAnnotation and /usertasks view.
R=go1.11

Change-Id: I5078ab714c8ac2c652e6ec496e01b063235a014a
Reviewed-on: https://go-review.googlesource.com/90216
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2018-02-20 19:25:37 +00:00
Austin Clements 21ced9c748 cmd/trace: encode selection in trace URL
This adds the ability to add a #x:y anchor to the trace view URL that
causes the viewer to initially select from x ms to y ms.

Change-Id: I4a980d8128ecc85dbe41f224e8ae336707a4eaab
Reviewed-on: https://go-review.googlesource.com/60794
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-02-20 18:22:54 +00:00
Hana Kim a58286c289 cmd/trace: init goroutine info entries with GoCreate event
golang.org/cl/81315 attempted to distinguish system goroutines
by examining the function name in the goroutine stack. It assumes that
the information would be available when GoSysBlock or GoInSyscall
events are processed, but it turned out the stack information is
set too late (when the goroutine gets a chance to run).

This change initializes the goroutine information entry when
processing GoCreate event which should be one of the very first
events for the every goroutine in trace.

Fixes #22574

Change-Id: I1ed37087ce2e78ed27c9b419b7d942eb4140cc69
Reviewed-on: https://go-review.googlesource.com/83595
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-12-20 23:04:21 +00:00
Hana Kim 662938850b cmd/trace: exclude threads in syscall on behalf of runtime
The number of threads in syscall presented by execution tracer's
trace view includes not only the threads calling system calls on behalf
of user created goroutines, but also those running on behalf of system
goroutines.

When the number of such system goroutines was small, the graph was
useful when examining where a program was saturating the CPU.
But as more and more system goroutines are invloved the graph became
less useful for the purpose - for example, after golang.org/cl/34784,
the timer goroutines dominate in the graph with large P
because the runtime creates per-P timer goroutines.

This change excludes the threads in syscall on behalf of runtime (system
goroutines) from the visualization. Alternatively, I could visualize the
count of such threads in a separate counter but in the same graph.
Given that many other debug endpoints (e.g. /debug/pprof/goroutine) hide
the system goroutines, including them in the same graph can confuse users.

Update #22574

Change-Id: If758cd6b9ed0596fde9a471e846b93246580b9d5
Reviewed-on: https://go-review.googlesource.com/81315
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-12-01 21:00:50 +00:00
Russ Cox 918b98ca70 cmd/internal/obj, cmd/trace: restore bounds checks dropped in CL 56950
CL 56950 correctly identified code with checks that were impossible.
But instead of correcting the checks it deleted them.
This CL corrects the code to check what was meant.

Change-Id: Ic89222184ee4fa5cacccae12d750601a9438ac8d
Reviewed-on: https://go-review.googlesource.com/78113
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-11-16 16:29:08 +00:00
Hana Kim 2f40dc79e5 cmd/trace: fix a javascript bug in handling import error
When traceviewer encounters a failure of json trace import
due to data error, onImportFail tried to access an error variable
which was not yet defined.

Change-Id: I431be03f179aafacaf1fd3c62a6337e8b5bd18fb
Reviewed-on: https://go-review.googlesource.com/71970
Reviewed-by: Austin Clements <austin@google.com>
2017-10-20 16:34:50 +00:00
Austin Clements b0392159f6 runtime,cmd/trace: trace GC STW events
Right now we only kind of sort of trace GC STW events. We emit events
around mark termination, but those start well after stopping the world
and end before starting it again, and we don't emit any events for
sweep termination.

Fix this by generalizing EvGCScanStart/EvGCScanDone. These were
already re-purposed to indicate mark termination (despite the names).
This commit renames them to EvGCSTWStart/EvGCSTWDone, adds an argument
to indicate the STW reason, and shuffles the runtime to generate them
right before stopping the world and right after starting the world,
respectively.

These events will make it possible to generate precise minimum mutator
utilization (MMU) graphs and could be useful in detecting
non-preemptible goroutines (e.g., #20792).

Change-Id: If95783f370781d8ef66addd94886028103a7c26f
Reviewed-on: https://go-review.googlesource.com/55411
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-08-29 21:54:55 +00:00
Hana Kim 05ff6bfe33 misc/trace: update trace-viewer
Generated with
 github.com/catapult/tracing/bin/vulcanize_trace_viewer
catapult @ ab4d571fa

Renamed trace_viewer_lean.html to trace_viewer_full.html
to make it clear we are using the full version of trace viewer
(waiting for https://github.com/catapult-project/catapult/issues/2247
to be fixed).

Update #15302

Change-Id: Ice808bb27ab79a1dec9fc863e0c5a761027ebfbe
Reviewed-on: https://go-review.googlesource.com/58750
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2017-08-25 15:50:29 +00:00
Daniel Martí 943dd0fe33 cmd/*: remove negative uint checks
All of these are uints of different sizes, so checking >= 0 or < 0 are
effectively no-ops.

Found with staticcheck.

Change-Id: I16ac900eb7007bc8f9018b302136d42e483a4180
Reviewed-on: https://go-review.googlesource.com/56950
Reviewed-by: Matt Layher <mdlayher@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Matt Layher <mdlayher@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-18 18:45:05 +00:00
Austin Clements a1eec645e1 cmd/trace: don't shift trace slices to 0
Currently all trace slices get shifted to start at time 0. This makes
it very difficult to find specific points in time unless they fall in
the first slice.

For example, right now when you click "View trace
(6.005646218s-8.155419698s)" on the trace tool's main page, the trace
view puts the first event in that slice at time 0. If you're looking
for something that happened at time 7s, you have to look at time
0.9943537s in the trace view. And if you want to subtract times taken
from different slices, you have to figure out what those time really
correspond to.

Fix this by telling the trace viewer not to shift the times when it
imports the trace. In the above example, this makes the view of that
second trace slice start at time 6.005646218s, so you don't have to do
any gymnastics to find or calculate times in later slices.

Change-Id: I04e0afda60f5573fdd8ad96238c24013297ef263
Reviewed-on: https://go-review.googlesource.com/54633
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-08-11 17:53:17 +00:00
Austin Clements 55f19b8d1d cmd/trace: update HTML; expand viewer to whole window
This updates the HTML served for the trace viewer to follow the latest
revision of the example from the upstream tracing project.

The main thing this adds is CSS for the trace viewer (which was
actually in the example at the originally referenced revision, so I'm
not sure why it got dropped). In particular, this expands the trace
viewer to use the entire browser client area, which fixes several
problems with the current page:

1. The details pane gets cut off at a strange place and can get a
scroll bar even if there's plenty of room below it on the page. This
fixes the bottom of the details pane to the bottom of the window.

2. If the track view is very tall (lots of procs), there's no way to
view the top tracks and the details pane at the same time. This fixes
this problem by limiting the height of the track view to something
less than the height of the window so it gets a scroll bar of its own
if necessary.

3. Dragging the divider between the track pane and the details pane
actually moves the bottom of the details pane without moving the
divider. Fixing the height of the trace viewer fixes this problem.

Change-Id: Ia811e72a7413417ca21c45e932c9db2724974633
Reviewed-on: https://go-review.googlesource.com/54632
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-08-11 17:52:41 +00:00
Austin Clements 7ca53f2f8b cmd/trace: show swept and reclaimed bytes
This displays the swept and reclaimed bytes for sweep events in the
lower panel of the trace viewer.

Change-Id: If1665a1c02bbc47700e0d9f515e574f013f3f285
Reviewed-on: https://go-review.googlesource.com/40812
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-04-19 18:31:16 +00:00
Heschi Kreinick 4de2efe927 cmd/trace: traces may end with pending mark assists
There's no guarantee that all in-progress mark assists will finish
before the trace does. Don't crash if that happens.

I haven't added a test because there's quite a bit of ceremony involved
and the bug is fairly straightforward.

Change-Id: Ia1369a8e2260fc6a328ad204a1eab1063d2e2c90
Reviewed-on: https://go-review.googlesource.com/37540
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 18:33:54 +00:00
Heschi Kreinick 2a74b9e814 cmd/trace: Record mark assists in execution traces
During the mark phase of garbage collection, goroutines that allocate
may be recruited to assist. This change creates trace events for mark
assists and displays them similarly to sweep assists in the trace
viewer.

Mark assists are different than sweeps in that they can be preempted, so
displaying them in the trace viewer is a little tricky -- we may need to
synthesize multiple slices for one mark assist. This could have been
done in the parser instead, but I thought it might be preferable to keep
the parser as true to the event stream as possible.

Change-Id: I381dcb1027a187a354b1858537851fa68a620ea7
Reviewed-on: https://go-review.googlesource.com/36015
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2017-02-10 18:03:42 +00:00
Austin Clements d0b3c169ac cmd/trace: fix goroutine view
Currently, trace processing interleaves state/statistics updates and
emitting trace viewer objects. As a result, if events are being
filtered, either by time or by goroutines, we'll miss those
state/statistics updates. At best, this leads to bad statistics;
however, since we're now strictly checking G state transitions, it
usually leads to a failure to process the trace if there is any
filtering.

Fix this by separating state updates from emitting trace object. State
updates are done before filtering, so we always have correct state
information and statistics. Trace objects are only emitted if we pass
the filter. To determine when we need to emit trace counters, rather
than duplicating the knowledge of which events might modify
statistics, we keep track of the previously emitted counters and emit
a trace counter object whenever these have changed.

Fixes #17719.

Change-Id: Ic66e3ddaef60d1acaaf2ff4c62baa5352799cf99
Reviewed-on: https://go-review.googlesource.com/32810
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-11-18 17:49:18 +00:00
Austin Clements 6da83c6fc0 runtime, cmd/trace: track goroutines blocked on GC assists
Currently when a goroutine blocks on a GC assist, it emits a generic
EvGoBlock event. Since assist blocking events and, in particular, the
length of the blocked assist queue, are important for diagnosing GC
behavior, this commit adds a new EvGoBlockGC event for blocking on a
GC assist. The trace viewer uses this event to report a "waiting on
GC" count in the "Goroutines" row. This makes sense because, unlike
other blocked goroutines, these goroutines do have work to do, so
being blocked on a GC assist is quite similar to being in the
"runnable" state, which we also report in the trace viewer.

Change-Id: Ic21a326992606b121ea3d3d00110d8d1fdc7a5ef
Reviewed-on: https://go-review.googlesource.com/30704
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-28 14:29:47 +00:00
Austin Clements 640e916915 cmd/trace: track each G's state explicitly
Currently the trace tool tracks an overall counts of goroutine states,
but not the states of any individual goroutine. We're about to add
more sophisticated blocked-state tracking, so add this tracking and
base the state counts off the tracked goroutine states.

Change-Id: I943ed61782436cf9540f4ee26c5561715c5b4a1d
Reviewed-on: https://go-review.googlesource.com/30703
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-28 14:29:44 +00:00
Austin Clements 6834839427 runtime, cmd/trace: annotate different mark worker types
Currently mark workers are shown in the trace as regular goroutines
labeled "runtime.gcBgMarkWorker". That's somewhat unhelpful to an end
user because of the opaque label and particularly unhelpful to runtime
developers because it doesn't distinguish the different types of mark
workers.

Fix this by introducing a variant of the GoStart event called
GoStartLabel that lets the runtime indicate a label for a goroutine
execution span and using this to label mark worker executions as "GC
(<mode>)" in the trace viewer.

Since this bumps the trace version to 1.8, we also add test data for
1.7 traces.

Change-Id: Id7b9c0536508430c661ffb9e40e436f3901ca121
Reviewed-on: https://go-review.googlesource.com/30702
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
2016-10-28 14:29:40 +00:00