This commit was dedicated to adding an example of using B.ReportMetrics
with B.RunParallel called ExampleB_ReportMetric_parallel. In this
example, the same algorithm for ExampleB_ReportMetric was used, instead
with a concurrent for loop using PB.Next instead of a standard one.
There is also notes noting when to use the B.ReportMetric methods when
running concurrent testing.
Fixes#50756
Change-Id: I2a621b4e367af5f4ec47d38a0da1035a8d52f628
Reviewed-on: https://go-review.googlesource.com/c/go/+/437815
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Add the description of Setenv that it cannot use if the test have
parallel ancestors.
Fixes#55128
Change-Id: Ia5a1deaa1a3116d1ebb439600a7d316c7d155412
Reviewed-on: https://go-review.googlesource.com/c/go/+/434115
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Add a testpoint to cover support routines used to help
implement "go test -cover".
Change-Id: Ic28bf884a4e0d2c0a6d8fd04fc29c0c949227f21
Reviewed-on: https://go-review.googlesource.com/c/go/+/432315
Reviewed-by: Bryan Mills <bcmills@google.com>
The existing implementation can call to T.Setenv in T.Run even after
calling to T.Parallel, so I changed it to cause a panic in that case.
Fixes#55128
Change-Id: Ib89d998ff56f00f96a5ca218af071bd35fdae53a
Reviewed-on: https://go-review.googlesource.com/c/go/+/431101
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
If GOEXPERIMENT=coverageredesign is in effect, introduce a new
top-level '-cover' option to "go build" to turn on new-style hybrid
code coverage instrumentation. Similarly, use the new instrumentation
for "go test -cover".
The main effects of "-cover" under the hood are to instrument files at
the package level using cmd/cover and to pass additional options to
the compiler when building instrumented packages.
The previous workflow for "go tool -cover mypkg" would expand to a
series of "go tool cover" commands (one per file) followed by a single
package compilation command to build the rewritten sources.
With the new workflow, the Go command will pass all of the Go files in
a package to the cover tool as a chunk (along with a config file
containing other parameters), then the cover tool will write
instrumented versions of the sources along with another "output"
config with info on coverage variable names for the the compiler. The
Go command will then kick off the compiler on the modified source
files, also passing in the config file generated by cmd/cover.
Updates #51430.
Change-Id: Id65621ff6a8c70a30168c1412c2d6f805ff3b9e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/355452
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
For proposal #41583, add a new 'go test -skip' flag to make it easy
to disable specific tests, benchmarks, examples, or fuzz targets.
Fixes#41583.
Change-Id: Id12a6575f505dafdce4a149aedc454a002e93afa
Reviewed-on: https://go-review.googlesource.com/c/go/+/421439
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
The existing documentation did not explain the difference between
placing a _test.go file in the same package as what is being
tested vs. adding it to a separate _test package. This explains the
distinction and adds an example.
Concept is explained well here: https://stackoverflow.com/a/31443271Fixes#25223
Change-Id: Iebaba15207d8aa24f0b370d8dd4062eadb504b5c
GitHub-Last-Rev: 7f49c5f462
GitHub-Pull-Request: golang/go#54160
Reviewed-on: https://go-review.googlesource.com/c/go/+/420415
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Elapsed returns the measured elapsed time of the benchmark,
but does not change the running state of the timer.
Fixes#43620.
Change-Id: Idd9f64c4632518eec759d2ffccbf0050d84fcc03
Reviewed-on: https://go-review.googlesource.com/c/go/+/420254
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: hopehook <hopehook@qq.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
$ cd $GOROOT/src/testing
$ go test
root
root
root
root
PASS
$
The root prints have been happening since Go 1.14.
There is a test in sub_test.go that calls b.Run directly
with a benchmark named "root", which triggers the print.
Silence them.
Change-Id: I2f0c186f04c6139bc24fab0e91975fcf0a8e80fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/421437
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
These are available since Go 1.7. The version used for bootstrap is Go
1.17 for Go 1.20.
For #44505
Change-Id: I497c9f617baefdeb273cd115b08b6e31bd10aad2
Reviewed-on: https://go-review.googlesource.com/c/go/+/421634
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
This reverts commit 4907c62f99.
Reason for revert: Race detector v3, which we just upgraded to, no longer has a goroutine limit.
(small caveat: openbsd/amd64 can't be updated, windows/amd64 isn't updated yet but should be by release time.)
Change-Id: I90017834501e81d3990d888f1b2baf3432452846
Reviewed-on: https://go-review.googlesource.com/c/go/+/405595
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Make some code more simple.
Change-Id: I801adf0dba5f6c515681345c732dbb907f945419
GitHub-Last-Rev: a505146bac
GitHub-Pull-Request: golang/go#49626
Reviewed-on: https://go-review.googlesource.com/c/go/+/364634
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
[This CL is part of a sequence implementing the proposal #51082.
The design doc is at https://go.dev/s/godocfmt-design.]
Run the updated gofmt, which reformats doc comments,
on the main repository. Vendored files are excluded.
For #51082.
Change-Id: I7332f099b60f716295fb34719c98c04eb1a85407
Reviewed-on: https://go-review.googlesource.com/c/go/+/384268
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It has been agreed that we should prefer the US spelling of words like
"canceling" over "cancelling"; for example, see https://go.dev/cl/14526.
Fix a few occurrences of the "canceling" inconsistency, as well as:
* signaling
* tunneling
* marshaling
Change-Id: I99f3ba0a700a9f0292bc6c1b110af31dd05f1ff0
Reviewed-on: https://go-review.googlesource.com/c/go/+/398734
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A future change to gofmt will rewrite
// Doc comment.
//
func f()
to
// Doc comment.
func f()
Apply that change preemptively to all doc comments.
For #51082.
Change-Id: I4023e16cfb0729b64a8590f071cd92f17343081d
Reviewed-on: https://go-review.googlesource.com/c/go/+/384259
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Otherwise, the behavior of a fuzz target that returns an error could
be confusing.
Fuzz is already documented to require a function “with no return
value”, so this fixes the implementation to match the existing
documentation.
Fixes#51222
Change-Id: I44ca7ee10960214c92f5ac066ac8484c8bb9cd6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/386175
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Nooras Saba <saba@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Only snapshot/reset coverage counters when we are actually fuzzing.
This prevents a race when running corpus/seed values during the testing
phase.
Fixes#50488
Change-Id: I7dd5a0353a296c0b13eede29ad9af7c78814fa2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/376554
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
The c.startParallel channel on the testContext is stuck
in t.Parallel() because c.running starts at 1 for the main
fuzz parent test, and is causing a deadlock because it is
never released. It would normally be released by tRunner,
but needs to instead be released by fRunner instead for fuzz
tests.
Fixes#50217
Change-Id: I2d010e9adddfd8e8321ff2f9dd2e43daf46c128f
Reviewed-on: https://go-review.googlesource.com/c/go/+/374054
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This works around what appears to be either a kernel bug or a Go
runtime or syscall bug affecting certain Windows versions
(possibly all pre-2016?).
The retry loop is a simplified version of the one used in
cmd/go/internal/robustio. We use the same 2-second arbitrary timeout
as was used in that package, since it seems to be reliable in practice
on the affected builders. (If it proves to be too short, we can
lengthen it, within reason, in a followup CL.)
Since this puts a higher-level workaround in place, we can also revert
the lower-level workaround added to a specific test in CL 345670.
This addresses the specific occurrences of the bug for users of
(*testing.T).TempDir, but does not fix the underlying bug for Go users
outside the "testing" package (which remains open as #25965).
Fixes#50051
Updates #48012
Updates #25965
Change-Id: I35be7125f32f05c8350787f5ca9a22974b8d0770
Reviewed-on: https://go-review.googlesource.com/c/go/+/371296
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Patrik Nyblom <pnyb@google.com>
Trust: Patrik Nyblom <pnyb@google.com>
Run-TryBot: Patrik Nyblom <pnyb@google.com>
Although most of the code seems to be already implemented
to support this for general use, it didn't make it in for
Go 1.18, so for now we should at least document that it's
only for use by the go command.
Change-Id: Id559e72d590aedeaaa50bcf880bca1a385d858dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/370954
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.
Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.
A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.
Fixes#49884.
Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change doesn't modify any functionality.
It also doesn't update all of the comments and
variable names of the internal code, but everything
user facing should be correct.
Updates #49185
Change-Id: Ia8b2c94b89ba45897c4085ea0c17a3d8896f7ec7
Reviewed-on: https://go-review.googlesource.com/c/go/+/362794
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
This is heavily based on CL 341336 by Joe Tsai and CL 351452 by
Jay Conrod.
T.Run and T.Name use a map[string]int64 to hold the next suffix to use
when duplicate names are passed to T.Run. This map necessarily retains
one entry per unique name. However, it's a waste of memory to retain
one entry per duplicate name: when we encounter the Nth duplicate, we
know that names 00 through N-1 have been used just by looking at N.
We do still need to store (and check for collisions againsts) explicit
names provided by the caller. For example, if the user passes in "a",
then "a#01", then "a" again, we cannot deduplicate the second "a" to
"a#01" — we need to instead skip ahead to "a#02". We can do so by
checking the count of "a", then generating a proposed deduplicated
name, then double-checking that proposed name against only the
explicit names so far.
This somewhat reduces memory usage for tests that spawn large numbers
of duplicate subtests, but doesn't solve the problem of memory growth
for fuzzing — we still have to track all of the explicit,
user-provided subtest names, and in a long-running fuzz test that set
alone may be unbounded.
This fixes memory growth for the example described in
https://golang.org/issue/44517#issuecomment-897104060,
but not the one in
https://golang.org/issue/44517#issuecomment-933825661.
For #44517
Change-Id: Ia159ecfcf44561ba67508d3af6377c27856df31d
Reviewed-on: https://go-review.googlesource.com/c/go/+/354749
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
When these packages are released as part of Go 1.18,
Go 1.16 will no longer be supported, so we can remove
the +build tags in these files.
Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs
as defined in src/cmd/dist/buildtool.go, which need to continue
to build with Go 1.4 for now.
Also reverted src/vendor and src/cmd/vendor, which will need
to be updated in their own repos first.
Manual changes in runtime/pprof/mprof_test.go to adjust line numbers.
For #41184.
Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e
Reviewed-on: https://go-review.googlesource.com/c/go/+/344955
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
When running benchmarks with high -count and no tests (either at all or
filtered with -run), the time for runTests to check for tests -count
times can add a significant delay to starting benchmarks.
To avoid this delay, make runTests bail out on the second iteration if
the first found no tests to run. We expect the same tests to run every
time, so there is no reason to duplicate work.
One caveat: the testing docs do not explicitly require the same subtests
to be run on every call, so this could break tests that depend on
multiple runs to actually run all tests. I consider such tests invalid,
but some may exist.
Fixes#49050
Change-Id: I7b34f3832b31493cc089ee0555e231f4dc690154
Reviewed-on: https://go-review.googlesource.com/c/go/+/356669
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This updates the testing package documentation to reference the
specification for the benchmark format, and points users to our
standard tools for working with benchmark data. (It's somewhat
remarkable how widely used benchstat appears to be given that we don't
mention it anywhere!)
Change-Id: Idbb4416d2fde9606ea7e6c15595f3b9e6a38f3b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/357589
Trust: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change also does some refactors around how
we prevent many (*F) methods from being called
inside (*F).Fuzz. Previously, there was a lot of
comment/code duplication, which was going to be
difficult to maintain and brittle. The refactor
lessens this duplication.
Previously, the methods Log, Logf, Failed, Name and
Skipped were the only (*common) methods that were
allowed to be called inside (*F).Fuzz. After this
change, Failed and Name are still allowed, but
Log, Logf, and Skipped are not (t.Log, t.Logf, or
t.Skipped should be used instead).
Fixes#48988
Change-Id: I4066247d551ea1908e8a2ca2889509fc68e3bb44
Reviewed-on: https://go-review.googlesource.com/c/go/+/356151
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
T.Run uses a map[string]int64 to keep track of subtest names that may
be returned through T.Name. T.Name can't return duplicate names for
subtests started with T.Run.
If a fuzz target calls T.Run, this map takes a large amount of memory,
since there are a very large number of subtests that would
otherwise have duplicate names, and the map stores one entry per subtest.
The unique suffixes are not useful (and may be confusing) since the
full sequence of tests cannot be re-run deterministically.
This change deletes all entries in the map before each call to the
function being fuzzed. There is a slight change in the contract of
T.Name while fuzzing.
This change was discussed in CL 351452.
Fixes#44517
Change-Id: I3093a2e5568099ce54aca1006fac84a6fd2c3ddf
Reviewed-on: https://go-review.googlesource.com/c/go/+/354430
Trust: Jay Conrod <jayconrod@google.com>
Trust: Katie Hockman <katie@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
This change fixes some issues with -run, and
the subsequent command line output when running
in verbose mode. It replaces CorpusEntry.Name
with CorpusEntry.Path, and refactors the code
accordingly.
This change also adds a lot of additional tests
which check explicit command line output when
fuzz targets are run without fuzzing. This will
be important to avoid regressions.
Updates #48149
Change-Id: If34b1f51db646317b7b51c3c38ae53231d01f568
Reviewed-on: https://go-review.googlesource.com/c/go/+/354632
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Like with -benchtime=1ns, if we find that the "discovery" round (run1)
has already crossed the -benchtime threshold, we skip running more
iterations.
Fixes#32051
Change-Id: I76aaef2ba521ba8ad6bbde2b14977e191aada5e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/331089
Trust: Caleb Spare <cespare@gmail.com>
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Based on comments in CL 348469.
Note that with this change, F.Fuzz no longer calls
runtime.Goexit. This simplifies our logic and makes F.Fuzz more
predictable.
Change-Id: I6c3c65b0e8e8f261621cbe2f17375e8164ef60a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/351316
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Rather than directly pointing at the underlying scratch slice, allocate
memory for strings. This prevents mutation of previous values we've
passed to the fuzz function, which may be retained by something that
expects them to be immutable.
Fixesgolang/go#48308
Change-Id: Iee9bed1a536fdc4188180e8e7c1c722f641271d2
Reviewed-on: https://go-review.googlesource.com/c/go/+/351312
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
This CL removes 'go help fuzz' but expands the testing package
documentation with much of the same information. It also removes
documentation for the unimplemented -keepfuzzing flag and makes a
number of other clarifications.
Addressing comments in CL 348469.
Updates #46629
Change-Id: I12ab5971c900c2e43f2c2d83c6705e8cd642388b
Reviewed-on: https://go-review.googlesource.com/c/go/+/351113
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Eddie Lopez
Nobuki Fujii
Than McIntosh
Russ Cox
cuiweixie
byarbrough
hopehook
Tobias Klauser
Keith Randall
Ryan Leung
Daniel Martí
Bryan C. Mills
Katie Hockman
Roland Shoemaker
Dan Kortschak
ichxxx
sivchari
Bharath Kumar Uppala
Alexander Melentyev
Cuong Manh Le
Michael Pratt
Austin Clements
Jay Conrod
Caleb Spare