Commit Graph

183 Commits

Author SHA1 Message Date
Bryan C. Mills 26c6b51bc1 [release-branch.go1.14] testing: flush test summaries to stdout atomically when streaming output
While debugging #40771, I realized that the chatty printer should only
ever print to a single io.Writer (normally os.Stdout). The other
Writer implementations in the chain write to local buffers, but if we
wrote a test's output to a local buffer, then we did *not* write it to
stdout and we should not store it as the most recently logged test.

Because the chatty printer should only ever print to one place, it
shouldn't receive an io.Writer as an argument — rather, it shouldn't
be used at all for destinations other than the main output stream.

On the other hand, when we flush the output buffer to stdout in the
top-level flushToParent call, it is important that we not allow some
other test's output to intrude between the test summary header and the
remainder of the test's output. cmd/test2json doesn't know how to
parse such an intrusion, and it's confusing to humans too.

No test because I couldn't reproduce the user-reported error without
modifying the testing package. (This behavior seems to be very
sensitive to output size and/or goroutine scheduling.)

Fixes #40880
Updates #40771
Updates #38458

Change-Id: Ic19bf1d535672b096ba1c8583a3b74aab6d6d766
Reviewed-on: https://go-review.googlesource.com/c/go/+/249026
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 51c0bdc6d1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/252638
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
2020-10-05 22:50:05 +00:00
Bryan C. Mills 2cab203bda [release-branch.go1.14] testing: treat PAUSE lines as changing the active test name
We could instead fix cmd/test2json to treat PAUSE lines as *not*
changing the active test name, but that seems like it would be more
confusing to humans, and also wouldn't fix tools that parse output
using existing builds of cmd/test2json.

Fixes #40848
Updates #40657

Change-Id: I937611778f5b1e7dd1d6e9f44424d7e725a589ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/248727
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jean de Klerk <deklerk@google.com>
(cherry picked from commit cdc77d34d7)
Reviewed-on: https://go-review.googlesource.com/c/go/+/249098
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-01 19:40:54 +00:00
Jean de Klerk 3445c3512b [release-branch.go1.14] testing: capture testname on --- PASS and --- FAIL lines
This fixes an issue raised at https://github.com/golang/go/issues/38458#issuecomment-635617670
in which --- PASS and --- FAIL lines would not trigger --- CONT lines
of other tests.

Updates #38458.
For #39308.

Change-Id: I0d8cc54d682a370d0a6ea6816a11b2e462a92efe
Reviewed-on: https://go-review.googlesource.com/c/go/+/235997
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242058
Reviewed-by: Jean de Klerk <deklerk@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2020-07-16 18:22:52 +00:00
Jean de Klerk 2ba9d45f9c [release-branch.go1.14] testing: reformat test chatty output
In #24929, we decided to stream chatty test output. It looks like,

foo_test.go:138: TestFoo/sub-1: hello from subtest 1
foo_test.go:138: TestFoo/sub-2: hello from subtest 2

In this CL, we refactor the output to be grouped by === CONT lines, preserving
the old test-file-before-log-line behavior:

=== CONT TestFoo/sub-1
    foo_test.go:138 hello from subtest 1
=== CONT TestFoo/sub-2
    foo_test.go:138 hello from subtest 2

This should remove a layer of verbosity from tests, and make it easier to group
together related lines. It also returns to a more familiar format (the
pre-streaming format), whilst still preserving the streaming feature.

Updates #38458.
Fixes #39308.

Change-Id: Iaef94c580d69cdd541b2ef055aa004f50d72d078
Reviewed-on: https://go-review.googlesource.com/c/go/+/229085
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-on: https://go-review.googlesource.com/c/go/+/242057
Reviewed-by: Jean de Klerk <deklerk@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Jean de Klerk <deklerk@google.com>
2020-07-16 18:20:50 +00:00
Changkun Ou ab9d037401 [release-branch.go1.14] testing: fix data race between parallel subtests
This CL fixes a race condition if there are two subtests, and
one finishing but the other is panicking.

For #37551
Fixes #37959

Change-Id: Ic33963eb338aec228964b95f7c34a0d207b91e00
Reviewed-on: https://go-review.googlesource.com/c/go/+/221322
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 93a9561b23)
Reviewed-on: https://go-review.googlesource.com/c/go/+/224257
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Changkun Ou <euryugasaki@gmail.com>
2020-03-20 19:05:17 +00:00
Ian Lance Taylor 6917529cc6 testing: remove obsolete comment in testing.(*T) docs
We now only accumulate logs when not using -v. Just drop the sentence
entirely rather than try to describe the current situation.

Updates #24929
Updates #37203

Change-Id: Ie3bf37894ab68b5b129eff54637893c7a129da03
Reviewed-on: https://go-review.googlesource.com/c/go/+/219540
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-02-15 16:40:12 +00:00
Roger Peppe ab7c174183 testing: make Cleanup work for benchmarks too.
Fixes #37073.

Change-Id: I6fb24a3f9d7b7adf3213ac6a8bcbf5fb43975b7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/218117
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-02-06 17:29:53 +00:00
Roger Peppe 03eeaa9656 testing: clarify that Cleanup is run after subtests complete.
It's good to be explicit, as it's not necessarily obvious (and indeed
the behavior has changed recently with https://go-review.googlesource.com/c/go/+/214822)
without an associated doc comment change).

Change-Id: I99d6398bf15b404b1b1b196e712e926e363251e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/215217
Reviewed-by: Paul Jolly <paul@myitcv.org.uk>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-18 08:45:35 +00:00
Ian Lance Taylor 998cbe2983 testing: don't run Cleanup functions until parallel subtests complete
Fixes #31651

Change-Id: Idbab0c4355fcc58520e210126795223435cf0078
Reviewed-on: https://go-review.googlesource.com/c/go/+/214822
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-16 21:32:12 +00:00
Alberto Donizetti 26f8b7074b testing: document that benchmark logs are dumped to stdout
The doc for testing.T and testing.B said that both test and benchmark
logs are printed to stderr, but in reality that wasn't the case.

CL 24311 fixed the doc for T, this change fixes it for B.

Fixes #36257

Change-Id: I0ff77ff44608f60320a1565b371c81e96039e71c
Reviewed-on: https://go-review.googlesource.com/c/go/+/212457
Reviewed-by: Gabriel Aszalos <gabriel.aszalos@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Gabriel Aszalos <gabriel.aszalos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-23 20:42:35 +00:00
vovapi 207a0b7933 testing: use conventional comments for exported internal funcs & structs
Change-Id: I2dd5ddc22bfff143b81d5945992d8c5fccf387f4
GitHub-Last-Rev: aa637756e7
GitHub-Pull-Request: golang/go#36054
Reviewed-on: https://go-review.googlesource.com/c/go/+/210497
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-12-10 21:11:20 +00:00
taisa 7bdef2dfa4 testing: fix testing docs
The Perm function return 0 or 1 or 2 or 3. 4 is not returned,
so that changed the argument to 5.

Change-Id: Ic980c71a9f29f522bdeef4fce70a6c2dd136d791
Reviewed-on: https://go-review.googlesource.com/c/go/+/209777
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-12-05 03:44:34 +00:00
Pantelis Sampaziotis 0994cc1af6 testing/iotest: add missing tests
Adds missing tests for all the types:
* OneByteReader
* HalfReader
* DataErrReader
* TimeoutReader
* TruncateWriter
* writeLogger
* readLogger

Fixes #33650

Change-Id: I1c773f9f1625ff33a1d0b5a045c72a73a9eca9ce
GitHub-Last-Rev: 2ab650677b
GitHub-Pull-Request: golang/go#33651
Reviewed-on: https://go-review.googlesource.com/c/go/+/190259
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-05 17:57:10 +00:00
Roger Peppe 1b2ff10136 testing: implement Cleanup method
Fixes #32111

Change-Id: I7078947889d1e126d9679fb28f27b3fa6ce133ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/201359
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-04 10:13:30 +00:00
Damien Neil 81a74b4e8d testing: provide additional information when test funcs panic
Flush the output log up to the root when a test panics. Prior to
this change, only the current test's output log was flushed to its
parent, resulting in no output when a subtest panics.

For the following test function:

	func Test(t *testing.T) {
		for i, test := range []int{1, 0, 2} {
			t.Run(fmt.Sprintf("%v/%v", i, test), func(t *testing.T) {
				_ = 1 / test
			})
		}
	}

Output before this change:

	panic: runtime error: integer divide by zero [recovered]
		panic: runtime error: integer divide by zero
	(stack trace follows)

Output after this change:

	--- FAIL: Test (0.00s)
	    --- FAIL: Test/1/0 (0.00s)
	panic: runtime error: integer divide by zero [recovered]
	(stack trace follows)

Fixes #32121

Change-Id: Ifee07ccc005f0493a902190a8be734943123b6b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/179599
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-30 20:46:44 +00:00
Jean de Klerk a813d3c788 testing: stream log output in verbose mode
Fixes #24929

Change-Id: Icc426068cd73b75b78001f55e1e5d81ccebbe854
Reviewed-on: https://go-review.googlesource.com/c/go/+/127120
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-22 19:21:02 +00:00
Bryan C. Mills 5e21032b3d Revert "cmd/go: move automatic testing.Init call into generated test code"
This reverts CL 176098.

Reason for revert: added complexity, but did not completely fix the
underlying problem. A complete solution would not be worth the
complexity, and as a partial solution this is probably not worth the
complexity either.

Updates #31859

Change-Id: Ifd34c292fd1b811c60afe3c339e5edd3f37190c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/186817
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Caleb Spare <cespare@gmail.com>
2019-07-22 21:42:51 +00:00
Dmitri Shuralyov bd61fc3da3 testing/internal/testdeps: delete didSetLogger variable
It was added in CL 83956 but never used.

Updates #23129

Change-Id: I70b50e974a56620069a77658386722af314cc857
Reviewed-on: https://go-review.googlesource.com/c/go/+/179138
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-28 16:07:50 +00:00
Russ Cox 06b0babf31 all: shorten some tests
Shorten some of the longest tests that run during all.bash.
Removes 7r 50u 21s from all.bash.

After this change, all.bash is under 5 minutes again on my laptop.

For #26473.

Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79
Reviewed-on: https://go-review.googlesource.com/c/go/+/177559
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-22 12:54:00 +00:00
Keith Randall 1ab063ce53 testing: callerName only needs one PC in the traceback
callerName requested 2 PCs from Callers, and that causes
both to be looked up in the file/line mapping.
We really only need to do the work for one PC.
(And in fact the caller doesn't need file/line at all, but
the Callers API can't express that.)

We used to request 2 PCs because in 1.11 and earlier we
stored an inline skip count in the second entry.
That's not necessary any more (as of 1.12).

Fixes #32093

Change-Id: I7b272626ef6496e848ee8af388cdaafd2556857b
Reviewed-on: https://go-review.googlesource.com/c/go/+/177858
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Caleb Spare <cespare@gmail.com>
2019-05-17 22:32:30 +00:00
Russ Cox f2694534cf testing: shorten go test -short testing
This cuts the time for 'go test -short testing' from 0.9s to < 0.1s.

Change-Id: Ib8402f80239e1e96ea5221dfd5cd0db08170d85b
Reviewed-on: https://go-review.googlesource.com/c/go/+/177420
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:25:08 +00:00
Russ Cox 6ab049b965 testing: panic on calls to Short/Verbose before Parse
CL 121936 added this diagnostic to avoid a panic accessing *short.
(Hence the "This shouldn't really be a panic" comment.)

That CL was right to produce a clearer error than a plain memory fault,
but I think wrong to print+exit instead of panicking. I just ran into
one of these in a real program, and there is no indication anywhere
of how the program reached this point. The panic will show that.
So change print+exit to a panic with a helpful message, in contrast
to the original panic with an unhelpful message and the current
helpful message without stack trace.

Change-Id: Ib2bae1dead4ccde92f00fa3a34c05241ff7690c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/177419
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-16 03:25:07 +00:00
Caleb Spare 49a1a01bb1 cmd/go: move automatic testing.Init call into generated test code
In CL 173722, we moved the flag registration in the testing package into
an Init function. In order to avoid needing changes to user code, we
called Init automatically as part of testing.MainStart.

However, that isn't early enough if flag.Parse is called before the
tests run, as part of package initialization.

Fix this by injecting a bit of code to call testing.Init into test
packages. This runs before any other initialization code in the user's
test package, so testing.Init will be called before any user code can
call flag.Parse.

Fixes #31859

Change-Id: Ib42cd8d3819150c49a3cecf7eef2472319d0c7e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/176098
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-10 18:10:48 +00:00
Caleb Spare fbc6a97222 testing: delay flag registration; move to an Init function
Any code that imports the testing package forces the testing flags to be
defined, even in non-test binaries. People work around this today by
defining a copy of the testing.TB interface just to avoid importing
testing.

Fix this by moving flag registration into a new function, testing.Init.
Delay calling Init until the testing binary begins to run, in
testing.MainStart.

Init is exported for cases where users need the testing flags to be
defined outside of a "go test" context. In particular, this may be
needed where testing.Benchmark is called outside of a test.

Fixes #21051

Change-Id: Ib7e02459e693c26ae1ba71bbae7d455a91118ee3
Reviewed-on: https://go-review.googlesource.com/c/go/+/173722
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-29 19:54:30 +00:00
Caleb Spare 562b7c27ec testing: fix minor bug with mutex profile error message
Change-Id: I92b7a44476cfd9e2f099477b9d7903bbaa2fd8da
Reviewed-on: https://go-review.googlesource.com/c/go/+/173720
Run-TryBot: Caleb Spare <cespare@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-24 19:48:39 +00:00
Rob Pike 02bd0fd39c testing/quick: simplify explanation of -quickchecks flag
CL 172698 documented this flag but the description was missing
punctuation and could be clearer.

Change-Id: I310d91ae8c6b947ce7d1ae7559882f49778f770a
Reviewed-on: https://go-review.googlesource.com/c/go/+/172817
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-18 21:21:21 +00:00
Andrew Z Allen e900964e0f testing/quick: clarify that Config.MaxCount is from a flag
Document that the default quickcheck configuration is to run 100 times
and that there is a flag that configures it called "quickchecks".

Change-Id: I46fdab9d572e132ccc23ef907f9cc6b2d06b37c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/172698
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-18 18:18:04 +00:00
Caleb Spare a591fd08dd testing: correct two spelling/grammar issues
Change-Id: Ic7f7a34e12cc2845e0385a5a872e694d5dca7372
Reviewed-on: https://go-review.googlesource.com/c/go/+/169158
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-25 22:26:21 +00:00
Austin Clements 1257d05088 testing: fix fractional ns/op printing
CL 166717 changed the way ns/op benchmark results were printed and
inadvertently rounded all ns/op results down to an integer, even if
they were small enough to print with digits after the decimal place.
For example, prior to this change, we got output like

    BenchmarkFast-8       380491575	 3.12 ns/op

CL 166717 changed this to

    BenchmarkFast-8       380491575      3.00 ns/op

This had the further side-effect that ns/op values between 0 and 1
would not be printed at all because they would be rounded down to 0.

This CL fixes this by always recomputing the float64 value of ns/op
instead of using the int64 truncation from BenchmarkResult.NsPerOp.

Fixes #30997. Fixes #31005.

Change-Id: I21f73b9d5cc5ad41e7ff535675d07ca00051ecd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/168937
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2019-03-22 21:19:22 +00:00
Josh Bleecher Snyder 03a79e94ac testing: stop rounding b.N
The original goal of rounding to readable b.N
was to make it easier to eyeball times.
However, proper analysis requires tooling
(such as benchstat) anyway.

Instead, take b.N as it comes.
This will reduce the impact of external noise
such as GC on benchmarks.

This requires reworking our iteration estimates.
We used to calculate the estimated ns/op
and then divide our target ns by that estimate.
However, this order of operations was destructive
when the ns/op was very small; rounding could
hide almost an order of magnitude of variation.
Instead, multiply first, then divide.
Also, make n an int64 to avoid overflow.

Prior to this change, we attempted to cap b.N at 1e9.
Due to rounding up, it was possible to get b.N as high as 2e9.
This change consistently enforces the 1e9 cap.

This change also reduces the wall time required to run benchmarks.

Here's the impact of this change on the wall time to run
all benchmarks once with benchtime=1s on some std packages:

name           old time/op       new time/op       delta
bytes                 306s ± 1%         238s ± 1%  -22.24%  (p=0.000 n=10+10)
encoding/json         112s ± 8%          99s ± 7%  -11.64%  (p=0.000 n=10+10)
net/http             54.7s ± 7%        44.9s ± 4%  -17.94%  (p=0.000 n=10+9)
runtime               957s ± 1%         714s ± 0%  -25.38%  (p=0.000 n=10+9)
strings               262s ± 1%         201s ± 1%  -23.27%  (p=0.000 n=10+10)
[Geo mean]            216s              172s       -20.23%

Updates #24735

Change-Id: I7e38efb8e23c804046bf4fc065b3f5f3991d0a15
Reviewed-on: https://go-review.googlesource.com/c/go/+/112155
Reviewed-by: Austin Clements <austin@google.com>
2019-03-20 21:19:16 +00:00
Austin Clements 834d229eb6 testing: add B.ReportMetric for custom benchmark metrics
This adds a ReportMetric method to testing.B that lets the user report
custom benchmark metrics and override built-in metrics.

Fixes #26037.

Change-Id: I8236fbde3683fc27bbe45cbbedfd377b435edf64
Reviewed-on: https://go-review.googlesource.com/c/go/+/166717
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2019-03-19 16:01:31 +00:00
Agniva De Sarker b6544a2a87 testing/cover: improve message when a package has no statements
Fixes #25492

Change-Id: Ic1496857524dad0c0a77f3bb80fa084c9bf00aa9
Reviewed-on: https://go-review.googlesource.com/c/go/+/155777
Run-TryBot: Agniva De Sarker <agniva.quicksilver@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2019-03-13 04:11:58 +00:00
Emmanuel T Odeke ac56baa09f testing: enable examples on js/wasm with non os.Pipe runExample
os.Pipe is not implemented on wasm/js so for that purpose use
a temporary file for js/wasm. This change creates two versions
of runExample:

* runExample verbatim that still uses os.Pipe for non js/wasm
* runExample that uses a temporary file

Also added a TODO to re-unify these function versions back into
example.go wasm/js gets an os.Pipe implementation.

Change-Id: I9f418a49b2c397e1667724c7442b7bbe8942225e
Reviewed-on: https://go-review.googlesource.com/c/go/+/165357
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-11 18:59:53 +00:00
Daniel Martí 340129e4c8 all: join a few chained ifs
I had been finding these over a year or so, but none were big enough
changes to warrant CLs. They're a handful now, so clean them all up in a
single commit.

The smaller bodies get a bit simpler, but most importantly, the larger
bodies get unindented.

Change-Id: I5707a6fee27d4c9ff9efd3d363af575d7a4bf2aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/165340
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-03-05 20:31:32 +00:00
Emmanuel T Odeke 6b04550820 testing: fix missing verb in StartTimer doc
Fixes StartTimer's doc with the verb 'be'
that was previously missing in 'can also used'.

Change-Id: I4b3e6103fbf62d676056d32fcce4618536b7c05c
Reviewed-on: https://go-review.googlesource.com/c/go/+/165117
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-04 23:07:12 +00:00
Leon Klingele d090429ea9 all: fix typos as reported by 'misspell'
Change-Id: I904b8655f21743189814bccf24073b6fbb9fc56d
GitHub-Last-Rev: b032c14394
GitHub-Pull-Request: golang/go#29997
Reviewed-on: https://go-review.googlesource.com/c/160421
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-26 23:02:05 +00:00
Ian Lance Taylor 006a5e7d00 testing: report the failing test in a late log panic
Updates #29388

Change-Id: Icb0e6048d05fde7a5486b923ff62147edb5c8dac
Reviewed-on: https://go-review.googlesource.com/c/157617
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2019-01-17 01:28:22 +00:00
Tobias Klauser 9e277f7d55 all: use "reports whether" consistently instead of "returns whether"
Follow-up for CL 147037 and after Brad noticed the "returns whether"
pattern during the review of CL 150621.

Go documentation style for boolean funcs is to say:

    // Foo reports whether ...
    func Foo() bool

(rather than "returns whether")

Created with:

    $ perl -i -npe 's/returns whether/reports whether/' $(git grep -l "returns whether" | grep -v vendor)

Change-Id: I15fe9ff99180ad97750cd05a10eceafdb12dc0b4
Reviewed-on: https://go-review.googlesource.com/c/150918
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-02 15:12:26 +00:00
Leigh McCulloch 8f4bc468a7 testing: add example to package doc
The package doc for the testing package doesn't have a simple
example demonstrating how to write a test with an expectation. The doc
has simple examples for benchmarks, examples, and skipping, and it would be
useful for people new to writing tests in Go.

Also moved the skip example further down because it references tests and
benchmarks but benchmarks haven't been discussed in detail until the
next section. Skip is also a less used feature and it seems misplaced to
sit so high up in the package documentation. As an example, Skip is used
570 times the Go code repository which is significantly less than Error
and Fatal that are used 23,303 times.

Also changed 'sample' to 'simple' in other places in the package documentation
to keep the language used consistent when describing the small examples.

Fixes #27839

Change-Id: Ie01a3751986ee61adf2a2f2eda59cc182342baa7
GitHub-Last-Rev: 7357bfdcd2
GitHub-Pull-Request: golang/go#27840
Reviewed-on: https://go-review.googlesource.com/c/137175
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-11-16 16:49:09 +00:00
Russ Cox 8e0aea162b testing: implement -benchtime=100x
When running benchmarks with profilers and trying to
compare one run against another, it is very useful to be
able to force each run to execute exactly the same number
of iterations.

Discussion on the proposal issue #24735 led to the decision
to overload -benchtime, so that instead of saying
-benchtime 10s to run a benchmark for 10 seconds,
you say -benchtime 100x to run a benchmark 100 times.

Fixes #24735.

Change-Id: Id17c5bd18bd09987bb48ed12420d61ae9e200fd7
Reviewed-on: https://go-review.googlesource.com/c/139258
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-12 17:48:31 +00:00
Wil Selwood 95d06ab6c9 testing: try to Log through parent if test has completed
If the test has already completed when a go routine with a panic
handler reports an error the location of the error call is lost.

Added logDepth to be able to log location of failure at different
depths down the stack.

Fixes #26720

Change-Id: I8b7789ddae757ef6f4bd315cb20356709f4fadec
Reviewed-on: https://go-review.googlesource.com/c/127596
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-10 01:02:14 +00:00
Brad Fitzpatrick da0d1a44ba all: use strings.ReplaceAll and bytes.ReplaceAll where applicable
I omitted vendor directories and anything necessary for bootstrapping.
(Tested by bootstrapping with Go 1.4)

Updates #27864

Change-Id: I7d9b68d0372d3a34dee22966cca323513ece7e8a
Reviewed-on: https://go-review.googlesource.com/137856
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-09-26 22:14:25 +00:00
Ian Lance Taylor c9b018918d testing: exit with error if testing.Short is called before flag.Parse
Change-Id: I2fa547d1074ef0931196066678fadd7250a1148d
Reviewed-on: https://go-review.googlesource.com/121936
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-08-20 21:51:11 +00:00
Rob Pike 6780042f0a testing/cover: improve comments on CoverBlock
The previous CL (https://go-review.googlesource.com/c/go/+/96756)
added comments that didn't really say much, but there is something
so say: what the units are and that they are indexed starting at 1.

Add a more helpful comment on the type, and also follow proper
style by using initial capitals and a period.

Change-Id: Id19cd5f392faf7c7bac034073f276cc770589075
Reviewed-on: https://go-review.googlesource.com/121875
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-07-01 08:07:47 +00:00
Stephen L 57da8dd459 src/testing/cover: document the CoverBlock struct fields
Fill in the missing descriptions for the CoverBlock struct fields

Change-Id: I9257881a19b01e5cfe61cf19a91375b6d7cc68ef
GitHub-Last-Rev: f5b9e1d49d
GitHub-Pull-Request: golang/go#24079
Reviewed-on: https://go-review.googlesource.com/96756
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-06-29 16:53:23 +00:00
Alberto Donizetti bfdf74be12 Revert "testing: only compute b.N once when passed -count > 1"
This reverts golang.org/cl/110775

Reason for revert: this is causing huge slow-dows on every run after
the 1st, on various benchmarks, on multiple architectures (see Issue
25622 for details). It's just a nice-to-have little optimization, and
we're near the 1st go1.11 beta release, so I'm reverting it.

Fixes #25622

Change-Id: I758ade4af4abf764abd8336d404396992d11a0c6
Reviewed-on: https://go-review.googlesource.com/115535
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2018-05-31 14:22:44 +00:00
Yury Smolsky fffb3a5c20 testing: make indentation consistent in sub-tests
Instead of mixed usage of spaces and tabs for indentation,
just use 4 spaces instead of one tab.

This test:

func TestX(t *testing.T) {
	t.Error("1\nnew line")
	t.Error("2")
	t.Error("3")
	t.Run("Y", func(t *testing.T) {
		t.Error("2")
		t.Error("2b\nnew line")
		t.Run("Z", func(t *testing.T) {
			t.Error("3\nnew line")
		})
	})
	t.Error("4")
}

produces following output:

--- FAIL: TestX (0.00s)
    indent_test.go:6: 1
	new line
    indent_test.go:7: 2
    indent_test.go:8: 3
    --- FAIL: TestX/Y (0.00s)
	indent_test.go:10: 2
	indent_test.go:11: 2b
	    new line
	--- FAIL: TestX/Y/Z (0.00s)
	    indent_test.go:13: 3
		new line
    indent_test.go:16: 4
FAIL

Fixes #25369

Change-Id: Ib3b5da45ab3ee670c6e8a23172e7cbefb94c5e60
Reviewed-on: https://go-review.googlesource.com/113177
Run-TryBot: Yury Smolsky <yury@smolsky.by>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2018-05-31 07:20:38 +00:00
Diogo Pinela 15f2cbf437 testing: allow marking subtest and subbenchmark functions as Helpers
Since subtests and subbenchmarks run in a separate goroutine, and thus
a separate stack, this entails capturing the stack trace at the point
tb.Run is called. The work of getting the file and line information from
this stack is only done when needed, however.

Continuing the search into the parent test also requires temporarily
holding its mutex. Since Run does not hold it while waiting for the
subtest to complete, there should be no risk of a deadlock due to this.

Fixes #24128

Change-Id: If0bb169f3ac96bd48794624e619ade7edb599f83
Reviewed-on: https://go-review.googlesource.com/108658
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2018-05-14 17:59:59 +00:00
Alberto Donizetti 05ca34093b testing: only compute b.N once when passed -count > 1
When running a benchmark multiple times, instead of re-computing the
value of b.N each time, use the value found by the first run.

For

  go test -bench=. -benchtime 3s -count 2 p_test.go

on the benchmark in the linked issue; before:

  BenchmarkBenchmark-4   	     500	  10180593 ns/op
  --- BENCH: BenchmarkBenchmark-4
  	  p_test.go:13: single call took 10.111079ms
  	  p_test.go:13: single call took 1.017298685s
  	  p_test.go:13: single call took 5.090096124s
  BenchmarkBenchmark-4   	     500	  10182164 ns/op
  --- BENCH: BenchmarkBenchmark-4
  	  p_test.go:13: single call took 10.098169ms
  	  p_test.go:13: single call took 1.017712905s
  	  p_test.go:13: single call took 5.090898517s
  PASS
  ok  	command-line-arguments	12.244s

and after:

  BenchmarkBenchmark-4   	     500	  10177076 ns/op
  --- BENCH: BenchmarkBenchmark-4
  	  p_test.go:13: single call took 10.091301ms
  	  p_test.go:13: single call took 1.016943125s
  	  p_test.go:13: single call took 5.088376028s
  BenchmarkBenchmark-4   	     500	  10171497 ns/op
  --- BENCH: BenchmarkBenchmark-4
  	  p_test.go:13: single call took 10.140245ms
  	  p_test.go:13: single call took 5.085605921s
  PASS
  ok  	command-line-arguments	11.218s

Fixes #23423

Change-Id: Ie66a8c5ac43881eb8741e14105db28745b4d56d3
Reviewed-on: https://go-review.googlesource.com/110775
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-14 17:07:43 +00:00
Anmol Sethi eff1e68528 testing: fix racey access to t.failed
We need to grab the mutex before we can access it.

Fixes #24438

Change-Id: Idd6130036691acec5bc5f8b40d6884f8db1d9d3c
Reviewed-on: https://go-review.googlesource.com/101283
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-05-02 17:51:26 +00:00