Commit Graph

488 Commits

Author SHA1 Message Date
Damien Neil 49cdf0c42e testing, testing/synctest: handle T.Helper in synctest bubbles
Fixes #74199

Change-Id: I6a15fbd59a3a3f8c496440f56d09d695e1504e4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/682576
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-06-20 12:29:58 -07:00
Damien Neil 4f86f22671 testing/synctest, runtime: avoid panic when using linker-alloc WG from bubble
We associate WaitGroups with synctest bubbles by attaching a
special to the WaitGroup. It is not possible to attach a special
to a linker-allocated value, such as:

    var wg sync.WaitGroup

Avoid panicking when accessing a linker-allocated WaitGroup
from a bubble. We have no way to associate these WaitGroups
with a bubble, so just treat them as always unbubbled.
This is probably fine, since the WaitGroup was always
created outside the bubble in this case.

Fixes #74005

Change-Id: Ic71514b0b8d0cecd62e45cc929ffcbeb16f54a55
Reviewed-on: https://go-review.googlesource.com/c/go/+/679695
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-06-10 15:02:26 -07:00
Mateusz Poliwczak 711ff943af testing: add Output method to TB
Updates #59928
Fixes #73937

Change-Id: Ibf7ec61758edccd245841c3acc9096563b44fcd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/677875
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2025-06-02 09:09:57 -07:00
Damien Neil 8cd7f17248 testing, testing/synctest: report correct duration after panics
Report the correct wall-clock test duration after handling a
panic in a synctest.Test bubble.

Fixes #73852

Change-Id: I053262e5eac2dd9d5938b17c3093cbc3fa115a0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/676695
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-05-30 09:23:37 -07:00
Damien Neil 21b7e60c6b runtime, testing/synctest: breaking bubble isolation with Cond is fatal
sync.Cond.Wait is durably blocking. Waking a goroutine out of Cond.Wait
from outside its bubble panics.

Make this panic a fatal panic, since it leaves the notifyList in an
inconsistent state. We could do some work to make this a recoverable
panic, but the complexity doesn't seem worth the outcome.

For #67434

Change-Id: I88874c1519c2e5c0063175297a9b120cedabcd07
Reviewed-on: https://go-review.googlesource.com/c/go/+/675617
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-05-29 12:49:36 -07:00
Damien Neil 555d425d17 testing, testing/synctest: write bubble errors to parent test log
Ensure that log messages written to the testing.T created by
synctest.Test appear in the test output when a test fails.

Fixes #73902

Change-Id: Ie97f5efe54eb003e6c0a5394c2def4cac1520ecb
Reviewed-on: https://go-review.googlesource.com/c/go/+/676995
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-05-29 12:46:41 -07:00
Damien Neil b170c7e94c runtime, internal/synctest, sync: associate WaitGroups with bubbles
Add support to internal/synctest for managing associations between
arbitrary pointers and synctest bubbles. (Implemented internally to
the runtime package by attaching a special to the pointer.)

Associate WaitGroups with bubbles.
Since WaitGroups don't have a constructor,
perform the association when Add is called.
All Add calls must be made from within the same bubble,
or outside any bubble.

When a bubbled goroutine calls WaitGroup.Wait,
the wait is durably blocking iff the WaitGroup is associated
with the current bubble.

Change-Id: I77e2701e734ac2fa2b32b28d5b0c853b7b2825c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/676656
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-05-29 10:26:00 -07:00
Damien Neil fce9d4515d runtime, testing/synctest: verify cleanups/finalizers run outside bubbles
Cleanup functions and finalizers must not run in a synctest bubble.
If they did, a function run by the GC at an unpredictable time
could unblock a bubble that synctest believes is durably
blocked.

Add a test verifying that cleanups and finalizers are always
run by non-bubbled goroutines. (This is already the case because
we never add system goroutines to a bubble.)

For #67434

Change-Id: I5a48db2b26f9712c3b0dc1f425d99814031a2fc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/675257
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-05-27 14:47:13 -07:00
Sean Liao c07ffe980a testing/synctest: correct duration in doc example
Fixes #73839

Change-Id: I961641c6d8244cdeb101a3c9ae91931828a893ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/676035
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-05-24 08:42:04 -07:00
Damien Neil 3cc8b532f9 testing: add Attr
Add a new Attr method to testing.TB that emits a test attribute.
An attribute is an arbitrary key/value pair.

Fixes #43936

Change-Id: I7ef299efae41f2cf39f2dc61ad4cdd4c3975cdb6
Reviewed-on: https://go-review.googlesource.com/c/go/+/662437
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-05-21 15:37:35 -07:00
Damien Neil 49a660e22c testing/synctest: add Test
Add a synctest.Test function, superseding the experimental
synctest.Run function. Promote the testing/synctest package
out of experimental status.

For #67434
For #73567

Change-Id: I3c5ba030860d90fe2ddb517a2f3536efd60181a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/671961
Auto-Submit: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2025-05-20 15:46:03 -07:00
Mateusz Poliwczak 4f1146e661 testing: use a pattern to match the elapsed time in TestTRun
Fixes #73723
Fixes #73737
Fixes #73739

Change-Id: I1ebd3614614285c3e660d48241389bb0f896be23
Reviewed-on: https://go-review.googlesource.com/c/go/+/674355
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-05-20 07:37:42 -07:00
Jonathan Amsterdam 6df855ebac testing: fix panic in t.Log
If a testing.TB is no longer on the stack, t.Log would panic because
its outputWriter is nil. Check for nil and drop the write, which
is the previous behavior.

Change-Id: Ifde97997a3aa26ae604ac9c218588c1980110cbf
Reviewed-on: https://go-review.googlesource.com/c/go/+/673215
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
2025-05-15 10:24:18 -07:00
suntala 9e026bf9cc testing: add Output
Output is a method on T, B and F. It provides an io.Writer that writes
to the same test output stream as TB.Log. The new output writer is
used to refactor the implementation of Log. It maintains the formatting
provided by Log while making call site information optional.
Additionally, it provides buffering of log messages. This fixes and
expands on
https://go-review.googlesource.com/c/go/+/646956.

For #59928.

Change-Id: I08179c35a681f601cf125c0f4aeb648bc10c7a9f
GitHub-Last-Rev: e6e202793c
GitHub-Pull-Request: golang/go#73703
Reviewed-on: https://go-review.googlesource.com/c/go/+/672395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
2025-05-13 15:43:42 -07:00
Jonathan Amsterdam afd6b0bc66 Revert "testing: add Output"
This reverts commit 8d189f188e.

Reason for revert: failing test

Change-Id: I951087eaef7818697acf87e3206003bcc8a81ee2
Reviewed-on: https://go-review.googlesource.com/c/go/+/672335
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-05-13 12:26:50 -07:00
suntala 8d189f188e testing: add Output
Output is a method on T, B and F. It provides an io.Writer that writes
to the same test output stream as TB.Log. The new output writer is
used to refactor the implementation of Log. It maintains the formatting
provided by Log while making call site information optional.
Additionally, it provides buffering of log messages.

Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com>

Fixes #59928.

Change-Id: I29090b3d4f61f7334388b373ec18750d5637aafa
GitHub-Last-Rev: 18af0e1526
GitHub-Pull-Request: golang/go#71575
Reviewed-on: https://go-review.googlesource.com/c/go/+/646956
Reviewed-by: Arati <artichaut2023@gmail.com>
Auto-Submit: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-05-13 08:06:08 -07:00
Sean Liao ac992f2614 testing: limit TempDir name length
Fixes #71742

Change-Id: Ibef8f7f0a36b25f181062c4d2f84279a97e467a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/671577
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2025-05-12 12:34:50 -07:00
Damien Neil d6c8bedc7b runtime, testing/synctest: stop advancing time when main goroutine exits
Once the goroutine started by synctest.Run exits, stop advancing
the fake clock in its bubble. This avoids confusing situations
where a bubble remains alive indefinitely while a background
goroutine reads from a time.Ticker or otherwise advances the clock.

For #67434

Change-Id: Id608ffe3c7d7b07747b56a21f365787fb9a057d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/662155
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-05-07 13:14:25 -07:00
Sean Liao 8270b858ee testing: use more doc links
Change-Id: Ide372735165b7510fd8d7588451a37fa743e59c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/668915
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
2025-04-30 08:21:07 -07:00
thepudds ab2926291b testing: clarify how B.Loop avoids optimizing away all the useful work
As discussed in #73137, we want to clarify the description of how
B.Loop avoids surprising optimizations, while also hinting that
the exact approach might change in the future.

Updates #73137

Change-Id: I8536540cd5d79804a47fba8cd6eec3821864309d
Reviewed-on: https://go-review.googlesource.com/c/go/+/662356
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-04-03 12:24:34 -07:00
junya koyama 99d97d7c4f testing/slogtest: test nested groups in empty record
Updates #62280

Change-Id: I1c80cb18bb174b47ff156974f72c37baf6b73635
GitHub-Last-Rev: d98b6cd57e
GitHub-Pull-Request: golang/go#65597
Reviewed-on: https://go-review.googlesource.com/c/go/+/562635
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sean Liao <sean@liao.dev>
2025-03-25 19:19:09 -07:00
Sean Liao 2bf57a3534 testing: document exit codes
Fixes #25989

Change-Id: I2e2a2a17854034ff68e69b8973018b1b2e7d59f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/660076
Reviewed-by: Thanyalak Detkhong (Palm’my) <pmy4416@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-25 11:52:07 -07:00
Austin Clements aaf9b46800 testing: streamline logic in loopSlowPath
There's a fair amount of duplication of logic between various return
branches of loopSlowPath and stopOrScaleBLoop. Restructure these so
there's a single "keep going" path and a single "we're done" path.

Change-Id: I38e4c7a616f8bd7707f3ca886f38ff21dbd78b6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/659658
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2025-03-24 11:41:13 -07:00
Austin Clements 5918101d67 testing: detect a stopped timer in B.Loop
Currently, if the user stops the timer in a B.Loop benchmark loop, the
benchmark will run until it hits the timeout and fails.

Fix this by detecting that the timer is stopped and failing the
benchmark right away. We avoid making the fast path more expensive for
this check by "poisoning" the B.Loop iteration counter when the timer
is stopped so that it falls back to the slow path, which can check the
timer.

This causes b to escape from B.Loop, which is totally harmless because
it was already definitely heap-allocated. But it causes the
test/inline_testingbloop.go errorcheck test to fail. I don't think the
escape messages actually mattered to that test, they just had to be
matched. To fix this, we drop the debug level to -m=1, since -m=2
prints a lot of extra information for escaping parameters that we
don't want to deal with, and change one error check to allow b to
escape.

Fixes #72971.

Change-Id: I7d4abbb1ec1e096685514536f91ba0d581cca6b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/659657
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-24 11:41:09 -07:00
Austin Clements b7f8c2a110 testing: detect early return from B.Loop
Currently, if a benchmark function returns prior to B.Loop() returning
false, we'll report a bogus result. While there was no way to detect
this with b.N-style benchmarks, one way b.Loop()-style benchmarks are
more robust is that we *can* detect it.

This CL adds a flag to B that tracks if B.Loop() has finished and
checks it after the benchmark completes. If there was an early exit
(not caused by another error), it reports a B.Error.

Fixes #72933.
Updates #72971.

Change-Id: I731c1350e6df938c0ffa08fcedc11dc147e78854
Reviewed-on: https://go-review.googlesource.com/c/go/+/659656
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-03-24 11:41:05 -07:00
Jake Bailey 710476495a testing: remove redundant variable capture from subtest example
Loop variable capturing hasn't been needed like this since Go 1.21;
remove it from the example.

Change-Id: I231dcfdb57832e32d524f156a605ba36d1c9d6d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/660176
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-03-24 10:16:54 -07:00
Austin Clements 9003154206 testing: separate b.Loop counter from b.N
Currently, b.Loop uses b.N as the iteration count target. However,
since it updates the target as it goes, the behavior is quite
different from a b.N-style benchmark. To avoid user confusion, this CL
gives b.Loop a separate, unexported iteration count target. It ensures
b.N is 0 within the b.Loop loop to help catch misuses, and commits the
final iteration count to b.N only once the loop is done (as the
documentation states "After Loop returns false, b.N contains the total
number of iterations that ran, so the benchmark may use b.N to compute
other average metrics.")

Since there are now two variables used by b.Loop, we put them in an
unnamed struct. Also, we rename b.loopN to b.loop.i because this
variable tracks the current iteration index (conventionally "i"), not
the target (conventionally "n").

Unfortunately, a simple renaming causes B.Loop to be too large for the
inliner. Thus, we make one simplification to B.Loop to keep it under
the threshold. We're about to lean into that simplification anyway in
a follow-up CL, so this is just temporary.

Prep for #72933 and #72971.

Change-Id: Ide1c4f1b9ca37f300f3beb0e60ba6202331b183e
Reviewed-on: https://go-review.googlesource.com/c/go/+/659655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Austin Clements <austin@google.com>
2025-03-21 16:12:09 -07:00
Junyang Shao b613d21ffd testing: allow manual timer control in testing.B.Loop
Fixes #72922

Change-Id: I56610d2d11d151a8f95b6434bbedbfcd5c11c317
Reviewed-on: https://go-review.googlesource.com/c/go/+/658975
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2025-03-20 12:42:51 -07:00
Sean Liao 2abe5ceb19 testing: warn against calling Log after a test completes
Fixes #40343

Change-Id: Id266f4b57131e9e148e5aa2be86b67fe6d73b20a
Reviewed-on: https://go-review.googlesource.com/c/go/+/656415
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-03-10 15:02:26 -07:00
Jes Cok 8abb8aa9a9 testing: modify got,want equal comparison for unordered example output
This change eliminates sortLines function to avoid strings.Join calls.

It's not a performance problem, this change tries to make the comparison
more straightforward.

Change-Id: I3a7ae877c9fc927833ab9f143205f7e007197f60
GitHub-Last-Rev: a71aa58c58
GitHub-Pull-Request: golang/go#72025
Reviewed-on: https://go-review.googlesource.com/c/go/+/653556
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-03-05 07:44:41 -08:00
Junyang Shao f48b53f0f6 testing: fix testing.B.Loop doc on loop condition
As mentioned by
https://github.com/golang/go/issues/61515#issuecomment-2656656554,
the documentation should be relaxed.

Change-Id: I9f18301e1a4e4d9a72c9fa0b1132b1ba3cc57b03
Reviewed-on: https://go-review.googlesource.com/c/go/+/651435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
2025-03-03 10:37:29 -08:00
qmuntal dceee2e983 all: use testenv.Executable instead of os.Executable and os.Args[0]
In test files, using testenv.Executable is more reliable than
os.Executable or os.Args[0].

Change-Id: I88e577efeabc20d02ada27bf706ae4523129128e
Reviewed-on: https://go-review.googlesource.com/c/go/+/651955
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-02-24 11:03:10 -08:00
Russ Cox baeab452d1 testing: panic in AllocsPerRun if parallel tests are running
If other tests are running, AllocsPerRun's result will be inherently flaky.
Saw this with CL 630136 and #70327.

Proposed in #70464.

Fixes #70464.

Change-Id: I190afdf26bc31299f6e5e8665b4fb420ffd554ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/630137
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-02-14 06:09:08 -08:00
Damien Neil cb47156e90 testing/synctest: add an example of testing networked code
For #67434

Change-Id: If7dcd3bf7bb69e1730856405d55cffc72ce0e132
Reviewed-on: https://go-review.googlesource.com/c/go/+/645675
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-02-07 09:32:22 -08:00
apocelipes 4cc7705e56 testing: use strings.SplitSeq and bytes.SplitSeq
To simplify the code. This is a follow-up for the CL 646216.

Change-Id: Ib09d1074a783482fb293527e9f1abeb3c02137c3
GitHub-Last-Rev: 2e7a6ad40c
GitHub-Pull-Request: golang/go#71568
Reviewed-on: https://go-review.googlesource.com/c/go/+/646755
Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Salah (Globlost) <globlost@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-02-05 15:53:08 -08:00
Ian Lance Taylor 41298239cf all: remove coverageredesign experiment
The coverageredesign experiment was turned on by default by
CL 436236 in September, 2022. We've documented it and people
are using it. This CL removes the ability to turn off the experiment.
This removes some old code that is no longer being executed.

For #51430

Change-Id: I88d4998c8b5ea98eef8145d7ca6ebd96f64fbc2b
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-darwin-amd64-longtest,gotip-linux-arm64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/644997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
2025-02-03 12:10:28 -08:00
Roxy Light f7b8dd9033 io/fs: add ReadLinkFS interface
Added implementations for *io/fs.subFS, os.DirFS, and testing/fstest.MapFS.
Amended testing/fstest.TestFS to check behavior.

Addressed TODOs in archive/tar and os.CopyFS around symbolic links.

I am deliberately not changing archive/zip in this CL,
since it currently does not resolve symlinks
as part of its filesystem implementation.
I am unsure of the compatibility restrictions on doing so,
so figured it would be better to address independently.

testing/fstest.MapFS now includes resolution of symlinks,
with MapFile.Data storing the symlink data.
The behavior change there seemed less intrusive,
especially given its intended usage in tests,
and it is especially helpful in testing the io/fs function implementations.

Fixes #49580

Change-Id: I58ec6915e8cc97341cdbfd9c24c67d1b60139447
Reviewed-on: https://go-review.googlesource.com/c/go/+/385534
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Funda Secgin <fundasecgin33@gmail.com>
2025-02-03 08:38:43 -08:00
Kir Kolyshkin 3aa7c5ef01 testing: fix reference to B.N in docstring
Currently, a documentation reference to a struct member (such as [B.N])
does not result in it being rendered as a link, and thus the square
brackets remain in the rendered documentation which is mildly confusing.

The issue can be seen at
https://pkg.go.dev/testing@master#hdr-b_N_style_benchmarks

Remove the square brackets to fix.

Change-Id: Id374fd2085bd511018220c5d663650f89672302e
Reviewed-on: https://go-review.googlesource.com/c/go/+/643496
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-01-21 04:36:54 -08:00
cuishuang befc43655b testing/fstest: fix function name and comment
Change-Id: I17bc68a2e3a96d0dc3d9ddcad40149df37fc4839
Reviewed-on: https://go-review.googlesource.com/c/go/+/642198
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-01-14 09:43:06 -08:00
Damien Neil 39f2032c17 testing/synctest: add some examples
For #67434

Change-Id: Iebcfc2559a62405fea7e3ceff8cf6c2f50b61def
Reviewed-on: https://go-review.googlesource.com/c/go/+/641176
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-01-07 20:05:28 -08:00
Ian Lance Taylor 971448ddf8 testing: support B.Context and F.Context
CL 603959 added T.Context for #36532.

The discussion on the proposal only mentions t.Context.
However, the implementation of CL 603959 also added B.Context and F.Context.
They were added to the API listing, and B.Context was mentioned in
the release notes.

Unfortunately, the new B.Context and F.Context methods always
returned nil, rather than a context.Context value.

This change adds a working implementation of B.Context and F.Context.

For #36532
Fixes #70866

Change-Id: I8a44e6649fb658e4f641ffb7efd08b4374f578ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/637236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-12-18 08:18:04 -08:00
Austin Clements 18b5435fc8 testing: don't measure cleanup time after B.Loop
B.Loop resets the timer on the first iteration so that setup code
isn't measured, but it currently leaves the timer running after the
last iteration, meaning that cleanup code will still be measured. Fix
this by stopping the timer when B.Loop returns false to indicate the
end of the benchmark.

Updates #61515

Change-Id: I0e0502cb2ce3c24cf872682b88d74e8be2c4529b
Reviewed-on: https://go-review.googlesource.com/c/go/+/635898
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-12-15 21:41:28 -08:00
Austin Clements c1f2542c8b testing: improve B.Loop test
This moves the B.Loop test from package testing_test to package
testing, where it can check on more of the internals of the benchmark
state.

Updates #61515.

Change-Id: Ia32d7104526125c5e8a1e35dab7660008afcbf80
Reviewed-on: https://go-review.googlesource.com/c/go/+/635897
Auto-Submit: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
2024-12-15 21:41:26 -08:00
Austin Clements 6bd56fcaeb testing: improve b.Loop example
The current b.Loop example doesn't focus on the basic usage of b.Loop.
Replace this with a new example that uses (slightly) more realistic
things to demonstrate the most salient points of b.Loop.

We also move the example into an example file so that we can write a
real Benchmark function and a real function to be benchmarks, which
makes this much closer to what a user would actually write.

Updates #61515.

Change-Id: I4d830b3bfe3eb3cd8cdecef469fea0541baebb43
Reviewed-on: https://go-review.googlesource.com/c/go/+/635896
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-12-15 21:41:23 -08:00
Austin Clements 090748d6c7 testing: improve B.Loop docs, use B.Loop in examples
This updates the testing documentation to frame B.Loop as the
canonical way to write benchmarks. We retain documentation on b.N
benchmarks because people will definitely continue to see them (and
write them), but it's demoted to clearly second class.

This also attempts to clarify and refine the B.Loop documentation
itself.

Updates #61515
Fixes #70787

Change-Id: If5123435bfe3a5883a753119ecdf7bbc41afd499
Reviewed-on: https://go-review.googlesource.com/c/go/+/635895
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Caleb Spare <cespare@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
2024-12-15 21:41:19 -08:00
Jorropo d3d4e7b0d1 testing: fix divide by zero when benchmark is faster than OS's clock resolution
I did not added a test because `benchmark_test.go` is `package testing_test`
and I don't care to change that because calling predictN is not testing the
thing I would want to test.

Ideally we would run benchmark in a VM with a highjacked clocksource that never
marches forward, or using faketime but that looks fairly involved for a quickie
fix.

Fixes #70709

Change-Id: I8b4d697aff7cba33da388cb0ae8e2c2b550b9690
Reviewed-on: https://go-review.googlesource.com/c/go/+/633419
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Jorropo <jorropo.pgm@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2024-12-06 06:25:44 +00:00
Junyang Shao 5213e1e56c testing: improve documentation, examples, release notes for
testing.b.Loop.

This CL added documentation of the no-inlining semantic of b.Loop, with
a concrete example. This CL also tries to improve the release note to be
more descriptive.

Fixes #61515

Change-Id: I1e13cc92d5d6bdbf40fb44f44475e249747b807f
Reviewed-on: https://go-review.googlesource.com/c/go/+/633536
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
2024-12-05 21:03:13 +00:00
Dmitri Shuralyov 3374abeb50 testing: consider -N suffix after benchmark name optional
A "-N" suffix is left out when GOMAXPROCS is 1.

Also match at least 1 space (\s+ instead of \s*), remove trailing '.*'
(it's a no-op), and make the test error message style more consistent
while here.

For #61515.
Fixes #70627.

Change-Id: Id0a17478ac31e2934a663dd0d3b1b37f24974989
Cq-Include-Trybots: luci.golang.try:gotip-plan9-386
Reviewed-on: https://go-review.googlesource.com/c/go/+/632655
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-12-02 18:59:26 +00:00
Junyang Shao 154fb4e1d4 testing: Update testing.B.Loop to save benchmark results.
This is fixing some the missing logic of CL 627755.

Change-Id: I35acffef7299331fce21da4a80a26185b2e909f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/630455
Commit-Queue: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-21 22:09:32 +00:00
Junyang Shao 558f5372fc cmd/compile,testing: implement one-time rampup logic for testing.B.Loop
testing.B.Loop now does its own loop scheduling without interaction with b.N.
b.N will be updated to the actual iterations b.Loop controls when b.Loop returns false.

This CL also added tests for fixed iteration count (benchtime=100x case).

This CL also ensured that b.Loop() is inlined.

For #61515

Change-Id: Ia15f4462f4830ef4ec51327520ff59910eb4bb58
Reviewed-on: https://go-review.googlesource.com/c/go/+/627755
Reviewed-by: Michael Pratt <mpratt@google.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-11-20 23:19:48 +00:00