Commit Graph

473 Commits

Author SHA1 Message Date
suntala 7ecfb1a2ea Remove call site from outputWriter 2025-04-10 09:39:41 +02:00
suntala a233f1675a Fix output indentation 2025-04-10 09:39:41 +02:00
suntala dfd2396755 Refactor output writer implementation 2025-04-10 09:39:41 +02:00
suntala cf4506e003 Add a new output writer and use it in t.Log
Co-authored-by: Aleks Fazlieva <britishrum@users.noreply.github.com>
2025-04-10 09:39:41 +02:00
thepudds ab2926291b testing: clarify how B.Loop avoids optimizing away all the useful work
As discussed in #73137, we want to clarify the description of how
B.Loop avoids surprising optimizations, while also hinting that
the exact approach might change in the future.

Updates #73137

Change-Id: I8536540cd5d79804a47fba8cd6eec3821864309d
Reviewed-on: https://go-review.googlesource.com/c/go/+/662356
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-04-03 12:24:34 -07:00
junya koyama 99d97d7c4f testing/slogtest: test nested groups in empty record
Updates #62280

Change-Id: I1c80cb18bb174b47ff156974f72c37baf6b73635
GitHub-Last-Rev: d98b6cd57e
GitHub-Pull-Request: golang/go#65597
Reviewed-on: https://go-review.googlesource.com/c/go/+/562635
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Sean Liao <sean@liao.dev>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Sean Liao <sean@liao.dev>
2025-03-25 19:19:09 -07:00
Sean Liao 2bf57a3534 testing: document exit codes
Fixes #25989

Change-Id: I2e2a2a17854034ff68e69b8973018b1b2e7d59f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/660076
Reviewed-by: Thanyalak Detkhong (Palm’my) <pmy4416@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-25 11:52:07 -07:00
Austin Clements aaf9b46800 testing: streamline logic in loopSlowPath
There's a fair amount of duplication of logic between various return
branches of loopSlowPath and stopOrScaleBLoop. Restructure these so
there's a single "keep going" path and a single "we're done" path.

Change-Id: I38e4c7a616f8bd7707f3ca886f38ff21dbd78b6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/659658
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2025-03-24 11:41:13 -07:00
Austin Clements 5918101d67 testing: detect a stopped timer in B.Loop
Currently, if the user stops the timer in a B.Loop benchmark loop, the
benchmark will run until it hits the timeout and fails.

Fix this by detecting that the timer is stopped and failing the
benchmark right away. We avoid making the fast path more expensive for
this check by "poisoning" the B.Loop iteration counter when the timer
is stopped so that it falls back to the slow path, which can check the
timer.

This causes b to escape from B.Loop, which is totally harmless because
it was already definitely heap-allocated. But it causes the
test/inline_testingbloop.go errorcheck test to fail. I don't think the
escape messages actually mattered to that test, they just had to be
matched. To fix this, we drop the debug level to -m=1, since -m=2
prints a lot of extra information for escaping parameters that we
don't want to deal with, and change one error check to allow b to
escape.

Fixes #72971.

Change-Id: I7d4abbb1ec1e096685514536f91ba0d581cca6b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/659657
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2025-03-24 11:41:09 -07:00
Austin Clements b7f8c2a110 testing: detect early return from B.Loop
Currently, if a benchmark function returns prior to B.Loop() returning
false, we'll report a bogus result. While there was no way to detect
this with b.N-style benchmarks, one way b.Loop()-style benchmarks are
more robust is that we *can* detect it.

This CL adds a flag to B that tracks if B.Loop() has finished and
checks it after the benchmark completes. If there was an early exit
(not caused by another error), it reports a B.Error.

Fixes #72933.
Updates #72971.

Change-Id: I731c1350e6df938c0ffa08fcedc11dc147e78854
Reviewed-on: https://go-review.googlesource.com/c/go/+/659656
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
2025-03-24 11:41:05 -07:00
Jake Bailey 710476495a testing: remove redundant variable capture from subtest example
Loop variable capturing hasn't been needed like this since Go 1.21;
remove it from the example.

Change-Id: I231dcfdb57832e32d524f156a605ba36d1c9d6d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/660176
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2025-03-24 10:16:54 -07:00
Austin Clements 9003154206 testing: separate b.Loop counter from b.N
Currently, b.Loop uses b.N as the iteration count target. However,
since it updates the target as it goes, the behavior is quite
different from a b.N-style benchmark. To avoid user confusion, this CL
gives b.Loop a separate, unexported iteration count target. It ensures
b.N is 0 within the b.Loop loop to help catch misuses, and commits the
final iteration count to b.N only once the loop is done (as the
documentation states "After Loop returns false, b.N contains the total
number of iterations that ran, so the benchmark may use b.N to compute
other average metrics.")

Since there are now two variables used by b.Loop, we put them in an
unnamed struct. Also, we rename b.loopN to b.loop.i because this
variable tracks the current iteration index (conventionally "i"), not
the target (conventionally "n").

Unfortunately, a simple renaming causes B.Loop to be too large for the
inliner. Thus, we make one simplification to B.Loop to keep it under
the threshold. We're about to lean into that simplification anyway in
a follow-up CL, so this is just temporary.

Prep for #72933 and #72971.

Change-Id: Ide1c4f1b9ca37f300f3beb0e60ba6202331b183e
Reviewed-on: https://go-review.googlesource.com/c/go/+/659655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Austin Clements <austin@google.com>
2025-03-21 16:12:09 -07:00
Junyang Shao b613d21ffd testing: allow manual timer control in testing.B.Loop
Fixes #72922

Change-Id: I56610d2d11d151a8f95b6434bbedbfcd5c11c317
Reviewed-on: https://go-review.googlesource.com/c/go/+/658975
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2025-03-20 12:42:51 -07:00
Sean Liao 2abe5ceb19 testing: warn against calling Log after a test completes
Fixes #40343

Change-Id: Id266f4b57131e9e148e5aa2be86b67fe6d73b20a
Reviewed-on: https://go-review.googlesource.com/c/go/+/656415
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-03-10 15:02:26 -07:00
Jes Cok 8abb8aa9a9 testing: modify got,want equal comparison for unordered example output
This change eliminates sortLines function to avoid strings.Join calls.

It's not a performance problem, this change tries to make the comparison
more straightforward.

Change-Id: I3a7ae877c9fc927833ab9f143205f7e007197f60
GitHub-Last-Rev: a71aa58c58
GitHub-Pull-Request: golang/go#72025
Reviewed-on: https://go-review.googlesource.com/c/go/+/653556
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-03-05 07:44:41 -08:00
Junyang Shao f48b53f0f6 testing: fix testing.B.Loop doc on loop condition
As mentioned by
https://github.com/golang/go/issues/61515#issuecomment-2656656554,
the documentation should be relaxed.

Change-Id: I9f18301e1a4e4d9a72c9fa0b1132b1ba3cc57b03
Reviewed-on: https://go-review.googlesource.com/c/go/+/651435
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
2025-03-03 10:37:29 -08:00
qmuntal dceee2e983 all: use testenv.Executable instead of os.Executable and os.Args[0]
In test files, using testenv.Executable is more reliable than
os.Executable or os.Args[0].

Change-Id: I88e577efeabc20d02ada27bf706ae4523129128e
Reviewed-on: https://go-review.googlesource.com/c/go/+/651955
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-02-24 11:03:10 -08:00
Russ Cox baeab452d1 testing: panic in AllocsPerRun if parallel tests are running
If other tests are running, AllocsPerRun's result will be inherently flaky.
Saw this with CL 630136 and #70327.

Proposed in #70464.

Fixes #70464.

Change-Id: I190afdf26bc31299f6e5e8665b4fb420ffd554ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/630137
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2025-02-14 06:09:08 -08:00
Damien Neil cb47156e90 testing/synctest: add an example of testing networked code
For #67434

Change-Id: If7dcd3bf7bb69e1730856405d55cffc72ce0e132
Reviewed-on: https://go-review.googlesource.com/c/go/+/645675
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-02-07 09:32:22 -08:00
apocelipes 4cc7705e56 testing: use strings.SplitSeq and bytes.SplitSeq
To simplify the code. This is a follow-up for the CL 646216.

Change-Id: Ib09d1074a783482fb293527e9f1abeb3c02137c3
GitHub-Last-Rev: 2e7a6ad40c
GitHub-Pull-Request: golang/go#71568
Reviewed-on: https://go-review.googlesource.com/c/go/+/646755
Reviewed-by: Jorropo <jorropo.pgm@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Salah (Globlost) <globlost@gmail.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-02-05 15:53:08 -08:00
Ian Lance Taylor 41298239cf all: remove coverageredesign experiment
The coverageredesign experiment was turned on by default by
CL 436236 in September, 2022. We've documented it and people
are using it. This CL removes the ability to turn off the experiment.
This removes some old code that is no longer being executed.

For #51430

Change-Id: I88d4998c8b5ea98eef8145d7ca6ebd96f64fbc2b
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-darwin-amd64-longtest,gotip-linux-arm64-longtest,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/644997
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
2025-02-03 12:10:28 -08:00
Roxy Light f7b8dd9033 io/fs: add ReadLinkFS interface
Added implementations for *io/fs.subFS, os.DirFS, and testing/fstest.MapFS.
Amended testing/fstest.TestFS to check behavior.

Addressed TODOs in archive/tar and os.CopyFS around symbolic links.

I am deliberately not changing archive/zip in this CL,
since it currently does not resolve symlinks
as part of its filesystem implementation.
I am unsure of the compatibility restrictions on doing so,
so figured it would be better to address independently.

testing/fstest.MapFS now includes resolution of symlinks,
with MapFile.Data storing the symlink data.
The behavior change there seemed less intrusive,
especially given its intended usage in tests,
and it is especially helpful in testing the io/fs function implementations.

Fixes #49580

Change-Id: I58ec6915e8cc97341cdbfd9c24c67d1b60139447
Reviewed-on: https://go-review.googlesource.com/c/go/+/385534
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Funda Secgin <fundasecgin33@gmail.com>
2025-02-03 08:38:43 -08:00
Kir Kolyshkin 3aa7c5ef01 testing: fix reference to B.N in docstring
Currently, a documentation reference to a struct member (such as [B.N])
does not result in it being rendered as a link, and thus the square
brackets remain in the rendered documentation which is mildly confusing.

The issue can be seen at
https://pkg.go.dev/testing@master#hdr-b_N_style_benchmarks

Remove the square brackets to fix.

Change-Id: Id374fd2085bd511018220c5d663650f89672302e
Reviewed-on: https://go-review.googlesource.com/c/go/+/643496
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2025-01-21 04:36:54 -08:00
cuishuang befc43655b testing/fstest: fix function name and comment
Change-Id: I17bc68a2e3a96d0dc3d9ddcad40149df37fc4839
Reviewed-on: https://go-review.googlesource.com/c/go/+/642198
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-01-14 09:43:06 -08:00
Damien Neil 39f2032c17 testing/synctest: add some examples
For #67434

Change-Id: Iebcfc2559a62405fea7e3ceff8cf6c2f50b61def
Reviewed-on: https://go-review.googlesource.com/c/go/+/641176
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Damien Neil <dneil@google.com>
2025-01-07 20:05:28 -08:00
Ian Lance Taylor 971448ddf8 testing: support B.Context and F.Context
CL 603959 added T.Context for #36532.

The discussion on the proposal only mentions t.Context.
However, the implementation of CL 603959 also added B.Context and F.Context.
They were added to the API listing, and B.Context was mentioned in
the release notes.

Unfortunately, the new B.Context and F.Context methods always
returned nil, rather than a context.Context value.

This change adds a working implementation of B.Context and F.Context.

For #36532
Fixes #70866

Change-Id: I8a44e6649fb658e4f641ffb7efd08b4374f578ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/637236
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-12-18 08:18:04 -08:00
Austin Clements 18b5435fc8 testing: don't measure cleanup time after B.Loop
B.Loop resets the timer on the first iteration so that setup code
isn't measured, but it currently leaves the timer running after the
last iteration, meaning that cleanup code will still be measured. Fix
this by stopping the timer when B.Loop returns false to indicate the
end of the benchmark.

Updates #61515

Change-Id: I0e0502cb2ce3c24cf872682b88d74e8be2c4529b
Reviewed-on: https://go-review.googlesource.com/c/go/+/635898
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Auto-Submit: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-12-15 21:41:28 -08:00
Austin Clements c1f2542c8b testing: improve B.Loop test
This moves the B.Loop test from package testing_test to package
testing, where it can check on more of the internals of the benchmark
state.

Updates #61515.

Change-Id: Ia32d7104526125c5e8a1e35dab7660008afcbf80
Reviewed-on: https://go-review.googlesource.com/c/go/+/635897
Auto-Submit: Austin Clements <austin@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
2024-12-15 21:41:26 -08:00
Austin Clements 6bd56fcaeb testing: improve b.Loop example
The current b.Loop example doesn't focus on the basic usage of b.Loop.
Replace this with a new example that uses (slightly) more realistic
things to demonstrate the most salient points of b.Loop.

We also move the example into an example file so that we can write a
real Benchmark function and a real function to be benchmarks, which
makes this much closer to what a user would actually write.

Updates #61515.

Change-Id: I4d830b3bfe3eb3cd8cdecef469fea0541baebb43
Reviewed-on: https://go-review.googlesource.com/c/go/+/635896
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-12-15 21:41:23 -08:00
Austin Clements 090748d6c7 testing: improve B.Loop docs, use B.Loop in examples
This updates the testing documentation to frame B.Loop as the
canonical way to write benchmarks. We retain documentation on b.N
benchmarks because people will definitely continue to see them (and
write them), but it's demoted to clearly second class.

This also attempts to clarify and refine the B.Loop documentation
itself.

Updates #61515
Fixes #70787

Change-Id: If5123435bfe3a5883a753119ecdf7bbc41afd499
Reviewed-on: https://go-review.googlesource.com/c/go/+/635895
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Caleb Spare <cespare@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Austin Clements <austin@google.com>
2024-12-15 21:41:19 -08:00
Jorropo d3d4e7b0d1 testing: fix divide by zero when benchmark is faster than OS's clock resolution
I did not added a test because `benchmark_test.go` is `package testing_test`
and I don't care to change that because calling predictN is not testing the
thing I would want to test.

Ideally we would run benchmark in a VM with a highjacked clocksource that never
marches forward, or using faketime but that looks fairly involved for a quickie
fix.

Fixes #70709

Change-Id: I8b4d697aff7cba33da388cb0ae8e2c2b550b9690
Reviewed-on: https://go-review.googlesource.com/c/go/+/633419
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Jorropo <jorropo.pgm@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2024-12-06 06:25:44 +00:00
Junyang Shao 5213e1e56c testing: improve documentation, examples, release notes for
testing.b.Loop.

This CL added documentation of the no-inlining semantic of b.Loop, with
a concrete example. This CL also tries to improve the release note to be
more descriptive.

Fixes #61515

Change-Id: I1e13cc92d5d6bdbf40fb44f44475e249747b807f
Reviewed-on: https://go-review.googlesource.com/c/go/+/633536
Reviewed-by: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
2024-12-05 21:03:13 +00:00
Dmitri Shuralyov 3374abeb50 testing: consider -N suffix after benchmark name optional
A "-N" suffix is left out when GOMAXPROCS is 1.

Also match at least 1 space (\s+ instead of \s*), remove trailing '.*'
(it's a no-op), and make the test error message style more consistent
while here.

For #61515.
Fixes #70627.

Change-Id: Id0a17478ac31e2934a663dd0d3b1b37f24974989
Cq-Include-Trybots: luci.golang.try:gotip-plan9-386
Reviewed-on: https://go-review.googlesource.com/c/go/+/632655
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-12-02 18:59:26 +00:00
Junyang Shao 154fb4e1d4 testing: Update testing.B.Loop to save benchmark results.
This is fixing some the missing logic of CL 627755.

Change-Id: I35acffef7299331fce21da4a80a26185b2e909f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/630455
Commit-Queue: Junyang Shao <shaojunyang@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-21 22:09:32 +00:00
Junyang Shao 558f5372fc cmd/compile,testing: implement one-time rampup logic for testing.B.Loop
testing.B.Loop now does its own loop scheduling without interaction with b.N.
b.N will be updated to the actual iterations b.Loop controls when b.Loop returns false.

This CL also added tests for fixed iteration count (benchtime=100x case).

This CL also ensured that b.Loop() is inlined.

For #61515

Change-Id: Ia15f4462f4830ef4ec51327520ff59910eb4bb58
Reviewed-on: https://go-review.googlesource.com/c/go/+/627755
Reviewed-by: Michael Pratt <mpratt@google.com>
Commit-Queue: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-11-20 23:19:48 +00:00
Damien Neil 76e4efdc77 testing/synctest: add experimental synctest package
The testing/synctest package is experimental,
subject to change or removal,
and only present when GOEXPERIMENT=synctest.

Fixes #69687

Change-Id: I6c79e7d1479a54bebcd2de754854c87bfd0a5fa1
Reviewed-on: https://go-review.googlesource.com/c/go/+/629735
Reviewed-by: Michael Pratt <mpratt@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-20 16:26:46 +00:00
Sean Liao 4310502385 testing/slogtest: reuse results obtained from previous call
Fixes #67605

Change-Id: Ib48ddfabb93887c88bfca347ba2e5f3c4b4e90b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/599836
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2024-11-13 19:03:44 +00:00
Constantin Konstantinidis f1add18530 testing: replace CRLF by LF on windows before comparing to the expected output
Fixes #51269

Change-Id: I06747db18ca078c1f1bda9b7bc60006f53191f4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/627035
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-11-13 18:37:25 +00:00
Tobias Klauser bb5339196f all: use slices.Sorted(maps.Keys(m))
Use slices.Sorted(maps.Keys(m)) to get a sorted slices of the keys in
a map.

Do not change packages built during bootstrap, as the bootstrap compiler
(currently 1.22.6) does not have the required maps and slices functions.

Change-Id: Ie35565d241fa14aca56b730a69af010127b659ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/617356
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
2024-10-02 14:22:59 +00:00
sunnymilk 402dc98759 testing: enable better loop time measurement for benchmarking.
With b.Loop() in place, the time measurement of loop scaling could be improved to be tighter. By identifying the first call to b.Loop(), we can avoid measuring the expensive ramp-up time by reset the timer tightly before the loop starts. The remaining loop scaling logic of b.N style loop is largely reused.

For #61515.

Change-Id: Ia7b8f0a8838f57c00ac6c5ef779d86f8d713c9b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/612835
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-09-20 19:10:01 +00:00
sunnymilk 6600a871ef testing: implement testing.B.Loop
Initial implementation for testing.B.Loop,
right now the calculation of b.N are still done in the old fasion way,
as of now b.Loop is merely an alias for the old loop over b.N.

For #61515.

Change-Id: If211d0acc5f0c33df530096dceafe0b947ab0c8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/608798
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Run-TryBot: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-09-20 19:09:41 +00:00
Kir Kolyshkin 6fa224a809 testing: use testenv.Executable
Note that this changes some nuances of how the tests work:
 - some tests had a fallback to using os.Args[0], which is removed;
 - some tests skipped (rather than failed) the test upon getting an
   error from os.Executable.

I think these changes are not practically relevant.

Change-Id: I0655add6d959a8b7e3359f94c38203aa06e8f490
Reviewed-on: https://go-review.googlesource.com/c/go/+/609303
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-09-03 18:03:28 +00:00
qiulaidongfeng 7303a283c4 testing: skip TestChdir/relative when on Windows when GOROOT and TMPDIR are on different drives
Fixes #69159

Change-Id: I0bbcf7075bdcf7a277a5053bcb543563a3074784
GitHub-Last-Rev: 86052a9ce3
GitHub-Pull-Request: golang/go#69160
Reviewed-on: https://go-review.googlesource.com/c/go/+/609304
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-09-03 18:01:12 +00:00
Vladimir Varankin 440c9ee73d testing: rename testContext to testState
Following up to CL 603959, update internals of testing package to
reduce the confusion around "context". The changes rename
testContext/benchContext/fuzzContext to testState/benchState/fuzzState.

Change-Id: Ib8855dab456d41ab343488fcf5fefff2431f7b72
Reviewed-on: https://go-review.googlesource.com/c/go/+/607555
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-08-21 20:52:09 +00:00
Dmitri Shuralyov b292799ddd testing: use temp dir without symlinks in TestChdir/relative
When paths with symlinks are involved, it's not viable to compare them
with string equality. Don't use a temporary directory with symlinks in
it as input, so the test works in more environments.

For #62516.

Change-Id: I95d774365cc2f90eb0ffcffa61229ed5cee43e3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/607035
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-08-20 17:15:47 +00:00
Vladimir Varankin f432b5f756 testing: add Context
Adds a new Context method to testing.T, that returns a context, that is
canceled before the end of its test function.

Fixes #36532.

Change-Id: I9315ad4dad25529d0b5be809e2d9db4e7528b5f2
GitHub-Last-Rev: 1c3fd6c4d8
GitHub-Pull-Request: golang/go#68828
Reviewed-on: https://go-review.googlesource.com/c/go/+/603959
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-08-20 14:58:54 +00:00
Kir Kolyshkin 79ca434ac6 testing: add Chdir
Some tests need to use os.Chdir, but the use is complicated because
 - they must change back to the old working directory;
 - they must not use t.Parallel.

Add Chdir that covers these cases, and sets PWD environment variable
to the new directory for the duration of the test for Unix platforms.
Unify the panic message when t.Parallel is used together with t.Setenv
or t.Chdir.

Add some tests.

For #62516.

Change-Id: Ib050d173b26eb28a27dba5a206b2d0d877d761c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/529895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-08-16 23:48:50 +00:00
apocelipes 05861ff90c os,path/filepath,testing: use slices to clean up tests
Replace reflect.DeepEqual with slices.Equal which is much faster.

Change-Id: I299db6f998738174983731f08c1021346b87dfaa
GitHub-Last-Rev: 44ca9015d6
GitHub-Pull-Request: golang/go#67613
Reviewed-on: https://go-review.googlesource.com/c/go/+/587938
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-07-25 00:23:06 +00:00
Julian Dax 87ec2c959c testing: remove call to os.Exit in documentation for TestMain
In the example for the TestMain function, os.Exit was called explicitly,
which is no longer necessary since Go 1.15 (see change #219639).

Updates #34129

Change-Id: Ia8021de0d5699742adbb7ab6d28b11c841e596dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/596977
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-07-08 22:10:16 +00:00
Than McIntosh a85b8810c4 cmd/go: fix problems with coverage percentage reporting w/ -coverpkg
This patch resolves a set of problems with "percent covered" metrics
reported when the "-coverpkg" is in effect; these bugs were introduced
in Go 1.22 with the rollout of CL 495452 and related changes.
Specifically, for runs with multiple packages selected but without
-coverpkg, "percent covered" metrics were generated for package P not
based just on P's statements but on the entire corpus of statements.

Fixes #65570.

Change-Id: I38d61886cb46ebd38d8c4313c326d671197c3568
Reviewed-on: https://go-review.googlesource.com/c/go/+/592205
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2024-06-14 11:46:05 +00:00