Commit Graph

435 Commits

Author SHA1 Message Date
Tobias Klauser bb5339196f all: use slices.Sorted(maps.Keys(m))
Use slices.Sorted(maps.Keys(m)) to get a sorted slices of the keys in
a map.

Do not change packages built during bootstrap, as the bootstrap compiler
(currently 1.22.6) does not have the required maps and slices functions.

Change-Id: Ie35565d241fa14aca56b730a69af010127b659ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/617356
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
2024-10-02 14:22:59 +00:00
sunnymilk 402dc98759 testing: enable better loop time measurement for benchmarking.
With b.Loop() in place, the time measurement of loop scaling could be improved to be tighter. By identifying the first call to b.Loop(), we can avoid measuring the expensive ramp-up time by reset the timer tightly before the loop starts. The remaining loop scaling logic of b.N style loop is largely reused.

For #61515.

Change-Id: Ia7b8f0a8838f57c00ac6c5ef779d86f8d713c9b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/612835
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-09-20 19:10:01 +00:00
sunnymilk 6600a871ef testing: implement testing.B.Loop
Initial implementation for testing.B.Loop,
right now the calculation of b.N are still done in the old fasion way,
as of now b.Loop is merely an alias for the old loop over b.N.

For #61515.

Change-Id: If211d0acc5f0c33df530096dceafe0b947ab0c8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/608798
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Junyang Shao <shaojunyang@google.com>
Run-TryBot: Junyang Shao <shaojunyang@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-09-20 19:09:41 +00:00
Kir Kolyshkin 6fa224a809 testing: use testenv.Executable
Note that this changes some nuances of how the tests work:
 - some tests had a fallback to using os.Args[0], which is removed;
 - some tests skipped (rather than failed) the test upon getting an
   error from os.Executable.

I think these changes are not practically relevant.

Change-Id: I0655add6d959a8b7e3359f94c38203aa06e8f490
Reviewed-on: https://go-review.googlesource.com/c/go/+/609303
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-09-03 18:03:28 +00:00
qiulaidongfeng 7303a283c4 testing: skip TestChdir/relative when on Windows when GOROOT and TMPDIR are on different drives
Fixes #69159

Change-Id: I0bbcf7075bdcf7a277a5053bcb543563a3074784
GitHub-Last-Rev: 86052a9ce3
GitHub-Pull-Request: golang/go#69160
Reviewed-on: https://go-review.googlesource.com/c/go/+/609304
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-09-03 18:01:12 +00:00
Vladimir Varankin 440c9ee73d testing: rename testContext to testState
Following up to CL 603959, update internals of testing package to
reduce the confusion around "context". The changes rename
testContext/benchContext/fuzzContext to testState/benchState/fuzzState.

Change-Id: Ib8855dab456d41ab343488fcf5fefff2431f7b72
Reviewed-on: https://go-review.googlesource.com/c/go/+/607555
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-08-21 20:52:09 +00:00
Dmitri Shuralyov b292799ddd testing: use temp dir without symlinks in TestChdir/relative
When paths with symlinks are involved, it's not viable to compare them
with string equality. Don't use a temporary directory with symlinks in
it as input, so the test works in more environments.

For #62516.

Change-Id: I95d774365cc2f90eb0ffcffa61229ed5cee43e3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/607035
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-08-20 17:15:47 +00:00
Vladimir Varankin f432b5f756 testing: add Context
Adds a new Context method to testing.T, that returns a context, that is
canceled before the end of its test function.

Fixes #36532.

Change-Id: I9315ad4dad25529d0b5be809e2d9db4e7528b5f2
GitHub-Last-Rev: 1c3fd6c4d8
GitHub-Pull-Request: golang/go#68828
Reviewed-on: https://go-review.googlesource.com/c/go/+/603959
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-08-20 14:58:54 +00:00
Kir Kolyshkin 79ca434ac6 testing: add Chdir
Some tests need to use os.Chdir, but the use is complicated because
 - they must change back to the old working directory;
 - they must not use t.Parallel.

Add Chdir that covers these cases, and sets PWD environment variable
to the new directory for the duration of the test for Unix platforms.
Unify the panic message when t.Parallel is used together with t.Setenv
or t.Chdir.

Add some tests.

For #62516.

Change-Id: Ib050d173b26eb28a27dba5a206b2d0d877d761c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/529895
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-08-16 23:48:50 +00:00
apocelipes 05861ff90c os,path/filepath,testing: use slices to clean up tests
Replace reflect.DeepEqual with slices.Equal which is much faster.

Change-Id: I299db6f998738174983731f08c1021346b87dfaa
GitHub-Last-Rev: 44ca9015d6
GitHub-Pull-Request: golang/go#67613
Reviewed-on: https://go-review.googlesource.com/c/go/+/587938
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-07-25 00:23:06 +00:00
Julian Dax 87ec2c959c testing: remove call to os.Exit in documentation for TestMain
In the example for the TestMain function, os.Exit was called explicitly,
which is no longer necessary since Go 1.15 (see change #219639).

Updates #34129

Change-Id: Ia8021de0d5699742adbb7ab6d28b11c841e596dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/596977
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-07-08 22:10:16 +00:00
Than McIntosh a85b8810c4 cmd/go: fix problems with coverage percentage reporting w/ -coverpkg
This patch resolves a set of problems with "percent covered" metrics
reported when the "-coverpkg" is in effect; these bugs were introduced
in Go 1.22 with the rollout of CL 495452 and related changes.
Specifically, for runs with multiple packages selected but without
-coverpkg, "percent covered" metrics were generated for package P not
based just on P's statements but on the entire corpus of statements.

Fixes #65570.

Change-Id: I38d61886cb46ebd38d8c4313c326d671197c3568
Reviewed-on: https://go-review.googlesource.com/c/go/+/592205
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2024-06-14 11:46:05 +00:00
Than McIntosh 9b5d27faf9 cmd/go,testdeps: move import of internal/coverage/cfile to testmain
Instead of having testing/internal/testdeps import the
internal/coverage/cfile package directly, have the code in testmain
pass in pointers to cfile functions during setup in the case that
we're running a "go test -cover" binary. This reduces the size of
regular non-coverage test binaries back to what they were before CL
585820.

Updates #67401.
Fixes #67588.

Change-Id: Iaf1a613bc7d3c9df9943189065d0161ca9120d34
Reviewed-on: https://go-review.googlesource.com/c/go/+/587795
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Russ Cox <rsc@golang.org>
2024-05-23 14:01:23 +00:00
Ian Lance Taylor b0b1d42db3 all: change from sort functions to slices functions where feasible
Doing this because the slices functions are slightly faster and
slightly easier to use. It also removes one dependency layer.

This CL does not change packages that are used during bootstrap,
as the bootstrap compiler does not have the required slices functions.
It does not change the go/scanner package because the ErrorList
Len, Swap, and Less methods are part of the Go 1 API.

Change-Id: If52899be791c829198e11d2408727720b91ebe8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/587655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-05-23 01:00:11 +00:00
Russ Cox cd41d71785 internal/coverage/cfile: remove //go:linkname into testing
This is just looking at a command-line flag. Look directly.

For #67401.

Change-Id: I7a1c3fc2d9cc85e5ffc5731444bf6db87abf6901
Reviewed-on: https://go-review.googlesource.com/c/go/+/585916
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-05-22 09:58:07 +00:00
Russ Cox 180ea45566 runtime/coverage: remove uses of //go:linkname
Move code to internal/coverage/cfile, making it possible to
access directly from testing/internal/testdeps, so that we can
avoid needing //go:linkname hacks.

For #67401.

Change-Id: I10b23a9970164afd2165e718ef3b2d9e86783883
Reviewed-on: https://go-review.googlesource.com/c/go/+/585820
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-05-21 19:41:02 +00:00
Cherry Mui c4772d30bf cmd/link: disallow pull-only linknames
As mentioned in CL 584598, linkname is a mechanism that, when
abused, can break API integrity and even safety of Go programs.
CL 584598 is a first step to restrict the use of linknames, by
implementing a blocklist. This CL takes a step further, tightening
up the restriction by allowing linkname references ("pull") only
when the definition side explicitly opts into it, by having a
linkname on the definition (possibly to itself). This way, it is at
least clear on the definition side that the symbol, despite being
unexported, is accessed outside of the package. Unexported symbols
without linkname can now be actually private. This is similar to
the symbol visibility rule used by gccgo for years (which defines
unexported non-linknamed symbols as C static symbols).

As there can be pull-only linknames in the wild that may be broken
by this change, we currently only enforce this rule for symbols
defined in the standard library. Push linknames are added in the
standard library to allow things build.

Linkname references to external (non-Go) symbols are still allowed,
as their visibility is controlled by the C symbol visibility rules
and enforced by the C (static or dynamic) linker.

Assembly symbols are treated similar to linknamed symbols.

This is controlled by -checklinkname linker flag, currently not
enabled by default. A follow-up CL will enable it by default.

Change-Id: I07344f5c7a02124dbbef0fbc8fec3b666a4b2b0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/585358
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
2024-05-15 19:57:43 +00:00
Alan Donovan 69e75c8581 runtime: properly frame panic values in tracebacks
This CL causes the printing of panic values to ensure that all
newlines in the output are immediately followed by a tab, so
that there is no way for a maliciously crafted panic value to
fool a program attempting to parse the traceback into thinking
that the panic value is in fact a goroutine stack.

See https://github.com/golang/go/issues/64590#issuecomment-1932675696

+ release note

Updates #64590
Updates #63455

Change-Id: I5142acb777383c0c122779d984e73879567dc627
Reviewed-on: https://go-review.googlesource.com/c/go/+/581215
Auto-Submit: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2024-05-08 19:10:41 +00:00
Jorropo 865cf5c5f6 testing: improve the documentation around b.N
Fixes #67137

- Make it clear the benchmark function is called multiple times.
- Demonstrate range over int.

Change-Id: I7e993d938b0351012cdd4aed8528951e0ad406ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/582835
Auto-Submit: Alan Donovan <adonovan@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-05-02 18:41:57 +00:00
Egon Elbre 644a4ff333 testing: use time.Since in highPrecisionTime
time.Since has optimizations for measuring monotonic time.

For #31160.

Change-Id: I0529b9f69b9f008e3414b8e386b6faa64af4a008
Reviewed-on: https://go-review.googlesource.com/c/go/+/582135
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-04-27 19:42:36 +00:00
Egon Elbre 1c4704991a testing: use QueryPerformanceCounter on Windows
Windows time.Now granularity is around 0.5ms on modern systems,
which introduces a significant noise into benchmark results.
Instead of relying time.Now use QueryPerformanceCounter, which
has significantly better granularity compared to time.Now.

 │ TimeNow-32  │        HighPrecisionTimeNow-32        │
 │   sec/op    │    sec/op     vs base                 │
   4.812n ± 0%   30.580n ± 0%  +535.43% (p=0.000 n=20)

Fixes #31160

Change-Id: Ib2a574d638c9c6762a2524212def02265574e267
Reviewed-on: https://go-review.googlesource.com/c/go/+/557315
Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2024-04-26 22:55:25 +00:00
Olivier Mengué 74cce866f8 testing/fstest: return structured errors in TestFS
TestFS now returns a structured error built with errors.Join to allow to
inspect errors using errors.Is and errors.As.

All errors are now wrapped using fmt.Errorf and %w.

Fixes #63675.

Change-Id: I8fc3363f8ae70085af4afdb84c16be9ca70d7731
Reviewed-on: https://go-review.googlesource.com/c/go/+/537015
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-04-05 17:45:39 +00:00
cui fliter 1e12eab870 all: fix a large number of comments
Partial typo corrections, following https://go.dev/wiki/Spelling

Change-Id: I2357906ff2ea04305c6357418e4e9556e20375d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/573776
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-26 19:58:28 +00:00
Ian Lance Taylor 27f41bb153 Revert "testing: add TB.SetGOMAXPROCS function"
This reverts CL 519235.

Reason for revert: Proposal is still in incoming.

For #62020

Change-Id: Icccb930209f36097f5d930c01eda6b5042bdddc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/573516
Reviewed-by: Than McIntosh <thanm@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-03-22 13:07:36 +00:00
sivchari 5d29578fd5 testing: add TB.SetGOMAXPROCS function
Add a new method TB.SetGOMAXPROCS which sets variable of GOMAXPROCS.
This method aims to set a variable for the isolated lifetime of the test and cleans up.
And unset this when the test ends.
This method disables the test or benchmark from running in
parallel.

Fixes: #62020

Change-Id: Iae44109d0def35cc47049c3ca4cd5306173d52ee
Signed-off-by: sivchari <shibuuuu5@gmail.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/519235
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-21 20:26:36 +00:00
guoguangwu 5d4c6088be testing: fix typo in comment
Change-Id: I5aa6093b0199df1ef5b0ad0fcfa651a4b990bfd5
GitHub-Last-Rev: b053d993eb
GitHub-Pull-Request: golang/go#65986
Reviewed-on: https://go-review.googlesource.com/c/go/+/567655
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Auto-Submit: Carlos Amedee <carlos@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-02-28 20:32:54 +00:00
Shang Ding ff0b46cfbb testing/fstest: return base name from mapfs FileInfo.Name
Change-Id: I5a68389a68875dbb2f6875de3f64f63dd7ca1af7
Reviewed-on: https://go-review.googlesource.com/c/go/+/565055
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2024-02-20 14:59:55 +00:00
Bryan C. Mills 26b1694d9a testing: add regression tests for reentrant calls to T.Run
These tests represent two patterns of usage, found in Google-internal
tests, that deadlocked after CL 506755.

TestConcurrentRun is a minor variation on TestParallelSub, with the
additional expectation that the concurrent calls to Run (without
explicit calls to Parallel) proceed without blocking. It replaces
TestParallelSub.

TestParentRun is similar, but instead of calling Run concurrently it
calls Run from within the subtest body. It almost certainly represents
an accidental misuse of T.Run, but since that pattern used to run to
completion we don't want to break it accidentally. (Perhaps it should
be diagnosed with a vet check instead?)

While we are testing concurrency, this also cleans up
TestConcurrentCleanup to use a clearer synchronization pattern.

Fixes #64402.

Change-Id: I14fc7e7085a994c284509eac28190c3a8feb04cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/546019
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2023-12-01 21:27:08 +00:00
Bryan C. Mills 76d90a34dd testing: remove tests from the running log while they are waiting on parallel subtests
The parallel subtests are themselves removed from the running map
while they are blocked on calls to t.Parallel, so it is misleading to
log their parents as if they are running when we know they cannot be
making any kind of meaningful progress.

Fixes #64404.

Change-Id: Iaad11d5d4f4c86d775d36e5285c49629dccddd74
Reviewed-on: https://go-review.googlesource.com/c/go/+/546018
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2023-11-30 16:41:22 +00:00
Lasse Folger 6317fb41e9 Revert "testing: simplify concurrency and cleanup logic"
reverts CL 506755

Reason for revert: This leads to deadlocks of tests in Google internal testing

For #64402

Change-Id: I78329fc9dcc2377e7e880b264ac1d18d577ef99a
Reviewed-on: https://go-review.googlesource.com/c/go/+/544895
Auto-Submit: Bryan Mills <bcmills@google.com>
Auto-Submit: Lasse Folger <lassefolger@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-11-27 16:49:24 +00:00
Bryan C. Mills e6b76bfc46 testing: simplify concurrency and cleanup logic
While investigating #60083, I found a couple of bugs (notably #61034)
that had slipped through code review in part because the concurrency
patterns used in the testing package were too complex for me to fully
reason about. This change adjusts those patterns to be more in line
with current idioms, and to reduce the number of special cases that
depend on details that should be orthogonal. (For example: the details
of how we invoke the Cleanup functions should not depend on whether
the test happened to run any parallel subtests.)

In the process, this change fixes a handful of bugs:

- Concurrent calls to Run (explicitly allowed by TestParallelSub)
  could previously drive the testcontext.running count negative,
  causing the number of running parallel tests to exceed the -parallel
  flag.

- The -failfast flag now takes effect immediately on failure. It no
  longer delays until the test finishes, and no longer misses failures
  during cleanup (fixing #61034).

- If a Cleanup function calls runtime.Goexit (typically via t.FailNow)
  during a panic, Cleanup functions from its parent tests are no
  longer skipped and buffered logs from its parent tests are now
  flushed.

- The time reported for a test with subtests now includes the time spent
  running those subtests, regardless of whether they are parallel.
  (Previously, non-parallel subtests were included but parallel subtests
  were not.)

- Calls to (*B).Run in iterations after the first are now diagnosed
  with a panic. (This diagnoses badly-behaved benchmarks: if Run is
  called during the first iteration, no subsequent iterations are
  supposed to occur.)

Fixes #61034.

Change-Id: I3797f6ef5210a3d2d5d6c2710d3f35c0219b02ea
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-amd64-longtest-race,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/506755
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-21 22:58:46 +00:00
Bryan C. Mills 0cb45bac01 testing: use subprocesses in TestTBHelper and TestTBHelperParallel
These tests are checking the output of test functions that call the
Helper methods. However, they were reaching into package internals
instead of running those test functions as actual tests.

That not only produced significant differences in formatting (such as
indentation for subtests), but also caused test flags such as
"-failfast" passed for the overall test run to interfere with the
output formatting.

Now, we run the test functions as real tests in a subprocess,
so that we get the real output and formatting of those tests.
This makes the tests not only more realistic, but also less
sensitive to otherwise-irrelevant implementation details
(such as the names and signatures of unexported types and
functions in the testing package).

Fixes #61016.

Change-Id: I646fbbd7cfeb00382054677f726c05fc9d35d0dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/506955
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-21 17:24:47 +00:00
Daniel Martí 4988a65f2a all: make more use of the new atomic integer types
Slightly simplifies the code and avoids human error.

Change-Id: Ib76575e8bc5b3a699ba6cc3870d63cd7a55e6416
Reviewed-on: https://go-review.googlesource.com/c/go/+/541476
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-11-17 16:39:52 +00:00
cui fliter 0889b39ff1 testing: add available godoc link
Change-Id: I8f4d097601796f53176d490cddf8832b7caa4c05
Reviewed-on: https://go-review.googlesource.com/c/go/+/539836
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-11-08 17:55:47 +00:00
Evan Jones 0354ee3aab testing: correct comments on runCleanup
The comment on runCleanup states "If catchPanic is true ...", but
there is no catchPanic argument or variable. This was introduced
in CL 214822, which introduced the panicHandling type. The code was
updated during code review, but the comment was missed.

Change-Id: Id14c5397e7a026bfdf98ea10ecb1e4c61ce2f924
Reviewed-on: https://go-review.googlesource.com/c/go/+/538695
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-11-06 18:01:18 +00:00
Bryan C. Mills 55b8e16b2e testing: use monotonic counts to attribute races in subtests
This implements the approach I described in
https://go-review.git.corp.google.com/c/go/+/494057/1#message-5c9773bded2f89b4058848cb036b860aa6716de3.

Specifically:

- Each level of test atomically records the cumulative number of races
  seen as of the last race-induced test failure.

- When a subtest fails, it logs the race error, and then updates its
  parents' counters so that they will not log the same error.

- We check each test or benchmark for races before it starts running
  each of its subtests or sub-benchmark, before unblocking parallel
  subtests, and after running any cleanup functions.

With this implementation, it should be the case that every test that
is running when a race is detected reports that race, and any race
reported for a subtest is not redundantly reported for its parent.

The regression tests are based on those added in CL 494057 and
CL 501895, with a few additions based on my own review of the code.

Fixes #60083.

Change-Id: I578ae929f192a7a951b31b17ecb560cbbf1ef7a1
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-linux-amd64-longtest-race,gotip-windows-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/506300
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-25 20:44:25 +00:00
Nakul Bajaj 983d90e11d testing/slogtest: test no source key with empty PC in record
Fixes #62280

Change-Id: Ideaffb797d8bb9cc70e635f0b019ae3cb90abc92
Reviewed-on: https://go-review.googlesource.com/c/go/+/536117
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
2023-10-24 15:06:07 +00:00
Olivier Mengué 53d000d75b testing/fstest: add godoc links
Change-Id: I611f04f12bb93197fea5ae40a08e5f60426f1c00
Reviewed-on: https://go-review.googlesource.com/c/go/+/534076
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: qiulaidongfeng <2645477756@qq.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-11 20:21:27 +00:00
Olivier Mengué 09aada24aa testing/fstest: MapFS: set perm 0555 on synthetized dirs
As MapFS ignores filemodes and always grant read and traverse access on
directories, let's make synthetized directory entries to expose filemode
0555 instead of 0000.
Fixes #63468.

Change-Id: I5d64a6bf2f2ac6082ca5dde55b3062669fb50b8d
Reviewed-on: https://go-review.googlesource.com/c/go/+/534075
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-11 20:20:55 +00:00
Ville Skyttä b201cf4afa testing: remove unnecessary TempDir implementation detail from its doc
The "by Cleanup" detail about the removal mechanism is not necessary for
users. The previous wording could even cause confusion whether they
should do something for Cleanup to occur.

Change-Id: I617a7a4e118a70a3ad8a83a2d81cd911dde8ee08
GitHub-Last-Rev: fba4cde636
GitHub-Pull-Request: golang/go#63479
Reviewed-on: https://go-review.googlesource.com/c/go/+/534016
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-10-10 00:25:53 +00:00
Tobias Klauser dd2279ee34 testing: use strings.LastIndexAny in (*common).decorate
Change-Id: I14c521f4e599e14c71a52342b0ead5d0d4007f1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/527655
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-09-13 18:06:09 +00:00
Jonathan Amsterdam a742ae493f testing/slogtest: add Run to run cases as subtests
This is an implementation of proposal #61758.

It adds a function to slogtest that runs each test case in a subtest,
instead of running them all at once.

That allows the caller to control which cases are run.

Fixes #61706.
Fixes #61758.

Change-Id: I95108b7b753675203ca7f0f00ccbc242bd9c2a9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/516076
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
2023-09-08 20:37:12 +00:00
Dmitri Shuralyov 0dfb22ed70 all: use ^TestName$ regular pattern for invoking a single test
Use ^ and $ in the -run flag regular expression value when the intention
is to invoke a single named test. This removes the reliance on there not
being another similarly named test to achieve the intended result.

In particular, package syscall has tests named TestUnshareMountNameSpace
and TestUnshareMountNameSpaceChroot that both trigger themselves setting
GO_WANT_HELPER_PROCESS=1 to run alternate code in a helper process. As a
consequence of overlap in their test names, the former was inadvertently
triggering one too many helpers.

Spotted while reviewing CL 525196. Apply the same change in other places
to make it easier for code readers to see that said tests aren't running
extraneous tests. The unlikely cases of -run=TestSomething intentionally
being used to run all tests that have the TestSomething substring in the
name can be better written as -run=^.*TestSomething.*$ or with a comment
so it is clear it wasn't an oversight.

Change-Id: Iba208aba3998acdbf8c6708e5d23ab88938bfc1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/524948
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-09-05 23:35:29 +00:00
go101 972cc3e7c2 testing: add doc to not call Init concurrently
Change-Id: I2e218805fbe4858be125df97bdaf921799315799
GitHub-Last-Rev: 32b798d75f
GitHub-Pull-Request: golang/go#62410
Reviewed-on: https://go-review.googlesource.com/c/go/+/525015
Auto-Submit: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-09-05 22:26:12 +00:00
Jonathan Amsterdam e016fabb55 testing/slogtest: clarify that passed handler must be enabled at info
Fixes #62115.

Change-Id: I1ecd3048ccbf2014bc1d561c7cd79a0c2fe8e4bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/520676
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-08-29 11:43:03 +00:00
Guoqi Chen 6a1182027c internal/sysinfo: print cpu type from cpuinfo when internal cpu name is empty on Linux
Supports all linux operating systems. currently tested on x86, mips64le and loong64.

Example output:
$ go test -bench=.*
goos: linux
goarch: loong64
pkg: runtime
cpu: Loongson-3A5000-HV @ 2500.00MHz
BenchmarkSemTable/OneAddrCollision/n=1000         	   19261	     62302 ns/op
    ...

Change-Id: I02db12d70c11327e4625bb6e59f30dfaf37c2db0
Reviewed-on: https://go-review.googlesource.com/c/go/+/508735
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
2023-08-17 21:40:36 +00:00
Muhammad Falak R Wani 215bcbc6ac testing: use the builtin max/min function
Change-Id: I1250b6a33f5f3509a216d307f1783ad4aa5937fc
GitHub-Last-Rev: 98dc6b1a11
GitHub-Pull-Request: golang/go#61809
Reviewed-on: https://go-review.googlesource.com/c/go/+/516616
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-08-07 18:49:54 +00:00
montag451 64c1be144f testing: improve the usage message of -benchtime
Specify that -benchtime can take the form Nx.

Change-Id: I1e711cdb2e19e3ff5eb2cea4e7c8843bc58696b1
GitHub-Last-Rev: 1cb13f7dba
GitHub-Pull-Request: golang/go#61756
Reviewed-on: https://go-review.googlesource.com/c/go/+/515801
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-08-07 18:49:02 +00:00
Jonathan Amsterdam 2bf8b3985f testing/slogtest: reword a confusing explanation
Updates #61758.

Change-Id: I4ce0359f676b6832e40203e1034ba1c4f6d2b2ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/516155
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Patrick Ohly <patrick.ohly@googlemail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
2023-08-07 18:23:51 +00:00
Jes Cok 3bc28402fa all: use built-in clear to clear maps
Change-Id: I7f4ac72fe3230d8b7486fab0c925015cefcbe355
GitHub-Last-Rev: 54455839b6
GitHub-Pull-Request: golang/go#61544
Reviewed-on: https://go-review.googlesource.com/c/go/+/512376
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-07-27 22:07:13 +00:00