signature-generator: add Go func signature fuzzing tools

Add a new subdirectory 'signature-fuzzer' to the x/tools/cmd
directory, with contents:

  internal/fuzz-generator      Guts of the signature fuzzer
  fuzz-driver                  Simple driver for the fuzzer
  fuzz-runner                  Test harness for doing iterative fuzzing

This set of tools is primarily of interest to compiler developers as
opposed to end users. The fuzzer generates random Go code with
"interesting" randomly chosen function signatures (random numbers of
parameters of randomly generated types), then emits code to call these
random functions with known inputs, as well as code that checks to
make sure parameter values were passed correctly and that returns
worked.  The generator tries to include odd scenarios and corner cases
(e.g. reflect calls, recursion, blank params, zero sized types, etc)
so as to explore all the nooks and crannies of the compiler.

This fuzzer was used during the bringup of the new register ABI (as
part of Go 1.17) to identify odd corner cases and flush out bugs. It
may be useful in the future if we have to support new ABI versions or
bring up register ABIs on new architectures.

See the README.md for more details.

Change-Id: Icbc89ddd07c3909a44a005deab43720c7a0dffde
Reviewed-on: https://go-review.googlesource.com/c/tools/+/350329
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This commit is contained in:
Than McIntosh 2021-09-16 12:46:51 -04:00
parent 5d35a75079
commit fb3622a928
17 changed files with 4675 additions and 0 deletions

View File

@ -0,0 +1,159 @@
# signature-fuzzer
This directory contains utilities for fuzz testing of Go function signatures, for use in developing/testing a Go compiler.
The basic idea of the fuzzer is that it emits source code for a stand-alone Go program; this generated program is a series of pairs of functions, a "Caller" function and a "Checker" function. The signature of the Checker function is generated randomly (random number of parameters and returns, each with randomly chosen types). The "Caller" func contains invocations of the "Checker" function, each passing randomly chosen values to the params of the "Checker", then the caller verifies that expected values are returned correctly. The "Checker" function in turn has code to verify that the expected values (more details below).
There are three main parts to the fuzzer: a generator package, a driver package, and a runner package.
The "generator" contains the guts of the fuzzer, the bits that actually emit the random code.
The "driver" is a stand-alone program that invokes the generator to create a single test program. It is not terribly useful on its own (since it doesn't actually build or run the generated program), but it is handy for debugging the generator or looking at examples of the emitted code.
The "runner" is a more complete test harness; it repeatedly runs the generator to create a new test program, builds the test program, then runs it (checking for errors along the way). If at any point a build or test fails, the "runner" harness attempts a minimization process to try to narrow down the failure to a single package and/or function.
## What the generated code looks like
Generated Go functions will have an "interesting" set of signatures (mix of
arrays, scalars, structs), intended to pick out corner cases and odd bits in the
Go compiler's code that handles function calls and returns.
The first generated file is genChecker.go, which contains function that look something
like this (simplified):
```
type StructF4S0 struct {
F0 float64
F1 int16
F2 uint16
}
// 0 returns 2 params
func Test4(p0 int8, p1 StructF4S0) {
c0 := int8(-1)
if p0 != c0 {
NoteFailure(4, "parm", 0)
}
c1 := StructF4S0{float64(2), int16(-3), uint16(4)}
if p1 != c1 {
NoteFailure(4, "parm", 1)
}
return
}
```
Here the test generator has randomly selected 0 return values and 2 params, then randomly generated types for the params.
The generator then emits code on the calling side into the file "genCaller.go", which might look like:
```
func Caller4() {
var p0 int8
p0 = int8(-1)
var p1 StructF4S0
p1 = StructF4S0{float64(2), int16(-3), uint16(4)}
// 0 returns 2 params
Test4(p0, p1)
}
```
The generator then emits some utility functions (ex: NoteFailure) and a main routine that cycles through all of the tests.
## Trying a single run of the generator
To generate a set of source files just to see what they look like, you can build and run the test generator as follows. This creates a new directory "cabiTest" containing generated test files:
```
$ git clone https://golang.org/x/tools
$ cd tools/cmd/signature-fuzzer/fuzz-driver
$ go build .
$ ./fuzz-driver -numpkgs 3 -numfcns 5 -seed 12345 -outdir /tmp/sigfuzzTest -pkgpath foobar
$ cd /tmp/sigfuzzTest
$ find . -type f -print
./genCaller1/genCaller1.go
./genUtils/genUtils.go
./genChecker1/genChecker1.go
./genChecker0/genChecker0.go
./genCaller2/genCaller2.go
./genCaller0/genCaller0.go
./genMain.go
./go.mod
./genChecker2/genChecker2.go
$
```
You can build and run the generated files in the usual way:
```
$ cd /tmp/sigfuzzTest
$ go build .
$ ./foobar
starting main
finished 15 tests
$
```
## Example usage for the test runner
The test runner orchestrates multiple runs of the fuzzer, iteratively emitting code, building it, and testing the resulting binary. To use the runner, build and invoke it with a specific number of iterations; it will select a new random seed on each invocation. The runner will terminate as soon as it finds a failure. Example:
```
$ git clone https://golang.org/x/tools
$ cd tools/cmd/signature-fuzzer/fuzz-runner
$ go build .
$ ./fuzz-runner -numit=3
... begin iteration 0 with current seed 67104558
starting main
finished 1000 tests
... begin iteration 1 with current seed 67104659
starting main
finished 1000 tests
... begin iteration 2 with current seed 67104760
starting main
finished 1000 tests
$
```
If the runner encounters a failure, it will try to perform test-case "minimization", e.g. attempt to isolate the failure
```
$ cd tools/cmd/signature-fuzzer/fuzz-runner
$ go build .
$ ./fuzz-runner -n=10
./fuzz-runner -n=10
... begin iteration 0 with current seed 40661762
Error: fail [reflect] |20|3|1| =Checker3.Test1= return 1
error executing cmd ./fzTest: exit status 1
... starting minimization for failed directory /tmp/fuzzrun1005327337/fuzzTest
package minimization succeeded: found bad pkg 3
function minimization succeeded: found bad fcn 1
$
```
Here the runner has generates a failure, minimized it down to a single function and package, and left the resulting program in the output directory /tmp/fuzzrun1005327337/fuzzTest.
## Limitations, future work
No support yet for variadic functions.
The set of generated types is still a bit thin; it has fairly limited support for interface values, and doesn't include channels.
Todos:
- better interface value coverage
- implement testing of reflect.MakeFunc
- extend to work with generic code of various types
- extend to work in a debugging scenario (e.g. instead of just emitting code,
emit a script of debugger commands to run the program with expected
responses from the debugger)
- rework things so that instead of always checking all of a given parameter
value, we sometimes skip over elements (or just check the length of a slice
or string as opposed to looking at its value)
- consider adding runtime.GC() calls at some points in the generated code

View File

@ -0,0 +1,168 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Stand-alone driver for emitting function-signature test code. This
// program is mainly just a wrapper around the code that lives in the
// fuzz-generator package; it is useful for generating a specific bad
// code scenario for a given seed, or for doing development on the
// fuzzer, but for doing actual fuzz testing, better to use
// fuzz-runner.
package main
import (
"flag"
"fmt"
"log"
"math/rand"
"os"
"time"
generator "golang.org/x/tools/cmd/signature-fuzzer/internal/fuzz-generator"
)
// Basic options
var numfcnflag = flag.Int("numfcns", 10, "Number of test func pairs to emit in each package")
var numpkgflag = flag.Int("numpkgs", 1, "Number of test packages to emit")
var seedflag = flag.Int64("seed", -1, "Random seed")
var tagflag = flag.String("tag", "gen", "Prefix name of go files/pkgs to generate")
var outdirflag = flag.String("outdir", "", "Output directory for generated files")
var pkgpathflag = flag.String("pkgpath", "gen", "Base package path for generated files")
// Options used for test case minimization.
var fcnmaskflag = flag.String("fcnmask", "", "Mask containing list of fcn numbers to emit")
var pkmaskflag = flag.String("pkgmask", "", "Mask containing list of pkg numbers to emit")
// Options used to control which features are used in the generated code.
var reflectflag = flag.Bool("reflect", true, "Include testing of reflect.Call.")
var deferflag = flag.Bool("defer", true, "Include testing of defer stmts.")
var recurflag = flag.Bool("recur", true, "Include testing of recursive calls.")
var takeaddrflag = flag.Bool("takeaddr", true, "Include functions that take the address of their parameters and results.")
var methodflag = flag.Bool("method", true, "Include testing of method calls.")
var inlimitflag = flag.Int("inmax", -1, "Max number of input params.")
var outlimitflag = flag.Int("outmax", -1, "Max number of input params.")
var pragmaflag = flag.String("pragma", "", "Tag generated test routines with pragma //go:<value>.")
var maxfailflag = flag.Int("maxfail", 10, "Maximum runtime failures before test self-terminates")
var stackforceflag = flag.Bool("forcestackgrowth", true, "Use hooks to force stack growth.")
// Debugging options
var verbflag = flag.Int("v", 0, "Verbose trace output level")
// Debugging/testing options. These tell the generator to emit "bad" code so as to
// test the logic for detecting errors and/or minimization (in the fuzz runner).
var emitbadflag = flag.Int("emitbad", 0, "[Testing only] force generator to emit 'bad' code.")
var selbadpkgflag = flag.Int("badpkgidx", 0, "[Testing only] select index of bad package (used with -emitbad)")
var selbadfcnflag = flag.Int("badfcnidx", 0, "[Testing only] select index of bad function (used with -emitbad)")
// Misc options
var goimpflag = flag.Bool("goimports", false, "Run 'goimports' on generated code.")
var randctlflag = flag.Int("randctl", generator.RandCtlChecks|generator.RandCtlPanic, "Wraprand control flag")
func verb(vlevel int, s string, a ...interface{}) {
if *verbflag >= vlevel {
fmt.Printf(s, a...)
fmt.Printf("\n")
}
}
func usage(msg string) {
if len(msg) > 0 {
fmt.Fprintf(os.Stderr, "error: %s\n", msg)
}
fmt.Fprintf(os.Stderr, "usage: fuzz-driver [flags]\n\n")
flag.PrintDefaults()
fmt.Fprintf(os.Stderr, "Example:\n\n")
fmt.Fprintf(os.Stderr, " fuzz-driver -numpkgs=23 -numfcns=19 -seed 10101 -outdir gendir\n\n")
fmt.Fprintf(os.Stderr, " \tgenerates a Go program with 437 test cases (23 packages, each \n")
fmt.Fprintf(os.Stderr, " \twith 19 functions, for a total of 437 funcs total) into a set of\n")
fmt.Fprintf(os.Stderr, " \tsub-directories in 'gendir', using random see 10101\n")
os.Exit(2)
}
func setupTunables() {
tunables := generator.DefaultTunables()
if !*reflectflag {
tunables.DisableReflectionCalls()
}
if !*deferflag {
tunables.DisableDefer()
}
if !*recurflag {
tunables.DisableRecursiveCalls()
}
if !*takeaddrflag {
tunables.DisableTakeAddr()
}
if !*methodflag {
tunables.DisableMethodCalls()
}
if *inlimitflag != -1 {
tunables.LimitInputs(*inlimitflag)
}
if *outlimitflag != -1 {
tunables.LimitOutputs(*outlimitflag)
}
generator.SetTunables(tunables)
}
func main() {
log.SetFlags(0)
log.SetPrefix("fuzz-driver: ")
flag.Parse()
generator.Verbctl = *verbflag
if *outdirflag == "" {
usage("select an output directory with -o flag")
}
verb(1, "in main verblevel=%d", *verbflag)
if *seedflag == -1 {
// user has not selected a specific seed -- pick one.
now := time.Now()
*seedflag = now.UnixNano() % 123456789
verb(0, "selected seed: %d", *seedflag)
}
rand.Seed(*seedflag)
if flag.NArg() != 0 {
usage("unknown extra arguments")
}
verb(1, "tag is %s", *tagflag)
fcnmask, err := generator.ParseMaskString(*fcnmaskflag, "fcn")
if err != nil {
usage(fmt.Sprintf("mangled fcn mask arg: %v", err))
}
pkmask, err := generator.ParseMaskString(*pkmaskflag, "pkg")
if err != nil {
usage(fmt.Sprintf("mangled pkg mask arg: %v", err))
}
verb(2, "pkg mask is %v", pkmask)
verb(2, "fn mask is %v", fcnmask)
verb(1, "starting generation")
setupTunables()
config := generator.GenConfig{
PkgPath: *pkgpathflag,
Tag: *tagflag,
OutDir: *outdirflag,
NumTestPackages: *numpkgflag,
NumTestFunctions: *numfcnflag,
Seed: *seedflag,
Pragma: *pragmaflag,
FcnMask: fcnmask,
PkgMask: pkmask,
MaxFail: *maxfailflag,
ForceStackGrowth: *stackforceflag,
RandCtl: *randctlflag,
RunGoImports: *goimpflag,
EmitBad: *emitbadflag,
BadPackageIdx: *selbadpkgflag,
BadFuncIdx: *selbadfcnflag,
}
errs := generator.Generate(config)
if errs != 0 {
log.Fatal("errors during generation")
}
verb(1, "... files written to directory %s", *outdirflag)
verb(1, "leaving main")
}

View File

@ -0,0 +1,73 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"os"
"os/exec"
"path/filepath"
"runtime"
"testing"
"golang.org/x/tools/internal/testenv"
)
// buildDriver builds the fuzz-driver executable, returning its path.
func buildDriver(t *testing.T) string {
t.Helper()
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
return ""
}
bindir := filepath.Join(t.TempDir(), "bin")
err := os.Mkdir(bindir, os.ModePerm)
if err != nil {
t.Fatal(err)
}
binary := filepath.Join(bindir, "driver")
if runtime.GOOS == "windows" {
binary += ".exe"
}
cmd := exec.Command("go", "build", "-o", binary)
if err := cmd.Run(); err != nil {
t.Fatalf("Building fuzz-driver: %v", err)
}
return binary
}
func TestEndToEndIntegration(t *testing.T) {
testenv.NeedsTool(t, "go")
td := t.TempDir()
// Build the fuzz-driver binary.
// Note: if more tests are added to this package, move this to single setup fcn, so
// that we don't have to redo the build each time.
binary := buildDriver(t)
// Kick off a run.
gendir := filepath.Join(td, "gen")
args := []string{"-numfcns", "3", "-numpkgs", "1", "-seed", "101", "-outdir", gendir}
c := exec.Command(binary, args...)
b, err := c.CombinedOutput()
if err != nil {
t.Fatalf("error invoking fuzz-driver: %v\n%s", err, b)
}
found := ""
walker := func(path string, info os.FileInfo, err error) error {
found = found + ":" + info.Name()
return nil
}
// Make sure it emits something.
err2 := filepath.Walk(gendir, walker)
if err2 != nil {
t.Fatalf("error from filepath.Walk: %v", err2)
}
const expected = ":gen:genCaller0:genCaller0.go:genChecker0:genChecker0.go:genMain.go:genUtils:genUtils.go:go.mod"
if found != expected {
t.Errorf("walk of generated code: got %s want %s", found, expected)
}
}

View File

@ -0,0 +1,145 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"runtime"
"strings"
"testing"
"golang.org/x/tools/internal/testenv"
)
func canRace(t *testing.T) bool {
_, err := exec.Command("go", "run", "-race", "./testdata/himom.go").CombinedOutput()
return err == nil
}
// buildRunner builds the fuzz-runner executable, returning its path.
func buildRunner(t *testing.T) string {
bindir := filepath.Join(t.TempDir(), "bin")
err := os.Mkdir(bindir, os.ModePerm)
if err != nil {
t.Fatal(err)
}
binary := filepath.Join(bindir, "runner")
if runtime.GOOS == "windows" {
binary += ".exe"
}
cmd := exec.Command("go", "build", "-o", binary)
if err := cmd.Run(); err != nil {
t.Fatalf("Building fuzz-runner: %v", err)
}
return binary
}
// TestRunner builds the binary, then kicks off a collection of sub-tests that invoke it.
func TestRunner(t *testing.T) {
testenv.NeedsTool(t, "go")
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
}
binaryPath := buildRunner(t)
// Sub-tests using the binary built above.
t.Run("Basic", func(t *testing.T) { testBasic(t, binaryPath) })
t.Run("Race", func(t *testing.T) { testRace(t, binaryPath) })
t.Run("Minimization1", func(t *testing.T) { testMinimization1(t, binaryPath) })
t.Run("Minimization2", func(t *testing.T) { testMinimization2(t, binaryPath) })
}
func testBasic(t *testing.T, binaryPath string) {
t.Parallel()
args := []string{"-numit=1", "-numfcns=1", "-numpkgs=1", "-seed=103", "-cleancache=0"}
c := exec.Command(binaryPath, args...)
b, err := c.CombinedOutput()
t.Logf("%s\n", b)
if err != nil {
t.Fatalf("error invoking fuzz-runner: %v", err)
}
}
func testRace(t *testing.T, binaryPath string) {
t.Parallel()
// For this test to work, the current test platform has to support the
// race detector. Check to see if that is the case by running a very
// simple Go program through it.
if !canRace(t) {
t.Skip("current platform does not appear to support the race detector")
}
args := []string{"-v=1", "-numit=1", "-race", "-numfcns=3", "-numpkgs=3", "-seed=987", "-cleancache=0"}
c := exec.Command(binaryPath, args...)
b, err := c.CombinedOutput()
t.Logf("%s\n", b)
if err != nil {
t.Fatalf("error invoking fuzz-runner: %v", err)
}
}
func testMinimization1(t *testing.T, binaryPath string) {
if binaryPath == "" {
t.Skipf("No runner binary")
}
t.Parallel()
// Fire off the runner passing it -emitbad=1, so that the generated code
// contains illegal Go code (which will force the build to fail). Verify that
// it does fail, that the error reflects the nature of the failure, and that
// we can minimize the error down to a single package.
args := []string{"-emitbad=1", "-badfcnidx=2", "-badpkgidx=2",
"-forcetmpclean", "-cleancache=0",
"-numit=1", "-numfcns=3", "-numpkgs=3", "-seed=909"}
invocation := fmt.Sprintf("%s %v", binaryPath, args)
c := exec.Command(binaryPath, args...)
b, err := c.CombinedOutput()
t.Logf("%s\n", b)
if err == nil {
t.Fatalf("unexpected pass of fuzz-runner (invocation %q): %v", invocation, err)
}
result := string(b)
if !strings.Contains(result, "syntax error") {
t.Fatalf("-emitbad=1 did not trigger syntax error (invocation %q): output: %s", invocation, result)
}
if !strings.Contains(result, "package minimization succeeded: found bad pkg 2") {
t.Fatalf("failed to minimize package (invocation %q): output: %s", invocation, result)
}
if !strings.Contains(result, "function minimization succeeded: found bad fcn 2") {
t.Fatalf("failed to minimize package (invocation %q): output: %s", invocation, result)
}
}
func testMinimization2(t *testing.T, binaryPath string) {
if binaryPath == "" {
t.Skipf("No runner binary")
}
t.Parallel()
// Fire off the runner passing it -emitbad=2, so that the
// generated code forces a runtime error. Verify that it does
// fail, and that the error is reflective.
args := []string{"-emitbad=2", "-badfcnidx=1", "-badpkgidx=1",
"-forcetmpclean", "-cleancache=0",
"-numit=1", "-numfcns=3", "-numpkgs=3", "-seed=55909"}
invocation := fmt.Sprintf("%s %v", binaryPath, args)
c := exec.Command(binaryPath, args...)
b, err := c.CombinedOutput()
t.Logf("%s\n", b)
if err == nil {
t.Fatalf("unexpected pass of fuzz-runner (invocation %q): %v", invocation, err)
}
result := string(b)
if !strings.Contains(result, "Error: fail") || !strings.Contains(result, "Checker1.Test1") {
t.Fatalf("-emitbad=2 did not trigger runtime error (invocation %q): output: %s", invocation, result)
}
if !strings.Contains(result, "package minimization succeeded: found bad pkg 1") {
t.Fatalf("failed to minimize package (invocation %q): output: %s", invocation, result)
}
if !strings.Contains(result, "function minimization succeeded: found bad fcn 1") {
t.Fatalf("failed to minimize package (invocation %q): output: %s", invocation, result)
}
}

View File

@ -0,0 +1,443 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Program for performing test runs using "fuzz-driver".
// Main loop iteratively runs "fuzz-driver" to create a corpus,
// then builds and runs the code. If a failure in the run is
// detected, then a testcase minimization phase kicks in.
package main
import (
"flag"
"fmt"
"io/ioutil"
"log"
"os"
"os/exec"
"path/filepath"
"runtime"
"strconv"
"strings"
"time"
generator "golang.org/x/tools/cmd/signature-fuzzer/internal/fuzz-generator"
)
const pkName = "fzTest"
// Basic options
var verbflag = flag.Int("v", 0, "Verbose trace output level")
var loopitflag = flag.Int("numit", 10, "Number of main loop iterations to run")
var seedflag = flag.Int64("seed", -1, "Random seed")
var execflag = flag.Bool("execdriver", false, "Exec fuzz-driver binary instead of invoking generator directly")
var numpkgsflag = flag.Int("numpkgs", 50, "Number of test packages")
var numfcnsflag = flag.Int("numfcns", 20, "Number of test functions per package.")
// Debugging/testing options. These tell the generator to emit "bad" code so as to
// test the logic for detecting errors and/or minimization.
var emitbadflag = flag.Int("emitbad", -1, "[Testing only] force generator to emit 'bad' code.")
var selbadpkgflag = flag.Int("badpkgidx", 0, "[Testing only] select index of bad package (used with -emitbad)")
var selbadfcnflag = flag.Int("badfcnidx", 0, "[Testing only] select index of bad function (used with -emitbad)")
var forcetmpcleanflag = flag.Bool("forcetmpclean", false, "[Testing only] force cleanup of temp dir")
var cleancacheflag = flag.Bool("cleancache", true, "[Testing only] don't clean the go cache")
var raceflag = flag.Bool("race", false, "[Testing only] build generated code with -race")
func verb(vlevel int, s string, a ...interface{}) {
if *verbflag >= vlevel {
fmt.Printf(s, a...)
fmt.Printf("\n")
}
}
func warn(s string, a ...interface{}) {
fmt.Fprintf(os.Stderr, s, a...)
fmt.Fprintf(os.Stderr, "\n")
}
func fatal(s string, a ...interface{}) {
fmt.Fprintf(os.Stderr, s, a...)
fmt.Fprintf(os.Stderr, "\n")
os.Exit(1)
}
type config struct {
generator.GenConfig
tmpdir string
gendir string
buildOutFile string
runOutFile string
gcflags string
nerrors int
}
func usage(msg string) {
if len(msg) > 0 {
fmt.Fprintf(os.Stderr, "error: %s\n", msg)
}
fmt.Fprintf(os.Stderr, "usage: fuzz-runner [flags]\n\n")
flag.PrintDefaults()
fmt.Fprintf(os.Stderr, "Example:\n\n")
fmt.Fprintf(os.Stderr, " fuzz-runner -numit=500 -numpkgs=11 -numfcns=13 -seed=10101\n\n")
fmt.Fprintf(os.Stderr, " \tRuns 500 rounds of test case generation\n")
fmt.Fprintf(os.Stderr, " \tusing random see 10101, in each round emitting\n")
fmt.Fprintf(os.Stderr, " \t11 packages each with 13 function pairs.\n")
os.Exit(2)
}
// docmd executes the specified command in the dir given and pipes the
// output to stderr. return status is 0 if command passed, 1
// otherwise.
func docmd(cmd []string, dir string) int {
verb(2, "docmd: %s", strings.Join(cmd, " "))
c := exec.Command(cmd[0], cmd[1:]...)
if dir != "" {
c.Dir = dir
}
b, err := c.CombinedOutput()
st := 0
if err != nil {
warn("error executing cmd %s: %v",
strings.Join(cmd, " "), err)
st = 1
}
os.Stderr.Write(b)
return st
}
// docodmout forks and execs command 'cmd' in dir 'dir', redirecting
// stderr and stdout from the execution to file 'outfile'.
func docmdout(cmd []string, dir string, outfile string) int {
of, err := os.OpenFile(outfile, os.O_RDWR|os.O_CREATE|os.O_TRUNC, 0644)
if err != nil {
fatal("opening outputfile %s: %v", outfile, err)
}
c := exec.Command(cmd[0], cmd[1:]...)
defer of.Close()
if dir != "" {
verb(2, "setting cmd.Dir to %s", dir)
c.Dir = dir
}
verb(2, "docmdout: %s > %s", strings.Join(cmd, " "), outfile)
c.Stdout = of
c.Stderr = of
err = c.Run()
st := 0
if err != nil {
warn("error executing cmd %s: %v",
strings.Join(cmd, " "), err)
st = 1
}
return st
}
// gen is the main hook for kicking off code generation. For
// non-minimization runs, 'singlepk' and 'singlefn' will both be -1
// (indicating that we want all functions and packages to be
// generated). If 'singlepk' is set to a non-negative value, then
// code generation will be restricted to the single package with that
// index (as a try at minimization), similarly with 'singlefn'
// restricting the codegen to a single specified function.
func (c *config) gen(singlepk int, singlefn int) {
// clean the output dir
verb(2, "cleaning outdir %s", c.gendir)
if err := os.RemoveAll(c.gendir); err != nil {
fatal("error cleaning gen dir %s: %v", c.gendir, err)
}
// emit code into the output dir. Here we either invoke the
// generator directly, or invoke fuzz-driver if -execflag is
// set. If the code generation process itself fails, this is
// typically a bug in the fuzzer itself, so it gets reported
// as a fatal error.
if *execflag {
args := []string{"fuzz-driver",
"-numpkgs", strconv.Itoa(c.NumTestPackages),
"-numfcns", strconv.Itoa(c.NumTestFunctions),
"-seed", strconv.Itoa(int(c.Seed)),
"-outdir", c.OutDir,
"-pkgpath", pkName,
"-maxfail", strconv.Itoa(c.MaxFail)}
if singlepk != -1 {
args = append(args, "-pkgmask", strconv.Itoa(singlepk))
}
if singlefn != -1 {
args = append(args, "-fcnmask", strconv.Itoa(singlefn))
}
if *emitbadflag != 0 {
args = append(args, "-emitbad", strconv.Itoa(*emitbadflag),
"-badpkgidx", strconv.Itoa(*selbadpkgflag),
"-badfcnidx", strconv.Itoa(*selbadfcnflag))
}
verb(1, "invoking fuzz-driver with args: %v", args)
st := docmd(args, "")
if st != 0 {
fatal("fatal error: generation failed, cmd was: %v", args)
}
} else {
if singlepk != -1 {
c.PkgMask = map[int]int{singlepk: 1}
}
if singlefn != -1 {
c.FcnMask = map[int]int{singlefn: 1}
}
verb(1, "invoking generator.Generate with config: %v", c.GenConfig)
errs := generator.Generate(c.GenConfig)
if errs != 0 {
log.Fatal("errors during generation")
}
}
}
// action performs a selected action/command in the generated code dir.
func (c *config) action(cmd []string, outfile string, emitout bool) int {
st := docmdout(cmd, c.gendir, outfile)
if emitout {
content, err := ioutil.ReadFile(outfile)
if err != nil {
log.Fatal(err)
}
fmt.Fprintf(os.Stderr, "%s", content)
}
return st
}
func binaryName() string {
if runtime.GOOS == "windows" {
return pkName + ".exe"
} else {
return "./" + pkName
}
}
// build builds a generated corpus of Go code. If 'emitout' is set, then dump out the
// results of the build after it completes (during minimization emitout is set to false,
// since there is no need to see repeated errors).
func (c *config) build(emitout bool) int {
// Issue a build of the generated code.
c.buildOutFile = filepath.Join(c.tmpdir, "build.err.txt")
cmd := []string{"go", "build", "-o", binaryName()}
if c.gcflags != "" {
cmd = append(cmd, "-gcflags=all="+c.gcflags)
}
if *raceflag {
cmd = append(cmd, "-race")
}
cmd = append(cmd, ".")
verb(1, "build command is: %v", cmd)
return c.action(cmd, c.buildOutFile, emitout)
}
// run invokes a binary built from a generated corpus of Go code. If
// 'emitout' is set, then dump out the results of the run after it
// completes.
func (c *config) run(emitout bool) int {
// Issue a run of the generated code.
c.runOutFile = filepath.Join(c.tmpdir, "run.err.txt")
cmd := []string{filepath.Join(c.gendir, binaryName())}
verb(1, "run command is: %v", cmd)
return c.action(cmd, c.runOutFile, emitout)
}
type minimizeMode int
const (
minimizeBuildFailure = iota
minimizeRuntimeFailure
)
// minimize tries to minimize a failing scenario down to a single
// package and/or function if possible. This is done using an
// iterative search. Here 'minimizeMode' tells us whether we're
// looking for a compile-time error or a runtime error.
func (c *config) minimize(mode minimizeMode) int {
verb(0, "... starting minimization for failed directory %s", c.gendir)
foundPkg := -1
foundFcn := -1
// Locate bad package. Uses brute-force linear search, could do better...
for pidx := 0; pidx < c.NumTestPackages; pidx++ {
verb(1, "minimization: trying package %d", pidx)
c.gen(pidx, -1)
st := c.build(false)
if mode == minimizeBuildFailure {
if st != 0 {
// Found.
foundPkg = pidx
c.nerrors++
break
}
} else {
if st != 0 {
warn("run minimization: unexpected build failed while searching for bad pkg")
return 1
}
st := c.run(false)
if st != 0 {
// Found.
c.nerrors++
verb(1, "run minimization found bad package: %d", pidx)
foundPkg = pidx
break
}
}
}
if foundPkg == -1 {
verb(0, "** minimization failed, could not locate bad package")
return 1
}
warn("package minimization succeeded: found bad pkg %d", foundPkg)
// clean unused packages
for pidx := 0; pidx < c.NumTestPackages; pidx++ {
if pidx != foundPkg {
chp := filepath.Join(c.gendir, fmt.Sprintf("%s%s%d", c.Tag, generator.CheckerName, pidx))
if err := os.RemoveAll(chp); err != nil {
fatal("failed to clean pkg subdir %s: %v", chp, err)
}
clp := filepath.Join(c.gendir, fmt.Sprintf("%s%s%d", c.Tag, generator.CallerName, pidx))
if err := os.RemoveAll(clp); err != nil {
fatal("failed to clean pkg subdir %s: %v", clp, err)
}
}
}
// Locate bad function. Again, brute force.
for fidx := 0; fidx < c.NumTestFunctions; fidx++ {
c.gen(foundPkg, fidx)
st := c.build(false)
if mode == minimizeBuildFailure {
if st != 0 {
// Found.
verb(1, "build minimization found bad function: %d", fidx)
foundFcn = fidx
break
}
} else {
if st != 0 {
warn("run minimization: unexpected build failed while searching for bad fcn")
return 1
}
st := c.run(false)
if st != 0 {
// Found.
verb(1, "run minimization found bad function: %d", fidx)
foundFcn = fidx
break
}
}
// not the function we want ... continue the hunt
}
if foundFcn == -1 {
verb(0, "** function minimization failed, could not locate bad function")
return 1
}
warn("function minimization succeeded: found bad fcn %d", foundFcn)
return 0
}
// cleanTemp removes the temp dir we've been working with.
func (c *config) cleanTemp() {
if !*forcetmpcleanflag {
if c.nerrors != 0 {
verb(1, "preserving temp dir %s", c.tmpdir)
return
}
}
verb(1, "cleaning temp dir %s", c.tmpdir)
os.RemoveAll(c.tmpdir)
}
// perform is the top level driver routine for the program, containing the
// main loop. Each iteration of the loop performs a generate/build/run
// sequence, and then updates the seed afterwards if no failure is found.
// If a failure is detected, we try to minimize it and then return without
// attempting any additional tests.
func (c *config) perform() int {
defer c.cleanTemp()
// Main loop
for iter := 0; iter < *loopitflag; iter++ {
if iter != 0 && iter%50 == 0 {
// Note: cleaning the Go cache periodically is
// pretty much a requirement if you want to do
// things like overnight runs of the fuzzer,
// but it is also a very unfriendly thing do
// to if we're executing as part of a unit
// test run (in which case there may be other
// tests running in parallel with this
// one). Check the "cleancache" flag before
// doing this.
if *cleancacheflag {
docmd([]string{"go", "clean", "-cache"}, "")
}
}
verb(0, "... begin iteration %d with current seed %d", iter, c.Seed)
c.gen(-1, -1)
st := c.build(true)
if st != 0 {
c.minimize(minimizeBuildFailure)
return 1
}
st = c.run(true)
if st != 0 {
c.minimize(minimizeRuntimeFailure)
return 1
}
// update seed so that we get different code on the next iter.
c.Seed += 101
}
return 0
}
func main() {
log.SetFlags(0)
log.SetPrefix("fuzz-runner: ")
flag.Parse()
if flag.NArg() != 0 {
usage("unknown extra arguments")
}
verb(1, "in main, verblevel=%d", *verbflag)
tmpdir, err := ioutil.TempDir("", "fuzzrun")
if err != nil {
fatal("creation of tempdir failed: %v", err)
}
gendir := filepath.Join(tmpdir, "fuzzTest")
// select starting seed
if *seedflag == -1 {
now := time.Now()
*seedflag = now.UnixNano() % 123456789
}
// set up params for this run
c := &config{
GenConfig: generator.GenConfig{
NumTestPackages: *numpkgsflag, // 100
NumTestFunctions: *numfcnsflag, // 20
Seed: *seedflag,
OutDir: gendir,
Pragma: "-maxfail=9999",
PkgPath: pkName,
EmitBad: *emitbadflag,
BadPackageIdx: *selbadpkgflag,
BadFuncIdx: *selbadfcnflag,
},
tmpdir: tmpdir,
gendir: gendir,
}
// kick off the main loop.
st := c.perform()
// done
verb(1, "leaving main, num errors=%d", c.nerrors)
os.Exit(st)
}

View File

@ -0,0 +1,9 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package main
func main() {
println("hi mom!")
}

View File

@ -0,0 +1,108 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"fmt"
)
// arrayparm describes a parameter of array type; it implements the
// "parm" interface.
type arrayparm struct {
aname string
qname string
nelements uint8
eltype parm
slice bool
isBlank
addrTakenHow
isGenValFunc
skipCompare
}
func (p arrayparm) IsControl() bool {
return false
}
func (p arrayparm) TypeName() string {
return p.aname
}
func (p arrayparm) QualName() string {
return p.qname
}
func (p arrayparm) Declare(b *bytes.Buffer, prefix string, suffix string, caller bool) {
n := p.aname
if caller {
n = p.qname
}
b.WriteString(fmt.Sprintf("%s %s%s", prefix, n, suffix))
}
func (p arrayparm) String() string {
return fmt.Sprintf("%s %d-element array of %s", p.aname, p.nelements, p.eltype.String())
}
func (p arrayparm) GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int) {
var buf bytes.Buffer
verb(5, "arrayparm.GenValue(%d)", value)
n := p.aname
if caller {
n = p.qname
}
buf.WriteString(fmt.Sprintf("%s{", n))
for i := 0; i < int(p.nelements); i++ {
var valstr string
valstr, value = s.GenValue(f, p.eltype, value, caller)
writeCom(&buf, i)
buf.WriteString(valstr)
}
buf.WriteString("}")
return buf.String(), value
}
func (p arrayparm) GenElemRef(elidx int, path string) (string, parm) {
ene := p.eltype.NumElements()
verb(4, "begin GenElemRef(%d,%s) on %s ene %d", elidx, path, p.String(), ene)
// For empty arrays, convention is to return empty string
if ene == 0 {
return "", &p
}
// Find slot within array of element of interest
slot := elidx / ene
// If this is the element we're interested in, return it
if ene == 1 {
verb(4, "hit scalar element")
epath := fmt.Sprintf("%s[%d]", path, slot)
if path == "_" || p.IsBlank() {
epath = "_"
}
return epath, p.eltype
}
verb(4, "recur slot=%d GenElemRef(%d,...)", slot, elidx-(slot*ene))
// Otherwise our victim is somewhere inside the slot
ppath := fmt.Sprintf("%s[%d]", path, slot)
if p.IsBlank() {
ppath = "_"
}
return p.eltype.GenElemRef(elidx-(slot*ene), ppath)
}
func (p arrayparm) NumElements() int {
return p.eltype.NumElements() * int(p.nelements)
}
func (p arrayparm) HasPointer() bool {
return p.eltype.HasPointer() || p.slice
}

View File

@ -0,0 +1,322 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"os"
"os/exec"
"path/filepath"
"runtime"
"testing"
"golang.org/x/tools/internal/testenv"
)
func mkGenState() *genstate {
return &genstate{
GenConfig: GenConfig{
Tag: "gen",
OutDir: "/tmp",
NumTestPackages: 1,
NumTestFunctions: 10,
},
ipref: "foo/",
derefFuncs: make(map[string]string),
assignFuncs: make(map[string]string),
allocFuncs: make(map[string]string),
globVars: make(map[string]string),
}
}
func TestBasic(t *testing.T) {
checkTunables(tunables)
s := mkGenState()
for i := 0; i < 1000; i++ {
s.wr = NewWrapRand(int64(i), RandCtlChecks|RandCtlPanic)
fp := s.GenFunc(i, i)
var buf bytes.Buffer
var b *bytes.Buffer = &buf
wr := NewWrapRand(int64(i), RandCtlChecks|RandCtlPanic)
s.wr = wr
s.emitCaller(fp, b, i)
s.wr = NewWrapRand(int64(i), RandCtlChecks|RandCtlPanic)
s.emitChecker(fp, b, i, true)
wr.Check(s.wr)
}
if s.errs != 0 {
t.Errorf("%d errors during Generate", s.errs)
}
}
func TestMoreComplicated(t *testing.T) {
saveit := tunables
defer func() { tunables = saveit }()
checkTunables(tunables)
s := mkGenState()
for i := 0; i < 10000; i++ {
s.wr = NewWrapRand(int64(i), RandCtlChecks|RandCtlPanic)
fp := s.GenFunc(i, i)
var buf bytes.Buffer
var b *bytes.Buffer = &buf
wr := NewWrapRand(int64(i), RandCtlChecks|RandCtlPanic)
s.wr = wr
s.emitCaller(fp, b, i)
verb(1, "finished iter %d caller", i)
s.wr = NewWrapRand(int64(i), RandCtlChecks|RandCtlPanic)
s.emitChecker(fp, b, i, true)
verb(1, "finished iter %d checker", i)
wr.Check(s.wr)
if s.errs != 0 {
t.Errorf("%d errors during Generate iter %d", s.errs, i)
}
}
}
func TestIsBuildable(t *testing.T) {
testenv.NeedsTool(t, "go")
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
}
td := t.TempDir()
verb(1, "generating into temp dir %s", td)
checkTunables(tunables)
pack := filepath.Base(td)
s := GenConfig{
Tag: "x",
OutDir: td,
PkgPath: pack,
NumTestFunctions: 10,
NumTestPackages: 10,
MaxFail: 10,
RandCtl: RandCtlChecks | RandCtlPanic,
}
errs := Generate(s)
if errs != 0 {
t.Errorf("%d errors during Generate", errs)
}
verb(1, "building %s\n", td)
cmd := exec.Command("go", "run", ".")
cmd.Dir = td
coutput, cerr := cmd.CombinedOutput()
if cerr != nil {
t.Errorf("go build command failed: %s\n", string(coutput))
}
verb(1, "output is: %s\n", string(coutput))
}
// TestExhaustive does a series of code genreation runs, starting with
// (relatively) simple code and then getting progressively more
// complex (more params, deeper structs, turning on additional
// features such as address-taken vars and reflect testing). The
// intent here is mainly to insure that the tester still works if you
// turn things on and off, e.g. that each feature is separately
// controllable and not linked to other things.
func TestExhaustive(t *testing.T) {
testenv.NeedsTool(t, "go")
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
}
if testing.Short() {
t.Skip("skipping test in short mode.")
}
td := t.TempDir()
verb(1, "generating into temp dir %s", td)
scenarios := []struct {
name string
adjuster func()
}{
{
"minimal",
func() {
tunables.nParmRange = 3
tunables.nReturnRange = 3
tunables.structDepth = 1
tunables.recurPerc = 0
tunables.methodPerc = 0
tunables.doReflectCall = false
tunables.doDefer = false
tunables.takeAddress = false
tunables.doFuncCallValues = false
tunables.doSkipCompare = false
checkTunables(tunables)
},
},
{
"moreparms",
func() {
tunables.nParmRange = 15
tunables.nReturnRange = 7
tunables.structDepth = 3
checkTunables(tunables)
},
},
{
"addrecur",
func() {
tunables.recurPerc = 20
checkTunables(tunables)
},
},
{
"addmethod",
func() {
tunables.methodPerc = 25
tunables.pointerMethodCallPerc = 30
checkTunables(tunables)
},
},
{
"addtakeaddr",
func() {
tunables.takeAddress = true
tunables.takenFraction = 20
checkTunables(tunables)
},
},
{
"addreflect",
func() {
tunables.doReflectCall = true
checkTunables(tunables)
},
},
{
"adddefer",
func() {
tunables.doDefer = true
checkTunables(tunables)
},
},
{
"addfuncval",
func() {
tunables.doFuncCallValues = true
checkTunables(tunables)
},
},
{
"addfuncval",
func() {
tunables.doSkipCompare = true
checkTunables(tunables)
},
},
}
// Loop over scenarios and make sure each one works properly.
for i, s := range scenarios {
t.Logf("running %s\n", s.name)
s.adjuster()
os.RemoveAll(td)
pack := filepath.Base(td)
c := GenConfig{
Tag: "x",
OutDir: td,
PkgPath: pack,
NumTestFunctions: 10,
NumTestPackages: 10,
Seed: int64(i + 9),
MaxFail: 10,
RandCtl: RandCtlChecks | RandCtlPanic,
}
errs := Generate(c)
if errs != 0 {
t.Errorf("%d errors during scenarios %q Generate", errs, s.name)
}
cmd := exec.Command("go", "run", ".")
cmd.Dir = td
coutput, cerr := cmd.CombinedOutput()
if cerr != nil {
t.Fatalf("run failed for scenario %q: %s\n", s.name, string(coutput))
}
verb(1, "output is: %s\n", string(coutput))
}
}
func TestEmitBadBuildFailure(t *testing.T) {
testenv.NeedsTool(t, "go")
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
}
td := t.TempDir()
verb(1, "generating into temp dir %s", td)
checkTunables(tunables)
pack := filepath.Base(td)
s := GenConfig{
Tag: "x",
OutDir: td,
PkgPath: pack,
NumTestFunctions: 10,
NumTestPackages: 10,
MaxFail: 10,
RandCtl: RandCtlChecks | RandCtlPanic,
EmitBad: 1,
}
errs := Generate(s)
if errs != 0 {
t.Errorf("%d errors during Generate", errs)
}
cmd := exec.Command("go", "build", ".")
cmd.Dir = td
coutput, cerr := cmd.CombinedOutput()
if cerr == nil {
t.Errorf("go build command passed, expected failure. output: %s\n", string(coutput))
}
}
func TestEmitBadRunFailure(t *testing.T) {
testenv.NeedsTool(t, "go")
if runtime.GOOS == "android" {
t.Skipf("the dependencies are not available on android")
}
td := t.TempDir()
verb(1, "generating into temp dir %s", td)
checkTunables(tunables)
pack := filepath.Base(td)
s := GenConfig{
Tag: "x",
OutDir: td,
PkgPath: pack,
NumTestFunctions: 10,
NumTestPackages: 10,
MaxFail: 10,
RandCtl: RandCtlChecks | RandCtlPanic,
EmitBad: 2,
}
errs := Generate(s)
if errs != 0 {
t.Errorf("%d errors during Generate", errs)
}
// build
cmd := exec.Command("go", "build", ".")
cmd.Dir = td
coutput, cerr := cmd.CombinedOutput()
if cerr != nil {
t.Fatalf("build failed: %s\n", string(coutput))
}
// run
cmd = exec.Command("./" + pack)
cmd.Dir = td
coutput, cerr = cmd.CombinedOutput()
if cerr == nil {
t.Fatalf("run passed, expected failure -- run output: %s", string(coutput))
}
}

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,91 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"fmt"
)
// mapparm describes a parameter of map type; it implements the
// "parm" interface.
type mapparm struct {
aname string
qname string
keytype parm
valtype parm
keytmp string
isBlank
addrTakenHow
isGenValFunc
skipCompare
}
func (p mapparm) IsControl() bool {
return false
}
func (p mapparm) TypeName() string {
return p.aname
}
func (p mapparm) QualName() string {
return p.qname
}
func (p mapparm) Declare(b *bytes.Buffer, prefix string, suffix string, caller bool) {
n := p.aname
if caller {
n = p.qname
}
b.WriteString(fmt.Sprintf("%s %s%s", prefix, n, suffix))
}
func (p mapparm) String() string {
return fmt.Sprintf("%s map[%s]%s", p.aname,
p.keytype.String(), p.valtype.String())
}
func (p mapparm) GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int) {
var buf bytes.Buffer
verb(5, "mapparm.GenValue(%d)", value)
n := p.aname
if caller {
n = p.qname
}
buf.WriteString(fmt.Sprintf("%s{", n))
buf.WriteString(p.keytmp + ": ")
var valstr string
valstr, value = s.GenValue(f, p.valtype, value, caller)
buf.WriteString(valstr + "}")
return buf.String(), value
}
func (p mapparm) GenElemRef(elidx int, path string) (string, parm) {
vne := p.valtype.NumElements()
verb(4, "begin GenElemRef(%d,%s) on %s %d", elidx, path, p.String(), vne)
ppath := fmt.Sprintf("%s[mkt.%s]", path, p.keytmp)
// otherwise dig into the value
verb(4, "recur GenElemRef(%d,...)", elidx)
// Otherwise our victim is somewhere inside the value
if p.IsBlank() {
ppath = "_"
}
return p.valtype.GenElemRef(elidx, ppath)
}
func (p mapparm) NumElements() int {
return p.valtype.NumElements()
}
func (p mapparm) HasPointer() bool {
return true
}

View File

@ -0,0 +1,144 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"fmt"
"math"
)
// numparm describes a numeric parameter type; it implements the
// "parm" interface.
type numparm struct {
tag string
widthInBits uint32
ctl bool
isBlank
addrTakenHow
isGenValFunc
skipCompare
}
var f32parm *numparm = &numparm{
tag: "float",
widthInBits: uint32(32),
ctl: false,
}
var f64parm *numparm = &numparm{
tag: "float",
widthInBits: uint32(64),
ctl: false,
}
func (p numparm) TypeName() string {
if p.tag == "byte" {
return "byte"
}
return fmt.Sprintf("%s%d", p.tag, p.widthInBits)
}
func (p numparm) QualName() string {
return p.TypeName()
}
func (p numparm) String() string {
if p.tag == "byte" {
return "byte"
}
ctl := ""
if p.ctl {
ctl = " [ctl=yes]"
}
return fmt.Sprintf("%s%s", p.TypeName(), ctl)
}
func (p numparm) NumElements() int {
return 1
}
func (p numparm) IsControl() bool {
return p.ctl
}
func (p numparm) GenElemRef(elidx int, path string) (string, parm) {
return path, &p
}
func (p numparm) Declare(b *bytes.Buffer, prefix string, suffix string, caller bool) {
t := fmt.Sprintf("%s%d%s", p.tag, p.widthInBits, suffix)
if p.tag == "byte" {
t = fmt.Sprintf("%s%s", p.tag, suffix)
}
b.WriteString(prefix + " " + t)
}
func (p numparm) genRandNum(s *genstate, value int) (string, int) {
which := uint8(s.wr.Intn(int64(100)))
if p.tag == "int" {
var v int64
if which < 3 {
// max
v = (1 << (p.widthInBits - 1)) - 1
} else if which < 5 {
// min
v = (-1 << (p.widthInBits - 1))
} else {
nrange := int64(1 << (p.widthInBits - 2))
v = s.wr.Intn(nrange)
if value%2 != 0 {
v = -v
}
}
return fmt.Sprintf("%s%d(%d)", p.tag, p.widthInBits, v), value + 1
}
if p.tag == "uint" || p.tag == "byte" {
nrange := int64(1 << (p.widthInBits - 2))
v := s.wr.Intn(nrange)
if p.tag == "byte" {
return fmt.Sprintf("%s(%d)", p.tag, v), value + 1
}
return fmt.Sprintf("%s%d(0x%x)", p.tag, p.widthInBits, v), value + 1
}
if p.tag == "float" {
if p.widthInBits == 32 {
rf := s.wr.Float32() * (math.MaxFloat32 / 4)
if value%2 != 0 {
rf = -rf
}
return fmt.Sprintf("%s%d(%v)", p.tag, p.widthInBits, rf), value + 1
}
if p.widthInBits == 64 {
return fmt.Sprintf("%s%d(%v)", p.tag, p.widthInBits,
s.wr.NormFloat64()), value + 1
}
panic("unknown float type")
}
if p.tag == "complex" {
if p.widthInBits == 64 {
f1, v2 := f32parm.genRandNum(s, value)
f2, v3 := f32parm.genRandNum(s, v2)
return fmt.Sprintf("complex(%s,%s)", f1, f2), v3
}
if p.widthInBits == 128 {
f1, v2 := f64parm.genRandNum(s, value)
f2, v3 := f64parm.genRandNum(s, v2)
return fmt.Sprintf("complex(%v,%v)", f1, f2), v3
}
panic("unknown complex type")
}
panic("unknown numeric type")
}
func (p numparm) GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int) {
r, nv := p.genRandNum(s, value)
verb(5, "numparm.GenValue(%d) = %s", value, r)
return r, nv
}
func (p numparm) HasPointer() bool {
return false
}

View File

@ -0,0 +1,216 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"fmt"
"os"
"sort"
)
// parm is an interface describing an abstract parameter var or return
// var; there will be concrete types of various sorts that implement
// this interface.
type parm interface {
// Declare emits text containing a declaration of this param
// or return var into the specified buffer. Prefix is a tag to
// prepend before the declaration (for example a variable
// name) followed by a space; suffix is an arbitrary string to
// tack onto the end of the param's type text. Here 'caller'
// is set to true if we're emitting the caller part of a test
// pair as opposed to the checker.
Declare(b *bytes.Buffer, prefix string, suffix string, caller bool)
// GenElemRef returns a pair [X,Y] corresponding to a
// component piece of some composite parm, where X is a string
// forming the reference (ex: ".field" if we're picking out a
// struct field) and Y is a parm object corresponding to the
// type of the element.
GenElemRef(elidx int, path string) (string, parm)
// GenValue constructs a new concrete random value appropriate
// for the type in question and returns it, along with a
// sequence number indicating how many random decisions we had
// to make. Here "s" is the current generator state, "f" is
// the current function we're emitting, value is a sequence
// number indicating how many random decisions have been made
// up until this point, and 'caller' is set to true if we're
// emitting the caller part of a test pair as opposed to the
// checker. Return value is a pair [V,I] where V is the text
// if the value, and I is a new sequence number reflecting any
// additional random choices we had to make. For example, if
// the parm is something like "type Foo struct { f1 int32; f2
// float64 }" then we might expect GenValue to emit something
// like "Foo{int32(-9), float64(123.123)}".
GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int)
// IsControl returns true if this specific param has been marked
// as the single param that controls recursion for a recursive
// checker function. The test code doesn't check this param for a specific
// value, but instead returns early if it has value 0 or decrements it
// on a recursive call.
IsControl() bool
// NumElements returns the total number of discrete elements contained
// in this parm. For non-composite types, this will always be 1.
NumElements() int
// String returns a descriptive string for this parm.
String() string
// TypeName returns the non-qualified type name for this parm.
TypeName() string
// QualName returns a package-qualified type name for this parm.
QualName() string
// HasPointer returns true if this parm is of pointer type, or
// if it is a composite that has a pointer element somewhere inside.
// Strings and slices return true for this hook.
HasPointer() bool
// IsBlank() returns true if the name of this parm is "_" (that is,
// if we randomly chose to make it a blank). SetBlank() is used
// to set the 'blank' property for this parm.
IsBlank() bool
SetBlank(v bool)
// AddrTaken() return a token indicating whether this parm should
// be address taken or not, the nature of the address-taken-ness (see
// below at the def of addrTakenHow). SetAddrTaken is used to set
// the address taken property of the parm.
AddrTaken() addrTakenHow
SetAddrTaken(val addrTakenHow)
// IsGenVal() returns true if the values of this type should
// be obtained by calling a helper func, as opposed to
// emitting code inline (as one would for things like numeric
// types). SetIsGenVal is used to set the gen-val property of
// the parm.
IsGenVal() bool
SetIsGenVal(val bool)
// SkipCompare() returns true if we've randomly decided that
// we don't want to compare the value for this param or
// return. SetSkipCompare is used to set the skip-compare
// property of the parm.
SkipCompare() skipCompare
SetSkipCompare(val skipCompare)
}
type addrTakenHow uint8
const (
// Param not address taken.
notAddrTaken addrTakenHow = 0
// Param address is taken and used for simple reads/writes.
addrTakenSimple addrTakenHow = 1
// Param address is taken and passed to a well-behaved function.
addrTakenPassed addrTakenHow = 2
// Param address is taken and stored to a global var.
addrTakenHeap addrTakenHow = 3
)
func (a *addrTakenHow) AddrTaken() addrTakenHow {
return *a
}
func (a *addrTakenHow) SetAddrTaken(val addrTakenHow) {
*a = val
}
type isBlank bool
func (b *isBlank) IsBlank() bool {
return bool(*b)
}
func (b *isBlank) SetBlank(val bool) {
*b = isBlank(val)
}
type isGenValFunc bool
func (g *isGenValFunc) IsGenVal() bool {
return bool(*g)
}
func (g *isGenValFunc) SetIsGenVal(val bool) {
*g = isGenValFunc(val)
}
type skipCompare int
const (
// Param not address taken.
SkipAll = -1
SkipNone = 0
SkipPayload = 1
)
func (s *skipCompare) SkipCompare() skipCompare {
return skipCompare(*s)
}
func (s *skipCompare) SetSkipCompare(val skipCompare) {
*s = skipCompare(val)
}
// containedParms takes an arbitrary param 'p' and returns a slice
// with 'p' itself plus any component parms contained within 'p'.
func containedParms(p parm) []parm {
visited := make(map[string]parm)
worklist := []parm{p}
addToWork := func(p parm) {
if p == nil {
panic("not expected")
}
if _, ok := visited[p.TypeName()]; !ok {
worklist = append(worklist, p)
}
}
for len(worklist) != 0 {
cp := worklist[0]
worklist = worklist[1:]
if _, ok := visited[cp.TypeName()]; ok {
continue
}
visited[cp.TypeName()] = cp
switch x := cp.(type) {
case *mapparm:
addToWork(x.keytype)
addToWork(x.valtype)
case *structparm:
for _, fld := range x.fields {
addToWork(fld)
}
case *arrayparm:
addToWork(x.eltype)
case *pointerparm:
addToWork(x.totype)
case *typedefparm:
addToWork(x.target)
}
}
rv := []parm{}
for _, v := range visited {
rv = append(rv, v)
}
sort.Slice(rv, func(i, j int) bool {
if rv[i].TypeName() == rv[j].TypeName() {
fmt.Fprintf(os.Stderr, "%d %d %+v %+v %s %s\n", i, j, rv[i], rv[i].String(), rv[j], rv[j].String())
panic("unexpected")
}
return rv[i].TypeName() < rv[j].TypeName()
})
return rv
}

View File

@ -0,0 +1,75 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"fmt"
)
// pointerparm describes a parameter of pointer type; it implements the
// "parm" interface.
type pointerparm struct {
tag string
totype parm
isBlank
addrTakenHow
isGenValFunc
skipCompare
}
func (p pointerparm) Declare(b *bytes.Buffer, prefix string, suffix string, caller bool) {
n := p.totype.TypeName()
if caller {
n = p.totype.QualName()
}
b.WriteString(fmt.Sprintf("%s *%s%s", prefix, n, suffix))
}
func (p pointerparm) GenElemRef(elidx int, path string) (string, parm) {
return path, &p
}
func (p pointerparm) GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int) {
pref := ""
if caller {
pref = s.checkerPkg(s.pkidx) + "."
}
var valstr string
valstr, value = s.GenValue(f, p.totype, value, caller)
fname := s.genAllocFunc(p.totype)
return fmt.Sprintf("%s%s(%s)", pref, fname, valstr), value
}
func (p pointerparm) IsControl() bool {
return false
}
func (p pointerparm) NumElements() int {
return 1
}
func (p pointerparm) String() string {
return fmt.Sprintf("*%s", p.totype)
}
func (p pointerparm) TypeName() string {
return fmt.Sprintf("*%s", p.totype.TypeName())
}
func (p pointerparm) QualName() string {
return fmt.Sprintf("*%s", p.totype.QualName())
}
func mkPointerParm(to parm) pointerparm {
var pp pointerparm
pp.tag = "pointer"
pp.totype = to
return pp
}
func (p pointerparm) HasPointer() bool {
return true
}

View File

@ -0,0 +1,64 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
)
// stringparm describes a parameter of string type; it implements the
// "parm" interface
type stringparm struct {
tag string
isBlank
addrTakenHow
isGenValFunc
skipCompare
}
func (p stringparm) Declare(b *bytes.Buffer, prefix string, suffix string, caller bool) {
b.WriteString(prefix + " string" + suffix)
}
func (p stringparm) GenElemRef(elidx int, path string) (string, parm) {
return path, &p
}
var letters = []rune("<>꿦3򂨃f6ꂅ8ˋ<􂊇񊶿(z̽|ϣᇊ񁗇򟄼q񧲥筁{ЂƜĽ")
func (p stringparm) GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int) {
ns := len(letters) - 9
nel := int(s.wr.Intn(8))
st := int(s.wr.Intn(int64(ns)))
en := st + nel
if en > ns {
en = ns
}
return "\"" + string(letters[st:en]) + "\"", value + 1
}
func (p stringparm) IsControl() bool {
return false
}
func (p stringparm) NumElements() int {
return 1
}
func (p stringparm) String() string {
return "string"
}
func (p stringparm) TypeName() string {
return "string"
}
func (p stringparm) QualName() string {
return "string"
}
func (p stringparm) HasPointer() bool {
return false
}

View File

@ -0,0 +1,163 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"fmt"
"strings"
)
// structparm describes a parameter of struct type; it implements the
// "parm" interface.
type structparm struct {
sname string
qname string
fields []parm
isBlank
addrTakenHow
isGenValFunc
skipCompare
}
func (p structparm) TypeName() string {
return p.sname
}
func (p structparm) QualName() string {
return p.qname
}
func (p structparm) Declare(b *bytes.Buffer, prefix string, suffix string, caller bool) {
n := p.sname
if caller {
n = p.qname
}
b.WriteString(fmt.Sprintf("%s %s%s", prefix, n, suffix))
}
func (p structparm) FieldName(i int) string {
if p.fields[i].IsBlank() {
return "_"
}
return fmt.Sprintf("F%d", i)
}
func (p structparm) String() string {
var buf bytes.Buffer
buf.WriteString(fmt.Sprintf("struct %s {\n", p.sname))
for fi, f := range p.fields {
buf.WriteString(fmt.Sprintf("%s %s\n", p.FieldName(fi), f.String()))
}
buf.WriteString("}")
return buf.String()
}
func (p structparm) GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int) {
var buf bytes.Buffer
verb(5, "structparm.GenValue(%d)", value)
n := p.sname
if caller {
n = p.qname
}
buf.WriteString(fmt.Sprintf("%s{", n))
nbfi := 0
for fi, fld := range p.fields {
var valstr string
valstr, value = s.GenValue(f, fld, value, caller)
if p.fields[fi].IsBlank() {
buf.WriteString("/* ")
valstr = strings.ReplaceAll(valstr, "/*", "[[")
valstr = strings.ReplaceAll(valstr, "*/", "]]")
} else {
writeCom(&buf, nbfi)
}
buf.WriteString(p.FieldName(fi) + ": ")
buf.WriteString(valstr)
if p.fields[fi].IsBlank() {
buf.WriteString(" */")
} else {
nbfi++
}
}
buf.WriteString("}")
return buf.String(), value
}
func (p structparm) IsControl() bool {
return false
}
func (p structparm) NumElements() int {
ne := 0
for _, f := range p.fields {
ne += f.NumElements()
}
return ne
}
func (p structparm) GenElemRef(elidx int, path string) (string, parm) {
ct := 0
verb(4, "begin GenElemRef(%d,%s) on %s", elidx, path, p.String())
for fi, f := range p.fields {
fne := f.NumElements()
//verb(4, "+ examining field %d fne %d ct %d", fi, fne, ct)
// Empty field. Continue on.
if elidx == ct && fne == 0 {
continue
}
// Is this field the element we're interested in?
if fne == 1 && elidx == ct {
// The field in question may be a composite that has only
// multiple elements but a single non-zero-sized element.
// If this is the case, keep going.
if sp, ok := f.(*structparm); ok {
if len(sp.fields) > 1 {
ppath := fmt.Sprintf("%s.F%d", path, fi)
if p.fields[fi].IsBlank() || path == "_" {
ppath = "_"
}
return f.GenElemRef(elidx-ct, ppath)
}
}
verb(4, "found field %d type %s in GenElemRef(%d,%s)", fi, f.TypeName(), elidx, path)
ppath := fmt.Sprintf("%s.F%d", path, fi)
if p.fields[fi].IsBlank() || path == "_" {
ppath = "_"
}
return ppath, f
}
// Is the element we want somewhere inside this field?
if fne > 1 && elidx >= ct && elidx < ct+fne {
ppath := fmt.Sprintf("%s.F%d", path, fi)
if p.fields[fi].IsBlank() || path == "_" {
ppath = "_"
}
return f.GenElemRef(elidx-ct, ppath)
}
ct += fne
}
panic(fmt.Sprintf("GenElemRef failed for struct %s elidx %d", p.TypeName(), elidx))
}
func (p structparm) HasPointer() bool {
for _, f := range p.fields {
if f.HasPointer() {
return true
}
}
return false
}

View File

@ -0,0 +1,90 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"bytes"
"fmt"
)
// typedefparm describes a parameter that is a typedef of some other
// type; it implements the "parm" interface
type typedefparm struct {
aname string
qname string
target parm
isBlank
addrTakenHow
isGenValFunc
skipCompare
}
func (p typedefparm) Declare(b *bytes.Buffer, prefix string, suffix string, caller bool) {
n := p.aname
if caller {
n = p.qname
}
b.WriteString(fmt.Sprintf("%s %s%s", prefix, n, suffix))
}
func (p typedefparm) GenElemRef(elidx int, path string) (string, parm) {
_, isarr := p.target.(*arrayparm)
_, isstruct := p.target.(*structparm)
_, ismap := p.target.(*mapparm)
rv, rp := p.target.GenElemRef(elidx, path)
// this is hacky, but I don't see a nicer way to do this
if isarr || isstruct || ismap {
return rv, rp
}
rp = &p
return rv, rp
}
func (p typedefparm) GenValue(s *genstate, f *funcdef, value int, caller bool) (string, int) {
n := p.aname
if caller {
n = p.qname
}
rv, v := s.GenValue(f, p.target, value, caller)
rv = n + "(" + rv + ")"
return rv, v
}
func (p typedefparm) IsControl() bool {
return false
}
func (p typedefparm) NumElements() int {
return p.target.NumElements()
}
func (p typedefparm) String() string {
return fmt.Sprintf("%s typedef of %s", p.aname, p.target.String())
}
func (p typedefparm) TypeName() string {
return p.aname
}
func (p typedefparm) QualName() string {
return p.qname
}
func (p typedefparm) HasPointer() bool {
return p.target.HasPointer()
}
func (s *genstate) makeTypedefParm(f *funcdef, target parm, pidx int) parm {
var tdp typedefparm
ns := len(f.typedefs)
tdp.aname = fmt.Sprintf("MyTypeF%dS%d", f.idx, ns)
tdp.qname = fmt.Sprintf("%s.MyTypeF%dS%d", s.checkerPkg(pidx), f.idx, ns)
tdp.target = target
tdp.SetBlank(uint8(s.wr.Intn(100)) < tunables.blankPerc)
f.typedefs = append(f.typedefs, tdp)
return &tdp
}

View File

@ -0,0 +1,136 @@
// Copyright 2021 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package generator
import (
"fmt"
"math/rand"
"os"
"runtime"
"strings"
)
const (
RandCtlNochecks = 0
RandCtlChecks = 1 << iota
RandCtlCapture
RandCtlPanic
)
func NewWrapRand(seed int64, ctl int) *wraprand {
rand.Seed(seed)
return &wraprand{seed: seed, ctl: ctl}
}
type wraprand struct {
f32calls int
f64calls int
intncalls int
seed int64
tag string
calls []string
ctl int
}
func (w *wraprand) captureCall(tag string, val string) {
call := tag + ": " + val + "\n"
pc := make([]uintptr, 10)
n := runtime.Callers(1, pc)
if n == 0 {
panic("why?")
}
pc = pc[:n] // pass only valid pcs to runtime.CallersFrames
frames := runtime.CallersFrames(pc)
for {
frame, more := frames.Next()
if strings.Contains(frame.File, "testing.") {
break
}
call += fmt.Sprintf("%s %s:%d\n", frame.Function, frame.File, frame.Line)
if !more {
break
}
}
w.calls = append(w.calls, call)
}
func (w *wraprand) Intn(n int64) int64 {
w.intncalls++
rv := rand.Int63n(n)
if w.ctl&RandCtlCapture != 0 {
w.captureCall("Intn", fmt.Sprintf("%d", rv))
}
return rv
}
func (w *wraprand) Float32() float32 {
w.f32calls++
rv := rand.Float32()
if w.ctl&RandCtlCapture != 0 {
w.captureCall("Float32", fmt.Sprintf("%f", rv))
}
return rv
}
func (w *wraprand) NormFloat64() float64 {
w.f64calls++
rv := rand.NormFloat64()
if w.ctl&RandCtlCapture != 0 {
w.captureCall("NormFloat64", fmt.Sprintf("%f", rv))
}
return rv
}
func (w *wraprand) emitCalls(fn string) {
outf, err := os.OpenFile(fn, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0666)
if err != nil {
panic(err)
}
for _, c := range w.calls {
fmt.Fprint(outf, c)
}
outf.Close()
}
func (w *wraprand) Equal(w2 *wraprand) bool {
return w.f32calls == w2.f32calls &&
w.f64calls == w2.f64calls &&
w.intncalls == w2.intncalls
}
func (w *wraprand) Check(w2 *wraprand) {
if w.ctl != 0 && !w.Equal(w2) {
fmt.Fprintf(os.Stderr, "wraprand consistency check failed:\n")
t := "w"
if w.tag != "" {
t = w.tag
}
t2 := "w2"
if w2.tag != "" {
t2 = w2.tag
}
fmt.Fprintf(os.Stderr, " %s: {f32:%d f64:%d i:%d}\n", t,
w.f32calls, w.f64calls, w.intncalls)
fmt.Fprintf(os.Stderr, " %s: {f32:%d f64:%d i:%d}\n", t2,
w2.f32calls, w2.f64calls, w2.intncalls)
if w.ctl&RandCtlCapture != 0 {
f := fmt.Sprintf("/tmp/%s.txt", t)
f2 := fmt.Sprintf("/tmp/%s.txt", t2)
w.emitCalls(f)
w2.emitCalls(f2)
fmt.Fprintf(os.Stderr, "=-= emitted calls to %s, %s\n", f, f2)
}
if w.ctl&RandCtlPanic != 0 {
panic("bad")
}
}
}
func (w *wraprand) Checkpoint(tag string) {
if w.ctl&RandCtlCapture != 0 {
w.calls = append(w.calls, "=-=\n"+tag+"\n=-=\n")
}
}