This on average reduces latency from 25ms to 12ms on internal codebase.
Updates golang/go#45686
Change-Id: I49c8f09f8e54b7b486d7ff7eb8f4ba9f0d90b278
Reviewed-on: https://go-review.googlesource.com/c/tools/+/413655
gopls-CI: kokoro <noreply+kokoro@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
This on average reduces latency from 34ms to 25ms on internal codebase.
Updates golang/go#45686
Change-Id: I57b05e5679620d8481b1f1a051645cf1cc00aca5
Reviewed-on: https://go-review.googlesource.com/c/tools/+/413654
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
Use treap (https://en.wikipedia.org/wiki/Treap) as a persistent map to avoid copying s.goFiles across generations.
Maintain an additional s.parseKeysByURIMap to avoid scanning s.goFiles on individual file's content invalidation.
This on average reduces didChange latency on internal codebase from 160ms to 150ms.
In a followup the same approach can be used to avoid copying s.files, s.packages, and s.knownSubdirs.
Updates golang/go#45686
Change-Id: Ic4a9b3c8fb2b66256f224adf9896ddcaaa6865b1
GitHub-Last-Rev: 0abd2570ae9b20ea7126ff31bee69aa0dc3f40aa
GitHub-Pull-Request: golang/tools#382
Reviewed-on: https://go-review.googlesource.com/c/tools/+/411554
Reviewed-by: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
When using Go workspaces, the go.work file should be used to determine
which packages are workspace packages.
For golang/go#48929
Change-Id: I1a8753ab7887daf193e093fca5070b4cc250a245
Reviewed-on: https://go-review.googlesource.com/c/tools/+/400822
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Rather than updating metadata directly in snapshot.clone, build a set of
updates to apply and call metadata.Clone.
After this change, metadata is only updated by cloning, so we can
eliminate some code that works with mutable metadata.
In the next CL we'll only update the metadata if something changed, but
this is intentionally left out of this CL to isolate the change.
Benchmark (didChange in kubernetes): ~55ms->65ms, because it is now more
work to compute uris.
For golang/go#45686
Change-Id: I048bed65760b266a209f67111c57fae29bd3e6f0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/340852
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
I had hoped to see a reduction in total allocation, but it does not
appear to be significant according to the included crude benchmark.
Nonetheless this is a slight code clarity improvement.
Change-Id: I94a503b377dd1146eb371ff11222a351cb5a43b7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/411655
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
This change replaces the single large map used for snapshot.goFiles
by a map of 256 stripes, each of which becomes immutable once shared.
This optimizes the common case in which the copy is nearly identical
to the original.
We still need to visit each map entry to see whether it needs to be
deleted (which is rare) and to inherit the handle in the usual case.
This is now done concurrently.
Also, share the (logically immutable) []PackageIDs slices across
old and new snapshots. This was worth 5% of CPU and 1/3 of allocations
(all small).
Benchmark on darwin/arm64 shows a 29% reduction for DidChange.
$ go test -v ./gopls/internal/regtest/bench -run=TestBenchmarkDidChange -didchange_dir=$HOME/w/kubernetes -didchange_file=pkg/util/hash/hash.go
Before:
BenchmarkStatistics 100 22955469 ns/op 11308095 B/op 47412 allocs/op
BenchmarkStatistics 100 23454630 ns/op 11226742 B/op 46882 allocs/op
BenchmarkStatistics 100 23618532 ns/op 11258619 B/op 47068 allocs/op
After goFilesMap:
BenchmarkStatistics 100 16643972 ns/op 8770787 B/op 46238 allocs/op
BenchmarkStatistics 100 17805864 ns/op 8862926 B/op 46762 allocs/op
BenchmarkStatistics 100 18618255 ns/op 9308864 B/op 49776 allocs/op
After goFilesMap and ids sharing:
BenchmarkStatistics 100 16703623 ns/op 8772626 B/op 33812 allocs/op
BenchmarkStatistics 100 16927378 ns/op 8529491 B/op 32328 allocs/op
BenchmarkStatistics 100 16632762 ns/op 8557533 B/op 32497 allocs/op
Also:
- Add comments documenting findings of profiling.
- preallocate slice for knownSubdirs.
- remove unwanted loop over slice in Generation.Inherit
Updates golang/go#45686
Change-Id: Id953699191b8404cf36ba3a7ab9cd78b1d19c0a2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/410176
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
In preparation for making metadata immutable, move metadata-related
fields to a new MetadataGraph type. Other than instantiating this type
when cloning, this CL contains no functional changes.
For golang/go#45686
Change-Id: I7ad29d1f331ba7e53dad3f012ad7ecdae4f7d4b7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/340730
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
As of golang/go#50827, gopls no longer supports building at 1.12, and so
usage of golang.org/x/xerrors can be replaced with the native support for
error wrapping introduced in Go 1.13.
Remove this usage as a step toward eliminating the xerrors dependency
from x/tools.
For golang/go#52442
Change-Id: Ibf459cc72402a30a6c2735dc620f76ed8a5e2525
Reviewed-on: https://go-review.googlesource.com/c/tools/+/401097
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Wired through support for calling x/mod's go.work formatter on go.work
files into LSP. Tested it by hand in editor using the "Format Document"
command. Added a test case to workspace_test regtest, though I'm not
totally sure the test is correct.
For golang/go#50930
Change-Id: Ied052ded514bb36f561737698f0e2d7b488158e7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/383774
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Remove the workspace_metadata command, as VS Code no longer needs this
to run workspace commands (it can use go.work instead).
Updates golang/go#44696
Change-Id: Ife579a15e64969c4301e4508e18b7c8a8b633b9f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/382235
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Honor the file kind provided by clients for overlays, by passing the
FileHandle into View.FileKind and checking for overlays.
For golang/vscode-go#1957
Change-Id: I4a767cb64dc5205f1d10c3126a2cbe67c21a34e4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/378314
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Suzy Mueller <suzmue@golang.org>
Make the language id (sent from the client) 'gotmpl' equivalent to 'tmpl'
Wherever a view is known, use its options to determine which files
are template files. Whenever the client sends an explicit
languageID, use that.
Partially fixesgolang/vscode-go#1957
Change-Id: I04cd630d6c6c80e0a78c2fafb6ddc1166ce86829
Reviewed-on: https://go-review.googlesource.com/c/tools/+/376854
Trust: Peter Weinberger <pjw@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
The Acquire method is nearly instantaneous; it only potentially blocks
on a small, constant sequence of cache misses, so there is no need to
avoid blocking in it when a Context is cancelled.
An early return when the passed-in Context is canceled was added in CL
242838 to avoid incrementing the Generation's WaitGroup after its
destruction has begun; however, that early return also bypasses the
WaitGroup accounting that blocks Destroy while the generation is still
in use. Instead, we need the invariant that Acquire is not called in
the first place after Destroy, which we can ensure by nilling out
the View's snapshot when we begin destroying it.
I was not able to reproduce golang/go#48774 locally, but I believe
that this CL will fix it. (It may, however, expose other races or
deadlocks that may have been masked by the early return, which we can
then fix separately.)
Fixesgolang/go#48774
Change-Id: Iac36fceb06485f849da5ba0250b44b55f937c44b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/367675
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
When we add a view, we should check if we already have a view for the
given folder.
Updates golang/go#48844
Change-Id: I0de27d420e2b4df3b33b913ae27b108bab6b7d12
Reviewed-on: https://go-review.googlesource.com/c/tools/+/356933
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
In preparation for moving metadata related functionality to a separate
package, move around some types and export some symbols. This is purely
to reduce diffs in subsequent CLs, and contains no functional changes.
Change-Id: I24d4fbd71df78e4d7a84f6598cdf820b41d542a2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/340729
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Coupling workspace symbols to package checking means that they do not
function when the workspace is contracted, and also forces us to do
duplicate work traversing file declarations.
This CL changes the workspace symbol implementation to precompute
symbols based only on syntactic information, allowing them to function
in degraded workspace mode, improving their performance, and laying the
groundwork for more significant performance improvement later on.
There is some loss of precision where we can't determine the kind of a
symbol from syntactic information alone, but this is minor: we fall back
on 'Class' if we can't determine whether a type definition is a basic
type, struct, or interface.
Benchmark ("test" in x/tools): 56ms->40ms
Benchmark ("test" in kuberneted): 874ms->799ms
Change-Id: Ic48df29b387bf029dd374d7d09720746bc27ae5e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/338692
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
In my testing, the gopls degraded memory mode (currently set via
"memoryMode": "DegradeClosed") did not save as much memory as expected
due to still type checking all packages in the workspace (even if in
ParseExported mode). It is also annoying to get incomplete results from
references and renaming.
I think we can (and should) fix both problems: don't even consider
packages that aren't 'reachable' via open files, but fully type check
the reverse transitive closure of the packages you're working on. This
CL does exactly that, by swapping out the concept of 'workspace
packages' with 'active packages'.
In testing, this decreased my memory footprint while working on std by
3-4x when compared to normal mode, and 2x when compared to the previous
implementation of DegradeClosed.
It still needs more testing before we move this option out of
experimental.
For golang/go#46902
Change-Id: I1e319d0b1607d344d27e797ce32de057d7a583f9
Reviewed-on: https://go-review.googlesource.com/c/tools/+/336410
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
The view workspace was written concurrently, meaning that changes to a
go.mod or go.sum could race to updating the view workspace, leaving it
in an incorrect state.
Move control over writing this workspace onto the view, and hold the
snapshotMu while writing.
Change-Id: I47f58769cc77860e9c9674c8f6bf5fd0793e7937
Reviewed-on: https://go-review.googlesource.com/c/tools/+/309289
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Gopls's serial processing of didChangeWatchedFile notifications has
historically been a pain point for clients that don't batch file events,
when branch switching or running go generate.
It's "not that tricky" for us to debounce and batch up watched file
notifications on our end, so this CL introduces this functionality as a
new experimental setting. Truth be told it ended up being harder than
expected, due to (1) our requirement for regtests to be able to
determine when diagnostics have completed, and (2) our reliance on
jsonrpc2 for sequencing changes to the server.
To address (1), I factored out the actual processing of change
notifications into a separate method (thus increasing our surprisingly
long chain of method calls). To address (2), I guarded the processing of
file changes with a mutex.
I also guarded some places where views and snapshots were accessed in
potentially racy ways. Our interaction with session.views was rather
complicated, so I had to switch session.viewsMu to a RWMutex.
Add this to the experimental regtest mode, and more generally enable all
experimental features in this mode, rather than just the experimental
workspace module.
Fixesgolang/go#41691
Change-Id: Ifccdbdf86263dbe2e37ffe9f7bbf2a2cd74218b4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/309269
Trust: Robert Findley <rfindley@google.com>
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Sometimes, we may want to report progress from functions inside of the
cache package, so move the progress tracker to the session to allow for
that.
Change-Id: I15409577a7a5080e7f0224a95d159de42856ffa7
Reviewed-on: https://go-review.googlesource.com/c/tools/+/319330
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Retrying CL 271477, this time with parts of CL 322650 incorporated.
This CL moves to a model where we don't automatically delete invalidated
metadata, but rather preserve it and mark it invalid. This way, we can
continue to use invalid metadata for all features even if there is an
issue with the user's workspace.
To keep track of the metadata's validity, we add an invalid flag to
track the status of the metadata. We still reload at the same rate--the
next CL changes the way we reload data.
We also add a configuration to opt-in (currently, this is off by
default).
In some cases, like switches between GOPATH and module modes, and when a
file is deleted, the metadata *must* be deleted outright.
Also, handle an empty GOMODCACHE in the directory filters (from a
previous CL).
Updates golang/go#42266
Change-Id: Idc778dc92cfcf1e4d14116c79754bcca0229e63d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/324394
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
This change treats the GOMODCACHE like a directory filter, and it
excludes any modules under the module cache from being considered part
of the workspace. This can happen when users open their entire GOPATH.
To do this, I had to propagate the view's root and gomodcache through
the exclusion functions, and I also had to add BuildGoplsMod to the
snapshot's interface.
Change-Id: Id80b359d73d09a525b380389917451e85357b784
Reviewed-on: https://go-review.googlesource.com/c/tools/+/326816
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
A lot of the time spent for every file change is recomputing the set of
known subdirectories in the workspace. We can easily memoize these known
subdirectories and avoid recomputing them on every file change. Do that
here and update the set as file creations and deletions come in.
Updates golang/go#45686Fixesgolang/go#45974
Change-Id: Ide07f7c90f0cafc3a3cc7b89ba14ab82d4e3ab28
Reviewed-on: https://go-review.googlesource.com/c/tools/+/317410
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
This reverts commit 46e69bf3b2.
Reason for revert: Still has bugs associated with it and want to do a release
Change-Id: Ifa80bde147aa23aa4029a157d5dbaf6479d53024
Reviewed-on: https://go-review.googlesource.com/c/tools/+/324290
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
This CL moves to a model where we don't automatically delete invalidated
metadata, but rather preserve it and mark it invalid. This way, we can
continue to use invalid metadata for all features even if there is an
issue with the user's workspace.
To keep track of the metadata's validity, we add an invalid flag to
track the status of the metadata. We still reload at the same rate--the
next CL changes the way we reload data.
We also add a configuration to opt-in (currently, this is off by
default).
In some cases, like switches between GOPATH and module modes, and when a
file is deleted, the metadata *must* be deleted outright.
Updates golang/go#42266
Change-Id: Iff5e10b641fdb4be270af0cd887a10ee97ac1a19
Reviewed-on: https://go-review.googlesource.com/c/tools/+/271477
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Locating the workspace module by convention has multiple problems:
+ gopls's view of $TMPDIR might be different from the client
+ there might be multiple views
+ there might be multiple gopls sessions per pid
Instead, assign a temp workspace directory for each workspace folder,
and provide a command to access this information.
Cleaning up all these temp directories was overcomplicated. Instead,
create a temp directory for the gopls server to nest them under, that
can be removed up on server shutdown.
Also fix a bug where the snapshot was not acquired before copying its
workspace.
Updates golang/go#42252
Change-Id: I0641cebe09cd376dfa27373cac30397711c64a8f
Reviewed-on: https://go-review.googlesource.com/c/tools/+/300409
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
findFile has a case that returns early if the file does not exist.
Handle this error in getFile to avoid inconsistently returning errors
when getting file handles for files that don't exist.
Unskip the test, since it is no longer flaky.
Fixesgolang/go#44227
Change-Id: I07a4f860cfc9f852728c31706bd924e419bd54e2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/291391
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
The Typescript source is still at version 3.16, but there are new
requests, more detailed client capabilities, and an attempt to be
more specific about ranges of number in the Typescript code.
Vscode defines types integer and uinteger (32-bit signed and unsigned),
so the Go code now uses int32 and uint32.
They've changed the use of TextDocument, so version information is sometimes
missing. cache/session.go:625 was changed correspondingly.
This CL also make CodeAction.Disabled into a pointer.
New requests or notifications:
DidCreateFiles, DidRenameFiles, DidDeleteFiles (notifications)
ShowDocument, WillCreateFiles,WillRenameFiles, WillDeleteFiles (request)
It's a lot of code; I've probably missed something.
Change-Id: I8449ad8473ac00947d0344c5f6133f9bd73b9e10
Reviewed-on: https://go-review.googlesource.com/c/tools/+/286192
Run-TryBot: Peter Weinberger <pjw@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Trust: Peter Weinberger <pjw@google.com>
In CL 271297, I disabled the constantly-running upgrade check, which
removed the upgrade commands for individual dependencies. This seems to
have been a relatively popular feature. Re-introduce it, but requiring
explicit user interaction.
We now run an upgrade check when the user clicks "Check for upgrades".
Those results are stored on the View and used to show diagnostics on
any requires they apply to. Right now we only check the go.mod the user
has open; in multi-module workspaces it might make sense to check all of
them, but I'm not sure.
Fixesgolang/go#42969.
Change-Id: I65205dc99a4fa9daafdb83145b0294b6f3be5336
Reviewed-on: https://go-review.googlesource.com/c/tools/+/286474
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
For every file change, we want to diagnose the resulting snapshots.
However, we also want to diagnose the file in its "best possible" view
(for example, for a nested module, we should use the inner module rather
than the outer module).
Add a bestView function that operates on a set of views, which allows
us to a pick a view out of the *views affected* by a file modification,
not out of *all* of the views in the session. This means that we can
pick the best view, if any, for each file change and then only return
the snapshots that we should diagnose, as well as the changed URIs for
those snapshots.
Fixesgolang/go#42890
Change-Id: Iad23d5ed4832dfd857f1424a7244cf3bd8900e3b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/274235
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Users working in large repositories may want to include only selected
directories in their workspace to avoid memory usage and performance
slowdowns. Add support for inclusion/exclusion filters that control what
directories are searched for workspace packages and modules. Packages
that are excluded by the filter may still be loaded as non-workspace
packages if other things depend on them.
For a description of the option's syntax, see the documentation.
Note that because we don't have any way to communicate the filters to
packages.Load, we still run go list on the unfiltered workspace scope,
then throw away the irrelevant packages. That may cost us, especially in
workspaces with many files.
Comments on the naming welcome. Also, if you know any places I may have
missed applying the filter, please do tell. One thing I thought of is
file watching, but that's covered because allKnownSubdirs works off of
workspace files and those are already filtered.
Possible enhancements:
- Support glob patterns.
- Apply filters during the goimports scan.
- Figure out how to apply the filters to packages.Load. I don't know
how to do it while still being build system neutral though.
Closesgolang/go#42473, assuming none of the enhancements are required.
Change-Id: I9006a7a361dc3bb3c11f78b05ff84981813035a0
Reviewed-on: https://go-review.googlesource.com/c/tools/+/275253
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
There was a race to the background context that allowed work for earlier
snapshots to run on the background context of later snapshots. For
example, the following sequence is possible and has been observed in
tests (admittedly at unrealistic time scales):
- snapshot N is created
- view.cancelBackground() is called
- snapshot N+1 is created
- snapshot N+1 is diagnosed
- snapshot N is diagnosed
- snapshot N is destroyed
Fix this by pinning the background context to the snapshot. This also
makes more sense; I'm guessing the background context being on view is a
vestige of when more work was done on the view.
For golang/go#42837
Change-Id: If9cbad7361de823da9bfd2f55cb2f8b3aed95075
Reviewed-on: https://go-review.googlesource.com/c/tools/+/275239
Run-TryBot: Robert Findley <rfindley@google.com>
Trust: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
We adjust the value of GO111MODULE to "auto" when running the go
command, since it handles more edge cases. However, in some cases, we
may want to know the environment's actual value of GO111MODULE, so
separate out these two concepts.
Also, as of Go 1.13, `go env` reports GO111MODULE, so add it to the
environmentVariables, but only for Go versions >= 1.13.
Removed a regression test that doesn't seem very important--looks like
even `go env` will fail with GO111MODULE=on and a malformed GOPATH.
I don't think that it's important that `gopls` continue working in such
a case, since the go command will also fail.
Change-Id: I8147782222261a190d0f1017fe9f6043f2ac0cdb
Reviewed-on: https://go-review.googlesource.com/c/tools/+/275013
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
We don't check the value of GO111MODULE before finding all of the
modules in the workspace, meaning we treat the workspace as if it has
modules even if modules are disabled. Before creating a workspace,
we now do check this.
Change-Id: I36b9d13f3b8382e65de36b4b4deed68f5b59e21d
Reviewed-on: https://go-review.googlesource.com/c/tools/+/273309
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Expanding the set of modifications in the cache package leads to a
mismatch when the didModifyFiles code in text_synchronization.go
processes the list of modifications.
Change-Id: Id0b7fea445103b34838fb9b03aee95a01e454306
Reviewed-on: https://go-review.googlesource.com/c/tools/+/274192
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
VS Code's file watching API doesn't send notifications about directory
deletions unless you register for them explicitly (see
https://github.com/microsoft/vscode/issues/109754). Rather than watch
every file in the workspace, keep track of every relevant directory and
register file watchers for it.
This CL moves the snapshot's WorkspaceDirectories function to the
session and changes it to a KnownDirectories function. It returns all of
the directories and subdirectories known the session at a given moment.
Top-level directories are marked as such so that their *.{go,mod,sum}
contents can be watched, while subdirectories are just watched by path
so we can be notified of deletions.
Fixesgolang/go#42348
Change-Id: Ic6d02dba55b5de89370522ea5d3cf1d198927997
Reviewed-on: https://go-review.googlesource.com/c/tools/+/271438
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Trust: Rebecca Stambler <rstambler@golang.org>
Our handling of diagnostics has gotten complicated and has recently been
a source of bugs. Heschi and I took a step back and refactored the
diagnostic pass, starting with a new data structure for tracking
diagnostics.
The LSP server now holds a map of URI to collection of diagnostic
reports, which is used to track diagnostics as they are computed by
multiple sources. Additionally, this collection holds a hash of the
last published diagnostic set.
This new information allows us to implement an algorithm for
incrementally updating diagnostics on the server: diagnostics reports
are stored as they are computed for a snapshot, and then published in
(possibly multiple) passes, with the last pass for a snapshot being
marked as 'final'. In non-final passes, 'publishReports' publishes any
diagnostics that have already been computed for the snapshot, but does
nothing with URIs for which no diagnostics have been computed. In final
passes all diagnostics are reported, with empty diagnostics being
published for any URIs with no diagnostic reports. Any URIs for which no
diagnostic reports were computed are pruned from the diagnostic set. In
all cases, tracking the hash of published diagnostics prevents us from
duplicate publication.
This enables some simplifications of the existing diagnostic logic.
Computing new diagnostics and tracking send diagnostics is now handled
by the same algorithm, avoiding some bookkeeping. It is also no longer
necessary to explicitly clear diagnostics for deletions. This fixes some
previously broken edge cases, for example when packages go out of scope
due to go.mod or gopls.mod changes.
Fixesgolang/go#42198
Change-Id: Id0d8d0f7a60f6ffa8c33f0ae41763461f61dab7b
Reviewed-on: https://go-review.googlesource.com/c/tools/+/269677
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Trust: Robert Findley <rfindley@google.com>
If the user opens a parent directory of a single module, we can load
their workspace without using some of the workarounds of experimental
workspace mode, simply by narrowing the workspace to this nested
subdirectory.
Do this by extending findAllModules to support a limit on the number of
modules and files to search.
Updates golang/go#41558Fixesgolang/go#42108
Change-Id: Idba800a0deaeee5137d46f16a30b2012ff902a36
Reviewed-on: https://go-review.googlesource.com/c/tools/+/268877
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Trust: Robert Findley <rfindley@google.com>
Throwing away all of the metadata in the package means it will be
expensive to reload, so it makes sense to keep it around until a go.mod
file has been saved, indicating that the user is more likely to be done
typing. Otherwise, we will be triggering full package metadata reloads
on every keystroke.
Fixesgolang/go#42529
Change-Id: Ibc4f4bc8937f8f7176da79e9fefe166468578993
Reviewed-on: https://go-review.googlesource.com/c/tools/+/271299
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
The current DidModifyFiles logic does not handle the possibility of a
directory URI in a didChangeWatchedFiles request. For creation and
changes, this is fine because only file URIs matter in those cases, but
for deletion, the deletion of a directory signifies the deletion of all
of the files in it.
Add handling for directories by looking up all of the known files in a
given directory. We then create file modifications for all of those
files, instead of the directory.
Fixesgolang/go#38422
Change-Id: I26d7b93285367fe116fd1cb3e8ec13a276f9e517
Reviewed-on: https://go-review.googlesource.com/c/tools/+/266138
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
Trust: Rebecca Stambler <rstambler@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Refactor to use goCommandInvocation to get the evironment for
goimports. That takes care of most of the work; all that's left is to
track the go.mod hash so we know when to clear the cache.
Fixesgolang/go#41836.
Change-Id: I614dacad04adfb879ead55e3b8640e57816534cb
Reviewed-on: https://go-review.googlesource.com/c/tools/+/270877
Trust: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
VS Code's file watching API doesn't send notifications about directory
deletions unless you register for them explicitly (see
https://github.com/microsoft/vscode/issues/109754). Rather than watch
every file in the workspace, keep track of every relevant directory and
register file watchers for it.
This CL moves the snapshot's WorkspaceDirectories function to the
session and changes it to a KnownDirectories function. It returns all of
the directories and subdirectories known the session at a given moment.
Top-level directories are marked as such so that their *.{go,mod,sum}
contents can be watched, while subdirectories are just watched by path
so we can be notified of deletions.
Fixesgolang/go#42348
Change-Id: If80a26cf6af5a2c2f7d87bb57b9ef5cb3dcf04ac
Reviewed-on: https://go-review.googlesource.com/c/tools/+/267120
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Editors need a way to run commands in the same workspace that gopls
sees. Longer term, we need a good solution for this that supports
multiple workspace folders, but for now just write the first folder's
workspace to a deterministic location:
$TMPDIR/gopls-<client PID>.workspace.
Using the client-provided PID allows this mechanism to work even for
multi-session daemons.
Along the way, simplify the snapshot reinitialization logic a bit.
Fixesgolang/go#42126
Change-Id: I5b9f454fcf1a1a8fa49a4b0a122e55e762d398b4
Reviewed-on: https://go-review.googlesource.com/c/tools/+/264618
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Robert Findley <rfindley@google.com>
The majority of the initialization logic is already based on the
snapshot, not the view, so it makes sense to move it there. The
initialization status of the snapshot is copied and invalidated in
clone.
Fixesgolang/go#41764
Change-Id: I93234b394318964e7af4696e5ebd465088a05728
Reviewed-on: https://go-review.googlesource.com/c/tools/+/266700
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
When incorporating the gopls.mod file, the invalidation logic for
workspace module information becomes quite complicated. For example:
+ if a modfile changes we should only invalidate if it is in the set of
active modules
+ the set of active modules can be provided by either the filesystem or
gopls.mod
+ if gopls.mod changes, we may gain or lose active modules in the
workspace
+ if gopls.mod is *deleted*, we may need to walk the filesystem to
actually find all active modules
Doing this with only concrete changes to the snapshot proved
prohibitively complex (at least for me), so a new workspace type is
introduced to manage the module state. This new abstraction is
responsible for tracking the set of active modules, the workspace
modfile, and the set of workspace directories. Its invalidation logic is
factored out of snapshot.clone, so that it can be tested and to
alleviate some of the growing complexity of snapshot.clone.
The workspace type is idempotent, allowing it to be shared across
snapshots without needing to use the cache. There is little benefit to
the cache in this case, since workspace module computation should be
infrequent, and the type itself consumes little memory.
This is made possible because the workspace type itself depends only on
file state, and therefore may be invalidated independently of the
snapshot. The new source.FileState interface is used in testing, and so
that the workspace module may be computed based on both the session file
state as well as the snapshot file state.
As a result of this change, the following improvements in functionality
are possible:
+ in the presence of a gopls.mod file, we avoid walking the filesystem
to detect modules. This could be helpful for working in large
monorepos or in GOPATH, when discovering all modules would be
expensive.
+ The set of active modules should always be consistent with the
gopls.mod file, likely fixing some bugs (for example, computing
diagnostics for modfiles that shouldn't be loaded)
For golang/go#41837
Change-Id: I2da888c097748b659ee892ca2d6b3fbe29c1942e
Reviewed-on: https://go-review.googlesource.com/c/tools/+/261237
Run-TryBot: Robert Findley <rfindley@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Trust: Robert Findley <rfindley@google.com>
Ruslan Nigmatullin
Robert Findley
Alan Donovan
Michael Matloob
pjw
Bryan C. Mills
Rebecca Stambler
Heschi Kreinick