Commit Graph

131 Commits

Author SHA1 Message Date
apocelipes 9cfe3a86d3 archive/zip,cmd/compile: simplify the split function
Use strings to simplify the code.

This is a follow-up for the CL 586715.

Change-Id: I9e5470ec271e8af1ad4ddbb5f01f43a8a4879557
GitHub-Last-Rev: b95d617978
GitHub-Pull-Request: golang/go#68713
Reviewed-on: https://go-review.googlesource.com/c/go/+/602697
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-08-03 01:05:29 +00:00
apocelipes bd6f911f85 archive: use slices and maps to clean up tests
Replace reflect.DeepEqual with slices.Equal/maps.Equal, which is
much faster.

Clean up some unnecessary helper functions.

Change-Id: I9b94bd43886302b9b327539ab065a435ce0d75d9
GitHub-Last-Rev: b9ca21f165
GitHub-Pull-Request: golang/go#67607
Reviewed-on: https://go-review.googlesource.com/c/go/+/587936
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2024-07-25 00:25:45 +00:00
Sean Liao 5c7f541386 archive/zip: document handling of duplicate names in Writer.Create
Fixes #66810

Change-Id: I6a7848dce245ae14941d61d2f78abaf0dc5c1247
Reviewed-on: https://go-review.googlesource.com/c/go/+/597978
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-07-16 18:02:59 +00:00
RTann c26be72d68 archive/zip: fix UncompressedSize godoc
Change-Id: I0c142eb18af7b03759041e894bbce0f365ea9d7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/588116
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-05-28 21:41:09 +00:00
Ian Lance Taylor b0b1d42db3 all: change from sort functions to slices functions where feasible
Doing this because the slices functions are slightly faster and
slightly easier to use. It also removes one dependency layer.

This CL does not change packages that are used during bootstrap,
as the bootstrap compiler does not have the required slices functions.
It does not change the go/scanner package because the ErrorList
Len, Swap, and Less methods are part of the Go 1 API.

Change-Id: If52899be791c829198e11d2408727720b91ebe8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/587655
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-05-23 01:00:11 +00:00
Damien Neil 33d725e575 archive/zip: treat truncated EOCDR comment as an error
When scanning for an end of central directory record,
treat an EOCDR signature with a record containing a truncated
comment as an error. Previously, we would skip over the invalid
record and look for another one. Other implementations do not
do this (they either consider this a hard error, or just ignore
the truncated comment). This parser misalignment allowed
presenting entirely different archive contents to Go programs
and other zip decoders.

Fixes #66869

Change-Id: I94e5cb028534bb5704588b8af27f1e22ea49c7c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/585397
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-05-16 15:56:19 +00:00
Liam Miller-Cushon da6cc218cb archive/zip: fix a broken URL in a comment in writeDataDescriptor
The comment contains a dead link to a OpenJDK bug ID 7073588
this change fixes the link.

Change-Id: Ib9b10362c707507e59bb6f340d52a0025f65e292
GitHub-Last-Rev: 37af15b947
GitHub-Pull-Request: golang/go#66669
Reviewed-on: https://go-review.googlesource.com/c/go/+/576335
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2024-04-04 14:28:57 +00:00
Jes Cok d4cc35c4fd all: make use of builtin clear
Change-Id: I1df0685c75fc1044ba46003a69ecc7dfc53bbc2b
Reviewed-on: https://go-review.googlesource.com/c/go/+/574675
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-27 18:23:49 +00:00
Jonathan Amsterdam 6f44cc88f5 archive/zip: reduce memory held by Writer.Copy
Make a copy of the argument File's FileHeader, and pass a pointer
to the copy to CreateRaw.

Passing the pointer directly causes the entire `File` to be referenced
by the receiver. The `File` includes a reference to the `ReaderAt`
underlying the `Reader`, so all its memory, which may be the entire
contents of the archive, is prevented from being garbage-collected.

Also, explain the issue in the doc comment for CreateRaw. We
cannot change its behavior because someone may depend on the
preserving the identity of its argument pointer.

For #65499.

Change-Id: Ieb4963a0ea30539d597547d3511accbd8c6b5c5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/560238
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Damien Neil <dneil@google.com>
2024-02-06 12:43:52 +00:00
cui fliter 22344034c5 archive: add available godoc link
Change-Id: I813aa09f8a65936796469fa637d0f23004d26098
Reviewed-on: https://go-review.googlesource.com/c/go/+/534757
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
2023-10-13 18:36:46 +00:00
Jes Cok 81c5d92f52 all: use the indefinite article an in comments
Change-Id: I8787458f9ccd3b5cdcdda820d8a45deb4f77eade
GitHub-Last-Rev: be865d67ef
GitHub-Pull-Request: golang/go#63165
Reviewed-on: https://go-review.googlesource.com/c/go/+/530120
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-09-25 14:29:30 +00:00
Jes Cok 9b883484a8 all: clean unnecessary casts for test files
This is a follow up of CL 528696.

Change-Id: I5b71eabedb12567c4b1b36f7182a3d2b0ed662a5
GitHub-Last-Rev: acaf3ac11c
GitHub-Pull-Request: golang/go#62713
Reviewed-on: https://go-review.googlesource.com/c/go/+/529197
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-09-22 16:22:42 +00:00
Jes Cok f4e7675d11 all: clean unnecessary casts
Run 'unconvert -safe -apply' (https://github.com/mdempsky/unconvert)

Change-Id: I24b7cd7d286cddce86431d8470d15c5f3f0d1106
GitHub-Last-Rev: 022e75384c
GitHub-Pull-Request: golang/go#62662
Reviewed-on: https://go-review.googlesource.com/c/go/+/528696
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-09-18 20:01:34 +00:00
Mauri de Souza Meneguzzo 52fc777dcc archive/zip: reject non-regular files in AddFS
When a filesystem with non-regular files is used
the resulting files inside the zip archive are empty.

In this case we can be explicit and return an error.

Fixes #61875

Change-Id: I1aeac544521002e0c4bad328e73f51852ad4eee3
GitHub-Last-Rev: 5e53dcfd7c
GitHub-Pull-Request: golang/go#61876
Reviewed-on: https://go-review.googlesource.com/c/go/+/517475
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Mauri de Souza Meneguzzo <mauri870@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: David Chase <drchase@google.com>
2023-09-15 19:04:06 +00:00
qmuntal 94200bf85c all: fix Microsoft links
This CL fixes the links to Microsoft documentation in the Go source
code. Some links were broken and some others were outdated.

Change-Id: I4c3bcd3aa3c07a31be1b7f94c25339dcc2e771e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/527556
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Quim Muntal <quimmuntal@gmail.com>
2023-09-12 16:42:41 +00:00
Yusuf Birader 6f11650855 archive/zip: update link to zip specification
Change-Id: Iad902ec2a13b0c5c594c60ecac0b40f641574a1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/524275
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Joseph Tsai <joetsai@digital-static.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-08-30 19:42:20 +00:00
qiulaidongfeng f8966594fa archive: use predeclared function min
Change-Id: I23e0005071fcbafeaecaa05f51712dd1de6eed01

Change-Id: I23e0005071fcbafeaecaa05f51712dd1de6eed01
GitHub-Last-Rev: 364d7c74fe
GitHub-Pull-Request: golang/go#61661
Reviewed-on: https://go-review.googlesource.com/c/go/+/514215
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-08-03 15:57:21 +00:00
Mauri de Souza Meneguzzo 845fff35c6 archive/zip: add AddFS method to zip Writer
The method AddFS can be used to add the contents of a fs.FS filesystem
to a zip archive.
This method walks the directory tree starting at the root of the filesystem
and adds each file to the archive.

Fixes #54898

Change-Id: I80511cbd91a1d7e09ee52d2d1b09fb5eed25f45f
GitHub-Last-Rev: 1e17f2e6ba
GitHub-Pull-Request: golang/go#61602
Reviewed-on: https://go-review.googlesource.com/c/go/+/513438
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-07-31 18:10:41 +00:00
Maksim Meshkov 08458804fb compress/flate, archive/zip: reduce memory allocations
The existing implementation allocates a new 4KB buffer each time it opens flate-encoded file in a zip archive. This commit allows the flate reader to reuse the buffer on call Reset instead of allocating a new one.

It is noticeable when a zip archive contains a huge amount of files, e.g. zip archive has 50_000 files, for each file 4KB buffer is allocated, so it is 200MB memory allocations. If files are read sequentially only one buffer is needed.

Fixes #59774

Change-Id: Ib16336b101ba58e8f0f30a45dc5fd4eeebc801a1
GitHub-Last-Rev: f3f395b2ad
GitHub-Pull-Request: golang/go#59775
Reviewed-on: https://go-review.googlesource.com/c/go/+/487675
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2023-05-24 23:23:54 +00:00
Ian Lance Taylor 1596aeec8e all: add String for fs.{FileInfo,DirEntry} implementations
The new String methods use the new FormatFileInfo and
FormatDirEntry functions.

Fixes #54451

Change-Id: I414cdfc212ec3c316fb2734756d2117842a23631
Reviewed-on: https://go-review.googlesource.com/c/go/+/491175
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-04 16:27:35 +00:00
Roland Shoemaker a8af76284d archive/zip: reject overflowing directorySize & directoryOffset
We added a check for incorrect baseOffset in CL 408734, but in doing so
we introduced a panic when directoryOffset overflowed a int64. The zip
spec uses uint64, but since io.SectionReader requires int64 we convert,
and possibly introduce an overflow. If offset < 0 && size-offset < 0,
SectionReader will panic when we attempt to read from it.

Since it's extremely unlikely we're ever going to process a zip file
larger than 1<<63-1 byte, just limit directory size and offset to the
max int64.

Change-Id: I1aaa755cf4da927a6e12ef59f97dfc83a3426d86
Reviewed-on: https://go-review.googlesource.com/c/go/+/488195
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
2023-04-24 20:28:37 +00:00
Imre Rad 3e8f5457ef archive/zip: return ErrInsecurePath for unsafe paths by OpenReader
zip.NewReader was recently improved to return ErrInsecurePath when
insecure entries are encountered.
This change adopts the same logic for the OpenReader interface as well.

Fixes #58641

Change-Id: I0d8be94d073cc14cf93a914dc250f85b19cec4ab
GitHub-Last-Rev: 68391dc515
GitHub-Pull-Request: golang/go#58658
Reviewed-on: https://go-review.googlesource.com/c/go/+/470735
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
2023-04-05 15:11:02 +00:00
cui fliter d8d22664c6 all: fix misuses of "a" vs "an"
Fixes the misuse of "a" vs "an", according to English grammatical
expectations and using https://www.a-or-an.com/

Change-Id: I53ac724070e3ff3d33c304483fe72c023c7cda47
Reviewed-on: https://go-review.googlesource.com/c/go/+/480536
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-04-04 14:20:53 +00:00
Oleksandr Redko 0487ba08a6 archive/zip: make receiver names consistent
Fixes revive linter receiver-naming warnings:

- receiver name f should be consistent with previous receiver name e for fileListEntry
- receiver name r should be consistent with previous receiver name z for Reader
- receiver name f should be consistent with previous receiver name h for FileHeader

Change-Id: Ibfa14b97f6ca7adc86e3a1df919c5bb5de9716dc
GitHub-Last-Rev: dd7315b09d
GitHub-Pull-Request: golang/go#58477
Reviewed-on: https://go-review.googlesource.com/c/go/+/467519
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@google.com>
2023-03-02 23:34:34 +00:00
Ian Lance Taylor 78558d5e10 archive/zip: use base offset 0 if it has a valid entry
In CL 408734 we introduced a fall back to base offset 0 if reading a
directory entry at the computed base offset failed. We have now found
a file in the wild for which the computed base offset is incorrect,
but happens to refer to a valid directory entry. In this CL, we change
the fallback such that if the first directory header relative to base
offset 0 is valid, we just use base offset 0.

Change-Id: Ia9ace20c1065d1f651035f16f7d91d741ab1dbf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/461598
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-01-20 01:04:11 +00:00
Russ Cox 213495a4a6 internal/godebug: export non-default-behavior counters in runtime/metrics
Allow GODEBUG users to report how many times a setting
resulted in non-default behavior.

Record non-default-behaviors for all existing GODEBUGs.

Also rework tests to ensure that runtime is in sync with runtime/metrics.All,
and generate docs mechanically from metrics.All.

For #56986.

Change-Id: Iefa1213e2a5c3f19ea16cd53298c487952ef05a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/453618
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-19 22:26:43 +00:00
Russ Cox 145dd38471 archive/tar, archive/zip: document ErrInsecurePath and GODEBUG setting
These are mentioned in the release notes but not the actual doc comments.
Nothing should exist only in release notes.

Change-Id: I8d10f25a2c9b2677231929ba3f393af9034b777b
Reviewed-on: https://go-review.googlesource.com/c/go/+/462195
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-17 14:18:16 +00:00
Damien Neil 2321abc5e9 archive/tar, archive/zip: revert documentation of ErrInsecurePath
CL 452616 disables path security checks by default, enabling them
only when GODEBUG=tarinsecurepath=0 or GODEBUG=zipinsecurepath=0
is set. Remove now-obsolete documenation of the path checks.

For #55356

Change-Id: I4ae57534efe9e27368d5e67773a502dd0e56eff4
Reviewed-on: https://go-review.googlesource.com/c/go/+/458875
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
2022-12-21 18:56:55 +00:00
Roland Shoemaker 5167e5cd64 archive/zip: only consider UncompressedSize when checking dirs
CL 454475 switched from checking CompressedSize to UncompressedSize
when determining if we should consider an archive malformed because
it contains data and added a test for an example of this (a JAR). We
should also remove the hasDataDescriptor check, since that is basically
an alias for CompressedSize > 0. The test didn't catch this because we
didn't actually attempt to read from the returned reader.

Change-Id: Ibc4c1aa9c3a733f3ebf4a956d1e2f8f4900a29cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/455523
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-12-06 20:19:47 +00:00
Russ Cox 5a4e440f48 archive/zip: use proper doc comment for Deprecated notes
End-of-line comments are not doc comments,
so Deprecated notes in them are not recognized
as deprecation notices. Rewrite the comments.

Change-Id: Idc4681924f9a7e9ead62f672ef8a763e70db1f0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/453616
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
2022-12-02 16:30:21 +00:00
Roland Shoemaker 791f758075 archive/zip: tolerate compressed directories with zero uncompressed size
In CL 449955 we made reading of directories with associated file data
an error, since it is a "must not" in the zip specification. It turns
out that a number of implementations make the mistake of not setting
the correct compression method on directories (in particular the Java
jar tool does this when storing the META-INF directory). If the
compression method used is not 0 (stored) then the compressed size of
the directory can be > 0, despite the uncompressed size still being 0.

Since this mistake is not uncommon, we are forced to tolerate it. We
still fail if the recorded uncompressed size is > 0, which should be
a significantly harder mistake to make.

Change-Id: Ia732b10787f26ab937ac9cf9869ac3042efb8118
Reviewed-on: https://go-review.googlesource.com/c/go/+/454475
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
2022-12-01 17:46:25 +00:00
Damien Neil 7a00f973a5 archive/tar, archive/zip: disable ErrInsecurePath by default
This change is being made late in the release cycle.
Disable it by default. Insecure path checks may be enabled by setting
GODEBUG=tarinsecurepath=0 or GODEBUG=zipinsecurepath=0.
We can enable this by default in Go 1.21 after publicizing the change
more broadly and giving users a chance to adapt to the change.

For #55356.

Change-Id: I549298b3c85d6c8c7fd607c41de1073083f79b1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/452616
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
2022-11-22 18:11:34 +00:00
Damien Neil 85a2c19b32 archive/tar, archive/zip: disable insecure file name checks with GODEBUG
Add GODEBUG=tarinsecurepath=1 and GODEBUG=zipinsecurepath=1 settings
to disable file name validation.

For #55356.

Change-Id: Iaacdc629189493e7ea3537a81660215a59dd40a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/452495
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-11-21 21:14:38 +00:00
Damien Neil a2d8157a7e archive/tar, archive/zip: return ErrInsecurePath for unsafe paths
Return a distinguishable error when reading an archive file
with a path that is:

	- absolute
	- escapes the current directory (../a)
	- on Windows, a reserved name such as NUL

Users may ignore this error and proceed if they do not need name
sanitization or intend to perform it themselves.

Fixes #25849
Fixes #55356

Change-Id: Ieefa163f00384bc285ab329ea21a6561d39d8096
Reviewed-on: https://go-review.googlesource.com/c/go/+/449937
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-11-16 23:36:48 +00:00
Alexander Yastrebov 40bdcbb483 archive/zip: don't read directories containing file data
Fixes #54801

Change-Id: I3d03516792975ddb09835b2621c57e12e7cbad35
GitHub-Last-Rev: 4faa7e14dc
GitHub-Pull-Request: golang/go#56714
Reviewed-on: https://go-review.googlesource.com/c/go/+/449955
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joedian Reid <joedian@golang.org>
2022-11-14 14:32:55 +00:00
cuiweixie 45c2421dc6 archive/zip: use strings.TrimPrefix
Change-Id: I2854b5e7b48c4c189df84cb7281b7b7de780eebd
Reviewed-on: https://go-review.googlesource.com/c/go/+/435938
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-09-29 20:45:56 +00:00
cuiweixie 6fcd9b42e7 archive: use strings.Builder
Change-Id: I07e022ffc022bca6aa12165871e723db3ed83141
Reviewed-on: https://go-review.googlesource.com/c/go/+/428255
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
2022-09-06 15:44:27 +00:00
Ian Lance Taylor aae0bef72f archive/zip: use bufio.Reset rather than NewReader
A clean up for CL 408734, suggested by Joe Tsai.

Change-Id: Ida9db0b8d31785d5640938c286c9c6c82c27f457
Reviewed-on: https://go-review.googlesource.com/c/go/+/410154
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
2022-06-02 17:17:19 +00:00
Ian Lance Taylor f70b93a6e9 archive/zip: if non-zero base offset fails, fall back to zero
This permits us to read files that earlier Go releases could read.
It is also compatible with other zip programs.

Change-Id: I7e2999f1073c4db5ba3f51f92681e0b149d55b3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/408734
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-06-02 16:25:34 +00:00
Russ Cox 946b4baaf6 all: gofmt main repo
Excluding vendor and testdata.
CL 384268 already reformatted most, but these slipped past.

The struct in the doc comment in debug/dwarf/type.go
was fixed up by hand to indent the first and last lines as well.

For #51082.

Change-Id: Iad020f83aafd671ff58238fe491907e85923d0c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/407137
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-19 15:49:05 +00:00
Ian Lance Taylor 8d074f61b7 archive/zip: error if using io/fs on zip with duplicate entries
Fixes #50390

Change-Id: I92787cdb3fa198ff88dcaadeccfcb49a3a6a88cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/374954
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-10 17:51:16 +00:00
Ian Lance Taylor df57592276 archive/zip: permit zip files to have prefixes
A Java jar file is a zip file, but it can have a prefix that is a bash
script that unpacks the zip file. Most zip programs ignore such prefixes.
This CL changes the archive/zip package to do the same.

Fixes #10464
Fixes #51337

Change-Id: I976e9c64684644317bd21077bc5b4a2baf626ee6
Reviewed-on: https://go-review.googlesource.com/c/go/+/387976
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-05-08 17:26:10 +00:00
Kale Blankenship 356c8c5452 archive/zip: remove unused File.descErr field
Found via staticcheck. Unused as of CL 357489.

Change-Id: I3aa409994ba4388912ac7e7809168529a5b6e31c
Reviewed-on: https://go-review.googlesource.com/c/go/+/403814
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Kale B <kale@lemnisys.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: David Chase <drchase@google.com>
2022-05-03 18:22:07 +00:00
Meng Zhuo b6fb3af6af archive/zip: fail fast if UncompressedSize64 < nread
The zip reader checks that the uncompressed file size is valid
after all compressed files read until EOF.
However in between reading each file, there could have already
been an overflow where nread > UncompressedSize64 hence this
change will now return ErrFormat in such situations.

Fixes #49791

Change-Id: If3584a57d173de6a97bf35c07d2a99ff6972f820
Reviewed-on: https://go-review.googlesource.com/c/go/+/366854
Trust: mzh <mzh@golangcn.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2022-04-11 01:24:31 +00:00
Russ Cox 690ac4071f all: remove trailing blank doc comment lines
A future change to gofmt will rewrite

	// Doc comment.
	//
	func f()

to

	// Doc comment.
	func f()

Apply that change preemptively to all doc comments.

For #51082.

Change-Id: I4023e16cfb0729b64a8590f071cd92f17343081d
Reviewed-on: https://go-review.googlesource.com/c/go/+/384259
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01 18:18:07 +00:00
Roland Shoemaker 4fa6e33f30 all: add a handful of fuzz targets
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip,
encoding/json, image/jpeg, image/gif, and image/png.

Second attempt, this time we don't use the archives in testdata when
fuzzing archive/tar, since those are rather memory intensive, and
were crashing a number of builders.

Change-Id: I4828d64fa4763c0d8c980392a6578e4dfd956e13
Reviewed-on: https://go-review.googlesource.com/c/go/+/378174
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-13 18:06:33 +00:00
Bryan Mills 6e8b7e4f42 Revert "all: add a handful of fuzz targets"
This reverts CL 352109.

Reason for revert: causing OOM failures on several builders, and may cause OOMs for end users with small machines as well.

Change-Id: I58308d09919969d5a6512ee5cee6aa5c4af6769b
Reviewed-on: https://go-review.googlesource.com/c/go/+/377934
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Katie Hockman <katie@golang.org>
2022-01-12 15:53:47 +00:00
Roland Shoemaker 9bce08999a all: add a handful of fuzz targets
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip,
encoding/json, image/jpeg, image/gif, and image/png.

Change-Id: Ide1a8de88a9421e786eeeaea3bb93f41e0bae347
Reviewed-on: https://go-review.googlesource.com/c/go/+/352109
Trust: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-12 00:12:10 +00:00
Russ Cox 2580d0e08d all: gofmt -w -r 'interface{} -> any' src
And then revert the bootstrap cmd directories and certain testdata.
And adjust tests as needed.

Not reverting the changes in std that are bootstrapped,
because some of those changes would appear in API docs,
and we want to use any consistently.
Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories
when preparing the bootstrap copy.

A few files changed as a result of running gofmt -w
not because of interface{} -> any but because they
hadn't been updated for the new //go:build lines.

Fixes #49884.

Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09
Reviewed-on: https://go-review.googlesource.com/c/go/+/368254
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13 18:45:54 +00:00
Arran Walker 85493d53e3 archive/zip: don't read data descriptor early
Go 1.17 introduced an unnecessary change to when a zip's data descriptor
is read for file entries, how it is parsed and how the crc32 field is
used.

Before Go 1.17, the data descriptor was read immediately after a file
entry's content. This continuous read is a pattern existing applications
have come to rely upon (for example, where reads at specific offsets
might be translated to HTTP range requests).

In Go 1.17, all data descriptors are immediately read upon opening the
file. This results in scattered and non-continuous reads of the archive,
and depending on the underlying reader, might have severe performance
implications. In addition, an additional object is now initialized for
each entry, but is mostly redundant.

Previously, the crc32 field in the data descriptor would return an error
if it did not match the central directory's entry. This check has
seemingly been unintentionally removed. If the central directory crc32
is invalid and a data descriptor is present, no error is returned.

This change reverts to the previous handling of data descriptors, before
CL 312310.

Fixes #48374
Fixes #49089

Change-Id: I5df2878c4fcc9e500064e7175f3ab9727c82f100
Reviewed-on: https://go-review.googlesource.com/c/go/+/357489
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-07 04:56:11 +00:00