Commit Graph

13 Commits

Author SHA1 Message Date
Marcel van Lohuizen 8ec5a052ec unicode: upgrade to Unicode 13.0.0
Fixes #40755

Change-Id: I14b3977317994095db8ae1bd873c174641209356
Reviewed-on: https://go-review.googlesource.com/c/go/+/248765
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-20 13:41:13 +00:00
Marcel van Lohuizen 79a00a3fe8 unicode: upgrade to Unicode 12
This does not include an upgrade of golang.org/x/net.
This is optional and best done as a separate CL.

Change-Id: Ifecc3fb6e3b7fe026b4ddefbe637186a3445b0bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/204658
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2019-11-05 14:34:56 +00:00
Marcel van Lohuizen 34fc6ced76 all: update to Unicode 11
This does *not* update the vendored tables.
A commit updating these tables should follow soon,

Mostly generated running UNICODE_VERSION=11.0.0 in x/text.

Manually updated next.txt file.

Updates golang/go#27945.

Change-Id: I939a01e235aeca898ee9afc99a531e7ad8444e12
Reviewed-on: https://go-review.googlesource.com/c/go/+/154420
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-04-25 20:33:22 +00:00
Marcel van Lohuizen e308d41fe5 unicode: update table using new generator in x/text
The changes in Unicode 11 exposes a bug in maketables.go.
We update the Unicode 10 tables using a new generator
to minimize the changes upgrading to Unicode 11.

This change switches over the generation from core to that in
x/text. To properly update the tables one needs to run the generate
in x/text anyway, so this makes that official.

The RangeTable generator in x/text also generates slightly compacter
tables.

Updates golang/go#27945

See CL 154443

Change-Id: I6c59e082d5b8cd9e9332a32d8971061228581d66
Reviewed-on: https://go-review.googlesource.com/c/go/+/169617
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-04-24 15:22:37 +00:00
Wil Selwood d704b5c956 unicode: improve generated comments for categories
The comments on the category range tables in the unicode package are fairly
redundent and require an external source to translate into human readable
category names.

This adds a look up table with the category descriptions and uses it if
available when generating the comments for the range tables.

Fixes #28954

Change-Id: I853e2d270def6492c2c1dd2ad0ec761a74c04e5d
Reviewed-on: https://go-review.googlesource.com/c/151297
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-28 03:12:48 +00:00
Marcel van Lohuizen 2fd9549797 unicode: update to Unicode 10.0.0
Also includes all derived values as well as
vendored packages.

Generated by running
    UNICODE_VERSION=10.0.0 go generate
in golang.org/x/text

and modified by hand to add the tests and
entries in next.txt for new script and properties.

Closes Issue #21471

Change-Id: I1d10ee3887bd1fd3d5a756ee0d04bd6ec2814ba1
Reviewed-on: https://go-review.googlesource.com/63953
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
2017-10-24 12:42:35 +00:00
Russ Cox 82abd4152d unicode: move scripts from FoldCategories to FoldScripts
Copy-and-paste bug was putting scripts in the categories map.

Fixes #18186.

Change-Id: Ife9d9bdd346fe24e578dbb2a0aac7ef6e889ae68
Reviewed-on: https://go-review.googlesource.com/45830
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-06-15 01:05:31 +00:00
Brad Fitzpatrick 6914b0e3e3 runtime, unicode: use consistent banner for generated code
Per golang.org/s/generatedcode

Updates #nnn

Change-Id: Ia7513ef6bd26c20b62b57b29f7770684a315d389
Reviewed-on: https://go-review.googlesource.com/45470
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Matt Layher <mdlayher@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-06-13 05:33:40 +00:00
Marcel van Lohuizen a2a4db7375 unicode: upgrade to version 9.0.0
Changes beyond generated tables:
- Now supports aliases to handle deprecated
  property classes.
- Some Mongolian letters are now modifiers.

Other changes:
- strconv: newly generated table to be in sync
- regexp/syntax: updated maxFold

Fixes #16191

Change-Id: I56bdf21ee2f775f2a82d0465b3772faf5c24cb61
Reviewed-on: https://go-review.googlesource.com/24496
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2016-06-28 15:08:11 +00:00
Egon Elbre e607abbfd6 unicode: improve SimpleFold performance for ascii
This change significantly speeds up case-insensitive regexp matching.

benchmark                      old ns/op      new ns/op      delta
BenchmarkMatchEasy0i_32-8      2690           1473           -45.24%
BenchmarkMatchEasy0i_1K-8      80404          42269          -47.43%
BenchmarkMatchEasy0i_32K-8     3272187        2076118        -36.55%
BenchmarkMatchEasy0i_1M-8      104805990      66503805       -36.55%
BenchmarkMatchEasy0i_32M-8     3360192200     2126121600     -36.73%

benchmark                      old MB/s     new MB/s     speedup
BenchmarkMatchEasy0i_32-8      11.90        21.72        1.83x
BenchmarkMatchEasy0i_1K-8      12.74        24.23        1.90x
BenchmarkMatchEasy0i_32K-8     10.01        15.78        1.58x
BenchmarkMatchEasy0i_1M-8      10.00        15.77        1.58x
BenchmarkMatchEasy0i_32M-8     9.99         15.78        1.58x

Issue #13288

Change-Id: I94af7bb29e75d60b4f6ee760124867ab271b9642
Reviewed-on: https://go-review.googlesource.com/16943
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-04-26 21:59:50 +00:00
Marcel van Lohuizen c714bbbfd3 unicode: include rune 0 in RangeTables.
All of Go passes. No changes for the text repo.

Fixes #10153

Change-Id: I313369bf471c8974390a6d42075e5c54f6a81750
Reviewed-on: https://go-review.googlesource.com/13667
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2015-08-24 15:05:48 +00:00
Marcel van Lohuizen fe15da62f7 unicode: upgrade to 8.0.0
Not sure if I'm on time for 1.5; Unicode 8 just got released.

Straighforward upgrade. Only changed maketables.go to prevent it from adding
the Cherokee upper and lower case mappings. This change causes the caseOrbit
table to NOT change. Added tests to verify that the relevant functions still
produce the correct result, even for Cherokee.

Fixes #11309

Change-Id: I42850f5b3399bde125b002efc78eff96dbd86a08
Reviewed-on: https://go-review.googlesource.com/11286
Reviewed-by: Russ Cox <rsc@golang.org>
2015-06-26 18:01:29 +00:00
Russ Cox c007ce824d build: move package sources from src/pkg to src
Preparation was in CL 134570043.
This CL contains only the effect of 'hg mv src/pkg/* src'.
For more about the move, see golang.org/s/go14nopkg.
2014-09-08 00:08:51 -04:00