Commit Graph

34 Commits

Author SHA1 Message Date
Bjoern A. Zeeb
298ee47e19 localedef: unbreak WITHOUT_LOCALES
After 0fa5403d49 ("pkgbase: move locales into their own package") we
need usr.bin/localedef as a bootstrap tool independent on where
WITHOUT_LOCALE was specified as we ALWAYS process C.UTF-8.

At the same time LOCALES= in the local Makefile is empty but
C.UTF-8 with WITHOUT_LOCALES. C.UTF-8 is excluded from FILES, and thus
after the replacement FILES= is set to only .LC_CTYPE which results in
a build failure not knowing how to build that. Tweak the substitution to
replace only non-empty words so that FILES remains harmlessly empty.

Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D31589
2021-08-19 12:38:17 -05:00
Emmanuel Vadot
86c71e97d1 pkgbase: locales: Also tag the files dir
Otherwise bsd.dirs.mk will create the directory with the default
package (utilities) and we end up with a bunch of empty dirs managed
by this package while it shouldn't be the case.
2021-08-06 14:37:45 +02:00
Kyle Evans
b3e87f1334 ctypedef: fix installation of C.UTF-8
The appropriate directory and name were assigned to the FILESDIR
grouping, but not the ALWAYS grouping where C.UTF-8 is actually
assigned. Add the appropriate bits for ALWAYSDIR, and remove an
obsolete *PACKAGE= assignment since C.UTF-8 is explicitly not included
in FILES.

Prior to this change, C.UTF-8 was being installed as
/usr/share/C.UTF-8.LC_CTYPE.

Reviewed by:	manu
Fixes:	0fa5403d49 ("pkgbase: move locales into their own package")
Differential Revision:	https://reviews.freebsd.org/D31429
2021-08-05 22:31:40 -05:00
Baptiste Daroussin
0fa5403d49 pkgbase: move locales into their own package
The only exception here being C.UTF-8 as this is the default
locales so it needs to always be installed

Reviewed by:	pkgbase (emaste)
Differential Revision:	https://reviews.freebsd.org/D31397
2021-08-03 18:35:26 +02:00
Baptiste Daroussin
30e1773fde locales: fix abuse of bsd.dirs.mk
the way SAMEDIRS was defined was an abuse of bsd.dirs.mk resulting in
all the directory to be created in one single command, but DESTDIR is
only prepend once on the first element of the list

Switch to the properway to use bsd.dirs.mk
2021-08-03 18:02:15 +02:00
Baptiste Daroussin
184d64af13 locales: stop hardcoding the directories in the mtree
The framework knows how to create directories and tag them properly
for a the creation of a mtree, not need to hardcode all the locales
entries in bsd.usr.mk

This simplifies addition of new locales but also allow people building
with WITHOUT_LOCALES to end up with a directory full of empty files
2021-08-03 14:25:00 +02:00
Faraz Vahedi
bd90d8bbb1 locale: Add Farsi/Persian locales
MFC after:	3 weeks
Reviewed by:	farrokhi
Differential Revision:	https://reviews.freebsd.org/D24359
2021-07-23 16:58:20 +02:00
Baptiste Daroussin
3a04284df0 locales: upgrade CLDR to 39.0 and Unicode to 13.0.0
MFC after:	3 weeks
Relnotes:	yes
2021-07-23 16:58:20 +02:00
Baptiste Daroussin
ecff3c30b7 locales: now that C.UTF-8 is the default locale, always install it
MFC after:	3 weeks
2021-07-23 16:58:20 +02:00
Baptiste Daroussin
0a36787e4c locales: separate unicode from other locales
Unicode locales are maintained and to evolved in cldr.
Generating the other locales from the unicode gets more and more painful
over the time.

Let's freeze the other locales to cldr 34.0.
2021-07-23 16:58:20 +02:00
Hiroki Sato
fa9012aef1 Fix CTYPE for ja_JP.eucJP and ja_JP.SJIS.
PR:		163168
MFC after:	3 days
2020-06-29 03:23:13 +00:00
Yuri Pankov
f01edb6f5d Mark "private use area" characters as printable.
At least some of the characters in E000-F8FF range are used by Powerline
fonts, and having no attributes for these ranges in UnicodeData.txt
other than "Other, Private Use" it should be safe to mark all of them as
printable.  Some actually were before r340491, so this fixes the
regression introduced there as well.

PR:		240911
Reviewed by:	bapt
Tested by:	Daniel Ponte <amigan@gmail.com>
Differential Revision:	https://reviews.freebsd.org/D21850
2019-10-05 22:17:54 +00:00
Yuri Pankov
ea22b5bd75 Add ga_IE.UTF-8 locale.
PR:		228587
Submitted by:	Micil <micheal.maclochlainn@oegaillimh.ie> (LC_TIME)
Reviewed by:	bapt
Approved by:	kib (mentor, implicit)
MFC after:	3 days
Differential revision:	https://reviews.freebsd.org/D17997
2018-11-26 19:39:49 +00:00
Yuri Pankov
dbcf594ac9 Use UnicodeData.txt to create UTF-8 ctype map.
This should provide more complete coverage of currently defined Unicode
characters as compared to manually assembled one we use currently.

Comparison of original and new UTF-8 ctype maps by character class:

TYPE    ORIG    NEW
alnum   94229   126029
alpha   93557   125419
blank   4       2
cntrl   73      137685
digit   469     622
graph   109615  137203
lower   1478    2145
print   109641  137222
punct   3428    797
rune    110481  274907
space   33      24
upper   983     1781
xdigit  469     622

Large number of added cntrl definitions is due to the fact that private-use
planes are currently defined as such, this can change in the future.

Discussed with:	bapt
Approved by:	kib (mentor, implicit)
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D17842
2018-11-17 10:36:00 +00:00
Yuri Pankov
dd7c41a378 Add hybrid C.UTF-8 locale being identical to default C locale except
that it uses the same ctype maps and functions as other UTF-8 locales.

Reviewed by:	bapt, cem, eadler
Approved by:	kib (mentor, implicit)
Differential Revision:	https://reviews.freebsd.org/D17833
2018-11-04 22:13:22 +00:00
Yuri Pankov
1ade1c4113 Update to CLDR 34 and UNICODE 11.
Discussed with:	bapt
Approved by:	kib (mentor)
MFC after:	1 month
2018-11-03 22:02:10 +00:00
Yuri Pankov
4644f9bef6 Add -b/-l options to localedef(1) to specify output endianness and use
it appropriately when building share/ctypedef and share/colldef.

This makes the resulting locale data in EL->EB (amd64->powerpc64) cross
build and in the native EB build match.  Revert the changes done to libc
in r308170 as they are no longer needed.

PR:		231965
Reviewed by:	bapt, emaste, sbruno, 0mp
Approved by:	kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D17603
2018-10-20 20:51:05 +00:00
Yuri Pankov
98fc050515 Restore some of the ctype definitions reported in the PR from pre-CLDR
data, namely 0xE000-0xF8FF private use area, and 0xFF00-0xFFF half- and
fullwidth punctuation.

While here, update tools/tools/locale/README based on my experience
rebuilding the locale data.

PR:		225692
Reviewed by:	bapt, cem (previous version)
Approved by:	re (gjb), kib (mentor)
Differential Revision:	https://reviews.freebsd.org/D17471
2018-10-11 18:30:12 +00:00
Baptiste Daroussin
fe90c6bf15 Update to CLDR 32 and Unicode 10
MFC after:	2 weeks
Relnotes:	yes
2017-11-03 13:52:34 +00:00
Baptiste Daroussin
50502545ce Readd Big5: some large databases setup are still requiring it.
Reported by:	"張君天(Chun-Tien Chang)" <tcs@kitty.2y.idv.tw>
2017-04-20 18:21:50 +00:00
Baptiste Daroussin
ababdab0ce Remove zh_TW.Big5 locale
After discussion with many Taiwanese, in IT or not. Big5 is not used anymore.
It is not able to represent lots of the characters used in the language.
2017-03-19 17:52:46 +00:00
Enji Cooper
e44885ac01 share: normalize paths using SRCTOP-relative paths or :H when possible
This simplifies make logic/output

MFC after:	1 month
Sponsored by:	Dell EMC Isilon
2017-03-04 11:26:40 +00:00
Baptiste Daroussin
28fded59b4 Update locales cldr to v30.0.3 and unicode to 9.0.0
MFC after:	1 month
Relnotes:	yes
2016-12-18 04:17:13 +00:00
Baptiste Daroussin
3cf6509d70 Posixify the locales name for variants
For all locales with variants:
- if no ambiguity on the locale (only one variant) just use the regular name
- if ambiguity, pick one as default and append @<variant> to the others
  respecting POSIX

As a result:
- All the 3 components locales added recently are renamed to the usual 2
  components version for all but sr_RS.UTF-8
- Set sr_RS.UTF-8 to the cyrillic variant
- Add sr_RS.UTF-8@latin
- Remove the symlinks aliases they were created to represent the 2 components
  version as aliasas and are now useless
- Update the OptionalObsoleteFiles.inc and ObsoleteFiles.inc to reflect those
  changes

Discussed with:	ache@
Approved by:	re@ (gjb)
2016-07-03 18:21:11 +00:00
Baptiste Daroussin
110cf8a6f1 Regenerates locales Makefiles and locales 2016-04-16 17:55:11 +00:00
Baptiste Daroussin
4149149e1e Use :E instead of regex in ctypedef Makefiles 2015-12-19 12:26:01 +00:00
Baptiste Daroussin
f8fd13a099 Simplify the locale generated Makefiles by using bmake multi variables for loops 2015-12-18 21:34:28 +00:00
Bryan Drewery
393608176b META MODE: Fix 'make the-lot' with recent locale changes
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 19:13:28 +00:00
Baptiste Daroussin
1d211085ac Bring back a couple of ISO8859-15 locales:
- af_ZA.ISO8859-15
- en_AU.ISO8859-15
- en_CA.ISO8859-15
- en_NZ.ISO8859-15
- en_US.ISO8859-15
- fr_CA.ISO8859-15
2015-11-16 12:58:47 +00:00
Baptiste Daroussin
c1c9ab6707 Regenerate locales after readding ISO8859-1 for locales that have ISO8859-15
Requested by:	arche
2015-11-15 14:51:10 +00:00
Bryan Drewery
d4ea5890a8 Use explicit filename when creating locale symlinks to avoid creating a
directory symlink when the target directory does not exist.  This will
cause an error instead of a broken setup.

Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:45:48 +00:00
Baptiste Daroussin
3c3feed41a Merge from head 2015-11-01 21:17:38 +00:00
Baptiste Daroussin
7cefd8ca2a Regenerate locales 2015-10-13 20:24:57 +00:00
Baptiste Daroussin
1a41e1f147 Add newly generated locales from CLDR 27.0.1
only skip timedef until I figure out how to workaround svn painful mime-type handling
2015-08-08 01:45:53 +00:00