freebsd-dev/lib/libarchive/test/test_pax_filename_encoding.tar.gz.uu
Tim Kientzle 60617bf578 A subtle point: "pax interchange format" mandates that all strings
(including pathname, gname, uname) be stored in UTF-8.  This usually
doesn't cause problems on FreeBSD because the "C" locale on FreeBSD
can convert any byte to Unicode/wchar_t and from there to UTF-8.  In
other locales (including the "C" locale on Linux which is really
ASCII), you can get into trouble with pathnames that cannot be
converted to UTF-8.

Libarchive's pax writer truncated pathnames and other strings at the
first nonconvertible character.  (ouch!)  Other archivers have worked
around this by storing unconvertible pathnames as raw binary, a
practice which has been sanctioned by the Austin group.  However,
libarchive's pax reader would segfault reading headers that weren't
proper UTF-8.  (ouch!)  Since bsdtar defaults to pax format, this
affects bsdtar rather heavily.

To correctly support the new "hdrcharset" header that is going into
SUS and to handle conversion failures in general, libarchive's pax reader
and writer have been overhauled fairly extensively.  They used to do
most of the pax header processing using wchar_t (Unicode); they now do
most of it using char so that common logic applies to either UTF-8 or
"binary" strings.

As a bonus, a number of extraneous conversions to/from wchar_t have
been eliminated, which should speed things up just a tad.

Thanks to: Bjoern Jacke for originally reporting this to me
Thanks to: Joerg Sonnenberger for noting a bad typo in my first draft of this
Thanks to: Gunnar Ritter for getting the standard fixed
MFC after: 5 days
2008-03-15 01:43:59 +00:00

11 lines
484 B
Plaintext

begin 644 test_pax_filename_encoding.tar.gz
M'XL(`)4;VT<``^V6STK#0!#&<\Y3[!/HS/Z-ASVHEQ1$BE[L<4T6$DP32:-$
MG\%'\Y%Z,*7$UEJLE"91NK_+P.P>OF'X^&9LZM":V):GYCYZ?YOFQ5W]\NH=
M%`"0G).FHA*P7I>@E`1!22E!`9,$4#"0'JD/*V,[3[/*E(V4*IW^^&_7^W(4
M\EG_"13)HZD2W6Y_WFS?IT"B9EZKD8(0+)"!6/3,EQZ5/BIR>QF.KB8GL7W6
M0>!3VC;2O-"<H>#\S,>@[>99FC]H](>>VM'2G>M7[/(_@-CP/V-4>`2Z$K3.
MD?L_M%E6#"W",1CC;_D_[SW_*;+-_!><N?SO@R;_D[B,$E/.;*4O1M?G-Q/_
L%T<!1\7V/@IP\<T=!7^![ER_8H_\%PI=_O>!RW^'P^$X3CX`98.>C@`4````
`
end