| commit | f9415807773a38c984ff579c33da81cbd2968705 | |
| author | Angel Ortega <angel@triptico.com> | |
| Tue, 16 Jun 2009 15:43:22 +0000 (16 17:43 +0200) | ||
| committer | Angel Ortega <angel@triptico.com> | |
| Tue, 16 Jun 2009 15:43:22 +0000 (16 17:43 +0200) | ||
| tree | e0b6b24bbf0830b315f53f801c8e532a4ebb39aa | treesnapshot (tar.gz zip) |
| parent | 3d00b0b259831e4a745faf8bb8125fc6cd3c16d2 | commitdiff |
Special utf-8 detection in read_auto().
After trying different UTF BOMs, a valid utf-8 footprint is
checked in the first bunch of characters in the file. If it's not
valid utf-8, encoding is set back to iso8859-1. This is not
really correct; it can be koi-8 or another oddity. The right thing
would be to *set* encoding to utf-8 if explicitly detected and fall
back to locale encoding otherwise, but this will fail again on
utf-8 environments.
After trying different UTF BOMs, a valid utf-8 footprint is
checked in the first bunch of characters in the file. If it's not
valid utf-8, encoding is set back to iso8859-1. This is not
really correct; it can be koi-8 or another oddity. The right thing
would be to *set* encoding to utf-8 if explicitly detected and fall
back to locale encoding otherwise, but this will fail again on
utf-8 environments.
| mpdm_f.c | diffblobblamehistory |