1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
3 2006 Free Software Foundation, Inc.
4 Copyright (C) 1995, 1997, 1998, 2002, 2003, 2004, 2005
5 National Institute of Advanced Industrial Science and Technology (AIST)
6 Registration Number H14PRO021
8 This file is part of GNU Emacs.
10 GNU Emacs is free software; you can redistribute it and/or modify
11 it under the terms of the GNU General Public License as published by
12 the Free Software Foundation; either version 2, or (at your option)
15 GNU Emacs is distributed in the hope that it will be useful,
16 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
18 GNU General Public License for more details.
20 You should have received a copy of the GNU General Public License
21 along with GNU Emacs; see the file COPYING. If not, write to
22 the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
23 Boston, MA 02110-1301, USA. */
25 /*** TABLE OF CONTENTS ***
29 2. Emacs' internal format (emacs-mule) handlers
31 4. Shift-JIS and BIG5 handlers
33 6. End-of-line handlers
34 7. C library functions
35 8. Emacs Lisp library functions
40 /*** 0. General comments ***/
43 /*** GENERAL NOTE on CODING SYSTEMS ***
45 A coding system is an encoding mechanism for one or more character
46 sets. Here's a list of coding systems which Emacs can handle. When
47 we say "decode", it means converting some other coding system to
48 Emacs' internal format (emacs-mule), and when we say "encode",
49 it means converting the coding system emacs-mule to some other
52 0. Emacs' internal format (emacs-mule)
54 Emacs itself holds a multi-lingual character in buffers and strings
55 in a special format. Details are described in section 2.
59 The most famous coding system for multiple character sets. X's
60 Compound Text, various EUCs (Extended Unix Code), and coding
61 systems used in Internet communication such as ISO-2022-JP are
62 all variants of ISO2022. Details are described in section 3.
64 2. SJIS (or Shift-JIS or MS-Kanji-Code)
66 A coding system to encode character sets: ASCII, JISX0201, and
67 JISX0208. Widely used for PC's in Japan. Details are described in
72 A coding system to encode the character sets ASCII and Big5. Widely
73 used for Chinese (mainly in Taiwan and Hong Kong). Details are
74 described in section 4. In this file, when we write "BIG5"
75 (all uppercase), we mean the coding system, and when we write
76 "Big5" (capitalized), we mean the character set.
80 A coding system for text containing random 8-bit code. Emacs does
81 no code conversion on such text except for end-of-line format.
85 If a user wants to read/write text encoded in a coding system not
86 listed above, he can supply a decoder and an encoder for it as CCL
87 (Code Conversion Language) programs. Emacs executes the CCL program
88 while reading/writing.
90 Emacs represents a coding system by a Lisp symbol that has a property
91 `coding-system'. But, before actually using the coding system, the
92 information about it is set in a structure of type `struct
93 coding_system' for rapid processing. See section 6 for more details.
97 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
99 How end-of-line of text is encoded depends on the operating system.
100 For instance, Unix's format is just one byte of `line-feed' code,
101 whereas DOS's format is two-byte sequence of `carriage-return' and
102 `line-feed' codes. MacOS's format is usually one byte of
105 Since text character encoding and end-of-line encoding are
106 independent, any coding system described above can have any
107 end-of-line format. So Emacs has information about end-of-line
108 format in each coding-system. See section 6 for more details.
112 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
114 These functions check if a text between SRC and SRC_END is encoded
115 in the coding system category XXX. Each returns an integer value in
116 which appropriate flag bits for the category XXX are set. The flag
117 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
118 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
119 of the range 0x80..0x9F are in multibyte form. */
122 detect_coding_emacs_mule (src
, src_end
, multibytep
)
123 unsigned char *src
, *src_end
;
130 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
132 These functions decode SRC_BYTES length of unibyte text at SOURCE
133 encoded in CODING to Emacs' internal format. The resulting
134 multibyte text goes to a place pointed to by DESTINATION, the length
135 of which should not exceed DST_BYTES.
137 These functions set the information about original and decoded texts
138 in the members `produced', `produced_char', `consumed', and
139 `consumed_char' of the structure *CODING. They also set the member
140 `result' to one of CODING_FINISH_XXX indicating how the decoding
143 DST_BYTES zero means that the source area and destination area are
144 overlapped, which means that we can produce a decoded text until it
145 reaches the head of the not-yet-decoded source text.
147 Below is a template for these functions. */
150 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
151 struct coding_system
*coding
;
152 const unsigned char *source
;
153 unsigned char *destination
;
154 int src_bytes
, dst_bytes
;
160 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
162 These functions encode SRC_BYTES length text at SOURCE from Emacs'
163 internal multibyte format to CODING. The resulting unibyte text
164 goes to a place pointed to by DESTINATION, the length of which
165 should not exceed DST_BYTES.
167 These functions set the information about original and encoded texts
168 in the members `produced', `produced_char', `consumed', and
169 `consumed_char' of the structure *CODING. They also set the member
170 `result' to one of CODING_FINISH_XXX indicating how the encoding
173 DST_BYTES zero means that the source area and destination area are
174 overlapped, which means that we can produce encoded text until it
175 reaches at the head of the not-yet-encoded source text.
177 Below is a template for these functions. */
180 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
181 struct coding_system
*coding
;
182 unsigned char *source
, *destination
;
183 int src_bytes
, dst_bytes
;
189 /*** COMMONLY USED MACROS ***/
191 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
192 get one, two, and three bytes from the source text respectively.
193 If there are not enough bytes in the source, they jump to
194 `label_end_of_loop'. The caller should set variables `coding',
195 `src' and `src_end' to appropriate pointer in advance. These
196 macros are called from decoding routines `decode_coding_XXX', thus
197 it is assumed that the source text is unibyte. */
199 #define ONE_MORE_BYTE(c1) \
201 if (src >= src_end) \
203 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
204 goto label_end_of_loop; \
209 #define TWO_MORE_BYTES(c1, c2) \
211 if (src + 1 >= src_end) \
213 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
214 goto label_end_of_loop; \
221 /* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
222 form if MULTIBYTEP is nonzero. */
224 #define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep) \
226 if (src >= src_end) \
228 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
229 goto label_end_of_loop; \
232 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
233 c1 = *src++ - 0x20; \
236 /* Set C to the next character at the source text pointed by `src'.
237 If there are not enough characters in the source, jump to
238 `label_end_of_loop'. The caller should set variables `coding'
239 `src', `src_end', and `translation_table' to appropriate pointers
240 in advance. This macro is used in encoding routines
241 `encode_coding_XXX', thus it assumes that the source text is in
242 multibyte form except for 8-bit characters. 8-bit characters are
243 in multibyte form if coding->src_multibyte is nonzero, else they
244 are represented by a single byte. */
246 #define ONE_MORE_CHAR(c) \
248 int len = src_end - src; \
252 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
253 goto label_end_of_loop; \
255 if (coding->src_multibyte \
256 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
257 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
259 c = *src, bytes = 1; \
260 if (!NILP (translation_table)) \
261 c = translate_char (translation_table, c, -1, 0, 0); \
266 /* Produce a multibyte form of character C to `dst'. Jump to
267 `label_end_of_loop' if there's not enough space at `dst'.
269 If we are now in the middle of a composition sequence, the decoded
270 character may be ALTCHAR (for the current composition). In that
271 case, the character goes to coding->cmp_data->data instead of
274 This macro is used in decoding routines. */
276 #define EMIT_CHAR(c) \
278 if (! COMPOSING_P (coding) \
279 || coding->composing == COMPOSITION_RELATIVE \
280 || coding->composing == COMPOSITION_WITH_RULE) \
282 int bytes = CHAR_BYTES (c); \
283 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
285 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
286 goto label_end_of_loop; \
288 dst += CHAR_STRING (c, dst); \
289 coding->produced_char++; \
292 if (COMPOSING_P (coding) \
293 && coding->composing != COMPOSITION_RELATIVE) \
295 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
296 coding->composition_rule_follows \
297 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
302 #define EMIT_ONE_BYTE(c) \
304 if (dst >= (dst_bytes ? dst_end : src)) \
306 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
307 goto label_end_of_loop; \
312 #define EMIT_TWO_BYTES(c1, c2) \
314 if (dst + 2 > (dst_bytes ? dst_end : src)) \
316 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
317 goto label_end_of_loop; \
319 *dst++ = c1, *dst++ = c2; \
322 #define EMIT_BYTES(from, to) \
324 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
326 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
327 goto label_end_of_loop; \
334 /*** 1. Preamble ***/
347 #include "composite.h"
351 #include "intervals.h"
353 #include "termhooks.h"
355 #else /* not emacs */
359 #endif /* not emacs */
361 Lisp_Object Qcoding_system
, Qeol_type
;
362 Lisp_Object Qbuffer_file_coding_system
;
363 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
364 Lisp_Object Qno_conversion
, Qundecided
;
365 Lisp_Object Qcoding_system_history
;
366 Lisp_Object Qsafe_chars
;
367 Lisp_Object Qvalid_codes
;
369 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
370 Lisp_Object Qcall_process
, Qcall_process_region
;
371 Lisp_Object Qstart_process
, Qopen_network_stream
;
372 Lisp_Object Qtarget_idx
;
374 /* If a symbol has this property, evaluate the value to define the
375 symbol as a coding system. */
376 Lisp_Object Qcoding_system_define_form
;
378 Lisp_Object Vselect_safe_coding_system_function
;
380 int coding_system_require_warning
;
382 /* Mnemonic string for each format of end-of-line. */
383 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
384 /* Mnemonic string to indicate format of end-of-line is not yet
386 Lisp_Object eol_mnemonic_undecided
;
388 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
389 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
394 /* Information about which coding system is safe for which chars.
395 The value has the form (GENERIC-LIST . NON-GENERIC-ALIST).
397 GENERIC-LIST is a list of generic coding systems which can encode
400 NON-GENERIC-ALIST is an alist of non generic coding systems vs the
401 corresponding char table that contains safe chars. */
402 Lisp_Object Vcoding_system_safe_chars
;
404 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
406 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
408 /* Coding system emacs-mule and raw-text are for converting only
409 end-of-line format. */
410 Lisp_Object Qemacs_mule
, Qraw_text
;
414 /* Coding-systems are handed between Emacs Lisp programs and C internal
415 routines by the following three variables. */
416 /* Coding-system for reading files and receiving data from process. */
417 Lisp_Object Vcoding_system_for_read
;
418 /* Coding-system for writing files and sending data to process. */
419 Lisp_Object Vcoding_system_for_write
;
420 /* Coding-system actually used in the latest I/O. */
421 Lisp_Object Vlast_coding_system_used
;
423 /* A vector of length 256 which contains information about special
424 Latin codes (especially for dealing with Microsoft codes). */
425 Lisp_Object Vlatin_extra_code_table
;
427 /* Flag to inhibit code conversion of end-of-line format. */
428 int inhibit_eol_conversion
;
430 /* Flag to inhibit ISO2022 escape sequence detection. */
431 int inhibit_iso_escape_detection
;
433 /* Flag to make buffer-file-coding-system inherit from process-coding. */
434 int inherit_process_coding_system
;
436 /* Coding system to be used to encode text for terminal display when
437 terminal coding system is nil. */
438 struct coding_system safe_terminal_coding
;
440 /* Default coding system to be used to write a file. */
441 struct coding_system default_buffer_file_coding
;
443 Lisp_Object Vfile_coding_system_alist
;
444 Lisp_Object Vprocess_coding_system_alist
;
445 Lisp_Object Vnetwork_coding_system_alist
;
447 Lisp_Object Vlocale_coding_system
;
451 Lisp_Object Qcoding_category
, Qcoding_category_index
;
453 /* List of symbols `coding-category-xxx' ordered by priority. */
454 Lisp_Object Vcoding_category_list
;
456 /* Table of coding categories (Lisp symbols). */
457 Lisp_Object Vcoding_category_table
;
459 /* Table of names of symbol for each coding-category. */
460 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
461 "coding-category-emacs-mule",
462 "coding-category-sjis",
463 "coding-category-iso-7",
464 "coding-category-iso-7-tight",
465 "coding-category-iso-8-1",
466 "coding-category-iso-8-2",
467 "coding-category-iso-7-else",
468 "coding-category-iso-8-else",
469 "coding-category-ccl",
470 "coding-category-big5",
471 "coding-category-utf-8",
472 "coding-category-utf-16-be",
473 "coding-category-utf-16-le",
474 "coding-category-raw-text",
475 "coding-category-binary"
478 /* Table of pointers to coding systems corresponding to each coding
480 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
482 /* Table of coding category masks. Nth element is a mask for a coding
483 category of which priority is Nth. */
485 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
487 /* Flag to tell if we look up translation table on character code
489 Lisp_Object Venable_character_translation
;
490 /* Standard translation table to look up on decoding (reading). */
491 Lisp_Object Vstandard_translation_table_for_decode
;
492 /* Standard translation table to look up on encoding (writing). */
493 Lisp_Object Vstandard_translation_table_for_encode
;
495 Lisp_Object Qtranslation_table
;
496 Lisp_Object Qtranslation_table_id
;
497 Lisp_Object Qtranslation_table_for_decode
;
498 Lisp_Object Qtranslation_table_for_encode
;
500 /* Alist of charsets vs revision number. */
501 Lisp_Object Vcharset_revision_alist
;
503 /* Default coding systems used for process I/O. */
504 Lisp_Object Vdefault_process_coding_system
;
506 /* Char table for translating Quail and self-inserting input. */
507 Lisp_Object Vtranslation_table_for_input
;
509 /* Global flag to tell that we can't call post-read-conversion and
510 pre-write-conversion functions. Usually the value is zero, but it
511 is set to 1 temporarily while such functions are running. This is
512 to avoid infinite recursive call. */
513 static int inhibit_pre_post_conversion
;
515 Lisp_Object Qchar_coding_system
;
517 /* Return `safe-chars' property of CODING_SYSTEM (symbol). Don't check
521 coding_safe_chars (coding_system
)
522 Lisp_Object coding_system
;
524 Lisp_Object coding_spec
, plist
, safe_chars
;
526 coding_spec
= Fget (coding_system
, Qcoding_system
);
527 plist
= XVECTOR (coding_spec
)->contents
[3];
528 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
529 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
532 #define CODING_SAFE_CHAR_P(safe_chars, c) \
533 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
536 /*** 2. Emacs internal format (emacs-mule) handlers ***/
538 /* Emacs' internal format for representation of multiple character
539 sets is a kind of multi-byte encoding, i.e. characters are
540 represented by variable-length sequences of one-byte codes.
542 ASCII characters and control characters (e.g. `tab', `newline') are
543 represented by one-byte sequences which are their ASCII codes, in
544 the range 0x00 through 0x7F.
546 8-bit characters of the range 0x80..0x9F are represented by
547 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
550 8-bit characters of the range 0xA0..0xFF are represented by
551 one-byte sequences which are their 8-bit code.
553 The other characters are represented by a sequence of `base
554 leading-code', optional `extended leading-code', and one or two
555 `position-code's. The length of the sequence is determined by the
556 base leading-code. Leading-code takes the range 0x81 through 0x9D,
557 whereas extended leading-code and position-code take the range 0xA0
558 through 0xFF. See `charset.h' for more details about leading-code
561 --- CODE RANGE of Emacs' internal format ---
565 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
566 eight-bit-graphic 0xA0..0xBF
567 ELSE 0x81..0x9D + [0xA0..0xFF]+
568 ---------------------------------------------
570 As this is the internal character representation, the format is
571 usually not used externally (i.e. in a file or in a data sent to a
572 process). But, it is possible to have a text externally in this
573 format (i.e. by encoding by the coding system `emacs-mule').
575 In that case, a sequence of one-byte codes has a slightly different
578 Firstly, all characters in eight-bit-control are represented by
579 one-byte sequences which are their 8-bit code.
581 Next, character composition data are represented by the byte
582 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
584 METHOD is 0xF0 plus one of composition method (enum
587 BYTES is 0xA0 plus the byte length of these composition data,
589 CHARS is 0xA0 plus the number of characters composed by these
592 COMPONENTs are characters of multibyte form or composition
593 rules encoded by two-byte of ASCII codes.
595 In addition, for backward compatibility, the following formats are
596 also recognized as composition data on decoding.
599 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
602 MSEQ is a multibyte form but in these special format:
603 ASCII: 0xA0 ASCII_CODE+0x80,
604 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
605 RULE is a one byte code of the range 0xA0..0xF0 that
606 represents a composition rule.
609 enum emacs_code_class_type emacs_code_class
[256];
611 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
612 Check if a text is encoded in Emacs' internal format. If it is,
613 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
616 detect_coding_emacs_mule (src
, src_end
, multibytep
)
617 unsigned char *src
, *src_end
;
622 /* Dummy for ONE_MORE_BYTE. */
623 struct coding_system dummy_coding
;
624 struct coding_system
*coding
= &dummy_coding
;
628 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
636 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
645 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
648 else if (c
>= 0x80 && c
< 0xA0)
651 /* Old leading code for a composite character. */
655 unsigned char *src_base
= src
- 1;
658 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
661 src
= src_base
+ bytes
;
666 return CODING_CATEGORY_MASK_EMACS_MULE
;
670 /* Record the starting position START and METHOD of one composition. */
672 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
674 struct composition_data *cmp_data = coding->cmp_data; \
675 int *data = cmp_data->data + cmp_data->used; \
676 coding->cmp_data_start = cmp_data->used; \
678 data[1] = cmp_data->char_offset + start; \
679 data[3] = (int) method; \
680 cmp_data->used += 4; \
683 /* Record the ending position END of the current composition. */
685 #define CODING_ADD_COMPOSITION_END(coding, end) \
687 struct composition_data *cmp_data = coding->cmp_data; \
688 int *data = cmp_data->data + coding->cmp_data_start; \
689 data[0] = cmp_data->used - coding->cmp_data_start; \
690 data[2] = cmp_data->char_offset + end; \
693 /* Record one COMPONENT (alternate character or composition rule). */
695 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
697 coding->cmp_data->data[coding->cmp_data->used++] = component; \
698 if (coding->cmp_data->used - coding->cmp_data_start \
699 == COMPOSITION_DATA_MAX_BUNCH_LENGTH) \
701 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
702 coding->composing = COMPOSITION_NO; \
707 /* Get one byte from a data pointed by SRC and increment SRC. If SRC
708 is not less than SRC_END, return -1 without incrementing Src. */
710 #define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
713 /* Decode a character represented as a component of composition
714 sequence of Emacs 20 style at SRC. Set C to that character, store
715 its multibyte form sequence at P, and set P to the end of that
716 sequence. If no valid character is found, set C to -1. */
718 #define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
722 c = SAFE_ONE_MORE_BYTE (); \
725 if (CHAR_HEAD_P (c)) \
727 else if (c == 0xA0) \
729 c = SAFE_ONE_MORE_BYTE (); \
738 else if (BASE_LEADING_CODE_P (c - 0x20)) \
740 unsigned char *p0 = p; \
744 bytes = BYTES_BY_CHAR_HEAD (c); \
747 c = SAFE_ONE_MORE_BYTE (); \
752 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes) \
753 || (coding->flags /* We are recovering a file. */ \
754 && p0[0] == LEADING_CODE_8_BIT_CONTROL \
755 && ! CHAR_HEAD_P (p0[1]))) \
756 c = STRING_CHAR (p0, bytes); \
765 /* Decode a composition rule represented as a component of composition
766 sequence of Emacs 20 style at SRC. Set C to the rule. If not
767 valid rule is found, set C to -1. */
769 #define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
771 c = SAFE_ONE_MORE_BYTE (); \
773 if (c < 0 || c >= 81) \
777 gref = c / 9, nref = c % 9; \
778 c = COMPOSITION_ENCODE_RULE (gref, nref); \
783 /* Decode composition sequence encoded by `emacs-mule' at the source
784 pointed by SRC. SRC_END is the end of source. Store information
785 of the composition in CODING->cmp_data.
787 For backward compatibility, decode also a composition sequence of
788 Emacs 20 style. In that case, the composition sequence contains
789 characters that should be extracted into a buffer or string. Store
790 those characters at *DESTINATION in multibyte form.
792 If we encounter an invalid byte sequence, return 0.
793 If we encounter an insufficient source or destination, or
794 insufficient space in CODING->cmp_data, return 1.
795 Otherwise, return consumed bytes in the source.
799 decode_composition_emacs_mule (coding
, src
, src_end
,
800 destination
, dst_end
, dst_bytes
)
801 struct coding_system
*coding
;
802 const unsigned char *src
, *src_end
;
803 unsigned char **destination
, *dst_end
;
806 unsigned char *dst
= *destination
;
807 int method
, data_len
, nchars
;
808 const unsigned char *src_base
= src
++;
809 /* Store components of composition. */
810 int component
[COMPOSITION_DATA_MAX_BUNCH_LENGTH
];
812 /* Store multibyte form of characters to be composed. This is for
813 Emacs 20 style composition sequence. */
814 unsigned char buf
[MAX_COMPOSITION_COMPONENTS
* MAX_MULTIBYTE_LENGTH
];
815 unsigned char *bufp
= buf
;
816 int c
, i
, gref
, nref
;
818 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
819 >= COMPOSITION_DATA_SIZE
)
821 coding
->result
= CODING_FINISH_INSUFFICIENT_CMP
;
826 if (c
- 0xF0 >= COMPOSITION_RELATIVE
827 && c
- 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS
)
832 with_rule
= (method
== COMPOSITION_WITH_RULE
833 || method
== COMPOSITION_WITH_RULE_ALTCHARS
);
837 || src_base
+ data_len
> src_end
)
843 for (ncomponent
= 0; src
< src_base
+ data_len
; ncomponent
++)
845 /* If it is longer than this, it can't be valid. */
846 if (ncomponent
>= COMPOSITION_DATA_MAX_BUNCH_LENGTH
)
849 if (ncomponent
% 2 && with_rule
)
851 ONE_MORE_BYTE (gref
);
853 ONE_MORE_BYTE (nref
);
855 c
= COMPOSITION_ENCODE_RULE (gref
, nref
);
860 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
861 || (coding
->flags
/* We are recovering a file. */
862 && src
[0] == LEADING_CODE_8_BIT_CONTROL
863 && ! CHAR_HEAD_P (src
[1])))
864 c
= STRING_CHAR (src
, bytes
);
869 component
[ncomponent
] = c
;
874 /* This may be an old Emacs 20 style format. See the comment at
875 the section 2 of this file. */
876 while (src
< src_end
&& !CHAR_HEAD_P (*src
)) src
++;
878 && !(coding
->mode
& CODING_MODE_LAST_BLOCK
))
879 goto label_end_of_loop
;
885 method
= COMPOSITION_RELATIVE
;
886 for (ncomponent
= 0; ncomponent
< MAX_COMPOSITION_COMPONENTS
;)
888 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
891 component
[ncomponent
++] = c
;
899 method
= COMPOSITION_WITH_RULE
;
901 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
906 ncomponent
< MAX_COMPOSITION_COMPONENTS
* 2 - 1;)
908 DECODE_EMACS_MULE_COMPOSITION_RULE (c
);
911 component
[ncomponent
++] = c
;
912 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
915 component
[ncomponent
++] = c
;
919 nchars
= (ncomponent
+ 1) / 2;
927 if (buf
== bufp
|| dst
+ (bufp
- buf
) <= (dst_bytes
? dst_end
: src
))
929 CODING_ADD_COMPOSITION_START (coding
, coding
->produced_char
, method
);
930 for (i
= 0; i
< ncomponent
; i
++)
931 CODING_ADD_COMPOSITION_COMPONENT (coding
, component
[i
]);
932 CODING_ADD_COMPOSITION_END (coding
, coding
->produced_char
+ nchars
);
935 unsigned char *p
= buf
;
936 EMIT_BYTES (p
, bufp
);
937 *destination
+= bufp
- buf
;
938 coding
->produced_char
+= nchars
;
940 return (src
- src_base
);
946 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
949 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
950 struct coding_system
*coding
;
951 const unsigned char *source
;
952 unsigned char *destination
;
953 int src_bytes
, dst_bytes
;
955 const unsigned char *src
= source
;
956 const unsigned char *src_end
= source
+ src_bytes
;
957 unsigned char *dst
= destination
;
958 unsigned char *dst_end
= destination
+ dst_bytes
;
959 /* SRC_BASE remembers the start position in source in each loop.
960 The loop will be exited when there's not enough source code, or
961 when there's not enough destination area to produce a
963 const unsigned char *src_base
;
965 coding
->produced_char
= 0;
966 while ((src_base
= src
) < src_end
)
968 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
];
969 const unsigned char *p
;
976 if (coding
->eol_type
== CODING_EOL_CR
)
978 else if (coding
->eol_type
== CODING_EOL_CRLF
)
988 coding
->produced_char
++;
991 else if (*src
== '\n')
993 if ((coding
->eol_type
== CODING_EOL_CR
994 || coding
->eol_type
== CODING_EOL_CRLF
)
995 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
997 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
998 goto label_end_of_loop
;
1001 coding
->produced_char
++;
1004 else if (*src
== 0x80 && coding
->cmp_data
)
1006 /* Start of composition data. */
1007 int consumed
= decode_composition_emacs_mule (coding
, src
, src_end
,
1011 goto label_end_of_loop
;
1012 else if (consumed
> 0)
1017 bytes
= CHAR_STRING (*src
, tmp
);
1021 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
1022 || (coding
->flags
/* We are recovering a file. */
1023 && src
[0] == LEADING_CODE_8_BIT_CONTROL
1024 && ! CHAR_HEAD_P (src
[1])))
1033 bytes
= BYTES_BY_CHAR_HEAD (*src
);
1035 for (i
= 1; i
< bytes
; i
++)
1038 if (CHAR_HEAD_P (c
))
1043 bytes
= CHAR_STRING (*src_base
, tmp
);
1052 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
1054 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
1057 while (bytes
--) *dst
++ = *p
++;
1058 coding
->produced_char
++;
1061 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1062 coding
->produced
= dst
- destination
;
1066 /* Encode composition data stored at DATA into a special byte sequence
1067 starting by 0x80. Update CODING->cmp_data_start and maybe
1068 CODING->cmp_data for the next call. */
1070 #define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1072 unsigned char buf[1024], *p0 = buf, *p; \
1073 int len = data[0]; \
1077 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1078 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1080 if (data[3] == COMPOSITION_WITH_RULE \
1081 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1083 p += CHAR_STRING (data[4], p); \
1084 for (i = 5; i < len; i += 2) \
1087 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1088 *p++ = 0x20 + gref; \
1089 *p++ = 0x20 + nref; \
1090 p += CHAR_STRING (data[i + 1], p); \
1095 for (i = 4; i < len; i++) \
1096 p += CHAR_STRING (data[i], p); \
1098 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1100 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1102 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1103 goto label_end_of_loop; \
1107 coding->cmp_data_start += data[0]; \
1108 if (coding->cmp_data_start == coding->cmp_data->used \
1109 && coding->cmp_data->next) \
1111 coding->cmp_data = coding->cmp_data->next; \
1112 coding->cmp_data_start = 0; \
1117 static void encode_eol
P_ ((struct coding_system
*, const unsigned char *,
1118 unsigned char *, int, int));
1121 encode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
1122 struct coding_system
*coding
;
1123 const unsigned char *source
;
1124 unsigned char *destination
;
1125 int src_bytes
, dst_bytes
;
1127 const unsigned char *src
= source
;
1128 const unsigned char *src_end
= source
+ src_bytes
;
1129 unsigned char *dst
= destination
;
1130 unsigned char *dst_end
= destination
+ dst_bytes
;
1131 const unsigned char *src_base
;
1136 Lisp_Object translation_table
;
1138 translation_table
= Qnil
;
1140 /* Optimization for the case that there's no composition. */
1141 if (!coding
->cmp_data
|| coding
->cmp_data
->used
== 0)
1143 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
1147 char_offset
= coding
->cmp_data
->char_offset
;
1148 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1153 /* If SRC starts a composition, encode the information about the
1154 composition in advance. */
1155 if (coding
->cmp_data_start
< coding
->cmp_data
->used
1156 && char_offset
+ coding
->consumed_char
== data
[1])
1158 ENCODE_COMPOSITION_EMACS_MULE (coding
, data
);
1159 char_offset
= coding
->cmp_data
->char_offset
;
1160 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1164 if (c
== '\n' && (coding
->eol_type
== CODING_EOL_CRLF
1165 || coding
->eol_type
== CODING_EOL_CR
))
1167 if (coding
->eol_type
== CODING_EOL_CRLF
)
1168 EMIT_TWO_BYTES ('\r', c
);
1170 EMIT_ONE_BYTE ('\r');
1172 else if (SINGLE_BYTE_CHAR_P (c
))
1174 if (coding
->flags
&& ! ASCII_BYTE_P (c
))
1176 /* As we are auto saving, retain the multibyte form for
1178 unsigned char buf
[MAX_MULTIBYTE_LENGTH
];
1179 int bytes
= CHAR_STRING (c
, buf
);
1182 EMIT_ONE_BYTE (buf
[0]);
1184 EMIT_TWO_BYTES (buf
[0], buf
[1]);
1190 EMIT_BYTES (src_base
, src
);
1191 coding
->consumed_char
++;
1194 coding
->consumed
= src_base
- source
;
1195 coding
->produced
= coding
->produced_char
= dst
- destination
;
1200 /*** 3. ISO2022 handlers ***/
1202 /* The following note describes the coding system ISO2022 briefly.
1203 Since the intention of this note is to help understand the
1204 functions in this file, some parts are NOT ACCURATE or are OVERLY
1205 SIMPLIFIED. For thorough understanding, please refer to the
1206 original document of ISO2022. This is equivalent to the standard
1207 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1209 ISO2022 provides many mechanisms to encode several character sets
1210 in 7-bit and 8-bit environments. For 7-bit environments, all text
1211 is encoded using bytes less than 128. This may make the encoded
1212 text a little bit longer, but the text passes more easily through
1213 several types of gateway, some of which strip off the MSB (Most
1216 There are two kinds of character sets: control character sets and
1217 graphic character sets. The former contain control characters such
1218 as `newline' and `escape' to provide control functions (control
1219 functions are also provided by escape sequences). The latter
1220 contain graphic characters such as 'A' and '-'. Emacs recognizes
1221 two control character sets and many graphic character sets.
1223 Graphic character sets are classified into one of the following
1224 four classes, according to the number of bytes (DIMENSION) and
1225 number of characters in one dimension (CHARS) of the set:
1226 - DIMENSION1_CHARS94
1227 - DIMENSION1_CHARS96
1228 - DIMENSION2_CHARS94
1229 - DIMENSION2_CHARS96
1231 In addition, each character set is assigned an identification tag,
1232 unique for each set, called the "final character" (denoted as <F>
1233 hereafter). The <F> of each character set is decided by ECMA(*)
1234 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1235 (0x30..0x3F are for private use only).
1237 Note (*): ECMA = European Computer Manufacturers Association
1239 Here are examples of graphic character sets [NAME(<F>)]:
1240 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1241 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1242 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1243 o DIMENSION2_CHARS96 -- none for the moment
1245 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
1246 C0 [0x00..0x1F] -- control character plane 0
1247 GL [0x20..0x7F] -- graphic character plane 0
1248 C1 [0x80..0x9F] -- control character plane 1
1249 GR [0xA0..0xFF] -- graphic character plane 1
1251 A control character set is directly designated and invoked to C0 or
1252 C1 by an escape sequence. The most common case is that:
1253 - ISO646's control character set is designated/invoked to C0, and
1254 - ISO6429's control character set is designated/invoked to C1,
1255 and usually these designations/invocations are omitted in encoded
1256 text. In a 7-bit environment, only C0 can be used, and a control
1257 character for C1 is encoded by an appropriate escape sequence to
1258 fit into the environment. All control characters for C1 are
1259 defined to have corresponding escape sequences.
1261 A graphic character set is at first designated to one of four
1262 graphic registers (G0 through G3), then these graphic registers are
1263 invoked to GL or GR. These designations and invocations can be
1264 done independently. The most common case is that G0 is invoked to
1265 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1266 these invocations and designations are omitted in encoded text.
1267 In a 7-bit environment, only GL can be used.
1269 When a graphic character set of CHARS94 is invoked to GL, codes
1270 0x20 and 0x7F of the GL area work as control characters SPACE and
1271 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1274 There are two ways of invocation: locking-shift and single-shift.
1275 With locking-shift, the invocation lasts until the next different
1276 invocation, whereas with single-shift, the invocation affects the
1277 following character only and doesn't affect the locking-shift
1278 state. Invocations are done by the following control characters or
1281 ----------------------------------------------------------------------
1282 abbrev function cntrl escape seq description
1283 ----------------------------------------------------------------------
1284 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1285 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1286 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1287 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1288 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1289 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1290 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1291 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1292 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
1293 ----------------------------------------------------------------------
1294 (*) These are not used by any known coding system.
1296 Control characters for these functions are defined by macros
1297 ISO_CODE_XXX in `coding.h'.
1299 Designations are done by the following escape sequences:
1300 ----------------------------------------------------------------------
1301 escape sequence description
1302 ----------------------------------------------------------------------
1303 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1304 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1305 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1306 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1307 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1308 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1309 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1310 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1311 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1312 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1313 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1314 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1315 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1316 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1317 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1318 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1319 ----------------------------------------------------------------------
1321 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
1322 of dimension 1, chars 94, and final character <F>, etc...
1324 Note (*): Although these designations are not allowed in ISO2022,
1325 Emacs accepts them on decoding, and produces them on encoding
1326 CHARS96 character sets in a coding system which is characterized as
1327 7-bit environment, non-locking-shift, and non-single-shift.
1329 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1330 '(' can be omitted. We refer to this as "short-form" hereafter.
1332 Now you may notice that there are a lot of ways of encoding the
1333 same multilingual text in ISO2022. Actually, there exist many
1334 coding systems such as Compound Text (used in X11's inter client
1335 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1336 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
1337 localized platforms), and all of these are variants of ISO2022.
1339 In addition to the above, Emacs handles two more kinds of escape
1340 sequences: ISO6429's direction specification and Emacs' private
1341 sequence for specifying character composition.
1343 ISO6429's direction specification takes the following form:
1344 o CSI ']' -- end of the current direction
1345 o CSI '0' ']' -- end of the current direction
1346 o CSI '1' ']' -- start of left-to-right text
1347 o CSI '2' ']' -- start of right-to-left text
1348 The control character CSI (0x9B: control sequence introducer) is
1349 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1351 Character composition specification takes the following form:
1352 o ESC '0' -- start relative composition
1353 o ESC '1' -- end composition
1354 o ESC '2' -- start rule-base composition (*)
1355 o ESC '3' -- start relative composition with alternate chars (**)
1356 o ESC '4' -- start rule-base composition with alternate chars (**)
1357 Since these are not standard escape sequences of any ISO standard,
1358 the use of them with these meanings is restricted to Emacs only.
1360 (*) This form is used only in Emacs 20.5 and older versions,
1361 but the newer versions can safely decode it.
1362 (**) This form is used only in Emacs 21.1 and newer versions,
1363 and the older versions can't decode it.
1365 Here's a list of example usages of these composition escape
1366 sequences (categorized by `enum composition_method').
1368 COMPOSITION_RELATIVE:
1369 ESC 0 CHAR [ CHAR ] ESC 1
1370 COMPOSITION_WITH_RULE:
1371 ESC 2 CHAR [ RULE CHAR ] ESC 1
1372 COMPOSITION_WITH_ALTCHARS:
1373 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
1374 COMPOSITION_WITH_RULE_ALTCHARS:
1375 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
1377 enum iso_code_class_type iso_code_class
[256];
1379 #define CHARSET_OK(idx, charset, c) \
1380 (coding_system_table[idx] \
1381 && (charset == CHARSET_ASCII \
1382 || (safe_chars = coding_safe_chars (coding_system_table[idx]->symbol), \
1383 CODING_SAFE_CHAR_P (safe_chars, c))) \
1384 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1386 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
1388 #define SHIFT_OUT_OK(idx) \
1389 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1391 #define COMPOSITION_OK(idx) \
1392 (coding_system_table[idx]->composing != COMPOSITION_DISABLED)
1394 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1395 Check if a text is encoded in ISO2022. If it is, return an
1396 integer in which appropriate flag bits any of:
1397 CODING_CATEGORY_MASK_ISO_7
1398 CODING_CATEGORY_MASK_ISO_7_TIGHT
1399 CODING_CATEGORY_MASK_ISO_8_1
1400 CODING_CATEGORY_MASK_ISO_8_2
1401 CODING_CATEGORY_MASK_ISO_7_ELSE
1402 CODING_CATEGORY_MASK_ISO_8_ELSE
1403 are set. If a code which should never appear in ISO2022 is found,
1407 detect_coding_iso2022 (src
, src_end
, multibytep
)
1408 unsigned char *src
, *src_end
;
1411 int mask
= CODING_CATEGORY_MASK_ISO
;
1413 int reg
[4], shift_out
= 0, single_shifting
= 0;
1415 /* Dummy for ONE_MORE_BYTE. */
1416 struct coding_system dummy_coding
;
1417 struct coding_system
*coding
= &dummy_coding
;
1418 Lisp_Object safe_chars
;
1420 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
1421 while (mask
&& src
< src_end
)
1423 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1428 if (inhibit_iso_escape_detection
)
1430 single_shifting
= 0;
1431 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1432 if (c
>= '(' && c
<= '/')
1434 /* Designation sequence for a charset of dimension 1. */
1435 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1436 if (c1
< ' ' || c1
>= 0x80
1437 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
1438 /* Invalid designation sequence. Just ignore. */
1440 reg
[(c
- '(') % 4] = charset
;
1444 /* Designation sequence for a charset of dimension 2. */
1445 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1446 if (c
>= '@' && c
<= 'B')
1447 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1448 reg
[0] = charset
= iso_charset_table
[1][0][c
];
1449 else if (c
>= '(' && c
<= '/')
1451 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1452 if (c1
< ' ' || c1
>= 0x80
1453 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
1454 /* Invalid designation sequence. Just ignore. */
1456 reg
[(c
- '(') % 4] = charset
;
1459 /* Invalid designation sequence. Just ignore. */
1462 else if (c
== 'N' || c
== 'O')
1464 /* ESC <Fe> for SS2 or SS3. */
1465 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1468 else if (c
>= '0' && c
<= '4')
1470 /* ESC <Fp> for start/end composition. */
1471 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7
))
1472 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1474 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1475 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
))
1476 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1478 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1479 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_1
))
1480 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1482 mask
&= ~CODING_CATEGORY_MASK_ISO_8_1
;
1483 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_2
))
1484 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1486 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1487 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
))
1488 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1490 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1491 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
))
1492 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1494 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1498 /* Invalid escape sequence. Just ignore. */
1501 /* We found a valid designation sequence for CHARSET. */
1502 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
1503 c
= MAKE_CHAR (charset
, 0, 0);
1504 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
1505 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1507 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1508 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
1509 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1511 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1512 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
1513 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1515 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1516 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
1517 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1519 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1523 if (inhibit_iso_escape_detection
)
1525 single_shifting
= 0;
1528 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
1529 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
1531 /* Locking shift out. */
1532 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1533 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1538 if (inhibit_iso_escape_detection
)
1540 single_shifting
= 0;
1543 /* Locking shift in. */
1544 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1545 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1550 single_shifting
= 0;
1554 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1556 if (inhibit_iso_escape_detection
)
1558 if (c
!= ISO_CODE_CSI
)
1560 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1561 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1562 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1563 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1564 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1565 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1566 single_shifting
= 1;
1568 if (VECTORP (Vlatin_extra_code_table
)
1569 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1571 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1572 & CODING_FLAG_ISO_LATIN_EXTRA
)
1573 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1574 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1575 & CODING_FLAG_ISO_LATIN_EXTRA
)
1576 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1579 mask_found
|= newmask
;
1586 single_shifting
= 0;
1591 single_shifting
= 0;
1592 if (VECTORP (Vlatin_extra_code_table
)
1593 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1597 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1598 & CODING_FLAG_ISO_LATIN_EXTRA
)
1599 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1600 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1601 & CODING_FLAG_ISO_LATIN_EXTRA
)
1602 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1604 mask_found
|= newmask
;
1611 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1612 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1613 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1614 /* Check the length of succeeding codes of the range
1615 0xA0..0FF. If the byte length is odd, we exclude
1616 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1617 when we are not single shifting. */
1618 if (!single_shifting
1619 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1624 while (src
< src_end
)
1626 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1632 if (i
& 1 && src
< src_end
)
1633 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1635 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1637 /* This means that we have read one extra byte. */
1645 return (mask
& mask_found
);
1648 /* Decode a character of which charset is CHARSET, the 1st position
1649 code is C1, the 2nd position code is C2, and return the decoded
1650 character code. If the variable `translation_table' is non-nil,
1651 returned the translated code. */
1653 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1654 (NILP (translation_table) \
1655 ? MAKE_CHAR (charset, c1, c2) \
1656 : translate_char (translation_table, -1, charset, c1, c2))
1658 /* Set designation state into CODING. */
1659 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1663 if (final_char < '0' || final_char >= 128) \
1664 goto label_invalid_code; \
1665 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1666 make_number (chars), \
1667 make_number (final_char)); \
1668 c = MAKE_CHAR (charset, 0, 0); \
1670 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1671 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1673 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1675 && charset == CHARSET_ASCII) \
1677 /* We should insert this designation sequence as is so \
1678 that it is surely written back to a file. */ \
1679 coding->spec.iso2022.last_invalid_designation_register = -1; \
1680 goto label_invalid_code; \
1682 coding->spec.iso2022.last_invalid_designation_register = -1; \
1683 if ((coding->mode & CODING_MODE_DIRECTION) \
1684 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1685 charset = CHARSET_REVERSE_CHARSET (charset); \
1686 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1690 coding->spec.iso2022.last_invalid_designation_register = reg; \
1691 goto label_invalid_code; \
1695 /* Allocate a memory block for storing information about compositions.
1696 The block is chained to the already allocated blocks. */
1699 coding_allocate_composition_data (coding
, char_offset
)
1700 struct coding_system
*coding
;
1703 struct composition_data
*cmp_data
1704 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1706 cmp_data
->char_offset
= char_offset
;
1708 cmp_data
->prev
= coding
->cmp_data
;
1709 cmp_data
->next
= NULL
;
1710 if (coding
->cmp_data
)
1711 coding
->cmp_data
->next
= cmp_data
;
1712 coding
->cmp_data
= cmp_data
;
1713 coding
->cmp_data_start
= 0;
1714 coding
->composing
= COMPOSITION_NO
;
1717 /* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1718 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1719 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1720 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1721 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1724 #define DECODE_COMPOSITION_START(c1) \
1726 if (coding->composing == COMPOSITION_DISABLED) \
1728 *dst++ = ISO_CODE_ESC; \
1729 *dst++ = c1 & 0x7f; \
1730 coding->produced_char += 2; \
1732 else if (!COMPOSING_P (coding)) \
1734 /* This is surely the start of a composition. We must be sure \
1735 that coding->cmp_data has enough space to store the \
1736 information about the composition. If not, terminate the \
1737 current decoding loop, allocate one more memory block for \
1738 coding->cmp_data in the caller, then start the decoding \
1739 loop again. We can't allocate memory here directly because \
1740 it may cause buffer/string relocation. */ \
1741 if (!coding->cmp_data \
1742 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1743 >= COMPOSITION_DATA_SIZE)) \
1745 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1746 goto label_end_of_loop; \
1748 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1749 : c1 == '2' ? COMPOSITION_WITH_RULE \
1750 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1751 : COMPOSITION_WITH_RULE_ALTCHARS); \
1752 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1753 coding->composing); \
1754 coding->composition_rule_follows = 0; \
1758 /* We are already handling a composition. If the method is \
1759 the following two, the codes following the current escape \
1760 sequence are actual characters stored in a buffer. */ \
1761 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1762 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1764 coding->composing = COMPOSITION_RELATIVE; \
1765 coding->composition_rule_follows = 0; \
1770 /* Handle composition end sequence ESC 1. */
1772 #define DECODE_COMPOSITION_END(c1) \
1774 if (! COMPOSING_P (coding)) \
1776 *dst++ = ISO_CODE_ESC; \
1778 coding->produced_char += 2; \
1782 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1783 coding->composing = COMPOSITION_NO; \
1787 /* Decode a composition rule from the byte C1 (and maybe one more byte
1788 from SRC) and store one encoded composition rule in
1789 coding->cmp_data. */
1791 #define DECODE_COMPOSITION_RULE(c1) \
1795 if (c1 < 81) /* old format (before ver.21) */ \
1797 int gref = (c1) / 9; \
1798 int nref = (c1) % 9; \
1799 if (gref == 4) gref = 10; \
1800 if (nref == 4) nref = 10; \
1801 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1803 else if (c1 < 93) /* new format (after ver.21) */ \
1805 ONE_MORE_BYTE (c2); \
1806 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1808 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1809 coding->composition_rule_follows = 0; \
1813 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1816 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1817 struct coding_system
*coding
;
1818 const unsigned char *source
;
1819 unsigned char *destination
;
1820 int src_bytes
, dst_bytes
;
1822 const unsigned char *src
= source
;
1823 const unsigned char *src_end
= source
+ src_bytes
;
1824 unsigned char *dst
= destination
;
1825 unsigned char *dst_end
= destination
+ dst_bytes
;
1826 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1827 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1828 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1829 /* SRC_BASE remembers the start position in source in each loop.
1830 The loop will be exited when there's not enough source code
1831 (within macro ONE_MORE_BYTE), or when there's not enough
1832 destination area to produce a character (within macro
1834 const unsigned char *src_base
;
1836 Lisp_Object translation_table
;
1837 Lisp_Object safe_chars
;
1839 safe_chars
= coding_safe_chars (coding
->symbol
);
1841 if (NILP (Venable_character_translation
))
1842 translation_table
= Qnil
;
1845 translation_table
= coding
->translation_table_for_decode
;
1846 if (NILP (translation_table
))
1847 translation_table
= Vstandard_translation_table_for_decode
;
1850 coding
->result
= CODING_FINISH_NORMAL
;
1859 /* We produce no character or one character. */
1860 switch (iso_code_class
[c1
])
1862 case ISO_0x20_or_0x7F
:
1863 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1865 DECODE_COMPOSITION_RULE (c1
);
1868 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1870 /* This is SPACE or DEL. */
1871 charset
= CHARSET_ASCII
;
1874 /* This is a graphic character, we fall down ... */
1876 case ISO_graphic_plane_0
:
1877 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1879 DECODE_COMPOSITION_RULE (c1
);
1885 case ISO_0xA0_or_0xFF
:
1886 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1887 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1888 goto label_invalid_code
;
1889 /* This is a graphic character, we fall down ... */
1891 case ISO_graphic_plane_1
:
1893 goto label_invalid_code
;
1898 if (COMPOSING_P (coding
))
1899 DECODE_COMPOSITION_END ('1');
1901 /* All ISO2022 control characters in this class have the
1902 same representation in Emacs internal format. */
1904 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1905 && (coding
->eol_type
== CODING_EOL_CR
1906 || coding
->eol_type
== CODING_EOL_CRLF
))
1908 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1909 goto label_end_of_loop
;
1911 charset
= CHARSET_ASCII
;
1915 if (COMPOSING_P (coding
))
1916 DECODE_COMPOSITION_END ('1');
1917 goto label_invalid_code
;
1919 case ISO_carriage_return
:
1920 if (COMPOSING_P (coding
))
1921 DECODE_COMPOSITION_END ('1');
1923 if (coding
->eol_type
== CODING_EOL_CR
)
1925 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1928 if (c1
!= ISO_CODE_LF
)
1934 charset
= CHARSET_ASCII
;
1938 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1939 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1940 goto label_invalid_code
;
1941 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1942 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1946 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1947 goto label_invalid_code
;
1948 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1949 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1952 case ISO_single_shift_2_7
:
1953 case ISO_single_shift_2
:
1954 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1955 goto label_invalid_code
;
1956 /* SS2 is handled as an escape sequence of ESC 'N' */
1958 goto label_escape_sequence
;
1960 case ISO_single_shift_3
:
1961 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1962 goto label_invalid_code
;
1963 /* SS2 is handled as an escape sequence of ESC 'O' */
1965 goto label_escape_sequence
;
1967 case ISO_control_sequence_introducer
:
1968 /* CSI is handled as an escape sequence of ESC '[' ... */
1970 goto label_escape_sequence
;
1974 label_escape_sequence
:
1975 /* Escape sequences handled by Emacs are invocation,
1976 designation, direction specification, and character
1977 composition specification. */
1980 case '&': /* revision of following character set */
1982 if (!(c1
>= '@' && c1
<= '~'))
1983 goto label_invalid_code
;
1985 if (c1
!= ISO_CODE_ESC
)
1986 goto label_invalid_code
;
1988 goto label_escape_sequence
;
1990 case '$': /* designation of 2-byte character set */
1991 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1992 goto label_invalid_code
;
1994 if (c1
>= '@' && c1
<= 'B')
1995 { /* designation of JISX0208.1978, GB2312.1980,
1997 DECODE_DESIGNATION (0, 2, 94, c1
);
1999 else if (c1
>= 0x28 && c1
<= 0x2B)
2000 { /* designation of DIMENSION2_CHARS94 character set */
2002 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
2004 else if (c1
>= 0x2C && c1
<= 0x2F)
2005 { /* designation of DIMENSION2_CHARS96 character set */
2007 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
2010 goto label_invalid_code
;
2011 /* We must update these variables now. */
2012 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2013 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2016 case 'n': /* invocation of locking-shift-2 */
2017 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2018 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2019 goto label_invalid_code
;
2020 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
2021 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2024 case 'o': /* invocation of locking-shift-3 */
2025 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2026 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2027 goto label_invalid_code
;
2028 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
2029 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2032 case 'N': /* invocation of single-shift-2 */
2033 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2034 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2035 goto label_invalid_code
;
2036 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
2038 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2039 goto label_invalid_code
;
2042 case 'O': /* invocation of single-shift-3 */
2043 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2044 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2045 goto label_invalid_code
;
2046 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
2048 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2049 goto label_invalid_code
;
2052 case '0': case '2': case '3': case '4': /* start composition */
2053 DECODE_COMPOSITION_START (c1
);
2056 case '1': /* end composition */
2057 DECODE_COMPOSITION_END (c1
);
2060 case '[': /* specification of direction */
2061 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
2062 goto label_invalid_code
;
2063 /* For the moment, nested direction is not supported.
2064 So, `coding->mode & CODING_MODE_DIRECTION' zero means
2065 left-to-right, and nonzero means right-to-left. */
2069 case ']': /* end of the current direction */
2070 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2072 case '0': /* end of the current direction */
2073 case '1': /* start of left-to-right direction */
2076 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2078 goto label_invalid_code
;
2081 case '2': /* start of right-to-left direction */
2084 coding
->mode
|= CODING_MODE_DIRECTION
;
2086 goto label_invalid_code
;
2090 goto label_invalid_code
;
2095 if (COMPOSING_P (coding
))
2096 DECODE_COMPOSITION_END ('1');
2100 /* CTEXT extended segment:
2101 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
2102 We keep these bytes as is for the moment.
2103 They may be decoded by post-read-conversion. */
2108 ONE_MORE_BYTE (dim
);
2111 size
= ((M
- 128) * 128) + (L
- 128);
2112 required
= 8 + size
* 2;
2113 if (dst
+ required
> (dst_bytes
? dst_end
: src
))
2114 goto label_end_of_loop
;
2115 *dst
++ = ISO_CODE_ESC
;
2120 dst
+= CHAR_STRING (M
, dst
), produced_chars
++;
2121 dst
+= CHAR_STRING (L
, dst
), produced_chars
++;
2125 dst
+= CHAR_STRING (c1
, dst
), produced_chars
++;
2127 coding
->produced_char
+= produced_chars
;
2131 unsigned char *d
= dst
;
2134 /* XFree86 extension for embedding UTF-8 in CTEXT:
2135 ESC % G --UTF-8-BYTES-- ESC % @
2136 We keep these bytes as is for the moment.
2137 They may be decoded by post-read-conversion. */
2138 if (d
+ 6 > (dst_bytes
? dst_end
: src
))
2139 goto label_end_of_loop
;
2140 *d
++ = ISO_CODE_ESC
;
2144 while (d
+ 1 < (dst_bytes
? dst_end
: src
))
2147 if (c1
== ISO_CODE_ESC
2148 && src
+ 1 < src_end
2155 d
+= CHAR_STRING (c1
, d
), produced_chars
++;
2157 if (d
+ 3 > (dst_bytes
? dst_end
: src
))
2158 goto label_end_of_loop
;
2159 *d
++ = ISO_CODE_ESC
;
2163 coding
->produced_char
+= produced_chars
+ 3;
2166 goto label_invalid_code
;
2170 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
2171 goto label_invalid_code
;
2172 if (c1
>= 0x28 && c1
<= 0x2B)
2173 { /* designation of DIMENSION1_CHARS94 character set */
2175 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
2177 else if (c1
>= 0x2C && c1
<= 0x2F)
2178 { /* designation of DIMENSION1_CHARS96 character set */
2180 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
2183 goto label_invalid_code
;
2184 /* We must update these variables now. */
2185 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2186 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2191 /* Now we know CHARSET and 1st position code C1 of a character.
2192 Produce a multibyte sequence for that character while getting
2193 2nd position code C2 if necessary. */
2194 if (CHARSET_DIMENSION (charset
) == 2)
2197 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
2198 /* C2 is not in a valid range. */
2199 goto label_invalid_code
;
2201 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2207 if (COMPOSING_P (coding
))
2208 DECODE_COMPOSITION_END ('1');
2211 if (! NILP (translation_table
))
2212 c
= translate_char (translation_table
, c
, 0, 0, 0);
2217 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2218 coding
->produced
= dst
- destination
;
2223 /* ISO2022 encoding stuff. */
2226 It is not enough to say just "ISO2022" on encoding, we have to
2227 specify more details. In Emacs, each ISO2022 coding system
2228 variant has the following specifications:
2229 1. Initial designation to G0 through G3.
2230 2. Allows short-form designation?
2231 3. ASCII should be designated to G0 before control characters?
2232 4. ASCII should be designated to G0 at end of line?
2233 5. 7-bit environment or 8-bit environment?
2234 6. Use locking-shift?
2235 7. Use Single-shift?
2236 And the following two are only for Japanese:
2237 8. Use ASCII in place of JIS0201-1976-Roman?
2238 9. Use JISX0208-1983 in place of JISX0208-1978?
2239 These specifications are encoded in `coding->flags' as flag bits
2240 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
2244 /* Produce codes (escape sequence) for designating CHARSET to graphic
2245 register REG at DST, and increment DST. If <final-char> of CHARSET is
2246 '@', 'A', or 'B' and the coding system CODING allows, produce
2247 designation sequence of short-form. */
2249 #define ENCODE_DESIGNATION(charset, reg, coding) \
2251 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2252 char *intermediate_char_94 = "()*+"; \
2253 char *intermediate_char_96 = ",-./"; \
2254 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
2256 if (revision < 255) \
2258 *dst++ = ISO_CODE_ESC; \
2260 *dst++ = '@' + revision; \
2262 *dst++ = ISO_CODE_ESC; \
2263 if (CHARSET_DIMENSION (charset) == 1) \
2265 if (CHARSET_CHARS (charset) == 94) \
2266 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2268 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2273 if (CHARSET_CHARS (charset) == 94) \
2275 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2277 || final_char < '@' || final_char > 'B') \
2278 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2281 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2283 *dst++ = final_char; \
2284 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2287 /* The following two macros produce codes (control character or escape
2288 sequence) for ISO2022 single-shift functions (single-shift-2 and
2291 #define ENCODE_SINGLE_SHIFT_2 \
2293 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2294 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2296 *dst++ = ISO_CODE_SS2; \
2297 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2300 #define ENCODE_SINGLE_SHIFT_3 \
2302 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2303 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2305 *dst++ = ISO_CODE_SS3; \
2306 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2309 /* The following four macros produce codes (control character or
2310 escape sequence) for ISO2022 locking-shift functions (shift-in,
2311 shift-out, locking-shift-2, and locking-shift-3). */
2313 #define ENCODE_SHIFT_IN \
2315 *dst++ = ISO_CODE_SI; \
2316 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2319 #define ENCODE_SHIFT_OUT \
2321 *dst++ = ISO_CODE_SO; \
2322 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2325 #define ENCODE_LOCKING_SHIFT_2 \
2327 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2328 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2331 #define ENCODE_LOCKING_SHIFT_3 \
2333 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
2334 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2337 /* Produce codes for a DIMENSION1 character whose character set is
2338 CHARSET and whose position-code is C1. Designation and invocation
2339 sequences are also produced in advance if necessary. */
2341 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2343 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2345 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2346 *dst++ = c1 & 0x7F; \
2348 *dst++ = c1 | 0x80; \
2349 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2352 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2354 *dst++ = c1 & 0x7F; \
2357 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2359 *dst++ = c1 | 0x80; \
2363 /* Since CHARSET is not yet invoked to any graphic planes, we \
2364 must invoke it, or, at first, designate it to some graphic \
2365 register. Then repeat the loop to actually produce the \
2367 dst = encode_invocation_designation (charset, coding, dst); \
2370 /* Produce codes for a DIMENSION2 character whose character set is
2371 CHARSET and whose position-codes are C1 and C2. Designation and
2372 invocation codes are also produced in advance if necessary. */
2374 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2376 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2378 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2379 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2381 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2382 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2385 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2387 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2390 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2392 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2396 /* Since CHARSET is not yet invoked to any graphic planes, we \
2397 must invoke it, or, at first, designate it to some graphic \
2398 register. Then repeat the loop to actually produce the \
2400 dst = encode_invocation_designation (charset, coding, dst); \
2403 #define ENCODE_ISO_CHARACTER(c) \
2405 int charset, c1, c2; \
2407 SPLIT_CHAR (c, charset, c1, c2); \
2408 if (CHARSET_DEFINED_P (charset)) \
2410 if (CHARSET_DIMENSION (charset) == 1) \
2412 if (charset == CHARSET_ASCII \
2413 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2414 charset = charset_latin_jisx0201; \
2415 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2419 if (charset == charset_jisx0208 \
2420 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2421 charset = charset_jisx0208_1978; \
2422 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2434 /* Instead of encoding character C, produce one or two `?'s. */
2436 #define ENCODE_UNSAFE_CHARACTER(c) \
2438 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2439 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2440 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2444 /* Produce designation and invocation codes at a place pointed by DST
2445 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2449 encode_invocation_designation (charset
, coding
, dst
)
2451 struct coding_system
*coding
;
2454 int reg
; /* graphic register number */
2456 /* At first, check designations. */
2457 for (reg
= 0; reg
< 4; reg
++)
2458 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
2463 /* CHARSET is not yet designated to any graphic registers. */
2464 /* At first check the requested designation. */
2465 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2466 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
2467 /* Since CHARSET requests no special designation, designate it
2468 to graphic register 0. */
2471 ENCODE_DESIGNATION (charset
, reg
, coding
);
2474 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
2475 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
2477 /* Since the graphic register REG is not invoked to any graphic
2478 planes, invoke it to graphic plane 0. */
2481 case 0: /* graphic register 0 */
2485 case 1: /* graphic register 1 */
2489 case 2: /* graphic register 2 */
2490 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2491 ENCODE_SINGLE_SHIFT_2
;
2493 ENCODE_LOCKING_SHIFT_2
;
2496 case 3: /* graphic register 3 */
2497 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2498 ENCODE_SINGLE_SHIFT_3
;
2500 ENCODE_LOCKING_SHIFT_3
;
2508 /* Produce 2-byte codes for encoded composition rule RULE. */
2510 #define ENCODE_COMPOSITION_RULE(rule) \
2513 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2514 *dst++ = 32 + 81 + gref; \
2515 *dst++ = 32 + nref; \
2518 /* Produce codes for indicating the start of a composition sequence
2519 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2520 which specify information about the composition. See the comment
2521 in coding.h for the format of DATA. */
2523 #define ENCODE_COMPOSITION_START(coding, data) \
2525 coding->composing = data[3]; \
2526 *dst++ = ISO_CODE_ESC; \
2527 if (coding->composing == COMPOSITION_RELATIVE) \
2531 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2533 coding->cmp_data_index = coding->cmp_data_start + 4; \
2534 coding->composition_rule_follows = 0; \
2538 /* Produce codes for indicating the end of the current composition. */
2540 #define ENCODE_COMPOSITION_END(coding, data) \
2542 *dst++ = ISO_CODE_ESC; \
2544 coding->cmp_data_start += data[0]; \
2545 coding->composing = COMPOSITION_NO; \
2546 if (coding->cmp_data_start == coding->cmp_data->used \
2547 && coding->cmp_data->next) \
2549 coding->cmp_data = coding->cmp_data->next; \
2550 coding->cmp_data_start = 0; \
2554 /* Produce composition start sequence ESC 0. Here, this sequence
2555 doesn't mean the start of a new composition but means that we have
2556 just produced components (alternate chars and composition rules) of
2557 the composition and the actual text follows in SRC. */
2559 #define ENCODE_COMPOSITION_FAKE_START(coding) \
2561 *dst++ = ISO_CODE_ESC; \
2563 coding->composing = COMPOSITION_RELATIVE; \
2566 /* The following three macros produce codes for indicating direction
2568 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2570 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2571 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2573 *dst++ = ISO_CODE_CSI; \
2576 #define ENCODE_DIRECTION_R2L \
2577 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2579 #define ENCODE_DIRECTION_L2R \
2580 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2582 /* Produce codes for designation and invocation to reset the graphic
2583 planes and registers to initial state. */
2584 #define ENCODE_RESET_PLANE_AND_REGISTER \
2587 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2589 for (reg = 0; reg < 4; reg++) \
2590 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2591 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2592 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2593 ENCODE_DESIGNATION \
2594 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
2597 /* Produce designation sequences of charsets in the line started from
2598 SRC to a place pointed by DST, and return updated DST.
2600 If the current block ends before any end-of-line, we may fail to
2601 find all the necessary designations. */
2603 static unsigned char *
2604 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
2605 struct coding_system
*coding
;
2606 Lisp_Object translation_table
;
2607 const unsigned char *src
, *src_end
;
2610 int charset
, c
, found
= 0, reg
;
2611 /* Table of charsets to be designated to each graphic register. */
2614 for (reg
= 0; reg
< 4; reg
++)
2623 charset
= CHAR_CHARSET (c
);
2624 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2625 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
2635 for (reg
= 0; reg
< 4; reg
++)
2637 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
2638 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2644 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2647 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2648 struct coding_system
*coding
;
2649 const unsigned char *source
;
2650 unsigned char *destination
;
2651 int src_bytes
, dst_bytes
;
2653 const unsigned char *src
= source
;
2654 const unsigned char *src_end
= source
+ src_bytes
;
2655 unsigned char *dst
= destination
;
2656 unsigned char *dst_end
= destination
+ dst_bytes
;
2657 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2658 from DST_END to assure overflow checking is necessary only at the
2660 unsigned char *adjusted_dst_end
= dst_end
- 19;
2661 /* SRC_BASE remembers the start position in source in each loop.
2662 The loop will be exited when there's not enough source text to
2663 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2664 there's not enough destination area to produce encoded codes
2665 (within macro EMIT_BYTES). */
2666 const unsigned char *src_base
;
2668 Lisp_Object translation_table
;
2669 Lisp_Object safe_chars
;
2671 if (coding
->flags
& CODING_FLAG_ISO_SAFE
)
2672 coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
2674 safe_chars
= coding_safe_chars (coding
->symbol
);
2676 if (NILP (Venable_character_translation
))
2677 translation_table
= Qnil
;
2680 translation_table
= coding
->translation_table_for_encode
;
2681 if (NILP (translation_table
))
2682 translation_table
= Vstandard_translation_table_for_encode
;
2685 coding
->consumed_char
= 0;
2691 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2693 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2697 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2698 && CODING_SPEC_ISO_BOL (coding
))
2700 /* We have to produce designation sequences if any now. */
2701 dst
= encode_designation_at_bol (coding
, translation_table
,
2703 CODING_SPEC_ISO_BOL (coding
) = 0;
2706 /* Check composition start and end. */
2707 if (coding
->composing
!= COMPOSITION_DISABLED
2708 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2710 struct composition_data
*cmp_data
= coding
->cmp_data
;
2711 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2712 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2714 if (coding
->composing
== COMPOSITION_RELATIVE
)
2716 if (this_pos
== data
[2])
2718 ENCODE_COMPOSITION_END (coding
, data
);
2719 cmp_data
= coding
->cmp_data
;
2720 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2723 else if (COMPOSING_P (coding
))
2725 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2726 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2727 /* We have consumed components of the composition.
2728 What follows in SRC is the composition's base
2730 ENCODE_COMPOSITION_FAKE_START (coding
);
2733 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2734 if (coding
->composition_rule_follows
)
2736 ENCODE_COMPOSITION_RULE (c
);
2737 coding
->composition_rule_follows
= 0;
2741 if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2742 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2743 ENCODE_UNSAFE_CHARACTER (c
);
2745 ENCODE_ISO_CHARACTER (c
);
2746 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2747 coding
->composition_rule_follows
= 1;
2752 if (!COMPOSING_P (coding
))
2754 if (this_pos
== data
[1])
2756 ENCODE_COMPOSITION_START (coding
, data
);
2764 /* Now encode the character C. */
2765 if (c
< 0x20 || c
== 0x7F)
2769 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2771 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2772 ENCODE_RESET_PLANE_AND_REGISTER
;
2776 /* fall down to treat '\r' as '\n' ... */
2781 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2782 ENCODE_RESET_PLANE_AND_REGISTER
;
2783 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2784 bcopy (coding
->spec
.iso2022
.initial_designation
,
2785 coding
->spec
.iso2022
.current_designation
,
2786 sizeof coding
->spec
.iso2022
.initial_designation
);
2787 if (coding
->eol_type
== CODING_EOL_LF
2788 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2789 *dst
++ = ISO_CODE_LF
;
2790 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2791 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2793 *dst
++ = ISO_CODE_CR
;
2794 CODING_SPEC_ISO_BOL (coding
) = 1;
2798 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2799 ENCODE_RESET_PLANE_AND_REGISTER
;
2803 else if (ASCII_BYTE_P (c
))
2804 ENCODE_ISO_CHARACTER (c
);
2805 else if (SINGLE_BYTE_CHAR_P (c
))
2810 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2811 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2812 ENCODE_UNSAFE_CHARACTER (c
);
2814 ENCODE_ISO_CHARACTER (c
);
2816 coding
->consumed_char
++;
2820 coding
->consumed
= src_base
- source
;
2821 coding
->produced
= coding
->produced_char
= dst
- destination
;
2825 /*** 4. SJIS and BIG5 handlers ***/
2827 /* Although SJIS and BIG5 are not ISO coding systems, they are used
2828 quite widely. So, for the moment, Emacs supports them in the bare
2829 C code. But, in the future, they may be supported only by CCL. */
2831 /* SJIS is a coding system encoding three character sets: ASCII, right
2832 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2833 as is. A character of charset katakana-jisx0201 is encoded by
2834 "position-code + 0x80". A character of charset japanese-jisx0208
2835 is encoded in 2-byte but two position-codes are divided and shifted
2836 so that it fits in the range below.
2838 --- CODE RANGE of SJIS ---
2839 (character set) (range)
2841 KATAKANA-JISX0201 0xA1 .. 0xDF
2842 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2843 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2844 -------------------------------
2848 /* BIG5 is a coding system encoding two character sets: ASCII and
2849 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2850 character set and is encoded in two bytes.
2852 --- CODE RANGE of BIG5 ---
2853 (character set) (range)
2855 Big5 (1st byte) 0xA1 .. 0xFE
2856 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2857 --------------------------
2859 Since the number of characters in Big5 is larger than maximum
2860 characters in Emacs' charset (96x96), it can't be handled as one
2861 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2862 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2863 contains frequently used characters and the latter contains less
2864 frequently used characters. */
2866 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2867 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2868 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
2869 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2871 /* Number of Big5 characters which have the same code in 1st byte. */
2872 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2874 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2877 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2879 charset = charset_big5_1; \
2882 charset = charset_big5_2; \
2883 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2885 c1 = temp / (0xFF - 0xA1) + 0x21; \
2886 c2 = temp % (0xFF - 0xA1) + 0x21; \
2889 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2891 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2892 if (charset == charset_big5_2) \
2893 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2894 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2895 b2 = temp % BIG5_SAME_ROW; \
2896 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2899 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2900 Check if a text is encoded in SJIS. If it is, return
2901 CODING_CATEGORY_MASK_SJIS, else return 0. */
2904 detect_coding_sjis (src
, src_end
, multibytep
)
2905 unsigned char *src
, *src_end
;
2909 /* Dummy for ONE_MORE_BYTE. */
2910 struct coding_system dummy_coding
;
2911 struct coding_system
*coding
= &dummy_coding
;
2915 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2918 if (c
== 0x80 || c
== 0xA0 || c
> 0xEF)
2920 if (c
<= 0x9F || c
>= 0xE0)
2922 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2923 if (c
< 0x40 || c
== 0x7F || c
> 0xFC)
2928 return CODING_CATEGORY_MASK_SJIS
;
2931 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2932 Check if a text is encoded in BIG5. If it is, return
2933 CODING_CATEGORY_MASK_BIG5, else return 0. */
2936 detect_coding_big5 (src
, src_end
, multibytep
)
2937 unsigned char *src
, *src_end
;
2941 /* Dummy for ONE_MORE_BYTE. */
2942 struct coding_system dummy_coding
;
2943 struct coding_system
*coding
= &dummy_coding
;
2947 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2950 if (c
< 0xA1 || c
> 0xFE)
2952 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2953 if (c
< 0x40 || (c
> 0x7F && c
< 0xA1) || c
> 0xFE)
2957 return CODING_CATEGORY_MASK_BIG5
;
2960 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2961 Check if a text is encoded in UTF-8. If it is, return
2962 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2964 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2965 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2966 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2967 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2968 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2969 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2970 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2973 detect_coding_utf_8 (src
, src_end
, multibytep
)
2974 unsigned char *src
, *src_end
;
2978 int seq_maybe_bytes
;
2979 /* Dummy for ONE_MORE_BYTE. */
2980 struct coding_system dummy_coding
;
2981 struct coding_system
*coding
= &dummy_coding
;
2985 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2986 if (UTF_8_1_OCTET_P (c
))
2988 else if (UTF_8_2_OCTET_LEADING_P (c
))
2989 seq_maybe_bytes
= 1;
2990 else if (UTF_8_3_OCTET_LEADING_P (c
))
2991 seq_maybe_bytes
= 2;
2992 else if (UTF_8_4_OCTET_LEADING_P (c
))
2993 seq_maybe_bytes
= 3;
2994 else if (UTF_8_5_OCTET_LEADING_P (c
))
2995 seq_maybe_bytes
= 4;
2996 else if (UTF_8_6_OCTET_LEADING_P (c
))
2997 seq_maybe_bytes
= 5;
3003 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3004 if (!UTF_8_EXTRA_OCTET_P (c
))
3008 while (seq_maybe_bytes
> 0);
3012 return CODING_CATEGORY_MASK_UTF_8
;
3015 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3016 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
3017 Little Endian (otherwise). If it is, return
3018 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
3021 #define UTF_16_INVALID_P(val) \
3022 (((val) == 0xFFFE) \
3023 || ((val) == 0xFFFF))
3025 #define UTF_16_HIGH_SURROGATE_P(val) \
3026 (((val) & 0xD800) == 0xD800)
3028 #define UTF_16_LOW_SURROGATE_P(val) \
3029 (((val) & 0xDC00) == 0xDC00)
3032 detect_coding_utf_16 (src
, src_end
, multibytep
)
3033 unsigned char *src
, *src_end
;
3036 unsigned char c1
, c2
;
3037 /* Dummy for ONE_MORE_BYTE_CHECK_MULTIBYTE. */
3038 struct coding_system dummy_coding
;
3039 struct coding_system
*coding
= &dummy_coding
;
3041 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
3042 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2
, multibytep
);
3044 if ((c1
== 0xFF) && (c2
== 0xFE))
3045 return CODING_CATEGORY_MASK_UTF_16_LE
;
3046 else if ((c1
== 0xFE) && (c2
== 0xFF))
3047 return CODING_CATEGORY_MASK_UTF_16_BE
;
3053 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
3054 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
3057 decode_coding_sjis_big5 (coding
, source
, destination
,
3058 src_bytes
, dst_bytes
, sjis_p
)
3059 struct coding_system
*coding
;
3060 const unsigned char *source
;
3061 unsigned char *destination
;
3062 int src_bytes
, dst_bytes
;
3065 const unsigned char *src
= source
;
3066 const unsigned char *src_end
= source
+ src_bytes
;
3067 unsigned char *dst
= destination
;
3068 unsigned char *dst_end
= destination
+ dst_bytes
;
3069 /* SRC_BASE remembers the start position in source in each loop.
3070 The loop will be exited when there's not enough source code
3071 (within macro ONE_MORE_BYTE), or when there's not enough
3072 destination area to produce a character (within macro
3074 const unsigned char *src_base
;
3075 Lisp_Object translation_table
;
3077 if (NILP (Venable_character_translation
))
3078 translation_table
= Qnil
;
3081 translation_table
= coding
->translation_table_for_decode
;
3082 if (NILP (translation_table
))
3083 translation_table
= Vstandard_translation_table_for_decode
;
3086 coding
->produced_char
= 0;
3089 int c
, charset
, c1
, c2
= 0;
3096 charset
= CHARSET_ASCII
;
3101 if (coding
->eol_type
== CODING_EOL_CRLF
)
3107 /* To process C2 again, SRC is subtracted by 1. */
3110 else if (coding
->eol_type
== CODING_EOL_CR
)
3114 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3115 && (coding
->eol_type
== CODING_EOL_CR
3116 || coding
->eol_type
== CODING_EOL_CRLF
))
3118 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3119 goto label_end_of_loop
;
3127 if (c1
== 0x80 || c1
== 0xA0 || c1
> 0xEF)
3128 goto label_invalid_code
;
3129 if (c1
<= 0x9F || c1
>= 0xE0)
3131 /* SJIS -> JISX0208 */
3133 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
3134 goto label_invalid_code
;
3135 DECODE_SJIS (c1
, c2
, c1
, c2
);
3136 charset
= charset_jisx0208
;
3139 /* SJIS -> JISX0201-Kana */
3140 charset
= charset_katakana_jisx0201
;
3145 if (c1
< 0xA0 || c1
> 0xFE)
3146 goto label_invalid_code
;
3148 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
3149 goto label_invalid_code
;
3150 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
3154 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
3166 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3167 coding
->produced
= dst
- destination
;
3171 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
3172 This function can encode charsets `ascii', `katakana-jisx0201',
3173 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
3174 are sure that all these charsets are registered as official charset
3175 (i.e. do not have extended leading-codes). Characters of other
3176 charsets are produced without any encoding. If SJIS_P is 1, encode
3177 SJIS text, else encode BIG5 text. */
3180 encode_coding_sjis_big5 (coding
, source
, destination
,
3181 src_bytes
, dst_bytes
, sjis_p
)
3182 struct coding_system
*coding
;
3183 unsigned char *source
, *destination
;
3184 int src_bytes
, dst_bytes
;
3187 unsigned char *src
= source
;
3188 unsigned char *src_end
= source
+ src_bytes
;
3189 unsigned char *dst
= destination
;
3190 unsigned char *dst_end
= destination
+ dst_bytes
;
3191 /* SRC_BASE remembers the start position in source in each loop.
3192 The loop will be exited when there's not enough source text to
3193 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3194 there's not enough destination area to produce encoded codes
3195 (within macro EMIT_BYTES). */
3196 unsigned char *src_base
;
3197 Lisp_Object translation_table
;
3199 if (NILP (Venable_character_translation
))
3200 translation_table
= Qnil
;
3203 translation_table
= coding
->translation_table_for_encode
;
3204 if (NILP (translation_table
))
3205 translation_table
= Vstandard_translation_table_for_encode
;
3210 int c
, charset
, c1
, c2
;
3215 /* Now encode the character C. */
3216 if (SINGLE_BYTE_CHAR_P (c
))
3221 if (!(coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
3228 if (coding
->eol_type
== CODING_EOL_CRLF
)
3230 EMIT_TWO_BYTES ('\r', c
);
3233 else if (coding
->eol_type
== CODING_EOL_CR
)
3241 SPLIT_CHAR (c
, charset
, c1
, c2
);
3244 if (charset
== charset_jisx0208
3245 || charset
== charset_jisx0208_1978
)
3247 ENCODE_SJIS (c1
, c2
, c1
, c2
);
3248 EMIT_TWO_BYTES (c1
, c2
);
3250 else if (charset
== charset_katakana_jisx0201
)
3251 EMIT_ONE_BYTE (c1
| 0x80);
3252 else if (charset
== charset_latin_jisx0201
)
3254 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3256 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3257 if (CHARSET_WIDTH (charset
) > 1)
3258 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3261 /* There's no way other than producing the internal
3263 EMIT_BYTES (src_base
, src
);
3267 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
3269 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
3270 EMIT_TWO_BYTES (c1
, c2
);
3272 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3274 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3275 if (CHARSET_WIDTH (charset
) > 1)
3276 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3279 /* There's no way other than producing the internal
3281 EMIT_BYTES (src_base
, src
);
3284 coding
->consumed_char
++;
3288 coding
->consumed
= src_base
- source
;
3289 coding
->produced
= coding
->produced_char
= dst
- destination
;
3293 /*** 5. CCL handlers ***/
3295 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3296 Check if a text is encoded in a coding system of which
3297 encoder/decoder are written in CCL program. If it is, return
3298 CODING_CATEGORY_MASK_CCL, else return 0. */
3301 detect_coding_ccl (src
, src_end
, multibytep
)
3302 unsigned char *src
, *src_end
;
3305 unsigned char *valid
;
3307 /* Dummy for ONE_MORE_BYTE. */
3308 struct coding_system dummy_coding
;
3309 struct coding_system
*coding
= &dummy_coding
;
3311 /* No coding system is assigned to coding-category-ccl. */
3312 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
3315 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
3318 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3323 return CODING_CATEGORY_MASK_CCL
;
3327 /*** 6. End-of-line handlers ***/
3329 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3332 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3333 struct coding_system
*coding
;
3334 const unsigned char *source
;
3335 unsigned char *destination
;
3336 int src_bytes
, dst_bytes
;
3338 const unsigned char *src
= source
;
3339 unsigned char *dst
= destination
;
3340 const unsigned char *src_end
= src
+ src_bytes
;
3341 unsigned char *dst_end
= dst
+ dst_bytes
;
3342 Lisp_Object translation_table
;
3343 /* SRC_BASE remembers the start position in source in each loop.
3344 The loop will be exited when there's not enough source code
3345 (within macro ONE_MORE_BYTE), or when there's not enough
3346 destination area to produce a character (within macro
3348 const unsigned char *src_base
;
3351 translation_table
= Qnil
;
3352 switch (coding
->eol_type
)
3354 case CODING_EOL_CRLF
:
3369 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
3371 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3372 goto label_end_of_loop
;
3385 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3387 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3388 goto label_end_of_loop
;
3397 default: /* no need for EOL handling */
3407 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3408 coding
->produced
= dst
- destination
;
3412 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
3413 format of end-of-line according to `coding->eol_type'. It also
3414 convert multibyte form 8-bit characters to unibyte if
3415 CODING->src_multibyte is nonzero. If `coding->mode &
3416 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3417 also means end-of-line. */
3420 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3421 struct coding_system
*coding
;
3422 const unsigned char *source
;
3423 unsigned char *destination
;
3424 int src_bytes
, dst_bytes
;
3426 const unsigned char *src
= source
;
3427 unsigned char *dst
= destination
;
3428 const unsigned char *src_end
= src
+ src_bytes
;
3429 unsigned char *dst_end
= dst
+ dst_bytes
;
3430 Lisp_Object translation_table
;
3431 /* SRC_BASE remembers the start position in source in each loop.
3432 The loop will be exited when there's not enough source text to
3433 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3434 there's not enough destination area to produce encoded codes
3435 (within macro EMIT_BYTES). */
3436 const unsigned char *src_base
;
3439 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
3441 translation_table
= Qnil
;
3442 if (coding
->src_multibyte
3443 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3447 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
3450 if (coding
->eol_type
== CODING_EOL_CRLF
)
3452 while (src
< src_end
)
3458 else if (c
== '\n' || (c
== '\r' && selective_display
))
3459 EMIT_TWO_BYTES ('\r', '\n');
3469 if (!dst_bytes
|| src_bytes
<= dst_bytes
)
3471 safe_bcopy (src
, dst
, src_bytes
);
3477 if (coding
->src_multibyte
3478 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3480 safe_bcopy (src
, dst
, dst_bytes
);
3481 src_base
= src
+ dst_bytes
;
3482 dst
= destination
+ dst_bytes
;
3483 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
3485 if (coding
->eol_type
== CODING_EOL_CR
)
3487 for (tmp
= destination
; tmp
< dst
; tmp
++)
3488 if (*tmp
== '\n') *tmp
= '\r';
3490 else if (selective_display
)
3492 for (tmp
= destination
; tmp
< dst
; tmp
++)
3493 if (*tmp
== '\r') *tmp
= '\n';
3496 if (coding
->src_multibyte
)
3497 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
3499 coding
->consumed
= src_base
- source
;
3500 coding
->produced
= dst
- destination
;
3501 coding
->produced_char
= coding
->produced
;
3505 /*** 7. C library functions ***/
3507 /* In Emacs Lisp, a coding system is represented by a Lisp symbol which
3508 has a property `coding-system'. The value of this property is a
3509 vector of length 5 (called the coding-vector). Among elements of
3510 this vector, the first (element[0]) and the fifth (element[4])
3511 carry important information for decoding/encoding. Before
3512 decoding/encoding, this information should be set in fields of a
3513 structure of type `coding_system'.
3515 The value of the property `coding-system' can be a symbol of another
3516 subsidiary coding-system. In that case, Emacs gets coding-vector
3519 `element[0]' contains information to be set in `coding->type'. The
3520 value and its meaning is as follows:
3522 0 -- coding_type_emacs_mule
3523 1 -- coding_type_sjis
3524 2 -- coding_type_iso2022
3525 3 -- coding_type_big5
3526 4 -- coding_type_ccl encoder/decoder written in CCL
3527 nil -- coding_type_no_conversion
3528 t -- coding_type_undecided (automatic conversion on decoding,
3529 no-conversion on encoding)
3531 `element[4]' contains information to be set in `coding->flags' and
3532 `coding->spec'. The meaning varies by `coding->type'.
3534 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3535 of length 32 (of which the first 13 sub-elements are used now).
3536 Meanings of these sub-elements are:
3538 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3539 If the value is an integer of valid charset, the charset is
3540 assumed to be designated to graphic register N initially.
3542 If the value is minus, it is a minus value of charset which
3543 reserves graphic register N, which means that the charset is
3544 not designated initially but should be designated to graphic
3545 register N just before encoding a character in that charset.
3547 If the value is nil, graphic register N is never used on
3550 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3551 Each value takes t or nil. See the section ISO2022 of
3552 `coding.h' for more information.
3554 If `coding->type' is `coding_type_big5', element[4] is t to denote
3555 BIG5-ETen or nil to denote BIG5-HKU.
3557 If `coding->type' takes the other value, element[4] is ignored.
3559 Emacs Lisp's coding systems also carry information about format of
3560 end-of-line in a value of property `eol-type'. If the value is
3561 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3562 means CODING_EOL_CR. If it is not integer, it should be a vector
3563 of subsidiary coding systems of which property `eol-type' has one
3564 of the above values.
3568 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3569 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3570 is setup so that no conversion is necessary and return -1, else
3574 setup_coding_system (coding_system
, coding
)
3575 Lisp_Object coding_system
;
3576 struct coding_system
*coding
;
3578 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
3581 /* At first, zero clear all members. */
3582 bzero (coding
, sizeof (struct coding_system
));
3584 /* Initialize some fields required for all kinds of coding systems. */
3585 coding
->symbol
= coding_system
;
3586 coding
->heading_ascii
= -1;
3587 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
3588 coding
->composing
= COMPOSITION_DISABLED
;
3589 coding
->cmp_data
= NULL
;
3591 if (NILP (coding_system
))
3592 goto label_invalid_coding_system
;
3594 coding_spec
= Fget (coding_system
, Qcoding_system
);
3596 if (!VECTORP (coding_spec
)
3597 || XVECTOR (coding_spec
)->size
!= 5
3598 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
3599 goto label_invalid_coding_system
;
3601 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
3602 if (VECTORP (eol_type
))
3604 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3605 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
3607 else if (XFASTINT (eol_type
) == 1)
3609 coding
->eol_type
= CODING_EOL_CRLF
;
3610 coding
->common_flags
3611 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3613 else if (XFASTINT (eol_type
) == 2)
3615 coding
->eol_type
= CODING_EOL_CR
;
3616 coding
->common_flags
3617 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3620 coding
->eol_type
= CODING_EOL_LF
;
3622 coding_type
= XVECTOR (coding_spec
)->contents
[0];
3623 /* Try short cut. */
3624 if (SYMBOLP (coding_type
))
3626 if (EQ (coding_type
, Qt
))
3628 coding
->type
= coding_type_undecided
;
3629 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
3632 coding
->type
= coding_type_no_conversion
;
3633 /* Initialize this member. Any thing other than
3634 CODING_CATEGORY_IDX_UTF_16_BE and
3635 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3636 special treatment in detect_eol. */
3637 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
3642 /* Get values of coding system properties:
3643 `post-read-conversion', `pre-write-conversion',
3644 `translation-table-for-decode', `translation-table-for-encode'. */
3645 plist
= XVECTOR (coding_spec
)->contents
[3];
3646 /* Pre & post conversion functions should be disabled if
3647 inhibit_eol_conversion is nonzero. This is the case that a code
3648 conversion function is called while those functions are running. */
3649 if (! inhibit_pre_post_conversion
)
3651 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
3652 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
3654 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
3656 val
= Fget (val
, Qtranslation_table_for_decode
);
3657 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3658 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
3660 val
= Fget (val
, Qtranslation_table_for_encode
);
3661 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3662 val
= Fplist_get (plist
, Qcoding_category
);
3665 val
= Fget (val
, Qcoding_category_index
);
3667 coding
->category_idx
= XINT (val
);
3669 goto label_invalid_coding_system
;
3672 goto label_invalid_coding_system
;
3674 /* If the coding system has non-nil `composition' property, enable
3675 composition handling. */
3676 val
= Fplist_get (plist
, Qcomposition
);
3678 coding
->composing
= COMPOSITION_NO
;
3680 switch (XFASTINT (coding_type
))
3683 coding
->type
= coding_type_emacs_mule
;
3684 coding
->common_flags
3685 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3686 if (!NILP (coding
->post_read_conversion
))
3687 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3688 if (!NILP (coding
->pre_write_conversion
))
3689 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3693 coding
->type
= coding_type_sjis
;
3694 coding
->common_flags
3695 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3699 coding
->type
= coding_type_iso2022
;
3700 coding
->common_flags
3701 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3703 Lisp_Object val
, temp
;
3705 int i
, charset
, reg_bits
= 0;
3707 val
= XVECTOR (coding_spec
)->contents
[4];
3709 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3710 goto label_invalid_coding_system
;
3712 flags
= XVECTOR (val
)->contents
;
3714 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3715 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3716 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3717 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3718 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3719 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3720 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3721 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3722 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3723 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3724 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3725 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3726 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3729 /* Invoke graphic register 0 to plane 0. */
3730 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3731 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3732 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3733 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3734 /* Not single shifting at first. */
3735 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3736 /* Beginning of buffer should also be regarded as bol. */
3737 CODING_SPEC_ISO_BOL (coding
) = 1;
3739 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3740 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3741 val
= Vcharset_revision_alist
;
3744 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3746 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3747 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3748 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3752 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3753 FLAGS[REG] can be one of below:
3754 integer CHARSET: CHARSET occupies register I,
3755 t: designate nothing to REG initially, but can be used
3757 list of integer, nil, or t: designate the first
3758 element (if integer) to REG initially, the remaining
3759 elements (if integer) is designated to REG on request,
3760 if an element is t, REG can be used by any charsets,
3761 nil: REG is never used. */
3762 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3763 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3764 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3765 for (i
= 0; i
< 4; i
++)
3767 if ((INTEGERP (flags
[i
])
3768 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
)))
3769 || (charset
= get_charset_id (flags
[i
])) >= 0)
3771 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3772 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3774 else if (EQ (flags
[i
], Qt
))
3776 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3778 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3780 else if (CONSP (flags
[i
]))
3785 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3786 if ((INTEGERP (XCAR (tail
))
3787 && (charset
= XINT (XCAR (tail
)),
3788 CHARSET_VALID_P (charset
)))
3789 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3791 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3792 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3795 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3797 while (CONSP (tail
))
3799 if ((INTEGERP (XCAR (tail
))
3800 && (charset
= XINT (XCAR (tail
)),
3801 CHARSET_VALID_P (charset
)))
3802 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3803 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3805 else if (EQ (XCAR (tail
), Qt
))
3811 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3813 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3814 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3817 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3819 /* REG 1 can be used only by locking shift in 7-bit env. */
3820 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3822 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3823 /* Without any shifting, only REG 0 and 1 can be used. */
3828 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3830 if (CHARSET_DEFINED_P (charset
)
3831 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3832 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3834 /* There exist some default graphic registers to be
3837 /* We had better avoid designating a charset of
3838 CHARS96 to REG 0 as far as possible. */
3839 if (CHARSET_CHARS (charset
) == 96)
3840 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3842 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3844 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3846 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3850 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3851 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3855 coding
->type
= coding_type_big5
;
3856 coding
->common_flags
3857 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3859 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3860 ? CODING_FLAG_BIG5_HKU
3861 : CODING_FLAG_BIG5_ETEN
);
3865 coding
->type
= coding_type_ccl
;
3866 coding
->common_flags
3867 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3869 val
= XVECTOR (coding_spec
)->contents
[4];
3871 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3873 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3875 goto label_invalid_coding_system
;
3877 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3878 val
= Fplist_get (plist
, Qvalid_codes
);
3883 for (; CONSP (val
); val
= XCDR (val
))
3887 && XINT (this) >= 0 && XINT (this) < 256)
3888 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3889 else if (CONSP (this)
3890 && INTEGERP (XCAR (this))
3891 && INTEGERP (XCDR (this)))
3893 int start
= XINT (XCAR (this));
3894 int end
= XINT (XCDR (this));
3896 if (start
>= 0 && start
<= end
&& end
< 256)
3897 while (start
<= end
)
3898 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3903 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3904 coding
->spec
.ccl
.cr_carryover
= 0;
3905 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
3909 coding
->type
= coding_type_raw_text
;
3913 goto label_invalid_coding_system
;
3917 label_invalid_coding_system
:
3918 coding
->type
= coding_type_no_conversion
;
3919 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3920 coding
->common_flags
= 0;
3921 coding
->eol_type
= CODING_EOL_LF
;
3922 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3926 /* Free memory blocks allocated for storing composition information. */
3929 coding_free_composition_data (coding
)
3930 struct coding_system
*coding
;
3932 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3936 /* Memory blocks are chained. At first, rewind to the first, then,
3937 free blocks one by one. */
3938 while (cmp_data
->prev
)
3939 cmp_data
= cmp_data
->prev
;
3942 next
= cmp_data
->next
;
3946 coding
->cmp_data
= NULL
;
3949 /* Set `char_offset' member of all memory blocks pointed by
3950 coding->cmp_data to POS. */
3953 coding_adjust_composition_offset (coding
, pos
)
3954 struct coding_system
*coding
;
3957 struct composition_data
*cmp_data
;
3959 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3960 cmp_data
->char_offset
= pos
;
3963 /* Setup raw-text or one of its subsidiaries in the structure
3964 coding_system CODING according to the already setup value eol_type
3965 in CODING. CODING should be setup for some coding system in
3969 setup_raw_text_coding_system (coding
)
3970 struct coding_system
*coding
;
3972 if (coding
->type
!= coding_type_raw_text
)
3974 coding
->symbol
= Qraw_text
;
3975 coding
->type
= coding_type_raw_text
;
3976 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3978 Lisp_Object subsidiaries
;
3979 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3981 if (VECTORP (subsidiaries
)
3982 && XVECTOR (subsidiaries
)->size
== 3)
3984 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3986 setup_coding_system (coding
->symbol
, coding
);
3991 /* Emacs has a mechanism to automatically detect a coding system if it
3992 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3993 it's impossible to distinguish some coding systems accurately
3994 because they use the same range of codes. So, at first, coding
3995 systems are categorized into 7, those are:
3997 o coding-category-emacs-mule
3999 The category for a coding system which has the same code range
4000 as Emacs' internal format. Assigned the coding-system (Lisp
4001 symbol) `emacs-mule' by default.
4003 o coding-category-sjis
4005 The category for a coding system which has the same code range
4006 as SJIS. Assigned the coding-system (Lisp
4007 symbol) `japanese-shift-jis' by default.
4009 o coding-category-iso-7
4011 The category for a coding system which has the same code range
4012 as ISO2022 of 7-bit environment. This doesn't use any locking
4013 shift and single shift functions. This can encode/decode all
4014 charsets. Assigned the coding-system (Lisp symbol)
4015 `iso-2022-7bit' by default.
4017 o coding-category-iso-7-tight
4019 Same as coding-category-iso-7 except that this can
4020 encode/decode only the specified charsets.
4022 o coding-category-iso-8-1
4024 The category for a coding system which has the same code range
4025 as ISO2022 of 8-bit environment and graphic plane 1 used only
4026 for DIMENSION1 charset. This doesn't use any locking shift
4027 and single shift functions. Assigned the coding-system (Lisp
4028 symbol) `iso-latin-1' by default.
4030 o coding-category-iso-8-2
4032 The category for a coding system which has the same code range
4033 as ISO2022 of 8-bit environment and graphic plane 1 used only
4034 for DIMENSION2 charset. This doesn't use any locking shift
4035 and single shift functions. Assigned the coding-system (Lisp
4036 symbol) `japanese-iso-8bit' by default.
4038 o coding-category-iso-7-else
4040 The category for a coding system which has the same code range
4041 as ISO2022 of 7-bit environment but uses locking shift or
4042 single shift functions. Assigned the coding-system (Lisp
4043 symbol) `iso-2022-7bit-lock' by default.
4045 o coding-category-iso-8-else
4047 The category for a coding system which has the same code range
4048 as ISO2022 of 8-bit environment but uses locking shift or
4049 single shift functions. Assigned the coding-system (Lisp
4050 symbol) `iso-2022-8bit-ss2' by default.
4052 o coding-category-big5
4054 The category for a coding system which has the same code range
4055 as BIG5. Assigned the coding-system (Lisp symbol)
4056 `cn-big5' by default.
4058 o coding-category-utf-8
4060 The category for a coding system which has the same code range
4061 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
4062 symbol) `utf-8' by default.
4064 o coding-category-utf-16-be
4066 The category for a coding system in which a text has an
4067 Unicode signature (cf. Unicode Standard) in the order of BIG
4068 endian at the head. Assigned the coding-system (Lisp symbol)
4069 `utf-16-be' by default.
4071 o coding-category-utf-16-le
4073 The category for a coding system in which a text has an
4074 Unicode signature (cf. Unicode Standard) in the order of
4075 LITTLE endian at the head. Assigned the coding-system (Lisp
4076 symbol) `utf-16-le' by default.
4078 o coding-category-ccl
4080 The category for a coding system of which encoder/decoder is
4081 written in CCL programs. The default value is nil, i.e., no
4082 coding system is assigned.
4084 o coding-category-binary
4086 The category for a coding system not categorized in any of the
4087 above. Assigned the coding-system (Lisp symbol)
4088 `no-conversion' by default.
4090 Each of them is a Lisp symbol and the value is an actual
4091 `coding-system' (this is also a Lisp symbol) assigned by a user.
4092 What Emacs does actually is to detect a category of coding system.
4093 Then, it uses a `coding-system' assigned to it. If Emacs can't
4094 decide a single possible category, it selects a category of the
4095 highest priority. Priorities of categories are also specified by a
4096 user in a Lisp variable `coding-category-list'.
4101 int ascii_skip_code
[256];
4103 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
4104 If it detects possible coding systems, return an integer in which
4105 appropriate flag bits are set. Flag bits are defined by macros
4106 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
4107 it should point the table `coding_priorities'. In that case, only
4108 the flag bit for a coding system of the highest priority is set in
4109 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
4110 range 0x80..0x9F are in multibyte form.
4112 How many ASCII characters are at the head is returned as *SKIP. */
4115 detect_coding_mask (source
, src_bytes
, priorities
, skip
, multibytep
)
4116 unsigned char *source
;
4117 int src_bytes
, *priorities
, *skip
;
4120 register unsigned char c
;
4121 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
4122 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
4125 /* At first, skip all ASCII characters and control characters except
4126 for three ISO2022 specific control characters. */
4127 ascii_skip_code
[ISO_CODE_SO
] = 0;
4128 ascii_skip_code
[ISO_CODE_SI
] = 0;
4129 ascii_skip_code
[ISO_CODE_ESC
] = 0;
4131 label_loop_detect_coding
:
4132 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
4133 *skip
= src
- source
;
4136 /* We found nothing other than ASCII. There's nothing to do. */
4140 /* The text seems to be encoded in some multilingual coding system.
4141 Now, try to find in which coding system the text is encoded. */
4144 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
4145 /* C is an ISO2022 specific control code of C0. */
4146 mask
= detect_coding_iso2022 (src
, src_end
, multibytep
);
4149 /* No valid ISO2022 code follows C. Try again. */
4151 if (c
== ISO_CODE_ESC
)
4152 ascii_skip_code
[ISO_CODE_ESC
] = 1;
4154 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
4155 goto label_loop_detect_coding
;
4159 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4161 if (mask
& priorities
[i
])
4162 return priorities
[i
];
4164 return CODING_CATEGORY_MASK_RAW_TEXT
;
4171 if (multibytep
&& c
== LEADING_CODE_8_BIT_CONTROL
)
4176 /* C is the first byte of SJIS character code,
4177 or a leading-code of Emacs' internal format (emacs-mule),
4178 or the first byte of UTF-16. */
4179 try = (CODING_CATEGORY_MASK_SJIS
4180 | CODING_CATEGORY_MASK_EMACS_MULE
4181 | CODING_CATEGORY_MASK_UTF_16_BE
4182 | CODING_CATEGORY_MASK_UTF_16_LE
);
4184 /* Or, if C is a special latin extra code,
4185 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4186 or is an ISO2022 control-sequence-introducer (CSI),
4187 we should also consider the possibility of ISO2022 codings. */
4188 if ((VECTORP (Vlatin_extra_code_table
)
4189 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
4190 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
4191 || (c
== ISO_CODE_CSI
4194 || ((*src
== '0' || *src
== '1' || *src
== '2')
4195 && src
+ 1 < src_end
4196 && src
[1] == ']')))))
4197 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4198 | CODING_CATEGORY_MASK_ISO_8BIT
);
4201 /* C is a character of ISO2022 in graphic plane right,
4202 or a SJIS's 1-byte character code (i.e. JISX0201),
4203 or the first byte of BIG5's 2-byte code,
4204 or the first byte of UTF-8/16. */
4205 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4206 | CODING_CATEGORY_MASK_ISO_8BIT
4207 | CODING_CATEGORY_MASK_SJIS
4208 | CODING_CATEGORY_MASK_BIG5
4209 | CODING_CATEGORY_MASK_UTF_8
4210 | CODING_CATEGORY_MASK_UTF_16_BE
4211 | CODING_CATEGORY_MASK_UTF_16_LE
);
4213 /* Or, we may have to consider the possibility of CCL. */
4214 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4215 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4216 ->spec
.ccl
.valid_codes
)[c
])
4217 try |= CODING_CATEGORY_MASK_CCL
;
4220 utf16_examined_p
= iso2022_examined_p
= 0;
4223 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4225 if (!iso2022_examined_p
4226 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
4228 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4229 iso2022_examined_p
= 1;
4231 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
4232 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4233 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
4234 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4235 else if (!utf16_examined_p
4236 && (priorities
[i
] & try &
4237 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
4239 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4240 utf16_examined_p
= 1;
4242 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
4243 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4244 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
4245 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4246 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
4247 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4248 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
4249 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
4250 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
4251 mask
|= CODING_CATEGORY_MASK_BINARY
;
4252 if (mask
& priorities
[i
])
4253 return priorities
[i
];
4255 return CODING_CATEGORY_MASK_RAW_TEXT
;
4257 if (try & CODING_CATEGORY_MASK_ISO
)
4258 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4259 if (try & CODING_CATEGORY_MASK_SJIS
)
4260 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4261 if (try & CODING_CATEGORY_MASK_BIG5
)
4262 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4263 if (try & CODING_CATEGORY_MASK_UTF_8
)
4264 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4265 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
4266 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4267 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
4268 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4269 if (try & CODING_CATEGORY_MASK_CCL
)
4270 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4272 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
4275 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4276 The information of the detected coding system is set in CODING. */
4279 detect_coding (coding
, src
, src_bytes
)
4280 struct coding_system
*coding
;
4281 const unsigned char *src
;
4288 val
= Vcoding_category_list
;
4289 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
,
4290 coding
->src_multibyte
);
4291 coding
->heading_ascii
= skip
;
4295 /* We found a single coding system of the highest priority in MASK. */
4297 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
4299 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
4301 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[idx
]);
4303 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4307 tmp
= Fget (val
, Qeol_type
);
4309 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
4312 /* Setup this new coding system while preserving some slots. */
4314 int src_multibyte
= coding
->src_multibyte
;
4315 int dst_multibyte
= coding
->dst_multibyte
;
4317 setup_coding_system (val
, coding
);
4318 coding
->src_multibyte
= src_multibyte
;
4319 coding
->dst_multibyte
= dst_multibyte
;
4320 coding
->heading_ascii
= skip
;
4324 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
4325 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4326 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4328 How many non-eol characters are at the head is returned as *SKIP. */
4330 #define MAX_EOL_CHECK_COUNT 3
4333 detect_eol_type (source
, src_bytes
, skip
)
4334 unsigned char *source
;
4335 int src_bytes
, *skip
;
4337 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4339 int total
= 0; /* How many end-of-lines are found so far. */
4340 int eol_type
= CODING_EOL_UNDECIDED
;
4345 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4348 if (c
== '\n' || c
== '\r')
4351 *skip
= src
- 1 - source
;
4354 this_eol_type
= CODING_EOL_LF
;
4355 else if (src
>= src_end
|| *src
!= '\n')
4356 this_eol_type
= CODING_EOL_CR
;
4358 this_eol_type
= CODING_EOL_CRLF
, src
++;
4360 if (eol_type
== CODING_EOL_UNDECIDED
)
4361 /* This is the first end-of-line. */
4362 eol_type
= this_eol_type
;
4363 else if (eol_type
!= this_eol_type
)
4365 /* The found type is different from what found before. */
4366 eol_type
= CODING_EOL_INCONSISTENT
;
4373 *skip
= src_end
- source
;
4377 /* Like detect_eol_type, but detect EOL type in 2-octet
4378 big-endian/little-endian format for coding systems utf-16-be and
4382 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
4383 unsigned char *source
;
4384 int src_bytes
, *skip
, big_endian_p
;
4386 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4387 unsigned int c1
, c2
;
4388 int total
= 0; /* How many end-of-lines are found so far. */
4389 int eol_type
= CODING_EOL_UNDECIDED
;
4400 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4402 c1
= (src
[msb
] << 8) | (src
[lsb
]);
4405 if (c1
== '\n' || c1
== '\r')
4408 *skip
= src
- 2 - source
;
4412 this_eol_type
= CODING_EOL_LF
;
4416 if ((src
+ 1) >= src_end
)
4418 this_eol_type
= CODING_EOL_CR
;
4422 c2
= (src
[msb
] << 8) | (src
[lsb
]);
4424 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
4426 this_eol_type
= CODING_EOL_CR
;
4430 if (eol_type
== CODING_EOL_UNDECIDED
)
4431 /* This is the first end-of-line. */
4432 eol_type
= this_eol_type
;
4433 else if (eol_type
!= this_eol_type
)
4435 /* The found type is different from what found before. */
4436 eol_type
= CODING_EOL_INCONSISTENT
;
4443 *skip
= src_end
- source
;
4447 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4448 is encoded. If it detects an appropriate format of end-of-line, it
4449 sets the information in *CODING. */
4452 detect_eol (coding
, src
, src_bytes
)
4453 struct coding_system
*coding
;
4454 const unsigned char *src
;
4461 switch (coding
->category_idx
)
4463 case CODING_CATEGORY_IDX_UTF_16_BE
:
4464 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
4466 case CODING_CATEGORY_IDX_UTF_16_LE
:
4467 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
4470 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
4474 if (coding
->heading_ascii
> skip
)
4475 coding
->heading_ascii
= skip
;
4477 skip
= coding
->heading_ascii
;
4479 if (eol_type
== CODING_EOL_UNDECIDED
)
4481 if (eol_type
== CODING_EOL_INCONSISTENT
)
4484 /* This code is suppressed until we find a better way to
4485 distinguish raw text file and binary file. */
4487 /* If we have already detected that the coding is raw-text, the
4488 coding should actually be no-conversion. */
4489 if (coding
->type
== coding_type_raw_text
)
4491 setup_coding_system (Qno_conversion
, coding
);
4494 /* Else, let's decode only text code anyway. */
4496 eol_type
= CODING_EOL_LF
;
4499 val
= Fget (coding
->symbol
, Qeol_type
);
4500 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4502 int src_multibyte
= coding
->src_multibyte
;
4503 int dst_multibyte
= coding
->dst_multibyte
;
4504 struct composition_data
*cmp_data
= coding
->cmp_data
;
4506 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
4507 coding
->src_multibyte
= src_multibyte
;
4508 coding
->dst_multibyte
= dst_multibyte
;
4509 coding
->heading_ascii
= skip
;
4510 coding
->cmp_data
= cmp_data
;
4514 #define CONVERSION_BUFFER_EXTRA_ROOM 256
4516 #define DECODING_BUFFER_MAG(coding) \
4517 (coding->type == coding_type_iso2022 \
4519 : (coding->type == coding_type_ccl \
4520 ? coding->spec.ccl.decoder.buf_magnification \
4523 /* Return maximum size (bytes) of a buffer enough for decoding
4524 SRC_BYTES of text encoded in CODING. */
4527 decoding_buffer_size (coding
, src_bytes
)
4528 struct coding_system
*coding
;
4531 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
4532 + CONVERSION_BUFFER_EXTRA_ROOM
);
4535 /* Return maximum size (bytes) of a buffer enough for encoding
4536 SRC_BYTES of text to CODING. */
4539 encoding_buffer_size (coding
, src_bytes
)
4540 struct coding_system
*coding
;
4545 if (coding
->type
== coding_type_ccl
)
4547 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
4548 if (coding
->eol_type
== CODING_EOL_CRLF
)
4551 else if (CODING_REQUIRE_ENCODING (coding
))
4556 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
4559 /* Working buffer for code conversion. */
4560 struct conversion_buffer
4562 int size
; /* size of data. */
4563 int on_stack
; /* 1 if allocated by alloca. */
4564 unsigned char *data
;
4567 /* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4568 #define allocate_conversion_buffer(buf, len) \
4570 if (len < MAX_ALLOCA) \
4572 buf.data = (unsigned char *) alloca (len); \
4577 buf.data = (unsigned char *) xmalloc (len); \
4583 /* Double the allocated memory for *BUF. */
4585 extend_conversion_buffer (buf
)
4586 struct conversion_buffer
*buf
;
4590 unsigned char *save
= buf
->data
;
4591 buf
->data
= (unsigned char *) xmalloc (buf
->size
* 2);
4592 bcopy (save
, buf
->data
, buf
->size
);
4597 buf
->data
= (unsigned char *) xrealloc (buf
->data
, buf
->size
* 2);
4602 /* Free the allocated memory for BUF if it is not on stack. */
4604 free_conversion_buffer (buf
)
4605 struct conversion_buffer
*buf
;
4612 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
4613 struct coding_system
*coding
;
4614 unsigned char *source
, *destination
;
4615 int src_bytes
, dst_bytes
, encodep
;
4617 struct ccl_program
*ccl
4618 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
4619 unsigned char *dst
= destination
;
4621 ccl
->suppress_error
= coding
->suppress_error
;
4622 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
4625 /* On encoding, EOL format is converted within ccl_driver. For
4626 that, setup proper information in the structure CCL. */
4627 ccl
->eol_type
= coding
->eol_type
;
4628 if (ccl
->eol_type
==CODING_EOL_UNDECIDED
)
4629 ccl
->eol_type
= CODING_EOL_LF
;
4630 ccl
->cr_consumed
= coding
->spec
.ccl
.cr_carryover
;
4631 ccl
->eight_bit_control
= coding
->dst_multibyte
;
4634 ccl
->eight_bit_control
= 1;
4635 ccl
->multibyte
= coding
->src_multibyte
;
4636 if (coding
->spec
.ccl
.eight_bit_carryover
[0] != 0)
4638 /* Move carryover bytes to DESTINATION. */
4639 unsigned char *p
= coding
->spec
.ccl
.eight_bit_carryover
;
4642 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4644 dst_bytes
-= dst
- destination
;
4647 coding
->produced
= (ccl_driver (ccl
, source
, dst
, src_bytes
, dst_bytes
,
4648 &(coding
->consumed
))
4649 + dst
- destination
);
4653 coding
->produced_char
= coding
->produced
;
4654 coding
->spec
.ccl
.cr_carryover
= ccl
->cr_consumed
;
4656 else if (!ccl
->eight_bit_control
)
4658 /* The produced bytes forms a valid multibyte sequence. */
4659 coding
->produced_char
4660 = multibyte_chars_in_text (destination
, coding
->produced
);
4661 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4665 /* On decoding, the destination should always multibyte. But,
4666 CCL program might have been generated an invalid multibyte
4667 sequence. Here we make such a sequence valid as
4670 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
4672 if ((coding
->consumed
< src_bytes
4673 || !ccl
->last_block
)
4674 && coding
->produced
>= 1
4675 && destination
[coding
->produced
- 1] >= 0x80)
4677 /* We should not convert the tailing 8-bit codes to
4678 multibyte form even if they doesn't form a valid
4679 multibyte sequence. They may form a valid sequence in
4683 if (destination
[coding
->produced
- 1] < 0xA0)
4685 else if (coding
->produced
>= 2)
4687 if (destination
[coding
->produced
- 2] >= 0x80)
4689 if (destination
[coding
->produced
- 2] < 0xA0)
4691 else if (coding
->produced
>= 3
4692 && destination
[coding
->produced
- 3] >= 0x80
4693 && destination
[coding
->produced
- 3] < 0xA0)
4699 BCOPY_SHORT (destination
+ coding
->produced
- carryover
,
4700 coding
->spec
.ccl
.eight_bit_carryover
,
4702 coding
->spec
.ccl
.eight_bit_carryover
[carryover
] = 0;
4703 coding
->produced
-= carryover
;
4706 coding
->produced
= str_as_multibyte (destination
, bytes
,
4708 &(coding
->produced_char
));
4711 switch (ccl
->status
)
4713 case CCL_STAT_SUSPEND_BY_SRC
:
4714 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
4716 case CCL_STAT_SUSPEND_BY_DST
:
4717 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
4720 case CCL_STAT_INVALID_CMD
:
4721 coding
->result
= CODING_FINISH_INTERRUPT
;
4724 coding
->result
= CODING_FINISH_NORMAL
;
4727 return coding
->result
;
4730 /* Decode EOL format of the text at PTR of BYTES length destructively
4731 according to CODING->eol_type. This is called after the CCL
4732 program produced a decoded text at PTR. If we do CRLF->LF
4733 conversion, update CODING->produced and CODING->produced_char. */
4736 decode_eol_post_ccl (coding
, ptr
, bytes
)
4737 struct coding_system
*coding
;
4741 Lisp_Object val
, saved_coding_symbol
;
4742 unsigned char *pend
= ptr
+ bytes
;
4745 /* Remember the current coding system symbol. We set it back when
4746 an inconsistent EOL is found so that `last-coding-system-used' is
4747 set to the coding system that doesn't specify EOL conversion. */
4748 saved_coding_symbol
= coding
->symbol
;
4750 coding
->spec
.ccl
.cr_carryover
= 0;
4751 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4753 /* Here, to avoid the call of setup_coding_system, we directly
4754 call detect_eol_type. */
4755 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
4756 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
4757 coding
->eol_type
= CODING_EOL_LF
;
4758 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4760 val
= Fget (coding
->symbol
, Qeol_type
);
4761 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4762 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
4764 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4767 if (coding
->eol_type
== CODING_EOL_LF
4768 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
4770 /* We have nothing to do. */
4773 else if (coding
->eol_type
== CODING_EOL_CRLF
)
4775 unsigned char *pstart
= ptr
, *p
= ptr
;
4777 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4778 && *(pend
- 1) == '\r')
4780 /* If the last character is CR, we can't handle it here
4781 because LF will be in the not-yet-decoded source text.
4782 Record that the CR is not yet processed. */
4783 coding
->spec
.ccl
.cr_carryover
= 1;
4785 coding
->produced_char
--;
4792 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4799 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4800 goto undo_eol_conversion
;
4804 else if (*ptr
== '\n'
4805 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4806 goto undo_eol_conversion
;
4811 undo_eol_conversion
:
4812 /* We have faced with inconsistent EOL format at PTR.
4813 Convert all LFs before PTR back to CRLFs. */
4814 for (p
--, ptr
--; p
>= pstart
; p
--)
4817 *ptr
-- = '\n', *ptr
-- = '\r';
4821 /* If carryover is recorded, cancel it because we don't
4822 convert CRLF anymore. */
4823 if (coding
->spec
.ccl
.cr_carryover
)
4825 coding
->spec
.ccl
.cr_carryover
= 0;
4827 coding
->produced_char
++;
4831 coding
->eol_type
= CODING_EOL_LF
;
4832 coding
->symbol
= saved_coding_symbol
;
4836 /* As each two-byte sequence CRLF was converted to LF, (PEND
4837 - P) is the number of deleted characters. */
4838 coding
->produced
-= pend
- p
;
4839 coding
->produced_char
-= pend
- p
;
4842 else /* i.e. coding->eol_type == CODING_EOL_CR */
4844 unsigned char *p
= ptr
;
4846 for (; ptr
< pend
; ptr
++)
4850 else if (*ptr
== '\n'
4851 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4853 for (; p
< ptr
; p
++)
4859 coding
->eol_type
= CODING_EOL_LF
;
4860 coding
->symbol
= saved_coding_symbol
;
4866 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4867 decoding, it may detect coding system and format of end-of-line if
4868 those are not yet decided. The source should be unibyte, the
4869 result is multibyte if CODING->dst_multibyte is nonzero, else
4873 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4874 struct coding_system
*coding
;
4875 const unsigned char *source
;
4876 unsigned char *destination
;
4877 int src_bytes
, dst_bytes
;
4881 if (coding
->type
== coding_type_undecided
)
4882 detect_coding (coding
, source
, src_bytes
);
4884 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4885 && coding
->type
!= coding_type_ccl
)
4887 detect_eol (coding
, source
, src_bytes
);
4888 /* We had better recover the original eol format if we
4889 encounter an inconsistent eol format while decoding. */
4890 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4893 coding
->produced
= coding
->produced_char
= 0;
4894 coding
->consumed
= coding
->consumed_char
= 0;
4896 coding
->result
= CODING_FINISH_NORMAL
;
4898 switch (coding
->type
)
4900 case coding_type_sjis
:
4901 decode_coding_sjis_big5 (coding
, source
, destination
,
4902 src_bytes
, dst_bytes
, 1);
4905 case coding_type_iso2022
:
4906 decode_coding_iso2022 (coding
, source
, destination
,
4907 src_bytes
, dst_bytes
);
4910 case coding_type_big5
:
4911 decode_coding_sjis_big5 (coding
, source
, destination
,
4912 src_bytes
, dst_bytes
, 0);
4915 case coding_type_emacs_mule
:
4916 decode_coding_emacs_mule (coding
, source
, destination
,
4917 src_bytes
, dst_bytes
);
4920 case coding_type_ccl
:
4921 if (coding
->spec
.ccl
.cr_carryover
)
4923 /* Put the CR which was not processed by the previous call
4924 of decode_eol_post_ccl in DESTINATION. It will be
4925 decoded together with the following LF by the call to
4926 decode_eol_post_ccl below. */
4927 *destination
= '\r';
4929 coding
->produced_char
++;
4931 extra
= coding
->spec
.ccl
.cr_carryover
;
4933 ccl_coding_driver (coding
, source
, destination
+ extra
,
4934 src_bytes
, dst_bytes
, 0);
4935 if (coding
->eol_type
!= CODING_EOL_LF
)
4937 coding
->produced
+= extra
;
4938 coding
->produced_char
+= extra
;
4939 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4944 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4947 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4948 && coding
->mode
& CODING_MODE_LAST_BLOCK
4949 && coding
->consumed
== src_bytes
)
4950 coding
->result
= CODING_FINISH_NORMAL
;
4952 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4953 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4955 const unsigned char *src
= source
+ coding
->consumed
;
4956 unsigned char *dst
= destination
+ coding
->produced
;
4958 src_bytes
-= coding
->consumed
;
4960 if (COMPOSING_P (coding
))
4961 DECODE_COMPOSITION_END ('1');
4965 dst
+= CHAR_STRING (c
, dst
);
4966 coding
->produced_char
++;
4968 coding
->consumed
= coding
->consumed_char
= src
- source
;
4969 coding
->produced
= dst
- destination
;
4970 coding
->result
= CODING_FINISH_NORMAL
;
4973 if (!coding
->dst_multibyte
)
4975 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4976 coding
->produced_char
= coding
->produced
;
4979 return coding
->result
;
4982 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4983 multibyteness of the source is CODING->src_multibyte, the
4984 multibyteness of the result is always unibyte. */
4987 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4988 struct coding_system
*coding
;
4989 const unsigned char *source
;
4990 unsigned char *destination
;
4991 int src_bytes
, dst_bytes
;
4993 coding
->produced
= coding
->produced_char
= 0;
4994 coding
->consumed
= coding
->consumed_char
= 0;
4996 coding
->result
= CODING_FINISH_NORMAL
;
4998 switch (coding
->type
)
5000 case coding_type_sjis
:
5001 encode_coding_sjis_big5 (coding
, source
, destination
,
5002 src_bytes
, dst_bytes
, 1);
5005 case coding_type_iso2022
:
5006 encode_coding_iso2022 (coding
, source
, destination
,
5007 src_bytes
, dst_bytes
);
5010 case coding_type_big5
:
5011 encode_coding_sjis_big5 (coding
, source
, destination
,
5012 src_bytes
, dst_bytes
, 0);
5015 case coding_type_emacs_mule
:
5016 encode_coding_emacs_mule (coding
, source
, destination
,
5017 src_bytes
, dst_bytes
);
5020 case coding_type_ccl
:
5021 ccl_coding_driver (coding
, source
, destination
,
5022 src_bytes
, dst_bytes
, 1);
5026 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
5029 if (coding
->mode
& CODING_MODE_LAST_BLOCK
5030 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
5032 const unsigned char *src
= source
+ coding
->consumed
;
5033 unsigned char *dst
= destination
+ coding
->produced
;
5035 if (coding
->type
== coding_type_iso2022
)
5036 ENCODE_RESET_PLANE_AND_REGISTER
;
5037 if (COMPOSING_P (coding
))
5038 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
5039 if (coding
->consumed
< src_bytes
)
5041 int len
= src_bytes
- coding
->consumed
;
5043 BCOPY_SHORT (src
, dst
, len
);
5044 if (coding
->src_multibyte
)
5045 len
= str_as_unibyte (dst
, len
);
5047 coding
->consumed
= src_bytes
;
5049 coding
->produced
= coding
->produced_char
= dst
- destination
;
5050 coding
->result
= CODING_FINISH_NORMAL
;
5053 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
5054 && coding
->consumed
== src_bytes
)
5055 coding
->result
= CODING_FINISH_NORMAL
;
5057 return coding
->result
;
5060 /* Scan text in the region between *BEG and *END (byte positions),
5061 skip characters which we don't have to decode by coding system
5062 CODING at the head and tail, then set *BEG and *END to the region
5063 of the text we actually have to convert. The caller should move
5064 the gap out of the region in advance if the region is from a
5067 If STR is not NULL, *BEG and *END are indices into STR. */
5070 shrink_decoding_region (beg
, end
, coding
, str
)
5072 struct coding_system
*coding
;
5075 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
5077 Lisp_Object translation_table
;
5079 if (coding
->type
== coding_type_ccl
5080 || coding
->type
== coding_type_undecided
5081 || coding
->eol_type
!= CODING_EOL_LF
5082 || !NILP (coding
->post_read_conversion
)
5083 || coding
->composing
!= COMPOSITION_DISABLED
)
5085 /* We can't skip any data. */
5088 if (coding
->type
== coding_type_no_conversion
5089 || coding
->type
== coding_type_raw_text
5090 || coding
->type
== coding_type_emacs_mule
)
5092 /* We need no conversion, but don't have to skip any data here.
5093 Decoding routine handles them effectively anyway. */
5097 translation_table
= coding
->translation_table_for_decode
;
5098 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5099 translation_table
= Vstandard_translation_table_for_decode
;
5100 if (CHAR_TABLE_P (translation_table
))
5103 for (i
= 0; i
< 128; i
++)
5104 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5107 /* Some ASCII character should be translated. We give up
5112 if (coding
->heading_ascii
>= 0)
5113 /* Detection routine has already found how much we can skip at the
5115 *beg
+= coding
->heading_ascii
;
5119 begp_orig
= begp
= str
+ *beg
;
5120 endp_orig
= endp
= str
+ *end
;
5124 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5125 endp_orig
= endp
= begp
+ *end
- *beg
;
5128 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5129 || coding
->eol_type
== CODING_EOL_CRLF
);
5131 switch (coding
->type
)
5133 case coding_type_sjis
:
5134 case coding_type_big5
:
5135 /* We can skip all ASCII characters at the head. */
5136 if (coding
->heading_ascii
< 0)
5139 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
5141 while (begp
< endp
&& *begp
< 0x80) begp
++;
5143 /* We can skip all ASCII characters at the tail except for the
5144 second byte of SJIS or BIG5 code. */
5146 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
5148 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5149 /* Do not consider LF as ascii if preceded by CR, since that
5150 confuses eol decoding. */
5151 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5153 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
5157 case coding_type_iso2022
:
5158 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5159 /* We can't skip any data. */
5161 if (coding
->heading_ascii
< 0)
5163 /* We can skip all ASCII characters at the head except for a
5164 few control codes. */
5165 while (begp
< endp
&& (c
= *begp
) < 0x80
5166 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
5167 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
5168 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
5171 switch (coding
->category_idx
)
5173 case CODING_CATEGORY_IDX_ISO_8_1
:
5174 case CODING_CATEGORY_IDX_ISO_8_2
:
5175 /* We can skip all ASCII characters at the tail. */
5177 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
5179 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5180 /* Do not consider LF as ascii if preceded by CR, since that
5181 confuses eol decoding. */
5182 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5186 case CODING_CATEGORY_IDX_ISO_7
:
5187 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
5189 /* We can skip all characters at the tail except for 8-bit
5190 codes and ESC and the following 2-byte at the tail. */
5191 unsigned char *eight_bit
= NULL
;
5195 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
5197 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5202 && (c
= endp
[-1]) != ISO_CODE_ESC
)
5204 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5207 /* Do not consider LF as ascii if preceded by CR, since that
5208 confuses eol decoding. */
5209 if (begp
< endp
&& endp
< endp_orig
5210 && endp
[-1] == '\r' && endp
[0] == '\n')
5212 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
5214 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
5215 /* This is an ASCII designation sequence. We can
5216 surely skip the tail. But, if we have
5217 encountered an 8-bit code, skip only the codes
5219 endp
= eight_bit
? eight_bit
: endp
+ 2;
5221 /* Hmmm, we can't skip the tail. */
5233 *beg
+= begp
- begp_orig
;
5234 *end
+= endp
- endp_orig
;
5238 /* Like shrink_decoding_region but for encoding. */
5241 shrink_encoding_region (beg
, end
, coding
, str
)
5243 struct coding_system
*coding
;
5246 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
5248 Lisp_Object translation_table
;
5250 if (coding
->type
== coding_type_ccl
5251 || coding
->eol_type
== CODING_EOL_CRLF
5252 || coding
->eol_type
== CODING_EOL_CR
5253 || (coding
->cmp_data
&& coding
->cmp_data
->used
> 0))
5255 /* We can't skip any data. */
5258 if (coding
->type
== coding_type_no_conversion
5259 || coding
->type
== coding_type_raw_text
5260 || coding
->type
== coding_type_emacs_mule
5261 || coding
->type
== coding_type_undecided
)
5263 /* We need no conversion, but don't have to skip any data here.
5264 Encoding routine handles them effectively anyway. */
5268 translation_table
= coding
->translation_table_for_encode
;
5269 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5270 translation_table
= Vstandard_translation_table_for_encode
;
5271 if (CHAR_TABLE_P (translation_table
))
5274 for (i
= 0; i
< 128; i
++)
5275 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5278 /* Some ASCII character should be translated. We give up
5285 begp_orig
= begp
= str
+ *beg
;
5286 endp_orig
= endp
= str
+ *end
;
5290 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5291 endp_orig
= endp
= begp
+ *end
- *beg
;
5294 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5295 || coding
->eol_type
== CODING_EOL_CRLF
);
5297 /* Here, we don't have to check coding->pre_write_conversion because
5298 the caller is expected to have handled it already. */
5299 switch (coding
->type
)
5301 case coding_type_iso2022
:
5302 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5303 /* We can't skip any data. */
5305 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
5307 unsigned char *bol
= begp
;
5308 while (begp
< endp
&& *begp
< 0x80)
5311 if (begp
[-1] == '\n')
5315 goto label_skip_tail
;
5319 case coding_type_sjis
:
5320 case coding_type_big5
:
5321 /* We can skip all ASCII characters at the head and tail. */
5323 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
5325 while (begp
< endp
&& *begp
< 0x80) begp
++;
5328 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
5330 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
5337 *beg
+= begp
- begp_orig
;
5338 *end
+= endp
- endp_orig
;
5342 /* As shrinking conversion region requires some overhead, we don't try
5343 shrinking if the length of conversion region is less than this
5345 static int shrink_conversion_region_threshhold
= 1024;
5347 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5349 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5351 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5352 else shrink_decoding_region (beg, end, coding, str); \
5356 /* ARG is (CODING BUFFER ...) where CODING is what to be set in
5357 Vlast_coding_system_used and the remaining elements are buffers to
5360 code_convert_region_unwind (arg
)
5363 struct gcpro gcpro1
;
5366 inhibit_pre_post_conversion
= 0;
5367 Vlast_coding_system_used
= XCAR (arg
);
5368 for (arg
= XCDR (arg
); ! NILP (arg
); arg
= XCDR (arg
))
5369 Fkill_buffer (XCAR (arg
));
5375 /* Store information about all compositions in the range FROM and TO
5376 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5377 buffer or a string, defaults to the current buffer. */
5380 coding_save_composition (coding
, from
, to
, obj
)
5381 struct coding_system
*coding
;
5388 if (coding
->composing
== COMPOSITION_DISABLED
)
5390 if (!coding
->cmp_data
)
5391 coding_allocate_composition_data (coding
, from
);
5392 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
5396 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
5399 coding
->composing
= COMPOSITION_NO
;
5402 if (COMPOSITION_VALID_P (start
, end
, prop
))
5404 enum composition_method method
= COMPOSITION_METHOD (prop
);
5405 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
5406 >= COMPOSITION_DATA_SIZE
)
5407 coding_allocate_composition_data (coding
, from
);
5408 /* For relative composition, we remember start and end
5409 positions, for the other compositions, we also remember
5411 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
5412 if (method
!= COMPOSITION_RELATIVE
)
5414 /* We must store a*/
5415 Lisp_Object val
, ch
;
5417 val
= COMPOSITION_COMPONENTS (prop
);
5421 ch
= XCAR (val
), val
= XCDR (val
);
5422 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5424 else if (VECTORP (val
) || STRINGP (val
))
5426 int len
= (VECTORP (val
)
5427 ? XVECTOR (val
)->size
: SCHARS (val
));
5429 for (i
= 0; i
< len
; i
++)
5432 ? Faref (val
, make_number (i
))
5433 : XVECTOR (val
)->contents
[i
]);
5434 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5437 else /* INTEGERP (val) */
5438 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
5440 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
5445 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
5448 /* Make coding->cmp_data point to the first memory block. */
5449 while (coding
->cmp_data
->prev
)
5450 coding
->cmp_data
= coding
->cmp_data
->prev
;
5451 coding
->cmp_data_start
= 0;
5454 /* Reflect the saved information about compositions to OBJ.
5455 CODING->cmp_data points to a memory block for the information. OBJ
5456 is a buffer or a string, defaults to the current buffer. */
5459 coding_restore_composition (coding
, obj
)
5460 struct coding_system
*coding
;
5463 struct composition_data
*cmp_data
= coding
->cmp_data
;
5468 while (cmp_data
->prev
)
5469 cmp_data
= cmp_data
->prev
;
5475 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
5476 i
+= cmp_data
->data
[i
])
5478 int *data
= cmp_data
->data
+ i
;
5479 enum composition_method method
= (enum composition_method
) data
[3];
5480 Lisp_Object components
;
5482 if (data
[0] < 0 || i
+ data
[0] > cmp_data
->used
)
5483 /* Invalid composition data. */
5486 if (method
== COMPOSITION_RELATIVE
)
5490 int len
= data
[0] - 4, j
;
5491 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
5493 if (method
== COMPOSITION_WITH_RULE_ALTCHARS
5497 /* Invalid composition data. */
5499 for (j
= 0; j
< len
; j
++)
5500 args
[j
] = make_number (data
[4 + j
]);
5501 components
= (method
== COMPOSITION_WITH_ALTCHARS
5502 ? Fstring (len
, args
)
5503 : Fvector (len
, args
));
5505 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
5507 cmp_data
= cmp_data
->next
;
5511 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
5512 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5513 coding system CODING, and return the status code of code conversion
5514 (currently, this value has no meaning).
5516 How many characters (and bytes) are converted to how many
5517 characters (and bytes) are recorded in members of the structure
5520 If REPLACE is nonzero, we do various things as if the original text
5521 is deleted and a new text is inserted. See the comments in
5522 replace_range (insdel.c) to know what we are doing.
5524 If REPLACE is zero, it is assumed that the source text is unibyte.
5525 Otherwise, it is assumed that the source text is multibyte. */
5528 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
5529 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
5530 struct coding_system
*coding
;
5532 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
5533 int nchars_del
= 0, nbytes_del
= 0;
5534 int require
, inserted
, inserted_byte
;
5535 int head_skip
, tail_skip
, total_skip
= 0;
5536 Lisp_Object saved_coding_symbol
;
5538 unsigned char *src
, *dst
;
5539 Lisp_Object deletion
;
5540 int orig_point
= PT
, orig_len
= len
;
5542 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
5545 saved_coding_symbol
= coding
->symbol
;
5547 if (from
< PT
&& PT
< to
)
5549 TEMP_SET_PT_BOTH (from
, from_byte
);
5555 int saved_from
= from
;
5556 int saved_inhibit_modification_hooks
;
5558 prepare_to_modify_buffer (from
, to
, &from
);
5559 if (saved_from
!= from
)
5562 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
5563 len_byte
= to_byte
- from_byte
;
5566 /* The code conversion routine can not preserve text properties
5567 for now. So, we must remove all text properties in the
5568 region. Here, we must suppress all modification hooks. */
5569 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
5570 inhibit_modification_hooks
= 1;
5571 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
5572 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
5575 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
5577 /* We must detect encoding of text and eol format. */
5579 if (from
< GPT
&& to
> GPT
)
5580 move_gap_both (from
, from_byte
);
5581 if (coding
->type
== coding_type_undecided
)
5583 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5584 if (coding
->type
== coding_type_undecided
)
5586 /* It seems that the text contains only ASCII, but we
5587 should not leave it undecided because the deeper
5588 decoding routine (decode_coding) tries to detect the
5589 encodings again in vain. */
5590 coding
->type
= coding_type_emacs_mule
;
5591 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5592 /* As emacs-mule decoder will handle composition, we
5593 need this setting to allocate coding->cmp_data
5595 coding
->composing
= COMPOSITION_NO
;
5598 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5599 && coding
->type
!= coding_type_ccl
)
5601 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5602 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5603 coding
->eol_type
= CODING_EOL_LF
;
5604 /* We had better recover the original eol format if we
5605 encounter an inconsistent eol format while decoding. */
5606 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5610 /* Now we convert the text. */
5612 /* For encoding, we must process pre-write-conversion in advance. */
5613 if (! inhibit_pre_post_conversion
5615 && SYMBOLP (coding
->pre_write_conversion
)
5616 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
5618 /* The function in pre-write-conversion may put a new text in a
5620 struct buffer
*prev
= current_buffer
;
5623 record_unwind_protect (code_convert_region_unwind
,
5624 Fcons (Vlast_coding_system_used
, Qnil
));
5625 /* We should not call any more pre-write/post-read-conversion
5626 functions while this pre-write-conversion is running. */
5627 inhibit_pre_post_conversion
= 1;
5628 call2 (coding
->pre_write_conversion
,
5629 make_number (from
), make_number (to
));
5630 inhibit_pre_post_conversion
= 0;
5631 /* Discard the unwind protect. */
5634 if (current_buffer
!= prev
)
5637 new = Fcurrent_buffer ();
5638 set_buffer_internal_1 (prev
);
5639 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
5640 TEMP_SET_PT_BOTH (from
, from_byte
);
5641 insert_from_buffer (XBUFFER (new), 1, len
, 0);
5643 if (orig_point
>= to
)
5644 orig_point
+= len
- orig_len
;
5645 else if (orig_point
> from
)
5649 from_byte
= CHAR_TO_BYTE (from
);
5650 to_byte
= CHAR_TO_BYTE (to
);
5651 len_byte
= to_byte
- from_byte
;
5652 TEMP_SET_PT_BOTH (from
, from_byte
);
5658 if (! EQ (current_buffer
->undo_list
, Qt
))
5659 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
5662 nchars_del
= to
- from
;
5663 nbytes_del
= to_byte
- from_byte
;
5667 if (coding
->composing
!= COMPOSITION_DISABLED
)
5670 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
5672 coding_allocate_composition_data (coding
, from
);
5675 /* Try to skip the heading and tailing ASCIIs. We can't skip them
5676 if we must run CCL program or there are compositions to
5678 if (coding
->type
!= coding_type_ccl
5679 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
5681 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
5683 if (from
< GPT
&& GPT
< to
)
5684 move_gap_both (from
, from_byte
);
5685 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
5686 if (from_byte
== to_byte
5687 && (encodep
|| NILP (coding
->post_read_conversion
))
5688 && ! CODING_REQUIRE_FLUSHING (coding
))
5690 coding
->produced
= len_byte
;
5691 coding
->produced_char
= len
;
5693 /* We must record and adjust for this new text now. */
5694 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
5695 coding_free_composition_data (coding
);
5699 head_skip
= from_byte
- from_byte_orig
;
5700 tail_skip
= to_byte_orig
- to_byte
;
5701 total_skip
= head_skip
+ tail_skip
;
5704 len
-= total_skip
; len_byte
-= total_skip
;
5707 /* For conversion, we must put the gap before the text in addition to
5708 making the gap larger for efficient decoding. The required gap
5709 size starts from 2000 which is the magic number used in make_gap.
5710 But, after one batch of conversion, it will be incremented if we
5711 find that it is not enough . */
5714 if (GAP_SIZE
< require
)
5715 make_gap (require
- GAP_SIZE
);
5716 move_gap_both (from
, from_byte
);
5718 inserted
= inserted_byte
= 0;
5720 GAP_SIZE
+= len_byte
;
5723 ZV_BYTE
-= len_byte
;
5726 if (GPT
- BEG
< BEG_UNCHANGED
)
5727 BEG_UNCHANGED
= GPT
- BEG
;
5728 if (Z
- GPT
< END_UNCHANGED
)
5729 END_UNCHANGED
= Z
- GPT
;
5731 if (!encodep
&& coding
->src_multibyte
)
5733 /* Decoding routines expects that the source text is unibyte.
5734 We must convert 8-bit characters of multibyte form to
5736 int len_byte_orig
= len_byte
;
5737 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
5738 if (len_byte
< len_byte_orig
)
5739 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
5741 coding
->src_multibyte
= 0;
5748 /* The buffer memory is now:
5749 +--------+converted-text+---------+-------original-text-------+---+
5750 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5751 |<---------------------- GAP ----------------------->| */
5752 src
= GAP_END_ADDR
- len_byte
;
5753 dst
= GPT_ADDR
+ inserted_byte
;
5756 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
5759 if (coding
->composing
!= COMPOSITION_DISABLED
)
5760 coding
->cmp_data
->char_offset
= from
+ inserted
;
5761 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
5764 /* The buffer memory is now:
5765 +--------+-------converted-text----+--+------original-text----+---+
5766 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5767 |<---------------------- GAP ----------------------->| */
5769 inserted
+= coding
->produced_char
;
5770 inserted_byte
+= coding
->produced
;
5771 len_byte
-= coding
->consumed
;
5773 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5775 coding_allocate_composition_data (coding
, from
+ inserted
);
5779 src
+= coding
->consumed
;
5780 dst
+= coding
->produced
;
5782 if (result
== CODING_FINISH_NORMAL
)
5787 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
5789 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
5790 Lisp_Object eol_type
;
5792 /* Encode LFs back to the original eol format (CR or CRLF). */
5793 if (coding
->eol_type
== CODING_EOL_CR
)
5795 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
5801 while (p
< pend
) if (*p
++ == '\n') count
++;
5802 if (src
- dst
< count
)
5804 /* We don't have sufficient room for encoding LFs
5805 back to CRLF. We must record converted and
5806 not-yet-converted text back to the buffer
5807 content, enlarge the gap, then record them out of
5808 the buffer contents again. */
5809 int add
= len_byte
+ inserted_byte
;
5812 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5813 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5814 make_gap (count
- GAP_SIZE
);
5816 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5817 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5818 /* Don't forget to update SRC, DST, and PEND. */
5819 src
= GAP_END_ADDR
- len_byte
;
5820 dst
= GPT_ADDR
+ inserted_byte
;
5824 inserted_byte
+= count
;
5825 coding
->produced
+= count
;
5826 p
= dst
= pend
+ count
;
5830 if (*p
== '\n') count
--, *--p
= '\r';
5834 /* Suppress eol-format conversion in the further conversion. */
5835 coding
->eol_type
= CODING_EOL_LF
;
5837 /* Set the coding system symbol to that for Unix-like EOL. */
5838 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5839 if (VECTORP (eol_type
)
5840 && XVECTOR (eol_type
)->size
== 3
5841 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
5842 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5844 coding
->symbol
= saved_coding_symbol
;
5850 if (coding
->type
!= coding_type_ccl
5851 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5853 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5856 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5858 /* The source text ends in invalid codes. Let's just
5859 make them valid buffer contents, and finish conversion. */
5862 unsigned char *start
= dst
;
5864 inserted
+= len_byte
;
5868 dst
+= CHAR_STRING (c
, dst
);
5871 inserted_byte
+= dst
- start
;
5875 inserted
+= len_byte
;
5876 inserted_byte
+= len_byte
;
5882 if (result
== CODING_FINISH_INTERRUPT
)
5884 /* The conversion procedure was interrupted by a user. */
5887 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5888 if (coding
->consumed
< 1)
5890 /* It's quite strange to require more memory without
5891 consuming any bytes. Perhaps CCL program bug. */
5896 /* We have just done the first batch of conversion which was
5897 stopped because of insufficient gap. Let's reconsider the
5898 required gap size (i.e. SRT - DST) now.
5900 We have converted ORIG bytes (== coding->consumed) into
5901 NEW bytes (coding->produced). To convert the remaining
5902 LEN bytes, we may need REQUIRE bytes of gap, where:
5903 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5904 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5905 Here, we are sure that NEW >= ORIG. */
5907 if (coding
->produced
<= coding
->consumed
)
5909 /* This happens because of CCL-based coding system with
5915 float ratio
= coding
->produced
- coding
->consumed
;
5916 ratio
/= coding
->consumed
;
5917 require
= len_byte
* ratio
;
5921 if ((src
- dst
) < (require
+ 2000))
5923 /* See the comment above the previous call of make_gap. */
5924 int add
= len_byte
+ inserted_byte
;
5927 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5928 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5929 make_gap (require
+ 2000);
5931 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5932 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5935 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5937 if (encodep
&& coding
->dst_multibyte
)
5939 /* The output is unibyte. We must convert 8-bit characters to
5941 if (inserted_byte
* 2 > GAP_SIZE
)
5943 GAP_SIZE
-= inserted_byte
;
5944 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5945 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5946 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5947 make_gap (inserted_byte
- GAP_SIZE
);
5948 GAP_SIZE
+= inserted_byte
;
5949 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5950 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5951 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5953 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5956 /* If we shrank the conversion area, adjust it now. */
5960 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5961 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5962 GAP_SIZE
+= total_skip
;
5963 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5964 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5965 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5966 from
-= head_skip
; from_byte
-= head_skip
;
5967 to
+= tail_skip
; to_byte
+= tail_skip
;
5971 if (! EQ (current_buffer
->undo_list
, Qt
))
5972 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5974 adjust_after_replace_noundo (from
, from_byte
, nchars_del
, nbytes_del
,
5975 inserted
, inserted_byte
);
5976 inserted
= Z
- prev_Z
;
5978 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5979 coding_restore_composition (coding
, Fcurrent_buffer ());
5980 coding_free_composition_data (coding
);
5982 if (! inhibit_pre_post_conversion
5983 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5986 Lisp_Object saved_coding_system
;
5989 TEMP_SET_PT_BOTH (from
, from_byte
);
5991 record_unwind_protect (code_convert_region_unwind
,
5992 Fcons (Vlast_coding_system_used
, Qnil
));
5993 saved_coding_system
= Vlast_coding_system_used
;
5994 Vlast_coding_system_used
= coding
->symbol
;
5995 /* We should not call any more pre-write/post-read-conversion
5996 functions while this post-read-conversion is running. */
5997 inhibit_pre_post_conversion
= 1;
5998 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
5999 inhibit_pre_post_conversion
= 0;
6000 coding
->symbol
= Vlast_coding_system_used
;
6001 Vlast_coding_system_used
= saved_coding_system
;
6002 /* Discard the unwind protect. */
6005 inserted
+= Z
- prev_Z
;
6008 if (orig_point
>= from
)
6010 if (orig_point
>= from
+ orig_len
)
6011 orig_point
+= inserted
- orig_len
;
6014 TEMP_SET_PT (orig_point
);
6019 signal_after_change (from
, to
- from
, inserted
);
6020 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
6024 coding
->consumed
= to_byte
- from_byte
;
6025 coding
->consumed_char
= to
- from
;
6026 coding
->produced
= inserted_byte
;
6027 coding
->produced_char
= inserted
;
6033 /* Name (or base name) of work buffer for code conversion. */
6034 static Lisp_Object Vcode_conversion_workbuf_name
;
6036 /* Set the current buffer to the working buffer prepared for
6037 code-conversion. MULTIBYTE specifies the multibyteness of the
6038 buffer. Return the buffer we set if it must be killed after use.
6039 Otherwise return Qnil. */
6042 set_conversion_work_buffer (multibyte
)
6045 Lisp_Object buffer
, buffer_to_kill
;
6048 buffer
= Fget_buffer_create (Vcode_conversion_workbuf_name
);
6049 buf
= XBUFFER (buffer
);
6050 if (buf
== current_buffer
)
6052 /* As we are already in the work buffer, we must generate a new
6053 buffer for the work. */
6056 name
= Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name
, Qnil
);
6057 buffer
= buffer_to_kill
= Fget_buffer_create (name
);
6058 buf
= XBUFFER (buffer
);
6061 buffer_to_kill
= Qnil
;
6063 delete_all_overlays (buf
);
6064 buf
->directory
= current_buffer
->directory
;
6065 buf
->read_only
= Qnil
;
6066 buf
->filename
= Qnil
;
6067 buf
->undo_list
= Qt
;
6068 eassert (buf
->overlays_before
== NULL
);
6069 eassert (buf
->overlays_after
== NULL
);
6070 set_buffer_internal (buf
);
6071 if (BEG
!= BEGV
|| Z
!= ZV
)
6073 del_range_2 (BEG
, BEG_BYTE
, Z
, Z_BYTE
, 0);
6074 buf
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
6075 return buffer_to_kill
;
6079 run_pre_post_conversion_on_str (str
, coding
, encodep
)
6081 struct coding_system
*coding
;
6084 int count
= SPECPDL_INDEX ();
6085 struct gcpro gcpro1
, gcpro2
;
6086 int multibyte
= STRING_MULTIBYTE (str
);
6087 Lisp_Object old_deactivate_mark
;
6088 Lisp_Object buffer_to_kill
;
6089 Lisp_Object unwind_arg
;
6091 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
6092 /* It is not crucial to specbind this. */
6093 old_deactivate_mark
= Vdeactivate_mark
;
6094 GCPRO2 (str
, old_deactivate_mark
);
6096 /* We must insert the contents of STR as is without
6097 unibyte<->multibyte conversion. For that, we adjust the
6098 multibyteness of the working buffer to that of STR. */
6099 buffer_to_kill
= set_conversion_work_buffer (multibyte
);
6100 if (NILP (buffer_to_kill
))
6101 unwind_arg
= Fcons (Vlast_coding_system_used
, Qnil
);
6103 unwind_arg
= list2 (Vlast_coding_system_used
, buffer_to_kill
);
6104 record_unwind_protect (code_convert_region_unwind
, unwind_arg
);
6106 insert_from_string (str
, 0, 0,
6107 SCHARS (str
), SBYTES (str
), 0);
6109 inhibit_pre_post_conversion
= 1;
6112 struct buffer
*prev
= current_buffer
;
6114 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
6115 if (prev
!= current_buffer
)
6116 /* We must kill the current buffer too. */
6117 Fsetcdr (unwind_arg
, Fcons (Fcurrent_buffer (), XCDR (unwind_arg
)));
6121 Vlast_coding_system_used
= coding
->symbol
;
6122 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
6123 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
6124 coding
->symbol
= Vlast_coding_system_used
;
6126 inhibit_pre_post_conversion
= 0;
6127 Vdeactivate_mark
= old_deactivate_mark
;
6128 str
= make_buffer_string (BEG
, Z
, 1);
6129 return unbind_to (count
, str
);
6133 /* Run pre-write-conversion function of CODING on NCHARS/NBYTES
6134 text in *STR. *SIZE is the allocated bytes for STR. As it
6135 is intended that this function is called from encode_terminal_code,
6136 the pre-write-conversion function is run by safe_call and thus
6137 "Error during redisplay: ..." is logged when an error occurs.
6139 Store the resulting text in *STR and set CODING->produced_char and
6140 CODING->produced to the number of characters and bytes
6141 respectively. If the size of *STR is too small, enlarge it by
6142 xrealloc and update *STR and *SIZE. */
6145 run_pre_write_conversin_on_c_str (str
, size
, nchars
, nbytes
, coding
)
6146 unsigned char **str
;
6147 int *size
, nchars
, nbytes
;
6148 struct coding_system
*coding
;
6150 struct gcpro gcpro1
, gcpro2
;
6151 struct buffer
*cur
= current_buffer
;
6152 struct buffer
*prev
;
6153 Lisp_Object old_deactivate_mark
, old_last_coding_system_used
;
6154 Lisp_Object args
[3];
6155 Lisp_Object buffer_to_kill
;
6157 /* It is not crucial to specbind this. */
6158 old_deactivate_mark
= Vdeactivate_mark
;
6159 old_last_coding_system_used
= Vlast_coding_system_used
;
6160 GCPRO2 (old_deactivate_mark
, old_last_coding_system_used
);
6162 /* We must insert the contents of STR as is without
6163 unibyte<->multibyte conversion. For that, we adjust the
6164 multibyteness of the working buffer to that of STR. */
6165 buffer_to_kill
= set_conversion_work_buffer (coding
->src_multibyte
);
6166 insert_1_both (*str
, nchars
, nbytes
, 0, 0, 0);
6168 inhibit_pre_post_conversion
= 1;
6169 prev
= current_buffer
;
6170 args
[0] = coding
->pre_write_conversion
;
6171 args
[1] = make_number (BEG
);
6172 args
[2] = make_number (Z
);
6173 safe_call (3, args
);
6174 inhibit_pre_post_conversion
= 0;
6175 Vdeactivate_mark
= old_deactivate_mark
;
6176 Vlast_coding_system_used
= old_last_coding_system_used
;
6177 coding
->produced_char
= Z
- BEG
;
6178 coding
->produced
= Z_BYTE
- BEG_BYTE
;
6179 if (coding
->produced
> *size
)
6181 *size
= coding
->produced
;
6182 *str
= xrealloc (*str
, *size
);
6184 if (BEG
< GPT
&& GPT
< Z
)
6186 bcopy (BEG_ADDR
, *str
, coding
->produced
);
6187 coding
->src_multibyte
6188 = ! NILP (current_buffer
->enable_multibyte_characters
);
6189 if (prev
!= current_buffer
)
6190 Fkill_buffer (Fcurrent_buffer ());
6191 set_buffer_internal (cur
);
6192 if (! NILP (buffer_to_kill
))
6193 Fkill_buffer (buffer_to_kill
);
6198 decode_coding_string (str
, coding
, nocopy
)
6200 struct coding_system
*coding
;
6204 struct conversion_buffer buf
;
6206 Lisp_Object saved_coding_symbol
;
6208 int require_decoding
;
6209 int shrinked_bytes
= 0;
6211 int consumed
, consumed_char
, produced
, produced_char
;
6214 to_byte
= SBYTES (str
);
6216 saved_coding_symbol
= coding
->symbol
;
6217 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6218 coding
->dst_multibyte
= 1;
6219 if (CODING_REQUIRE_DETECTION (coding
))
6221 /* See the comments in code_convert_region. */
6222 if (coding
->type
== coding_type_undecided
)
6224 detect_coding (coding
, SDATA (str
), to_byte
);
6225 if (coding
->type
== coding_type_undecided
)
6227 coding
->type
= coding_type_emacs_mule
;
6228 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
6229 /* As emacs-mule decoder will handle composition, we
6230 need this setting to allocate coding->cmp_data
6232 coding
->composing
= COMPOSITION_NO
;
6235 if (coding
->eol_type
== CODING_EOL_UNDECIDED
6236 && coding
->type
!= coding_type_ccl
)
6238 saved_coding_symbol
= coding
->symbol
;
6239 detect_eol (coding
, SDATA (str
), to_byte
);
6240 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
6241 coding
->eol_type
= CODING_EOL_LF
;
6242 /* We had better recover the original eol format if we
6243 encounter an inconsistent eol format while decoding. */
6244 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
6248 if (coding
->type
== coding_type_no_conversion
6249 || coding
->type
== coding_type_raw_text
)
6250 coding
->dst_multibyte
= 0;
6252 require_decoding
= CODING_REQUIRE_DECODING (coding
);
6254 if (STRING_MULTIBYTE (str
))
6256 /* Decoding routines expect the source text to be unibyte. */
6257 str
= Fstring_as_unibyte (str
);
6258 to_byte
= SBYTES (str
);
6260 coding
->src_multibyte
= 0;
6263 /* Try to skip the heading and tailing ASCIIs. */
6264 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
6266 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6268 if (from
== to_byte
)
6269 require_decoding
= 0;
6270 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6273 if (!require_decoding
6274 && !(SYMBOLP (coding
->post_read_conversion
)
6275 && !NILP (Ffboundp (coding
->post_read_conversion
))))
6277 coding
->consumed
= SBYTES (str
);
6278 coding
->consumed_char
= SCHARS (str
);
6279 if (coding
->dst_multibyte
)
6281 str
= Fstring_as_multibyte (str
);
6284 coding
->produced
= SBYTES (str
);
6285 coding
->produced_char
= SCHARS (str
);
6286 return (nocopy
? str
: Fcopy_sequence (str
));
6289 if (coding
->composing
!= COMPOSITION_DISABLED
)
6290 coding_allocate_composition_data (coding
, from
);
6291 len
= decoding_buffer_size (coding
, to_byte
- from
);
6292 allocate_conversion_buffer (buf
, len
);
6294 consumed
= consumed_char
= produced
= produced_char
= 0;
6297 result
= decode_coding (coding
, SDATA (str
) + from
+ consumed
,
6298 buf
.data
+ produced
, to_byte
- from
- consumed
,
6299 buf
.size
- produced
);
6300 consumed
+= coding
->consumed
;
6301 consumed_char
+= coding
->consumed_char
;
6302 produced
+= coding
->produced
;
6303 produced_char
+= coding
->produced_char
;
6304 if (result
== CODING_FINISH_NORMAL
6305 || result
== CODING_FINISH_INTERRUPT
6306 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6307 && coding
->consumed
== 0))
6309 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
6310 coding_allocate_composition_data (coding
, from
+ produced_char
);
6311 else if (result
== CODING_FINISH_INSUFFICIENT_DST
)
6312 extend_conversion_buffer (&buf
);
6313 else if (result
== CODING_FINISH_INCONSISTENT_EOL
)
6315 Lisp_Object eol_type
;
6317 /* Recover the original EOL format. */
6318 if (coding
->eol_type
== CODING_EOL_CR
)
6321 for (p
= buf
.data
; p
< buf
.data
+ produced
; p
++)
6322 if (*p
== '\n') *p
= '\r';
6324 else if (coding
->eol_type
== CODING_EOL_CRLF
)
6327 unsigned char *p0
, *p1
;
6328 for (p0
= buf
.data
, p1
= p0
+ produced
; p0
< p1
; p0
++)
6329 if (*p0
== '\n') num_eol
++;
6330 if (produced
+ num_eol
>= buf
.size
)
6331 extend_conversion_buffer (&buf
);
6332 for (p0
= buf
.data
+ produced
, p1
= p0
+ num_eol
; p0
> buf
.data
;)
6335 if (*p0
== '\n') *--p1
= '\r';
6337 produced
+= num_eol
;
6338 produced_char
+= num_eol
;
6340 /* Suppress eol-format conversion in the further conversion. */
6341 coding
->eol_type
= CODING_EOL_LF
;
6343 /* Set the coding system symbol to that for Unix-like EOL. */
6344 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
6345 if (VECTORP (eol_type
)
6346 && XVECTOR (eol_type
)->size
== 3
6347 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
6348 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
6350 coding
->symbol
= saved_coding_symbol
;
6356 coding
->consumed
= consumed
;
6357 coding
->consumed_char
= consumed_char
;
6358 coding
->produced
= produced
;
6359 coding
->produced_char
= produced_char
;
6361 if (coding
->dst_multibyte
)
6362 newstr
= make_uninit_multibyte_string (produced_char
+ shrinked_bytes
,
6363 produced
+ shrinked_bytes
);
6365 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6367 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6368 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6369 if (shrinked_bytes
> from
)
6370 STRING_COPYIN (newstr
, from
+ produced
,
6371 SDATA (str
) + to_byte
,
6372 shrinked_bytes
- from
);
6373 free_conversion_buffer (&buf
);
6375 coding
->consumed
+= shrinked_bytes
;
6376 coding
->consumed_char
+= shrinked_bytes
;
6377 coding
->produced
+= shrinked_bytes
;
6378 coding
->produced_char
+= shrinked_bytes
;
6380 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
6381 coding_restore_composition (coding
, newstr
);
6382 coding_free_composition_data (coding
);
6384 if (SYMBOLP (coding
->post_read_conversion
)
6385 && !NILP (Ffboundp (coding
->post_read_conversion
)))
6386 newstr
= run_pre_post_conversion_on_str (newstr
, coding
, 0);
6392 encode_coding_string (str
, coding
, nocopy
)
6394 struct coding_system
*coding
;
6398 struct conversion_buffer buf
;
6399 int from
, to
, to_byte
;
6401 int shrinked_bytes
= 0;
6403 int consumed
, consumed_char
, produced
, produced_char
;
6405 if (SYMBOLP (coding
->pre_write_conversion
)
6406 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
6408 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
6409 /* As STR is just newly generated, we don't have to copy it
6416 to_byte
= SBYTES (str
);
6418 /* Encoding routines determine the multibyteness of the source text
6419 by coding->src_multibyte. */
6420 coding
->src_multibyte
= SCHARS (str
) < SBYTES (str
);
6421 coding
->dst_multibyte
= 0;
6422 if (! CODING_REQUIRE_ENCODING (coding
))
6423 goto no_need_of_encoding
;
6425 if (coding
->composing
!= COMPOSITION_DISABLED
)
6426 coding_save_composition (coding
, from
, to
, str
);
6428 /* Try to skip the heading and tailing ASCIIs. We can't skip them
6429 if we must run CCL program or there are compositions to
6431 if (coding
->type
!= coding_type_ccl
6432 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
6434 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6436 if (from
== to_byte
)
6438 coding_free_composition_data (coding
);
6439 goto no_need_of_encoding
;
6441 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6444 len
= encoding_buffer_size (coding
, to_byte
- from
);
6445 allocate_conversion_buffer (buf
, len
);
6447 consumed
= consumed_char
= produced
= produced_char
= 0;
6450 result
= encode_coding (coding
, SDATA (str
) + from
+ consumed
,
6451 buf
.data
+ produced
, to_byte
- from
- consumed
,
6452 buf
.size
- produced
);
6453 consumed
+= coding
->consumed
;
6454 consumed_char
+= coding
->consumed_char
;
6455 produced
+= coding
->produced
;
6456 produced_char
+= coding
->produced_char
;
6457 if (result
== CODING_FINISH_NORMAL
6458 || result
== CODING_FINISH_INTERRUPT
6459 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6460 && coding
->consumed
== 0))
6462 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6463 extend_conversion_buffer (&buf
);
6466 coding
->consumed
= consumed
;
6467 coding
->consumed_char
= consumed_char
;
6468 coding
->produced
= produced
;
6469 coding
->produced_char
= produced_char
;
6471 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6473 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6474 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6475 if (shrinked_bytes
> from
)
6476 STRING_COPYIN (newstr
, from
+ produced
,
6477 SDATA (str
) + to_byte
,
6478 shrinked_bytes
- from
);
6480 free_conversion_buffer (&buf
);
6481 coding_free_composition_data (coding
);
6485 no_need_of_encoding
:
6486 coding
->consumed
= SBYTES (str
);
6487 coding
->consumed_char
= SCHARS (str
);
6488 if (STRING_MULTIBYTE (str
))
6491 /* We are sure that STR doesn't contain a multibyte
6493 STRING_SET_UNIBYTE (str
);
6496 str
= Fstring_as_unibyte (str
);
6500 coding
->produced
= SBYTES (str
);
6501 coding
->produced_char
= SCHARS (str
);
6502 return (nocopy
? str
: Fcopy_sequence (str
));
6507 /*** 8. Emacs Lisp library functions ***/
6509 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
6510 doc
: /* Return t if OBJECT is nil or a coding-system.
6511 See the documentation of `make-coding-system' for information
6512 about coding-system objects. */)
6520 if (! NILP (Fget (obj
, Qcoding_system_define_form
)))
6522 /* Get coding-spec vector for OBJ. */
6523 obj
= Fget (obj
, Qcoding_system
);
6524 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
6528 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
6529 Sread_non_nil_coding_system
, 1, 1, 0,
6530 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6537 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6538 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
6540 while (SCHARS (val
) == 0);
6541 return (Fintern (val
, Qnil
));
6544 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
6545 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6546 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
6547 (prompt
, default_coding_system
)
6548 Lisp_Object prompt
, default_coding_system
;
6551 if (SYMBOLP (default_coding_system
))
6552 default_coding_system
= SYMBOL_NAME (default_coding_system
);
6553 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6554 Qt
, Qnil
, Qcoding_system_history
,
6555 default_coding_system
, Qnil
);
6556 return (SCHARS (val
) == 0 ? Qnil
: Fintern (val
, Qnil
));
6559 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
6561 doc
: /* Check validity of CODING-SYSTEM.
6562 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6563 It is valid if it is nil or a symbol with a non-nil `coding-system' property.
6564 The value of this property should be a vector of length 5. */)
6566 Lisp_Object coding_system
;
6568 Lisp_Object define_form
;
6570 define_form
= Fget (coding_system
, Qcoding_system_define_form
);
6571 if (! NILP (define_form
))
6573 Fput (coding_system
, Qcoding_system_define_form
, Qnil
);
6574 safe_eval (define_form
);
6576 if (!NILP (Fcoding_system_p (coding_system
)))
6577 return coding_system
;
6579 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
6583 detect_coding_system (src
, src_bytes
, highest
, multibytep
)
6584 const unsigned char *src
;
6585 int src_bytes
, highest
;
6588 int coding_mask
, eol_type
;
6589 Lisp_Object val
, tmp
;
6592 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
, multibytep
);
6593 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
6594 if (eol_type
== CODING_EOL_INCONSISTENT
)
6595 eol_type
= CODING_EOL_UNDECIDED
;
6600 if (eol_type
!= CODING_EOL_UNDECIDED
)
6603 val2
= Fget (Qundecided
, Qeol_type
);
6605 val
= XVECTOR (val2
)->contents
[eol_type
];
6607 return (highest
? val
: Fcons (val
, Qnil
));
6610 /* At first, gather possible coding systems in VAL. */
6612 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
6614 Lisp_Object category_val
, category_index
;
6616 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
6617 category_val
= Fsymbol_value (XCAR (tmp
));
6618 if (!NILP (category_val
)
6619 && NATNUMP (category_index
)
6620 && (coding_mask
& (1 << XFASTINT (category_index
))))
6622 val
= Fcons (category_val
, val
);
6628 val
= Fnreverse (val
);
6630 /* Then, replace the elements with subsidiary coding systems. */
6631 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
6633 if (eol_type
!= CODING_EOL_UNDECIDED
6634 && eol_type
!= CODING_EOL_INCONSISTENT
)
6637 eol
= Fget (XCAR (tmp
), Qeol_type
);
6639 XSETCAR (tmp
, XVECTOR (eol
)->contents
[eol_type
]);
6642 return (highest
? XCAR (val
) : val
);
6645 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
6647 doc
: /* Detect how the byte sequence in the region is encoded.
6648 Return a list of possible coding systems used on decoding a byte
6649 sequence containing the bytes in the region between START and END when
6650 the coding system `undecided' is specified. The list is ordered by
6651 priority decided in the current language environment.
6653 If only ASCII characters are found, it returns a list of single element
6654 `undecided' or its subsidiary coding system according to a detected
6657 If optional argument HIGHEST is non-nil, return the coding system of
6658 highest priority. */)
6659 (start
, end
, highest
)
6660 Lisp_Object start
, end
, highest
;
6663 int from_byte
, to_byte
;
6664 int include_anchor_byte
= 0;
6666 CHECK_NUMBER_COERCE_MARKER (start
);
6667 CHECK_NUMBER_COERCE_MARKER (end
);
6669 validate_region (&start
, &end
);
6670 from
= XINT (start
), to
= XINT (end
);
6671 from_byte
= CHAR_TO_BYTE (from
);
6672 to_byte
= CHAR_TO_BYTE (to
);
6674 if (from
< GPT
&& to
>= GPT
)
6675 move_gap_both (to
, to_byte
);
6676 /* If we an anchor byte `\0' follows the region, we include it in
6677 the detecting source. Then code detectors can handle the tailing
6678 byte sequence more accurately.
6680 Fix me: This is not a perfect solution. It is better that we
6681 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6683 if (to
== Z
|| (to
== GPT
&& GAP_SIZE
> 0))
6684 include_anchor_byte
= 1;
6685 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
6686 to_byte
- from_byte
+ include_anchor_byte
,
6688 !NILP (current_buffer
6689 ->enable_multibyte_characters
));
6692 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
6694 doc
: /* Detect how the byte sequence in STRING is encoded.
6695 Return a list of possible coding systems used on decoding a byte
6696 sequence containing the bytes in STRING when the coding system
6697 `undecided' is specified. The list is ordered by priority decided in
6698 the current language environment.
6700 If only ASCII characters are found, it returns a list of single element
6701 `undecided' or its subsidiary coding system according to a detected
6704 If optional argument HIGHEST is non-nil, return the coding system of
6705 highest priority. */)
6707 Lisp_Object string
, highest
;
6709 CHECK_STRING (string
);
6711 return detect_coding_system (SDATA (string
),
6712 /* "+ 1" is to include the anchor byte
6713 `\0'. With this, code detectors can
6714 handle the tailing bytes more
6716 SBYTES (string
) + 1,
6718 STRING_MULTIBYTE (string
));
6721 /* Subroutine for Ffind_coding_systems_region_internal.
6723 Return a list of coding systems that safely encode the multibyte
6724 text between P and PEND. SAFE_CODINGS, if non-nil, is an alist of
6725 possible coding systems. If it is nil, it means that we have not
6726 yet found any coding systems.
6728 WORK_TABLE a char-table of which element is set to t once the
6729 element is looked up.
6731 If a non-ASCII single byte char is found, set
6732 *single_byte_char_found to 1. */
6735 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
6736 unsigned char *p
, *pend
;
6737 Lisp_Object safe_codings
, work_table
;
6738 int *single_byte_char_found
;
6741 Lisp_Object val
, ch
;
6742 Lisp_Object prev
, tail
;
6744 if (NILP (safe_codings
))
6745 goto done_safe_codings
;
6748 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6750 if (ASCII_BYTE_P (c
))
6751 /* We can ignore ASCII characters here. */
6753 if (SINGLE_BYTE_CHAR_P (c
))
6754 *single_byte_char_found
= 1;
6755 /* Check the safe coding systems for C. */
6756 ch
= make_number (c
);
6757 val
= Faref (work_table
, ch
);
6759 /* This element was already checked. Ignore it. */
6761 /* Remember that we checked this element. */
6762 Faset (work_table
, ch
, Qt
);
6764 for (prev
= tail
= safe_codings
; CONSP (tail
); tail
= XCDR (tail
))
6766 Lisp_Object elt
, translation_table
, hash_table
, accept_latin_extra
;
6770 if (CONSP (XCDR (elt
)))
6772 /* This entry has this format now:
6773 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6774 ACCEPT-LATIN-EXTRA ) */
6776 encodable
= ! NILP (Faref (XCAR (val
), ch
));
6780 translation_table
= XCAR (val
);
6781 hash_table
= XCAR (XCDR (val
));
6782 accept_latin_extra
= XCAR (XCDR (XCDR (val
)));
6787 /* This entry has this format now: ( CODING . SAFE-CHARS) */
6788 encodable
= ! NILP (Faref (XCDR (elt
), ch
));
6791 /* Transform the format to:
6792 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6793 ACCEPT-LATIN-EXTRA ) */
6794 val
= Fget (XCAR (elt
), Qcoding_system
);
6796 = Fplist_get (AREF (val
, 3),
6797 Qtranslation_table_for_encode
);
6798 if (SYMBOLP (translation_table
))
6799 translation_table
= Fget (translation_table
,
6800 Qtranslation_table
);
6802 = (CHAR_TABLE_P (translation_table
)
6803 ? XCHAR_TABLE (translation_table
)->extras
[1]
6806 = ((EQ (AREF (val
, 0), make_number (2))
6807 && VECTORP (AREF (val
, 4)))
6808 ? AREF (AREF (val
, 4), 16)
6810 XSETCAR (tail
, list5 (XCAR (elt
), XCDR (elt
),
6811 translation_table
, hash_table
,
6812 accept_latin_extra
));
6817 && ((CHAR_TABLE_P (translation_table
)
6818 && ! NILP (Faref (translation_table
, ch
)))
6819 || (HASH_TABLE_P (hash_table
)
6820 && ! NILP (Fgethash (ch
, hash_table
, Qnil
)))
6821 || (SINGLE_BYTE_CHAR_P (c
)
6822 && ! NILP (accept_latin_extra
)
6823 && VECTORP (Vlatin_extra_code_table
)
6824 && ! NILP (AREF (Vlatin_extra_code_table
, c
)))))
6830 /* Exclude this coding system from SAFE_CODINGS. */
6831 if (EQ (tail
, safe_codings
))
6833 safe_codings
= XCDR (safe_codings
);
6834 if (NILP (safe_codings
))
6835 goto done_safe_codings
;
6838 XSETCDR (prev
, XCDR (tail
));
6844 /* If the above loop was terminated before P reaches PEND, it means
6845 SAFE_CODINGS was set to nil. If we have not yet found an
6846 non-ASCII single-byte char, check it now. */
6847 if (! *single_byte_char_found
)
6850 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6852 if (! ASCII_BYTE_P (c
)
6853 && SINGLE_BYTE_CHAR_P (c
))
6855 *single_byte_char_found
= 1;
6859 return safe_codings
;
6862 DEFUN ("find-coding-systems-region-internal",
6863 Ffind_coding_systems_region_internal
,
6864 Sfind_coding_systems_region_internal
, 2, 2, 0,
6865 doc
: /* Internal use only. */)
6867 Lisp_Object start
, end
;
6869 Lisp_Object work_table
, safe_codings
;
6870 int non_ascii_p
= 0;
6871 int single_byte_char_found
= 0;
6872 const unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
6874 if (STRINGP (start
))
6876 if (!STRING_MULTIBYTE (start
))
6878 p1
= SDATA (start
), p1end
= p1
+ SBYTES (start
);
6880 if (SCHARS (start
) != SBYTES (start
))
6887 CHECK_NUMBER_COERCE_MARKER (start
);
6888 CHECK_NUMBER_COERCE_MARKER (end
);
6889 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
6890 args_out_of_range (start
, end
);
6891 if (NILP (current_buffer
->enable_multibyte_characters
))
6893 from
= CHAR_TO_BYTE (XINT (start
));
6894 to
= CHAR_TO_BYTE (XINT (end
));
6895 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
6896 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
6900 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
6901 if (XINT (end
) - XINT (start
) != to
- from
)
6907 /* We are sure that the text contains no multibyte character.
6908 Check if it contains eight-bit-graphic. */
6910 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
6913 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
6919 /* The text contains non-ASCII characters. */
6921 work_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
6922 safe_codings
= Fcopy_sequence (XCDR (Vcoding_system_safe_chars
));
6924 safe_codings
= find_safe_codings (p1
, p1end
, safe_codings
, work_table
,
6925 &single_byte_char_found
);
6927 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
6928 &single_byte_char_found
);
6929 if (EQ (safe_codings
, XCDR (Vcoding_system_safe_chars
)))
6933 /* Turn safe_codings to a list of coding systems... */
6936 if (single_byte_char_found
)
6937 /* ... and append these for eight-bit chars. */
6938 val
= Fcons (Qraw_text
,
6939 Fcons (Qemacs_mule
, Fcons (Qno_conversion
, Qnil
)));
6941 /* ... and append generic coding systems. */
6942 val
= Fcopy_sequence (XCAR (Vcoding_system_safe_chars
));
6944 for (; CONSP (safe_codings
); safe_codings
= XCDR (safe_codings
))
6945 val
= Fcons (XCAR (XCAR (safe_codings
)), val
);
6949 return safe_codings
;
6953 /* Search from position POS for such characters that are unencodable
6954 accoding to SAFE_CHARS, and return a list of their positions. P
6955 points where in the memory the character at POS exists. Limit the
6956 search at PEND or when Nth unencodable characters are found.
6958 If SAFE_CHARS is a char table, an element for an unencodable
6961 If SAFE_CHARS is nil, all non-ASCII characters are unencodable.
6963 Otherwise, SAFE_CHARS is t, and only eight-bit-contrl and
6964 eight-bit-graphic characters are unencodable. */
6967 unencodable_char_position (safe_chars
, pos
, p
, pend
, n
)
6968 Lisp_Object safe_chars
;
6970 unsigned char *p
, *pend
;
6973 Lisp_Object pos_list
;
6979 int c
= STRING_CHAR_AND_LENGTH (p
, MAX_MULTIBYTE_LENGTH
, len
);
6982 && (CHAR_TABLE_P (safe_chars
)
6983 ? NILP (CHAR_TABLE_REF (safe_chars
, c
))
6984 : (NILP (safe_chars
) || c
< 256)))
6986 pos_list
= Fcons (make_number (pos
), pos_list
);
6993 return Fnreverse (pos_list
);
6997 DEFUN ("unencodable-char-position", Funencodable_char_position
,
6998 Sunencodable_char_position
, 3, 5, 0,
7000 Return position of first un-encodable character in a region.
7001 START and END specfiy the region and CODING-SYSTEM specifies the
7002 encoding to check. Return nil if CODING-SYSTEM does encode the region.
7004 If optional 4th argument COUNT is non-nil, it specifies at most how
7005 many un-encodable characters to search. In this case, the value is a
7008 If optional 5th argument STRING is non-nil, it is a string to search
7009 for un-encodable characters. In that case, START and END are indexes
7011 (start
, end
, coding_system
, count
, string
)
7012 Lisp_Object start
, end
, coding_system
, count
, string
;
7015 Lisp_Object safe_chars
;
7016 struct coding_system coding
;
7017 Lisp_Object positions
;
7019 unsigned char *p
, *pend
;
7023 validate_region (&start
, &end
);
7024 from
= XINT (start
);
7026 if (NILP (current_buffer
->enable_multibyte_characters
))
7028 p
= CHAR_POS_ADDR (from
);
7032 pend
= CHAR_POS_ADDR (to
);
7036 CHECK_STRING (string
);
7037 CHECK_NATNUM (start
);
7039 from
= XINT (start
);
7042 || to
> SCHARS (string
))
7043 args_out_of_range_3 (string
, start
, end
);
7044 if (! STRING_MULTIBYTE (string
))
7046 p
= SDATA (string
) + string_char_to_byte (string
, from
);
7047 pend
= SDATA (string
) + string_char_to_byte (string
, to
);
7050 setup_coding_system (Fcheck_coding_system (coding_system
), &coding
);
7056 CHECK_NATNUM (count
);
7060 if (coding
.type
== coding_type_no_conversion
7061 || coding
.type
== coding_type_raw_text
)
7064 if (coding
.type
== coding_type_undecided
)
7067 safe_chars
= coding_safe_chars (coding_system
);
7069 if (STRINGP (string
)
7070 || from
>= GPT
|| to
<= GPT
)
7071 positions
= unencodable_char_position (safe_chars
, from
, p
, pend
, n
);
7074 Lisp_Object args
[2];
7076 args
[0] = unencodable_char_position (safe_chars
, from
, p
, GPT_ADDR
, n
);
7077 n
-= XINT (Flength (args
[0]));
7079 positions
= args
[0];
7082 args
[1] = unencodable_char_position (safe_chars
, GPT
, GAP_END_ADDR
,
7084 positions
= Fappend (2, args
);
7088 return (NILP (count
) ? Fcar (positions
) : positions
);
7093 code_convert_region1 (start
, end
, coding_system
, encodep
)
7094 Lisp_Object start
, end
, coding_system
;
7097 struct coding_system coding
;
7100 CHECK_NUMBER_COERCE_MARKER (start
);
7101 CHECK_NUMBER_COERCE_MARKER (end
);
7102 CHECK_SYMBOL (coding_system
);
7104 validate_region (&start
, &end
);
7105 from
= XFASTINT (start
);
7106 to
= XFASTINT (end
);
7108 if (NILP (coding_system
))
7109 return make_number (to
- from
);
7111 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7112 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7114 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7115 coding
.src_multibyte
= coding
.dst_multibyte
7116 = !NILP (current_buffer
->enable_multibyte_characters
);
7117 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
7118 &coding
, encodep
, 1);
7119 Vlast_coding_system_used
= coding
.symbol
;
7120 return make_number (coding
.produced_char
);
7123 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
7124 3, 3, "r\nzCoding system: ",
7125 doc
: /* Decode the current region from the specified coding system.
7126 When called from a program, takes three arguments:
7127 START, END, and CODING-SYSTEM. START and END are buffer positions.
7128 This function sets `last-coding-system-used' to the precise coding system
7129 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7130 not fully specified.)
7131 It returns the length of the decoded text. */)
7132 (start
, end
, coding_system
)
7133 Lisp_Object start
, end
, coding_system
;
7135 return code_convert_region1 (start
, end
, coding_system
, 0);
7138 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
7139 3, 3, "r\nzCoding system: ",
7140 doc
: /* Encode the current region into the specified coding system.
7141 When called from a program, takes three arguments:
7142 START, END, and CODING-SYSTEM. START and END are buffer positions.
7143 This function sets `last-coding-system-used' to the precise coding system
7144 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7145 not fully specified.)
7146 It returns the length of the encoded text. */)
7147 (start
, end
, coding_system
)
7148 Lisp_Object start
, end
, coding_system
;
7150 return code_convert_region1 (start
, end
, coding_system
, 1);
7154 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
7155 Lisp_Object string
, coding_system
, nocopy
;
7158 struct coding_system coding
;
7160 CHECK_STRING (string
);
7161 CHECK_SYMBOL (coding_system
);
7163 if (NILP (coding_system
))
7164 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
7166 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7167 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7169 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7171 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
7172 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
7173 Vlast_coding_system_used
= coding
.symbol
;
7178 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
7180 doc
: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7181 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7182 if the decoding operation is trivial.
7183 This function sets `last-coding-system-used' to the precise coding system
7184 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7185 not fully specified.) */)
7186 (string
, coding_system
, nocopy
)
7187 Lisp_Object string
, coding_system
, nocopy
;
7189 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
7192 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
7194 doc
: /* Encode STRING to CODING-SYSTEM, and return the result.
7195 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7196 if the encoding operation is trivial.
7197 This function sets `last-coding-system-used' to the precise coding system
7198 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7199 not fully specified.) */)
7200 (string
, coding_system
, nocopy
)
7201 Lisp_Object string
, coding_system
, nocopy
;
7203 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
7206 /* Encode or decode STRING according to CODING_SYSTEM.
7207 Do not set Vlast_coding_system_used.
7209 This function is called only from macros DECODE_FILE and
7210 ENCODE_FILE, thus we ignore character composition. */
7213 code_convert_string_norecord (string
, coding_system
, encodep
)
7214 Lisp_Object string
, coding_system
;
7217 struct coding_system coding
;
7219 CHECK_STRING (string
);
7220 CHECK_SYMBOL (coding_system
);
7222 if (NILP (coding_system
))
7225 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7226 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7228 coding
.composing
= COMPOSITION_DISABLED
;
7229 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7231 ? encode_coding_string (string
, &coding
, 1)
7232 : decode_coding_string (string
, &coding
, 1));
7235 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
7236 doc
: /* Decode a Japanese character which has CODE in shift_jis encoding.
7237 Return the corresponding character. */)
7241 unsigned char c1
, c2
, s1
, s2
;
7244 CHECK_NUMBER (code
);
7245 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
7249 XSETFASTINT (val
, s2
);
7250 else if (s2
>= 0xA0 || s2
<= 0xDF)
7251 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
7253 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7257 if ((s1
< 0x80 || (s1
> 0x9F && s1
< 0xE0) || s1
> 0xEF)
7258 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
7259 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7260 DECODE_SJIS (s1
, s2
, c1
, c2
);
7261 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
7266 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
7267 doc
: /* Encode a Japanese character CHAR to shift_jis encoding.
7268 Return the corresponding code in SJIS. */)
7272 int charset
, c1
, c2
, s1
, s2
;
7276 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7277 if (charset
== CHARSET_ASCII
)
7281 else if (charset
== charset_jisx0208
7282 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
7284 ENCODE_SJIS (c1
, c2
, s1
, s2
);
7285 XSETFASTINT (val
, (s1
<< 8) | s2
);
7287 else if (charset
== charset_katakana_jisx0201
7288 && c1
> 0x20 && c2
< 0xE0)
7290 XSETFASTINT (val
, c1
| 0x80);
7293 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
7297 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
7298 doc
: /* Decode a Big5 character which has CODE in BIG5 coding system.
7299 Return the corresponding character. */)
7304 unsigned char b1
, b2
, c1
, c2
;
7307 CHECK_NUMBER (code
);
7308 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
7312 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7317 if ((b1
< 0xA1 || b1
> 0xFE)
7318 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
7319 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7320 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
7321 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
7326 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
7327 doc
: /* Encode the Big5 character CHAR to BIG5 coding system.
7328 Return the corresponding character code in Big5. */)
7332 int charset
, c1
, c2
, b1
, b2
;
7336 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7337 if (charset
== CHARSET_ASCII
)
7341 else if ((charset
== charset_big5_1
7342 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
7343 || (charset
== charset_big5_2
7344 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
7346 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
7347 XSETFASTINT (val
, (b1
<< 8) | b2
);
7350 error ("Can't encode to Big5: %d", XFASTINT (ch
));
7354 DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal
,
7355 Sset_terminal_coding_system_internal
, 1, 2, 0,
7356 doc
: /* Internal use only. */)
7357 (coding_system
, terminal
)
7358 Lisp_Object coding_system
;
7359 Lisp_Object terminal
;
7361 struct coding_system
*terminal_coding
= TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1));
7362 CHECK_SYMBOL (coding_system
);
7363 setup_coding_system (Fcheck_coding_system (coding_system
), terminal_coding
);
7364 /* We had better not send unsafe characters to terminal. */
7365 terminal_coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
7366 /* Character composition should be disabled. */
7367 terminal_coding
->composing
= COMPOSITION_DISABLED
;
7368 /* Error notification should be suppressed. */
7369 terminal_coding
->suppress_error
= 1;
7370 terminal_coding
->src_multibyte
= 1;
7371 terminal_coding
->dst_multibyte
= 0;
7375 DEFUN ("set-safe-terminal-coding-system-internal", Fset_safe_terminal_coding_system_internal
,
7376 Sset_safe_terminal_coding_system_internal
, 1, 1, 0,
7377 doc
: /* Internal use only. */)
7379 Lisp_Object coding_system
;
7381 CHECK_SYMBOL (coding_system
);
7382 setup_coding_system (Fcheck_coding_system (coding_system
),
7383 &safe_terminal_coding
);
7384 /* Character composition should be disabled. */
7385 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
7386 /* Error notification should be suppressed. */
7387 safe_terminal_coding
.suppress_error
= 1;
7388 safe_terminal_coding
.src_multibyte
= 1;
7389 safe_terminal_coding
.dst_multibyte
= 0;
7393 DEFUN ("terminal-coding-system", Fterminal_coding_system
,
7394 Sterminal_coding_system
, 0, 1, 0,
7395 doc
: /* Return coding system specified for terminal output on the given terminal.
7396 TERMINAL may be a terminal id, a frame, or nil for the selected
7397 frame's terminal device. */)
7399 Lisp_Object terminal
;
7401 return TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1))->symbol
;
7404 DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal
,
7405 Sset_keyboard_coding_system_internal
, 1, 2, 0,
7406 doc
: /* Internal use only. */)
7407 (coding_system
, terminal
)
7408 Lisp_Object coding_system
;
7409 Lisp_Object terminal
;
7411 struct terminal
*t
= get_terminal (terminal
, 1);
7412 CHECK_SYMBOL (coding_system
);
7414 setup_coding_system (Fcheck_coding_system (coding_system
),
7415 TERMINAL_KEYBOARD_CODING (t
));
7416 /* Character composition should be disabled. */
7417 TERMINAL_KEYBOARD_CODING (t
)->composing
= COMPOSITION_DISABLED
;
7421 DEFUN ("keyboard-coding-system", Fkeyboard_coding_system
,
7422 Skeyboard_coding_system
, 0, 1, 0,
7423 doc
: /* Return coding system for decoding keyboard input on TERMINAL.
7424 TERMINAL may be a terminal id, a frame, or nil for the selected
7425 frame's terminal device. */)
7427 Lisp_Object terminal
;
7429 return TERMINAL_KEYBOARD_CODING (get_terminal (terminal
, 1))->symbol
;
7433 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
7434 Sfind_operation_coding_system
, 1, MANY
, 0,
7435 doc
: /* Choose a coding system for an operation based on the target name.
7436 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
7437 DECODING-SYSTEM is the coding system to use for decoding
7438 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
7439 for encoding (in case OPERATION does encoding).
7441 The first argument OPERATION specifies an I/O primitive:
7442 For file I/O, `insert-file-contents' or `write-region'.
7443 For process I/O, `call-process', `call-process-region', or `start-process'.
7444 For network I/O, `open-network-stream'.
7446 The remaining arguments should be the same arguments that were passed
7447 to the primitive. Depending on which primitive, one of those arguments
7448 is selected as the TARGET. For example, if OPERATION does file I/O,
7449 whichever argument specifies the file name is TARGET.
7451 TARGET has a meaning which depends on OPERATION:
7452 For file I/O, TARGET is a file name.
7453 For process I/O, TARGET is a process name.
7454 For network I/O, TARGET is a service name or a port number
7456 This function looks up what specified for TARGET in,
7457 `file-coding-system-alist', `process-coding-system-alist',
7458 or `network-coding-system-alist' depending on OPERATION.
7459 They may specify a coding system, a cons of coding systems,
7460 or a function symbol to call.
7461 In the last case, we call the function with one argument,
7462 which is a list of all the arguments given to this function.
7464 usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
7469 Lisp_Object operation
, target_idx
, target
, val
;
7470 register Lisp_Object chain
;
7473 error ("Too few arguments");
7474 operation
= args
[0];
7475 if (!SYMBOLP (operation
)
7476 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
7477 error ("Invalid first argument");
7478 if (nargs
< 1 + XINT (target_idx
))
7479 error ("Too few arguments for operation: %s",
7480 SDATA (SYMBOL_NAME (operation
)));
7481 /* For write-region, if the 6th argument (i.e. VISIT, the 5th
7482 argument to write-region) is string, it must be treated as a
7483 target file name. */
7484 if (EQ (operation
, Qwrite_region
)
7486 && STRINGP (args
[5]))
7487 target_idx
= make_number (4);
7488 target
= args
[XINT (target_idx
) + 1];
7489 if (!(STRINGP (target
)
7490 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
7491 error ("Invalid argument %d", XINT (target_idx
) + 1);
7493 chain
= ((EQ (operation
, Qinsert_file_contents
)
7494 || EQ (operation
, Qwrite_region
))
7495 ? Vfile_coding_system_alist
7496 : (EQ (operation
, Qopen_network_stream
)
7497 ? Vnetwork_coding_system_alist
7498 : Vprocess_coding_system_alist
));
7502 for (; CONSP (chain
); chain
= XCDR (chain
))
7508 && ((STRINGP (target
)
7509 && STRINGP (XCAR (elt
))
7510 && fast_string_match (XCAR (elt
), target
) >= 0)
7511 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
7514 /* Here, if VAL is both a valid coding system and a valid
7515 function symbol, we return VAL as a coding system. */
7518 if (! SYMBOLP (val
))
7520 if (! NILP (Fcoding_system_p (val
)))
7521 return Fcons (val
, val
);
7522 if (! NILP (Ffboundp (val
)))
7524 val
= call1 (val
, Flist (nargs
, args
));
7527 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
7528 return Fcons (val
, val
);
7536 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
7537 Supdate_coding_systems_internal
, 0, 0, 0,
7538 doc
: /* Update internal database for ISO2022 and CCL based coding systems.
7539 When values of any coding categories are changed, you must
7540 call this function. */)
7545 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7549 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[i
]);
7552 if (! coding_system_table
[i
])
7553 coding_system_table
[i
] = ((struct coding_system
*)
7554 xmalloc (sizeof (struct coding_system
)));
7555 setup_coding_system (val
, coding_system_table
[i
]);
7557 else if (coding_system_table
[i
])
7559 xfree (coding_system_table
[i
]);
7560 coding_system_table
[i
] = NULL
;
7567 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
7568 Sset_coding_priority_internal
, 0, 0, 0,
7569 doc
: /* Update internal database for the current value of `coding-category-list'.
7570 This function is internal use only. */)
7576 val
= Vcoding_category_list
;
7578 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
7580 if (! SYMBOLP (XCAR (val
)))
7582 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
7583 if (idx
>= CODING_CATEGORY_IDX_MAX
)
7585 coding_priorities
[i
++] = (1 << idx
);
7588 /* If coding-category-list is valid and contains all coding
7589 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
7590 the following code saves Emacs from crashing. */
7591 while (i
< CODING_CATEGORY_IDX_MAX
)
7592 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
7597 DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal
,
7598 Sdefine_coding_system_internal
, 1, 1, 0,
7599 doc
: /* Register CODING-SYSTEM as a base coding system.
7600 This function is internal use only. */)
7602 Lisp_Object coding_system
;
7604 Lisp_Object safe_chars
, slot
;
7606 if (NILP (Fcheck_coding_system (coding_system
)))
7607 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
7608 safe_chars
= coding_safe_chars (coding_system
);
7609 if (! EQ (safe_chars
, Qt
) && ! CHAR_TABLE_P (safe_chars
))
7610 error ("No valid safe-chars property for %s",
7611 SDATA (SYMBOL_NAME (coding_system
)));
7612 if (EQ (safe_chars
, Qt
))
7614 if (NILP (Fmemq (coding_system
, XCAR (Vcoding_system_safe_chars
))))
7615 XSETCAR (Vcoding_system_safe_chars
,
7616 Fcons (coding_system
, XCAR (Vcoding_system_safe_chars
)));
7620 slot
= Fassq (coding_system
, XCDR (Vcoding_system_safe_chars
));
7622 XSETCDR (Vcoding_system_safe_chars
,
7623 nconc2 (XCDR (Vcoding_system_safe_chars
),
7624 Fcons (Fcons (coding_system
, safe_chars
), Qnil
)));
7626 XSETCDR (slot
, safe_chars
);
7634 /*** 9. Post-amble ***/
7641 /* Emacs' internal format specific initialize routine. */
7642 for (i
= 0; i
<= 0x20; i
++)
7643 emacs_code_class
[i
] = EMACS_control_code
;
7644 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
7645 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
7646 for (i
= 0x21 ; i
< 0x7F; i
++)
7647 emacs_code_class
[i
] = EMACS_ascii_code
;
7648 emacs_code_class
[0x7F] = EMACS_control_code
;
7649 for (i
= 0x80; i
< 0xFF; i
++)
7650 emacs_code_class
[i
] = EMACS_invalid_code
;
7651 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
7652 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
7653 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
7654 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
7656 /* ISO2022 specific initialize routine. */
7657 for (i
= 0; i
< 0x20; i
++)
7658 iso_code_class
[i
] = ISO_control_0
;
7659 for (i
= 0x21; i
< 0x7F; i
++)
7660 iso_code_class
[i
] = ISO_graphic_plane_0
;
7661 for (i
= 0x80; i
< 0xA0; i
++)
7662 iso_code_class
[i
] = ISO_control_1
;
7663 for (i
= 0xA1; i
< 0xFF; i
++)
7664 iso_code_class
[i
] = ISO_graphic_plane_1
;
7665 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
7666 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
7667 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
7668 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
7669 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
7670 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
7671 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
7672 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
7673 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
7674 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
7676 setup_coding_system (Qnil
, &safe_terminal_coding
);
7677 setup_coding_system (Qnil
, &default_buffer_file_coding
);
7679 bzero (coding_system_table
, sizeof coding_system_table
);
7681 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
7682 for (i
= 0; i
< 128; i
++)
7683 ascii_skip_code
[i
] = 1;
7685 #if defined (MSDOS) || defined (WINDOWSNT)
7686 system_eol_type
= CODING_EOL_CRLF
;
7688 system_eol_type
= CODING_EOL_LF
;
7691 inhibit_pre_post_conversion
= 0;
7699 staticpro (&Vcode_conversion_workbuf_name
);
7700 Vcode_conversion_workbuf_name
= build_string (" *code-conversion-work*");
7702 Qtarget_idx
= intern ("target-idx");
7703 staticpro (&Qtarget_idx
);
7705 Qcoding_system_history
= intern ("coding-system-history");
7706 staticpro (&Qcoding_system_history
);
7707 Fset (Qcoding_system_history
, Qnil
);
7709 /* Target FILENAME is the first argument. */
7710 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
7711 /* Target FILENAME is the third argument. */
7712 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
7714 Qcall_process
= intern ("call-process");
7715 staticpro (&Qcall_process
);
7716 /* Target PROGRAM is the first argument. */
7717 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
7719 Qcall_process_region
= intern ("call-process-region");
7720 staticpro (&Qcall_process_region
);
7721 /* Target PROGRAM is the third argument. */
7722 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
7724 Qstart_process
= intern ("start-process");
7725 staticpro (&Qstart_process
);
7726 /* Target PROGRAM is the third argument. */
7727 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
7729 Qopen_network_stream
= intern ("open-network-stream");
7730 staticpro (&Qopen_network_stream
);
7731 /* Target SERVICE is the fourth argument. */
7732 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
7734 Qcoding_system
= intern ("coding-system");
7735 staticpro (&Qcoding_system
);
7737 Qeol_type
= intern ("eol-type");
7738 staticpro (&Qeol_type
);
7740 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
7741 staticpro (&Qbuffer_file_coding_system
);
7743 Qpost_read_conversion
= intern ("post-read-conversion");
7744 staticpro (&Qpost_read_conversion
);
7746 Qpre_write_conversion
= intern ("pre-write-conversion");
7747 staticpro (&Qpre_write_conversion
);
7749 Qno_conversion
= intern ("no-conversion");
7750 staticpro (&Qno_conversion
);
7752 Qundecided
= intern ("undecided");
7753 staticpro (&Qundecided
);
7755 Qcoding_system_p
= intern ("coding-system-p");
7756 staticpro (&Qcoding_system_p
);
7758 Qcoding_system_error
= intern ("coding-system-error");
7759 staticpro (&Qcoding_system_error
);
7761 Fput (Qcoding_system_error
, Qerror_conditions
,
7762 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
7763 Fput (Qcoding_system_error
, Qerror_message
,
7764 build_string ("Invalid coding system"));
7766 Qcoding_category
= intern ("coding-category");
7767 staticpro (&Qcoding_category
);
7768 Qcoding_category_index
= intern ("coding-category-index");
7769 staticpro (&Qcoding_category_index
);
7771 Vcoding_category_table
7772 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
7773 staticpro (&Vcoding_category_table
);
7776 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7778 XVECTOR (Vcoding_category_table
)->contents
[i
]
7779 = intern (coding_category_name
[i
]);
7780 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
7781 Qcoding_category_index
, make_number (i
));
7785 Vcoding_system_safe_chars
= Fcons (Qnil
, Qnil
);
7786 staticpro (&Vcoding_system_safe_chars
);
7788 Qtranslation_table
= intern ("translation-table");
7789 staticpro (&Qtranslation_table
);
7790 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (2));
7792 Qtranslation_table_id
= intern ("translation-table-id");
7793 staticpro (&Qtranslation_table_id
);
7795 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
7796 staticpro (&Qtranslation_table_for_decode
);
7798 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
7799 staticpro (&Qtranslation_table_for_encode
);
7801 Qsafe_chars
= intern ("safe-chars");
7802 staticpro (&Qsafe_chars
);
7804 Qchar_coding_system
= intern ("char-coding-system");
7805 staticpro (&Qchar_coding_system
);
7807 /* Intern this now in case it isn't already done.
7808 Setting this variable twice is harmless.
7809 But don't staticpro it here--that is done in alloc.c. */
7810 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
7811 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
7812 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (0));
7814 Qvalid_codes
= intern ("valid-codes");
7815 staticpro (&Qvalid_codes
);
7817 Qemacs_mule
= intern ("emacs-mule");
7818 staticpro (&Qemacs_mule
);
7820 Qraw_text
= intern ("raw-text");
7821 staticpro (&Qraw_text
);
7823 Qutf_8
= intern ("utf-8");
7824 staticpro (&Qutf_8
);
7826 Qcoding_system_define_form
= intern ("coding-system-define-form");
7827 staticpro (&Qcoding_system_define_form
);
7829 defsubr (&Scoding_system_p
);
7830 defsubr (&Sread_coding_system
);
7831 defsubr (&Sread_non_nil_coding_system
);
7832 defsubr (&Scheck_coding_system
);
7833 defsubr (&Sdetect_coding_region
);
7834 defsubr (&Sdetect_coding_string
);
7835 defsubr (&Sfind_coding_systems_region_internal
);
7836 defsubr (&Sunencodable_char_position
);
7837 defsubr (&Sdecode_coding_region
);
7838 defsubr (&Sencode_coding_region
);
7839 defsubr (&Sdecode_coding_string
);
7840 defsubr (&Sencode_coding_string
);
7841 defsubr (&Sdecode_sjis_char
);
7842 defsubr (&Sencode_sjis_char
);
7843 defsubr (&Sdecode_big5_char
);
7844 defsubr (&Sencode_big5_char
);
7845 defsubr (&Sset_terminal_coding_system_internal
);
7846 defsubr (&Sset_safe_terminal_coding_system_internal
);
7847 defsubr (&Sterminal_coding_system
);
7848 defsubr (&Sset_keyboard_coding_system_internal
);
7849 defsubr (&Skeyboard_coding_system
);
7850 defsubr (&Sfind_operation_coding_system
);
7851 defsubr (&Supdate_coding_systems_internal
);
7852 defsubr (&Sset_coding_priority_internal
);
7853 defsubr (&Sdefine_coding_system_internal
);
7855 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
7856 doc
: /* List of coding systems.
7858 Do not alter the value of this variable manually. This variable should be
7859 updated by the functions `make-coding-system' and
7860 `define-coding-system-alias'. */);
7861 Vcoding_system_list
= Qnil
;
7863 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
7864 doc
: /* Alist of coding system names.
7865 Each element is one element list of coding system name.
7866 This variable is given to `completing-read' as TABLE argument.
7868 Do not alter the value of this variable manually. This variable should be
7869 updated by the functions `make-coding-system' and
7870 `define-coding-system-alias'. */);
7871 Vcoding_system_alist
= Qnil
;
7873 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
7874 doc
: /* List of coding-categories (symbols) ordered by priority.
7876 On detecting a coding system, Emacs tries code detection algorithms
7877 associated with each coding-category one by one in this order. When
7878 one algorithm agrees with a byte sequence of source text, the coding
7879 system bound to the corresponding coding-category is selected.
7881 Don't modify this variable directly, but use `set-coding-priority'. */);
7885 Vcoding_category_list
= Qnil
;
7886 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
7887 Vcoding_category_list
7888 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
7889 Vcoding_category_list
);
7892 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
7893 doc
: /* Specify the coding system for read operations.
7894 It is useful to bind this variable with `let', but do not set it globally.
7895 If the value is a coding system, it is used for decoding on read operation.
7896 If not, an appropriate element is used from one of the coding system alists:
7897 There are three such tables, `file-coding-system-alist',
7898 `process-coding-system-alist', and `network-coding-system-alist'. */);
7899 Vcoding_system_for_read
= Qnil
;
7901 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
7902 doc
: /* Specify the coding system for write operations.
7903 Programs bind this variable with `let', but you should not set it globally.
7904 If the value is a coding system, it is used for encoding of output,
7905 when writing it to a file and when sending it to a file or subprocess.
7907 If this does not specify a coding system, an appropriate element
7908 is used from one of the coding system alists:
7909 There are three such tables, `file-coding-system-alist',
7910 `process-coding-system-alist', and `network-coding-system-alist'.
7911 For output to files, if the above procedure does not specify a coding system,
7912 the value of `buffer-file-coding-system' is used. */);
7913 Vcoding_system_for_write
= Qnil
;
7915 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
7916 doc
: /* Coding system used in the latest file or process I/O.
7917 Also set by `encode-coding-region', `decode-coding-region',
7918 `encode-coding-string' and `decode-coding-string'. */);
7919 Vlast_coding_system_used
= Qnil
;
7921 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
7922 doc
: /* *Non-nil means always inhibit code conversion of end-of-line format.
7923 See info node `Coding Systems' and info node `Text and Binary' concerning
7924 such conversion. */);
7925 inhibit_eol_conversion
= 0;
7927 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
7928 doc
: /* Non-nil means process buffer inherits coding system of process output.
7929 Bind it to t if the process output is to be treated as if it were a file
7930 read from some filesystem. */);
7931 inherit_process_coding_system
= 0;
7933 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
7934 doc
: /* Alist to decide a coding system to use for a file I/O operation.
7935 The format is ((PATTERN . VAL) ...),
7936 where PATTERN is a regular expression matching a file name,
7937 VAL is a coding system, a cons of coding systems, or a function symbol.
7938 If VAL is a coding system, it is used for both decoding and encoding
7940 If VAL is a cons of coding systems, the car part is used for decoding,
7941 and the cdr part is used for encoding.
7942 If VAL is a function symbol, the function must return a coding system
7943 or a cons of coding systems which are used as above. The function gets
7944 the arguments with which `find-operation-coding-system' was called.
7946 See also the function `find-operation-coding-system'
7947 and the variable `auto-coding-alist'. */);
7948 Vfile_coding_system_alist
= Qnil
;
7950 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
7951 doc
: /* Alist to decide a coding system to use for a process I/O operation.
7952 The format is ((PATTERN . VAL) ...),
7953 where PATTERN is a regular expression matching a program name,
7954 VAL is a coding system, a cons of coding systems, or a function symbol.
7955 If VAL is a coding system, it is used for both decoding what received
7956 from the program and encoding what sent to the program.
7957 If VAL is a cons of coding systems, the car part is used for decoding,
7958 and the cdr part is used for encoding.
7959 If VAL is a function symbol, the function must return a coding system
7960 or a cons of coding systems which are used as above.
7962 See also the function `find-operation-coding-system'. */);
7963 Vprocess_coding_system_alist
= Qnil
;
7965 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
7966 doc
: /* Alist to decide a coding system to use for a network I/O operation.
7967 The format is ((PATTERN . VAL) ...),
7968 where PATTERN is a regular expression matching a network service name
7969 or is a port number to connect to,
7970 VAL is a coding system, a cons of coding systems, or a function symbol.
7971 If VAL is a coding system, it is used for both decoding what received
7972 from the network stream and encoding what sent to the network stream.
7973 If VAL is a cons of coding systems, the car part is used for decoding,
7974 and the cdr part is used for encoding.
7975 If VAL is a function symbol, the function must return a coding system
7976 or a cons of coding systems which are used as above.
7978 See also the function `find-operation-coding-system'. */);
7979 Vnetwork_coding_system_alist
= Qnil
;
7981 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
7982 doc
: /* Coding system to use with system messages.
7983 Also used for decoding keyboard input on X Window system. */);
7984 Vlocale_coding_system
= Qnil
;
7986 /* The eol mnemonics are reset in startup.el system-dependently. */
7987 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
7988 doc
: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7989 eol_mnemonic_unix
= build_string (":");
7991 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
7992 doc
: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7993 eol_mnemonic_dos
= build_string ("\\");
7995 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
7996 doc
: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
7997 eol_mnemonic_mac
= build_string ("/");
7999 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
8000 doc
: /* *String displayed in mode line when end-of-line format is not yet determined. */);
8001 eol_mnemonic_undecided
= build_string (":");
8003 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
8004 doc
: /* *Non-nil enables character translation while encoding and decoding. */);
8005 Venable_character_translation
= Qt
;
8007 DEFVAR_LISP ("standard-translation-table-for-decode",
8008 &Vstandard_translation_table_for_decode
,
8009 doc
: /* Table for translating characters while decoding. */);
8010 Vstandard_translation_table_for_decode
= Qnil
;
8012 DEFVAR_LISP ("standard-translation-table-for-encode",
8013 &Vstandard_translation_table_for_encode
,
8014 doc
: /* Table for translating characters while encoding. */);
8015 Vstandard_translation_table_for_encode
= Qnil
;
8017 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
8018 doc
: /* Alist of charsets vs revision numbers.
8019 While encoding, if a charset (car part of an element) is found,
8020 designate it with the escape sequence identifying revision (cdr part of the element). */);
8021 Vcharset_revision_alist
= Qnil
;
8023 DEFVAR_LISP ("default-process-coding-system",
8024 &Vdefault_process_coding_system
,
8025 doc
: /* Cons of coding systems used for process I/O by default.
8026 The car part is used for decoding a process output,
8027 the cdr part is used for encoding a text to be sent to a process. */);
8028 Vdefault_process_coding_system
= Qnil
;
8030 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
8031 doc
: /* Table of extra Latin codes in the range 128..159 (inclusive).
8032 This is a vector of length 256.
8033 If Nth element is non-nil, the existence of code N in a file
8034 \(or output of subprocess) doesn't prevent it to be detected as
8035 a coding system of ISO 2022 variant which has a flag
8036 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
8037 or reading output of a subprocess.
8038 Only 128th through 159th elements has a meaning. */);
8039 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
8041 DEFVAR_LISP ("select-safe-coding-system-function",
8042 &Vselect_safe_coding_system_function
,
8043 doc
: /* Function to call to select safe coding system for encoding a text.
8045 If set, this function is called to force a user to select a proper
8046 coding system which can encode the text in the case that a default
8047 coding system used in each operation can't encode the text.
8049 The default value is `select-safe-coding-system' (which see). */);
8050 Vselect_safe_coding_system_function
= Qnil
;
8052 DEFVAR_BOOL ("coding-system-require-warning",
8053 &coding_system_require_warning
,
8054 doc
: /* Internal use only.
8055 If non-nil, on writing a file, `select-safe-coding-system-function' is
8056 called even if `coding-system-for-write' is non-nil. The command
8057 `universal-coding-system-argument' binds this variable to t temporarily. */);
8058 coding_system_require_warning
= 0;
8061 DEFVAR_BOOL ("inhibit-iso-escape-detection",
8062 &inhibit_iso_escape_detection
,
8063 doc
: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
8065 By default, on reading a file, Emacs tries to detect how the text is
8066 encoded. This code detection is sensitive to escape sequences. If
8067 the sequence is valid as ISO2022, the code is determined as one of
8068 the ISO2022 encodings, and the file is decoded by the corresponding
8069 coding system (e.g. `iso-2022-7bit').
8071 However, there may be a case that you want to read escape sequences in
8072 a file as is. In such a case, you can set this variable to non-nil.
8073 Then, as the code detection ignores any escape sequences, no file is
8074 detected as encoded in some ISO2022 encoding. The result is that all
8075 escape sequences become visible in a buffer.
8077 The default value is nil, and it is strongly recommended not to change
8078 it. That is because many Emacs Lisp source files that contain
8079 non-ASCII characters are encoded by the coding system `iso-2022-7bit'
8080 in Emacs's distribution, and they won't be decoded correctly on
8081 reading if you suppress escape sequence detection.
8083 The other way to read escape sequences in a file without decoding is
8084 to explicitly specify some coding system that doesn't use ISO2022's
8085 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
8086 inhibit_iso_escape_detection
= 0;
8088 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input
,
8089 doc
: /* Char table for translating self-inserting characters.
8090 This is applied to the result of input methods, not their input. See also
8091 `keyboard-translate-table'. */);
8092 Vtranslation_table_for_input
= Qnil
;
8096 emacs_strerror (error_number
)
8101 synchronize_system_messages_locale ();
8102 str
= strerror (error_number
);
8104 if (! NILP (Vlocale_coding_system
))
8106 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
8107 Vlocale_coding_system
,
8109 str
= (char *) SDATA (dec
);
8117 /* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
8118 (do not change this comment) */