1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 2001, 2002, 2003, 2004, 2005,
3 2006, 2007 Free Software Foundation, Inc.
4 Copyright (C) 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004,
6 National Institute of Advanced Industrial Science and Technology (AIST)
7 Registration Number H14PRO021
9 This file is part of GNU Emacs.
11 GNU Emacs is free software; you can redistribute it and/or modify
12 it under the terms of the GNU General Public License as published by
13 the Free Software Foundation; either version 2, or (at your option)
16 GNU Emacs is distributed in the hope that it will be useful,
17 but WITHOUT ANY WARRANTY; without even the implied warranty of
18 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
19 GNU General Public License for more details.
21 You should have received a copy of the GNU General Public License
22 along with GNU Emacs; see the file COPYING. If not, write to
23 the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
24 Boston, MA 02110-1301, USA. */
26 /*** TABLE OF CONTENTS ***
30 2. Emacs' internal format (emacs-mule) handlers
32 4. Shift-JIS and BIG5 handlers
34 6. End-of-line handlers
35 7. C library functions
36 8. Emacs Lisp library functions
41 /*** 0. General comments ***/
44 /*** GENERAL NOTE on CODING SYSTEMS ***
46 A coding system is an encoding mechanism for one or more character
47 sets. Here's a list of coding systems which Emacs can handle. When
48 we say "decode", it means converting some other coding system to
49 Emacs' internal format (emacs-mule), and when we say "encode",
50 it means converting the coding system emacs-mule to some other
53 0. Emacs' internal format (emacs-mule)
55 Emacs itself holds a multi-lingual character in buffers and strings
56 in a special format. Details are described in section 2.
60 The most famous coding system for multiple character sets. X's
61 Compound Text, various EUCs (Extended Unix Code), and coding
62 systems used in Internet communication such as ISO-2022-JP are
63 all variants of ISO2022. Details are described in section 3.
65 2. SJIS (or Shift-JIS or MS-Kanji-Code)
67 A coding system to encode character sets: ASCII, JISX0201, and
68 JISX0208. Widely used for PC's in Japan. Details are described in
73 A coding system to encode the character sets ASCII and Big5. Widely
74 used for Chinese (mainly in Taiwan and Hong Kong). Details are
75 described in section 4. In this file, when we write "BIG5"
76 (all uppercase), we mean the coding system, and when we write
77 "Big5" (capitalized), we mean the character set.
81 A coding system for text containing random 8-bit code. Emacs does
82 no code conversion on such text except for end-of-line format.
86 If a user wants to read/write text encoded in a coding system not
87 listed above, he can supply a decoder and an encoder for it as CCL
88 (Code Conversion Language) programs. Emacs executes the CCL program
89 while reading/writing.
91 Emacs represents a coding system by a Lisp symbol that has a property
92 `coding-system'. But, before actually using the coding system, the
93 information about it is set in a structure of type `struct
94 coding_system' for rapid processing. See section 6 for more details.
98 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
100 How end-of-line of text is encoded depends on the operating system.
101 For instance, Unix's format is just one byte of `line-feed' code,
102 whereas DOS's format is two-byte sequence of `carriage-return' and
103 `line-feed' codes. MacOS's format is usually one byte of
106 Since text character encoding and end-of-line encoding are
107 independent, any coding system described above can have any
108 end-of-line format. So Emacs has information about end-of-line
109 format in each coding-system. See section 6 for more details.
113 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
115 These functions check if a text between SRC and SRC_END is encoded
116 in the coding system category XXX. Each returns an integer value in
117 which appropriate flag bits for the category XXX are set. The flag
118 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
119 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
120 of the range 0x80..0x9F are in multibyte form. */
123 detect_coding_emacs_mule (src
, src_end
, multibytep
)
124 unsigned char *src
, *src_end
;
131 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
133 These functions decode SRC_BYTES length of unibyte text at SOURCE
134 encoded in CODING to Emacs' internal format. The resulting
135 multibyte text goes to a place pointed to by DESTINATION, the length
136 of which should not exceed DST_BYTES.
138 These functions set the information about original and decoded texts
139 in the members `produced', `produced_char', `consumed', and
140 `consumed_char' of the structure *CODING. They also set the member
141 `result' to one of CODING_FINISH_XXX indicating how the decoding
144 DST_BYTES zero means that the source area and destination area are
145 overlapped, which means that we can produce a decoded text until it
146 reaches the head of the not-yet-decoded source text.
148 Below is a template for these functions. */
151 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
152 struct coding_system
*coding
;
153 const unsigned char *source
;
154 unsigned char *destination
;
155 int src_bytes
, dst_bytes
;
161 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
163 These functions encode SRC_BYTES length text at SOURCE from Emacs'
164 internal multibyte format to CODING. The resulting unibyte text
165 goes to a place pointed to by DESTINATION, the length of which
166 should not exceed DST_BYTES.
168 These functions set the information about original and encoded texts
169 in the members `produced', `produced_char', `consumed', and
170 `consumed_char' of the structure *CODING. They also set the member
171 `result' to one of CODING_FINISH_XXX indicating how the encoding
174 DST_BYTES zero means that the source area and destination area are
175 overlapped, which means that we can produce encoded text until it
176 reaches at the head of the not-yet-encoded source text.
178 Below is a template for these functions. */
181 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
182 struct coding_system
*coding
;
183 unsigned char *source
, *destination
;
184 int src_bytes
, dst_bytes
;
190 /*** COMMONLY USED MACROS ***/
192 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
193 get one, two, and three bytes from the source text respectively.
194 If there are not enough bytes in the source, they jump to
195 `label_end_of_loop'. The caller should set variables `coding',
196 `src' and `src_end' to appropriate pointer in advance. These
197 macros are called from decoding routines `decode_coding_XXX', thus
198 it is assumed that the source text is unibyte. */
200 #define ONE_MORE_BYTE(c1) \
202 if (src >= src_end) \
204 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
205 goto label_end_of_loop; \
210 #define TWO_MORE_BYTES(c1, c2) \
212 if (src + 1 >= src_end) \
214 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
215 goto label_end_of_loop; \
222 /* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
223 form if MULTIBYTEP is nonzero. In addition, if SRC is not less
224 than SRC_END, return with RET. */
226 #define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep, ret) \
228 if (src >= src_end) \
230 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
234 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
235 c1 = *src++ - 0x20; \
238 /* Set C to the next character at the source text pointed by `src'.
239 If there are not enough characters in the source, jump to
240 `label_end_of_loop'. The caller should set variables `coding'
241 `src', `src_end', and `translation_table' to appropriate pointers
242 in advance. This macro is used in encoding routines
243 `encode_coding_XXX', thus it assumes that the source text is in
244 multibyte form except for 8-bit characters. 8-bit characters are
245 in multibyte form if coding->src_multibyte is nonzero, else they
246 are represented by a single byte. */
248 #define ONE_MORE_CHAR(c) \
250 int len = src_end - src; \
254 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
255 goto label_end_of_loop; \
257 if (coding->src_multibyte \
258 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
259 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
261 c = *src, bytes = 1; \
262 if (!NILP (translation_table)) \
263 c = translate_char (translation_table, c, -1, 0, 0); \
268 /* Produce a multibyte form of character C to `dst'. Jump to
269 `label_end_of_loop' if there's not enough space at `dst'.
271 If we are now in the middle of a composition sequence, the decoded
272 character may be ALTCHAR (for the current composition). In that
273 case, the character goes to coding->cmp_data->data instead of
276 This macro is used in decoding routines. */
278 #define EMIT_CHAR(c) \
280 if (! COMPOSING_P (coding) \
281 || coding->composing == COMPOSITION_RELATIVE \
282 || coding->composing == COMPOSITION_WITH_RULE) \
284 int bytes = CHAR_BYTES (c); \
285 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
287 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
288 goto label_end_of_loop; \
290 dst += CHAR_STRING (c, dst); \
291 coding->produced_char++; \
294 if (COMPOSING_P (coding) \
295 && coding->composing != COMPOSITION_RELATIVE) \
297 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
298 coding->composition_rule_follows \
299 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
304 #define EMIT_ONE_BYTE(c) \
306 if (dst >= (dst_bytes ? dst_end : src)) \
308 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
309 goto label_end_of_loop; \
314 #define EMIT_TWO_BYTES(c1, c2) \
316 if (dst + 2 > (dst_bytes ? dst_end : src)) \
318 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
319 goto label_end_of_loop; \
321 *dst++ = c1, *dst++ = c2; \
324 #define EMIT_BYTES(from, to) \
326 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
328 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
329 goto label_end_of_loop; \
336 /*** 1. Preamble ***/
349 #include "composite.h"
353 #include "intervals.h"
355 #include "termhooks.h"
357 #else /* not emacs */
361 #endif /* not emacs */
363 Lisp_Object Qcoding_system
, Qeol_type
;
364 Lisp_Object Qbuffer_file_coding_system
;
365 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
366 Lisp_Object Qno_conversion
, Qundecided
;
367 Lisp_Object Qcoding_system_history
;
368 Lisp_Object Qsafe_chars
;
369 Lisp_Object Qvalid_codes
;
370 Lisp_Object Qascii_incompatible
;
372 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
373 Lisp_Object Qcall_process
, Qcall_process_region
;
374 Lisp_Object Qstart_process
, Qopen_network_stream
;
375 Lisp_Object Qtarget_idx
;
377 /* If a symbol has this property, evaluate the value to define the
378 symbol as a coding system. */
379 Lisp_Object Qcoding_system_define_form
;
381 Lisp_Object Vselect_safe_coding_system_function
;
383 int coding_system_require_warning
;
385 /* Mnemonic string for each format of end-of-line. */
386 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
387 /* Mnemonic string to indicate format of end-of-line is not yet
389 Lisp_Object eol_mnemonic_undecided
;
391 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
392 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac.
393 This has an effect only for external encoding (i.e. for output to
394 file and process), not for in-buffer or Lisp string encoding. */
399 /* Information about which coding system is safe for which chars.
400 The value has the form (GENERIC-LIST . NON-GENERIC-ALIST).
402 GENERIC-LIST is a list of generic coding systems which can encode
405 NON-GENERIC-ALIST is an alist of non generic coding systems vs the
406 corresponding char table that contains safe chars. */
407 Lisp_Object Vcoding_system_safe_chars
;
409 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
411 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
413 /* Coding system emacs-mule and raw-text are for converting only
414 end-of-line format. */
415 Lisp_Object Qemacs_mule
, Qraw_text
;
419 /* Coding-systems are handed between Emacs Lisp programs and C internal
420 routines by the following three variables. */
421 /* Coding-system for reading files and receiving data from process. */
422 Lisp_Object Vcoding_system_for_read
;
423 /* Coding-system for writing files and sending data to process. */
424 Lisp_Object Vcoding_system_for_write
;
425 /* Coding-system actually used in the latest I/O. */
426 Lisp_Object Vlast_coding_system_used
;
428 /* A vector of length 256 which contains information about special
429 Latin codes (especially for dealing with Microsoft codes). */
430 Lisp_Object Vlatin_extra_code_table
;
432 /* Flag to inhibit code conversion of end-of-line format. */
433 int inhibit_eol_conversion
;
435 /* Flag to inhibit ISO2022 escape sequence detection. */
436 int inhibit_iso_escape_detection
;
438 /* Flag to make buffer-file-coding-system inherit from process-coding. */
439 int inherit_process_coding_system
;
441 /* Coding system to be used to encode text for terminal display when
442 terminal coding system is nil. */
443 struct coding_system safe_terminal_coding
;
445 /* Default coding system to be used to write a file. */
446 struct coding_system default_buffer_file_coding
;
448 Lisp_Object Vfile_coding_system_alist
;
449 Lisp_Object Vprocess_coding_system_alist
;
450 Lisp_Object Vnetwork_coding_system_alist
;
452 Lisp_Object Vlocale_coding_system
;
456 Lisp_Object Qcoding_category
, Qcoding_category_index
;
458 /* List of symbols `coding-category-xxx' ordered by priority. */
459 Lisp_Object Vcoding_category_list
;
461 /* Table of coding categories (Lisp symbols). */
462 Lisp_Object Vcoding_category_table
;
464 /* Table of names of symbol for each coding-category. */
465 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
466 "coding-category-emacs-mule",
467 "coding-category-sjis",
468 "coding-category-iso-7",
469 "coding-category-iso-7-tight",
470 "coding-category-iso-8-1",
471 "coding-category-iso-8-2",
472 "coding-category-iso-7-else",
473 "coding-category-iso-8-else",
474 "coding-category-ccl",
475 "coding-category-big5",
476 "coding-category-utf-8",
477 "coding-category-utf-16-be",
478 "coding-category-utf-16-le",
479 "coding-category-raw-text",
480 "coding-category-binary"
483 /* Table of pointers to coding systems corresponding to each coding
485 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
487 /* Table of coding category masks. Nth element is a mask for a coding
488 category of which priority is Nth. */
490 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
492 /* Flag to tell if we look up translation table on character code
494 Lisp_Object Venable_character_translation
;
495 /* Standard translation table to look up on decoding (reading). */
496 Lisp_Object Vstandard_translation_table_for_decode
;
497 /* Standard translation table to look up on encoding (writing). */
498 Lisp_Object Vstandard_translation_table_for_encode
;
500 Lisp_Object Qtranslation_table
;
501 Lisp_Object Qtranslation_table_id
;
502 Lisp_Object Qtranslation_table_for_decode
;
503 Lisp_Object Qtranslation_table_for_encode
;
505 /* Alist of charsets vs revision number. */
506 Lisp_Object Vcharset_revision_alist
;
508 /* Default coding systems used for process I/O. */
509 Lisp_Object Vdefault_process_coding_system
;
511 /* Char table for translating Quail and self-inserting input. */
512 Lisp_Object Vtranslation_table_for_input
;
514 /* Global flag to tell that we can't call post-read-conversion and
515 pre-write-conversion functions. Usually the value is zero, but it
516 is set to 1 temporarily while such functions are running. This is
517 to avoid infinite recursive call. */
518 static int inhibit_pre_post_conversion
;
520 Lisp_Object Qchar_coding_system
;
522 /* Return `safe-chars' property of CODING_SYSTEM (symbol). Don't check
526 coding_safe_chars (coding_system
)
527 Lisp_Object coding_system
;
529 Lisp_Object coding_spec
, plist
, safe_chars
;
531 coding_spec
= Fget (coding_system
, Qcoding_system
);
532 plist
= XVECTOR (coding_spec
)->contents
[3];
533 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
534 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
537 #define CODING_SAFE_CHAR_P(safe_chars, c) \
538 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
541 /*** 2. Emacs internal format (emacs-mule) handlers ***/
543 /* Emacs' internal format for representation of multiple character
544 sets is a kind of multi-byte encoding, i.e. characters are
545 represented by variable-length sequences of one-byte codes.
547 ASCII characters and control characters (e.g. `tab', `newline') are
548 represented by one-byte sequences which are their ASCII codes, in
549 the range 0x00 through 0x7F.
551 8-bit characters of the range 0x80..0x9F are represented by
552 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
555 8-bit characters of the range 0xA0..0xFF are represented by
556 one-byte sequences which are their 8-bit code.
558 The other characters are represented by a sequence of `base
559 leading-code', optional `extended leading-code', and one or two
560 `position-code's. The length of the sequence is determined by the
561 base leading-code. Leading-code takes the range 0x81 through 0x9D,
562 whereas extended leading-code and position-code take the range 0xA0
563 through 0xFF. See `charset.h' for more details about leading-code
566 --- CODE RANGE of Emacs' internal format ---
570 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
571 eight-bit-graphic 0xA0..0xBF
572 ELSE 0x81..0x9D + [0xA0..0xFF]+
573 ---------------------------------------------
575 As this is the internal character representation, the format is
576 usually not used externally (i.e. in a file or in a data sent to a
577 process). But, it is possible to have a text externally in this
578 format (i.e. by encoding by the coding system `emacs-mule').
580 In that case, a sequence of one-byte codes has a slightly different
583 Firstly, all characters in eight-bit-control are represented by
584 one-byte sequences which are their 8-bit code.
586 Next, character composition data are represented by the byte
587 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
589 METHOD is 0xF0 plus one of composition method (enum
592 BYTES is 0xA0 plus the byte length of these composition data,
594 CHARS is 0xA0 plus the number of characters composed by these
597 COMPONENTs are characters of multibyte form or composition
598 rules encoded by two-byte of ASCII codes.
600 In addition, for backward compatibility, the following formats are
601 also recognized as composition data on decoding.
604 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
607 MSEQ is a multibyte form but in these special format:
608 ASCII: 0xA0 ASCII_CODE+0x80,
609 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
610 RULE is a one byte code of the range 0xA0..0xF0 that
611 represents a composition rule.
614 enum emacs_code_class_type emacs_code_class
[256];
616 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
617 Check if a text is encoded in Emacs' internal format. If it is,
618 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
621 detect_coding_emacs_mule (src
, src_end
, multibytep
)
622 unsigned char *src
, *src_end
;
627 /* Dummy for ONE_MORE_BYTE. */
628 struct coding_system dummy_coding
;
629 struct coding_system
*coding
= &dummy_coding
;
633 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
,
634 CODING_CATEGORY_MASK_EMACS_MULE
);
641 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
650 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
653 else if (c
>= 0x80 && c
< 0xA0)
656 /* Old leading code for a composite character. */
660 unsigned char *src_base
= src
- 1;
663 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
666 src
= src_base
+ bytes
;
673 /* Record the starting position START and METHOD of one composition. */
675 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
677 struct composition_data *cmp_data = coding->cmp_data; \
678 int *data = cmp_data->data + cmp_data->used; \
679 coding->cmp_data_start = cmp_data->used; \
681 data[1] = cmp_data->char_offset + start; \
682 data[3] = (int) method; \
683 cmp_data->used += 4; \
686 /* Record the ending position END of the current composition. */
688 #define CODING_ADD_COMPOSITION_END(coding, end) \
690 struct composition_data *cmp_data = coding->cmp_data; \
691 int *data = cmp_data->data + coding->cmp_data_start; \
692 data[0] = cmp_data->used - coding->cmp_data_start; \
693 data[2] = cmp_data->char_offset + end; \
696 /* Record one COMPONENT (alternate character or composition rule). */
698 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
700 coding->cmp_data->data[coding->cmp_data->used++] = component; \
701 if (coding->cmp_data->used - coding->cmp_data_start \
702 == COMPOSITION_DATA_MAX_BUNCH_LENGTH) \
704 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
705 coding->composing = COMPOSITION_NO; \
710 /* Get one byte from a data pointed by SRC and increment SRC. If SRC
711 is not less than SRC_END, return -1 without incrementing Src. */
713 #define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
716 /* Decode a character represented as a component of composition
717 sequence of Emacs 20 style at SRC. Set C to that character, store
718 its multibyte form sequence at P, and set P to the end of that
719 sequence. If no valid character is found, set C to -1. */
721 #define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
725 c = SAFE_ONE_MORE_BYTE (); \
728 if (CHAR_HEAD_P (c)) \
730 else if (c == 0xA0) \
732 c = SAFE_ONE_MORE_BYTE (); \
741 else if (BASE_LEADING_CODE_P (c - 0x20)) \
743 unsigned char *p0 = p; \
747 bytes = BYTES_BY_CHAR_HEAD (c); \
750 c = SAFE_ONE_MORE_BYTE (); \
755 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes) \
756 || (coding->flags /* We are recovering a file. */ \
757 && p0[0] == LEADING_CODE_8_BIT_CONTROL \
758 && ! CHAR_HEAD_P (p0[1]))) \
759 c = STRING_CHAR (p0, bytes); \
768 /* Decode a composition rule represented as a component of composition
769 sequence of Emacs 20 style at SRC. Set C to the rule. If not
770 valid rule is found, set C to -1. */
772 #define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
774 c = SAFE_ONE_MORE_BYTE (); \
776 if (c < 0 || c >= 81) \
780 gref = c / 9, nref = c % 9; \
781 c = COMPOSITION_ENCODE_RULE (gref, nref); \
786 /* Decode composition sequence encoded by `emacs-mule' at the source
787 pointed by SRC. SRC_END is the end of source. Store information
788 of the composition in CODING->cmp_data.
790 For backward compatibility, decode also a composition sequence of
791 Emacs 20 style. In that case, the composition sequence contains
792 characters that should be extracted into a buffer or string. Store
793 those characters at *DESTINATION in multibyte form.
795 If we encounter an invalid byte sequence, return 0.
796 If we encounter an insufficient source or destination, or
797 insufficient space in CODING->cmp_data, return 1.
798 Otherwise, return consumed bytes in the source.
802 decode_composition_emacs_mule (coding
, src
, src_end
,
803 destination
, dst_end
, dst_bytes
)
804 struct coding_system
*coding
;
805 const unsigned char *src
, *src_end
;
806 unsigned char **destination
, *dst_end
;
809 unsigned char *dst
= *destination
;
810 int method
, data_len
, nchars
;
811 const unsigned char *src_base
= src
++;
812 /* Store components of composition. */
813 int component
[COMPOSITION_DATA_MAX_BUNCH_LENGTH
];
815 /* Store multibyte form of characters to be composed. This is for
816 Emacs 20 style composition sequence. */
817 unsigned char buf
[MAX_COMPOSITION_COMPONENTS
* MAX_MULTIBYTE_LENGTH
];
818 unsigned char *bufp
= buf
;
819 int c
, i
, gref
, nref
;
821 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
822 >= COMPOSITION_DATA_SIZE
)
824 coding
->result
= CODING_FINISH_INSUFFICIENT_CMP
;
829 if (c
- 0xF0 >= COMPOSITION_RELATIVE
830 && c
- 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS
)
835 with_rule
= (method
== COMPOSITION_WITH_RULE
836 || method
== COMPOSITION_WITH_RULE_ALTCHARS
);
840 || src_base
+ data_len
> src_end
)
846 for (ncomponent
= 0; src
< src_base
+ data_len
; ncomponent
++)
848 /* If it is longer than this, it can't be valid. */
849 if (ncomponent
>= COMPOSITION_DATA_MAX_BUNCH_LENGTH
)
852 if (ncomponent
% 2 && with_rule
)
854 ONE_MORE_BYTE (gref
);
856 ONE_MORE_BYTE (nref
);
858 c
= COMPOSITION_ENCODE_RULE (gref
, nref
);
863 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
864 || (coding
->flags
/* We are recovering a file. */
865 && src
[0] == LEADING_CODE_8_BIT_CONTROL
866 && ! CHAR_HEAD_P (src
[1])))
867 c
= STRING_CHAR (src
, bytes
);
872 component
[ncomponent
] = c
;
877 /* This may be an old Emacs 20 style format. See the comment at
878 the section 2 of this file. */
879 while (src
< src_end
&& !CHAR_HEAD_P (*src
)) src
++;
881 && !(coding
->mode
& CODING_MODE_LAST_BLOCK
))
882 goto label_end_of_loop
;
888 method
= COMPOSITION_RELATIVE
;
889 for (ncomponent
= 0; ncomponent
< MAX_COMPOSITION_COMPONENTS
;)
891 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
894 component
[ncomponent
++] = c
;
902 method
= COMPOSITION_WITH_RULE
;
904 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
909 ncomponent
< MAX_COMPOSITION_COMPONENTS
* 2 - 1;)
911 DECODE_EMACS_MULE_COMPOSITION_RULE (c
);
914 component
[ncomponent
++] = c
;
915 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
918 component
[ncomponent
++] = c
;
922 nchars
= (ncomponent
+ 1) / 2;
930 if (buf
== bufp
|| dst
+ (bufp
- buf
) <= (dst_bytes
? dst_end
: src
))
932 CODING_ADD_COMPOSITION_START (coding
, coding
->produced_char
, method
);
933 for (i
= 0; i
< ncomponent
; i
++)
934 CODING_ADD_COMPOSITION_COMPONENT (coding
, component
[i
]);
935 CODING_ADD_COMPOSITION_END (coding
, coding
->produced_char
+ nchars
);
938 unsigned char *p
= buf
;
939 EMIT_BYTES (p
, bufp
);
940 *destination
+= bufp
- buf
;
941 coding
->produced_char
+= nchars
;
943 return (src
- src_base
);
949 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
952 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
953 struct coding_system
*coding
;
954 const unsigned char *source
;
955 unsigned char *destination
;
956 int src_bytes
, dst_bytes
;
958 const unsigned char *src
= source
;
959 const unsigned char *src_end
= source
+ src_bytes
;
960 unsigned char *dst
= destination
;
961 unsigned char *dst_end
= destination
+ dst_bytes
;
962 /* SRC_BASE remembers the start position in source in each loop.
963 The loop will be exited when there's not enough source code, or
964 when there's not enough destination area to produce a
966 const unsigned char *src_base
;
968 coding
->produced_char
= 0;
969 while ((src_base
= src
) < src_end
)
971 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
];
972 const unsigned char *p
;
979 if (coding
->eol_type
== CODING_EOL_CR
)
981 else if (coding
->eol_type
== CODING_EOL_CRLF
)
991 coding
->produced_char
++;
994 else if (*src
== '\n')
996 if ((coding
->eol_type
== CODING_EOL_CR
997 || coding
->eol_type
== CODING_EOL_CRLF
)
998 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1000 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1001 goto label_end_of_loop
;
1004 coding
->produced_char
++;
1007 else if (*src
== 0x80 && coding
->cmp_data
)
1009 /* Start of composition data. */
1010 int consumed
= decode_composition_emacs_mule (coding
, src
, src_end
,
1014 goto label_end_of_loop
;
1015 else if (consumed
> 0)
1020 bytes
= CHAR_STRING (*src
, tmp
);
1024 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
1025 || (coding
->flags
/* We are recovering a file. */
1026 && src
[0] == LEADING_CODE_8_BIT_CONTROL
1027 && ! CHAR_HEAD_P (src
[1])))
1036 bytes
= BYTES_BY_CHAR_HEAD (*src
);
1038 for (i
= 1; i
< bytes
; i
++)
1041 if (CHAR_HEAD_P (c
))
1046 bytes
= CHAR_STRING (*src_base
, tmp
);
1055 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
1057 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
1060 while (bytes
--) *dst
++ = *p
++;
1061 coding
->produced_char
++;
1064 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1065 coding
->produced
= dst
- destination
;
1069 /* Encode composition data stored at DATA into a special byte sequence
1070 starting by 0x80. Update CODING->cmp_data_start and maybe
1071 CODING->cmp_data for the next call. */
1073 #define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1075 unsigned char buf[1024], *p0 = buf, *p; \
1076 int len = data[0]; \
1080 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1081 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1083 if (data[3] == COMPOSITION_WITH_RULE \
1084 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1086 p += CHAR_STRING (data[4], p); \
1087 for (i = 5; i < len; i += 2) \
1090 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1091 *p++ = 0x20 + gref; \
1092 *p++ = 0x20 + nref; \
1093 p += CHAR_STRING (data[i + 1], p); \
1098 for (i = 4; i < len; i++) \
1099 p += CHAR_STRING (data[i], p); \
1101 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1103 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1105 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1106 goto label_end_of_loop; \
1110 coding->cmp_data_start += data[0]; \
1111 if (coding->cmp_data_start == coding->cmp_data->used \
1112 && coding->cmp_data->next) \
1114 coding->cmp_data = coding->cmp_data->next; \
1115 coding->cmp_data_start = 0; \
1120 static void encode_eol
P_ ((struct coding_system
*, const unsigned char *,
1121 unsigned char *, int, int));
1124 encode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
1125 struct coding_system
*coding
;
1126 const unsigned char *source
;
1127 unsigned char *destination
;
1128 int src_bytes
, dst_bytes
;
1130 const unsigned char *src
= source
;
1131 const unsigned char *src_end
= source
+ src_bytes
;
1132 unsigned char *dst
= destination
;
1133 unsigned char *dst_end
= destination
+ dst_bytes
;
1134 const unsigned char *src_base
;
1139 Lisp_Object translation_table
;
1141 translation_table
= Qnil
;
1143 /* Optimization for the case that there's no composition. */
1144 if (!coding
->cmp_data
|| coding
->cmp_data
->used
== 0)
1146 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
1150 char_offset
= coding
->cmp_data
->char_offset
;
1151 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1156 /* If SRC starts a composition, encode the information about the
1157 composition in advance. */
1158 if (coding
->cmp_data_start
< coding
->cmp_data
->used
1159 && char_offset
+ coding
->consumed_char
== data
[1])
1161 ENCODE_COMPOSITION_EMACS_MULE (coding
, data
);
1162 char_offset
= coding
->cmp_data
->char_offset
;
1163 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1167 if (c
== '\n' && (coding
->eol_type
== CODING_EOL_CRLF
1168 || coding
->eol_type
== CODING_EOL_CR
))
1170 if (coding
->eol_type
== CODING_EOL_CRLF
)
1171 EMIT_TWO_BYTES ('\r', c
);
1173 EMIT_ONE_BYTE ('\r');
1175 else if (SINGLE_BYTE_CHAR_P (c
))
1177 if (coding
->flags
&& ! ASCII_BYTE_P (c
))
1179 /* As we are auto saving, retain the multibyte form for
1181 unsigned char buf
[MAX_MULTIBYTE_LENGTH
];
1182 int bytes
= CHAR_STRING (c
, buf
);
1185 EMIT_ONE_BYTE (buf
[0]);
1187 EMIT_TWO_BYTES (buf
[0], buf
[1]);
1193 EMIT_BYTES (src_base
, src
);
1194 coding
->consumed_char
++;
1197 coding
->consumed
= src_base
- source
;
1198 coding
->produced
= coding
->produced_char
= dst
- destination
;
1203 /*** 3. ISO2022 handlers ***/
1205 /* The following note describes the coding system ISO2022 briefly.
1206 Since the intention of this note is to help understand the
1207 functions in this file, some parts are NOT ACCURATE or are OVERLY
1208 SIMPLIFIED. For thorough understanding, please refer to the
1209 original document of ISO2022. This is equivalent to the standard
1210 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1212 ISO2022 provides many mechanisms to encode several character sets
1213 in 7-bit and 8-bit environments. For 7-bit environments, all text
1214 is encoded using bytes less than 128. This may make the encoded
1215 text a little bit longer, but the text passes more easily through
1216 several types of gateway, some of which strip off the MSB (Most
1219 There are two kinds of character sets: control character sets and
1220 graphic character sets. The former contain control characters such
1221 as `newline' and `escape' to provide control functions (control
1222 functions are also provided by escape sequences). The latter
1223 contain graphic characters such as 'A' and '-'. Emacs recognizes
1224 two control character sets and many graphic character sets.
1226 Graphic character sets are classified into one of the following
1227 four classes, according to the number of bytes (DIMENSION) and
1228 number of characters in one dimension (CHARS) of the set:
1229 - DIMENSION1_CHARS94
1230 - DIMENSION1_CHARS96
1231 - DIMENSION2_CHARS94
1232 - DIMENSION2_CHARS96
1234 In addition, each character set is assigned an identification tag,
1235 unique for each set, called the "final character" (denoted as <F>
1236 hereafter). The <F> of each character set is decided by ECMA(*)
1237 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1238 (0x30..0x3F are for private use only).
1240 Note (*): ECMA = European Computer Manufacturers Association
1242 Here are examples of graphic character sets [NAME(<F>)]:
1243 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1244 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1245 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1246 o DIMENSION2_CHARS96 -- none for the moment
1248 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
1249 C0 [0x00..0x1F] -- control character plane 0
1250 GL [0x20..0x7F] -- graphic character plane 0
1251 C1 [0x80..0x9F] -- control character plane 1
1252 GR [0xA0..0xFF] -- graphic character plane 1
1254 A control character set is directly designated and invoked to C0 or
1255 C1 by an escape sequence. The most common case is that:
1256 - ISO646's control character set is designated/invoked to C0, and
1257 - ISO6429's control character set is designated/invoked to C1,
1258 and usually these designations/invocations are omitted in encoded
1259 text. In a 7-bit environment, only C0 can be used, and a control
1260 character for C1 is encoded by an appropriate escape sequence to
1261 fit into the environment. All control characters for C1 are
1262 defined to have corresponding escape sequences.
1264 A graphic character set is at first designated to one of four
1265 graphic registers (G0 through G3), then these graphic registers are
1266 invoked to GL or GR. These designations and invocations can be
1267 done independently. The most common case is that G0 is invoked to
1268 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1269 these invocations and designations are omitted in encoded text.
1270 In a 7-bit environment, only GL can be used.
1272 When a graphic character set of CHARS94 is invoked to GL, codes
1273 0x20 and 0x7F of the GL area work as control characters SPACE and
1274 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1277 There are two ways of invocation: locking-shift and single-shift.
1278 With locking-shift, the invocation lasts until the next different
1279 invocation, whereas with single-shift, the invocation affects the
1280 following character only and doesn't affect the locking-shift
1281 state. Invocations are done by the following control characters or
1284 ----------------------------------------------------------------------
1285 abbrev function cntrl escape seq description
1286 ----------------------------------------------------------------------
1287 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1288 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1289 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1290 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1291 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1292 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1293 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1294 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1295 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
1296 ----------------------------------------------------------------------
1297 (*) These are not used by any known coding system.
1299 Control characters for these functions are defined by macros
1300 ISO_CODE_XXX in `coding.h'.
1302 Designations are done by the following escape sequences:
1303 ----------------------------------------------------------------------
1304 escape sequence description
1305 ----------------------------------------------------------------------
1306 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1307 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1308 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1309 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1310 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1311 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1312 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1313 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1314 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1315 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1316 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1317 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1318 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1319 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1320 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1321 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1322 ----------------------------------------------------------------------
1324 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
1325 of dimension 1, chars 94, and final character <F>, etc...
1327 Note (*): Although these designations are not allowed in ISO2022,
1328 Emacs accepts them on decoding, and produces them on encoding
1329 CHARS96 character sets in a coding system which is characterized as
1330 7-bit environment, non-locking-shift, and non-single-shift.
1332 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1333 '(' can be omitted. We refer to this as "short-form" hereafter.
1335 Now you may notice that there are a lot of ways of encoding the
1336 same multilingual text in ISO2022. Actually, there exist many
1337 coding systems such as Compound Text (used in X11's inter client
1338 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1339 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
1340 localized platforms), and all of these are variants of ISO2022.
1342 In addition to the above, Emacs handles two more kinds of escape
1343 sequences: ISO6429's direction specification and Emacs' private
1344 sequence for specifying character composition.
1346 ISO6429's direction specification takes the following form:
1347 o CSI ']' -- end of the current direction
1348 o CSI '0' ']' -- end of the current direction
1349 o CSI '1' ']' -- start of left-to-right text
1350 o CSI '2' ']' -- start of right-to-left text
1351 The control character CSI (0x9B: control sequence introducer) is
1352 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1354 Character composition specification takes the following form:
1355 o ESC '0' -- start relative composition
1356 o ESC '1' -- end composition
1357 o ESC '2' -- start rule-base composition (*)
1358 o ESC '3' -- start relative composition with alternate chars (**)
1359 o ESC '4' -- start rule-base composition with alternate chars (**)
1360 Since these are not standard escape sequences of any ISO standard,
1361 the use of them with these meanings is restricted to Emacs only.
1363 (*) This form is used only in Emacs 20.5 and older versions,
1364 but the newer versions can safely decode it.
1365 (**) This form is used only in Emacs 21.1 and newer versions,
1366 and the older versions can't decode it.
1368 Here's a list of example usages of these composition escape
1369 sequences (categorized by `enum composition_method').
1371 COMPOSITION_RELATIVE:
1372 ESC 0 CHAR [ CHAR ] ESC 1
1373 COMPOSITION_WITH_RULE:
1374 ESC 2 CHAR [ RULE CHAR ] ESC 1
1375 COMPOSITION_WITH_ALTCHARS:
1376 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
1377 COMPOSITION_WITH_RULE_ALTCHARS:
1378 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
1380 enum iso_code_class_type iso_code_class
[256];
1382 #define CHARSET_OK(idx, charset, c) \
1383 (coding_system_table[idx] \
1384 && (charset == CHARSET_ASCII \
1385 || (safe_chars = coding_safe_chars (coding_system_table[idx]->symbol), \
1386 CODING_SAFE_CHAR_P (safe_chars, c))) \
1387 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1389 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
1391 #define SHIFT_OUT_OK(idx) \
1392 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1394 #define COMPOSITION_OK(idx) \
1395 (coding_system_table[idx]->composing != COMPOSITION_DISABLED)
1397 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1398 Check if a text is encoded in ISO2022. If it is, return an
1399 integer in which appropriate flag bits any of:
1400 CODING_CATEGORY_MASK_ISO_7
1401 CODING_CATEGORY_MASK_ISO_7_TIGHT
1402 CODING_CATEGORY_MASK_ISO_8_1
1403 CODING_CATEGORY_MASK_ISO_8_2
1404 CODING_CATEGORY_MASK_ISO_7_ELSE
1405 CODING_CATEGORY_MASK_ISO_8_ELSE
1406 are set. If a code which should never appear in ISO2022 is found,
1410 detect_coding_iso2022 (src
, src_end
, multibytep
)
1411 unsigned char *src
, *src_end
;
1414 int mask
= CODING_CATEGORY_MASK_ISO
;
1416 int reg
[4], shift_out
= 0, single_shifting
= 0;
1418 /* Dummy for ONE_MORE_BYTE. */
1419 struct coding_system dummy_coding
;
1420 struct coding_system
*coding
= &dummy_coding
;
1421 Lisp_Object safe_chars
;
1423 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
1426 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, mask
& mask_found
);
1431 if (inhibit_iso_escape_detection
)
1433 single_shifting
= 0;
1434 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, mask
& mask_found
);
1435 if (c
>= '(' && c
<= '/')
1437 /* Designation sequence for a charset of dimension 1. */
1438 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
, mask
& mask_found
);
1439 if (c1
< ' ' || c1
>= 0x80
1440 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
1441 /* Invalid designation sequence. Just ignore. */
1443 reg
[(c
- '(') % 4] = charset
;
1447 /* Designation sequence for a charset of dimension 2. */
1448 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, mask
& mask_found
);
1449 if (c
>= '@' && c
<= 'B')
1450 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1451 reg
[0] = charset
= iso_charset_table
[1][0][c
];
1452 else if (c
>= '(' && c
<= '/')
1454 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
,
1456 if (c1
< ' ' || c1
>= 0x80
1457 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
1458 /* Invalid designation sequence. Just ignore. */
1460 reg
[(c
- '(') % 4] = charset
;
1463 /* Invalid designation sequence. Just ignore. */
1466 else if (c
== 'N' || c
== 'O')
1468 /* ESC <Fe> for SS2 or SS3. */
1469 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1472 else if (c
>= '0' && c
<= '4')
1474 /* ESC <Fp> for start/end composition. */
1475 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7
))
1476 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1478 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1479 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
))
1480 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1482 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1483 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_1
))
1484 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1486 mask
&= ~CODING_CATEGORY_MASK_ISO_8_1
;
1487 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_2
))
1488 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1490 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1491 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
))
1492 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1494 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1495 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
))
1496 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1498 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1502 /* Invalid escape sequence. Just ignore. */
1505 /* We found a valid designation sequence for CHARSET. */
1506 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
1507 c
= MAKE_CHAR (charset
, 0, 0);
1508 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
1509 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1511 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1512 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
1513 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1515 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1516 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
1517 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1519 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1520 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
1521 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1523 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1527 if (inhibit_iso_escape_detection
)
1529 single_shifting
= 0;
1532 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
1533 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
1535 /* Locking shift out. */
1536 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1537 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1542 if (inhibit_iso_escape_detection
)
1544 single_shifting
= 0;
1547 /* Locking shift in. */
1548 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1549 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1554 single_shifting
= 0;
1558 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1560 if (inhibit_iso_escape_detection
)
1562 if (c
!= ISO_CODE_CSI
)
1564 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1565 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1566 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1567 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1568 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1569 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1570 single_shifting
= 1;
1572 if (VECTORP (Vlatin_extra_code_table
)
1573 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1575 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1576 & CODING_FLAG_ISO_LATIN_EXTRA
)
1577 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1578 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1579 & CODING_FLAG_ISO_LATIN_EXTRA
)
1580 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1583 mask_found
|= newmask
;
1590 single_shifting
= 0;
1595 single_shifting
= 0;
1596 if (VECTORP (Vlatin_extra_code_table
)
1597 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1601 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1602 & CODING_FLAG_ISO_LATIN_EXTRA
)
1603 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1604 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1605 & CODING_FLAG_ISO_LATIN_EXTRA
)
1606 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1608 mask_found
|= newmask
;
1615 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1616 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1617 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1618 /* Check the length of succeeding codes of the range
1619 0xA0..0FF. If the byte length is odd, we exclude
1620 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1621 when we are not single shifting. */
1622 if (!single_shifting
1623 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1628 while (src
< src_end
)
1630 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
,
1637 if (i
& 1 && src
< src_end
)
1638 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1640 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1642 /* This means that we have read one extra byte. */
1649 return (mask
& mask_found
);
1652 /* Decode a character of which charset is CHARSET, the 1st position
1653 code is C1, the 2nd position code is C2, and return the decoded
1654 character code. If the variable `translation_table' is non-nil,
1655 returned the translated code. */
1657 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1658 (NILP (translation_table) \
1659 ? MAKE_CHAR (charset, c1, c2) \
1660 : translate_char (translation_table, -1, charset, c1, c2))
1662 /* Set designation state into CODING. */
1663 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1667 if (final_char < '0' || final_char >= 128) \
1668 goto label_invalid_code; \
1669 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1670 make_number (chars), \
1671 make_number (final_char)); \
1672 c = MAKE_CHAR (charset, 0, 0); \
1674 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1675 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1677 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1679 && charset == CHARSET_ASCII) \
1681 /* We should insert this designation sequence as is so \
1682 that it is surely written back to a file. */ \
1683 coding->spec.iso2022.last_invalid_designation_register = -1; \
1684 goto label_invalid_code; \
1686 coding->spec.iso2022.last_invalid_designation_register = -1; \
1687 if ((coding->mode & CODING_MODE_DIRECTION) \
1688 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1689 charset = CHARSET_REVERSE_CHARSET (charset); \
1690 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1694 coding->spec.iso2022.last_invalid_designation_register = reg; \
1695 goto label_invalid_code; \
1699 /* Allocate a memory block for storing information about compositions.
1700 The block is chained to the already allocated blocks. */
1703 coding_allocate_composition_data (coding
, char_offset
)
1704 struct coding_system
*coding
;
1707 struct composition_data
*cmp_data
1708 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1710 cmp_data
->char_offset
= char_offset
;
1712 cmp_data
->prev
= coding
->cmp_data
;
1713 cmp_data
->next
= NULL
;
1714 if (coding
->cmp_data
)
1715 coding
->cmp_data
->next
= cmp_data
;
1716 coding
->cmp_data
= cmp_data
;
1717 coding
->cmp_data_start
= 0;
1718 coding
->composing
= COMPOSITION_NO
;
1721 /* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1722 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1723 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1724 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1725 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1728 #define DECODE_COMPOSITION_START(c1) \
1730 if (coding->composing == COMPOSITION_DISABLED) \
1732 *dst++ = ISO_CODE_ESC; \
1733 *dst++ = c1 & 0x7f; \
1734 coding->produced_char += 2; \
1736 else if (!COMPOSING_P (coding)) \
1738 /* This is surely the start of a composition. We must be sure \
1739 that coding->cmp_data has enough space to store the \
1740 information about the composition. If not, terminate the \
1741 current decoding loop, allocate one more memory block for \
1742 coding->cmp_data in the caller, then start the decoding \
1743 loop again. We can't allocate memory here directly because \
1744 it may cause buffer/string relocation. */ \
1745 if (!coding->cmp_data \
1746 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1747 >= COMPOSITION_DATA_SIZE)) \
1749 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1750 goto label_end_of_loop; \
1752 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1753 : c1 == '2' ? COMPOSITION_WITH_RULE \
1754 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1755 : COMPOSITION_WITH_RULE_ALTCHARS); \
1756 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1757 coding->composing); \
1758 coding->composition_rule_follows = 0; \
1762 /* We are already handling a composition. If the method is \
1763 the following two, the codes following the current escape \
1764 sequence are actual characters stored in a buffer. */ \
1765 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1766 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1768 coding->composing = COMPOSITION_RELATIVE; \
1769 coding->composition_rule_follows = 0; \
1774 /* Handle composition end sequence ESC 1. */
1776 #define DECODE_COMPOSITION_END(c1) \
1778 if (! COMPOSING_P (coding)) \
1780 *dst++ = ISO_CODE_ESC; \
1782 coding->produced_char += 2; \
1786 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1787 coding->composing = COMPOSITION_NO; \
1791 /* Decode a composition rule from the byte C1 (and maybe one more byte
1792 from SRC) and store one encoded composition rule in
1793 coding->cmp_data. */
1795 #define DECODE_COMPOSITION_RULE(c1) \
1799 if (c1 < 81) /* old format (before ver.21) */ \
1801 int gref = (c1) / 9; \
1802 int nref = (c1) % 9; \
1803 if (gref == 4) gref = 10; \
1804 if (nref == 4) nref = 10; \
1805 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1807 else if (c1 < 93) /* new format (after ver.21) */ \
1809 ONE_MORE_BYTE (c2); \
1810 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1812 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1813 coding->composition_rule_follows = 0; \
1817 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1820 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1821 struct coding_system
*coding
;
1822 const unsigned char *source
;
1823 unsigned char *destination
;
1824 int src_bytes
, dst_bytes
;
1826 const unsigned char *src
= source
;
1827 const unsigned char *src_end
= source
+ src_bytes
;
1828 unsigned char *dst
= destination
;
1829 unsigned char *dst_end
= destination
+ dst_bytes
;
1830 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1831 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1832 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1833 /* SRC_BASE remembers the start position in source in each loop.
1834 The loop will be exited when there's not enough source code
1835 (within macro ONE_MORE_BYTE), or when there's not enough
1836 destination area to produce a character (within macro
1838 const unsigned char *src_base
;
1840 Lisp_Object translation_table
;
1841 Lisp_Object safe_chars
;
1843 safe_chars
= coding_safe_chars (coding
->symbol
);
1845 if (NILP (Venable_character_translation
))
1846 translation_table
= Qnil
;
1849 translation_table
= coding
->translation_table_for_decode
;
1850 if (NILP (translation_table
))
1851 translation_table
= Vstandard_translation_table_for_decode
;
1854 coding
->result
= CODING_FINISH_NORMAL
;
1863 /* We produce no character or one character. */
1864 switch (iso_code_class
[c1
])
1866 case ISO_0x20_or_0x7F
:
1867 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1869 DECODE_COMPOSITION_RULE (c1
);
1872 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1874 /* This is SPACE or DEL. */
1875 charset
= CHARSET_ASCII
;
1878 /* This is a graphic character, we fall down ... */
1880 case ISO_graphic_plane_0
:
1881 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1883 DECODE_COMPOSITION_RULE (c1
);
1889 case ISO_0xA0_or_0xFF
:
1890 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1891 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1892 goto label_invalid_code
;
1893 /* This is a graphic character, we fall down ... */
1895 case ISO_graphic_plane_1
:
1897 goto label_invalid_code
;
1902 if (COMPOSING_P (coding
))
1903 DECODE_COMPOSITION_END ('1');
1905 /* All ISO2022 control characters in this class have the
1906 same representation in Emacs internal format. */
1908 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1909 && (coding
->eol_type
== CODING_EOL_CR
1910 || coding
->eol_type
== CODING_EOL_CRLF
))
1912 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1913 goto label_end_of_loop
;
1915 charset
= CHARSET_ASCII
;
1919 if (COMPOSING_P (coding
))
1920 DECODE_COMPOSITION_END ('1');
1921 goto label_invalid_code
;
1923 case ISO_carriage_return
:
1924 if (COMPOSING_P (coding
))
1925 DECODE_COMPOSITION_END ('1');
1927 if (coding
->eol_type
== CODING_EOL_CR
)
1929 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1932 if (c1
!= ISO_CODE_LF
)
1938 charset
= CHARSET_ASCII
;
1942 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1943 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1944 goto label_invalid_code
;
1945 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1946 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1950 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1951 goto label_invalid_code
;
1952 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1953 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1956 case ISO_single_shift_2_7
:
1957 case ISO_single_shift_2
:
1958 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1959 goto label_invalid_code
;
1960 /* SS2 is handled as an escape sequence of ESC 'N' */
1962 goto label_escape_sequence
;
1964 case ISO_single_shift_3
:
1965 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1966 goto label_invalid_code
;
1967 /* SS2 is handled as an escape sequence of ESC 'O' */
1969 goto label_escape_sequence
;
1971 case ISO_control_sequence_introducer
:
1972 /* CSI is handled as an escape sequence of ESC '[' ... */
1974 goto label_escape_sequence
;
1978 label_escape_sequence
:
1979 /* Escape sequences handled by Emacs are invocation,
1980 designation, direction specification, and character
1981 composition specification. */
1984 case '&': /* revision of following character set */
1986 if (!(c1
>= '@' && c1
<= '~'))
1987 goto label_invalid_code
;
1989 if (c1
!= ISO_CODE_ESC
)
1990 goto label_invalid_code
;
1992 goto label_escape_sequence
;
1994 case '$': /* designation of 2-byte character set */
1995 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1996 goto label_invalid_code
;
1998 if (c1
>= '@' && c1
<= 'B')
1999 { /* designation of JISX0208.1978, GB2312.1980,
2001 DECODE_DESIGNATION (0, 2, 94, c1
);
2003 else if (c1
>= 0x28 && c1
<= 0x2B)
2004 { /* designation of DIMENSION2_CHARS94 character set */
2006 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
2008 else if (c1
>= 0x2C && c1
<= 0x2F)
2009 { /* designation of DIMENSION2_CHARS96 character set */
2011 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
2014 goto label_invalid_code
;
2015 /* We must update these variables now. */
2016 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2017 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2020 case 'n': /* invocation of locking-shift-2 */
2021 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2022 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2023 goto label_invalid_code
;
2024 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
2025 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2028 case 'o': /* invocation of locking-shift-3 */
2029 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2030 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2031 goto label_invalid_code
;
2032 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
2033 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2036 case 'N': /* invocation of single-shift-2 */
2037 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2038 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2039 goto label_invalid_code
;
2040 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
2042 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2043 goto label_invalid_code
;
2046 case 'O': /* invocation of single-shift-3 */
2047 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2048 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2049 goto label_invalid_code
;
2050 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
2052 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2053 goto label_invalid_code
;
2056 case '0': case '2': case '3': case '4': /* start composition */
2057 DECODE_COMPOSITION_START (c1
);
2060 case '1': /* end composition */
2061 DECODE_COMPOSITION_END (c1
);
2064 case '[': /* specification of direction */
2065 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
2066 goto label_invalid_code
;
2067 /* For the moment, nested direction is not supported.
2068 So, `coding->mode & CODING_MODE_DIRECTION' zero means
2069 left-to-right, and nonzero means right-to-left. */
2073 case ']': /* end of the current direction */
2074 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2076 case '0': /* end of the current direction */
2077 case '1': /* start of left-to-right direction */
2080 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2082 goto label_invalid_code
;
2085 case '2': /* start of right-to-left direction */
2088 coding
->mode
|= CODING_MODE_DIRECTION
;
2090 goto label_invalid_code
;
2094 goto label_invalid_code
;
2099 if (COMPOSING_P (coding
))
2100 DECODE_COMPOSITION_END ('1');
2104 /* CTEXT extended segment:
2105 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
2106 We keep these bytes as is for the moment.
2107 They may be decoded by post-read-conversion. */
2112 ONE_MORE_BYTE (dim
);
2115 size
= ((M
- 128) * 128) + (L
- 128);
2116 required
= 8 + size
* 2;
2117 if (dst
+ required
> (dst_bytes
? dst_end
: src
))
2118 goto label_end_of_loop
;
2119 *dst
++ = ISO_CODE_ESC
;
2124 dst
+= CHAR_STRING (M
, dst
), produced_chars
++;
2125 dst
+= CHAR_STRING (L
, dst
), produced_chars
++;
2129 dst
+= CHAR_STRING (c1
, dst
), produced_chars
++;
2131 coding
->produced_char
+= produced_chars
;
2135 unsigned char *d
= dst
;
2138 /* XFree86 extension for embedding UTF-8 in CTEXT:
2139 ESC % G --UTF-8-BYTES-- ESC % @
2140 We keep these bytes as is for the moment.
2141 They may be decoded by post-read-conversion. */
2142 if (d
+ 6 > (dst_bytes
? dst_end
: src
))
2143 goto label_end_of_loop
;
2144 *d
++ = ISO_CODE_ESC
;
2148 while (d
+ 1 < (dst_bytes
? dst_end
: src
))
2151 if (c1
== ISO_CODE_ESC
2152 && src
+ 1 < src_end
2159 d
+= CHAR_STRING (c1
, d
), produced_chars
++;
2161 if (d
+ 3 > (dst_bytes
? dst_end
: src
))
2162 goto label_end_of_loop
;
2163 *d
++ = ISO_CODE_ESC
;
2167 coding
->produced_char
+= produced_chars
+ 3;
2170 goto label_invalid_code
;
2174 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
2175 goto label_invalid_code
;
2176 if (c1
>= 0x28 && c1
<= 0x2B)
2177 { /* designation of DIMENSION1_CHARS94 character set */
2179 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
2181 else if (c1
>= 0x2C && c1
<= 0x2F)
2182 { /* designation of DIMENSION1_CHARS96 character set */
2184 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
2187 goto label_invalid_code
;
2188 /* We must update these variables now. */
2189 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2190 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2195 /* Now we know CHARSET and 1st position code C1 of a character.
2196 Produce a multibyte sequence for that character while getting
2197 2nd position code C2 if necessary. */
2198 if (CHARSET_DIMENSION (charset
) == 2)
2201 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
2202 /* C2 is not in a valid range. */
2203 goto label_invalid_code
;
2205 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2211 if (COMPOSING_P (coding
))
2212 DECODE_COMPOSITION_END ('1');
2215 if (! NILP (translation_table
))
2216 c
= translate_char (translation_table
, c
, 0, 0, 0);
2221 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2222 coding
->produced
= dst
- destination
;
2227 /* ISO2022 encoding stuff. */
2230 It is not enough to say just "ISO2022" on encoding, we have to
2231 specify more details. In Emacs, each ISO2022 coding system
2232 variant has the following specifications:
2233 1. Initial designation to G0 through G3.
2234 2. Allows short-form designation?
2235 3. ASCII should be designated to G0 before control characters?
2236 4. ASCII should be designated to G0 at end of line?
2237 5. 7-bit environment or 8-bit environment?
2238 6. Use locking-shift?
2239 7. Use Single-shift?
2240 And the following two are only for Japanese:
2241 8. Use ASCII in place of JIS0201-1976-Roman?
2242 9. Use JISX0208-1983 in place of JISX0208-1978?
2243 These specifications are encoded in `coding->flags' as flag bits
2244 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
2248 /* Produce codes (escape sequence) for designating CHARSET to graphic
2249 register REG at DST, and increment DST. If <final-char> of CHARSET is
2250 '@', 'A', or 'B' and the coding system CODING allows, produce
2251 designation sequence of short-form. */
2253 #define ENCODE_DESIGNATION(charset, reg, coding) \
2255 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2256 char *intermediate_char_94 = "()*+"; \
2257 char *intermediate_char_96 = ",-./"; \
2258 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
2260 if (revision < 255) \
2262 *dst++ = ISO_CODE_ESC; \
2264 *dst++ = '@' + revision; \
2266 *dst++ = ISO_CODE_ESC; \
2267 if (CHARSET_DIMENSION (charset) == 1) \
2269 if (CHARSET_CHARS (charset) == 94) \
2270 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2272 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2277 if (CHARSET_CHARS (charset) == 94) \
2279 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2281 || final_char < '@' || final_char > 'B') \
2282 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2285 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2287 *dst++ = final_char; \
2288 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2291 /* The following two macros produce codes (control character or escape
2292 sequence) for ISO2022 single-shift functions (single-shift-2 and
2295 #define ENCODE_SINGLE_SHIFT_2 \
2297 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2298 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2300 *dst++ = ISO_CODE_SS2; \
2301 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2304 #define ENCODE_SINGLE_SHIFT_3 \
2306 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2307 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2309 *dst++ = ISO_CODE_SS3; \
2310 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2313 /* The following four macros produce codes (control character or
2314 escape sequence) for ISO2022 locking-shift functions (shift-in,
2315 shift-out, locking-shift-2, and locking-shift-3). */
2317 #define ENCODE_SHIFT_IN \
2319 *dst++ = ISO_CODE_SI; \
2320 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2323 #define ENCODE_SHIFT_OUT \
2325 *dst++ = ISO_CODE_SO; \
2326 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2329 #define ENCODE_LOCKING_SHIFT_2 \
2331 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2332 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2335 #define ENCODE_LOCKING_SHIFT_3 \
2337 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
2338 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2341 /* Produce codes for a DIMENSION1 character whose character set is
2342 CHARSET and whose position-code is C1. Designation and invocation
2343 sequences are also produced in advance if necessary. */
2345 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2347 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2349 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2350 *dst++ = c1 & 0x7F; \
2352 *dst++ = c1 | 0x80; \
2353 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2356 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2358 *dst++ = c1 & 0x7F; \
2361 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2363 *dst++ = c1 | 0x80; \
2367 /* Since CHARSET is not yet invoked to any graphic planes, we \
2368 must invoke it, or, at first, designate it to some graphic \
2369 register. Then repeat the loop to actually produce the \
2371 dst = encode_invocation_designation (charset, coding, dst); \
2374 /* Produce codes for a DIMENSION2 character whose character set is
2375 CHARSET and whose position-codes are C1 and C2. Designation and
2376 invocation codes are also produced in advance if necessary. */
2378 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2380 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2382 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2383 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2385 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2386 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2389 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2391 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2394 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2396 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2400 /* Since CHARSET is not yet invoked to any graphic planes, we \
2401 must invoke it, or, at first, designate it to some graphic \
2402 register. Then repeat the loop to actually produce the \
2404 dst = encode_invocation_designation (charset, coding, dst); \
2407 #define ENCODE_ISO_CHARACTER(c) \
2409 int charset, c1, c2; \
2411 SPLIT_CHAR (c, charset, c1, c2); \
2412 if (CHARSET_DEFINED_P (charset)) \
2414 if (CHARSET_DIMENSION (charset) == 1) \
2416 if (charset == CHARSET_ASCII \
2417 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2418 charset = charset_latin_jisx0201; \
2419 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2423 if (charset == charset_jisx0208 \
2424 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2425 charset = charset_jisx0208_1978; \
2426 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2438 /* Instead of encoding character C, produce one or two `?'s. */
2440 #define ENCODE_UNSAFE_CHARACTER(c) \
2442 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2443 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2444 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2448 /* Produce designation and invocation codes at a place pointed by DST
2449 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2453 encode_invocation_designation (charset
, coding
, dst
)
2455 struct coding_system
*coding
;
2458 int reg
; /* graphic register number */
2460 /* At first, check designations. */
2461 for (reg
= 0; reg
< 4; reg
++)
2462 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
2467 /* CHARSET is not yet designated to any graphic registers. */
2468 /* At first check the requested designation. */
2469 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2470 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
2471 /* Since CHARSET requests no special designation, designate it
2472 to graphic register 0. */
2475 ENCODE_DESIGNATION (charset
, reg
, coding
);
2478 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
2479 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
2481 /* Since the graphic register REG is not invoked to any graphic
2482 planes, invoke it to graphic plane 0. */
2485 case 0: /* graphic register 0 */
2489 case 1: /* graphic register 1 */
2493 case 2: /* graphic register 2 */
2494 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2495 ENCODE_SINGLE_SHIFT_2
;
2497 ENCODE_LOCKING_SHIFT_2
;
2500 case 3: /* graphic register 3 */
2501 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2502 ENCODE_SINGLE_SHIFT_3
;
2504 ENCODE_LOCKING_SHIFT_3
;
2512 /* Produce 2-byte codes for encoded composition rule RULE. */
2514 #define ENCODE_COMPOSITION_RULE(rule) \
2517 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2518 *dst++ = 32 + 81 + gref; \
2519 *dst++ = 32 + nref; \
2522 /* Produce codes for indicating the start of a composition sequence
2523 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2524 which specify information about the composition. See the comment
2525 in coding.h for the format of DATA. */
2527 #define ENCODE_COMPOSITION_START(coding, data) \
2529 coding->composing = data[3]; \
2530 *dst++ = ISO_CODE_ESC; \
2531 if (coding->composing == COMPOSITION_RELATIVE) \
2535 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2537 coding->cmp_data_index = coding->cmp_data_start + 4; \
2538 coding->composition_rule_follows = 0; \
2542 /* Produce codes for indicating the end of the current composition. */
2544 #define ENCODE_COMPOSITION_END(coding, data) \
2546 *dst++ = ISO_CODE_ESC; \
2548 coding->cmp_data_start += data[0]; \
2549 coding->composing = COMPOSITION_NO; \
2550 if (coding->cmp_data_start == coding->cmp_data->used \
2551 && coding->cmp_data->next) \
2553 coding->cmp_data = coding->cmp_data->next; \
2554 coding->cmp_data_start = 0; \
2558 /* Produce composition start sequence ESC 0. Here, this sequence
2559 doesn't mean the start of a new composition but means that we have
2560 just produced components (alternate chars and composition rules) of
2561 the composition and the actual text follows in SRC. */
2563 #define ENCODE_COMPOSITION_FAKE_START(coding) \
2565 *dst++ = ISO_CODE_ESC; \
2567 coding->composing = COMPOSITION_RELATIVE; \
2570 /* The following three macros produce codes for indicating direction
2572 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2574 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2575 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2577 *dst++ = ISO_CODE_CSI; \
2580 #define ENCODE_DIRECTION_R2L \
2581 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2583 #define ENCODE_DIRECTION_L2R \
2584 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2586 /* Produce codes for designation and invocation to reset the graphic
2587 planes and registers to initial state. */
2588 #define ENCODE_RESET_PLANE_AND_REGISTER \
2591 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2593 for (reg = 0; reg < 4; reg++) \
2594 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2595 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2596 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2597 ENCODE_DESIGNATION \
2598 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
2601 /* Produce designation sequences of charsets in the line started from
2602 SRC to a place pointed by DST, and return updated DST.
2604 If the current block ends before any end-of-line, we may fail to
2605 find all the necessary designations. */
2607 static unsigned char *
2608 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
2609 struct coding_system
*coding
;
2610 Lisp_Object translation_table
;
2611 const unsigned char *src
, *src_end
;
2614 int charset
, c
, found
= 0, reg
;
2615 /* Table of charsets to be designated to each graphic register. */
2618 for (reg
= 0; reg
< 4; reg
++)
2627 charset
= CHAR_CHARSET (c
);
2628 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2629 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
2639 for (reg
= 0; reg
< 4; reg
++)
2641 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
2642 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2648 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2651 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2652 struct coding_system
*coding
;
2653 const unsigned char *source
;
2654 unsigned char *destination
;
2655 int src_bytes
, dst_bytes
;
2657 const unsigned char *src
= source
;
2658 const unsigned char *src_end
= source
+ src_bytes
;
2659 unsigned char *dst
= destination
;
2660 unsigned char *dst_end
= destination
+ dst_bytes
;
2661 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2662 from DST_END to assure overflow checking is necessary only at the
2664 unsigned char *adjusted_dst_end
= dst_end
- 19;
2665 /* SRC_BASE remembers the start position in source in each loop.
2666 The loop will be exited when there's not enough source text to
2667 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2668 there's not enough destination area to produce encoded codes
2669 (within macro EMIT_BYTES). */
2670 const unsigned char *src_base
;
2672 Lisp_Object translation_table
;
2673 Lisp_Object safe_chars
;
2675 if (coding
->flags
& CODING_FLAG_ISO_SAFE
)
2676 coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
2678 safe_chars
= coding_safe_chars (coding
->symbol
);
2680 if (NILP (Venable_character_translation
))
2681 translation_table
= Qnil
;
2684 translation_table
= coding
->translation_table_for_encode
;
2685 if (NILP (translation_table
))
2686 translation_table
= Vstandard_translation_table_for_encode
;
2689 coding
->consumed_char
= 0;
2695 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2697 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2701 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2702 && CODING_SPEC_ISO_BOL (coding
))
2704 /* We have to produce designation sequences if any now. */
2705 dst
= encode_designation_at_bol (coding
, translation_table
,
2707 CODING_SPEC_ISO_BOL (coding
) = 0;
2710 /* Check composition start and end. */
2711 if (coding
->composing
!= COMPOSITION_DISABLED
2712 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2714 struct composition_data
*cmp_data
= coding
->cmp_data
;
2715 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2716 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2718 if (coding
->composing
== COMPOSITION_RELATIVE
)
2720 if (this_pos
== data
[2])
2722 ENCODE_COMPOSITION_END (coding
, data
);
2723 cmp_data
= coding
->cmp_data
;
2724 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2727 else if (COMPOSING_P (coding
))
2729 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2730 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2731 /* We have consumed components of the composition.
2732 What follows in SRC is the composition's base
2734 ENCODE_COMPOSITION_FAKE_START (coding
);
2737 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2738 if (coding
->composition_rule_follows
)
2740 ENCODE_COMPOSITION_RULE (c
);
2741 coding
->composition_rule_follows
= 0;
2745 if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2746 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2747 ENCODE_UNSAFE_CHARACTER (c
);
2749 ENCODE_ISO_CHARACTER (c
);
2750 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2751 coding
->composition_rule_follows
= 1;
2756 if (!COMPOSING_P (coding
))
2758 if (this_pos
== data
[1])
2760 ENCODE_COMPOSITION_START (coding
, data
);
2768 /* Now encode the character C. */
2769 if (c
< 0x20 || c
== 0x7F)
2773 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2775 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2776 ENCODE_RESET_PLANE_AND_REGISTER
;
2780 /* fall down to treat '\r' as '\n' ... */
2785 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2786 ENCODE_RESET_PLANE_AND_REGISTER
;
2787 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2788 bcopy (coding
->spec
.iso2022
.initial_designation
,
2789 coding
->spec
.iso2022
.current_designation
,
2790 sizeof coding
->spec
.iso2022
.initial_designation
);
2791 if (coding
->eol_type
== CODING_EOL_LF
2792 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2793 *dst
++ = ISO_CODE_LF
;
2794 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2795 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2797 *dst
++ = ISO_CODE_CR
;
2798 CODING_SPEC_ISO_BOL (coding
) = 1;
2802 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2803 ENCODE_RESET_PLANE_AND_REGISTER
;
2807 else if (ASCII_BYTE_P (c
))
2808 ENCODE_ISO_CHARACTER (c
);
2809 else if (SINGLE_BYTE_CHAR_P (c
))
2814 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2815 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2816 ENCODE_UNSAFE_CHARACTER (c
);
2818 ENCODE_ISO_CHARACTER (c
);
2820 coding
->consumed_char
++;
2824 coding
->consumed
= src_base
- source
;
2825 coding
->produced
= coding
->produced_char
= dst
- destination
;
2829 /*** 4. SJIS and BIG5 handlers ***/
2831 /* Although SJIS and BIG5 are not ISO coding systems, they are used
2832 quite widely. So, for the moment, Emacs supports them in the bare
2833 C code. But, in the future, they may be supported only by CCL. */
2835 /* SJIS is a coding system encoding three character sets: ASCII, right
2836 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2837 as is. A character of charset katakana-jisx0201 is encoded by
2838 "position-code + 0x80". A character of charset japanese-jisx0208
2839 is encoded in 2-byte but two position-codes are divided and shifted
2840 so that it fits in the range below.
2842 --- CODE RANGE of SJIS ---
2843 (character set) (range)
2845 KATAKANA-JISX0201 0xA1 .. 0xDF
2846 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2847 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2848 -------------------------------
2852 /* BIG5 is a coding system encoding two character sets: ASCII and
2853 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2854 character set and is encoded in two bytes.
2856 --- CODE RANGE of BIG5 ---
2857 (character set) (range)
2859 Big5 (1st byte) 0xA1 .. 0xFE
2860 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2861 --------------------------
2863 Since the number of characters in Big5 is larger than maximum
2864 characters in Emacs' charset (96x96), it can't be handled as one
2865 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2866 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2867 contains frequently used characters and the latter contains less
2868 frequently used characters. */
2870 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2871 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2872 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
2873 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2875 /* Number of Big5 characters which have the same code in 1st byte. */
2876 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2878 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2881 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2883 charset = charset_big5_1; \
2886 charset = charset_big5_2; \
2887 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2889 c1 = temp / (0xFF - 0xA1) + 0x21; \
2890 c2 = temp % (0xFF - 0xA1) + 0x21; \
2893 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2895 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2896 if (charset == charset_big5_2) \
2897 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2898 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2899 b2 = temp % BIG5_SAME_ROW; \
2900 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2903 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2904 Check if a text is encoded in SJIS. If it is, return
2905 CODING_CATEGORY_MASK_SJIS, else return 0. */
2908 detect_coding_sjis (src
, src_end
, multibytep
)
2909 unsigned char *src
, *src_end
;
2913 /* Dummy for ONE_MORE_BYTE. */
2914 struct coding_system dummy_coding
;
2915 struct coding_system
*coding
= &dummy_coding
;
2919 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_SJIS
);
2922 if (c
== 0x80 || c
== 0xA0 || c
> 0xEF)
2924 if (c
<= 0x9F || c
>= 0xE0)
2926 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
2927 if (c
< 0x40 || c
== 0x7F || c
> 0xFC)
2933 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2934 Check if a text is encoded in BIG5. If it is, return
2935 CODING_CATEGORY_MASK_BIG5, else return 0. */
2938 detect_coding_big5 (src
, src_end
, multibytep
)
2939 unsigned char *src
, *src_end
;
2943 /* Dummy for ONE_MORE_BYTE. */
2944 struct coding_system dummy_coding
;
2945 struct coding_system
*coding
= &dummy_coding
;
2949 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_BIG5
);
2952 if (c
< 0xA1 || c
> 0xFE)
2954 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
2955 if (c
< 0x40 || (c
> 0x7F && c
< 0xA1) || c
> 0xFE)
2960 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2961 Check if a text is encoded in UTF-8. If it is, return
2962 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2964 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2965 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2966 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2967 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2968 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2969 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2970 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2973 detect_coding_utf_8 (src
, src_end
, multibytep
)
2974 unsigned char *src
, *src_end
;
2978 int seq_maybe_bytes
;
2979 /* Dummy for ONE_MORE_BYTE. */
2980 struct coding_system dummy_coding
;
2981 struct coding_system
*coding
= &dummy_coding
;
2985 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_UTF_8
);
2986 if (UTF_8_1_OCTET_P (c
))
2988 else if (UTF_8_2_OCTET_LEADING_P (c
))
2989 seq_maybe_bytes
= 1;
2990 else if (UTF_8_3_OCTET_LEADING_P (c
))
2991 seq_maybe_bytes
= 2;
2992 else if (UTF_8_4_OCTET_LEADING_P (c
))
2993 seq_maybe_bytes
= 3;
2994 else if (UTF_8_5_OCTET_LEADING_P (c
))
2995 seq_maybe_bytes
= 4;
2996 else if (UTF_8_6_OCTET_LEADING_P (c
))
2997 seq_maybe_bytes
= 5;
3003 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, 0);
3004 if (!UTF_8_EXTRA_OCTET_P (c
))
3008 while (seq_maybe_bytes
> 0);
3012 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3013 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
3014 Little Endian (otherwise). If it is, return
3015 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
3018 #define UTF_16_INVALID_P(val) \
3019 (((val) == 0xFFFE) \
3020 || ((val) == 0xFFFF))
3022 #define UTF_16_HIGH_SURROGATE_P(val) \
3023 (((val) & 0xD800) == 0xD800)
3025 #define UTF_16_LOW_SURROGATE_P(val) \
3026 (((val) & 0xDC00) == 0xDC00)
3029 detect_coding_utf_16 (src
, src_end
, multibytep
)
3030 unsigned char *src
, *src_end
;
3033 unsigned char c1
, c2
;
3034 /* Dummy for ONE_MORE_BYTE_CHECK_MULTIBYTE. */
3035 struct coding_system dummy_coding
;
3036 struct coding_system
*coding
= &dummy_coding
;
3038 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
, 0);
3039 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2
, multibytep
, 0);
3041 if ((c1
== 0xFF) && (c2
== 0xFE))
3042 return CODING_CATEGORY_MASK_UTF_16_LE
;
3043 else if ((c1
== 0xFE) && (c2
== 0xFF))
3044 return CODING_CATEGORY_MASK_UTF_16_BE
;
3048 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
3049 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
3052 decode_coding_sjis_big5 (coding
, source
, destination
,
3053 src_bytes
, dst_bytes
, sjis_p
)
3054 struct coding_system
*coding
;
3055 const unsigned char *source
;
3056 unsigned char *destination
;
3057 int src_bytes
, dst_bytes
;
3060 const unsigned char *src
= source
;
3061 const unsigned char *src_end
= source
+ src_bytes
;
3062 unsigned char *dst
= destination
;
3063 unsigned char *dst_end
= destination
+ dst_bytes
;
3064 /* SRC_BASE remembers the start position in source in each loop.
3065 The loop will be exited when there's not enough source code
3066 (within macro ONE_MORE_BYTE), or when there's not enough
3067 destination area to produce a character (within macro
3069 const unsigned char *src_base
;
3070 Lisp_Object translation_table
;
3072 if (NILP (Venable_character_translation
))
3073 translation_table
= Qnil
;
3076 translation_table
= coding
->translation_table_for_decode
;
3077 if (NILP (translation_table
))
3078 translation_table
= Vstandard_translation_table_for_decode
;
3081 coding
->produced_char
= 0;
3084 int c
, charset
, c1
, c2
= 0;
3091 charset
= CHARSET_ASCII
;
3096 if (coding
->eol_type
== CODING_EOL_CRLF
)
3102 /* To process C2 again, SRC is subtracted by 1. */
3105 else if (coding
->eol_type
== CODING_EOL_CR
)
3109 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3110 && (coding
->eol_type
== CODING_EOL_CR
3111 || coding
->eol_type
== CODING_EOL_CRLF
))
3113 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3114 goto label_end_of_loop
;
3122 if (c1
== 0x80 || c1
== 0xA0 || c1
> 0xEF)
3123 goto label_invalid_code
;
3124 if (c1
<= 0x9F || c1
>= 0xE0)
3126 /* SJIS -> JISX0208 */
3128 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
3129 goto label_invalid_code
;
3130 DECODE_SJIS (c1
, c2
, c1
, c2
);
3131 charset
= charset_jisx0208
;
3134 /* SJIS -> JISX0201-Kana */
3135 charset
= charset_katakana_jisx0201
;
3140 if (c1
< 0xA0 || c1
> 0xFE)
3141 goto label_invalid_code
;
3143 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
3144 goto label_invalid_code
;
3145 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
3149 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
3161 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3162 coding
->produced
= dst
- destination
;
3166 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
3167 This function can encode charsets `ascii', `katakana-jisx0201',
3168 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
3169 are sure that all these charsets are registered as official charset
3170 (i.e. do not have extended leading-codes). Characters of other
3171 charsets are produced without any encoding. If SJIS_P is 1, encode
3172 SJIS text, else encode BIG5 text. */
3175 encode_coding_sjis_big5 (coding
, source
, destination
,
3176 src_bytes
, dst_bytes
, sjis_p
)
3177 struct coding_system
*coding
;
3178 unsigned char *source
, *destination
;
3179 int src_bytes
, dst_bytes
;
3182 unsigned char *src
= source
;
3183 unsigned char *src_end
= source
+ src_bytes
;
3184 unsigned char *dst
= destination
;
3185 unsigned char *dst_end
= destination
+ dst_bytes
;
3186 /* SRC_BASE remembers the start position in source in each loop.
3187 The loop will be exited when there's not enough source text to
3188 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3189 there's not enough destination area to produce encoded codes
3190 (within macro EMIT_BYTES). */
3191 unsigned char *src_base
;
3192 Lisp_Object translation_table
;
3194 if (NILP (Venable_character_translation
))
3195 translation_table
= Qnil
;
3198 translation_table
= coding
->translation_table_for_encode
;
3199 if (NILP (translation_table
))
3200 translation_table
= Vstandard_translation_table_for_encode
;
3205 int c
, charset
, c1
, c2
;
3210 /* Now encode the character C. */
3211 if (SINGLE_BYTE_CHAR_P (c
))
3216 if (!(coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
3223 if (coding
->eol_type
== CODING_EOL_CRLF
)
3225 EMIT_TWO_BYTES ('\r', c
);
3228 else if (coding
->eol_type
== CODING_EOL_CR
)
3236 SPLIT_CHAR (c
, charset
, c1
, c2
);
3239 if (charset
== charset_jisx0208
3240 || charset
== charset_jisx0208_1978
)
3242 ENCODE_SJIS (c1
, c2
, c1
, c2
);
3243 EMIT_TWO_BYTES (c1
, c2
);
3245 else if (charset
== charset_katakana_jisx0201
)
3246 EMIT_ONE_BYTE (c1
| 0x80);
3247 else if (charset
== charset_latin_jisx0201
)
3249 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3251 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3252 if (CHARSET_WIDTH (charset
) > 1)
3253 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3256 /* There's no way other than producing the internal
3258 EMIT_BYTES (src_base
, src
);
3262 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
3264 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
3265 EMIT_TWO_BYTES (c1
, c2
);
3267 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3269 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3270 if (CHARSET_WIDTH (charset
) > 1)
3271 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3274 /* There's no way other than producing the internal
3276 EMIT_BYTES (src_base
, src
);
3279 coding
->consumed_char
++;
3283 coding
->consumed
= src_base
- source
;
3284 coding
->produced
= coding
->produced_char
= dst
- destination
;
3288 /*** 5. CCL handlers ***/
3290 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3291 Check if a text is encoded in a coding system of which
3292 encoder/decoder are written in CCL program. If it is, return
3293 CODING_CATEGORY_MASK_CCL, else return 0. */
3296 detect_coding_ccl (src
, src_end
, multibytep
)
3297 unsigned char *src
, *src_end
;
3300 unsigned char *valid
;
3302 /* Dummy for ONE_MORE_BYTE. */
3303 struct coding_system dummy_coding
;
3304 struct coding_system
*coding
= &dummy_coding
;
3306 /* No coding system is assigned to coding-category-ccl. */
3307 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
3310 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
3313 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
, CODING_CATEGORY_MASK_CCL
);
3320 /*** 6. End-of-line handlers ***/
3322 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3325 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3326 struct coding_system
*coding
;
3327 const unsigned char *source
;
3328 unsigned char *destination
;
3329 int src_bytes
, dst_bytes
;
3331 const unsigned char *src
= source
;
3332 unsigned char *dst
= destination
;
3333 const unsigned char *src_end
= src
+ src_bytes
;
3334 unsigned char *dst_end
= dst
+ dst_bytes
;
3335 Lisp_Object translation_table
;
3336 /* SRC_BASE remembers the start position in source in each loop.
3337 The loop will be exited when there's not enough source code
3338 (within macro ONE_MORE_BYTE), or when there's not enough
3339 destination area to produce a character (within macro
3341 const unsigned char *src_base
;
3344 translation_table
= Qnil
;
3345 switch (coding
->eol_type
)
3347 case CODING_EOL_CRLF
:
3362 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
3364 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3365 goto label_end_of_loop
;
3378 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3380 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3381 goto label_end_of_loop
;
3390 default: /* no need for EOL handling */
3400 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3401 coding
->produced
= dst
- destination
;
3405 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
3406 format of end-of-line according to `coding->eol_type'. It also
3407 convert multibyte form 8-bit characters to unibyte if
3408 CODING->src_multibyte is nonzero. If `coding->mode &
3409 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3410 also means end-of-line. */
3413 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3414 struct coding_system
*coding
;
3415 const unsigned char *source
;
3416 unsigned char *destination
;
3417 int src_bytes
, dst_bytes
;
3419 const unsigned char *src
= source
;
3420 unsigned char *dst
= destination
;
3421 const unsigned char *src_end
= src
+ src_bytes
;
3422 unsigned char *dst_end
= dst
+ dst_bytes
;
3423 Lisp_Object translation_table
;
3424 /* SRC_BASE remembers the start position in source in each loop.
3425 The loop will be exited when there's not enough source text to
3426 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3427 there's not enough destination area to produce encoded codes
3428 (within macro EMIT_BYTES). */
3429 const unsigned char *src_base
;
3432 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
3434 translation_table
= Qnil
;
3435 if (coding
->src_multibyte
3436 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3440 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
3443 if (coding
->eol_type
== CODING_EOL_CRLF
)
3445 while (src
< src_end
)
3451 else if (c
== '\n' || (c
== '\r' && selective_display
))
3452 EMIT_TWO_BYTES ('\r', '\n');
3462 if (!dst_bytes
|| src_bytes
<= dst_bytes
)
3464 safe_bcopy (src
, dst
, src_bytes
);
3470 if (coding
->src_multibyte
3471 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3473 safe_bcopy (src
, dst
, dst_bytes
);
3474 src_base
= src
+ dst_bytes
;
3475 dst
= destination
+ dst_bytes
;
3476 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
3478 if (coding
->eol_type
== CODING_EOL_CR
)
3480 for (tmp
= destination
; tmp
< dst
; tmp
++)
3481 if (*tmp
== '\n') *tmp
= '\r';
3483 else if (selective_display
)
3485 for (tmp
= destination
; tmp
< dst
; tmp
++)
3486 if (*tmp
== '\r') *tmp
= '\n';
3489 if (coding
->src_multibyte
)
3490 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
3492 coding
->consumed
= src_base
- source
;
3493 coding
->produced
= dst
- destination
;
3494 coding
->produced_char
= coding
->produced
;
3498 /*** 7. C library functions ***/
3500 /* In Emacs Lisp, a coding system is represented by a Lisp symbol which
3501 has a property `coding-system'. The value of this property is a
3502 vector of length 5 (called the coding-vector). Among elements of
3503 this vector, the first (element[0]) and the fifth (element[4])
3504 carry important information for decoding/encoding. Before
3505 decoding/encoding, this information should be set in fields of a
3506 structure of type `coding_system'.
3508 The value of the property `coding-system' can be a symbol of another
3509 subsidiary coding-system. In that case, Emacs gets coding-vector
3512 `element[0]' contains information to be set in `coding->type'. The
3513 value and its meaning is as follows:
3515 0 -- coding_type_emacs_mule
3516 1 -- coding_type_sjis
3517 2 -- coding_type_iso2022
3518 3 -- coding_type_big5
3519 4 -- coding_type_ccl encoder/decoder written in CCL
3520 nil -- coding_type_no_conversion
3521 t -- coding_type_undecided (automatic conversion on decoding,
3522 no-conversion on encoding)
3524 `element[4]' contains information to be set in `coding->flags' and
3525 `coding->spec'. The meaning varies by `coding->type'.
3527 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3528 of length 32 (of which the first 13 sub-elements are used now).
3529 Meanings of these sub-elements are:
3531 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3532 If the value is an integer of valid charset, the charset is
3533 assumed to be designated to graphic register N initially.
3535 If the value is minus, it is a minus value of charset which
3536 reserves graphic register N, which means that the charset is
3537 not designated initially but should be designated to graphic
3538 register N just before encoding a character in that charset.
3540 If the value is nil, graphic register N is never used on
3543 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3544 Each value takes t or nil. See the section ISO2022 of
3545 `coding.h' for more information.
3547 If `coding->type' is `coding_type_big5', element[4] is t to denote
3548 BIG5-ETen or nil to denote BIG5-HKU.
3550 If `coding->type' takes the other value, element[4] is ignored.
3552 Emacs Lisp's coding systems also carry information about format of
3553 end-of-line in a value of property `eol-type'. If the value is
3554 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3555 means CODING_EOL_CR. If it is not integer, it should be a vector
3556 of subsidiary coding systems of which property `eol-type' has one
3557 of the above values.
3561 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3562 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3563 is setup so that no conversion is necessary and return -1, else
3567 setup_coding_system (coding_system
, coding
)
3568 Lisp_Object coding_system
;
3569 struct coding_system
*coding
;
3571 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
3574 /* At first, zero clear all members. */
3575 bzero (coding
, sizeof (struct coding_system
));
3577 /* Initialize some fields required for all kinds of coding systems. */
3578 coding
->symbol
= coding_system
;
3579 coding
->heading_ascii
= -1;
3580 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
3581 coding
->composing
= COMPOSITION_DISABLED
;
3582 coding
->cmp_data
= NULL
;
3584 if (NILP (coding_system
))
3585 goto label_invalid_coding_system
;
3587 coding_spec
= Fget (coding_system
, Qcoding_system
);
3589 if (!VECTORP (coding_spec
)
3590 || XVECTOR (coding_spec
)->size
!= 5
3591 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
3592 goto label_invalid_coding_system
;
3594 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
3595 if (VECTORP (eol_type
))
3597 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3598 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
3599 if (system_eol_type
!= CODING_EOL_LF
)
3600 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3602 else if (XFASTINT (eol_type
) == 1)
3604 coding
->eol_type
= CODING_EOL_CRLF
;
3605 coding
->common_flags
3606 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3608 else if (XFASTINT (eol_type
) == 2)
3610 coding
->eol_type
= CODING_EOL_CR
;
3611 coding
->common_flags
3612 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3616 coding
->common_flags
= 0;
3617 coding
->eol_type
= CODING_EOL_LF
;
3620 coding_type
= XVECTOR (coding_spec
)->contents
[0];
3621 /* Try short cut. */
3622 if (SYMBOLP (coding_type
))
3624 if (EQ (coding_type
, Qt
))
3626 coding
->type
= coding_type_undecided
;
3627 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
3630 coding
->type
= coding_type_no_conversion
;
3631 /* Initialize this member. Any thing other than
3632 CODING_CATEGORY_IDX_UTF_16_BE and
3633 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3634 special treatment in detect_eol. */
3635 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
3640 /* Get values of coding system properties:
3641 `post-read-conversion', `pre-write-conversion',
3642 `translation-table-for-decode', `translation-table-for-encode'. */
3643 plist
= XVECTOR (coding_spec
)->contents
[3];
3644 /* Pre & post conversion functions should be disabled if
3645 inhibit_eol_conversion is nonzero. This is the case that a code
3646 conversion function is called while those functions are running. */
3647 if (! inhibit_pre_post_conversion
)
3649 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
3650 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
3652 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
3654 val
= Fget (val
, Qtranslation_table_for_decode
);
3655 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3656 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
3658 val
= Fget (val
, Qtranslation_table_for_encode
);
3659 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3660 val
= Fplist_get (plist
, Qcoding_category
);
3663 val
= Fget (val
, Qcoding_category_index
);
3665 coding
->category_idx
= XINT (val
);
3667 goto label_invalid_coding_system
;
3670 goto label_invalid_coding_system
;
3672 /* If the coding system has non-nil `composition' property, enable
3673 composition handling. */
3674 val
= Fplist_get (plist
, Qcomposition
);
3676 coding
->composing
= COMPOSITION_NO
;
3678 /* If the coding system is ascii-incompatible, record it in
3680 val
= Fplist_get (plist
, Qascii_incompatible
);
3682 coding
->common_flags
|= CODING_ASCII_INCOMPATIBLE_MASK
;
3684 switch (XFASTINT (coding_type
))
3687 coding
->type
= coding_type_emacs_mule
;
3688 coding
->common_flags
3689 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3690 if (!NILP (coding
->post_read_conversion
))
3691 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3692 if (!NILP (coding
->pre_write_conversion
))
3693 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3697 coding
->type
= coding_type_sjis
;
3698 coding
->common_flags
3699 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3703 coding
->type
= coding_type_iso2022
;
3704 coding
->common_flags
3705 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3707 Lisp_Object val
, temp
;
3709 int i
, charset
, reg_bits
= 0;
3711 val
= XVECTOR (coding_spec
)->contents
[4];
3713 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3714 goto label_invalid_coding_system
;
3716 flags
= XVECTOR (val
)->contents
;
3718 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3719 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3720 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3721 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3722 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3723 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3724 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3725 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3726 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3727 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3728 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3729 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3730 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3733 /* Invoke graphic register 0 to plane 0. */
3734 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3735 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3736 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3737 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3738 /* Not single shifting at first. */
3739 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3740 /* Beginning of buffer should also be regarded as bol. */
3741 CODING_SPEC_ISO_BOL (coding
) = 1;
3743 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3744 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3745 val
= Vcharset_revision_alist
;
3748 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3750 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3751 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3752 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3756 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3757 FLAGS[REG] can be one of below:
3758 integer CHARSET: CHARSET occupies register I,
3759 t: designate nothing to REG initially, but can be used
3761 list of integer, nil, or t: designate the first
3762 element (if integer) to REG initially, the remaining
3763 elements (if integer) is designated to REG on request,
3764 if an element is t, REG can be used by any charsets,
3765 nil: REG is never used. */
3766 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3767 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3768 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3769 for (i
= 0; i
< 4; i
++)
3771 if ((INTEGERP (flags
[i
])
3772 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
)))
3773 || (charset
= get_charset_id (flags
[i
])) >= 0)
3775 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3776 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3778 else if (EQ (flags
[i
], Qt
))
3780 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3782 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3784 else if (CONSP (flags
[i
]))
3789 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3790 if ((INTEGERP (XCAR (tail
))
3791 && (charset
= XINT (XCAR (tail
)),
3792 CHARSET_VALID_P (charset
)))
3793 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3795 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3796 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3799 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3801 while (CONSP (tail
))
3803 if ((INTEGERP (XCAR (tail
))
3804 && (charset
= XINT (XCAR (tail
)),
3805 CHARSET_VALID_P (charset
)))
3806 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3807 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3809 else if (EQ (XCAR (tail
), Qt
))
3815 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3817 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3818 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3821 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3823 /* REG 1 can be used only by locking shift in 7-bit env. */
3824 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3826 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3827 /* Without any shifting, only REG 0 and 1 can be used. */
3832 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3834 if (CHARSET_DEFINED_P (charset
)
3835 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3836 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3838 /* There exist some default graphic registers to be
3841 /* We had better avoid designating a charset of
3842 CHARS96 to REG 0 as far as possible. */
3843 if (CHARSET_CHARS (charset
) == 96)
3844 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3846 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3848 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3850 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3854 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3855 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3859 coding
->type
= coding_type_big5
;
3860 coding
->common_flags
3861 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3863 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3864 ? CODING_FLAG_BIG5_HKU
3865 : CODING_FLAG_BIG5_ETEN
);
3869 coding
->type
= coding_type_ccl
;
3870 coding
->common_flags
3871 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3873 val
= XVECTOR (coding_spec
)->contents
[4];
3875 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3877 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3879 goto label_invalid_coding_system
;
3881 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3882 val
= Fplist_get (plist
, Qvalid_codes
);
3887 for (; CONSP (val
); val
= XCDR (val
))
3891 && XINT (this) >= 0 && XINT (this) < 256)
3892 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3893 else if (CONSP (this)
3894 && INTEGERP (XCAR (this))
3895 && INTEGERP (XCDR (this)))
3897 int start
= XINT (XCAR (this));
3898 int end
= XINT (XCDR (this));
3900 if (start
>= 0 && start
<= end
&& end
< 256)
3901 while (start
<= end
)
3902 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3907 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3908 coding
->spec
.ccl
.cr_carryover
= 0;
3909 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
3913 coding
->type
= coding_type_raw_text
;
3917 goto label_invalid_coding_system
;
3921 label_invalid_coding_system
:
3922 coding
->type
= coding_type_no_conversion
;
3923 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3924 coding
->common_flags
= 0;
3925 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3926 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3927 return NILP (coding_system
) ? 0 : -1;
3930 /* Free memory blocks allocated for storing composition information. */
3933 coding_free_composition_data (coding
)
3934 struct coding_system
*coding
;
3936 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3940 /* Memory blocks are chained. At first, rewind to the first, then,
3941 free blocks one by one. */
3942 while (cmp_data
->prev
)
3943 cmp_data
= cmp_data
->prev
;
3946 next
= cmp_data
->next
;
3950 coding
->cmp_data
= NULL
;
3953 /* Set `char_offset' member of all memory blocks pointed by
3954 coding->cmp_data to POS. */
3957 coding_adjust_composition_offset (coding
, pos
)
3958 struct coding_system
*coding
;
3961 struct composition_data
*cmp_data
;
3963 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3964 cmp_data
->char_offset
= pos
;
3967 /* Setup raw-text or one of its subsidiaries in the structure
3968 coding_system CODING according to the already setup value eol_type
3969 in CODING. CODING should be setup for some coding system in
3973 setup_raw_text_coding_system (coding
)
3974 struct coding_system
*coding
;
3976 if (coding
->type
!= coding_type_raw_text
)
3978 coding
->symbol
= Qraw_text
;
3979 coding
->type
= coding_type_raw_text
;
3980 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3982 Lisp_Object subsidiaries
;
3983 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3985 if (VECTORP (subsidiaries
)
3986 && XVECTOR (subsidiaries
)->size
== 3)
3988 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3990 setup_coding_system (coding
->symbol
, coding
);
3995 /* Emacs has a mechanism to automatically detect a coding system if it
3996 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3997 it's impossible to distinguish some coding systems accurately
3998 because they use the same range of codes. So, at first, coding
3999 systems are categorized into 7, those are:
4001 o coding-category-emacs-mule
4003 The category for a coding system which has the same code range
4004 as Emacs' internal format. Assigned the coding-system (Lisp
4005 symbol) `emacs-mule' by default.
4007 o coding-category-sjis
4009 The category for a coding system which has the same code range
4010 as SJIS. Assigned the coding-system (Lisp
4011 symbol) `japanese-shift-jis' by default.
4013 o coding-category-iso-7
4015 The category for a coding system which has the same code range
4016 as ISO2022 of 7-bit environment. This doesn't use any locking
4017 shift and single shift functions. This can encode/decode all
4018 charsets. Assigned the coding-system (Lisp symbol)
4019 `iso-2022-7bit' by default.
4021 o coding-category-iso-7-tight
4023 Same as coding-category-iso-7 except that this can
4024 encode/decode only the specified charsets.
4026 o coding-category-iso-8-1
4028 The category for a coding system which has the same code range
4029 as ISO2022 of 8-bit environment and graphic plane 1 used only
4030 for DIMENSION1 charset. This doesn't use any locking shift
4031 and single shift functions. Assigned the coding-system (Lisp
4032 symbol) `iso-latin-1' by default.
4034 o coding-category-iso-8-2
4036 The category for a coding system which has the same code range
4037 as ISO2022 of 8-bit environment and graphic plane 1 used only
4038 for DIMENSION2 charset. This doesn't use any locking shift
4039 and single shift functions. Assigned the coding-system (Lisp
4040 symbol) `japanese-iso-8bit' by default.
4042 o coding-category-iso-7-else
4044 The category for a coding system which has the same code range
4045 as ISO2022 of 7-bit environment but uses locking shift or
4046 single shift functions. Assigned the coding-system (Lisp
4047 symbol) `iso-2022-7bit-lock' by default.
4049 o coding-category-iso-8-else
4051 The category for a coding system which has the same code range
4052 as ISO2022 of 8-bit environment but uses locking shift or
4053 single shift functions. Assigned the coding-system (Lisp
4054 symbol) `iso-2022-8bit-ss2' by default.
4056 o coding-category-big5
4058 The category for a coding system which has the same code range
4059 as BIG5. Assigned the coding-system (Lisp symbol)
4060 `cn-big5' by default.
4062 o coding-category-utf-8
4064 The category for a coding system which has the same code range
4065 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
4066 symbol) `utf-8' by default.
4068 o coding-category-utf-16-be
4070 The category for a coding system in which a text has an
4071 Unicode signature (cf. Unicode Standard) in the order of BIG
4072 endian at the head. Assigned the coding-system (Lisp symbol)
4073 `utf-16-be' by default.
4075 o coding-category-utf-16-le
4077 The category for a coding system in which a text has an
4078 Unicode signature (cf. Unicode Standard) in the order of
4079 LITTLE endian at the head. Assigned the coding-system (Lisp
4080 symbol) `utf-16-le' by default.
4082 o coding-category-ccl
4084 The category for a coding system of which encoder/decoder is
4085 written in CCL programs. The default value is nil, i.e., no
4086 coding system is assigned.
4088 o coding-category-binary
4090 The category for a coding system not categorized in any of the
4091 above. Assigned the coding-system (Lisp symbol)
4092 `no-conversion' by default.
4094 Each of them is a Lisp symbol and the value is an actual
4095 `coding-system' (this is also a Lisp symbol) assigned by a user.
4096 What Emacs does actually is to detect a category of coding system.
4097 Then, it uses a `coding-system' assigned to it. If Emacs can't
4098 decide a single possible category, it selects a category of the
4099 highest priority. Priorities of categories are also specified by a
4100 user in a Lisp variable `coding-category-list'.
4105 int ascii_skip_code
[256];
4107 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
4108 If it detects possible coding systems, return an integer in which
4109 appropriate flag bits are set. Flag bits are defined by macros
4110 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
4111 it should point the table `coding_priorities'. In that case, only
4112 the flag bit for a coding system of the highest priority is set in
4113 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
4114 range 0x80..0x9F are in multibyte form.
4116 How many ASCII characters are at the head is returned as *SKIP. */
4119 detect_coding_mask (source
, src_bytes
, priorities
, skip
, multibytep
)
4120 unsigned char *source
;
4121 int src_bytes
, *priorities
, *skip
;
4124 register unsigned char c
;
4125 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
4126 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
4129 /* At first, skip all ASCII characters and control characters except
4130 for three ISO2022 specific control characters. */
4131 ascii_skip_code
[ISO_CODE_SO
] = 0;
4132 ascii_skip_code
[ISO_CODE_SI
] = 0;
4133 ascii_skip_code
[ISO_CODE_ESC
] = 0;
4135 label_loop_detect_coding
:
4136 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
4137 *skip
= src
- source
;
4140 /* We found nothing other than ASCII. There's nothing to do. */
4144 /* The text seems to be encoded in some multilingual coding system.
4145 Now, try to find in which coding system the text is encoded. */
4148 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
4149 /* C is an ISO2022 specific control code of C0. */
4150 mask
= detect_coding_iso2022 (src
, src_end
, multibytep
);
4153 /* No valid ISO2022 code follows C. Try again. */
4155 if (c
== ISO_CODE_ESC
)
4156 ascii_skip_code
[ISO_CODE_ESC
] = 1;
4158 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
4159 goto label_loop_detect_coding
;
4163 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4165 if (mask
& priorities
[i
])
4166 return priorities
[i
];
4168 return CODING_CATEGORY_MASK_RAW_TEXT
;
4175 if (multibytep
&& c
== LEADING_CODE_8_BIT_CONTROL
)
4180 /* C is the first byte of SJIS character code,
4181 or a leading-code of Emacs' internal format (emacs-mule),
4182 or the first byte of UTF-16. */
4183 try = (CODING_CATEGORY_MASK_SJIS
4184 | CODING_CATEGORY_MASK_EMACS_MULE
4185 | CODING_CATEGORY_MASK_UTF_16_BE
4186 | CODING_CATEGORY_MASK_UTF_16_LE
);
4188 /* Or, if C is a special latin extra code,
4189 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4190 or is an ISO2022 control-sequence-introducer (CSI),
4191 we should also consider the possibility of ISO2022 codings. */
4192 if ((VECTORP (Vlatin_extra_code_table
)
4193 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
4194 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
4195 || (c
== ISO_CODE_CSI
4198 || ((*src
== '0' || *src
== '1' || *src
== '2')
4199 && src
+ 1 < src_end
4200 && src
[1] == ']')))))
4201 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4202 | CODING_CATEGORY_MASK_ISO_8BIT
);
4205 /* C is a character of ISO2022 in graphic plane right,
4206 or a SJIS's 1-byte character code (i.e. JISX0201),
4207 or the first byte of BIG5's 2-byte code,
4208 or the first byte of UTF-8/16. */
4209 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4210 | CODING_CATEGORY_MASK_ISO_8BIT
4211 | CODING_CATEGORY_MASK_SJIS
4212 | CODING_CATEGORY_MASK_BIG5
4213 | CODING_CATEGORY_MASK_UTF_8
4214 | CODING_CATEGORY_MASK_UTF_16_BE
4215 | CODING_CATEGORY_MASK_UTF_16_LE
);
4217 /* Or, we may have to consider the possibility of CCL. */
4218 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4219 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4220 ->spec
.ccl
.valid_codes
)[c
])
4221 try |= CODING_CATEGORY_MASK_CCL
;
4224 utf16_examined_p
= iso2022_examined_p
= 0;
4227 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4229 if (!iso2022_examined_p
4230 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
4232 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4233 iso2022_examined_p
= 1;
4235 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
4236 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4237 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
4238 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4239 else if (!utf16_examined_p
4240 && (priorities
[i
] & try &
4241 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
4243 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4244 utf16_examined_p
= 1;
4246 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
4247 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4248 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
4249 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4250 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
4251 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4252 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
4253 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
4254 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
4255 mask
|= CODING_CATEGORY_MASK_BINARY
;
4256 if (mask
& priorities
[i
])
4257 return priorities
[i
];
4259 return CODING_CATEGORY_MASK_RAW_TEXT
;
4261 if (try & CODING_CATEGORY_MASK_ISO
)
4262 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4263 if (try & CODING_CATEGORY_MASK_SJIS
)
4264 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4265 if (try & CODING_CATEGORY_MASK_BIG5
)
4266 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4267 if (try & CODING_CATEGORY_MASK_UTF_8
)
4268 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4269 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
4270 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4271 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
4272 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4273 if (try & CODING_CATEGORY_MASK_CCL
)
4274 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4276 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
4279 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4280 The information of the detected coding system is set in CODING. */
4283 detect_coding (coding
, src
, src_bytes
)
4284 struct coding_system
*coding
;
4285 const unsigned char *src
;
4292 val
= Vcoding_category_list
;
4293 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
,
4294 coding
->src_multibyte
);
4295 coding
->heading_ascii
= skip
;
4299 /* We found a single coding system of the highest priority in MASK. */
4301 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
4303 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
4305 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[idx
]);
4307 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4311 tmp
= Fget (val
, Qeol_type
);
4313 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
4316 /* Setup this new coding system while preserving some slots. */
4318 int src_multibyte
= coding
->src_multibyte
;
4319 int dst_multibyte
= coding
->dst_multibyte
;
4321 setup_coding_system (val
, coding
);
4322 coding
->src_multibyte
= src_multibyte
;
4323 coding
->dst_multibyte
= dst_multibyte
;
4324 coding
->heading_ascii
= skip
;
4328 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
4329 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4330 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4332 How many non-eol characters are at the head is returned as *SKIP. */
4334 #define MAX_EOL_CHECK_COUNT 3
4337 detect_eol_type (source
, src_bytes
, skip
)
4338 unsigned char *source
;
4339 int src_bytes
, *skip
;
4341 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4343 int total
= 0; /* How many end-of-lines are found so far. */
4344 int eol_type
= CODING_EOL_UNDECIDED
;
4349 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4352 if (c
== '\n' || c
== '\r')
4355 *skip
= src
- 1 - source
;
4358 this_eol_type
= CODING_EOL_LF
;
4359 else if (src
>= src_end
|| *src
!= '\n')
4360 this_eol_type
= CODING_EOL_CR
;
4362 this_eol_type
= CODING_EOL_CRLF
, src
++;
4364 if (eol_type
== CODING_EOL_UNDECIDED
)
4365 /* This is the first end-of-line. */
4366 eol_type
= this_eol_type
;
4367 else if (eol_type
!= this_eol_type
)
4369 /* The found type is different from what found before. */
4370 eol_type
= CODING_EOL_INCONSISTENT
;
4377 *skip
= src_end
- source
;
4381 /* Like detect_eol_type, but detect EOL type in 2-octet
4382 big-endian/little-endian format for coding systems utf-16-be and
4386 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
4387 unsigned char *source
;
4388 int src_bytes
, *skip
, big_endian_p
;
4390 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4391 unsigned int c1
, c2
;
4392 int total
= 0; /* How many end-of-lines are found so far. */
4393 int eol_type
= CODING_EOL_UNDECIDED
;
4404 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4406 c1
= (src
[msb
] << 8) | (src
[lsb
]);
4409 if (c1
== '\n' || c1
== '\r')
4412 *skip
= src
- 2 - source
;
4416 this_eol_type
= CODING_EOL_LF
;
4420 if ((src
+ 1) >= src_end
)
4422 this_eol_type
= CODING_EOL_CR
;
4426 c2
= (src
[msb
] << 8) | (src
[lsb
]);
4428 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
4430 this_eol_type
= CODING_EOL_CR
;
4434 if (eol_type
== CODING_EOL_UNDECIDED
)
4435 /* This is the first end-of-line. */
4436 eol_type
= this_eol_type
;
4437 else if (eol_type
!= this_eol_type
)
4439 /* The found type is different from what found before. */
4440 eol_type
= CODING_EOL_INCONSISTENT
;
4447 *skip
= src_end
- source
;
4451 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4452 is encoded. If it detects an appropriate format of end-of-line, it
4453 sets the information in *CODING. */
4456 detect_eol (coding
, src
, src_bytes
)
4457 struct coding_system
*coding
;
4458 const unsigned char *src
;
4465 switch (coding
->category_idx
)
4467 case CODING_CATEGORY_IDX_UTF_16_BE
:
4468 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
4470 case CODING_CATEGORY_IDX_UTF_16_LE
:
4471 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
4474 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
4478 if (coding
->heading_ascii
> skip
)
4479 coding
->heading_ascii
= skip
;
4481 skip
= coding
->heading_ascii
;
4483 if (eol_type
== CODING_EOL_UNDECIDED
)
4485 if (eol_type
== CODING_EOL_INCONSISTENT
)
4488 /* This code is suppressed until we find a better way to
4489 distinguish raw text file and binary file. */
4491 /* If we have already detected that the coding is raw-text, the
4492 coding should actually be no-conversion. */
4493 if (coding
->type
== coding_type_raw_text
)
4495 setup_coding_system (Qno_conversion
, coding
);
4498 /* Else, let's decode only text code anyway. */
4500 eol_type
= CODING_EOL_LF
;
4503 val
= Fget (coding
->symbol
, Qeol_type
);
4504 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4506 int src_multibyte
= coding
->src_multibyte
;
4507 int dst_multibyte
= coding
->dst_multibyte
;
4508 struct composition_data
*cmp_data
= coding
->cmp_data
;
4510 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
4511 coding
->src_multibyte
= src_multibyte
;
4512 coding
->dst_multibyte
= dst_multibyte
;
4513 coding
->heading_ascii
= skip
;
4514 coding
->cmp_data
= cmp_data
;
4518 #define CONVERSION_BUFFER_EXTRA_ROOM 256
4520 #define DECODING_BUFFER_MAG(coding) \
4521 (coding->type == coding_type_iso2022 \
4523 : (coding->type == coding_type_ccl \
4524 ? coding->spec.ccl.decoder.buf_magnification \
4527 /* Return maximum size (bytes) of a buffer enough for decoding
4528 SRC_BYTES of text encoded in CODING. */
4531 decoding_buffer_size (coding
, src_bytes
)
4532 struct coding_system
*coding
;
4535 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
4536 + CONVERSION_BUFFER_EXTRA_ROOM
);
4539 /* Return maximum size (bytes) of a buffer enough for encoding
4540 SRC_BYTES of text to CODING. */
4543 encoding_buffer_size (coding
, src_bytes
)
4544 struct coding_system
*coding
;
4549 if (coding
->type
== coding_type_ccl
)
4551 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
4552 if (coding
->eol_type
== CODING_EOL_CRLF
)
4555 else if (CODING_REQUIRE_ENCODING (coding
))
4560 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
4563 /* Working buffer for code conversion. */
4564 struct conversion_buffer
4566 int size
; /* size of data. */
4567 int on_stack
; /* 1 if allocated by alloca. */
4568 unsigned char *data
;
4571 /* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4572 #define allocate_conversion_buffer(buf, len) \
4574 if (len < MAX_ALLOCA) \
4576 buf.data = (unsigned char *) alloca (len); \
4581 buf.data = (unsigned char *) xmalloc (len); \
4587 /* Double the allocated memory for *BUF. */
4589 extend_conversion_buffer (buf
)
4590 struct conversion_buffer
*buf
;
4594 unsigned char *save
= buf
->data
;
4595 buf
->data
= (unsigned char *) xmalloc (buf
->size
* 2);
4596 bcopy (save
, buf
->data
, buf
->size
);
4601 buf
->data
= (unsigned char *) xrealloc (buf
->data
, buf
->size
* 2);
4606 /* Free the allocated memory for BUF if it is not on stack. */
4608 free_conversion_buffer (buf
)
4609 struct conversion_buffer
*buf
;
4616 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
4617 struct coding_system
*coding
;
4618 unsigned char *source
, *destination
;
4619 int src_bytes
, dst_bytes
, encodep
;
4621 struct ccl_program
*ccl
4622 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
4623 unsigned char *dst
= destination
;
4625 ccl
->suppress_error
= coding
->suppress_error
;
4626 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
4629 /* On encoding, EOL format is converted within ccl_driver. For
4630 that, setup proper information in the structure CCL. */
4631 ccl
->eol_type
= coding
->eol_type
;
4632 if (ccl
->eol_type
==CODING_EOL_UNDECIDED
)
4633 ccl
->eol_type
= CODING_EOL_LF
;
4634 ccl
->cr_consumed
= coding
->spec
.ccl
.cr_carryover
;
4635 ccl
->eight_bit_control
= coding
->dst_multibyte
;
4638 ccl
->eight_bit_control
= 1;
4639 ccl
->multibyte
= coding
->src_multibyte
;
4640 if (coding
->spec
.ccl
.eight_bit_carryover
[0] != 0)
4642 /* Move carryover bytes to DESTINATION. */
4643 unsigned char *p
= coding
->spec
.ccl
.eight_bit_carryover
;
4646 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4648 dst_bytes
-= dst
- destination
;
4651 coding
->produced
= (ccl_driver (ccl
, source
, dst
, src_bytes
, dst_bytes
,
4652 &(coding
->consumed
))
4653 + dst
- destination
);
4657 coding
->produced_char
= coding
->produced
;
4658 coding
->spec
.ccl
.cr_carryover
= ccl
->cr_consumed
;
4660 else if (!ccl
->eight_bit_control
)
4662 /* The produced bytes forms a valid multibyte sequence. */
4663 coding
->produced_char
4664 = multibyte_chars_in_text (destination
, coding
->produced
);
4665 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4669 /* On decoding, the destination should always multibyte. But,
4670 CCL program might have been generated an invalid multibyte
4671 sequence. Here we make such a sequence valid as
4674 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
4676 if ((coding
->consumed
< src_bytes
4677 || !ccl
->last_block
)
4678 && coding
->produced
>= 1
4679 && destination
[coding
->produced
- 1] >= 0x80)
4681 /* We should not convert the tailing 8-bit codes to
4682 multibyte form even if they doesn't form a valid
4683 multibyte sequence. They may form a valid sequence in
4687 if (destination
[coding
->produced
- 1] < 0xA0)
4689 else if (coding
->produced
>= 2)
4691 if (destination
[coding
->produced
- 2] >= 0x80)
4693 if (destination
[coding
->produced
- 2] < 0xA0)
4695 else if (coding
->produced
>= 3
4696 && destination
[coding
->produced
- 3] >= 0x80
4697 && destination
[coding
->produced
- 3] < 0xA0)
4703 BCOPY_SHORT (destination
+ coding
->produced
- carryover
,
4704 coding
->spec
.ccl
.eight_bit_carryover
,
4706 coding
->spec
.ccl
.eight_bit_carryover
[carryover
] = 0;
4707 coding
->produced
-= carryover
;
4710 coding
->produced
= str_as_multibyte (destination
, bytes
,
4712 &(coding
->produced_char
));
4715 switch (ccl
->status
)
4717 case CCL_STAT_SUSPEND_BY_SRC
:
4718 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
4720 case CCL_STAT_SUSPEND_BY_DST
:
4721 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
4724 case CCL_STAT_INVALID_CMD
:
4725 coding
->result
= CODING_FINISH_INTERRUPT
;
4728 coding
->result
= CODING_FINISH_NORMAL
;
4731 return coding
->result
;
4734 /* Decode EOL format of the text at PTR of BYTES length destructively
4735 according to CODING->eol_type. This is called after the CCL
4736 program produced a decoded text at PTR. If we do CRLF->LF
4737 conversion, update CODING->produced and CODING->produced_char. */
4740 decode_eol_post_ccl (coding
, ptr
, bytes
)
4741 struct coding_system
*coding
;
4745 Lisp_Object val
, saved_coding_symbol
;
4746 unsigned char *pend
= ptr
+ bytes
;
4749 /* Remember the current coding system symbol. We set it back when
4750 an inconsistent EOL is found so that `last-coding-system-used' is
4751 set to the coding system that doesn't specify EOL conversion. */
4752 saved_coding_symbol
= coding
->symbol
;
4754 coding
->spec
.ccl
.cr_carryover
= 0;
4755 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4757 /* Here, to avoid the call of setup_coding_system, we directly
4758 call detect_eol_type. */
4759 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
4760 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
4761 coding
->eol_type
= CODING_EOL_LF
;
4762 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4764 val
= Fget (coding
->symbol
, Qeol_type
);
4765 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4766 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
4768 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4771 if (coding
->eol_type
== CODING_EOL_LF
4772 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
4774 /* We have nothing to do. */
4777 else if (coding
->eol_type
== CODING_EOL_CRLF
)
4779 unsigned char *pstart
= ptr
, *p
= ptr
;
4781 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4782 && *(pend
- 1) == '\r')
4784 /* If the last character is CR, we can't handle it here
4785 because LF will be in the not-yet-decoded source text.
4786 Record that the CR is not yet processed. */
4787 coding
->spec
.ccl
.cr_carryover
= 1;
4789 coding
->produced_char
--;
4796 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4803 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4804 goto undo_eol_conversion
;
4808 else if (*ptr
== '\n'
4809 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4810 goto undo_eol_conversion
;
4815 undo_eol_conversion
:
4816 /* We have faced with inconsistent EOL format at PTR.
4817 Convert all LFs before PTR back to CRLFs. */
4818 for (p
--, ptr
--; p
>= pstart
; p
--)
4821 *ptr
-- = '\n', *ptr
-- = '\r';
4825 /* If carryover is recorded, cancel it because we don't
4826 convert CRLF anymore. */
4827 if (coding
->spec
.ccl
.cr_carryover
)
4829 coding
->spec
.ccl
.cr_carryover
= 0;
4831 coding
->produced_char
++;
4835 coding
->eol_type
= CODING_EOL_LF
;
4836 coding
->symbol
= saved_coding_symbol
;
4840 /* As each two-byte sequence CRLF was converted to LF, (PEND
4841 - P) is the number of deleted characters. */
4842 coding
->produced
-= pend
- p
;
4843 coding
->produced_char
-= pend
- p
;
4846 else /* i.e. coding->eol_type == CODING_EOL_CR */
4848 unsigned char *p
= ptr
;
4850 for (; ptr
< pend
; ptr
++)
4854 else if (*ptr
== '\n'
4855 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4857 for (; p
< ptr
; p
++)
4863 coding
->eol_type
= CODING_EOL_LF
;
4864 coding
->symbol
= saved_coding_symbol
;
4870 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4871 decoding, it may detect coding system and format of end-of-line if
4872 those are not yet decided. The source should be unibyte, the
4873 result is multibyte if CODING->dst_multibyte is nonzero, else
4877 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4878 struct coding_system
*coding
;
4879 const unsigned char *source
;
4880 unsigned char *destination
;
4881 int src_bytes
, dst_bytes
;
4885 if (coding
->type
== coding_type_undecided
)
4886 detect_coding (coding
, source
, src_bytes
);
4888 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4889 && coding
->type
!= coding_type_ccl
)
4891 detect_eol (coding
, source
, src_bytes
);
4892 /* We had better recover the original eol format if we
4893 encounter an inconsistent eol format while decoding. */
4894 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4897 coding
->produced
= coding
->produced_char
= 0;
4898 coding
->consumed
= coding
->consumed_char
= 0;
4900 coding
->result
= CODING_FINISH_NORMAL
;
4902 switch (coding
->type
)
4904 case coding_type_sjis
:
4905 decode_coding_sjis_big5 (coding
, source
, destination
,
4906 src_bytes
, dst_bytes
, 1);
4909 case coding_type_iso2022
:
4910 decode_coding_iso2022 (coding
, source
, destination
,
4911 src_bytes
, dst_bytes
);
4914 case coding_type_big5
:
4915 decode_coding_sjis_big5 (coding
, source
, destination
,
4916 src_bytes
, dst_bytes
, 0);
4919 case coding_type_emacs_mule
:
4920 decode_coding_emacs_mule (coding
, source
, destination
,
4921 src_bytes
, dst_bytes
);
4924 case coding_type_ccl
:
4925 if (coding
->spec
.ccl
.cr_carryover
)
4927 /* Put the CR which was not processed by the previous call
4928 of decode_eol_post_ccl in DESTINATION. It will be
4929 decoded together with the following LF by the call to
4930 decode_eol_post_ccl below. */
4931 *destination
= '\r';
4933 coding
->produced_char
++;
4935 extra
= coding
->spec
.ccl
.cr_carryover
;
4937 ccl_coding_driver (coding
, source
, destination
+ extra
,
4938 src_bytes
, dst_bytes
, 0);
4939 if (coding
->eol_type
!= CODING_EOL_LF
)
4941 coding
->produced
+= extra
;
4942 coding
->produced_char
+= extra
;
4943 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4948 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4951 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4952 && coding
->mode
& CODING_MODE_LAST_BLOCK
4953 && coding
->consumed
== src_bytes
)
4954 coding
->result
= CODING_FINISH_NORMAL
;
4956 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4957 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4959 const unsigned char *src
= source
+ coding
->consumed
;
4960 unsigned char *dst
= destination
+ coding
->produced
;
4962 src_bytes
-= coding
->consumed
;
4964 if (COMPOSING_P (coding
))
4965 DECODE_COMPOSITION_END ('1');
4969 dst
+= CHAR_STRING (c
, dst
);
4970 coding
->produced_char
++;
4972 coding
->consumed
= coding
->consumed_char
= src
- source
;
4973 coding
->produced
= dst
- destination
;
4974 coding
->result
= CODING_FINISH_NORMAL
;
4977 if (!coding
->dst_multibyte
)
4979 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4980 coding
->produced_char
= coding
->produced
;
4983 return coding
->result
;
4986 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4987 multibyteness of the source is CODING->src_multibyte, the
4988 multibyteness of the result is always unibyte. */
4991 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4992 struct coding_system
*coding
;
4993 const unsigned char *source
;
4994 unsigned char *destination
;
4995 int src_bytes
, dst_bytes
;
4997 coding
->produced
= coding
->produced_char
= 0;
4998 coding
->consumed
= coding
->consumed_char
= 0;
5000 coding
->result
= CODING_FINISH_NORMAL
;
5001 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5002 coding
->eol_type
= CODING_EOL_LF
;
5004 switch (coding
->type
)
5006 case coding_type_sjis
:
5007 encode_coding_sjis_big5 (coding
, source
, destination
,
5008 src_bytes
, dst_bytes
, 1);
5011 case coding_type_iso2022
:
5012 encode_coding_iso2022 (coding
, source
, destination
,
5013 src_bytes
, dst_bytes
);
5016 case coding_type_big5
:
5017 encode_coding_sjis_big5 (coding
, source
, destination
,
5018 src_bytes
, dst_bytes
, 0);
5021 case coding_type_emacs_mule
:
5022 encode_coding_emacs_mule (coding
, source
, destination
,
5023 src_bytes
, dst_bytes
);
5026 case coding_type_ccl
:
5027 ccl_coding_driver (coding
, source
, destination
,
5028 src_bytes
, dst_bytes
, 1);
5032 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
5035 if (coding
->mode
& CODING_MODE_LAST_BLOCK
5036 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
5038 const unsigned char *src
= source
+ coding
->consumed
;
5039 unsigned char *dst
= destination
+ coding
->produced
;
5041 if (coding
->type
== coding_type_iso2022
)
5042 ENCODE_RESET_PLANE_AND_REGISTER
;
5043 if (COMPOSING_P (coding
))
5044 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
5045 if (coding
->consumed
< src_bytes
)
5047 int len
= src_bytes
- coding
->consumed
;
5049 BCOPY_SHORT (src
, dst
, len
);
5050 if (coding
->src_multibyte
)
5051 len
= str_as_unibyte (dst
, len
);
5053 coding
->consumed
= src_bytes
;
5055 coding
->produced
= coding
->produced_char
= dst
- destination
;
5056 coding
->result
= CODING_FINISH_NORMAL
;
5059 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
5060 && coding
->consumed
== src_bytes
)
5061 coding
->result
= CODING_FINISH_NORMAL
;
5063 return coding
->result
;
5066 /* Scan text in the region between *BEG and *END (byte positions),
5067 skip characters which we don't have to decode by coding system
5068 CODING at the head and tail, then set *BEG and *END to the region
5069 of the text we actually have to convert. The caller should move
5070 the gap out of the region in advance if the region is from a
5073 If STR is not NULL, *BEG and *END are indices into STR. */
5076 shrink_decoding_region (beg
, end
, coding
, str
)
5078 struct coding_system
*coding
;
5081 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
5083 Lisp_Object translation_table
;
5085 if (coding
->type
== coding_type_ccl
5086 || coding
->type
== coding_type_undecided
5087 || coding
->eol_type
!= CODING_EOL_LF
5088 || !NILP (coding
->post_read_conversion
)
5089 || coding
->composing
!= COMPOSITION_DISABLED
)
5091 /* We can't skip any data. */
5094 if (coding
->type
== coding_type_no_conversion
5095 || coding
->type
== coding_type_raw_text
5096 || coding
->type
== coding_type_emacs_mule
)
5098 /* We need no conversion, but don't have to skip any data here.
5099 Decoding routine handles them effectively anyway. */
5103 translation_table
= coding
->translation_table_for_decode
;
5104 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5105 translation_table
= Vstandard_translation_table_for_decode
;
5106 if (CHAR_TABLE_P (translation_table
))
5109 for (i
= 0; i
< 128; i
++)
5110 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5113 /* Some ASCII character should be translated. We give up
5118 if (coding
->heading_ascii
>= 0)
5119 /* Detection routine has already found how much we can skip at the
5121 *beg
+= coding
->heading_ascii
;
5125 begp_orig
= begp
= str
+ *beg
;
5126 endp_orig
= endp
= str
+ *end
;
5130 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5131 endp_orig
= endp
= begp
+ *end
- *beg
;
5134 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5135 || coding
->eol_type
== CODING_EOL_CRLF
);
5137 switch (coding
->type
)
5139 case coding_type_sjis
:
5140 case coding_type_big5
:
5141 /* We can skip all ASCII characters at the head. */
5142 if (coding
->heading_ascii
< 0)
5145 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
5147 while (begp
< endp
&& *begp
< 0x80) begp
++;
5149 /* We can skip all ASCII characters at the tail except for the
5150 second byte of SJIS or BIG5 code. */
5152 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
5154 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5155 /* Do not consider LF as ascii if preceded by CR, since that
5156 confuses eol decoding. */
5157 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5159 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
5163 case coding_type_iso2022
:
5164 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5165 /* We can't skip any data. */
5167 if (coding
->heading_ascii
< 0)
5169 /* We can skip all ASCII characters at the head except for a
5170 few control codes. */
5171 while (begp
< endp
&& (c
= *begp
) < 0x80
5172 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
5173 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
5174 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
5177 switch (coding
->category_idx
)
5179 case CODING_CATEGORY_IDX_ISO_8_1
:
5180 case CODING_CATEGORY_IDX_ISO_8_2
:
5181 /* We can skip all ASCII characters at the tail. */
5183 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
5185 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5186 /* Do not consider LF as ascii if preceded by CR, since that
5187 confuses eol decoding. */
5188 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5192 case CODING_CATEGORY_IDX_ISO_7
:
5193 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
5195 /* We can skip all characters at the tail except for 8-bit
5196 codes and ESC and the following 2-byte at the tail. */
5197 unsigned char *eight_bit
= NULL
;
5201 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
5203 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5208 && (c
= endp
[-1]) != ISO_CODE_ESC
)
5210 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5213 /* Do not consider LF as ascii if preceded by CR, since that
5214 confuses eol decoding. */
5215 if (begp
< endp
&& endp
< endp_orig
5216 && endp
[-1] == '\r' && endp
[0] == '\n')
5218 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
5220 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
5221 /* This is an ASCII designation sequence. We can
5222 surely skip the tail. But, if we have
5223 encountered an 8-bit code, skip only the codes
5225 endp
= eight_bit
? eight_bit
: endp
+ 2;
5227 /* Hmmm, we can't skip the tail. */
5239 *beg
+= begp
- begp_orig
;
5240 *end
+= endp
- endp_orig
;
5244 /* Like shrink_decoding_region but for encoding. */
5247 shrink_encoding_region (beg
, end
, coding
, str
)
5249 struct coding_system
*coding
;
5252 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
5254 Lisp_Object translation_table
;
5256 if (coding
->type
== coding_type_ccl
5257 || coding
->eol_type
== CODING_EOL_CRLF
5258 || coding
->eol_type
== CODING_EOL_CR
5259 || (coding
->cmp_data
&& coding
->cmp_data
->used
> 0))
5261 /* We can't skip any data. */
5264 if (coding
->type
== coding_type_no_conversion
5265 || coding
->type
== coding_type_raw_text
5266 || coding
->type
== coding_type_emacs_mule
5267 || coding
->type
== coding_type_undecided
)
5269 /* We need no conversion, but don't have to skip any data here.
5270 Encoding routine handles them effectively anyway. */
5274 translation_table
= coding
->translation_table_for_encode
;
5275 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5276 translation_table
= Vstandard_translation_table_for_encode
;
5277 if (CHAR_TABLE_P (translation_table
))
5280 for (i
= 0; i
< 128; i
++)
5281 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5284 /* Some ASCII character should be translated. We give up
5291 begp_orig
= begp
= str
+ *beg
;
5292 endp_orig
= endp
= str
+ *end
;
5296 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5297 endp_orig
= endp
= begp
+ *end
- *beg
;
5300 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5301 || coding
->eol_type
== CODING_EOL_CRLF
);
5303 /* Here, we don't have to check coding->pre_write_conversion because
5304 the caller is expected to have handled it already. */
5305 switch (coding
->type
)
5307 case coding_type_iso2022
:
5308 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5309 /* We can't skip any data. */
5311 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
5313 unsigned char *bol
= begp
;
5314 while (begp
< endp
&& *begp
< 0x80)
5317 if (begp
[-1] == '\n')
5321 goto label_skip_tail
;
5325 case coding_type_sjis
:
5326 case coding_type_big5
:
5327 /* We can skip all ASCII characters at the head and tail. */
5329 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
5331 while (begp
< endp
&& *begp
< 0x80) begp
++;
5334 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
5336 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
5343 *beg
+= begp
- begp_orig
;
5344 *end
+= endp
- endp_orig
;
5348 /* As shrinking conversion region requires some overhead, we don't try
5349 shrinking if the length of conversion region is less than this
5351 static int shrink_conversion_region_threshhold
= 1024;
5353 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5355 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5357 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5358 else shrink_decoding_region (beg, end, coding, str); \
5362 /* ARG is (CODING BUFFER ...) where CODING is what to be set in
5363 Vlast_coding_system_used and the remaining elements are buffers to
5366 code_convert_region_unwind (arg
)
5369 struct gcpro gcpro1
;
5372 inhibit_pre_post_conversion
= 0;
5373 Vlast_coding_system_used
= XCAR (arg
);
5374 for (arg
= XCDR (arg
); ! NILP (arg
); arg
= XCDR (arg
))
5375 Fkill_buffer (XCAR (arg
));
5381 /* Store information about all compositions in the range FROM and TO
5382 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5383 buffer or a string, defaults to the current buffer. */
5386 coding_save_composition (coding
, from
, to
, obj
)
5387 struct coding_system
*coding
;
5394 if (coding
->composing
== COMPOSITION_DISABLED
)
5396 if (!coding
->cmp_data
)
5397 coding_allocate_composition_data (coding
, from
);
5398 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
5402 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
5405 coding
->composing
= COMPOSITION_NO
;
5408 if (COMPOSITION_VALID_P (start
, end
, prop
))
5410 enum composition_method method
= COMPOSITION_METHOD (prop
);
5411 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
5412 >= COMPOSITION_DATA_SIZE
)
5413 coding_allocate_composition_data (coding
, from
);
5414 /* For relative composition, we remember start and end
5415 positions, for the other compositions, we also remember
5417 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
5418 if (method
!= COMPOSITION_RELATIVE
)
5420 /* We must store a*/
5421 Lisp_Object val
, ch
;
5423 val
= COMPOSITION_COMPONENTS (prop
);
5427 ch
= XCAR (val
), val
= XCDR (val
);
5428 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5430 else if (VECTORP (val
) || STRINGP (val
))
5432 int len
= (VECTORP (val
)
5433 ? XVECTOR (val
)->size
: SCHARS (val
));
5435 for (i
= 0; i
< len
; i
++)
5438 ? Faref (val
, make_number (i
))
5439 : XVECTOR (val
)->contents
[i
]);
5440 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5443 else /* INTEGERP (val) */
5444 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
5446 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
5451 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
5454 /* Make coding->cmp_data point to the first memory block. */
5455 while (coding
->cmp_data
->prev
)
5456 coding
->cmp_data
= coding
->cmp_data
->prev
;
5457 coding
->cmp_data_start
= 0;
5460 /* Reflect the saved information about compositions to OBJ.
5461 CODING->cmp_data points to a memory block for the information. OBJ
5462 is a buffer or a string, defaults to the current buffer. */
5465 coding_restore_composition (coding
, obj
)
5466 struct coding_system
*coding
;
5469 struct composition_data
*cmp_data
= coding
->cmp_data
;
5474 while (cmp_data
->prev
)
5475 cmp_data
= cmp_data
->prev
;
5481 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
5482 i
+= cmp_data
->data
[i
])
5484 int *data
= cmp_data
->data
+ i
;
5485 enum composition_method method
= (enum composition_method
) data
[3];
5486 Lisp_Object components
;
5488 if (data
[0] < 0 || i
+ data
[0] > cmp_data
->used
)
5489 /* Invalid composition data. */
5492 if (method
== COMPOSITION_RELATIVE
)
5496 int len
= data
[0] - 4, j
;
5497 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
5499 if (method
== COMPOSITION_WITH_RULE_ALTCHARS
5503 /* Invalid composition data. */
5505 for (j
= 0; j
< len
; j
++)
5506 args
[j
] = make_number (data
[4 + j
]);
5507 components
= (method
== COMPOSITION_WITH_ALTCHARS
5508 ? Fstring (len
, args
)
5509 : Fvector (len
, args
));
5511 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
5513 cmp_data
= cmp_data
->next
;
5517 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
5518 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5519 coding system CODING, and return the status code of code conversion
5520 (currently, this value has no meaning).
5522 How many characters (and bytes) are converted to how many
5523 characters (and bytes) are recorded in members of the structure
5526 If REPLACE is nonzero, we do various things as if the original text
5527 is deleted and a new text is inserted. See the comments in
5528 replace_range (insdel.c) to know what we are doing.
5530 If REPLACE is zero, it is assumed that the source text is unibyte.
5531 Otherwise, it is assumed that the source text is multibyte. */
5534 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
5535 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
5536 struct coding_system
*coding
;
5538 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
5539 int nchars_del
= 0, nbytes_del
= 0;
5540 int require
, inserted
, inserted_byte
;
5541 int head_skip
, tail_skip
, total_skip
= 0;
5542 Lisp_Object saved_coding_symbol
;
5544 unsigned char *src
, *dst
;
5545 Lisp_Object deletion
;
5546 int orig_point
= PT
, orig_len
= len
;
5548 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
5551 saved_coding_symbol
= coding
->symbol
;
5553 if (from
< PT
&& PT
< to
)
5555 TEMP_SET_PT_BOTH (from
, from_byte
);
5561 int saved_from
= from
;
5562 int saved_inhibit_modification_hooks
;
5564 prepare_to_modify_buffer (from
, to
, &from
);
5565 if (saved_from
!= from
)
5568 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
5569 len_byte
= to_byte
- from_byte
;
5572 /* The code conversion routine can not preserve text properties
5573 for now. So, we must remove all text properties in the
5574 region. Here, we must suppress all modification hooks. */
5575 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
5576 inhibit_modification_hooks
= 1;
5577 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
5578 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
5581 coding
->heading_ascii
= 0;
5583 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
5585 /* We must detect encoding of text and eol format. */
5587 if (from
< GPT
&& to
> GPT
)
5588 move_gap_both (from
, from_byte
);
5589 if (coding
->type
== coding_type_undecided
)
5591 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5592 if (coding
->type
== coding_type_undecided
)
5594 /* It seems that the text contains only ASCII, but we
5595 should not leave it undecided because the deeper
5596 decoding routine (decode_coding) tries to detect the
5597 encodings again in vain. */
5598 coding
->type
= coding_type_emacs_mule
;
5599 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5600 /* As emacs-mule decoder will handle composition, we
5601 need this setting to allocate coding->cmp_data
5603 coding
->composing
= COMPOSITION_NO
;
5606 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5607 && coding
->type
!= coding_type_ccl
)
5609 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5610 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5611 coding
->eol_type
= CODING_EOL_LF
;
5612 /* We had better recover the original eol format if we
5613 encounter an inconsistent eol format while decoding. */
5614 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5618 /* Now we convert the text. */
5620 /* For encoding, we must process pre-write-conversion in advance. */
5621 if (! inhibit_pre_post_conversion
5623 && SYMBOLP (coding
->pre_write_conversion
)
5624 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
5626 /* The function in pre-write-conversion may put a new text in a
5628 struct buffer
*prev
= current_buffer
;
5631 record_unwind_protect (code_convert_region_unwind
,
5632 Fcons (Vlast_coding_system_used
, Qnil
));
5633 /* We should not call any more pre-write/post-read-conversion
5634 functions while this pre-write-conversion is running. */
5635 inhibit_pre_post_conversion
= 1;
5636 call2 (coding
->pre_write_conversion
,
5637 make_number (from
), make_number (to
));
5638 inhibit_pre_post_conversion
= 0;
5639 /* Discard the unwind protect. */
5642 if (current_buffer
!= prev
)
5645 new = Fcurrent_buffer ();
5646 set_buffer_internal_1 (prev
);
5647 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
5648 TEMP_SET_PT_BOTH (from
, from_byte
);
5649 insert_from_buffer (XBUFFER (new), 1, len
, 0);
5651 if (orig_point
>= to
)
5652 orig_point
+= len
- orig_len
;
5653 else if (orig_point
> from
)
5657 from_byte
= CHAR_TO_BYTE (from
);
5658 to_byte
= CHAR_TO_BYTE (to
);
5659 len_byte
= to_byte
- from_byte
;
5660 TEMP_SET_PT_BOTH (from
, from_byte
);
5666 if (! EQ (current_buffer
->undo_list
, Qt
))
5667 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
5670 nchars_del
= to
- from
;
5671 nbytes_del
= to_byte
- from_byte
;
5675 if (coding
->composing
!= COMPOSITION_DISABLED
)
5678 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
5680 coding_allocate_composition_data (coding
, from
);
5683 /* Try to skip the heading and tailing ASCIIs. We can't skip them
5684 if we must run CCL program or there are compositions to
5686 if (coding
->type
!= coding_type_ccl
5687 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
5689 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
5691 if (from
< GPT
&& GPT
< to
)
5692 move_gap_both (from
, from_byte
);
5693 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
5694 if (from_byte
== to_byte
5695 && (encodep
|| NILP (coding
->post_read_conversion
))
5696 && ! CODING_REQUIRE_FLUSHING (coding
))
5698 coding
->produced
= len_byte
;
5699 coding
->produced_char
= len
;
5701 /* We must record and adjust for this new text now. */
5702 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
5703 coding_free_composition_data (coding
);
5707 head_skip
= from_byte
- from_byte_orig
;
5708 tail_skip
= to_byte_orig
- to_byte
;
5709 total_skip
= head_skip
+ tail_skip
;
5712 len
-= total_skip
; len_byte
-= total_skip
;
5715 /* For conversion, we must put the gap before the text in addition to
5716 making the gap larger for efficient decoding. The required gap
5717 size starts from 2000 which is the magic number used in make_gap.
5718 But, after one batch of conversion, it will be incremented if we
5719 find that it is not enough . */
5722 if (GAP_SIZE
< require
)
5723 make_gap (require
- GAP_SIZE
);
5724 move_gap_both (from
, from_byte
);
5726 inserted
= inserted_byte
= 0;
5728 GAP_SIZE
+= len_byte
;
5731 ZV_BYTE
-= len_byte
;
5734 if (GPT
- BEG
< BEG_UNCHANGED
)
5735 BEG_UNCHANGED
= GPT
- BEG
;
5736 if (Z
- GPT
< END_UNCHANGED
)
5737 END_UNCHANGED
= Z
- GPT
;
5739 if (!encodep
&& coding
->src_multibyte
)
5741 /* Decoding routines expects that the source text is unibyte.
5742 We must convert 8-bit characters of multibyte form to
5744 int len_byte_orig
= len_byte
;
5745 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
5746 if (len_byte
< len_byte_orig
)
5747 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
5749 coding
->src_multibyte
= 0;
5756 /* The buffer memory is now:
5757 +--------+converted-text+---------+-------original-text-------+---+
5758 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5759 |<---------------------- GAP ----------------------->| */
5760 src
= GAP_END_ADDR
- len_byte
;
5761 dst
= GPT_ADDR
+ inserted_byte
;
5764 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
5767 if (coding
->composing
!= COMPOSITION_DISABLED
)
5768 coding
->cmp_data
->char_offset
= from
+ inserted
;
5769 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
5772 /* The buffer memory is now:
5773 +--------+-------converted-text----+--+------original-text----+---+
5774 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5775 |<---------------------- GAP ----------------------->| */
5777 inserted
+= coding
->produced_char
;
5778 inserted_byte
+= coding
->produced
;
5779 len_byte
-= coding
->consumed
;
5781 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5783 coding_allocate_composition_data (coding
, from
+ inserted
);
5787 src
+= coding
->consumed
;
5788 dst
+= coding
->produced
;
5790 if (result
== CODING_FINISH_NORMAL
)
5795 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
5797 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
5798 Lisp_Object eol_type
;
5800 /* Encode LFs back to the original eol format (CR or CRLF). */
5801 if (coding
->eol_type
== CODING_EOL_CR
)
5803 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
5809 while (p
< pend
) if (*p
++ == '\n') count
++;
5810 if (src
- dst
< count
)
5812 /* We don't have sufficient room for encoding LFs
5813 back to CRLF. We must record converted and
5814 not-yet-converted text back to the buffer
5815 content, enlarge the gap, then record them out of
5816 the buffer contents again. */
5817 int add
= len_byte
+ inserted_byte
;
5820 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5821 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5822 make_gap (count
- GAP_SIZE
);
5824 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5825 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5826 /* Don't forget to update SRC, DST, and PEND. */
5827 src
= GAP_END_ADDR
- len_byte
;
5828 dst
= GPT_ADDR
+ inserted_byte
;
5832 inserted_byte
+= count
;
5833 coding
->produced
+= count
;
5834 p
= dst
= pend
+ count
;
5838 if (*p
== '\n') count
--, *--p
= '\r';
5842 /* Suppress eol-format conversion in the further conversion. */
5843 coding
->eol_type
= CODING_EOL_LF
;
5845 /* Set the coding system symbol to that for Unix-like EOL. */
5846 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5847 if (VECTORP (eol_type
)
5848 && XVECTOR (eol_type
)->size
== 3
5849 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
5850 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5852 coding
->symbol
= saved_coding_symbol
;
5858 if (coding
->type
!= coding_type_ccl
5859 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5861 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5864 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5866 /* The source text ends in invalid codes. Let's just
5867 make them valid buffer contents, and finish conversion. */
5870 unsigned char *start
= dst
;
5872 inserted
+= len_byte
;
5876 dst
+= CHAR_STRING (c
, dst
);
5879 inserted_byte
+= dst
- start
;
5883 inserted
+= len_byte
;
5884 inserted_byte
+= len_byte
;
5890 if (result
== CODING_FINISH_INTERRUPT
)
5892 /* The conversion procedure was interrupted by a user. */
5895 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5896 if (coding
->consumed
< 1)
5898 /* It's quite strange to require more memory without
5899 consuming any bytes. Perhaps CCL program bug. */
5904 /* We have just done the first batch of conversion which was
5905 stopped because of insufficient gap. Let's reconsider the
5906 required gap size (i.e. SRT - DST) now.
5908 We have converted ORIG bytes (== coding->consumed) into
5909 NEW bytes (coding->produced). To convert the remaining
5910 LEN bytes, we may need REQUIRE bytes of gap, where:
5911 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5912 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5913 Here, we are sure that NEW >= ORIG. */
5915 if (coding
->produced
<= coding
->consumed
)
5917 /* This happens because of CCL-based coding system with
5923 float ratio
= coding
->produced
- coding
->consumed
;
5924 ratio
/= coding
->consumed
;
5925 require
= len_byte
* ratio
;
5929 if ((src
- dst
) < (require
+ 2000))
5931 /* See the comment above the previous call of make_gap. */
5932 int add
= len_byte
+ inserted_byte
;
5935 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5936 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5937 make_gap (require
+ 2000);
5939 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5940 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5943 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5945 if (encodep
&& coding
->dst_multibyte
)
5947 /* The output is unibyte. We must convert 8-bit characters to
5949 if (inserted_byte
* 2 > GAP_SIZE
)
5951 GAP_SIZE
-= inserted_byte
;
5952 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5953 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5954 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5955 make_gap (inserted_byte
- GAP_SIZE
);
5956 GAP_SIZE
+= inserted_byte
;
5957 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5958 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5959 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5961 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5964 /* If we shrank the conversion area, adjust it now. */
5968 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5969 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5970 GAP_SIZE
+= total_skip
;
5971 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5972 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5973 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5974 from
-= head_skip
; from_byte
-= head_skip
;
5975 to
+= tail_skip
; to_byte
+= tail_skip
;
5979 if (! EQ (current_buffer
->undo_list
, Qt
))
5980 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5982 adjust_after_replace_noundo (from
, from_byte
, nchars_del
, nbytes_del
,
5983 inserted
, inserted_byte
);
5984 inserted
= Z
- prev_Z
;
5986 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5987 coding_restore_composition (coding
, Fcurrent_buffer ());
5988 coding_free_composition_data (coding
);
5990 if (! inhibit_pre_post_conversion
5991 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5994 Lisp_Object saved_coding_system
;
5997 TEMP_SET_PT_BOTH (from
, from_byte
);
5999 record_unwind_protect (code_convert_region_unwind
,
6000 Fcons (Vlast_coding_system_used
, Qnil
));
6001 saved_coding_system
= Vlast_coding_system_used
;
6002 Vlast_coding_system_used
= coding
->symbol
;
6003 /* We should not call any more pre-write/post-read-conversion
6004 functions while this post-read-conversion is running. */
6005 inhibit_pre_post_conversion
= 1;
6006 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
6007 inhibit_pre_post_conversion
= 0;
6008 coding
->symbol
= Vlast_coding_system_used
;
6009 Vlast_coding_system_used
= saved_coding_system
;
6010 /* Discard the unwind protect. */
6013 inserted
+= Z
- prev_Z
;
6016 if (orig_point
>= from
)
6018 if (orig_point
>= from
+ orig_len
)
6019 orig_point
+= inserted
- orig_len
;
6022 TEMP_SET_PT (orig_point
);
6027 signal_after_change (from
, to
- from
, inserted
);
6028 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
6032 coding
->consumed
= to_byte
- from_byte
;
6033 coding
->consumed_char
= to
- from
;
6034 coding
->produced
= inserted_byte
;
6035 coding
->produced_char
= inserted
;
6041 /* Name (or base name) of work buffer for code conversion. */
6042 static Lisp_Object Vcode_conversion_workbuf_name
;
6044 /* Set the current buffer to the working buffer prepared for
6045 code-conversion. MULTIBYTE specifies the multibyteness of the
6046 buffer. Return the buffer we set if it must be killed after use.
6047 Otherwise return Qnil. */
6050 set_conversion_work_buffer (multibyte
)
6053 Lisp_Object buffer
, buffer_to_kill
;
6056 buffer
= Fget_buffer_create (Vcode_conversion_workbuf_name
);
6057 buf
= XBUFFER (buffer
);
6058 if (buf
== current_buffer
)
6060 /* As we are already in the work buffer, we must generate a new
6061 buffer for the work. */
6064 name
= Fgenerate_new_buffer_name (Vcode_conversion_workbuf_name
, Qnil
);
6065 buffer
= buffer_to_kill
= Fget_buffer_create (name
);
6066 buf
= XBUFFER (buffer
);
6069 buffer_to_kill
= Qnil
;
6071 delete_all_overlays (buf
);
6072 buf
->directory
= current_buffer
->directory
;
6073 buf
->read_only
= Qnil
;
6074 buf
->filename
= Qnil
;
6075 buf
->undo_list
= Qt
;
6076 eassert (buf
->overlays_before
== NULL
);
6077 eassert (buf
->overlays_after
== NULL
);
6078 set_buffer_internal (buf
);
6079 if (BEG
!= BEGV
|| Z
!= ZV
)
6081 del_range_2 (BEG
, BEG_BYTE
, Z
, Z_BYTE
, 0);
6082 buf
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
6083 return buffer_to_kill
;
6087 run_pre_post_conversion_on_str (str
, coding
, encodep
)
6089 struct coding_system
*coding
;
6092 int count
= SPECPDL_INDEX ();
6093 struct gcpro gcpro1
, gcpro2
;
6094 int multibyte
= STRING_MULTIBYTE (str
);
6095 Lisp_Object old_deactivate_mark
;
6096 Lisp_Object buffer_to_kill
;
6097 Lisp_Object unwind_arg
;
6099 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
6100 /* It is not crucial to specbind this. */
6101 old_deactivate_mark
= Vdeactivate_mark
;
6102 GCPRO2 (str
, old_deactivate_mark
);
6104 /* We must insert the contents of STR as is without
6105 unibyte<->multibyte conversion. For that, we adjust the
6106 multibyteness of the working buffer to that of STR. */
6107 buffer_to_kill
= set_conversion_work_buffer (multibyte
);
6108 if (NILP (buffer_to_kill
))
6109 unwind_arg
= Fcons (Vlast_coding_system_used
, Qnil
);
6111 unwind_arg
= list2 (Vlast_coding_system_used
, buffer_to_kill
);
6112 record_unwind_protect (code_convert_region_unwind
, unwind_arg
);
6114 insert_from_string (str
, 0, 0,
6115 SCHARS (str
), SBYTES (str
), 0);
6117 inhibit_pre_post_conversion
= 1;
6120 struct buffer
*prev
= current_buffer
;
6122 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
6123 if (prev
!= current_buffer
)
6124 /* We must kill the current buffer too. */
6125 Fsetcdr (unwind_arg
, Fcons (Fcurrent_buffer (), XCDR (unwind_arg
)));
6129 Vlast_coding_system_used
= coding
->symbol
;
6130 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
6131 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
6132 coding
->symbol
= Vlast_coding_system_used
;
6134 inhibit_pre_post_conversion
= 0;
6135 Vdeactivate_mark
= old_deactivate_mark
;
6136 str
= make_buffer_string (BEG
, Z
, 1);
6137 return unbind_to (count
, str
);
6141 /* Run pre-write-conversion function of CODING on NCHARS/NBYTES
6142 text in *STR. *SIZE is the allocated bytes for STR. As it
6143 is intended that this function is called from encode_terminal_code,
6144 the pre-write-conversion function is run by safe_call and thus
6145 "Error during redisplay: ..." is logged when an error occurs.
6147 Store the resulting text in *STR and set CODING->produced_char and
6148 CODING->produced to the number of characters and bytes
6149 respectively. If the size of *STR is too small, enlarge it by
6150 xrealloc and update *STR and *SIZE. */
6153 run_pre_write_conversin_on_c_str (str
, size
, nchars
, nbytes
, coding
)
6154 unsigned char **str
;
6155 int *size
, nchars
, nbytes
;
6156 struct coding_system
*coding
;
6158 struct gcpro gcpro1
, gcpro2
;
6159 struct buffer
*cur
= current_buffer
;
6160 struct buffer
*prev
;
6161 Lisp_Object old_deactivate_mark
, old_last_coding_system_used
;
6162 Lisp_Object args
[3];
6163 Lisp_Object buffer_to_kill
;
6165 /* It is not crucial to specbind this. */
6166 old_deactivate_mark
= Vdeactivate_mark
;
6167 old_last_coding_system_used
= Vlast_coding_system_used
;
6168 GCPRO2 (old_deactivate_mark
, old_last_coding_system_used
);
6170 /* We must insert the contents of STR as is without
6171 unibyte<->multibyte conversion. For that, we adjust the
6172 multibyteness of the working buffer to that of STR. */
6173 buffer_to_kill
= set_conversion_work_buffer (coding
->src_multibyte
);
6174 insert_1_both (*str
, nchars
, nbytes
, 0, 0, 0);
6176 inhibit_pre_post_conversion
= 1;
6177 prev
= current_buffer
;
6178 args
[0] = coding
->pre_write_conversion
;
6179 args
[1] = make_number (BEG
);
6180 args
[2] = make_number (Z
);
6181 safe_call (3, args
);
6182 inhibit_pre_post_conversion
= 0;
6183 Vdeactivate_mark
= old_deactivate_mark
;
6184 Vlast_coding_system_used
= old_last_coding_system_used
;
6185 coding
->produced_char
= Z
- BEG
;
6186 coding
->produced
= Z_BYTE
- BEG_BYTE
;
6187 if (coding
->produced
> *size
)
6189 *size
= coding
->produced
;
6190 *str
= xrealloc (*str
, *size
);
6192 if (BEG
< GPT
&& GPT
< Z
)
6194 bcopy (BEG_ADDR
, *str
, coding
->produced
);
6195 coding
->src_multibyte
6196 = ! NILP (current_buffer
->enable_multibyte_characters
);
6197 if (prev
!= current_buffer
)
6198 Fkill_buffer (Fcurrent_buffer ());
6199 set_buffer_internal (cur
);
6200 if (! NILP (buffer_to_kill
))
6201 Fkill_buffer (buffer_to_kill
);
6206 decode_coding_string (str
, coding
, nocopy
)
6208 struct coding_system
*coding
;
6212 struct conversion_buffer buf
;
6214 Lisp_Object saved_coding_symbol
;
6216 int require_decoding
;
6217 int shrinked_bytes
= 0;
6219 int consumed
, consumed_char
, produced
, produced_char
;
6222 to_byte
= SBYTES (str
);
6224 saved_coding_symbol
= coding
->symbol
;
6225 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6226 coding
->dst_multibyte
= 1;
6227 coding
->heading_ascii
= 0;
6229 if (CODING_REQUIRE_DETECTION (coding
))
6231 /* See the comments in code_convert_region. */
6232 if (coding
->type
== coding_type_undecided
)
6234 detect_coding (coding
, SDATA (str
), to_byte
);
6235 if (coding
->type
== coding_type_undecided
)
6237 coding
->type
= coding_type_emacs_mule
;
6238 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
6239 /* As emacs-mule decoder will handle composition, we
6240 need this setting to allocate coding->cmp_data
6242 coding
->composing
= COMPOSITION_NO
;
6245 if (coding
->eol_type
== CODING_EOL_UNDECIDED
6246 && coding
->type
!= coding_type_ccl
)
6248 saved_coding_symbol
= coding
->symbol
;
6249 detect_eol (coding
, SDATA (str
), to_byte
);
6250 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
6251 coding
->eol_type
= CODING_EOL_LF
;
6252 /* We had better recover the original eol format if we
6253 encounter an inconsistent eol format while decoding. */
6254 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
6258 if (coding
->type
== coding_type_no_conversion
6259 || coding
->type
== coding_type_raw_text
)
6260 coding
->dst_multibyte
= 0;
6262 require_decoding
= CODING_REQUIRE_DECODING (coding
);
6264 if (STRING_MULTIBYTE (str
))
6266 /* Decoding routines expect the source text to be unibyte. */
6267 str
= Fstring_as_unibyte (str
);
6268 to_byte
= SBYTES (str
);
6270 coding
->src_multibyte
= 0;
6273 /* Try to skip the heading and tailing ASCIIs. */
6274 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
6276 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6278 if (from
== to_byte
)
6279 require_decoding
= 0;
6280 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6283 if (!require_decoding
6284 && !(SYMBOLP (coding
->post_read_conversion
)
6285 && !NILP (Ffboundp (coding
->post_read_conversion
))))
6287 coding
->consumed
= SBYTES (str
);
6288 coding
->consumed_char
= SCHARS (str
);
6289 if (coding
->dst_multibyte
)
6291 str
= Fstring_as_multibyte (str
);
6294 coding
->produced
= SBYTES (str
);
6295 coding
->produced_char
= SCHARS (str
);
6296 return (nocopy
? str
: Fcopy_sequence (str
));
6299 if (coding
->composing
!= COMPOSITION_DISABLED
)
6300 coding_allocate_composition_data (coding
, from
);
6301 len
= decoding_buffer_size (coding
, to_byte
- from
);
6302 allocate_conversion_buffer (buf
, len
);
6304 consumed
= consumed_char
= produced
= produced_char
= 0;
6307 result
= decode_coding (coding
, SDATA (str
) + from
+ consumed
,
6308 buf
.data
+ produced
, to_byte
- from
- consumed
,
6309 buf
.size
- produced
);
6310 consumed
+= coding
->consumed
;
6311 consumed_char
+= coding
->consumed_char
;
6312 produced
+= coding
->produced
;
6313 produced_char
+= coding
->produced_char
;
6314 if (result
== CODING_FINISH_NORMAL
6315 || result
== CODING_FINISH_INTERRUPT
6316 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6317 && coding
->consumed
== 0))
6319 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
6320 coding_allocate_composition_data (coding
, from
+ produced_char
);
6321 else if (result
== CODING_FINISH_INSUFFICIENT_DST
)
6322 extend_conversion_buffer (&buf
);
6323 else if (result
== CODING_FINISH_INCONSISTENT_EOL
)
6325 Lisp_Object eol_type
;
6327 /* Recover the original EOL format. */
6328 if (coding
->eol_type
== CODING_EOL_CR
)
6331 for (p
= buf
.data
; p
< buf
.data
+ produced
; p
++)
6332 if (*p
== '\n') *p
= '\r';
6334 else if (coding
->eol_type
== CODING_EOL_CRLF
)
6337 unsigned char *p0
, *p1
;
6338 for (p0
= buf
.data
, p1
= p0
+ produced
; p0
< p1
; p0
++)
6339 if (*p0
== '\n') num_eol
++;
6340 if (produced
+ num_eol
>= buf
.size
)
6341 extend_conversion_buffer (&buf
);
6342 for (p0
= buf
.data
+ produced
, p1
= p0
+ num_eol
; p0
> buf
.data
;)
6345 if (*p0
== '\n') *--p1
= '\r';
6347 produced
+= num_eol
;
6348 produced_char
+= num_eol
;
6350 /* Suppress eol-format conversion in the further conversion. */
6351 coding
->eol_type
= CODING_EOL_LF
;
6353 /* Set the coding system symbol to that for Unix-like EOL. */
6354 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
6355 if (VECTORP (eol_type
)
6356 && XVECTOR (eol_type
)->size
== 3
6357 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
6358 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
6360 coding
->symbol
= saved_coding_symbol
;
6366 coding
->consumed
= consumed
;
6367 coding
->consumed_char
= consumed_char
;
6368 coding
->produced
= produced
;
6369 coding
->produced_char
= produced_char
;
6371 if (coding
->dst_multibyte
)
6372 newstr
= make_uninit_multibyte_string (produced_char
+ shrinked_bytes
,
6373 produced
+ shrinked_bytes
);
6375 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6377 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6378 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6379 if (shrinked_bytes
> from
)
6380 STRING_COPYIN (newstr
, from
+ produced
,
6381 SDATA (str
) + to_byte
,
6382 shrinked_bytes
- from
);
6383 free_conversion_buffer (&buf
);
6385 coding
->consumed
+= shrinked_bytes
;
6386 coding
->consumed_char
+= shrinked_bytes
;
6387 coding
->produced
+= shrinked_bytes
;
6388 coding
->produced_char
+= shrinked_bytes
;
6390 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
6391 coding_restore_composition (coding
, newstr
);
6392 coding_free_composition_data (coding
);
6394 if (SYMBOLP (coding
->post_read_conversion
)
6395 && !NILP (Ffboundp (coding
->post_read_conversion
)))
6396 newstr
= run_pre_post_conversion_on_str (newstr
, coding
, 0);
6402 encode_coding_string (str
, coding
, nocopy
)
6404 struct coding_system
*coding
;
6408 struct conversion_buffer buf
;
6409 int from
, to
, to_byte
;
6411 int shrinked_bytes
= 0;
6413 int consumed
, consumed_char
, produced
, produced_char
;
6415 if (SYMBOLP (coding
->pre_write_conversion
)
6416 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
6418 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
6419 /* As STR is just newly generated, we don't have to copy it
6426 to_byte
= SBYTES (str
);
6428 /* Encoding routines determine the multibyteness of the source text
6429 by coding->src_multibyte. */
6430 coding
->src_multibyte
= SCHARS (str
) < SBYTES (str
);
6431 coding
->dst_multibyte
= 0;
6432 if (! CODING_REQUIRE_ENCODING (coding
))
6433 goto no_need_of_encoding
;
6435 if (coding
->composing
!= COMPOSITION_DISABLED
)
6436 coding_save_composition (coding
, from
, to
, str
);
6438 /* Try to skip the heading and tailing ASCIIs. We can't skip them
6439 if we must run CCL program or there are compositions to
6441 coding
->heading_ascii
= 0;
6442 if (coding
->type
!= coding_type_ccl
6443 && (! coding
->cmp_data
|| coding
->cmp_data
->used
== 0))
6445 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6447 if (from
== to_byte
)
6449 coding_free_composition_data (coding
);
6450 goto no_need_of_encoding
;
6452 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6455 len
= encoding_buffer_size (coding
, to_byte
- from
);
6456 allocate_conversion_buffer (buf
, len
);
6458 consumed
= consumed_char
= produced
= produced_char
= 0;
6461 result
= encode_coding (coding
, SDATA (str
) + from
+ consumed
,
6462 buf
.data
+ produced
, to_byte
- from
- consumed
,
6463 buf
.size
- produced
);
6464 consumed
+= coding
->consumed
;
6465 consumed_char
+= coding
->consumed_char
;
6466 produced
+= coding
->produced
;
6467 produced_char
+= coding
->produced_char
;
6468 if (result
== CODING_FINISH_NORMAL
6469 || result
== CODING_FINISH_INTERRUPT
6470 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6471 && coding
->consumed
== 0))
6473 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6474 extend_conversion_buffer (&buf
);
6477 coding
->consumed
= consumed
;
6478 coding
->consumed_char
= consumed_char
;
6479 coding
->produced
= produced
;
6480 coding
->produced_char
= produced_char
;
6482 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6484 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6485 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6486 if (shrinked_bytes
> from
)
6487 STRING_COPYIN (newstr
, from
+ produced
,
6488 SDATA (str
) + to_byte
,
6489 shrinked_bytes
- from
);
6491 free_conversion_buffer (&buf
);
6492 coding_free_composition_data (coding
);
6496 no_need_of_encoding
:
6497 coding
->consumed
= SBYTES (str
);
6498 coding
->consumed_char
= SCHARS (str
);
6499 if (STRING_MULTIBYTE (str
))
6502 /* We are sure that STR doesn't contain a multibyte
6504 STRING_SET_UNIBYTE (str
);
6507 str
= Fstring_as_unibyte (str
);
6511 coding
->produced
= SBYTES (str
);
6512 coding
->produced_char
= SCHARS (str
);
6513 return (nocopy
? str
: Fcopy_sequence (str
));
6518 /*** 8. Emacs Lisp library functions ***/
6520 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
6521 doc
: /* Return t if OBJECT is nil or a coding-system.
6522 See the documentation of `make-coding-system' for information
6523 about coding-system objects. */)
6531 if (! NILP (Fget (obj
, Qcoding_system_define_form
)))
6533 /* Get coding-spec vector for OBJ. */
6534 obj
= Fget (obj
, Qcoding_system
);
6535 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
6539 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
6540 Sread_non_nil_coding_system
, 1, 1, 0,
6541 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6548 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6549 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
6551 while (SCHARS (val
) == 0);
6552 return (Fintern (val
, Qnil
));
6555 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
6556 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6557 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
6558 (prompt
, default_coding_system
)
6559 Lisp_Object prompt
, default_coding_system
;
6562 if (SYMBOLP (default_coding_system
))
6563 default_coding_system
= SYMBOL_NAME (default_coding_system
);
6564 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6565 Qt
, Qnil
, Qcoding_system_history
,
6566 default_coding_system
, Qnil
);
6567 return (SCHARS (val
) == 0 ? Qnil
: Fintern (val
, Qnil
));
6570 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
6572 doc
: /* Check validity of CODING-SYSTEM.
6573 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6574 It is valid if it is nil or a symbol with a non-nil `coding-system' property.
6575 The value of this property should be a vector of length 5. */)
6577 Lisp_Object coding_system
;
6579 Lisp_Object define_form
;
6581 define_form
= Fget (coding_system
, Qcoding_system_define_form
);
6582 if (! NILP (define_form
))
6584 Fput (coding_system
, Qcoding_system_define_form
, Qnil
);
6585 safe_eval (define_form
);
6587 if (!NILP (Fcoding_system_p (coding_system
)))
6588 return coding_system
;
6589 xsignal1 (Qcoding_system_error
, coding_system
);
6593 detect_coding_system (src
, src_bytes
, highest
, multibytep
)
6594 const unsigned char *src
;
6595 int src_bytes
, highest
;
6598 int coding_mask
, eol_type
;
6599 Lisp_Object val
, tmp
;
6602 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
, multibytep
);
6603 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
6604 if (eol_type
== CODING_EOL_INCONSISTENT
)
6605 eol_type
= CODING_EOL_UNDECIDED
;
6610 if (eol_type
!= CODING_EOL_UNDECIDED
)
6613 val2
= Fget (Qundecided
, Qeol_type
);
6615 val
= XVECTOR (val2
)->contents
[eol_type
];
6617 return (highest
? val
: Fcons (val
, Qnil
));
6620 /* At first, gather possible coding systems in VAL. */
6622 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
6624 Lisp_Object category_val
, category_index
;
6626 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
6627 category_val
= Fsymbol_value (XCAR (tmp
));
6628 if (!NILP (category_val
)
6629 && NATNUMP (category_index
)
6630 && (coding_mask
& (1 << XFASTINT (category_index
))))
6632 val
= Fcons (category_val
, val
);
6638 val
= Fnreverse (val
);
6640 /* Then, replace the elements with subsidiary coding systems. */
6641 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
6643 if (eol_type
!= CODING_EOL_UNDECIDED
6644 && eol_type
!= CODING_EOL_INCONSISTENT
)
6647 eol
= Fget (XCAR (tmp
), Qeol_type
);
6649 XSETCAR (tmp
, XVECTOR (eol
)->contents
[eol_type
]);
6652 return (highest
? XCAR (val
) : val
);
6655 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
6657 doc
: /* Detect how the byte sequence in the region is encoded.
6658 Return a list of possible coding systems used on decoding a byte
6659 sequence containing the bytes in the region between START and END when
6660 the coding system `undecided' is specified. The list is ordered by
6661 priority decided in the current language environment.
6663 If only ASCII characters are found (except for such ISO-2022 control
6664 characters ISO-2022 as ESC), it returns a list of single element
6665 `undecided' or its subsidiary coding system according to a detected
6668 If optional argument HIGHEST is non-nil, return the coding system of
6669 highest priority. */)
6670 (start
, end
, highest
)
6671 Lisp_Object start
, end
, highest
;
6674 int from_byte
, to_byte
;
6675 int include_anchor_byte
= 0;
6677 CHECK_NUMBER_COERCE_MARKER (start
);
6678 CHECK_NUMBER_COERCE_MARKER (end
);
6680 validate_region (&start
, &end
);
6681 from
= XINT (start
), to
= XINT (end
);
6682 from_byte
= CHAR_TO_BYTE (from
);
6683 to_byte
= CHAR_TO_BYTE (to
);
6685 if (from
< GPT
&& to
>= GPT
)
6686 move_gap_both (to
, to_byte
);
6687 /* If we an anchor byte `\0' follows the region, we include it in
6688 the detecting source. Then code detectors can handle the tailing
6689 byte sequence more accurately.
6691 Fix me: This is not a perfect solution. It is better that we
6692 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6694 if (to
== Z
|| (to
== GPT
&& GAP_SIZE
> 0))
6695 include_anchor_byte
= 1;
6696 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
6697 to_byte
- from_byte
+ include_anchor_byte
,
6699 !NILP (current_buffer
6700 ->enable_multibyte_characters
));
6703 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
6705 doc
: /* Detect how the byte sequence in STRING is encoded.
6706 Return a list of possible coding systems used on decoding a byte
6707 sequence containing the bytes in STRING when the coding system
6708 `undecided' is specified. The list is ordered by priority decided in
6709 the current language environment.
6711 If only ASCII characters are found (except for such ISO-2022 control
6712 characters ISO-2022 as ESC), it returns a list of single element
6713 `undecided' or its subsidiary coding system according to a detected
6716 If optional argument HIGHEST is non-nil, return the coding system of
6717 highest priority. */)
6719 Lisp_Object string
, highest
;
6721 CHECK_STRING (string
);
6723 return detect_coding_system (SDATA (string
),
6724 /* "+ 1" is to include the anchor byte
6725 `\0'. With this, code detectors can
6726 handle the tailing bytes more
6728 SBYTES (string
) + 1,
6730 STRING_MULTIBYTE (string
));
6733 /* Subroutine for Ffind_coding_systems_region_internal.
6735 Return a list of coding systems that safely encode the multibyte
6736 text between P and PEND. SAFE_CODINGS, if non-nil, is an alist of
6737 possible coding systems. If it is nil, it means that we have not
6738 yet found any coding systems.
6740 WORK_TABLE a char-table of which element is set to t once the
6741 element is looked up.
6743 If a non-ASCII single byte char is found, set
6744 *single_byte_char_found to 1. */
6747 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
6748 unsigned char *p
, *pend
;
6749 Lisp_Object safe_codings
, work_table
;
6750 int *single_byte_char_found
;
6753 Lisp_Object val
, ch
;
6754 Lisp_Object prev
, tail
;
6756 if (NILP (safe_codings
))
6757 goto done_safe_codings
;
6760 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6762 if (ASCII_BYTE_P (c
))
6763 /* We can ignore ASCII characters here. */
6765 if (SINGLE_BYTE_CHAR_P (c
))
6766 *single_byte_char_found
= 1;
6767 /* Check the safe coding systems for C. */
6768 ch
= make_number (c
);
6769 val
= Faref (work_table
, ch
);
6771 /* This element was already checked. Ignore it. */
6773 /* Remember that we checked this element. */
6774 Faset (work_table
, ch
, Qt
);
6776 for (prev
= tail
= safe_codings
; CONSP (tail
); tail
= XCDR (tail
))
6778 Lisp_Object elt
, translation_table
, hash_table
, accept_latin_extra
;
6782 if (CONSP (XCDR (elt
)))
6784 /* This entry has this format now:
6785 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6786 ACCEPT-LATIN-EXTRA ) */
6788 encodable
= ! NILP (Faref (XCAR (val
), ch
));
6792 translation_table
= XCAR (val
);
6793 hash_table
= XCAR (XCDR (val
));
6794 accept_latin_extra
= XCAR (XCDR (XCDR (val
)));
6799 /* This entry has this format now: ( CODING . SAFE-CHARS) */
6800 encodable
= ! NILP (Faref (XCDR (elt
), ch
));
6803 /* Transform the format to:
6804 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6805 ACCEPT-LATIN-EXTRA ) */
6806 val
= Fget (XCAR (elt
), Qcoding_system
);
6808 = Fplist_get (AREF (val
, 3),
6809 Qtranslation_table_for_encode
);
6810 if (SYMBOLP (translation_table
))
6811 translation_table
= Fget (translation_table
,
6812 Qtranslation_table
);
6814 = (CHAR_TABLE_P (translation_table
)
6815 ? XCHAR_TABLE (translation_table
)->extras
[1]
6818 = ((EQ (AREF (val
, 0), make_number (2))
6819 && VECTORP (AREF (val
, 4)))
6820 ? AREF (AREF (val
, 4), 16)
6822 XSETCAR (tail
, list5 (XCAR (elt
), XCDR (elt
),
6823 translation_table
, hash_table
,
6824 accept_latin_extra
));
6829 && ((CHAR_TABLE_P (translation_table
)
6830 && ! NILP (Faref (translation_table
, ch
)))
6831 || (HASH_TABLE_P (hash_table
)
6832 && ! NILP (Fgethash (ch
, hash_table
, Qnil
)))
6833 || (SINGLE_BYTE_CHAR_P (c
)
6834 && ! NILP (accept_latin_extra
)
6835 && VECTORP (Vlatin_extra_code_table
)
6836 && ! NILP (AREF (Vlatin_extra_code_table
, c
)))))
6842 /* Exclude this coding system from SAFE_CODINGS. */
6843 if (EQ (tail
, safe_codings
))
6845 safe_codings
= XCDR (safe_codings
);
6846 if (NILP (safe_codings
))
6847 goto done_safe_codings
;
6850 XSETCDR (prev
, XCDR (tail
));
6856 /* If the above loop was terminated before P reaches PEND, it means
6857 SAFE_CODINGS was set to nil. If we have not yet found an
6858 non-ASCII single-byte char, check it now. */
6859 if (! *single_byte_char_found
)
6862 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6864 if (! ASCII_BYTE_P (c
)
6865 && SINGLE_BYTE_CHAR_P (c
))
6867 *single_byte_char_found
= 1;
6871 return safe_codings
;
6874 DEFUN ("find-coding-systems-region-internal",
6875 Ffind_coding_systems_region_internal
,
6876 Sfind_coding_systems_region_internal
, 2, 2, 0,
6877 doc
: /* Internal use only. */)
6879 Lisp_Object start
, end
;
6881 Lisp_Object work_table
, safe_codings
;
6882 int non_ascii_p
= 0;
6883 int single_byte_char_found
= 0;
6884 const unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
6886 if (STRINGP (start
))
6888 if (!STRING_MULTIBYTE (start
))
6890 p1
= SDATA (start
), p1end
= p1
+ SBYTES (start
);
6892 if (SCHARS (start
) != SBYTES (start
))
6899 CHECK_NUMBER_COERCE_MARKER (start
);
6900 CHECK_NUMBER_COERCE_MARKER (end
);
6901 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
6902 args_out_of_range (start
, end
);
6903 if (NILP (current_buffer
->enable_multibyte_characters
))
6905 from
= CHAR_TO_BYTE (XINT (start
));
6906 to
= CHAR_TO_BYTE (XINT (end
));
6907 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
6908 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
6912 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
6913 if (XINT (end
) - XINT (start
) != to
- from
)
6919 /* We are sure that the text contains no multibyte character.
6920 Check if it contains eight-bit-graphic. */
6922 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
6925 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
6931 /* The text contains non-ASCII characters. */
6933 work_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
6934 safe_codings
= Fcopy_sequence (XCDR (Vcoding_system_safe_chars
));
6936 safe_codings
= find_safe_codings (p1
, p1end
, safe_codings
, work_table
,
6937 &single_byte_char_found
);
6939 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
6940 &single_byte_char_found
);
6941 if (EQ (safe_codings
, XCDR (Vcoding_system_safe_chars
)))
6945 /* Turn safe_codings to a list of coding systems... */
6948 if (single_byte_char_found
)
6949 /* ... and append these for eight-bit chars. */
6950 val
= Fcons (Qraw_text
,
6951 Fcons (Qemacs_mule
, Fcons (Qno_conversion
, Qnil
)));
6953 /* ... and append generic coding systems. */
6954 val
= Fcopy_sequence (XCAR (Vcoding_system_safe_chars
));
6956 for (; CONSP (safe_codings
); safe_codings
= XCDR (safe_codings
))
6957 val
= Fcons (XCAR (XCAR (safe_codings
)), val
);
6961 return safe_codings
;
6965 /* Search from position POS for such characters that are unencodable
6966 accoding to SAFE_CHARS, and return a list of their positions. P
6967 points where in the memory the character at POS exists. Limit the
6968 search at PEND or when Nth unencodable characters are found.
6970 If SAFE_CHARS is a char table, an element for an unencodable
6973 If SAFE_CHARS is nil, all non-ASCII characters are unencodable.
6975 Otherwise, SAFE_CHARS is t, and only eight-bit-contrl and
6976 eight-bit-graphic characters are unencodable. */
6979 unencodable_char_position (safe_chars
, pos
, p
, pend
, n
)
6980 Lisp_Object safe_chars
;
6982 unsigned char *p
, *pend
;
6985 Lisp_Object pos_list
;
6991 int c
= STRING_CHAR_AND_LENGTH (p
, MAX_MULTIBYTE_LENGTH
, len
);
6994 && (CHAR_TABLE_P (safe_chars
)
6995 ? NILP (CHAR_TABLE_REF (safe_chars
, c
))
6996 : (NILP (safe_chars
) || c
< 256)))
6998 pos_list
= Fcons (make_number (pos
), pos_list
);
7005 return Fnreverse (pos_list
);
7009 DEFUN ("unencodable-char-position", Funencodable_char_position
,
7010 Sunencodable_char_position
, 3, 5, 0,
7012 Return position of first un-encodable character in a region.
7013 START and END specfiy the region and CODING-SYSTEM specifies the
7014 encoding to check. Return nil if CODING-SYSTEM does encode the region.
7016 If optional 4th argument COUNT is non-nil, it specifies at most how
7017 many un-encodable characters to search. In this case, the value is a
7020 If optional 5th argument STRING is non-nil, it is a string to search
7021 for un-encodable characters. In that case, START and END are indexes
7023 (start
, end
, coding_system
, count
, string
)
7024 Lisp_Object start
, end
, coding_system
, count
, string
;
7027 Lisp_Object safe_chars
;
7028 struct coding_system coding
;
7029 Lisp_Object positions
;
7031 unsigned char *p
, *pend
;
7035 validate_region (&start
, &end
);
7036 from
= XINT (start
);
7038 if (NILP (current_buffer
->enable_multibyte_characters
))
7040 p
= CHAR_POS_ADDR (from
);
7044 pend
= CHAR_POS_ADDR (to
);
7048 CHECK_STRING (string
);
7049 CHECK_NATNUM (start
);
7051 from
= XINT (start
);
7054 || to
> SCHARS (string
))
7055 args_out_of_range_3 (string
, start
, end
);
7056 if (! STRING_MULTIBYTE (string
))
7058 p
= SDATA (string
) + string_char_to_byte (string
, from
);
7059 pend
= SDATA (string
) + string_char_to_byte (string
, to
);
7062 setup_coding_system (Fcheck_coding_system (coding_system
), &coding
);
7068 CHECK_NATNUM (count
);
7072 if (coding
.type
== coding_type_no_conversion
7073 || coding
.type
== coding_type_raw_text
)
7076 if (coding
.type
== coding_type_undecided
)
7079 safe_chars
= coding_safe_chars (coding_system
);
7081 if (STRINGP (string
)
7082 || from
>= GPT
|| to
<= GPT
)
7083 positions
= unencodable_char_position (safe_chars
, from
, p
, pend
, n
);
7086 Lisp_Object args
[2];
7088 args
[0] = unencodable_char_position (safe_chars
, from
, p
, GPT_ADDR
, n
);
7089 n
-= XINT (Flength (args
[0]));
7091 positions
= args
[0];
7094 args
[1] = unencodable_char_position (safe_chars
, GPT
, GAP_END_ADDR
,
7096 positions
= Fappend (2, args
);
7100 return (NILP (count
) ? Fcar (positions
) : positions
);
7105 code_convert_region1 (start
, end
, coding_system
, encodep
)
7106 Lisp_Object start
, end
, coding_system
;
7109 struct coding_system coding
;
7112 CHECK_NUMBER_COERCE_MARKER (start
);
7113 CHECK_NUMBER_COERCE_MARKER (end
);
7114 CHECK_SYMBOL (coding_system
);
7116 validate_region (&start
, &end
);
7117 from
= XFASTINT (start
);
7118 to
= XFASTINT (end
);
7120 if (NILP (coding_system
))
7121 return make_number (to
- from
);
7123 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7124 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7126 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7127 coding
.src_multibyte
= coding
.dst_multibyte
7128 = !NILP (current_buffer
->enable_multibyte_characters
);
7129 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
7130 &coding
, encodep
, 1);
7131 Vlast_coding_system_used
= coding
.symbol
;
7132 return make_number (coding
.produced_char
);
7135 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
7136 3, 3, "r\nzCoding system: ",
7137 doc
: /* Decode the current region from the specified coding system.
7138 When called from a program, takes three arguments:
7139 START, END, and CODING-SYSTEM. START and END are buffer positions.
7140 This function sets `last-coding-system-used' to the precise coding system
7141 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7142 not fully specified.)
7143 It returns the length of the decoded text. */)
7144 (start
, end
, coding_system
)
7145 Lisp_Object start
, end
, coding_system
;
7147 return code_convert_region1 (start
, end
, coding_system
, 0);
7150 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
7151 3, 3, "r\nzCoding system: ",
7152 doc
: /* Encode the current region into the specified coding system.
7153 When called from a program, takes three arguments:
7154 START, END, and CODING-SYSTEM. START and END are buffer positions.
7155 This function sets `last-coding-system-used' to the precise coding system
7156 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7157 not fully specified.)
7158 It returns the length of the encoded text. */)
7159 (start
, end
, coding_system
)
7160 Lisp_Object start
, end
, coding_system
;
7162 return code_convert_region1 (start
, end
, coding_system
, 1);
7166 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
7167 Lisp_Object string
, coding_system
, nocopy
;
7170 struct coding_system coding
;
7172 CHECK_STRING (string
);
7173 CHECK_SYMBOL (coding_system
);
7175 if (NILP (coding_system
))
7176 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
7178 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7179 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7181 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7183 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
7184 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
7185 Vlast_coding_system_used
= coding
.symbol
;
7190 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
7192 doc
: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7193 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7194 if the decoding operation is trivial.
7195 This function sets `last-coding-system-used' to the precise coding system
7196 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7197 not fully specified.) */)
7198 (string
, coding_system
, nocopy
)
7199 Lisp_Object string
, coding_system
, nocopy
;
7201 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
7204 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
7206 doc
: /* Encode STRING to CODING-SYSTEM, and return the result.
7207 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7208 if the encoding operation is trivial.
7209 This function sets `last-coding-system-used' to the precise coding system
7210 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7211 not fully specified.) */)
7212 (string
, coding_system
, nocopy
)
7213 Lisp_Object string
, coding_system
, nocopy
;
7215 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
7218 /* Encode or decode STRING according to CODING_SYSTEM.
7219 Do not set Vlast_coding_system_used.
7221 This function is called only from macros DECODE_FILE and
7222 ENCODE_FILE, thus we ignore character composition. */
7225 code_convert_string_norecord (string
, coding_system
, encodep
)
7226 Lisp_Object string
, coding_system
;
7229 struct coding_system coding
;
7231 CHECK_STRING (string
);
7232 CHECK_SYMBOL (coding_system
);
7234 if (NILP (coding_system
))
7237 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7238 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7240 coding
.composing
= COMPOSITION_DISABLED
;
7241 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7243 ? encode_coding_string (string
, &coding
, 1)
7244 : decode_coding_string (string
, &coding
, 1));
7247 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
7248 doc
: /* Decode a Japanese character which has CODE in shift_jis encoding.
7249 Return the corresponding character. */)
7253 unsigned char c1
, c2
, s1
, s2
;
7256 CHECK_NUMBER (code
);
7257 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
7261 XSETFASTINT (val
, s2
);
7262 else if (s2
>= 0xA0 || s2
<= 0xDF)
7263 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
7265 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7269 if ((s1
< 0x80 || (s1
> 0x9F && s1
< 0xE0) || s1
> 0xEF)
7270 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
7271 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7272 DECODE_SJIS (s1
, s2
, c1
, c2
);
7273 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
7278 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
7279 doc
: /* Encode a Japanese character CH to shift_jis encoding.
7280 Return the corresponding code in SJIS. */)
7284 int charset
, c1
, c2
, s1
, s2
;
7288 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7289 if (charset
== CHARSET_ASCII
)
7293 else if (charset
== charset_jisx0208
7294 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
7296 ENCODE_SJIS (c1
, c2
, s1
, s2
);
7297 XSETFASTINT (val
, (s1
<< 8) | s2
);
7299 else if (charset
== charset_katakana_jisx0201
7300 && c1
> 0x20 && c2
< 0xE0)
7302 XSETFASTINT (val
, c1
| 0x80);
7305 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
7309 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
7310 doc
: /* Decode a Big5 character which has CODE in BIG5 coding system.
7311 Return the corresponding character. */)
7316 unsigned char b1
, b2
, c1
, c2
;
7319 CHECK_NUMBER (code
);
7320 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
7324 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7329 if ((b1
< 0xA1 || b1
> 0xFE)
7330 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
7331 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7332 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
7333 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
7338 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
7339 doc
: /* Encode the Big5 character CH to BIG5 coding system.
7340 Return the corresponding character code in Big5. */)
7344 int charset
, c1
, c2
, b1
, b2
;
7348 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7349 if (charset
== CHARSET_ASCII
)
7353 else if ((charset
== charset_big5_1
7354 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
7355 || (charset
== charset_big5_2
7356 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
7358 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
7359 XSETFASTINT (val
, (b1
<< 8) | b2
);
7362 error ("Can't encode to Big5: %d", XFASTINT (ch
));
7366 DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal
,
7367 Sset_terminal_coding_system_internal
, 1, 2, 0,
7368 doc
: /* Internal use only. */)
7369 (coding_system
, terminal
)
7370 Lisp_Object coding_system
;
7371 Lisp_Object terminal
;
7373 struct coding_system
*terminal_coding
= TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1));
7374 CHECK_SYMBOL (coding_system
);
7375 setup_coding_system (Fcheck_coding_system (coding_system
), terminal_coding
);
7376 /* We had better not send unsafe characters to terminal. */
7377 terminal_coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
7378 /* Character composition should be disabled. */
7379 terminal_coding
->composing
= COMPOSITION_DISABLED
;
7380 /* Error notification should be suppressed. */
7381 terminal_coding
->suppress_error
= 1;
7382 terminal_coding
->src_multibyte
= 1;
7383 terminal_coding
->dst_multibyte
= 0;
7387 DEFUN ("set-safe-terminal-coding-system-internal", Fset_safe_terminal_coding_system_internal
,
7388 Sset_safe_terminal_coding_system_internal
, 1, 1, 0,
7389 doc
: /* Internal use only. */)
7391 Lisp_Object coding_system
;
7393 CHECK_SYMBOL (coding_system
);
7394 setup_coding_system (Fcheck_coding_system (coding_system
),
7395 &safe_terminal_coding
);
7396 /* Character composition should be disabled. */
7397 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
7398 /* Error notification should be suppressed. */
7399 safe_terminal_coding
.suppress_error
= 1;
7400 safe_terminal_coding
.src_multibyte
= 1;
7401 safe_terminal_coding
.dst_multibyte
= 0;
7405 DEFUN ("terminal-coding-system", Fterminal_coding_system
,
7406 Sterminal_coding_system
, 0, 1, 0,
7407 doc
: /* Return coding system specified for terminal output on the given terminal.
7408 TERMINAL may be a terminal id, a frame, or nil for the selected
7409 frame's terminal device. */)
7411 Lisp_Object terminal
;
7413 return TERMINAL_TERMINAL_CODING (get_terminal (terminal
, 1))->symbol
;
7416 DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal
,
7417 Sset_keyboard_coding_system_internal
, 1, 2, 0,
7418 doc
: /* Internal use only. */)
7419 (coding_system
, terminal
)
7420 Lisp_Object coding_system
;
7421 Lisp_Object terminal
;
7423 struct terminal
*t
= get_terminal (terminal
, 1);
7424 CHECK_SYMBOL (coding_system
);
7426 setup_coding_system (Fcheck_coding_system (coding_system
),
7427 TERMINAL_KEYBOARD_CODING (t
));
7428 /* Character composition should be disabled. */
7429 TERMINAL_KEYBOARD_CODING (t
)->composing
= COMPOSITION_DISABLED
;
7433 DEFUN ("keyboard-coding-system", Fkeyboard_coding_system
,
7434 Skeyboard_coding_system
, 0, 1, 0,
7435 doc
: /* Return coding system for decoding keyboard input on TERMINAL.
7436 TERMINAL may be a terminal id, a frame, or nil for the selected
7437 frame's terminal device. */)
7439 Lisp_Object terminal
;
7441 return TERMINAL_KEYBOARD_CODING (get_terminal (terminal
, 1))->symbol
;
7445 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
7446 Sfind_operation_coding_system
, 1, MANY
, 0,
7447 doc
: /* Choose a coding system for an operation based on the target name.
7448 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
7449 DECODING-SYSTEM is the coding system to use for decoding
7450 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
7451 for encoding (in case OPERATION does encoding).
7453 The first argument OPERATION specifies an I/O primitive:
7454 For file I/O, `insert-file-contents' or `write-region'.
7455 For process I/O, `call-process', `call-process-region', or `start-process'.
7456 For network I/O, `open-network-stream'.
7458 The remaining arguments should be the same arguments that were passed
7459 to the primitive. Depending on which primitive, one of those arguments
7460 is selected as the TARGET. For example, if OPERATION does file I/O,
7461 whichever argument specifies the file name is TARGET.
7463 TARGET has a meaning which depends on OPERATION:
7464 For file I/O, TARGET is a file name (except for the special case below).
7465 For process I/O, TARGET is a process name.
7466 For network I/O, TARGET is a service name or a port number
7468 This function looks up what specified for TARGET in,
7469 `file-coding-system-alist', `process-coding-system-alist',
7470 or `network-coding-system-alist' depending on OPERATION.
7471 They may specify a coding system, a cons of coding systems,
7472 or a function symbol to call.
7473 In the last case, we call the function with one argument,
7474 which is a list of all the arguments given to this function.
7475 If the function can't decide a coding system, it can return
7476 `undecided' so that the normal code-detection is performed.
7478 If OPERATION is `insert-file-contents', the argument corresponding to
7479 TARGET may be a cons (FILENAME . BUFFER). In that case, FILENAME is a
7480 file name to look up, and BUFFER is a buffer that contains the file's
7481 contents (not yet decoded). If `file-coding-system-alist' specifies a
7482 function to call for FILENAME, that function should examine the
7483 contents of BUFFER instead of reading the file.
7485 usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
7490 Lisp_Object operation
, target_idx
, target
, val
;
7491 register Lisp_Object chain
;
7494 error ("Too few arguments");
7495 operation
= args
[0];
7496 if (!SYMBOLP (operation
)
7497 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
7498 error ("Invalid first argument");
7499 if (nargs
< 1 + XINT (target_idx
))
7500 error ("Too few arguments for operation: %s",
7501 SDATA (SYMBOL_NAME (operation
)));
7502 /* For write-region, if the 6th argument (i.e. VISIT, the 5th
7503 argument to write-region) is string, it must be treated as a
7504 target file name. */
7505 if (EQ (operation
, Qwrite_region
)
7507 && STRINGP (args
[5]))
7508 target_idx
= make_number (4);
7509 target
= args
[XINT (target_idx
) + 1];
7510 if (!(STRINGP (target
)
7511 || (EQ (operation
, Qinsert_file_contents
) && CONSP (target
)
7512 && STRINGP (XCAR (target
)) && BUFFERP (XCDR (target
)))
7513 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
7514 error ("Invalid argument %d", XINT (target_idx
) + 1);
7516 target
= XCAR (target
);
7518 chain
= ((EQ (operation
, Qinsert_file_contents
)
7519 || EQ (operation
, Qwrite_region
))
7520 ? Vfile_coding_system_alist
7521 : (EQ (operation
, Qopen_network_stream
)
7522 ? Vnetwork_coding_system_alist
7523 : Vprocess_coding_system_alist
));
7527 for (; CONSP (chain
); chain
= XCDR (chain
))
7533 && ((STRINGP (target
)
7534 && STRINGP (XCAR (elt
))
7535 && fast_string_match (XCAR (elt
), target
) >= 0)
7536 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
7539 /* Here, if VAL is both a valid coding system and a valid
7540 function symbol, we return VAL as a coding system. */
7543 if (! SYMBOLP (val
))
7545 if (! NILP (Fcoding_system_p (val
)))
7546 return Fcons (val
, val
);
7547 if (! NILP (Ffboundp (val
)))
7549 /* We use call1 rather than safe_call1
7550 so as to get bug reports about functions called here
7551 which don't handle the current interface. */
7552 val
= call1 (val
, Flist (nargs
, args
));
7555 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
7556 return Fcons (val
, val
);
7564 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
7565 Supdate_coding_systems_internal
, 0, 0, 0,
7566 doc
: /* Update internal database for ISO2022 and CCL based coding systems.
7567 When values of any coding categories are changed, you must
7568 call this function. */)
7573 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7577 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[i
]);
7580 if (! coding_system_table
[i
])
7581 coding_system_table
[i
] = ((struct coding_system
*)
7582 xmalloc (sizeof (struct coding_system
)));
7583 setup_coding_system (val
, coding_system_table
[i
]);
7585 else if (coding_system_table
[i
])
7587 xfree (coding_system_table
[i
]);
7588 coding_system_table
[i
] = NULL
;
7595 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
7596 Sset_coding_priority_internal
, 0, 0, 0,
7597 doc
: /* Update internal database for the current value of `coding-category-list'.
7598 This function is internal use only. */)
7604 val
= Vcoding_category_list
;
7606 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
7608 if (! SYMBOLP (XCAR (val
)))
7610 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
7611 if (idx
>= CODING_CATEGORY_IDX_MAX
)
7613 coding_priorities
[i
++] = (1 << idx
);
7616 /* If coding-category-list is valid and contains all coding
7617 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
7618 the following code saves Emacs from crashing. */
7619 while (i
< CODING_CATEGORY_IDX_MAX
)
7620 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
7625 DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal
,
7626 Sdefine_coding_system_internal
, 1, 1, 0,
7627 doc
: /* Register CODING-SYSTEM as a base coding system.
7628 This function is internal use only. */)
7630 Lisp_Object coding_system
;
7632 Lisp_Object safe_chars
, slot
;
7634 if (NILP (Fcheck_coding_system (coding_system
)))
7635 xsignal1 (Qcoding_system_error
, coding_system
);
7637 safe_chars
= coding_safe_chars (coding_system
);
7638 if (! EQ (safe_chars
, Qt
) && ! CHAR_TABLE_P (safe_chars
))
7639 error ("No valid safe-chars property for %s",
7640 SDATA (SYMBOL_NAME (coding_system
)));
7642 if (EQ (safe_chars
, Qt
))
7644 if (NILP (Fmemq (coding_system
, XCAR (Vcoding_system_safe_chars
))))
7645 XSETCAR (Vcoding_system_safe_chars
,
7646 Fcons (coding_system
, XCAR (Vcoding_system_safe_chars
)));
7650 slot
= Fassq (coding_system
, XCDR (Vcoding_system_safe_chars
));
7652 XSETCDR (Vcoding_system_safe_chars
,
7653 nconc2 (XCDR (Vcoding_system_safe_chars
),
7654 Fcons (Fcons (coding_system
, safe_chars
), Qnil
)));
7656 XSETCDR (slot
, safe_chars
);
7664 /*** 9. Post-amble ***/
7671 /* Emacs' internal format specific initialize routine. */
7672 for (i
= 0; i
<= 0x20; i
++)
7673 emacs_code_class
[i
] = EMACS_control_code
;
7674 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
7675 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
7676 for (i
= 0x21 ; i
< 0x7F; i
++)
7677 emacs_code_class
[i
] = EMACS_ascii_code
;
7678 emacs_code_class
[0x7F] = EMACS_control_code
;
7679 for (i
= 0x80; i
< 0xFF; i
++)
7680 emacs_code_class
[i
] = EMACS_invalid_code
;
7681 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
7682 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
7683 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
7684 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
7686 /* ISO2022 specific initialize routine. */
7687 for (i
= 0; i
< 0x20; i
++)
7688 iso_code_class
[i
] = ISO_control_0
;
7689 for (i
= 0x21; i
< 0x7F; i
++)
7690 iso_code_class
[i
] = ISO_graphic_plane_0
;
7691 for (i
= 0x80; i
< 0xA0; i
++)
7692 iso_code_class
[i
] = ISO_control_1
;
7693 for (i
= 0xA1; i
< 0xFF; i
++)
7694 iso_code_class
[i
] = ISO_graphic_plane_1
;
7695 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
7696 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
7697 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
7698 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
7699 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
7700 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
7701 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
7702 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
7703 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
7704 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
7706 setup_coding_system (Qnil
, &safe_terminal_coding
);
7707 setup_coding_system (Qnil
, &default_buffer_file_coding
);
7709 bzero (coding_system_table
, sizeof coding_system_table
);
7711 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
7712 for (i
= 0; i
< 128; i
++)
7713 ascii_skip_code
[i
] = 1;
7715 #if defined (MSDOS) || defined (WINDOWSNT)
7716 system_eol_type
= CODING_EOL_CRLF
;
7718 system_eol_type
= CODING_EOL_LF
;
7721 inhibit_pre_post_conversion
= 0;
7729 staticpro (&Vcode_conversion_workbuf_name
);
7730 Vcode_conversion_workbuf_name
= build_string (" *code-conversion-work*");
7732 Qtarget_idx
= intern ("target-idx");
7733 staticpro (&Qtarget_idx
);
7735 Qcoding_system_history
= intern ("coding-system-history");
7736 staticpro (&Qcoding_system_history
);
7737 Fset (Qcoding_system_history
, Qnil
);
7739 /* Target FILENAME is the first argument. */
7740 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
7741 /* Target FILENAME is the third argument. */
7742 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
7744 Qcall_process
= intern ("call-process");
7745 staticpro (&Qcall_process
);
7746 /* Target PROGRAM is the first argument. */
7747 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
7749 Qcall_process_region
= intern ("call-process-region");
7750 staticpro (&Qcall_process_region
);
7751 /* Target PROGRAM is the third argument. */
7752 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
7754 Qstart_process
= intern ("start-process");
7755 staticpro (&Qstart_process
);
7756 /* Target PROGRAM is the third argument. */
7757 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
7759 Qopen_network_stream
= intern ("open-network-stream");
7760 staticpro (&Qopen_network_stream
);
7761 /* Target SERVICE is the fourth argument. */
7762 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
7764 Qcoding_system
= intern ("coding-system");
7765 staticpro (&Qcoding_system
);
7767 Qeol_type
= intern ("eol-type");
7768 staticpro (&Qeol_type
);
7770 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
7771 staticpro (&Qbuffer_file_coding_system
);
7773 Qpost_read_conversion
= intern ("post-read-conversion");
7774 staticpro (&Qpost_read_conversion
);
7776 Qpre_write_conversion
= intern ("pre-write-conversion");
7777 staticpro (&Qpre_write_conversion
);
7779 Qno_conversion
= intern ("no-conversion");
7780 staticpro (&Qno_conversion
);
7782 Qundecided
= intern ("undecided");
7783 staticpro (&Qundecided
);
7785 Qcoding_system_p
= intern ("coding-system-p");
7786 staticpro (&Qcoding_system_p
);
7788 Qcoding_system_error
= intern ("coding-system-error");
7789 staticpro (&Qcoding_system_error
);
7791 Fput (Qcoding_system_error
, Qerror_conditions
,
7792 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
7793 Fput (Qcoding_system_error
, Qerror_message
,
7794 build_string ("Invalid coding system"));
7796 Qcoding_category
= intern ("coding-category");
7797 staticpro (&Qcoding_category
);
7798 Qcoding_category_index
= intern ("coding-category-index");
7799 staticpro (&Qcoding_category_index
);
7801 Vcoding_category_table
7802 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
7803 staticpro (&Vcoding_category_table
);
7806 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7808 XVECTOR (Vcoding_category_table
)->contents
[i
]
7809 = intern (coding_category_name
[i
]);
7810 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
7811 Qcoding_category_index
, make_number (i
));
7815 Vcoding_system_safe_chars
= Fcons (Qnil
, Qnil
);
7816 staticpro (&Vcoding_system_safe_chars
);
7818 Qtranslation_table
= intern ("translation-table");
7819 staticpro (&Qtranslation_table
);
7820 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (2));
7822 Qtranslation_table_id
= intern ("translation-table-id");
7823 staticpro (&Qtranslation_table_id
);
7825 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
7826 staticpro (&Qtranslation_table_for_decode
);
7828 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
7829 staticpro (&Qtranslation_table_for_encode
);
7831 Qsafe_chars
= intern ("safe-chars");
7832 staticpro (&Qsafe_chars
);
7834 Qchar_coding_system
= intern ("char-coding-system");
7835 staticpro (&Qchar_coding_system
);
7837 /* Intern this now in case it isn't already done.
7838 Setting this variable twice is harmless.
7839 But don't staticpro it here--that is done in alloc.c. */
7840 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
7841 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
7842 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (0));
7844 Qvalid_codes
= intern ("valid-codes");
7845 staticpro (&Qvalid_codes
);
7847 Qascii_incompatible
= intern ("ascii-incompatible");
7848 staticpro (&Qascii_incompatible
);
7850 Qemacs_mule
= intern ("emacs-mule");
7851 staticpro (&Qemacs_mule
);
7853 Qraw_text
= intern ("raw-text");
7854 staticpro (&Qraw_text
);
7856 Qutf_8
= intern ("utf-8");
7857 staticpro (&Qutf_8
);
7859 Qcoding_system_define_form
= intern ("coding-system-define-form");
7860 staticpro (&Qcoding_system_define_form
);
7862 defsubr (&Scoding_system_p
);
7863 defsubr (&Sread_coding_system
);
7864 defsubr (&Sread_non_nil_coding_system
);
7865 defsubr (&Scheck_coding_system
);
7866 defsubr (&Sdetect_coding_region
);
7867 defsubr (&Sdetect_coding_string
);
7868 defsubr (&Sfind_coding_systems_region_internal
);
7869 defsubr (&Sunencodable_char_position
);
7870 defsubr (&Sdecode_coding_region
);
7871 defsubr (&Sencode_coding_region
);
7872 defsubr (&Sdecode_coding_string
);
7873 defsubr (&Sencode_coding_string
);
7874 defsubr (&Sdecode_sjis_char
);
7875 defsubr (&Sencode_sjis_char
);
7876 defsubr (&Sdecode_big5_char
);
7877 defsubr (&Sencode_big5_char
);
7878 defsubr (&Sset_terminal_coding_system_internal
);
7879 defsubr (&Sset_safe_terminal_coding_system_internal
);
7880 defsubr (&Sterminal_coding_system
);
7881 defsubr (&Sset_keyboard_coding_system_internal
);
7882 defsubr (&Skeyboard_coding_system
);
7883 defsubr (&Sfind_operation_coding_system
);
7884 defsubr (&Supdate_coding_systems_internal
);
7885 defsubr (&Sset_coding_priority_internal
);
7886 defsubr (&Sdefine_coding_system_internal
);
7888 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
7889 doc
: /* List of coding systems.
7891 Do not alter the value of this variable manually. This variable should be
7892 updated by the functions `make-coding-system' and
7893 `define-coding-system-alias'. */);
7894 Vcoding_system_list
= Qnil
;
7896 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
7897 doc
: /* Alist of coding system names.
7898 Each element is one element list of coding system name.
7899 This variable is given to `completing-read' as TABLE argument.
7901 Do not alter the value of this variable manually. This variable should be
7902 updated by the functions `make-coding-system' and
7903 `define-coding-system-alias'. */);
7904 Vcoding_system_alist
= Qnil
;
7906 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
7907 doc
: /* List of coding-categories (symbols) ordered by priority.
7909 On detecting a coding system, Emacs tries code detection algorithms
7910 associated with each coding-category one by one in this order. When
7911 one algorithm agrees with a byte sequence of source text, the coding
7912 system bound to the corresponding coding-category is selected.
7914 Don't modify this variable directly, but use `set-coding-priority'. */);
7918 Vcoding_category_list
= Qnil
;
7919 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
7920 Vcoding_category_list
7921 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
7922 Vcoding_category_list
);
7925 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
7926 doc
: /* Specify the coding system for read operations.
7927 It is useful to bind this variable with `let', but do not set it globally.
7928 If the value is a coding system, it is used for decoding on read operation.
7929 If not, an appropriate element is used from one of the coding system alists:
7930 There are three such tables, `file-coding-system-alist',
7931 `process-coding-system-alist', and `network-coding-system-alist'. */);
7932 Vcoding_system_for_read
= Qnil
;
7934 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
7935 doc
: /* Specify the coding system for write operations.
7936 Programs bind this variable with `let', but you should not set it globally.
7937 If the value is a coding system, it is used for encoding of output,
7938 when writing it to a file and when sending it to a file or subprocess.
7940 If this does not specify a coding system, an appropriate element
7941 is used from one of the coding system alists:
7942 There are three such tables, `file-coding-system-alist',
7943 `process-coding-system-alist', and `network-coding-system-alist'.
7944 For output to files, if the above procedure does not specify a coding system,
7945 the value of `buffer-file-coding-system' is used. */);
7946 Vcoding_system_for_write
= Qnil
;
7948 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
7949 doc
: /* Coding system used in the latest file or process I/O.
7950 Also set by `encode-coding-region', `decode-coding-region',
7951 `encode-coding-string' and `decode-coding-string'. */);
7952 Vlast_coding_system_used
= Qnil
;
7954 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
7955 doc
: /* *Non-nil means always inhibit code conversion of end-of-line format.
7956 See info node `Coding Systems' and info node `Text and Binary' concerning
7957 such conversion. */);
7958 inhibit_eol_conversion
= 0;
7960 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
7961 doc
: /* Non-nil means process buffer inherits coding system of process output.
7962 Bind it to t if the process output is to be treated as if it were a file
7963 read from some filesystem. */);
7964 inherit_process_coding_system
= 0;
7966 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
7967 doc
: /* Alist to decide a coding system to use for a file I/O operation.
7968 The format is ((PATTERN . VAL) ...),
7969 where PATTERN is a regular expression matching a file name,
7970 VAL is a coding system, a cons of coding systems, or a function symbol.
7971 If VAL is a coding system, it is used for both decoding and encoding
7973 If VAL is a cons of coding systems, the car part is used for decoding,
7974 and the cdr part is used for encoding.
7975 If VAL is a function symbol, the function must return a coding system
7976 or a cons of coding systems which are used as above. The function is
7977 called with an argument that is a list of the arguments with which
7978 `find-operation-coding-system' was called. If the function can't decide
7979 a coding system, it can return `undecided' so that the normal
7980 code-detection is performed.
7982 See also the function `find-operation-coding-system'
7983 and the variable `auto-coding-alist'. */);
7984 Vfile_coding_system_alist
= Qnil
;
7986 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
7987 doc
: /* Alist to decide a coding system to use for a process I/O operation.
7988 The format is ((PATTERN . VAL) ...),
7989 where PATTERN is a regular expression matching a program name,
7990 VAL is a coding system, a cons of coding systems, or a function symbol.
7991 If VAL is a coding system, it is used for both decoding what received
7992 from the program and encoding what sent to the program.
7993 If VAL is a cons of coding systems, the car part is used for decoding,
7994 and the cdr part is used for encoding.
7995 If VAL is a function symbol, the function must return a coding system
7996 or a cons of coding systems which are used as above.
7998 See also the function `find-operation-coding-system'. */);
7999 Vprocess_coding_system_alist
= Qnil
;
8001 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
8002 doc
: /* Alist to decide a coding system to use for a network I/O operation.
8003 The format is ((PATTERN . VAL) ...),
8004 where PATTERN is a regular expression matching a network service name
8005 or is a port number to connect to,
8006 VAL is a coding system, a cons of coding systems, or a function symbol.
8007 If VAL is a coding system, it is used for both decoding what received
8008 from the network stream and encoding what sent to the network stream.
8009 If VAL is a cons of coding systems, the car part is used for decoding,
8010 and the cdr part is used for encoding.
8011 If VAL is a function symbol, the function must return a coding system
8012 or a cons of coding systems which are used as above.
8014 See also the function `find-operation-coding-system'. */);
8015 Vnetwork_coding_system_alist
= Qnil
;
8017 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
8018 doc
: /* Coding system to use with system messages.
8019 Also used for decoding keyboard input on X Window system. */);
8020 Vlocale_coding_system
= Qnil
;
8022 /* The eol mnemonics are reset in startup.el system-dependently. */
8023 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
8024 doc
: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
8025 eol_mnemonic_unix
= build_string (":");
8027 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
8028 doc
: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
8029 eol_mnemonic_dos
= build_string ("\\");
8031 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
8032 doc
: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
8033 eol_mnemonic_mac
= build_string ("/");
8035 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
8036 doc
: /* *String displayed in mode line when end-of-line format is not yet determined. */);
8037 eol_mnemonic_undecided
= build_string (":");
8039 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
8040 doc
: /* *Non-nil enables character translation while encoding and decoding. */);
8041 Venable_character_translation
= Qt
;
8043 DEFVAR_LISP ("standard-translation-table-for-decode",
8044 &Vstandard_translation_table_for_decode
,
8045 doc
: /* Table for translating characters while decoding. */);
8046 Vstandard_translation_table_for_decode
= Qnil
;
8048 DEFVAR_LISP ("standard-translation-table-for-encode",
8049 &Vstandard_translation_table_for_encode
,
8050 doc
: /* Table for translating characters while encoding. */);
8051 Vstandard_translation_table_for_encode
= Qnil
;
8053 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
8054 doc
: /* Alist of charsets vs revision numbers.
8055 While encoding, if a charset (car part of an element) is found,
8056 designate it with the escape sequence identifying revision (cdr part of the element). */);
8057 Vcharset_revision_alist
= Qnil
;
8059 DEFVAR_LISP ("default-process-coding-system",
8060 &Vdefault_process_coding_system
,
8061 doc
: /* Cons of coding systems used for process I/O by default.
8062 The car part is used for decoding a process output,
8063 the cdr part is used for encoding a text to be sent to a process. */);
8064 Vdefault_process_coding_system
= Qnil
;
8066 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
8067 doc
: /* Table of extra Latin codes in the range 128..159 (inclusive).
8068 This is a vector of length 256.
8069 If Nth element is non-nil, the existence of code N in a file
8070 \(or output of subprocess) doesn't prevent it to be detected as
8071 a coding system of ISO 2022 variant which has a flag
8072 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
8073 or reading output of a subprocess.
8074 Only 128th through 159th elements has a meaning. */);
8075 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
8077 DEFVAR_LISP ("select-safe-coding-system-function",
8078 &Vselect_safe_coding_system_function
,
8079 doc
: /* Function to call to select safe coding system for encoding a text.
8081 If set, this function is called to force a user to select a proper
8082 coding system which can encode the text in the case that a default
8083 coding system used in each operation can't encode the text.
8085 The default value is `select-safe-coding-system' (which see). */);
8086 Vselect_safe_coding_system_function
= Qnil
;
8088 DEFVAR_BOOL ("coding-system-require-warning",
8089 &coding_system_require_warning
,
8090 doc
: /* Internal use only.
8091 If non-nil, on writing a file, `select-safe-coding-system-function' is
8092 called even if `coding-system-for-write' is non-nil. The command
8093 `universal-coding-system-argument' binds this variable to t temporarily. */);
8094 coding_system_require_warning
= 0;
8097 DEFVAR_BOOL ("inhibit-iso-escape-detection",
8098 &inhibit_iso_escape_detection
,
8099 doc
: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
8101 By default, on reading a file, Emacs tries to detect how the text is
8102 encoded. This code detection is sensitive to escape sequences. If
8103 the sequence is valid as ISO2022, the code is determined as one of
8104 the ISO2022 encodings, and the file is decoded by the corresponding
8105 coding system (e.g. `iso-2022-7bit').
8107 However, there may be a case that you want to read escape sequences in
8108 a file as is. In such a case, you can set this variable to non-nil.
8109 Then, as the code detection ignores any escape sequences, no file is
8110 detected as encoded in some ISO2022 encoding. The result is that all
8111 escape sequences become visible in a buffer.
8113 The default value is nil, and it is strongly recommended not to change
8114 it. That is because many Emacs Lisp source files that contain
8115 non-ASCII characters are encoded by the coding system `iso-2022-7bit'
8116 in Emacs's distribution, and they won't be decoded correctly on
8117 reading if you suppress escape sequence detection.
8119 The other way to read escape sequences in a file without decoding is
8120 to explicitly specify some coding system that doesn't use ISO2022's
8121 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
8122 inhibit_iso_escape_detection
= 0;
8124 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input
,
8125 doc
: /* Char table for translating self-inserting characters.
8126 This is applied to the result of input methods, not their input. See also
8127 `keyboard-translate-table'. */);
8128 Vtranslation_table_for_input
= Qnil
;
8132 emacs_strerror (error_number
)
8137 synchronize_system_messages_locale ();
8138 str
= strerror (error_number
);
8140 if (! NILP (Vlocale_coding_system
))
8142 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
8143 Vlocale_coding_system
,
8145 str
= (char *) SDATA (dec
);
8153 /* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
8154 (do not change this comment) */