1 /* Coding system handler (conversion, detection, and etc).
2 Copyright (C) 1995,97,1998,2002,2003 Electrotechnical Laboratory, JAPAN.
3 Licensed to the Free Software Foundation.
4 Copyright (C) 2001,2002,2003 Free Software Foundation, Inc.
6 This file is part of GNU Emacs.
8 GNU Emacs is free software; you can redistribute it and/or modify
9 it under the terms of the GNU General Public License as published by
10 the Free Software Foundation; either version 2, or (at your option)
13 GNU Emacs is distributed in the hope that it will be useful,
14 but WITHOUT ANY WARRANTY; without even the implied warranty of
15 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
16 GNU General Public License for more details.
18 You should have received a copy of the GNU General Public License
19 along with GNU Emacs; see the file COPYING. If not, write to
20 the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
21 Boston, MA 02111-1307, USA. */
23 /*** TABLE OF CONTENTS ***
27 2. Emacs' internal format (emacs-mule) handlers
29 4. Shift-JIS and BIG5 handlers
31 6. End-of-line handlers
32 7. C library functions
33 8. Emacs Lisp library functions
38 /*** 0. General comments ***/
41 /*** GENERAL NOTE on CODING SYSTEMS ***
43 A coding system is an encoding mechanism for one or more character
44 sets. Here's a list of coding systems which Emacs can handle. When
45 we say "decode", it means converting some other coding system to
46 Emacs' internal format (emacs-mule), and when we say "encode",
47 it means converting the coding system emacs-mule to some other
50 0. Emacs' internal format (emacs-mule)
52 Emacs itself holds a multi-lingual character in buffers and strings
53 in a special format. Details are described in section 2.
57 The most famous coding system for multiple character sets. X's
58 Compound Text, various EUCs (Extended Unix Code), and coding
59 systems used in Internet communication such as ISO-2022-JP are
60 all variants of ISO2022. Details are described in section 3.
62 2. SJIS (or Shift-JIS or MS-Kanji-Code)
64 A coding system to encode character sets: ASCII, JISX0201, and
65 JISX0208. Widely used for PC's in Japan. Details are described in
70 A coding system to encode the character sets ASCII and Big5. Widely
71 used for Chinese (mainly in Taiwan and Hong Kong). Details are
72 described in section 4. In this file, when we write "BIG5"
73 (all uppercase), we mean the coding system, and when we write
74 "Big5" (capitalized), we mean the character set.
78 A coding system for text containing random 8-bit code. Emacs does
79 no code conversion on such text except for end-of-line format.
83 If a user wants to read/write text encoded in a coding system not
84 listed above, he can supply a decoder and an encoder for it as CCL
85 (Code Conversion Language) programs. Emacs executes the CCL program
86 while reading/writing.
88 Emacs represents a coding system by a Lisp symbol that has a property
89 `coding-system'. But, before actually using the coding system, the
90 information about it is set in a structure of type `struct
91 coding_system' for rapid processing. See section 6 for more details.
95 /*** GENERAL NOTES on END-OF-LINE FORMAT ***
97 How end-of-line of text is encoded depends on the operating system.
98 For instance, Unix's format is just one byte of `line-feed' code,
99 whereas DOS's format is two-byte sequence of `carriage-return' and
100 `line-feed' codes. MacOS's format is usually one byte of
103 Since text character encoding and end-of-line encoding are
104 independent, any coding system described above can have any
105 end-of-line format. So Emacs has information about end-of-line
106 format in each coding-system. See section 6 for more details.
110 /*** GENERAL NOTES on `detect_coding_XXX ()' functions ***
112 These functions check if a text between SRC and SRC_END is encoded
113 in the coding system category XXX. Each returns an integer value in
114 which appropriate flag bits for the category XXX are set. The flag
115 bits are defined in macros CODING_CATEGORY_MASK_XXX. Below is the
116 template for these functions. If MULTIBYTEP is nonzero, 8-bit codes
117 of the range 0x80..0x9F are in multibyte form. */
120 detect_coding_emacs_mule (src
, src_end
, multibytep
)
121 unsigned char *src
, *src_end
;
128 /*** GENERAL NOTES on `decode_coding_XXX ()' functions ***
130 These functions decode SRC_BYTES length of unibyte text at SOURCE
131 encoded in CODING to Emacs' internal format. The resulting
132 multibyte text goes to a place pointed to by DESTINATION, the length
133 of which should not exceed DST_BYTES.
135 These functions set the information about original and decoded texts
136 in the members `produced', `produced_char', `consumed', and
137 `consumed_char' of the structure *CODING. They also set the member
138 `result' to one of CODING_FINISH_XXX indicating how the decoding
141 DST_BYTES zero means that the source area and destination area are
142 overlapped, which means that we can produce a decoded text until it
143 reaches the head of the not-yet-decoded source text.
145 Below is a template for these functions. */
148 decode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
149 struct coding_system
*coding
;
150 unsigned char *source
, *destination
;
151 int src_bytes
, dst_bytes
;
157 /*** GENERAL NOTES on `encode_coding_XXX ()' functions ***
159 These functions encode SRC_BYTES length text at SOURCE from Emacs'
160 internal multibyte format to CODING. The resulting unibyte text
161 goes to a place pointed to by DESTINATION, the length of which
162 should not exceed DST_BYTES.
164 These functions set the information about original and encoded texts
165 in the members `produced', `produced_char', `consumed', and
166 `consumed_char' of the structure *CODING. They also set the member
167 `result' to one of CODING_FINISH_XXX indicating how the encoding
170 DST_BYTES zero means that the source area and destination area are
171 overlapped, which means that we can produce encoded text until it
172 reaches at the head of the not-yet-encoded source text.
174 Below is a template for these functions. */
177 encode_coding_XXX (coding
, source
, destination
, src_bytes
, dst_bytes
)
178 struct coding_system
*coding
;
179 unsigned char *source
, *destination
;
180 int src_bytes
, dst_bytes
;
186 /*** COMMONLY USED MACROS ***/
188 /* The following two macros ONE_MORE_BYTE and TWO_MORE_BYTES safely
189 get one, two, and three bytes from the source text respectively.
190 If there are not enough bytes in the source, they jump to
191 `label_end_of_loop'. The caller should set variables `coding',
192 `src' and `src_end' to appropriate pointer in advance. These
193 macros are called from decoding routines `decode_coding_XXX', thus
194 it is assumed that the source text is unibyte. */
196 #define ONE_MORE_BYTE(c1) \
198 if (src >= src_end) \
200 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
201 goto label_end_of_loop; \
206 #define TWO_MORE_BYTES(c1, c2) \
208 if (src + 1 >= src_end) \
210 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
211 goto label_end_of_loop; \
218 /* Like ONE_MORE_BYTE, but 8-bit bytes of data at SRC are in multibyte
219 form if MULTIBYTEP is nonzero. */
221 #define ONE_MORE_BYTE_CHECK_MULTIBYTE(c1, multibytep) \
223 if (src >= src_end) \
225 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
226 goto label_end_of_loop; \
229 if (multibytep && c1 == LEADING_CODE_8_BIT_CONTROL) \
230 c1 = *src++ - 0x20; \
233 /* Set C to the next character at the source text pointed by `src'.
234 If there are not enough characters in the source, jump to
235 `label_end_of_loop'. The caller should set variables `coding'
236 `src', `src_end', and `translation_table' to appropriate pointers
237 in advance. This macro is used in encoding routines
238 `encode_coding_XXX', thus it assumes that the source text is in
239 multibyte form except for 8-bit characters. 8-bit characters are
240 in multibyte form if coding->src_multibyte is nonzero, else they
241 are represented by a single byte. */
243 #define ONE_MORE_CHAR(c) \
245 int len = src_end - src; \
249 coding->result = CODING_FINISH_INSUFFICIENT_SRC; \
250 goto label_end_of_loop; \
252 if (coding->src_multibyte \
253 || UNIBYTE_STR_AS_MULTIBYTE_P (src, len, bytes)) \
254 c = STRING_CHAR_AND_LENGTH (src, len, bytes); \
256 c = *src, bytes = 1; \
257 if (!NILP (translation_table)) \
258 c = translate_char (translation_table, c, -1, 0, 0); \
263 /* Produce a multibyte form of character C to `dst'. Jump to
264 `label_end_of_loop' if there's not enough space at `dst'.
266 If we are now in the middle of a composition sequence, the decoded
267 character may be ALTCHAR (for the current composition). In that
268 case, the character goes to coding->cmp_data->data instead of
271 This macro is used in decoding routines. */
273 #define EMIT_CHAR(c) \
275 if (! COMPOSING_P (coding) \
276 || coding->composing == COMPOSITION_RELATIVE \
277 || coding->composing == COMPOSITION_WITH_RULE) \
279 int bytes = CHAR_BYTES (c); \
280 if ((dst + bytes) > (dst_bytes ? dst_end : src)) \
282 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
283 goto label_end_of_loop; \
285 dst += CHAR_STRING (c, dst); \
286 coding->produced_char++; \
289 if (COMPOSING_P (coding) \
290 && coding->composing != COMPOSITION_RELATIVE) \
292 CODING_ADD_COMPOSITION_COMPONENT (coding, c); \
293 coding->composition_rule_follows \
294 = coding->composing != COMPOSITION_WITH_ALTCHARS; \
299 #define EMIT_ONE_BYTE(c) \
301 if (dst >= (dst_bytes ? dst_end : src)) \
303 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
304 goto label_end_of_loop; \
309 #define EMIT_TWO_BYTES(c1, c2) \
311 if (dst + 2 > (dst_bytes ? dst_end : src)) \
313 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
314 goto label_end_of_loop; \
316 *dst++ = c1, *dst++ = c2; \
319 #define EMIT_BYTES(from, to) \
321 if (dst + (to - from) > (dst_bytes ? dst_end : src)) \
323 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
324 goto label_end_of_loop; \
331 /*** 1. Preamble ***/
344 #include "composite.h"
348 #include "intervals.h"
350 #else /* not emacs */
354 #endif /* not emacs */
356 Lisp_Object Qcoding_system
, Qeol_type
;
357 Lisp_Object Qbuffer_file_coding_system
;
358 Lisp_Object Qpost_read_conversion
, Qpre_write_conversion
;
359 Lisp_Object Qno_conversion
, Qundecided
;
360 Lisp_Object Qcoding_system_history
;
361 Lisp_Object Qsafe_chars
;
362 Lisp_Object Qvalid_codes
;
364 extern Lisp_Object Qinsert_file_contents
, Qwrite_region
;
365 Lisp_Object Qcall_process
, Qcall_process_region
, Qprocess_argument
;
366 Lisp_Object Qstart_process
, Qopen_network_stream
;
367 Lisp_Object Qtarget_idx
;
369 /* If a symbol has this property, evaluate the value to define the
370 symbol as a coding system. */
371 Lisp_Object Qcoding_system_define_form
;
373 Lisp_Object Vselect_safe_coding_system_function
;
375 int coding_system_require_warning
;
377 /* Mnemonic string for each format of end-of-line. */
378 Lisp_Object eol_mnemonic_unix
, eol_mnemonic_dos
, eol_mnemonic_mac
;
379 /* Mnemonic string to indicate format of end-of-line is not yet
381 Lisp_Object eol_mnemonic_undecided
;
383 /* Format of end-of-line decided by system. This is CODING_EOL_LF on
384 Unix, CODING_EOL_CRLF on DOS/Windows, and CODING_EOL_CR on Mac. */
389 /* Information about which coding system is safe for which chars.
390 The value has the form (GENERIC-LIST . NON-GENERIC-ALIST).
392 GENERIC-LIST is a list of generic coding systems which can encode
395 NON-GENERIC-ALIST is an alist of non generic coding systems vs the
396 corresponding char table that contains safe chars. */
397 Lisp_Object Vcoding_system_safe_chars
;
399 Lisp_Object Vcoding_system_list
, Vcoding_system_alist
;
401 Lisp_Object Qcoding_system_p
, Qcoding_system_error
;
403 /* Coding system emacs-mule and raw-text are for converting only
404 end-of-line format. */
405 Lisp_Object Qemacs_mule
, Qraw_text
;
409 /* Coding-systems are handed between Emacs Lisp programs and C internal
410 routines by the following three variables. */
411 /* Coding-system for reading files and receiving data from process. */
412 Lisp_Object Vcoding_system_for_read
;
413 /* Coding-system for writing files and sending data to process. */
414 Lisp_Object Vcoding_system_for_write
;
415 /* Coding-system actually used in the latest I/O. */
416 Lisp_Object Vlast_coding_system_used
;
418 /* A vector of length 256 which contains information about special
419 Latin codes (especially for dealing with Microsoft codes). */
420 Lisp_Object Vlatin_extra_code_table
;
422 /* Flag to inhibit code conversion of end-of-line format. */
423 int inhibit_eol_conversion
;
425 /* Flag to inhibit ISO2022 escape sequence detection. */
426 int inhibit_iso_escape_detection
;
428 /* Flag to make buffer-file-coding-system inherit from process-coding. */
429 int inherit_process_coding_system
;
431 /* Coding system to be used to encode text for terminal display. */
432 struct coding_system terminal_coding
;
434 /* Coding system to be used to encode text for terminal display when
435 terminal coding system is nil. */
436 struct coding_system safe_terminal_coding
;
438 /* Coding system of what is sent from terminal keyboard. */
439 struct coding_system keyboard_coding
;
441 /* Default coding system to be used to write a file. */
442 struct coding_system default_buffer_file_coding
;
444 Lisp_Object Vfile_coding_system_alist
;
445 Lisp_Object Vprocess_coding_system_alist
;
446 Lisp_Object Vnetwork_coding_system_alist
;
448 Lisp_Object Vlocale_coding_system
;
452 Lisp_Object Qcoding_category
, Qcoding_category_index
;
454 /* List of symbols `coding-category-xxx' ordered by priority. */
455 Lisp_Object Vcoding_category_list
;
457 /* Table of coding categories (Lisp symbols). */
458 Lisp_Object Vcoding_category_table
;
460 /* Table of names of symbol for each coding-category. */
461 char *coding_category_name
[CODING_CATEGORY_IDX_MAX
] = {
462 "coding-category-emacs-mule",
463 "coding-category-sjis",
464 "coding-category-iso-7",
465 "coding-category-iso-7-tight",
466 "coding-category-iso-8-1",
467 "coding-category-iso-8-2",
468 "coding-category-iso-7-else",
469 "coding-category-iso-8-else",
470 "coding-category-ccl",
471 "coding-category-big5",
472 "coding-category-utf-8",
473 "coding-category-utf-16-be",
474 "coding-category-utf-16-le",
475 "coding-category-raw-text",
476 "coding-category-binary"
479 /* Table of pointers to coding systems corresponding to each coding
481 struct coding_system
*coding_system_table
[CODING_CATEGORY_IDX_MAX
];
483 /* Table of coding category masks. Nth element is a mask for a coding
484 category of which priority is Nth. */
486 int coding_priorities
[CODING_CATEGORY_IDX_MAX
];
488 /* Flag to tell if we look up translation table on character code
490 Lisp_Object Venable_character_translation
;
491 /* Standard translation table to look up on decoding (reading). */
492 Lisp_Object Vstandard_translation_table_for_decode
;
493 /* Standard translation table to look up on encoding (writing). */
494 Lisp_Object Vstandard_translation_table_for_encode
;
496 Lisp_Object Qtranslation_table
;
497 Lisp_Object Qtranslation_table_id
;
498 Lisp_Object Qtranslation_table_for_decode
;
499 Lisp_Object Qtranslation_table_for_encode
;
501 /* Alist of charsets vs revision number. */
502 Lisp_Object Vcharset_revision_alist
;
504 /* Default coding systems used for process I/O. */
505 Lisp_Object Vdefault_process_coding_system
;
507 /* Char table for translating Quail and self-inserting input. */
508 Lisp_Object Vtranslation_table_for_input
;
510 /* Global flag to tell that we can't call post-read-conversion and
511 pre-write-conversion functions. Usually the value is zero, but it
512 is set to 1 temporarily while such functions are running. This is
513 to avoid infinite recursive call. */
514 static int inhibit_pre_post_conversion
;
516 Lisp_Object Qchar_coding_system
;
518 /* Return `safe-chars' property of CODING_SYSTEM (symbol). Don't check
522 coding_safe_chars (coding_system
)
523 Lisp_Object coding_system
;
525 Lisp_Object coding_spec
, plist
, safe_chars
;
527 coding_spec
= Fget (coding_system
, Qcoding_system
);
528 plist
= XVECTOR (coding_spec
)->contents
[3];
529 safe_chars
= Fplist_get (XVECTOR (coding_spec
)->contents
[3], Qsafe_chars
);
530 return (CHAR_TABLE_P (safe_chars
) ? safe_chars
: Qt
);
533 #define CODING_SAFE_CHAR_P(safe_chars, c) \
534 (EQ (safe_chars, Qt) || !NILP (CHAR_TABLE_REF (safe_chars, c)))
537 /*** 2. Emacs internal format (emacs-mule) handlers ***/
539 /* Emacs' internal format for representation of multiple character
540 sets is a kind of multi-byte encoding, i.e. characters are
541 represented by variable-length sequences of one-byte codes.
543 ASCII characters and control characters (e.g. `tab', `newline') are
544 represented by one-byte sequences which are their ASCII codes, in
545 the range 0x00 through 0x7F.
547 8-bit characters of the range 0x80..0x9F are represented by
548 two-byte sequences of LEADING_CODE_8_BIT_CONTROL and (their 8-bit
551 8-bit characters of the range 0xA0..0xFF are represented by
552 one-byte sequences which are their 8-bit code.
554 The other characters are represented by a sequence of `base
555 leading-code', optional `extended leading-code', and one or two
556 `position-code's. The length of the sequence is determined by the
557 base leading-code. Leading-code takes the range 0x81 through 0x9D,
558 whereas extended leading-code and position-code take the range 0xA0
559 through 0xFF. See `charset.h' for more details about leading-code
562 --- CODE RANGE of Emacs' internal format ---
566 eight-bit-control LEADING_CODE_8_BIT_CONTROL + 0xA0..0xBF
567 eight-bit-graphic 0xA0..0xBF
568 ELSE 0x81..0x9D + [0xA0..0xFF]+
569 ---------------------------------------------
571 As this is the internal character representation, the format is
572 usually not used externally (i.e. in a file or in a data sent to a
573 process). But, it is possible to have a text externally in this
574 format (i.e. by encoding by the coding system `emacs-mule').
576 In that case, a sequence of one-byte codes has a slightly different
579 Firstly, all characters in eight-bit-control are represented by
580 one-byte sequences which are their 8-bit code.
582 Next, character composition data are represented by the byte
583 sequence of the form: 0x80 METHOD BYTES CHARS COMPONENT ...,
585 METHOD is 0xF0 plus one of composition method (enum
588 BYTES is 0xA0 plus the byte length of these composition data,
590 CHARS is 0xA0 plus the number of characters composed by these
593 COMPONENTs are characters of multibyte form or composition
594 rules encoded by two-byte of ASCII codes.
596 In addition, for backward compatibility, the following formats are
597 also recognized as composition data on decoding.
600 0x80 0xFF MSEQ RULE MSEQ RULE ... MSEQ
603 MSEQ is a multibyte form but in these special format:
604 ASCII: 0xA0 ASCII_CODE+0x80,
605 other: LEADING_CODE+0x20 FOLLOWING-BYTE ...,
606 RULE is a one byte code of the range 0xA0..0xF0 that
607 represents a composition rule.
610 enum emacs_code_class_type emacs_code_class
[256];
612 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
613 Check if a text is encoded in Emacs' internal format. If it is,
614 return CODING_CATEGORY_MASK_EMACS_MULE, else return 0. */
617 detect_coding_emacs_mule (src
, src_end
, multibytep
)
618 unsigned char *src
, *src_end
;
623 /* Dummy for ONE_MORE_BYTE. */
624 struct coding_system dummy_coding
;
625 struct coding_system
*coding
= &dummy_coding
;
629 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
637 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
646 if (c
== ISO_CODE_ESC
|| c
== ISO_CODE_SI
|| c
== ISO_CODE_SO
)
649 else if (c
>= 0x80 && c
< 0xA0)
652 /* Old leading code for a composite character. */
656 unsigned char *src_base
= src
- 1;
659 if (!UNIBYTE_STR_AS_MULTIBYTE_P (src_base
, src_end
- src_base
,
662 src
= src_base
+ bytes
;
667 return CODING_CATEGORY_MASK_EMACS_MULE
;
671 /* Record the starting position START and METHOD of one composition. */
673 #define CODING_ADD_COMPOSITION_START(coding, start, method) \
675 struct composition_data *cmp_data = coding->cmp_data; \
676 int *data = cmp_data->data + cmp_data->used; \
677 coding->cmp_data_start = cmp_data->used; \
679 data[1] = cmp_data->char_offset + start; \
680 data[3] = (int) method; \
681 cmp_data->used += 4; \
684 /* Record the ending position END of the current composition. */
686 #define CODING_ADD_COMPOSITION_END(coding, end) \
688 struct composition_data *cmp_data = coding->cmp_data; \
689 int *data = cmp_data->data + coding->cmp_data_start; \
690 data[0] = cmp_data->used - coding->cmp_data_start; \
691 data[2] = cmp_data->char_offset + end; \
694 /* Record one COMPONENT (alternate character or composition rule). */
696 #define CODING_ADD_COMPOSITION_COMPONENT(coding, component) \
698 coding->cmp_data->data[coding->cmp_data->used++] = component; \
699 if (coding->cmp_data->used - coding->cmp_data_start \
700 == COMPOSITION_DATA_MAX_BUNCH_LENGTH) \
702 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
703 coding->composing = COMPOSITION_NO; \
708 /* Get one byte from a data pointed by SRC and increment SRC. If SRC
709 is not less than SRC_END, return -1 without incrementing Src. */
711 #define SAFE_ONE_MORE_BYTE() (src >= src_end ? -1 : *src++)
714 /* Decode a character represented as a component of composition
715 sequence of Emacs 20 style at SRC. Set C to that character, store
716 its multibyte form sequence at P, and set P to the end of that
717 sequence. If no valid character is found, set C to -1. */
719 #define DECODE_EMACS_MULE_COMPOSITION_CHAR(c, p) \
723 c = SAFE_ONE_MORE_BYTE (); \
726 if (CHAR_HEAD_P (c)) \
728 else if (c == 0xA0) \
730 c = SAFE_ONE_MORE_BYTE (); \
739 else if (BASE_LEADING_CODE_P (c - 0x20)) \
741 unsigned char *p0 = p; \
745 bytes = BYTES_BY_CHAR_HEAD (c); \
748 c = SAFE_ONE_MORE_BYTE (); \
753 if (UNIBYTE_STR_AS_MULTIBYTE_P (p0, p - p0, bytes) \
754 || (coding->flags /* We are recovering a file. */ \
755 && p0[0] == LEADING_CODE_8_BIT_CONTROL \
756 && ! CHAR_HEAD_P (p0[1]))) \
757 c = STRING_CHAR (p0, bytes); \
766 /* Decode a composition rule represented as a component of composition
767 sequence of Emacs 20 style at SRC. Set C to the rule. If not
768 valid rule is found, set C to -1. */
770 #define DECODE_EMACS_MULE_COMPOSITION_RULE(c) \
772 c = SAFE_ONE_MORE_BYTE (); \
774 if (c < 0 || c >= 81) \
778 gref = c / 9, nref = c % 9; \
779 c = COMPOSITION_ENCODE_RULE (gref, nref); \
784 /* Decode composition sequence encoded by `emacs-mule' at the source
785 pointed by SRC. SRC_END is the end of source. Store information
786 of the composition in CODING->cmp_data.
788 For backward compatibility, decode also a composition sequence of
789 Emacs 20 style. In that case, the composition sequence contains
790 characters that should be extracted into a buffer or string. Store
791 those characters at *DESTINATION in multibyte form.
793 If we encounter an invalid byte sequence, return 0.
794 If we encounter an insufficient source or destination, or
795 insufficient space in CODING->cmp_data, return 1.
796 Otherwise, return consumed bytes in the source.
800 decode_composition_emacs_mule (coding
, src
, src_end
,
801 destination
, dst_end
, dst_bytes
)
802 struct coding_system
*coding
;
803 unsigned char *src
, *src_end
, **destination
, *dst_end
;
806 unsigned char *dst
= *destination
;
807 int method
, data_len
, nchars
;
808 unsigned char *src_base
= src
++;
809 /* Store components of composition. */
810 int component
[COMPOSITION_DATA_MAX_BUNCH_LENGTH
];
812 /* Store multibyte form of characters to be composed. This is for
813 Emacs 20 style composition sequence. */
814 unsigned char buf
[MAX_COMPOSITION_COMPONENTS
* MAX_MULTIBYTE_LENGTH
];
815 unsigned char *bufp
= buf
;
816 int c
, i
, gref
, nref
;
818 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
819 >= COMPOSITION_DATA_SIZE
)
821 coding
->result
= CODING_FINISH_INSUFFICIENT_CMP
;
826 if (c
- 0xF0 >= COMPOSITION_RELATIVE
827 && c
- 0xF0 <= COMPOSITION_WITH_RULE_ALTCHARS
)
832 with_rule
= (method
== COMPOSITION_WITH_RULE
833 || method
== COMPOSITION_WITH_RULE_ALTCHARS
);
837 || src_base
+ data_len
> src_end
)
843 for (ncomponent
= 0; src
< src_base
+ data_len
; ncomponent
++)
845 /* If it is longer than this, it can't be valid. */
846 if (ncomponent
>= COMPOSITION_DATA_MAX_BUNCH_LENGTH
)
849 if (ncomponent
% 2 && with_rule
)
851 ONE_MORE_BYTE (gref
);
853 ONE_MORE_BYTE (nref
);
855 c
= COMPOSITION_ENCODE_RULE (gref
, nref
);
860 if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
861 || (coding
->flags
/* We are recovering a file. */
862 && src
[0] == LEADING_CODE_8_BIT_CONTROL
863 && ! CHAR_HEAD_P (src
[1])))
864 c
= STRING_CHAR (src
, bytes
);
869 component
[ncomponent
] = c
;
874 /* This may be an old Emacs 20 style format. See the comment at
875 the section 2 of this file. */
876 while (src
< src_end
&& !CHAR_HEAD_P (*src
)) src
++;
878 && !(coding
->mode
& CODING_MODE_LAST_BLOCK
))
879 goto label_end_of_loop
;
885 method
= COMPOSITION_RELATIVE
;
886 for (ncomponent
= 0; ncomponent
< MAX_COMPOSITION_COMPONENTS
;)
888 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
891 component
[ncomponent
++] = c
;
899 method
= COMPOSITION_WITH_RULE
;
901 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
906 ncomponent
< MAX_COMPOSITION_COMPONENTS
* 2 - 1;)
908 DECODE_EMACS_MULE_COMPOSITION_RULE (c
);
911 component
[ncomponent
++] = c
;
912 DECODE_EMACS_MULE_COMPOSITION_CHAR (c
, bufp
);
915 component
[ncomponent
++] = c
;
919 nchars
= (ncomponent
+ 1) / 2;
925 if (buf
== bufp
|| dst
+ (bufp
- buf
) <= (dst_bytes
? dst_end
: src
))
927 CODING_ADD_COMPOSITION_START (coding
, coding
->produced_char
, method
);
928 for (i
= 0; i
< ncomponent
; i
++)
929 CODING_ADD_COMPOSITION_COMPONENT (coding
, component
[i
]);
930 CODING_ADD_COMPOSITION_END (coding
, coding
->produced_char
+ nchars
);
933 unsigned char *p
= buf
;
934 EMIT_BYTES (p
, bufp
);
935 *destination
+= bufp
- buf
;
936 coding
->produced_char
+= nchars
;
938 return (src
- src_base
);
944 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
947 decode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
948 struct coding_system
*coding
;
949 unsigned char *source
, *destination
;
950 int src_bytes
, dst_bytes
;
952 unsigned char *src
= source
;
953 unsigned char *src_end
= source
+ src_bytes
;
954 unsigned char *dst
= destination
;
955 unsigned char *dst_end
= destination
+ dst_bytes
;
956 /* SRC_BASE remembers the start position in source in each loop.
957 The loop will be exited when there's not enough source code, or
958 when there's not enough destination area to produce a
960 unsigned char *src_base
;
962 coding
->produced_char
= 0;
963 while ((src_base
= src
) < src_end
)
965 unsigned char tmp
[MAX_MULTIBYTE_LENGTH
], *p
;
972 if (coding
->eol_type
== CODING_EOL_CR
)
974 else if (coding
->eol_type
== CODING_EOL_CRLF
)
984 coding
->produced_char
++;
987 else if (*src
== '\n')
989 if ((coding
->eol_type
== CODING_EOL_CR
990 || coding
->eol_type
== CODING_EOL_CRLF
)
991 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
993 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
994 goto label_end_of_loop
;
997 coding
->produced_char
++;
1000 else if (*src
== 0x80 && coding
->cmp_data
)
1002 /* Start of composition data. */
1003 int consumed
= decode_composition_emacs_mule (coding
, src
, src_end
,
1007 goto label_end_of_loop
;
1008 else if (consumed
> 0)
1013 bytes
= CHAR_STRING (*src
, tmp
);
1017 else if (UNIBYTE_STR_AS_MULTIBYTE_P (src
, src_end
- src
, bytes
)
1018 || (coding
->flags
/* We are recovering a file. */
1019 && src
[0] == LEADING_CODE_8_BIT_CONTROL
1020 && ! CHAR_HEAD_P (src
[1])))
1029 bytes
= BYTES_BY_CHAR_HEAD (*src
);
1031 for (i
= 1; i
< bytes
; i
++)
1034 if (CHAR_HEAD_P (c
))
1039 bytes
= CHAR_STRING (*src_base
, tmp
);
1048 if (dst
+ bytes
>= (dst_bytes
? dst_end
: src
))
1050 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
1053 while (bytes
--) *dst
++ = *p
++;
1054 coding
->produced_char
++;
1057 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
1058 coding
->produced
= dst
- destination
;
1062 /* Encode composition data stored at DATA into a special byte sequence
1063 starting by 0x80. Update CODING->cmp_data_start and maybe
1064 CODING->cmp_data for the next call. */
1066 #define ENCODE_COMPOSITION_EMACS_MULE(coding, data) \
1068 unsigned char buf[1024], *p0 = buf, *p; \
1069 int len = data[0]; \
1073 buf[1] = 0xF0 + data[3]; /* METHOD */ \
1074 buf[3] = 0xA0 + (data[2] - data[1]); /* COMPOSED-CHARS */ \
1076 if (data[3] == COMPOSITION_WITH_RULE \
1077 || data[3] == COMPOSITION_WITH_RULE_ALTCHARS) \
1079 p += CHAR_STRING (data[4], p); \
1080 for (i = 5; i < len; i += 2) \
1083 COMPOSITION_DECODE_RULE (data[i], gref, nref); \
1084 *p++ = 0x20 + gref; \
1085 *p++ = 0x20 + nref; \
1086 p += CHAR_STRING (data[i + 1], p); \
1091 for (i = 4; i < len; i++) \
1092 p += CHAR_STRING (data[i], p); \
1094 buf[2] = 0xA0 + (p - buf); /* COMPONENTS-BYTES */ \
1096 if (dst + (p - buf) + 4 > (dst_bytes ? dst_end : src)) \
1098 coding->result = CODING_FINISH_INSUFFICIENT_DST; \
1099 goto label_end_of_loop; \
1103 coding->cmp_data_start += data[0]; \
1104 if (coding->cmp_data_start == coding->cmp_data->used \
1105 && coding->cmp_data->next) \
1107 coding->cmp_data = coding->cmp_data->next; \
1108 coding->cmp_data_start = 0; \
1113 static void encode_eol
P_ ((struct coding_system
*, const unsigned char *,
1114 unsigned char *, int, int));
1117 encode_coding_emacs_mule (coding
, source
, destination
, src_bytes
, dst_bytes
)
1118 struct coding_system
*coding
;
1119 unsigned char *source
, *destination
;
1120 int src_bytes
, dst_bytes
;
1122 unsigned char *src
= source
;
1123 unsigned char *src_end
= source
+ src_bytes
;
1124 unsigned char *dst
= destination
;
1125 unsigned char *dst_end
= destination
+ dst_bytes
;
1126 unsigned char *src_base
;
1131 Lisp_Object translation_table
;
1133 translation_table
= Qnil
;
1135 /* Optimization for the case that there's no composition. */
1136 if (!coding
->cmp_data
|| coding
->cmp_data
->used
== 0)
1138 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
1142 char_offset
= coding
->cmp_data
->char_offset
;
1143 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1148 /* If SRC starts a composition, encode the information about the
1149 composition in advance. */
1150 if (coding
->cmp_data_start
< coding
->cmp_data
->used
1151 && char_offset
+ coding
->consumed_char
== data
[1])
1153 ENCODE_COMPOSITION_EMACS_MULE (coding
, data
);
1154 char_offset
= coding
->cmp_data
->char_offset
;
1155 data
= coding
->cmp_data
->data
+ coding
->cmp_data_start
;
1159 if (c
== '\n' && (coding
->eol_type
== CODING_EOL_CRLF
1160 || coding
->eol_type
== CODING_EOL_CR
))
1162 if (coding
->eol_type
== CODING_EOL_CRLF
)
1163 EMIT_TWO_BYTES ('\r', c
);
1165 EMIT_ONE_BYTE ('\r');
1167 else if (SINGLE_BYTE_CHAR_P (c
))
1169 if (coding
->flags
&& ! ASCII_BYTE_P (c
))
1171 /* As we are auto saving, retain the multibyte form for
1173 unsigned char buf
[MAX_MULTIBYTE_LENGTH
];
1174 int bytes
= CHAR_STRING (c
, buf
);
1177 EMIT_ONE_BYTE (buf
[0]);
1179 EMIT_TWO_BYTES (buf
[0], buf
[1]);
1185 EMIT_BYTES (src_base
, src
);
1186 coding
->consumed_char
++;
1189 coding
->consumed
= src_base
- source
;
1190 coding
->produced
= coding
->produced_char
= dst
- destination
;
1195 /*** 3. ISO2022 handlers ***/
1197 /* The following note describes the coding system ISO2022 briefly.
1198 Since the intention of this note is to help understand the
1199 functions in this file, some parts are NOT ACCURATE or are OVERLY
1200 SIMPLIFIED. For thorough understanding, please refer to the
1201 original document of ISO2022. This is equivalent to the standard
1202 ECMA-35, obtainable from <URL:http://www.ecma.ch/> (*).
1204 ISO2022 provides many mechanisms to encode several character sets
1205 in 7-bit and 8-bit environments. For 7-bit environments, all text
1206 is encoded using bytes less than 128. This may make the encoded
1207 text a little bit longer, but the text passes more easily through
1208 several types of gateway, some of which strip off the MSB (Most
1211 There are two kinds of character sets: control character sets and
1212 graphic character sets. The former contain control characters such
1213 as `newline' and `escape' to provide control functions (control
1214 functions are also provided by escape sequences). The latter
1215 contain graphic characters such as 'A' and '-'. Emacs recognizes
1216 two control character sets and many graphic character sets.
1218 Graphic character sets are classified into one of the following
1219 four classes, according to the number of bytes (DIMENSION) and
1220 number of characters in one dimension (CHARS) of the set:
1221 - DIMENSION1_CHARS94
1222 - DIMENSION1_CHARS96
1223 - DIMENSION2_CHARS94
1224 - DIMENSION2_CHARS96
1226 In addition, each character set is assigned an identification tag,
1227 unique for each set, called the "final character" (denoted as <F>
1228 hereafter). The <F> of each character set is decided by ECMA(*)
1229 when it is registered in ISO. The code range of <F> is 0x30..0x7F
1230 (0x30..0x3F are for private use only).
1232 Note (*): ECMA = European Computer Manufacturers Association
1234 Here are examples of graphic character sets [NAME(<F>)]:
1235 o DIMENSION1_CHARS94 -- ASCII('B'), right-half-of-JISX0201('I'), ...
1236 o DIMENSION1_CHARS96 -- right-half-of-ISO8859-1('A'), ...
1237 o DIMENSION2_CHARS94 -- GB2312('A'), JISX0208('B'), ...
1238 o DIMENSION2_CHARS96 -- none for the moment
1240 A code area (1 byte=8 bits) is divided into 4 areas, C0, GL, C1, and GR.
1241 C0 [0x00..0x1F] -- control character plane 0
1242 GL [0x20..0x7F] -- graphic character plane 0
1243 C1 [0x80..0x9F] -- control character plane 1
1244 GR [0xA0..0xFF] -- graphic character plane 1
1246 A control character set is directly designated and invoked to C0 or
1247 C1 by an escape sequence. The most common case is that:
1248 - ISO646's control character set is designated/invoked to C0, and
1249 - ISO6429's control character set is designated/invoked to C1,
1250 and usually these designations/invocations are omitted in encoded
1251 text. In a 7-bit environment, only C0 can be used, and a control
1252 character for C1 is encoded by an appropriate escape sequence to
1253 fit into the environment. All control characters for C1 are
1254 defined to have corresponding escape sequences.
1256 A graphic character set is at first designated to one of four
1257 graphic registers (G0 through G3), then these graphic registers are
1258 invoked to GL or GR. These designations and invocations can be
1259 done independently. The most common case is that G0 is invoked to
1260 GL, G1 is invoked to GR, and ASCII is designated to G0. Usually
1261 these invocations and designations are omitted in encoded text.
1262 In a 7-bit environment, only GL can be used.
1264 When a graphic character set of CHARS94 is invoked to GL, codes
1265 0x20 and 0x7F of the GL area work as control characters SPACE and
1266 DEL respectively, and codes 0xA0 and 0xFF of the GR area should not
1269 There are two ways of invocation: locking-shift and single-shift.
1270 With locking-shift, the invocation lasts until the next different
1271 invocation, whereas with single-shift, the invocation affects the
1272 following character only and doesn't affect the locking-shift
1273 state. Invocations are done by the following control characters or
1276 ----------------------------------------------------------------------
1277 abbrev function cntrl escape seq description
1278 ----------------------------------------------------------------------
1279 SI/LS0 (shift-in) 0x0F none invoke G0 into GL
1280 SO/LS1 (shift-out) 0x0E none invoke G1 into GL
1281 LS2 (locking-shift-2) none ESC 'n' invoke G2 into GL
1282 LS3 (locking-shift-3) none ESC 'o' invoke G3 into GL
1283 LS1R (locking-shift-1 right) none ESC '~' invoke G1 into GR (*)
1284 LS2R (locking-shift-2 right) none ESC '}' invoke G2 into GR (*)
1285 LS3R (locking-shift 3 right) none ESC '|' invoke G3 into GR (*)
1286 SS2 (single-shift-2) 0x8E ESC 'N' invoke G2 for one char
1287 SS3 (single-shift-3) 0x8F ESC 'O' invoke G3 for one char
1288 ----------------------------------------------------------------------
1289 (*) These are not used by any known coding system.
1291 Control characters for these functions are defined by macros
1292 ISO_CODE_XXX in `coding.h'.
1294 Designations are done by the following escape sequences:
1295 ----------------------------------------------------------------------
1296 escape sequence description
1297 ----------------------------------------------------------------------
1298 ESC '(' <F> designate DIMENSION1_CHARS94<F> to G0
1299 ESC ')' <F> designate DIMENSION1_CHARS94<F> to G1
1300 ESC '*' <F> designate DIMENSION1_CHARS94<F> to G2
1301 ESC '+' <F> designate DIMENSION1_CHARS94<F> to G3
1302 ESC ',' <F> designate DIMENSION1_CHARS96<F> to G0 (*)
1303 ESC '-' <F> designate DIMENSION1_CHARS96<F> to G1
1304 ESC '.' <F> designate DIMENSION1_CHARS96<F> to G2
1305 ESC '/' <F> designate DIMENSION1_CHARS96<F> to G3
1306 ESC '$' '(' <F> designate DIMENSION2_CHARS94<F> to G0 (**)
1307 ESC '$' ')' <F> designate DIMENSION2_CHARS94<F> to G1
1308 ESC '$' '*' <F> designate DIMENSION2_CHARS94<F> to G2
1309 ESC '$' '+' <F> designate DIMENSION2_CHARS94<F> to G3
1310 ESC '$' ',' <F> designate DIMENSION2_CHARS96<F> to G0 (*)
1311 ESC '$' '-' <F> designate DIMENSION2_CHARS96<F> to G1
1312 ESC '$' '.' <F> designate DIMENSION2_CHARS96<F> to G2
1313 ESC '$' '/' <F> designate DIMENSION2_CHARS96<F> to G3
1314 ----------------------------------------------------------------------
1316 In this list, "DIMENSION1_CHARS94<F>" means a graphic character set
1317 of dimension 1, chars 94, and final character <F>, etc...
1319 Note (*): Although these designations are not allowed in ISO2022,
1320 Emacs accepts them on decoding, and produces them on encoding
1321 CHARS96 character sets in a coding system which is characterized as
1322 7-bit environment, non-locking-shift, and non-single-shift.
1324 Note (**): If <F> is '@', 'A', or 'B', the intermediate character
1325 '(' can be omitted. We refer to this as "short-form" hereafter.
1327 Now you may notice that there are a lot of ways of encoding the
1328 same multilingual text in ISO2022. Actually, there exist many
1329 coding systems such as Compound Text (used in X11's inter client
1330 communication, ISO-2022-JP (used in Japanese Internet), ISO-2022-KR
1331 (used in Korean Internet), EUC (Extended UNIX Code, used in Asian
1332 localized platforms), and all of these are variants of ISO2022.
1334 In addition to the above, Emacs handles two more kinds of escape
1335 sequences: ISO6429's direction specification and Emacs' private
1336 sequence for specifying character composition.
1338 ISO6429's direction specification takes the following form:
1339 o CSI ']' -- end of the current direction
1340 o CSI '0' ']' -- end of the current direction
1341 o CSI '1' ']' -- start of left-to-right text
1342 o CSI '2' ']' -- start of right-to-left text
1343 The control character CSI (0x9B: control sequence introducer) is
1344 abbreviated to the escape sequence ESC '[' in a 7-bit environment.
1346 Character composition specification takes the following form:
1347 o ESC '0' -- start relative composition
1348 o ESC '1' -- end composition
1349 o ESC '2' -- start rule-base composition (*)
1350 o ESC '3' -- start relative composition with alternate chars (**)
1351 o ESC '4' -- start rule-base composition with alternate chars (**)
1352 Since these are not standard escape sequences of any ISO standard,
1353 the use of them with these meanings is restricted to Emacs only.
1355 (*) This form is used only in Emacs 20.5 and older versions,
1356 but the newer versions can safely decode it.
1357 (**) This form is used only in Emacs 21.1 and newer versions,
1358 and the older versions can't decode it.
1360 Here's a list of example usages of these composition escape
1361 sequences (categorized by `enum composition_method').
1363 COMPOSITION_RELATIVE:
1364 ESC 0 CHAR [ CHAR ] ESC 1
1365 COMPOSITION_WITH_RULE:
1366 ESC 2 CHAR [ RULE CHAR ] ESC 1
1367 COMPOSITION_WITH_ALTCHARS:
1368 ESC 3 ALTCHAR [ ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1
1369 COMPOSITION_WITH_RULE_ALTCHARS:
1370 ESC 4 ALTCHAR [ RULE ALTCHAR ] ESC 0 CHAR [ CHAR ] ESC 1 */
1372 enum iso_code_class_type iso_code_class
[256];
1374 #define CHARSET_OK(idx, charset, c) \
1375 (coding_system_table[idx] \
1376 && (charset == CHARSET_ASCII \
1377 || (safe_chars = coding_safe_chars (coding_system_table[idx]->symbol), \
1378 CODING_SAFE_CHAR_P (safe_chars, c))) \
1379 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding_system_table[idx], \
1381 != CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION))
1383 #define SHIFT_OUT_OK(idx) \
1384 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding_system_table[idx], 1) >= 0)
1386 #define COMPOSITION_OK(idx) \
1387 (coding_system_table[idx]->composing != COMPOSITION_DISABLED)
1389 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
1390 Check if a text is encoded in ISO2022. If it is, return an
1391 integer in which appropriate flag bits any of:
1392 CODING_CATEGORY_MASK_ISO_7
1393 CODING_CATEGORY_MASK_ISO_7_TIGHT
1394 CODING_CATEGORY_MASK_ISO_8_1
1395 CODING_CATEGORY_MASK_ISO_8_2
1396 CODING_CATEGORY_MASK_ISO_7_ELSE
1397 CODING_CATEGORY_MASK_ISO_8_ELSE
1398 are set. If a code which should never appear in ISO2022 is found,
1402 detect_coding_iso2022 (src
, src_end
, multibytep
)
1403 unsigned char *src
, *src_end
;
1406 int mask
= CODING_CATEGORY_MASK_ISO
;
1408 int reg
[4], shift_out
= 0, single_shifting
= 0;
1410 /* Dummy for ONE_MORE_BYTE. */
1411 struct coding_system dummy_coding
;
1412 struct coding_system
*coding
= &dummy_coding
;
1413 Lisp_Object safe_chars
;
1415 reg
[0] = CHARSET_ASCII
, reg
[1] = reg
[2] = reg
[3] = -1;
1416 while (mask
&& src
< src_end
)
1418 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1423 if (inhibit_iso_escape_detection
)
1425 single_shifting
= 0;
1426 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1427 if (c
>= '(' && c
<= '/')
1429 /* Designation sequence for a charset of dimension 1. */
1430 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1431 if (c1
< ' ' || c1
>= 0x80
1432 || (charset
= iso_charset_table
[0][c
>= ','][c1
]) < 0)
1433 /* Invalid designation sequence. Just ignore. */
1435 reg
[(c
- '(') % 4] = charset
;
1439 /* Designation sequence for a charset of dimension 2. */
1440 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1441 if (c
>= '@' && c
<= 'B')
1442 /* Designation for JISX0208.1978, GB2312, or JISX0208. */
1443 reg
[0] = charset
= iso_charset_table
[1][0][c
];
1444 else if (c
>= '(' && c
<= '/')
1446 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
1447 if (c1
< ' ' || c1
>= 0x80
1448 || (charset
= iso_charset_table
[1][c
>= ','][c1
]) < 0)
1449 /* Invalid designation sequence. Just ignore. */
1451 reg
[(c
- '(') % 4] = charset
;
1454 /* Invalid designation sequence. Just ignore. */
1457 else if (c
== 'N' || c
== 'O')
1459 /* ESC <Fe> for SS2 or SS3. */
1460 mask
&= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1463 else if (c
>= '0' && c
<= '4')
1465 /* ESC <Fp> for start/end composition. */
1466 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7
))
1467 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1469 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1470 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
))
1471 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1473 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1474 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_1
))
1475 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1477 mask
&= ~CODING_CATEGORY_MASK_ISO_8_1
;
1478 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_2
))
1479 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1481 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1482 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
))
1483 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1485 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1486 if (COMPOSITION_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
))
1487 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1489 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1493 /* Invalid escape sequence. Just ignore. */
1496 /* We found a valid designation sequence for CHARSET. */
1497 mask
&= ~CODING_CATEGORY_MASK_ISO_8BIT
;
1498 c
= MAKE_CHAR (charset
, 0, 0);
1499 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7
, charset
, c
))
1500 mask_found
|= CODING_CATEGORY_MASK_ISO_7
;
1502 mask
&= ~CODING_CATEGORY_MASK_ISO_7
;
1503 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_TIGHT
, charset
, c
))
1504 mask_found
|= CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1506 mask
&= ~CODING_CATEGORY_MASK_ISO_7_TIGHT
;
1507 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
, charset
, c
))
1508 mask_found
|= CODING_CATEGORY_MASK_ISO_7_ELSE
;
1510 mask
&= ~CODING_CATEGORY_MASK_ISO_7_ELSE
;
1511 if (CHARSET_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
, charset
, c
))
1512 mask_found
|= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1514 mask
&= ~CODING_CATEGORY_MASK_ISO_8_ELSE
;
1518 if (inhibit_iso_escape_detection
)
1520 single_shifting
= 0;
1523 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_7_ELSE
)
1524 || SHIFT_OUT_OK (CODING_CATEGORY_IDX_ISO_8_ELSE
)))
1526 /* Locking shift out. */
1527 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1528 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1533 if (inhibit_iso_escape_detection
)
1535 single_shifting
= 0;
1538 /* Locking shift in. */
1539 mask
&= ~CODING_CATEGORY_MASK_ISO_7BIT
;
1540 mask_found
|= CODING_CATEGORY_MASK_ISO_SHIFT
;
1545 single_shifting
= 0;
1549 int newmask
= CODING_CATEGORY_MASK_ISO_8_ELSE
;
1551 if (inhibit_iso_escape_detection
)
1553 if (c
!= ISO_CODE_CSI
)
1555 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1556 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1557 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1558 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1559 & CODING_FLAG_ISO_SINGLE_SHIFT
)
1560 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1561 single_shifting
= 1;
1563 if (VECTORP (Vlatin_extra_code_table
)
1564 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1566 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1567 & CODING_FLAG_ISO_LATIN_EXTRA
)
1568 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1569 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1570 & CODING_FLAG_ISO_LATIN_EXTRA
)
1571 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1574 mask_found
|= newmask
;
1581 single_shifting
= 0;
1586 single_shifting
= 0;
1587 if (VECTORP (Vlatin_extra_code_table
)
1588 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
1592 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_1
]->flags
1593 & CODING_FLAG_ISO_LATIN_EXTRA
)
1594 newmask
|= CODING_CATEGORY_MASK_ISO_8_1
;
1595 if (coding_system_table
[CODING_CATEGORY_IDX_ISO_8_2
]->flags
1596 & CODING_FLAG_ISO_LATIN_EXTRA
)
1597 newmask
|= CODING_CATEGORY_MASK_ISO_8_2
;
1599 mask_found
|= newmask
;
1606 mask
&= ~(CODING_CATEGORY_MASK_ISO_7BIT
1607 | CODING_CATEGORY_MASK_ISO_7_ELSE
);
1608 mask_found
|= CODING_CATEGORY_MASK_ISO_8_1
;
1609 /* Check the length of succeeding codes of the range
1610 0xA0..0FF. If the byte length is odd, we exclude
1611 CODING_CATEGORY_MASK_ISO_8_2. We can check this only
1612 when we are not single shifting. */
1613 if (!single_shifting
1614 && mask
& CODING_CATEGORY_MASK_ISO_8_2
)
1619 while (src
< src_end
)
1621 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
1627 if (i
& 1 && src
< src_end
)
1628 mask
&= ~CODING_CATEGORY_MASK_ISO_8_2
;
1630 mask_found
|= CODING_CATEGORY_MASK_ISO_8_2
;
1632 /* This means that we have read one extra byte. */
1640 return (mask
& mask_found
);
1643 /* Decode a character of which charset is CHARSET, the 1st position
1644 code is C1, the 2nd position code is C2, and return the decoded
1645 character code. If the variable `translation_table' is non-nil,
1646 returned the translated code. */
1648 #define DECODE_ISO_CHARACTER(charset, c1, c2) \
1649 (NILP (translation_table) \
1650 ? MAKE_CHAR (charset, c1, c2) \
1651 : translate_char (translation_table, -1, charset, c1, c2))
1653 /* Set designation state into CODING. */
1654 #define DECODE_DESIGNATION(reg, dimension, chars, final_char) \
1658 if (final_char < '0' || final_char >= 128) \
1659 goto label_invalid_code; \
1660 charset = ISO_CHARSET_TABLE (make_number (dimension), \
1661 make_number (chars), \
1662 make_number (final_char)); \
1663 c = MAKE_CHAR (charset, 0, 0); \
1665 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding, charset) == reg \
1666 || CODING_SAFE_CHAR_P (safe_chars, c))) \
1668 if (coding->spec.iso2022.last_invalid_designation_register == 0 \
1670 && charset == CHARSET_ASCII) \
1672 /* We should insert this designation sequence as is so \
1673 that it is surely written back to a file. */ \
1674 coding->spec.iso2022.last_invalid_designation_register = -1; \
1675 goto label_invalid_code; \
1677 coding->spec.iso2022.last_invalid_designation_register = -1; \
1678 if ((coding->mode & CODING_MODE_DIRECTION) \
1679 && CHARSET_REVERSE_CHARSET (charset) >= 0) \
1680 charset = CHARSET_REVERSE_CHARSET (charset); \
1681 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
1685 coding->spec.iso2022.last_invalid_designation_register = reg; \
1686 goto label_invalid_code; \
1690 /* Allocate a memory block for storing information about compositions.
1691 The block is chained to the already allocated blocks. */
1694 coding_allocate_composition_data (coding
, char_offset
)
1695 struct coding_system
*coding
;
1698 struct composition_data
*cmp_data
1699 = (struct composition_data
*) xmalloc (sizeof *cmp_data
);
1701 cmp_data
->char_offset
= char_offset
;
1703 cmp_data
->prev
= coding
->cmp_data
;
1704 cmp_data
->next
= NULL
;
1705 if (coding
->cmp_data
)
1706 coding
->cmp_data
->next
= cmp_data
;
1707 coding
->cmp_data
= cmp_data
;
1708 coding
->cmp_data_start
= 0;
1709 coding
->composing
= COMPOSITION_NO
;
1712 /* Handle composition start sequence ESC 0, ESC 2, ESC 3, or ESC 4.
1713 ESC 0 : relative composition : ESC 0 CHAR ... ESC 1
1714 ESC 2 : rulebase composition : ESC 2 CHAR RULE CHAR RULE ... CHAR ESC 1
1715 ESC 3 : altchar composition : ESC 3 ALT ... ESC 0 CHAR ... ESC 1
1716 ESC 4 : alt&rule composition : ESC 4 ALT RULE .. ALT ESC 0 CHAR ... ESC 1
1719 #define DECODE_COMPOSITION_START(c1) \
1721 if (coding->composing == COMPOSITION_DISABLED) \
1723 *dst++ = ISO_CODE_ESC; \
1724 *dst++ = c1 & 0x7f; \
1725 coding->produced_char += 2; \
1727 else if (!COMPOSING_P (coding)) \
1729 /* This is surely the start of a composition. We must be sure \
1730 that coding->cmp_data has enough space to store the \
1731 information about the composition. If not, terminate the \
1732 current decoding loop, allocate one more memory block for \
1733 coding->cmp_data in the caller, then start the decoding \
1734 loop again. We can't allocate memory here directly because \
1735 it may cause buffer/string relocation. */ \
1736 if (!coding->cmp_data \
1737 || (coding->cmp_data->used + COMPOSITION_DATA_MAX_BUNCH_LENGTH \
1738 >= COMPOSITION_DATA_SIZE)) \
1740 coding->result = CODING_FINISH_INSUFFICIENT_CMP; \
1741 goto label_end_of_loop; \
1743 coding->composing = (c1 == '0' ? COMPOSITION_RELATIVE \
1744 : c1 == '2' ? COMPOSITION_WITH_RULE \
1745 : c1 == '3' ? COMPOSITION_WITH_ALTCHARS \
1746 : COMPOSITION_WITH_RULE_ALTCHARS); \
1747 CODING_ADD_COMPOSITION_START (coding, coding->produced_char, \
1748 coding->composing); \
1749 coding->composition_rule_follows = 0; \
1753 /* We are already handling a composition. If the method is \
1754 the following two, the codes following the current escape \
1755 sequence are actual characters stored in a buffer. */ \
1756 if (coding->composing == COMPOSITION_WITH_ALTCHARS \
1757 || coding->composing == COMPOSITION_WITH_RULE_ALTCHARS) \
1759 coding->composing = COMPOSITION_RELATIVE; \
1760 coding->composition_rule_follows = 0; \
1765 /* Handle composition end sequence ESC 1. */
1767 #define DECODE_COMPOSITION_END(c1) \
1769 if (! COMPOSING_P (coding)) \
1771 *dst++ = ISO_CODE_ESC; \
1773 coding->produced_char += 2; \
1777 CODING_ADD_COMPOSITION_END (coding, coding->produced_char); \
1778 coding->composing = COMPOSITION_NO; \
1782 /* Decode a composition rule from the byte C1 (and maybe one more byte
1783 from SRC) and store one encoded composition rule in
1784 coding->cmp_data. */
1786 #define DECODE_COMPOSITION_RULE(c1) \
1790 if (c1 < 81) /* old format (before ver.21) */ \
1792 int gref = (c1) / 9; \
1793 int nref = (c1) % 9; \
1794 if (gref == 4) gref = 10; \
1795 if (nref == 4) nref = 10; \
1796 rule = COMPOSITION_ENCODE_RULE (gref, nref); \
1798 else if (c1 < 93) /* new format (after ver.21) */ \
1800 ONE_MORE_BYTE (c2); \
1801 rule = COMPOSITION_ENCODE_RULE (c1 - 81, c2 - 32); \
1803 CODING_ADD_COMPOSITION_COMPONENT (coding, rule); \
1804 coding->composition_rule_follows = 0; \
1808 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
1811 decode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
1812 struct coding_system
*coding
;
1813 unsigned char *source
, *destination
;
1814 int src_bytes
, dst_bytes
;
1816 unsigned char *src
= source
;
1817 unsigned char *src_end
= source
+ src_bytes
;
1818 unsigned char *dst
= destination
;
1819 unsigned char *dst_end
= destination
+ dst_bytes
;
1820 /* Charsets invoked to graphic plane 0 and 1 respectively. */
1821 int charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1822 int charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
1823 /* SRC_BASE remembers the start position in source in each loop.
1824 The loop will be exited when there's not enough source code
1825 (within macro ONE_MORE_BYTE), or when there's not enough
1826 destination area to produce a character (within macro
1828 unsigned char *src_base
;
1830 Lisp_Object translation_table
;
1831 Lisp_Object safe_chars
;
1833 safe_chars
= coding_safe_chars (coding
->symbol
);
1835 if (NILP (Venable_character_translation
))
1836 translation_table
= Qnil
;
1839 translation_table
= coding
->translation_table_for_decode
;
1840 if (NILP (translation_table
))
1841 translation_table
= Vstandard_translation_table_for_decode
;
1844 coding
->result
= CODING_FINISH_NORMAL
;
1853 /* We produce no character or one character. */
1854 switch (iso_code_class
[c1
])
1856 case ISO_0x20_or_0x7F
:
1857 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1859 DECODE_COMPOSITION_RULE (c1
);
1862 if (charset0
< 0 || CHARSET_CHARS (charset0
) == 94)
1864 /* This is SPACE or DEL. */
1865 charset
= CHARSET_ASCII
;
1868 /* This is a graphic character, we fall down ... */
1870 case ISO_graphic_plane_0
:
1871 if (COMPOSING_P (coding
) && coding
->composition_rule_follows
)
1873 DECODE_COMPOSITION_RULE (c1
);
1879 case ISO_0xA0_or_0xFF
:
1880 if (charset1
< 0 || CHARSET_CHARS (charset1
) == 94
1881 || coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
1882 goto label_invalid_code
;
1883 /* This is a graphic character, we fall down ... */
1885 case ISO_graphic_plane_1
:
1887 goto label_invalid_code
;
1892 if (COMPOSING_P (coding
))
1893 DECODE_COMPOSITION_END ('1');
1895 /* All ISO2022 control characters in this class have the
1896 same representation in Emacs internal format. */
1898 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
1899 && (coding
->eol_type
== CODING_EOL_CR
1900 || coding
->eol_type
== CODING_EOL_CRLF
))
1902 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
1903 goto label_end_of_loop
;
1905 charset
= CHARSET_ASCII
;
1909 if (COMPOSING_P (coding
))
1910 DECODE_COMPOSITION_END ('1');
1911 goto label_invalid_code
;
1913 case ISO_carriage_return
:
1914 if (COMPOSING_P (coding
))
1915 DECODE_COMPOSITION_END ('1');
1917 if (coding
->eol_type
== CODING_EOL_CR
)
1919 else if (coding
->eol_type
== CODING_EOL_CRLF
)
1922 if (c1
!= ISO_CODE_LF
)
1928 charset
= CHARSET_ASCII
;
1932 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
1933 || CODING_SPEC_ISO_DESIGNATION (coding
, 1) < 0)
1934 goto label_invalid_code
;
1935 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 1;
1936 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1940 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
1941 goto label_invalid_code
;
1942 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
1943 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
1946 case ISO_single_shift_2_7
:
1947 case ISO_single_shift_2
:
1948 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1949 goto label_invalid_code
;
1950 /* SS2 is handled as an escape sequence of ESC 'N' */
1952 goto label_escape_sequence
;
1954 case ISO_single_shift_3
:
1955 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
1956 goto label_invalid_code
;
1957 /* SS2 is handled as an escape sequence of ESC 'O' */
1959 goto label_escape_sequence
;
1961 case ISO_control_sequence_introducer
:
1962 /* CSI is handled as an escape sequence of ESC '[' ... */
1964 goto label_escape_sequence
;
1968 label_escape_sequence
:
1969 /* Escape sequences handled by Emacs are invocation,
1970 designation, direction specification, and character
1971 composition specification. */
1974 case '&': /* revision of following character set */
1976 if (!(c1
>= '@' && c1
<= '~'))
1977 goto label_invalid_code
;
1979 if (c1
!= ISO_CODE_ESC
)
1980 goto label_invalid_code
;
1982 goto label_escape_sequence
;
1984 case '$': /* designation of 2-byte character set */
1985 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
1986 goto label_invalid_code
;
1988 if (c1
>= '@' && c1
<= 'B')
1989 { /* designation of JISX0208.1978, GB2312.1980,
1991 DECODE_DESIGNATION (0, 2, 94, c1
);
1993 else if (c1
>= 0x28 && c1
<= 0x2B)
1994 { /* designation of DIMENSION2_CHARS94 character set */
1996 DECODE_DESIGNATION (c1
- 0x28, 2, 94, c2
);
1998 else if (c1
>= 0x2C && c1
<= 0x2F)
1999 { /* designation of DIMENSION2_CHARS96 character set */
2001 DECODE_DESIGNATION (c1
- 0x2C, 2, 96, c2
);
2004 goto label_invalid_code
;
2005 /* We must update these variables now. */
2006 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2007 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2010 case 'n': /* invocation of locking-shift-2 */
2011 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2012 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2013 goto label_invalid_code
;
2014 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 2;
2015 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2018 case 'o': /* invocation of locking-shift-3 */
2019 if (! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
)
2020 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2021 goto label_invalid_code
;
2022 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 3;
2023 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2026 case 'N': /* invocation of single-shift-2 */
2027 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2028 || CODING_SPEC_ISO_DESIGNATION (coding
, 2) < 0)
2029 goto label_invalid_code
;
2030 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 2);
2032 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2033 goto label_invalid_code
;
2036 case 'O': /* invocation of single-shift-3 */
2037 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2038 || CODING_SPEC_ISO_DESIGNATION (coding
, 3) < 0)
2039 goto label_invalid_code
;
2040 charset
= CODING_SPEC_ISO_DESIGNATION (coding
, 3);
2042 if (c1
< 0x20 || (c1
>= 0x80 && c1
< 0xA0))
2043 goto label_invalid_code
;
2046 case '0': case '2': case '3': case '4': /* start composition */
2047 DECODE_COMPOSITION_START (c1
);
2050 case '1': /* end composition */
2051 DECODE_COMPOSITION_END (c1
);
2054 case '[': /* specification of direction */
2055 if (coding
->flags
& CODING_FLAG_ISO_NO_DIRECTION
)
2056 goto label_invalid_code
;
2057 /* For the moment, nested direction is not supported.
2058 So, `coding->mode & CODING_MODE_DIRECTION' zero means
2059 left-to-right, and nonzero means right-to-left. */
2063 case ']': /* end of the current direction */
2064 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2066 case '0': /* end of the current direction */
2067 case '1': /* start of left-to-right direction */
2070 coding
->mode
&= ~CODING_MODE_DIRECTION
;
2072 goto label_invalid_code
;
2075 case '2': /* start of right-to-left direction */
2078 coding
->mode
|= CODING_MODE_DIRECTION
;
2080 goto label_invalid_code
;
2084 goto label_invalid_code
;
2089 if (COMPOSING_P (coding
))
2090 DECODE_COMPOSITION_END ('1');
2094 /* CTEXT extended segment:
2095 ESC % / [0-4] M L --ENCODING-NAME-- \002 --BYTES--
2096 We keep these bytes as is for the moment.
2097 They may be decoded by post-read-conversion. */
2102 ONE_MORE_BYTE (dim
);
2105 size
= ((M
- 128) * 128) + (L
- 128);
2106 required
= 8 + size
* 2;
2107 if (dst
+ required
> (dst_bytes
? dst_end
: src
))
2108 goto label_end_of_loop
;
2109 *dst
++ = ISO_CODE_ESC
;
2114 dst
+= CHAR_STRING (M
, dst
), produced_chars
++;
2115 dst
+= CHAR_STRING (L
, dst
), produced_chars
++;
2119 dst
+= CHAR_STRING (c1
, dst
), produced_chars
++;
2121 coding
->produced_char
+= produced_chars
;
2125 unsigned char *d
= dst
;
2128 /* XFree86 extension for embedding UTF-8 in CTEXT:
2129 ESC % G --UTF-8-BYTES-- ESC % @
2130 We keep these bytes as is for the moment.
2131 They may be decoded by post-read-conversion. */
2132 if (d
+ 6 > (dst_bytes
? dst_end
: src
))
2133 goto label_end_of_loop
;
2134 *d
++ = ISO_CODE_ESC
;
2138 while (d
+ 1 < (dst_bytes
? dst_end
: src
))
2141 if (c1
== ISO_CODE_ESC
2142 && src
+ 1 < src_end
2149 d
+= CHAR_STRING (c1
, d
), produced_chars
++;
2151 if (d
+ 3 > (dst_bytes
? dst_end
: src
))
2152 goto label_end_of_loop
;
2153 *d
++ = ISO_CODE_ESC
;
2157 coding
->produced_char
+= produced_chars
+ 3;
2160 goto label_invalid_code
;
2164 if (! (coding
->flags
& CODING_FLAG_ISO_DESIGNATION
))
2165 goto label_invalid_code
;
2166 if (c1
>= 0x28 && c1
<= 0x2B)
2167 { /* designation of DIMENSION1_CHARS94 character set */
2169 DECODE_DESIGNATION (c1
- 0x28, 1, 94, c2
);
2171 else if (c1
>= 0x2C && c1
<= 0x2F)
2172 { /* designation of DIMENSION1_CHARS96 character set */
2174 DECODE_DESIGNATION (c1
- 0x2C, 1, 96, c2
);
2177 goto label_invalid_code
;
2178 /* We must update these variables now. */
2179 charset0
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 0);
2180 charset1
= CODING_SPEC_ISO_PLANE_CHARSET (coding
, 1);
2185 /* Now we know CHARSET and 1st position code C1 of a character.
2186 Produce a multibyte sequence for that character while getting
2187 2nd position code C2 if necessary. */
2188 if (CHARSET_DIMENSION (charset
) == 2)
2191 if (c1
< 0x80 ? c2
< 0x20 || c2
>= 0x80 : c2
< 0xA0)
2192 /* C2 is not in a valid range. */
2193 goto label_invalid_code
;
2195 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
2201 if (COMPOSING_P (coding
))
2202 DECODE_COMPOSITION_END ('1');
2209 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
2210 coding
->produced
= dst
- destination
;
2215 /* ISO2022 encoding stuff. */
2218 It is not enough to say just "ISO2022" on encoding, we have to
2219 specify more details. In Emacs, each ISO2022 coding system
2220 variant has the following specifications:
2221 1. Initial designation to G0 through G3.
2222 2. Allows short-form designation?
2223 3. ASCII should be designated to G0 before control characters?
2224 4. ASCII should be designated to G0 at end of line?
2225 5. 7-bit environment or 8-bit environment?
2226 6. Use locking-shift?
2227 7. Use Single-shift?
2228 And the following two are only for Japanese:
2229 8. Use ASCII in place of JIS0201-1976-Roman?
2230 9. Use JISX0208-1983 in place of JISX0208-1978?
2231 These specifications are encoded in `coding->flags' as flag bits
2232 defined by macros CODING_FLAG_ISO_XXX. See `coding.h' for more
2236 /* Produce codes (escape sequence) for designating CHARSET to graphic
2237 register REG at DST, and increment DST. If <final-char> of CHARSET is
2238 '@', 'A', or 'B' and the coding system CODING allows, produce
2239 designation sequence of short-form. */
2241 #define ENCODE_DESIGNATION(charset, reg, coding) \
2243 unsigned char final_char = CHARSET_ISO_FINAL_CHAR (charset); \
2244 char *intermediate_char_94 = "()*+"; \
2245 char *intermediate_char_96 = ",-./"; \
2246 int revision = CODING_SPEC_ISO_REVISION_NUMBER(coding, charset); \
2248 if (revision < 255) \
2250 *dst++ = ISO_CODE_ESC; \
2252 *dst++ = '@' + revision; \
2254 *dst++ = ISO_CODE_ESC; \
2255 if (CHARSET_DIMENSION (charset) == 1) \
2257 if (CHARSET_CHARS (charset) == 94) \
2258 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2260 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2265 if (CHARSET_CHARS (charset) == 94) \
2267 if (! (coding->flags & CODING_FLAG_ISO_SHORT_FORM) \
2269 || final_char < '@' || final_char > 'B') \
2270 *dst++ = (unsigned char) (intermediate_char_94[reg]); \
2273 *dst++ = (unsigned char) (intermediate_char_96[reg]); \
2275 *dst++ = final_char; \
2276 CODING_SPEC_ISO_DESIGNATION (coding, reg) = charset; \
2279 /* The following two macros produce codes (control character or escape
2280 sequence) for ISO2022 single-shift functions (single-shift-2 and
2283 #define ENCODE_SINGLE_SHIFT_2 \
2285 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2286 *dst++ = ISO_CODE_ESC, *dst++ = 'N'; \
2288 *dst++ = ISO_CODE_SS2; \
2289 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2292 #define ENCODE_SINGLE_SHIFT_3 \
2294 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2295 *dst++ = ISO_CODE_ESC, *dst++ = 'O'; \
2297 *dst++ = ISO_CODE_SS3; \
2298 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 1; \
2301 /* The following four macros produce codes (control character or
2302 escape sequence) for ISO2022 locking-shift functions (shift-in,
2303 shift-out, locking-shift-2, and locking-shift-3). */
2305 #define ENCODE_SHIFT_IN \
2307 *dst++ = ISO_CODE_SI; \
2308 CODING_SPEC_ISO_INVOCATION (coding, 0) = 0; \
2311 #define ENCODE_SHIFT_OUT \
2313 *dst++ = ISO_CODE_SO; \
2314 CODING_SPEC_ISO_INVOCATION (coding, 0) = 1; \
2317 #define ENCODE_LOCKING_SHIFT_2 \
2319 *dst++ = ISO_CODE_ESC, *dst++ = 'n'; \
2320 CODING_SPEC_ISO_INVOCATION (coding, 0) = 2; \
2323 #define ENCODE_LOCKING_SHIFT_3 \
2325 *dst++ = ISO_CODE_ESC, *dst++ = 'o'; \
2326 CODING_SPEC_ISO_INVOCATION (coding, 0) = 3; \
2329 /* Produce codes for a DIMENSION1 character whose character set is
2330 CHARSET and whose position-code is C1. Designation and invocation
2331 sequences are also produced in advance if necessary. */
2333 #define ENCODE_ISO_CHARACTER_DIMENSION1(charset, c1) \
2335 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2337 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2338 *dst++ = c1 & 0x7F; \
2340 *dst++ = c1 | 0x80; \
2341 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2344 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2346 *dst++ = c1 & 0x7F; \
2349 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2351 *dst++ = c1 | 0x80; \
2355 /* Since CHARSET is not yet invoked to any graphic planes, we \
2356 must invoke it, or, at first, designate it to some graphic \
2357 register. Then repeat the loop to actually produce the \
2359 dst = encode_invocation_designation (charset, coding, dst); \
2362 /* Produce codes for a DIMENSION2 character whose character set is
2363 CHARSET and whose position-codes are C1 and C2. Designation and
2364 invocation codes are also produced in advance if necessary. */
2366 #define ENCODE_ISO_CHARACTER_DIMENSION2(charset, c1, c2) \
2368 if (CODING_SPEC_ISO_SINGLE_SHIFTING (coding)) \
2370 if (coding->flags & CODING_FLAG_ISO_SEVEN_BITS) \
2371 *dst++ = c1 & 0x7F, *dst++ = c2 & 0x7F; \
2373 *dst++ = c1 | 0x80, *dst++ = c2 | 0x80; \
2374 CODING_SPEC_ISO_SINGLE_SHIFTING (coding) = 0; \
2377 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 0)) \
2379 *dst++ = c1 & 0x7F, *dst++= c2 & 0x7F; \
2382 else if (charset == CODING_SPEC_ISO_PLANE_CHARSET (coding, 1)) \
2384 *dst++ = c1 | 0x80, *dst++= c2 | 0x80; \
2388 /* Since CHARSET is not yet invoked to any graphic planes, we \
2389 must invoke it, or, at first, designate it to some graphic \
2390 register. Then repeat the loop to actually produce the \
2392 dst = encode_invocation_designation (charset, coding, dst); \
2395 #define ENCODE_ISO_CHARACTER(c) \
2397 int charset, c1, c2; \
2399 SPLIT_CHAR (c, charset, c1, c2); \
2400 if (CHARSET_DEFINED_P (charset)) \
2402 if (CHARSET_DIMENSION (charset) == 1) \
2404 if (charset == CHARSET_ASCII \
2405 && coding->flags & CODING_FLAG_ISO_USE_ROMAN) \
2406 charset = charset_latin_jisx0201; \
2407 ENCODE_ISO_CHARACTER_DIMENSION1 (charset, c1); \
2411 if (charset == charset_jisx0208 \
2412 && coding->flags & CODING_FLAG_ISO_USE_OLDJIS) \
2413 charset = charset_jisx0208_1978; \
2414 ENCODE_ISO_CHARACTER_DIMENSION2 (charset, c1, c2); \
2426 /* Instead of encoding character C, produce one or two `?'s. */
2428 #define ENCODE_UNSAFE_CHARACTER(c) \
2430 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2431 if (CHARSET_WIDTH (CHAR_CHARSET (c)) > 1) \
2432 ENCODE_ISO_CHARACTER (CODING_REPLACEMENT_CHARACTER); \
2436 /* Produce designation and invocation codes at a place pointed by DST
2437 to use CHARSET. The element `spec.iso2022' of *CODING is updated.
2441 encode_invocation_designation (charset
, coding
, dst
)
2443 struct coding_system
*coding
;
2446 int reg
; /* graphic register number */
2448 /* At first, check designations. */
2449 for (reg
= 0; reg
< 4; reg
++)
2450 if (charset
== CODING_SPEC_ISO_DESIGNATION (coding
, reg
))
2455 /* CHARSET is not yet designated to any graphic registers. */
2456 /* At first check the requested designation. */
2457 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2458 if (reg
== CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
)
2459 /* Since CHARSET requests no special designation, designate it
2460 to graphic register 0. */
2463 ENCODE_DESIGNATION (charset
, reg
, coding
);
2466 if (CODING_SPEC_ISO_INVOCATION (coding
, 0) != reg
2467 && CODING_SPEC_ISO_INVOCATION (coding
, 1) != reg
)
2469 /* Since the graphic register REG is not invoked to any graphic
2470 planes, invoke it to graphic plane 0. */
2473 case 0: /* graphic register 0 */
2477 case 1: /* graphic register 1 */
2481 case 2: /* graphic register 2 */
2482 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2483 ENCODE_SINGLE_SHIFT_2
;
2485 ENCODE_LOCKING_SHIFT_2
;
2488 case 3: /* graphic register 3 */
2489 if (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
)
2490 ENCODE_SINGLE_SHIFT_3
;
2492 ENCODE_LOCKING_SHIFT_3
;
2500 /* Produce 2-byte codes for encoded composition rule RULE. */
2502 #define ENCODE_COMPOSITION_RULE(rule) \
2505 COMPOSITION_DECODE_RULE (rule, gref, nref); \
2506 *dst++ = 32 + 81 + gref; \
2507 *dst++ = 32 + nref; \
2510 /* Produce codes for indicating the start of a composition sequence
2511 (ESC 0, ESC 3, or ESC 4). DATA points to an array of integers
2512 which specify information about the composition. See the comment
2513 in coding.h for the format of DATA. */
2515 #define ENCODE_COMPOSITION_START(coding, data) \
2517 coding->composing = data[3]; \
2518 *dst++ = ISO_CODE_ESC; \
2519 if (coding->composing == COMPOSITION_RELATIVE) \
2523 *dst++ = (coding->composing == COMPOSITION_WITH_ALTCHARS \
2525 coding->cmp_data_index = coding->cmp_data_start + 4; \
2526 coding->composition_rule_follows = 0; \
2530 /* Produce codes for indicating the end of the current composition. */
2532 #define ENCODE_COMPOSITION_END(coding, data) \
2534 *dst++ = ISO_CODE_ESC; \
2536 coding->cmp_data_start += data[0]; \
2537 coding->composing = COMPOSITION_NO; \
2538 if (coding->cmp_data_start == coding->cmp_data->used \
2539 && coding->cmp_data->next) \
2541 coding->cmp_data = coding->cmp_data->next; \
2542 coding->cmp_data_start = 0; \
2546 /* Produce composition start sequence ESC 0. Here, this sequence
2547 doesn't mean the start of a new composition but means that we have
2548 just produced components (alternate chars and composition rules) of
2549 the composition and the actual text follows in SRC. */
2551 #define ENCODE_COMPOSITION_FAKE_START(coding) \
2553 *dst++ = ISO_CODE_ESC; \
2555 coding->composing = COMPOSITION_RELATIVE; \
2558 /* The following three macros produce codes for indicating direction
2560 #define ENCODE_CONTROL_SEQUENCE_INTRODUCER \
2562 if (coding->flags == CODING_FLAG_ISO_SEVEN_BITS) \
2563 *dst++ = ISO_CODE_ESC, *dst++ = '['; \
2565 *dst++ = ISO_CODE_CSI; \
2568 #define ENCODE_DIRECTION_R2L \
2569 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '2', *dst++ = ']'
2571 #define ENCODE_DIRECTION_L2R \
2572 ENCODE_CONTROL_SEQUENCE_INTRODUCER (dst), *dst++ = '0', *dst++ = ']'
2574 /* Produce codes for designation and invocation to reset the graphic
2575 planes and registers to initial state. */
2576 #define ENCODE_RESET_PLANE_AND_REGISTER \
2579 if (CODING_SPEC_ISO_INVOCATION (coding, 0) != 0) \
2581 for (reg = 0; reg < 4; reg++) \
2582 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg) >= 0 \
2583 && (CODING_SPEC_ISO_DESIGNATION (coding, reg) \
2584 != CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg))) \
2585 ENCODE_DESIGNATION \
2586 (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding, reg), reg, coding); \
2589 /* Produce designation sequences of charsets in the line started from
2590 SRC to a place pointed by DST, and return updated DST.
2592 If the current block ends before any end-of-line, we may fail to
2593 find all the necessary designations. */
2595 static unsigned char *
2596 encode_designation_at_bol (coding
, translation_table
, src
, src_end
, dst
)
2597 struct coding_system
*coding
;
2598 Lisp_Object translation_table
;
2599 unsigned char *src
, *src_end
, *dst
;
2601 int charset
, c
, found
= 0, reg
;
2602 /* Table of charsets to be designated to each graphic register. */
2605 for (reg
= 0; reg
< 4; reg
++)
2614 charset
= CHAR_CHARSET (c
);
2615 reg
= CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
);
2616 if (reg
!= CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
&& r
[reg
] < 0)
2626 for (reg
= 0; reg
< 4; reg
++)
2628 && CODING_SPEC_ISO_DESIGNATION (coding
, reg
) != r
[reg
])
2629 ENCODE_DESIGNATION (r
[reg
], reg
, coding
);
2635 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions". */
2638 encode_coding_iso2022 (coding
, source
, destination
, src_bytes
, dst_bytes
)
2639 struct coding_system
*coding
;
2640 unsigned char *source
, *destination
;
2641 int src_bytes
, dst_bytes
;
2643 unsigned char *src
= source
;
2644 unsigned char *src_end
= source
+ src_bytes
;
2645 unsigned char *dst
= destination
;
2646 unsigned char *dst_end
= destination
+ dst_bytes
;
2647 /* Since the maximum bytes produced by each loop is 20, we subtract 19
2648 from DST_END to assure overflow checking is necessary only at the
2650 unsigned char *adjusted_dst_end
= dst_end
- 19;
2651 /* SRC_BASE remembers the start position in source in each loop.
2652 The loop will be exited when there's not enough source text to
2653 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
2654 there's not enough destination area to produce encoded codes
2655 (within macro EMIT_BYTES). */
2656 unsigned char *src_base
;
2658 Lisp_Object translation_table
;
2659 Lisp_Object safe_chars
;
2661 if (coding
->flags
& CODING_FLAG_ISO_SAFE
)
2662 coding
->mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
2664 safe_chars
= coding_safe_chars (coding
->symbol
);
2666 if (NILP (Venable_character_translation
))
2667 translation_table
= Qnil
;
2670 translation_table
= coding
->translation_table_for_encode
;
2671 if (NILP (translation_table
))
2672 translation_table
= Vstandard_translation_table_for_encode
;
2675 coding
->consumed_char
= 0;
2681 if (dst
>= (dst_bytes
? adjusted_dst_end
: (src
- 19)))
2683 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
2687 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
2688 && CODING_SPEC_ISO_BOL (coding
))
2690 /* We have to produce designation sequences if any now. */
2691 dst
= encode_designation_at_bol (coding
, translation_table
,
2693 CODING_SPEC_ISO_BOL (coding
) = 0;
2696 /* Check composition start and end. */
2697 if (coding
->composing
!= COMPOSITION_DISABLED
2698 && coding
->cmp_data_start
< coding
->cmp_data
->used
)
2700 struct composition_data
*cmp_data
= coding
->cmp_data
;
2701 int *data
= cmp_data
->data
+ coding
->cmp_data_start
;
2702 int this_pos
= cmp_data
->char_offset
+ coding
->consumed_char
;
2704 if (coding
->composing
== COMPOSITION_RELATIVE
)
2706 if (this_pos
== data
[2])
2708 ENCODE_COMPOSITION_END (coding
, data
);
2709 cmp_data
= coding
->cmp_data
;
2710 data
= cmp_data
->data
+ coding
->cmp_data_start
;
2713 else if (COMPOSING_P (coding
))
2715 /* COMPOSITION_WITH_ALTCHARS or COMPOSITION_WITH_RULE_ALTCHAR */
2716 if (coding
->cmp_data_index
== coding
->cmp_data_start
+ data
[0])
2717 /* We have consumed components of the composition.
2718 What follows in SRC is the composition's base
2720 ENCODE_COMPOSITION_FAKE_START (coding
);
2723 int c
= cmp_data
->data
[coding
->cmp_data_index
++];
2724 if (coding
->composition_rule_follows
)
2726 ENCODE_COMPOSITION_RULE (c
);
2727 coding
->composition_rule_follows
= 0;
2731 if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2732 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2733 ENCODE_UNSAFE_CHARACTER (c
);
2735 ENCODE_ISO_CHARACTER (c
);
2736 if (coding
->composing
== COMPOSITION_WITH_RULE_ALTCHARS
)
2737 coding
->composition_rule_follows
= 1;
2742 if (!COMPOSING_P (coding
))
2744 if (this_pos
== data
[1])
2746 ENCODE_COMPOSITION_START (coding
, data
);
2754 /* Now encode the character C. */
2755 if (c
< 0x20 || c
== 0x7F)
2759 if (! (coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
2761 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2762 ENCODE_RESET_PLANE_AND_REGISTER
;
2766 /* fall down to treat '\r' as '\n' ... */
2771 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_EOL
)
2772 ENCODE_RESET_PLANE_AND_REGISTER
;
2773 if (coding
->flags
& CODING_FLAG_ISO_INIT_AT_BOL
)
2774 bcopy (coding
->spec
.iso2022
.initial_designation
,
2775 coding
->spec
.iso2022
.current_designation
,
2776 sizeof coding
->spec
.iso2022
.initial_designation
);
2777 if (coding
->eol_type
== CODING_EOL_LF
2778 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
2779 *dst
++ = ISO_CODE_LF
;
2780 else if (coding
->eol_type
== CODING_EOL_CRLF
)
2781 *dst
++ = ISO_CODE_CR
, *dst
++ = ISO_CODE_LF
;
2783 *dst
++ = ISO_CODE_CR
;
2784 CODING_SPEC_ISO_BOL (coding
) = 1;
2788 if (coding
->flags
& CODING_FLAG_ISO_RESET_AT_CNTL
)
2789 ENCODE_RESET_PLANE_AND_REGISTER
;
2793 else if (ASCII_BYTE_P (c
))
2794 ENCODE_ISO_CHARACTER (c
);
2795 else if (SINGLE_BYTE_CHAR_P (c
))
2800 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
2801 && ! CODING_SAFE_CHAR_P (safe_chars
, c
))
2802 ENCODE_UNSAFE_CHARACTER (c
);
2804 ENCODE_ISO_CHARACTER (c
);
2806 coding
->consumed_char
++;
2810 coding
->consumed
= src_base
- source
;
2811 coding
->produced
= coding
->produced_char
= dst
- destination
;
2815 /*** 4. SJIS and BIG5 handlers ***/
2817 /* Although SJIS and BIG5 are not ISO coding systems, they are used
2818 quite widely. So, for the moment, Emacs supports them in the bare
2819 C code. But, in the future, they may be supported only by CCL. */
2821 /* SJIS is a coding system encoding three character sets: ASCII, right
2822 half of JISX0201-Kana, and JISX0208. An ASCII character is encoded
2823 as is. A character of charset katakana-jisx0201 is encoded by
2824 "position-code + 0x80". A character of charset japanese-jisx0208
2825 is encoded in 2-byte but two position-codes are divided and shifted
2826 so that it fits in the range below.
2828 --- CODE RANGE of SJIS ---
2829 (character set) (range)
2831 KATAKANA-JISX0201 0xA1 .. 0xDF
2832 JISX0208 (1st byte) 0x81 .. 0x9F and 0xE0 .. 0xEF
2833 (2nd byte) 0x40 .. 0x7E and 0x80 .. 0xFC
2834 -------------------------------
2838 /* BIG5 is a coding system encoding two character sets: ASCII and
2839 Big5. An ASCII character is encoded as is. Big5 is a two-byte
2840 character set and is encoded in two bytes.
2842 --- CODE RANGE of BIG5 ---
2843 (character set) (range)
2845 Big5 (1st byte) 0xA1 .. 0xFE
2846 (2nd byte) 0x40 .. 0x7E and 0xA1 .. 0xFE
2847 --------------------------
2849 Since the number of characters in Big5 is larger than maximum
2850 characters in Emacs' charset (96x96), it can't be handled as one
2851 charset. So, in Emacs, Big5 is divided into two: `charset-big5-1'
2852 and `charset-big5-2'. Both are DIMENSION2 and CHARS94. The former
2853 contains frequently used characters and the latter contains less
2854 frequently used characters. */
2856 /* Macros to decode or encode a character of Big5 in BIG5. B1 and B2
2857 are the 1st and 2nd position-codes of Big5 in BIG5 coding system.
2858 C1 and C2 are the 1st and 2nd position-codes of Emacs' internal
2859 format. CHARSET is `charset_big5_1' or `charset_big5_2'. */
2861 /* Number of Big5 characters which have the same code in 1st byte. */
2862 #define BIG5_SAME_ROW (0xFF - 0xA1 + 0x7F - 0x40)
2864 #define DECODE_BIG5(b1, b2, charset, c1, c2) \
2867 = (b1 - 0xA1) * BIG5_SAME_ROW + b2 - (b2 < 0x7F ? 0x40 : 0x62); \
2869 charset = charset_big5_1; \
2872 charset = charset_big5_2; \
2873 temp -= (0xC9 - 0xA1) * BIG5_SAME_ROW; \
2875 c1 = temp / (0xFF - 0xA1) + 0x21; \
2876 c2 = temp % (0xFF - 0xA1) + 0x21; \
2879 #define ENCODE_BIG5(charset, c1, c2, b1, b2) \
2881 unsigned int temp = (c1 - 0x21) * (0xFF - 0xA1) + (c2 - 0x21); \
2882 if (charset == charset_big5_2) \
2883 temp += BIG5_SAME_ROW * (0xC9 - 0xA1); \
2884 b1 = temp / BIG5_SAME_ROW + 0xA1; \
2885 b2 = temp % BIG5_SAME_ROW; \
2886 b2 += b2 < 0x3F ? 0x40 : 0x62; \
2889 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2890 Check if a text is encoded in SJIS. If it is, return
2891 CODING_CATEGORY_MASK_SJIS, else return 0. */
2894 detect_coding_sjis (src
, src_end
, multibytep
)
2895 unsigned char *src
, *src_end
;
2899 /* Dummy for ONE_MORE_BYTE. */
2900 struct coding_system dummy_coding
;
2901 struct coding_system
*coding
= &dummy_coding
;
2905 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2908 if (c
== 0x80 || c
== 0xA0 || c
> 0xEF)
2910 if (c
<= 0x9F || c
>= 0xE0)
2912 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2913 if (c
< 0x40 || c
== 0x7F || c
> 0xFC)
2918 return CODING_CATEGORY_MASK_SJIS
;
2921 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2922 Check if a text is encoded in BIG5. If it is, return
2923 CODING_CATEGORY_MASK_BIG5, else return 0. */
2926 detect_coding_big5 (src
, src_end
, multibytep
)
2927 unsigned char *src
, *src_end
;
2931 /* Dummy for ONE_MORE_BYTE. */
2932 struct coding_system dummy_coding
;
2933 struct coding_system
*coding
= &dummy_coding
;
2937 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2940 if (c
< 0xA1 || c
> 0xFE)
2942 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2943 if (c
< 0x40 || (c
> 0x7F && c
< 0xA1) || c
> 0xFE)
2947 return CODING_CATEGORY_MASK_BIG5
;
2950 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
2951 Check if a text is encoded in UTF-8. If it is, return
2952 CODING_CATEGORY_MASK_UTF_8, else return 0. */
2954 #define UTF_8_1_OCTET_P(c) ((c) < 0x80)
2955 #define UTF_8_EXTRA_OCTET_P(c) (((c) & 0xC0) == 0x80)
2956 #define UTF_8_2_OCTET_LEADING_P(c) (((c) & 0xE0) == 0xC0)
2957 #define UTF_8_3_OCTET_LEADING_P(c) (((c) & 0xF0) == 0xE0)
2958 #define UTF_8_4_OCTET_LEADING_P(c) (((c) & 0xF8) == 0xF0)
2959 #define UTF_8_5_OCTET_LEADING_P(c) (((c) & 0xFC) == 0xF8)
2960 #define UTF_8_6_OCTET_LEADING_P(c) (((c) & 0xFE) == 0xFC)
2963 detect_coding_utf_8 (src
, src_end
, multibytep
)
2964 unsigned char *src
, *src_end
;
2968 int seq_maybe_bytes
;
2969 /* Dummy for ONE_MORE_BYTE. */
2970 struct coding_system dummy_coding
;
2971 struct coding_system
*coding
= &dummy_coding
;
2975 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2976 if (UTF_8_1_OCTET_P (c
))
2978 else if (UTF_8_2_OCTET_LEADING_P (c
))
2979 seq_maybe_bytes
= 1;
2980 else if (UTF_8_3_OCTET_LEADING_P (c
))
2981 seq_maybe_bytes
= 2;
2982 else if (UTF_8_4_OCTET_LEADING_P (c
))
2983 seq_maybe_bytes
= 3;
2984 else if (UTF_8_5_OCTET_LEADING_P (c
))
2985 seq_maybe_bytes
= 4;
2986 else if (UTF_8_6_OCTET_LEADING_P (c
))
2987 seq_maybe_bytes
= 5;
2993 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
2994 if (!UTF_8_EXTRA_OCTET_P (c
))
2998 while (seq_maybe_bytes
> 0);
3002 return CODING_CATEGORY_MASK_UTF_8
;
3005 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3006 Check if a text is encoded in UTF-16 Big Endian (endian == 1) or
3007 Little Endian (otherwise). If it is, return
3008 CODING_CATEGORY_MASK_UTF_16_BE or CODING_CATEGORY_MASK_UTF_16_LE,
3011 #define UTF_16_INVALID_P(val) \
3012 (((val) == 0xFFFE) \
3013 || ((val) == 0xFFFF))
3015 #define UTF_16_HIGH_SURROGATE_P(val) \
3016 (((val) & 0xD800) == 0xD800)
3018 #define UTF_16_LOW_SURROGATE_P(val) \
3019 (((val) & 0xDC00) == 0xDC00)
3022 detect_coding_utf_16 (src
, src_end
, multibytep
)
3023 unsigned char *src
, *src_end
;
3026 unsigned char c1
, c2
;
3027 /* Dummy for ONE_MORE_BYTE_CHECK_MULTIBYTE. */
3028 struct coding_system dummy_coding
;
3029 struct coding_system
*coding
= &dummy_coding
;
3031 ONE_MORE_BYTE_CHECK_MULTIBYTE (c1
, multibytep
);
3032 ONE_MORE_BYTE_CHECK_MULTIBYTE (c2
, multibytep
);
3034 if ((c1
== 0xFF) && (c2
== 0xFE))
3035 return CODING_CATEGORY_MASK_UTF_16_LE
;
3036 else if ((c1
== 0xFE) && (c2
== 0xFF))
3037 return CODING_CATEGORY_MASK_UTF_16_BE
;
3043 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions".
3044 If SJIS_P is 1, decode SJIS text, else decode BIG5 test. */
3047 decode_coding_sjis_big5 (coding
, source
, destination
,
3048 src_bytes
, dst_bytes
, sjis_p
)
3049 struct coding_system
*coding
;
3050 unsigned char *source
, *destination
;
3051 int src_bytes
, dst_bytes
;
3054 unsigned char *src
= source
;
3055 unsigned char *src_end
= source
+ src_bytes
;
3056 unsigned char *dst
= destination
;
3057 unsigned char *dst_end
= destination
+ dst_bytes
;
3058 /* SRC_BASE remembers the start position in source in each loop.
3059 The loop will be exited when there's not enough source code
3060 (within macro ONE_MORE_BYTE), or when there's not enough
3061 destination area to produce a character (within macro
3063 unsigned char *src_base
;
3064 Lisp_Object translation_table
;
3066 if (NILP (Venable_character_translation
))
3067 translation_table
= Qnil
;
3070 translation_table
= coding
->translation_table_for_decode
;
3071 if (NILP (translation_table
))
3072 translation_table
= Vstandard_translation_table_for_decode
;
3075 coding
->produced_char
= 0;
3078 int c
, charset
, c1
, c2
= 0;
3085 charset
= CHARSET_ASCII
;
3090 if (coding
->eol_type
== CODING_EOL_CRLF
)
3096 /* To process C2 again, SRC is subtracted by 1. */
3099 else if (coding
->eol_type
== CODING_EOL_CR
)
3103 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3104 && (coding
->eol_type
== CODING_EOL_CR
3105 || coding
->eol_type
== CODING_EOL_CRLF
))
3107 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3108 goto label_end_of_loop
;
3116 if (c1
== 0x80 || c1
== 0xA0 || c1
> 0xEF)
3117 goto label_invalid_code
;
3118 if (c1
<= 0x9F || c1
>= 0xE0)
3120 /* SJIS -> JISX0208 */
3122 if (c2
< 0x40 || c2
== 0x7F || c2
> 0xFC)
3123 goto label_invalid_code
;
3124 DECODE_SJIS (c1
, c2
, c1
, c2
);
3125 charset
= charset_jisx0208
;
3128 /* SJIS -> JISX0201-Kana */
3129 charset
= charset_katakana_jisx0201
;
3134 if (c1
< 0xA0 || c1
> 0xFE)
3135 goto label_invalid_code
;
3137 if (c2
< 0x40 || (c2
> 0x7E && c2
< 0xA1) || c2
> 0xFE)
3138 goto label_invalid_code
;
3139 DECODE_BIG5 (c1
, c2
, charset
, c1
, c2
);
3143 c
= DECODE_ISO_CHARACTER (charset
, c1
, c2
);
3155 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3156 coding
->produced
= dst
- destination
;
3160 /* See the above "GENERAL NOTES on `encode_coding_XXX ()' functions".
3161 This function can encode charsets `ascii', `katakana-jisx0201',
3162 `japanese-jisx0208', `chinese-big5-1', and `chinese-big5-2'. We
3163 are sure that all these charsets are registered as official charset
3164 (i.e. do not have extended leading-codes). Characters of other
3165 charsets are produced without any encoding. If SJIS_P is 1, encode
3166 SJIS text, else encode BIG5 text. */
3169 encode_coding_sjis_big5 (coding
, source
, destination
,
3170 src_bytes
, dst_bytes
, sjis_p
)
3171 struct coding_system
*coding
;
3172 unsigned char *source
, *destination
;
3173 int src_bytes
, dst_bytes
;
3176 unsigned char *src
= source
;
3177 unsigned char *src_end
= source
+ src_bytes
;
3178 unsigned char *dst
= destination
;
3179 unsigned char *dst_end
= destination
+ dst_bytes
;
3180 /* SRC_BASE remembers the start position in source in each loop.
3181 The loop will be exited when there's not enough source text to
3182 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3183 there's not enough destination area to produce encoded codes
3184 (within macro EMIT_BYTES). */
3185 unsigned char *src_base
;
3186 Lisp_Object translation_table
;
3188 if (NILP (Venable_character_translation
))
3189 translation_table
= Qnil
;
3192 translation_table
= coding
->translation_table_for_encode
;
3193 if (NILP (translation_table
))
3194 translation_table
= Vstandard_translation_table_for_encode
;
3199 int c
, charset
, c1
, c2
;
3204 /* Now encode the character C. */
3205 if (SINGLE_BYTE_CHAR_P (c
))
3210 if (!(coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
))
3217 if (coding
->eol_type
== CODING_EOL_CRLF
)
3219 EMIT_TWO_BYTES ('\r', c
);
3222 else if (coding
->eol_type
== CODING_EOL_CR
)
3230 SPLIT_CHAR (c
, charset
, c1
, c2
);
3233 if (charset
== charset_jisx0208
3234 || charset
== charset_jisx0208_1978
)
3236 ENCODE_SJIS (c1
, c2
, c1
, c2
);
3237 EMIT_TWO_BYTES (c1
, c2
);
3239 else if (charset
== charset_katakana_jisx0201
)
3240 EMIT_ONE_BYTE (c1
| 0x80);
3241 else if (charset
== charset_latin_jisx0201
)
3243 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3245 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3246 if (CHARSET_WIDTH (charset
) > 1)
3247 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3250 /* There's no way other than producing the internal
3252 EMIT_BYTES (src_base
, src
);
3256 if (charset
== charset_big5_1
|| charset
== charset_big5_2
)
3258 ENCODE_BIG5 (charset
, c1
, c2
, c1
, c2
);
3259 EMIT_TWO_BYTES (c1
, c2
);
3261 else if (coding
->mode
& CODING_MODE_INHIBIT_UNENCODABLE_CHAR
)
3263 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3264 if (CHARSET_WIDTH (charset
) > 1)
3265 EMIT_ONE_BYTE (CODING_REPLACEMENT_CHARACTER
);
3268 /* There's no way other than producing the internal
3270 EMIT_BYTES (src_base
, src
);
3273 coding
->consumed_char
++;
3277 coding
->consumed
= src_base
- source
;
3278 coding
->produced
= coding
->produced_char
= dst
- destination
;
3282 /*** 5. CCL handlers ***/
3284 /* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
3285 Check if a text is encoded in a coding system of which
3286 encoder/decoder are written in CCL program. If it is, return
3287 CODING_CATEGORY_MASK_CCL, else return 0. */
3290 detect_coding_ccl (src
, src_end
, multibytep
)
3291 unsigned char *src
, *src_end
;
3294 unsigned char *valid
;
3296 /* Dummy for ONE_MORE_BYTE. */
3297 struct coding_system dummy_coding
;
3298 struct coding_system
*coding
= &dummy_coding
;
3300 /* No coding system is assigned to coding-category-ccl. */
3301 if (!coding_system_table
[CODING_CATEGORY_IDX_CCL
])
3304 valid
= coding_system_table
[CODING_CATEGORY_IDX_CCL
]->spec
.ccl
.valid_codes
;
3307 ONE_MORE_BYTE_CHECK_MULTIBYTE (c
, multibytep
);
3312 return CODING_CATEGORY_MASK_CCL
;
3316 /*** 6. End-of-line handlers ***/
3318 /* See the above "GENERAL NOTES on `decode_coding_XXX ()' functions". */
3321 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3322 struct coding_system
*coding
;
3323 unsigned char *source
, *destination
;
3324 int src_bytes
, dst_bytes
;
3326 unsigned char *src
= source
;
3327 unsigned char *dst
= destination
;
3328 unsigned char *src_end
= src
+ src_bytes
;
3329 unsigned char *dst_end
= dst
+ dst_bytes
;
3330 Lisp_Object translation_table
;
3331 /* SRC_BASE remembers the start position in source in each loop.
3332 The loop will be exited when there's not enough source code
3333 (within macro ONE_MORE_BYTE), or when there's not enough
3334 destination area to produce a character (within macro
3336 unsigned char *src_base
;
3339 translation_table
= Qnil
;
3340 switch (coding
->eol_type
)
3342 case CODING_EOL_CRLF
:
3357 && (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
))
3359 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3360 goto label_end_of_loop
;
3373 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
3375 coding
->result
= CODING_FINISH_INCONSISTENT_EOL
;
3376 goto label_end_of_loop
;
3385 default: /* no need for EOL handling */
3395 coding
->consumed
= coding
->consumed_char
= src_base
- source
;
3396 coding
->produced
= dst
- destination
;
3400 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". Encode
3401 format of end-of-line according to `coding->eol_type'. It also
3402 convert multibyte form 8-bit characters to unibyte if
3403 CODING->src_multibyte is nonzero. If `coding->mode &
3404 CODING_MODE_SELECTIVE_DISPLAY' is nonzero, code '\r' in source text
3405 also means end-of-line. */
3408 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
)
3409 struct coding_system
*coding
;
3410 const unsigned char *source
;
3411 unsigned char *destination
;
3412 int src_bytes
, dst_bytes
;
3414 const unsigned char *src
= source
;
3415 unsigned char *dst
= destination
;
3416 const unsigned char *src_end
= src
+ src_bytes
;
3417 unsigned char *dst_end
= dst
+ dst_bytes
;
3418 Lisp_Object translation_table
;
3419 /* SRC_BASE remembers the start position in source in each loop.
3420 The loop will be exited when there's not enough source text to
3421 analyze multi-byte codes (within macro ONE_MORE_CHAR), or when
3422 there's not enough destination area to produce encoded codes
3423 (within macro EMIT_BYTES). */
3424 const unsigned char *src_base
;
3427 int selective_display
= coding
->mode
& CODING_MODE_SELECTIVE_DISPLAY
;
3429 translation_table
= Qnil
;
3430 if (coding
->src_multibyte
3431 && *(src_end
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3435 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
3438 if (coding
->eol_type
== CODING_EOL_CRLF
)
3440 while (src
< src_end
)
3446 else if (c
== '\n' || (c
== '\r' && selective_display
))
3447 EMIT_TWO_BYTES ('\r', '\n');
3457 if (!dst_bytes
|| src_bytes
<= dst_bytes
)
3459 safe_bcopy (src
, dst
, src_bytes
);
3465 if (coding
->src_multibyte
3466 && *(src
+ dst_bytes
- 1) == LEADING_CODE_8_BIT_CONTROL
)
3468 safe_bcopy (src
, dst
, dst_bytes
);
3469 src_base
= src
+ dst_bytes
;
3470 dst
= destination
+ dst_bytes
;
3471 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
3473 if (coding
->eol_type
== CODING_EOL_CR
)
3475 for (tmp
= destination
; tmp
< dst
; tmp
++)
3476 if (*tmp
== '\n') *tmp
= '\r';
3478 else if (selective_display
)
3480 for (tmp
= destination
; tmp
< dst
; tmp
++)
3481 if (*tmp
== '\r') *tmp
= '\n';
3484 if (coding
->src_multibyte
)
3485 dst
= destination
+ str_as_unibyte (destination
, dst
- destination
);
3487 coding
->consumed
= src_base
- source
;
3488 coding
->produced
= dst
- destination
;
3489 coding
->produced_char
= coding
->produced
;
3493 /*** 7. C library functions ***/
3495 /* In Emacs Lisp, a coding system is represented by a Lisp symbol which
3496 has a property `coding-system'. The value of this property is a
3497 vector of length 5 (called the coding-vector). Among elements of
3498 this vector, the first (element[0]) and the fifth (element[4])
3499 carry important information for decoding/encoding. Before
3500 decoding/encoding, this information should be set in fields of a
3501 structure of type `coding_system'.
3503 The value of the property `coding-system' can be a symbol of another
3504 subsidiary coding-system. In that case, Emacs gets coding-vector
3507 `element[0]' contains information to be set in `coding->type'. The
3508 value and its meaning is as follows:
3510 0 -- coding_type_emacs_mule
3511 1 -- coding_type_sjis
3512 2 -- coding_type_iso2022
3513 3 -- coding_type_big5
3514 4 -- coding_type_ccl encoder/decoder written in CCL
3515 nil -- coding_type_no_conversion
3516 t -- coding_type_undecided (automatic conversion on decoding,
3517 no-conversion on encoding)
3519 `element[4]' contains information to be set in `coding->flags' and
3520 `coding->spec'. The meaning varies by `coding->type'.
3522 If `coding->type' is `coding_type_iso2022', element[4] is a vector
3523 of length 32 (of which the first 13 sub-elements are used now).
3524 Meanings of these sub-elements are:
3526 sub-element[N] where N is 0 through 3: to be set in `coding->spec.iso2022'
3527 If the value is an integer of valid charset, the charset is
3528 assumed to be designated to graphic register N initially.
3530 If the value is minus, it is a minus value of charset which
3531 reserves graphic register N, which means that the charset is
3532 not designated initially but should be designated to graphic
3533 register N just before encoding a character in that charset.
3535 If the value is nil, graphic register N is never used on
3538 sub-element[N] where N is 4 through 11: to be set in `coding->flags'
3539 Each value takes t or nil. See the section ISO2022 of
3540 `coding.h' for more information.
3542 If `coding->type' is `coding_type_big5', element[4] is t to denote
3543 BIG5-ETen or nil to denote BIG5-HKU.
3545 If `coding->type' takes the other value, element[4] is ignored.
3547 Emacs Lisp's coding systems also carry information about format of
3548 end-of-line in a value of property `eol-type'. If the value is
3549 integer, 0 means CODING_EOL_LF, 1 means CODING_EOL_CRLF, and 2
3550 means CODING_EOL_CR. If it is not integer, it should be a vector
3551 of subsidiary coding systems of which property `eol-type' has one
3552 of the above values.
3556 /* Extract information for decoding/encoding from CODING_SYSTEM_SYMBOL
3557 and set it in CODING. If CODING_SYSTEM_SYMBOL is invalid, CODING
3558 is setup so that no conversion is necessary and return -1, else
3562 setup_coding_system (coding_system
, coding
)
3563 Lisp_Object coding_system
;
3564 struct coding_system
*coding
;
3566 Lisp_Object coding_spec
, coding_type
, eol_type
, plist
;
3569 /* At first, zero clear all members. */
3570 bzero (coding
, sizeof (struct coding_system
));
3572 /* Initialize some fields required for all kinds of coding systems. */
3573 coding
->symbol
= coding_system
;
3574 coding
->heading_ascii
= -1;
3575 coding
->post_read_conversion
= coding
->pre_write_conversion
= Qnil
;
3576 coding
->composing
= COMPOSITION_DISABLED
;
3577 coding
->cmp_data
= NULL
;
3579 if (NILP (coding_system
))
3580 goto label_invalid_coding_system
;
3582 coding_spec
= Fget (coding_system
, Qcoding_system
);
3584 if (!VECTORP (coding_spec
)
3585 || XVECTOR (coding_spec
)->size
!= 5
3586 || !CONSP (XVECTOR (coding_spec
)->contents
[3]))
3587 goto label_invalid_coding_system
;
3589 eol_type
= inhibit_eol_conversion
? Qnil
: Fget (coding_system
, Qeol_type
);
3590 if (VECTORP (eol_type
))
3592 coding
->eol_type
= CODING_EOL_UNDECIDED
;
3593 coding
->common_flags
= CODING_REQUIRE_DETECTION_MASK
;
3595 else if (XFASTINT (eol_type
) == 1)
3597 coding
->eol_type
= CODING_EOL_CRLF
;
3598 coding
->common_flags
3599 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3601 else if (XFASTINT (eol_type
) == 2)
3603 coding
->eol_type
= CODING_EOL_CR
;
3604 coding
->common_flags
3605 = CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3608 coding
->eol_type
= CODING_EOL_LF
;
3610 coding_type
= XVECTOR (coding_spec
)->contents
[0];
3611 /* Try short cut. */
3612 if (SYMBOLP (coding_type
))
3614 if (EQ (coding_type
, Qt
))
3616 coding
->type
= coding_type_undecided
;
3617 coding
->common_flags
|= CODING_REQUIRE_DETECTION_MASK
;
3620 coding
->type
= coding_type_no_conversion
;
3621 /* Initialize this member. Any thing other than
3622 CODING_CATEGORY_IDX_UTF_16_BE and
3623 CODING_CATEGORY_IDX_UTF_16_LE are ok because they have
3624 special treatment in detect_eol. */
3625 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
3630 /* Get values of coding system properties:
3631 `post-read-conversion', `pre-write-conversion',
3632 `translation-table-for-decode', `translation-table-for-encode'. */
3633 plist
= XVECTOR (coding_spec
)->contents
[3];
3634 /* Pre & post conversion functions should be disabled if
3635 inhibit_eol_conversion is nonzero. This is the case that a code
3636 conversion function is called while those functions are running. */
3637 if (! inhibit_pre_post_conversion
)
3639 coding
->post_read_conversion
= Fplist_get (plist
, Qpost_read_conversion
);
3640 coding
->pre_write_conversion
= Fplist_get (plist
, Qpre_write_conversion
);
3642 val
= Fplist_get (plist
, Qtranslation_table_for_decode
);
3644 val
= Fget (val
, Qtranslation_table_for_decode
);
3645 coding
->translation_table_for_decode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3646 val
= Fplist_get (plist
, Qtranslation_table_for_encode
);
3648 val
= Fget (val
, Qtranslation_table_for_encode
);
3649 coding
->translation_table_for_encode
= CHAR_TABLE_P (val
) ? val
: Qnil
;
3650 val
= Fplist_get (plist
, Qcoding_category
);
3653 val
= Fget (val
, Qcoding_category_index
);
3655 coding
->category_idx
= XINT (val
);
3657 goto label_invalid_coding_system
;
3660 goto label_invalid_coding_system
;
3662 /* If the coding system has non-nil `composition' property, enable
3663 composition handling. */
3664 val
= Fplist_get (plist
, Qcomposition
);
3666 coding
->composing
= COMPOSITION_NO
;
3668 switch (XFASTINT (coding_type
))
3671 coding
->type
= coding_type_emacs_mule
;
3672 coding
->common_flags
3673 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3674 if (!NILP (coding
->post_read_conversion
))
3675 coding
->common_flags
|= CODING_REQUIRE_DECODING_MASK
;
3676 if (!NILP (coding
->pre_write_conversion
))
3677 coding
->common_flags
|= CODING_REQUIRE_ENCODING_MASK
;
3681 coding
->type
= coding_type_sjis
;
3682 coding
->common_flags
3683 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3687 coding
->type
= coding_type_iso2022
;
3688 coding
->common_flags
3689 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3691 Lisp_Object val
, temp
;
3693 int i
, charset
, reg_bits
= 0;
3695 val
= XVECTOR (coding_spec
)->contents
[4];
3697 if (!VECTORP (val
) || XVECTOR (val
)->size
!= 32)
3698 goto label_invalid_coding_system
;
3700 flags
= XVECTOR (val
)->contents
;
3702 = ((NILP (flags
[4]) ? 0 : CODING_FLAG_ISO_SHORT_FORM
)
3703 | (NILP (flags
[5]) ? 0 : CODING_FLAG_ISO_RESET_AT_EOL
)
3704 | (NILP (flags
[6]) ? 0 : CODING_FLAG_ISO_RESET_AT_CNTL
)
3705 | (NILP (flags
[7]) ? 0 : CODING_FLAG_ISO_SEVEN_BITS
)
3706 | (NILP (flags
[8]) ? 0 : CODING_FLAG_ISO_LOCKING_SHIFT
)
3707 | (NILP (flags
[9]) ? 0 : CODING_FLAG_ISO_SINGLE_SHIFT
)
3708 | (NILP (flags
[10]) ? 0 : CODING_FLAG_ISO_USE_ROMAN
)
3709 | (NILP (flags
[11]) ? 0 : CODING_FLAG_ISO_USE_OLDJIS
)
3710 | (NILP (flags
[12]) ? 0 : CODING_FLAG_ISO_NO_DIRECTION
)
3711 | (NILP (flags
[13]) ? 0 : CODING_FLAG_ISO_INIT_AT_BOL
)
3712 | (NILP (flags
[14]) ? 0 : CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
3713 | (NILP (flags
[15]) ? 0 : CODING_FLAG_ISO_SAFE
)
3714 | (NILP (flags
[16]) ? 0 : CODING_FLAG_ISO_LATIN_EXTRA
)
3717 /* Invoke graphic register 0 to plane 0. */
3718 CODING_SPEC_ISO_INVOCATION (coding
, 0) = 0;
3719 /* Invoke graphic register 1 to plane 1 if we can use full 8-bit. */
3720 CODING_SPEC_ISO_INVOCATION (coding
, 1)
3721 = (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
? -1 : 1);
3722 /* Not single shifting at first. */
3723 CODING_SPEC_ISO_SINGLE_SHIFTING (coding
) = 0;
3724 /* Beginning of buffer should also be regarded as bol. */
3725 CODING_SPEC_ISO_BOL (coding
) = 1;
3727 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3728 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = 255;
3729 val
= Vcharset_revision_alist
;
3732 charset
= get_charset_id (Fcar_safe (XCAR (val
)));
3734 && (temp
= Fcdr_safe (XCAR (val
)), INTEGERP (temp
))
3735 && (i
= XINT (temp
), (i
>= 0 && (i
+ '@') < 128)))
3736 CODING_SPEC_ISO_REVISION_NUMBER (coding
, charset
) = i
;
3740 /* Checks FLAGS[REG] (REG = 0, 1, 2 3) and decide designations.
3741 FLAGS[REG] can be one of below:
3742 integer CHARSET: CHARSET occupies register I,
3743 t: designate nothing to REG initially, but can be used
3745 list of integer, nil, or t: designate the first
3746 element (if integer) to REG initially, the remaining
3747 elements (if integer) is designated to REG on request,
3748 if an element is t, REG can be used by any charsets,
3749 nil: REG is never used. */
3750 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3751 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3752 = CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
;
3753 for (i
= 0; i
< 4; i
++)
3755 if ((INTEGERP (flags
[i
])
3756 && (charset
= XINT (flags
[i
]), CHARSET_VALID_P (charset
)))
3757 || (charset
= get_charset_id (flags
[i
])) >= 0)
3759 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3760 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) = i
;
3762 else if (EQ (flags
[i
], Qt
))
3764 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3766 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3768 else if (CONSP (flags
[i
]))
3773 coding
->flags
|= CODING_FLAG_ISO_DESIGNATION
;
3774 if ((INTEGERP (XCAR (tail
))
3775 && (charset
= XINT (XCAR (tail
)),
3776 CHARSET_VALID_P (charset
)))
3777 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3779 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = charset
;
3780 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
) =i
;
3783 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3785 while (CONSP (tail
))
3787 if ((INTEGERP (XCAR (tail
))
3788 && (charset
= XINT (XCAR (tail
)),
3789 CHARSET_VALID_P (charset
)))
3790 || (charset
= get_charset_id (XCAR (tail
))) >= 0)
3791 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3793 else if (EQ (XCAR (tail
), Qt
))
3799 CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
) = -1;
3801 CODING_SPEC_ISO_DESIGNATION (coding
, i
)
3802 = CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, i
);
3805 if (reg_bits
&& ! (coding
->flags
& CODING_FLAG_ISO_LOCKING_SHIFT
))
3807 /* REG 1 can be used only by locking shift in 7-bit env. */
3808 if (coding
->flags
& CODING_FLAG_ISO_SEVEN_BITS
)
3810 if (! (coding
->flags
& CODING_FLAG_ISO_SINGLE_SHIFT
))
3811 /* Without any shifting, only REG 0 and 1 can be used. */
3816 for (charset
= 0; charset
<= MAX_CHARSET
; charset
++)
3818 if (CHARSET_DEFINED_P (charset
)
3819 && (CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3820 == CODING_SPEC_ISO_NO_REQUESTED_DESIGNATION
))
3822 /* There exist some default graphic registers to be
3825 /* We had better avoid designating a charset of
3826 CHARS96 to REG 0 as far as possible. */
3827 if (CHARSET_CHARS (charset
) == 96)
3828 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3830 ? 1 : (reg_bits
& 4 ? 2 : (reg_bits
& 8 ? 3 : 0)));
3832 CODING_SPEC_ISO_REQUESTED_DESIGNATION (coding
, charset
)
3834 ? 0 : (reg_bits
& 2 ? 1 : (reg_bits
& 4 ? 2 : 3)));
3838 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3839 coding
->spec
.iso2022
.last_invalid_designation_register
= -1;
3843 coding
->type
= coding_type_big5
;
3844 coding
->common_flags
3845 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3847 = (NILP (XVECTOR (coding_spec
)->contents
[4])
3848 ? CODING_FLAG_BIG5_HKU
3849 : CODING_FLAG_BIG5_ETEN
);
3853 coding
->type
= coding_type_ccl
;
3854 coding
->common_flags
3855 |= CODING_REQUIRE_DECODING_MASK
| CODING_REQUIRE_ENCODING_MASK
;
3857 val
= XVECTOR (coding_spec
)->contents
[4];
3859 || setup_ccl_program (&(coding
->spec
.ccl
.decoder
),
3861 || setup_ccl_program (&(coding
->spec
.ccl
.encoder
),
3863 goto label_invalid_coding_system
;
3865 bzero (coding
->spec
.ccl
.valid_codes
, 256);
3866 val
= Fplist_get (plist
, Qvalid_codes
);
3871 for (; CONSP (val
); val
= XCDR (val
))
3875 && XINT (this) >= 0 && XINT (this) < 256)
3876 coding
->spec
.ccl
.valid_codes
[XINT (this)] = 1;
3877 else if (CONSP (this)
3878 && INTEGERP (XCAR (this))
3879 && INTEGERP (XCDR (this)))
3881 int start
= XINT (XCAR (this));
3882 int end
= XINT (XCDR (this));
3884 if (start
>= 0 && start
<= end
&& end
< 256)
3885 while (start
<= end
)
3886 coding
->spec
.ccl
.valid_codes
[start
++] = 1;
3891 coding
->common_flags
|= CODING_REQUIRE_FLUSHING_MASK
;
3892 coding
->spec
.ccl
.cr_carryover
= 0;
3893 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
3897 coding
->type
= coding_type_raw_text
;
3901 goto label_invalid_coding_system
;
3905 label_invalid_coding_system
:
3906 coding
->type
= coding_type_no_conversion
;
3907 coding
->category_idx
= CODING_CATEGORY_IDX_BINARY
;
3908 coding
->common_flags
= 0;
3909 coding
->eol_type
= CODING_EOL_LF
;
3910 coding
->pre_write_conversion
= coding
->post_read_conversion
= Qnil
;
3914 /* Free memory blocks allocated for storing composition information. */
3917 coding_free_composition_data (coding
)
3918 struct coding_system
*coding
;
3920 struct composition_data
*cmp_data
= coding
->cmp_data
, *next
;
3924 /* Memory blocks are chained. At first, rewind to the first, then,
3925 free blocks one by one. */
3926 while (cmp_data
->prev
)
3927 cmp_data
= cmp_data
->prev
;
3930 next
= cmp_data
->next
;
3934 coding
->cmp_data
= NULL
;
3937 /* Set `char_offset' member of all memory blocks pointed by
3938 coding->cmp_data to POS. */
3941 coding_adjust_composition_offset (coding
, pos
)
3942 struct coding_system
*coding
;
3945 struct composition_data
*cmp_data
;
3947 for (cmp_data
= coding
->cmp_data
; cmp_data
; cmp_data
= cmp_data
->next
)
3948 cmp_data
->char_offset
= pos
;
3951 /* Setup raw-text or one of its subsidiaries in the structure
3952 coding_system CODING according to the already setup value eol_type
3953 in CODING. CODING should be setup for some coding system in
3957 setup_raw_text_coding_system (coding
)
3958 struct coding_system
*coding
;
3960 if (coding
->type
!= coding_type_raw_text
)
3962 coding
->symbol
= Qraw_text
;
3963 coding
->type
= coding_type_raw_text
;
3964 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
3966 Lisp_Object subsidiaries
;
3967 subsidiaries
= Fget (Qraw_text
, Qeol_type
);
3969 if (VECTORP (subsidiaries
)
3970 && XVECTOR (subsidiaries
)->size
== 3)
3972 = XVECTOR (subsidiaries
)->contents
[coding
->eol_type
];
3974 setup_coding_system (coding
->symbol
, coding
);
3979 /* Emacs has a mechanism to automatically detect a coding system if it
3980 is one of Emacs' internal format, ISO2022, SJIS, and BIG5. But,
3981 it's impossible to distinguish some coding systems accurately
3982 because they use the same range of codes. So, at first, coding
3983 systems are categorized into 7, those are:
3985 o coding-category-emacs-mule
3987 The category for a coding system which has the same code range
3988 as Emacs' internal format. Assigned the coding-system (Lisp
3989 symbol) `emacs-mule' by default.
3991 o coding-category-sjis
3993 The category for a coding system which has the same code range
3994 as SJIS. Assigned the coding-system (Lisp
3995 symbol) `japanese-shift-jis' by default.
3997 o coding-category-iso-7
3999 The category for a coding system which has the same code range
4000 as ISO2022 of 7-bit environment. This doesn't use any locking
4001 shift and single shift functions. This can encode/decode all
4002 charsets. Assigned the coding-system (Lisp symbol)
4003 `iso-2022-7bit' by default.
4005 o coding-category-iso-7-tight
4007 Same as coding-category-iso-7 except that this can
4008 encode/decode only the specified charsets.
4010 o coding-category-iso-8-1
4012 The category for a coding system which has the same code range
4013 as ISO2022 of 8-bit environment and graphic plane 1 used only
4014 for DIMENSION1 charset. This doesn't use any locking shift
4015 and single shift functions. Assigned the coding-system (Lisp
4016 symbol) `iso-latin-1' by default.
4018 o coding-category-iso-8-2
4020 The category for a coding system which has the same code range
4021 as ISO2022 of 8-bit environment and graphic plane 1 used only
4022 for DIMENSION2 charset. This doesn't use any locking shift
4023 and single shift functions. Assigned the coding-system (Lisp
4024 symbol) `japanese-iso-8bit' by default.
4026 o coding-category-iso-7-else
4028 The category for a coding system which has the same code range
4029 as ISO2022 of 7-bit environment but uses locking shift or
4030 single shift functions. Assigned the coding-system (Lisp
4031 symbol) `iso-2022-7bit-lock' by default.
4033 o coding-category-iso-8-else
4035 The category for a coding system which has the same code range
4036 as ISO2022 of 8-bit environment but uses locking shift or
4037 single shift functions. Assigned the coding-system (Lisp
4038 symbol) `iso-2022-8bit-ss2' by default.
4040 o coding-category-big5
4042 The category for a coding system which has the same code range
4043 as BIG5. Assigned the coding-system (Lisp symbol)
4044 `cn-big5' by default.
4046 o coding-category-utf-8
4048 The category for a coding system which has the same code range
4049 as UTF-8 (cf. RFC3629). Assigned the coding-system (Lisp
4050 symbol) `utf-8' by default.
4052 o coding-category-utf-16-be
4054 The category for a coding system in which a text has an
4055 Unicode signature (cf. Unicode Standard) in the order of BIG
4056 endian at the head. Assigned the coding-system (Lisp symbol)
4057 `utf-16-be' by default.
4059 o coding-category-utf-16-le
4061 The category for a coding system in which a text has an
4062 Unicode signature (cf. Unicode Standard) in the order of
4063 LITTLE endian at the head. Assigned the coding-system (Lisp
4064 symbol) `utf-16-le' by default.
4066 o coding-category-ccl
4068 The category for a coding system of which encoder/decoder is
4069 written in CCL programs. The default value is nil, i.e., no
4070 coding system is assigned.
4072 o coding-category-binary
4074 The category for a coding system not categorized in any of the
4075 above. Assigned the coding-system (Lisp symbol)
4076 `no-conversion' by default.
4078 Each of them is a Lisp symbol and the value is an actual
4079 `coding-system' (this is also a Lisp symbol) assigned by a user.
4080 What Emacs does actually is to detect a category of coding system.
4081 Then, it uses a `coding-system' assigned to it. If Emacs can't
4082 decide a single possible category, it selects a category of the
4083 highest priority. Priorities of categories are also specified by a
4084 user in a Lisp variable `coding-category-list'.
4089 int ascii_skip_code
[256];
4091 /* Detect how a text of length SRC_BYTES pointed by SOURCE is encoded.
4092 If it detects possible coding systems, return an integer in which
4093 appropriate flag bits are set. Flag bits are defined by macros
4094 CODING_CATEGORY_MASK_XXX in `coding.h'. If PRIORITIES is non-NULL,
4095 it should point the table `coding_priorities'. In that case, only
4096 the flag bit for a coding system of the highest priority is set in
4097 the returned value. If MULTIBYTEP is nonzero, 8-bit codes of the
4098 range 0x80..0x9F are in multibyte form.
4100 How many ASCII characters are at the head is returned as *SKIP. */
4103 detect_coding_mask (source
, src_bytes
, priorities
, skip
, multibytep
)
4104 unsigned char *source
;
4105 int src_bytes
, *priorities
, *skip
;
4108 register unsigned char c
;
4109 unsigned char *src
= source
, *src_end
= source
+ src_bytes
;
4110 unsigned int mask
, utf16_examined_p
, iso2022_examined_p
;
4113 /* At first, skip all ASCII characters and control characters except
4114 for three ISO2022 specific control characters. */
4115 ascii_skip_code
[ISO_CODE_SO
] = 0;
4116 ascii_skip_code
[ISO_CODE_SI
] = 0;
4117 ascii_skip_code
[ISO_CODE_ESC
] = 0;
4119 label_loop_detect_coding
:
4120 while (src
< src_end
&& ascii_skip_code
[*src
]) src
++;
4121 *skip
= src
- source
;
4124 /* We found nothing other than ASCII. There's nothing to do. */
4128 /* The text seems to be encoded in some multilingual coding system.
4129 Now, try to find in which coding system the text is encoded. */
4132 /* i.e. (c == ISO_CODE_ESC || c == ISO_CODE_SI || c == ISO_CODE_SO) */
4133 /* C is an ISO2022 specific control code of C0. */
4134 mask
= detect_coding_iso2022 (src
, src_end
, multibytep
);
4137 /* No valid ISO2022 code follows C. Try again. */
4139 if (c
== ISO_CODE_ESC
)
4140 ascii_skip_code
[ISO_CODE_ESC
] = 1;
4142 ascii_skip_code
[ISO_CODE_SO
] = ascii_skip_code
[ISO_CODE_SI
] = 1;
4143 goto label_loop_detect_coding
;
4147 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4149 if (mask
& priorities
[i
])
4150 return priorities
[i
];
4152 return CODING_CATEGORY_MASK_RAW_TEXT
;
4159 if (multibytep
&& c
== LEADING_CODE_8_BIT_CONTROL
)
4164 /* C is the first byte of SJIS character code,
4165 or a leading-code of Emacs' internal format (emacs-mule),
4166 or the first byte of UTF-16. */
4167 try = (CODING_CATEGORY_MASK_SJIS
4168 | CODING_CATEGORY_MASK_EMACS_MULE
4169 | CODING_CATEGORY_MASK_UTF_16_BE
4170 | CODING_CATEGORY_MASK_UTF_16_LE
);
4172 /* Or, if C is a special latin extra code,
4173 or is an ISO2022 specific control code of C1 (SS2 or SS3),
4174 or is an ISO2022 control-sequence-introducer (CSI),
4175 we should also consider the possibility of ISO2022 codings. */
4176 if ((VECTORP (Vlatin_extra_code_table
)
4177 && !NILP (XVECTOR (Vlatin_extra_code_table
)->contents
[c
]))
4178 || (c
== ISO_CODE_SS2
|| c
== ISO_CODE_SS3
)
4179 || (c
== ISO_CODE_CSI
4182 || ((*src
== '0' || *src
== '1' || *src
== '2')
4183 && src
+ 1 < src_end
4184 && src
[1] == ']')))))
4185 try |= (CODING_CATEGORY_MASK_ISO_8_ELSE
4186 | CODING_CATEGORY_MASK_ISO_8BIT
);
4189 /* C is a character of ISO2022 in graphic plane right,
4190 or a SJIS's 1-byte character code (i.e. JISX0201),
4191 or the first byte of BIG5's 2-byte code,
4192 or the first byte of UTF-8/16. */
4193 try = (CODING_CATEGORY_MASK_ISO_8_ELSE
4194 | CODING_CATEGORY_MASK_ISO_8BIT
4195 | CODING_CATEGORY_MASK_SJIS
4196 | CODING_CATEGORY_MASK_BIG5
4197 | CODING_CATEGORY_MASK_UTF_8
4198 | CODING_CATEGORY_MASK_UTF_16_BE
4199 | CODING_CATEGORY_MASK_UTF_16_LE
);
4201 /* Or, we may have to consider the possibility of CCL. */
4202 if (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4203 && (coding_system_table
[CODING_CATEGORY_IDX_CCL
]
4204 ->spec
.ccl
.valid_codes
)[c
])
4205 try |= CODING_CATEGORY_MASK_CCL
;
4208 utf16_examined_p
= iso2022_examined_p
= 0;
4211 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
4213 if (!iso2022_examined_p
4214 && (priorities
[i
] & try & CODING_CATEGORY_MASK_ISO
))
4216 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4217 iso2022_examined_p
= 1;
4219 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_SJIS
)
4220 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4221 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_UTF_8
)
4222 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4223 else if (!utf16_examined_p
4224 && (priorities
[i
] & try &
4225 CODING_CATEGORY_MASK_UTF_16_BE_LE
))
4227 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4228 utf16_examined_p
= 1;
4230 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_BIG5
)
4231 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4232 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_EMACS_MULE
)
4233 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4234 else if (priorities
[i
] & try & CODING_CATEGORY_MASK_CCL
)
4235 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4236 else if (priorities
[i
] & CODING_CATEGORY_MASK_RAW_TEXT
)
4237 mask
|= CODING_CATEGORY_MASK_RAW_TEXT
;
4238 else if (priorities
[i
] & CODING_CATEGORY_MASK_BINARY
)
4239 mask
|= CODING_CATEGORY_MASK_BINARY
;
4240 if (mask
& priorities
[i
])
4241 return priorities
[i
];
4243 return CODING_CATEGORY_MASK_RAW_TEXT
;
4245 if (try & CODING_CATEGORY_MASK_ISO
)
4246 mask
|= detect_coding_iso2022 (src
, src_end
, multibytep
);
4247 if (try & CODING_CATEGORY_MASK_SJIS
)
4248 mask
|= detect_coding_sjis (src
, src_end
, multibytep
);
4249 if (try & CODING_CATEGORY_MASK_BIG5
)
4250 mask
|= detect_coding_big5 (src
, src_end
, multibytep
);
4251 if (try & CODING_CATEGORY_MASK_UTF_8
)
4252 mask
|= detect_coding_utf_8 (src
, src_end
, multibytep
);
4253 if (try & CODING_CATEGORY_MASK_UTF_16_BE_LE
)
4254 mask
|= detect_coding_utf_16 (src
, src_end
, multibytep
);
4255 if (try & CODING_CATEGORY_MASK_EMACS_MULE
)
4256 mask
|= detect_coding_emacs_mule (src
, src_end
, multibytep
);
4257 if (try & CODING_CATEGORY_MASK_CCL
)
4258 mask
|= detect_coding_ccl (src
, src_end
, multibytep
);
4260 return (mask
| CODING_CATEGORY_MASK_RAW_TEXT
| CODING_CATEGORY_MASK_BINARY
);
4263 /* Detect how a text of length SRC_BYTES pointed by SRC is encoded.
4264 The information of the detected coding system is set in CODING. */
4267 detect_coding (coding
, src
, src_bytes
)
4268 struct coding_system
*coding
;
4269 const unsigned char *src
;
4276 val
= Vcoding_category_list
;
4277 mask
= detect_coding_mask (src
, src_bytes
, coding_priorities
, &skip
,
4278 coding
->src_multibyte
);
4279 coding
->heading_ascii
= skip
;
4283 /* We found a single coding system of the highest priority in MASK. */
4285 while (mask
&& ! (mask
& 1)) mask
>>= 1, idx
++;
4287 idx
= CODING_CATEGORY_IDX_RAW_TEXT
;
4289 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[idx
]);
4291 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4295 tmp
= Fget (val
, Qeol_type
);
4297 val
= XVECTOR (tmp
)->contents
[coding
->eol_type
];
4300 /* Setup this new coding system while preserving some slots. */
4302 int src_multibyte
= coding
->src_multibyte
;
4303 int dst_multibyte
= coding
->dst_multibyte
;
4305 setup_coding_system (val
, coding
);
4306 coding
->src_multibyte
= src_multibyte
;
4307 coding
->dst_multibyte
= dst_multibyte
;
4308 coding
->heading_ascii
= skip
;
4312 /* Detect how end-of-line of a text of length SRC_BYTES pointed by
4313 SOURCE is encoded. Return one of CODING_EOL_LF, CODING_EOL_CRLF,
4314 CODING_EOL_CR, and CODING_EOL_UNDECIDED.
4316 How many non-eol characters are at the head is returned as *SKIP. */
4318 #define MAX_EOL_CHECK_COUNT 3
4321 detect_eol_type (source
, src_bytes
, skip
)
4322 unsigned char *source
;
4323 int src_bytes
, *skip
;
4325 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4327 int total
= 0; /* How many end-of-lines are found so far. */
4328 int eol_type
= CODING_EOL_UNDECIDED
;
4333 while (src
< src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4336 if (c
== '\n' || c
== '\r')
4339 *skip
= src
- 1 - source
;
4342 this_eol_type
= CODING_EOL_LF
;
4343 else if (src
>= src_end
|| *src
!= '\n')
4344 this_eol_type
= CODING_EOL_CR
;
4346 this_eol_type
= CODING_EOL_CRLF
, src
++;
4348 if (eol_type
== CODING_EOL_UNDECIDED
)
4349 /* This is the first end-of-line. */
4350 eol_type
= this_eol_type
;
4351 else if (eol_type
!= this_eol_type
)
4353 /* The found type is different from what found before. */
4354 eol_type
= CODING_EOL_INCONSISTENT
;
4361 *skip
= src_end
- source
;
4365 /* Like detect_eol_type, but detect EOL type in 2-octet
4366 big-endian/little-endian format for coding systems utf-16-be and
4370 detect_eol_type_in_2_octet_form (source
, src_bytes
, skip
, big_endian_p
)
4371 unsigned char *source
;
4372 int src_bytes
, *skip
, big_endian_p
;
4374 unsigned char *src
= source
, *src_end
= src
+ src_bytes
;
4375 unsigned int c1
, c2
;
4376 int total
= 0; /* How many end-of-lines are found so far. */
4377 int eol_type
= CODING_EOL_UNDECIDED
;
4388 while ((src
+ 1) < src_end
&& total
< MAX_EOL_CHECK_COUNT
)
4390 c1
= (src
[msb
] << 8) | (src
[lsb
]);
4393 if (c1
== '\n' || c1
== '\r')
4396 *skip
= src
- 2 - source
;
4400 this_eol_type
= CODING_EOL_LF
;
4404 if ((src
+ 1) >= src_end
)
4406 this_eol_type
= CODING_EOL_CR
;
4410 c2
= (src
[msb
] << 8) | (src
[lsb
]);
4412 this_eol_type
= CODING_EOL_CRLF
, src
+= 2;
4414 this_eol_type
= CODING_EOL_CR
;
4418 if (eol_type
== CODING_EOL_UNDECIDED
)
4419 /* This is the first end-of-line. */
4420 eol_type
= this_eol_type
;
4421 else if (eol_type
!= this_eol_type
)
4423 /* The found type is different from what found before. */
4424 eol_type
= CODING_EOL_INCONSISTENT
;
4431 *skip
= src_end
- source
;
4435 /* Detect how end-of-line of a text of length SRC_BYTES pointed by SRC
4436 is encoded. If it detects an appropriate format of end-of-line, it
4437 sets the information in *CODING. */
4440 detect_eol (coding
, src
, src_bytes
)
4441 struct coding_system
*coding
;
4442 const unsigned char *src
;
4449 switch (coding
->category_idx
)
4451 case CODING_CATEGORY_IDX_UTF_16_BE
:
4452 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 1);
4454 case CODING_CATEGORY_IDX_UTF_16_LE
:
4455 eol_type
= detect_eol_type_in_2_octet_form (src
, src_bytes
, &skip
, 0);
4458 eol_type
= detect_eol_type (src
, src_bytes
, &skip
);
4462 if (coding
->heading_ascii
> skip
)
4463 coding
->heading_ascii
= skip
;
4465 skip
= coding
->heading_ascii
;
4467 if (eol_type
== CODING_EOL_UNDECIDED
)
4469 if (eol_type
== CODING_EOL_INCONSISTENT
)
4472 /* This code is suppressed until we find a better way to
4473 distinguish raw text file and binary file. */
4475 /* If we have already detected that the coding is raw-text, the
4476 coding should actually be no-conversion. */
4477 if (coding
->type
== coding_type_raw_text
)
4479 setup_coding_system (Qno_conversion
, coding
);
4482 /* Else, let's decode only text code anyway. */
4484 eol_type
= CODING_EOL_LF
;
4487 val
= Fget (coding
->symbol
, Qeol_type
);
4488 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4490 int src_multibyte
= coding
->src_multibyte
;
4491 int dst_multibyte
= coding
->dst_multibyte
;
4492 struct composition_data
*cmp_data
= coding
->cmp_data
;
4494 setup_coding_system (XVECTOR (val
)->contents
[eol_type
], coding
);
4495 coding
->src_multibyte
= src_multibyte
;
4496 coding
->dst_multibyte
= dst_multibyte
;
4497 coding
->heading_ascii
= skip
;
4498 coding
->cmp_data
= cmp_data
;
4502 #define CONVERSION_BUFFER_EXTRA_ROOM 256
4504 #define DECODING_BUFFER_MAG(coding) \
4505 (coding->type == coding_type_iso2022 \
4507 : (coding->type == coding_type_ccl \
4508 ? coding->spec.ccl.decoder.buf_magnification \
4511 /* Return maximum size (bytes) of a buffer enough for decoding
4512 SRC_BYTES of text encoded in CODING. */
4515 decoding_buffer_size (coding
, src_bytes
)
4516 struct coding_system
*coding
;
4519 return (src_bytes
* DECODING_BUFFER_MAG (coding
)
4520 + CONVERSION_BUFFER_EXTRA_ROOM
);
4523 /* Return maximum size (bytes) of a buffer enough for encoding
4524 SRC_BYTES of text to CODING. */
4527 encoding_buffer_size (coding
, src_bytes
)
4528 struct coding_system
*coding
;
4533 if (coding
->type
== coding_type_ccl
)
4535 magnification
= coding
->spec
.ccl
.encoder
.buf_magnification
;
4536 if (coding
->eol_type
== CODING_EOL_CRLF
)
4539 else if (CODING_REQUIRE_ENCODING (coding
))
4544 return (src_bytes
* magnification
+ CONVERSION_BUFFER_EXTRA_ROOM
);
4547 /* Working buffer for code conversion. */
4548 struct conversion_buffer
4550 int size
; /* size of data. */
4551 int on_stack
; /* 1 if allocated by alloca. */
4552 unsigned char *data
;
4555 /* Allocate LEN bytes of memory for BUF (struct conversion_buffer). */
4556 #define allocate_conversion_buffer(buf, len) \
4558 if (len < MAX_ALLOCA) \
4560 buf.data = (unsigned char *) alloca (len); \
4565 buf.data = (unsigned char *) xmalloc (len); \
4571 /* Double the allocated memory for *BUF. */
4573 extend_conversion_buffer (buf
)
4574 struct conversion_buffer
*buf
;
4578 unsigned char *save
= buf
->data
;
4579 buf
->data
= (unsigned char *) xmalloc (buf
->size
* 2);
4580 bcopy (save
, buf
->data
, buf
->size
);
4585 buf
->data
= (unsigned char *) xrealloc (buf
->data
, buf
->size
* 2);
4590 /* Free the allocated memory for BUF if it is not on stack. */
4592 free_conversion_buffer (buf
)
4593 struct conversion_buffer
*buf
;
4600 ccl_coding_driver (coding
, source
, destination
, src_bytes
, dst_bytes
, encodep
)
4601 struct coding_system
*coding
;
4602 unsigned char *source
, *destination
;
4603 int src_bytes
, dst_bytes
, encodep
;
4605 struct ccl_program
*ccl
4606 = encodep
? &coding
->spec
.ccl
.encoder
: &coding
->spec
.ccl
.decoder
;
4607 unsigned char *dst
= destination
;
4609 ccl
->suppress_error
= coding
->suppress_error
;
4610 ccl
->last_block
= coding
->mode
& CODING_MODE_LAST_BLOCK
;
4613 /* On encoding, EOL format is converted within ccl_driver. For
4614 that, setup proper information in the structure CCL. */
4615 ccl
->eol_type
= coding
->eol_type
;
4616 if (ccl
->eol_type
==CODING_EOL_UNDECIDED
)
4617 ccl
->eol_type
= CODING_EOL_LF
;
4618 ccl
->cr_consumed
= coding
->spec
.ccl
.cr_carryover
;
4619 ccl
->eight_bit_control
= coding
->dst_multibyte
;
4622 ccl
->eight_bit_control
= 1;
4623 ccl
->multibyte
= coding
->src_multibyte
;
4624 if (coding
->spec
.ccl
.eight_bit_carryover
[0] != 0)
4626 /* Move carryover bytes to DESTINATION. */
4627 unsigned char *p
= coding
->spec
.ccl
.eight_bit_carryover
;
4630 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4632 dst_bytes
-= dst
- destination
;
4635 coding
->produced
= (ccl_driver (ccl
, source
, dst
, src_bytes
, dst_bytes
,
4636 &(coding
->consumed
))
4637 + dst
- destination
);
4641 coding
->produced_char
= coding
->produced
;
4642 coding
->spec
.ccl
.cr_carryover
= ccl
->cr_consumed
;
4644 else if (!ccl
->eight_bit_control
)
4646 /* The produced bytes forms a valid multibyte sequence. */
4647 coding
->produced_char
4648 = multibyte_chars_in_text (destination
, coding
->produced
);
4649 coding
->spec
.ccl
.eight_bit_carryover
[0] = 0;
4653 /* On decoding, the destination should always multibyte. But,
4654 CCL program might have been generated an invalid multibyte
4655 sequence. Here we make such a sequence valid as
4658 = dst_bytes
? dst_bytes
: source
+ coding
->consumed
- destination
;
4660 if ((coding
->consumed
< src_bytes
4661 || !ccl
->last_block
)
4662 && coding
->produced
>= 1
4663 && destination
[coding
->produced
- 1] >= 0x80)
4665 /* We should not convert the tailing 8-bit codes to
4666 multibyte form even if they doesn't form a valid
4667 multibyte sequence. They may form a valid sequence in
4671 if (destination
[coding
->produced
- 1] < 0xA0)
4673 else if (coding
->produced
>= 2)
4675 if (destination
[coding
->produced
- 2] >= 0x80)
4677 if (destination
[coding
->produced
- 2] < 0xA0)
4679 else if (coding
->produced
>= 3
4680 && destination
[coding
->produced
- 3] >= 0x80
4681 && destination
[coding
->produced
- 3] < 0xA0)
4687 BCOPY_SHORT (destination
+ coding
->produced
- carryover
,
4688 coding
->spec
.ccl
.eight_bit_carryover
,
4690 coding
->spec
.ccl
.eight_bit_carryover
[carryover
] = 0;
4691 coding
->produced
-= carryover
;
4694 coding
->produced
= str_as_multibyte (destination
, bytes
,
4696 &(coding
->produced_char
));
4699 switch (ccl
->status
)
4701 case CCL_STAT_SUSPEND_BY_SRC
:
4702 coding
->result
= CODING_FINISH_INSUFFICIENT_SRC
;
4704 case CCL_STAT_SUSPEND_BY_DST
:
4705 coding
->result
= CODING_FINISH_INSUFFICIENT_DST
;
4708 case CCL_STAT_INVALID_CMD
:
4709 coding
->result
= CODING_FINISH_INTERRUPT
;
4712 coding
->result
= CODING_FINISH_NORMAL
;
4715 return coding
->result
;
4718 /* Decode EOL format of the text at PTR of BYTES length destructively
4719 according to CODING->eol_type. This is called after the CCL
4720 program produced a decoded text at PTR. If we do CRLF->LF
4721 conversion, update CODING->produced and CODING->produced_char. */
4724 decode_eol_post_ccl (coding
, ptr
, bytes
)
4725 struct coding_system
*coding
;
4729 Lisp_Object val
, saved_coding_symbol
;
4730 unsigned char *pend
= ptr
+ bytes
;
4733 /* Remember the current coding system symbol. We set it back when
4734 an inconsistent EOL is found so that `last-coding-system-used' is
4735 set to the coding system that doesn't specify EOL conversion. */
4736 saved_coding_symbol
= coding
->symbol
;
4738 coding
->spec
.ccl
.cr_carryover
= 0;
4739 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
4741 /* Here, to avoid the call of setup_coding_system, we directly
4742 call detect_eol_type. */
4743 coding
->eol_type
= detect_eol_type (ptr
, bytes
, &dummy
);
4744 if (coding
->eol_type
== CODING_EOL_INCONSISTENT
)
4745 coding
->eol_type
= CODING_EOL_LF
;
4746 if (coding
->eol_type
!= CODING_EOL_UNDECIDED
)
4748 val
= Fget (coding
->symbol
, Qeol_type
);
4749 if (VECTORP (val
) && XVECTOR (val
)->size
== 3)
4750 coding
->symbol
= XVECTOR (val
)->contents
[coding
->eol_type
];
4752 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4755 if (coding
->eol_type
== CODING_EOL_LF
4756 || coding
->eol_type
== CODING_EOL_UNDECIDED
)
4758 /* We have nothing to do. */
4761 else if (coding
->eol_type
== CODING_EOL_CRLF
)
4763 unsigned char *pstart
= ptr
, *p
= ptr
;
4765 if (! (coding
->mode
& CODING_MODE_LAST_BLOCK
)
4766 && *(pend
- 1) == '\r')
4768 /* If the last character is CR, we can't handle it here
4769 because LF will be in the not-yet-decoded source text.
4770 Record that the CR is not yet processed. */
4771 coding
->spec
.ccl
.cr_carryover
= 1;
4773 coding
->produced_char
--;
4780 if (ptr
+ 1 < pend
&& *(ptr
+ 1) == '\n')
4787 if (coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4788 goto undo_eol_conversion
;
4792 else if (*ptr
== '\n'
4793 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4794 goto undo_eol_conversion
;
4799 undo_eol_conversion
:
4800 /* We have faced with inconsistent EOL format at PTR.
4801 Convert all LFs before PTR back to CRLFs. */
4802 for (p
--, ptr
--; p
>= pstart
; p
--)
4805 *ptr
-- = '\n', *ptr
-- = '\r';
4809 /* If carryover is recorded, cancel it because we don't
4810 convert CRLF anymore. */
4811 if (coding
->spec
.ccl
.cr_carryover
)
4813 coding
->spec
.ccl
.cr_carryover
= 0;
4815 coding
->produced_char
++;
4819 coding
->eol_type
= CODING_EOL_LF
;
4820 coding
->symbol
= saved_coding_symbol
;
4824 /* As each two-byte sequence CRLF was converted to LF, (PEND
4825 - P) is the number of deleted characters. */
4826 coding
->produced
-= pend
- p
;
4827 coding
->produced_char
-= pend
- p
;
4830 else /* i.e. coding->eol_type == CODING_EOL_CR */
4832 unsigned char *p
= ptr
;
4834 for (; ptr
< pend
; ptr
++)
4838 else if (*ptr
== '\n'
4839 && coding
->mode
& CODING_MODE_INHIBIT_INCONSISTENT_EOL
)
4841 for (; p
< ptr
; p
++)
4847 coding
->eol_type
= CODING_EOL_LF
;
4848 coding
->symbol
= saved_coding_symbol
;
4854 /* See "GENERAL NOTES about `decode_coding_XXX ()' functions". Before
4855 decoding, it may detect coding system and format of end-of-line if
4856 those are not yet decided. The source should be unibyte, the
4857 result is multibyte if CODING->dst_multibyte is nonzero, else
4861 decode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4862 struct coding_system
*coding
;
4863 const unsigned char *source
;
4864 unsigned char *destination
;
4865 int src_bytes
, dst_bytes
;
4869 if (coding
->type
== coding_type_undecided
)
4870 detect_coding (coding
, source
, src_bytes
);
4872 if (coding
->eol_type
== CODING_EOL_UNDECIDED
4873 && coding
->type
!= coding_type_ccl
)
4875 detect_eol (coding
, source
, src_bytes
);
4876 /* We had better recover the original eol format if we
4877 encounter an inconsistent eol format while decoding. */
4878 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
4881 coding
->produced
= coding
->produced_char
= 0;
4882 coding
->consumed
= coding
->consumed_char
= 0;
4884 coding
->result
= CODING_FINISH_NORMAL
;
4886 switch (coding
->type
)
4888 case coding_type_sjis
:
4889 decode_coding_sjis_big5 (coding
, source
, destination
,
4890 src_bytes
, dst_bytes
, 1);
4893 case coding_type_iso2022
:
4894 decode_coding_iso2022 (coding
, source
, destination
,
4895 src_bytes
, dst_bytes
);
4898 case coding_type_big5
:
4899 decode_coding_sjis_big5 (coding
, source
, destination
,
4900 src_bytes
, dst_bytes
, 0);
4903 case coding_type_emacs_mule
:
4904 decode_coding_emacs_mule (coding
, source
, destination
,
4905 src_bytes
, dst_bytes
);
4908 case coding_type_ccl
:
4909 if (coding
->spec
.ccl
.cr_carryover
)
4911 /* Put the CR which was not processed by the previous call
4912 of decode_eol_post_ccl in DESTINATION. It will be
4913 decoded together with the following LF by the call to
4914 decode_eol_post_ccl below. */
4915 *destination
= '\r';
4917 coding
->produced_char
++;
4919 extra
= coding
->spec
.ccl
.cr_carryover
;
4921 ccl_coding_driver (coding
, source
, destination
+ extra
,
4922 src_bytes
, dst_bytes
, 0);
4923 if (coding
->eol_type
!= CODING_EOL_LF
)
4925 coding
->produced
+= extra
;
4926 coding
->produced_char
+= extra
;
4927 decode_eol_post_ccl (coding
, destination
, coding
->produced
);
4932 decode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
4935 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
4936 && coding
->mode
& CODING_MODE_LAST_BLOCK
4937 && coding
->consumed
== src_bytes
)
4938 coding
->result
= CODING_FINISH_NORMAL
;
4940 if (coding
->mode
& CODING_MODE_LAST_BLOCK
4941 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
4943 const unsigned char *src
= source
+ coding
->consumed
;
4944 unsigned char *dst
= destination
+ coding
->produced
;
4946 src_bytes
-= coding
->consumed
;
4948 if (COMPOSING_P (coding
))
4949 DECODE_COMPOSITION_END ('1');
4953 dst
+= CHAR_STRING (c
, dst
);
4954 coding
->produced_char
++;
4956 coding
->consumed
= coding
->consumed_char
= src
- source
;
4957 coding
->produced
= dst
- destination
;
4958 coding
->result
= CODING_FINISH_NORMAL
;
4961 if (!coding
->dst_multibyte
)
4963 coding
->produced
= str_as_unibyte (destination
, coding
->produced
);
4964 coding
->produced_char
= coding
->produced
;
4967 return coding
->result
;
4970 /* See "GENERAL NOTES about `encode_coding_XXX ()' functions". The
4971 multibyteness of the source is CODING->src_multibyte, the
4972 multibyteness of the result is always unibyte. */
4975 encode_coding (coding
, source
, destination
, src_bytes
, dst_bytes
)
4976 struct coding_system
*coding
;
4977 const unsigned char *source
;
4978 unsigned char *destination
;
4979 int src_bytes
, dst_bytes
;
4981 coding
->produced
= coding
->produced_char
= 0;
4982 coding
->consumed
= coding
->consumed_char
= 0;
4984 coding
->result
= CODING_FINISH_NORMAL
;
4986 switch (coding
->type
)
4988 case coding_type_sjis
:
4989 encode_coding_sjis_big5 (coding
, source
, destination
,
4990 src_bytes
, dst_bytes
, 1);
4993 case coding_type_iso2022
:
4994 encode_coding_iso2022 (coding
, source
, destination
,
4995 src_bytes
, dst_bytes
);
4998 case coding_type_big5
:
4999 encode_coding_sjis_big5 (coding
, source
, destination
,
5000 src_bytes
, dst_bytes
, 0);
5003 case coding_type_emacs_mule
:
5004 encode_coding_emacs_mule (coding
, source
, destination
,
5005 src_bytes
, dst_bytes
);
5008 case coding_type_ccl
:
5009 ccl_coding_driver (coding
, source
, destination
,
5010 src_bytes
, dst_bytes
, 1);
5014 encode_eol (coding
, source
, destination
, src_bytes
, dst_bytes
);
5017 if (coding
->mode
& CODING_MODE_LAST_BLOCK
5018 && coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
)
5020 const unsigned char *src
= source
+ coding
->consumed
;
5021 unsigned char *dst
= destination
+ coding
->produced
;
5023 if (coding
->type
== coding_type_iso2022
)
5024 ENCODE_RESET_PLANE_AND_REGISTER
;
5025 if (COMPOSING_P (coding
))
5026 *dst
++ = ISO_CODE_ESC
, *dst
++ = '1';
5027 if (coding
->consumed
< src_bytes
)
5029 int len
= src_bytes
- coding
->consumed
;
5031 BCOPY_SHORT (src
, dst
, len
);
5032 if (coding
->src_multibyte
)
5033 len
= str_as_unibyte (dst
, len
);
5035 coding
->consumed
= src_bytes
;
5037 coding
->produced
= coding
->produced_char
= dst
- destination
;
5038 coding
->result
= CODING_FINISH_NORMAL
;
5041 if (coding
->result
== CODING_FINISH_INSUFFICIENT_SRC
5042 && coding
->consumed
== src_bytes
)
5043 coding
->result
= CODING_FINISH_NORMAL
;
5045 return coding
->result
;
5048 /* Scan text in the region between *BEG and *END (byte positions),
5049 skip characters which we don't have to decode by coding system
5050 CODING at the head and tail, then set *BEG and *END to the region
5051 of the text we actually have to convert. The caller should move
5052 the gap out of the region in advance if the region is from a
5055 If STR is not NULL, *BEG and *END are indices into STR. */
5058 shrink_decoding_region (beg
, end
, coding
, str
)
5060 struct coding_system
*coding
;
5063 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
, c
;
5065 Lisp_Object translation_table
;
5067 if (coding
->type
== coding_type_ccl
5068 || coding
->type
== coding_type_undecided
5069 || coding
->eol_type
!= CODING_EOL_LF
5070 || !NILP (coding
->post_read_conversion
)
5071 || coding
->composing
!= COMPOSITION_DISABLED
)
5073 /* We can't skip any data. */
5076 if (coding
->type
== coding_type_no_conversion
5077 || coding
->type
== coding_type_raw_text
5078 || coding
->type
== coding_type_emacs_mule
)
5080 /* We need no conversion, but don't have to skip any data here.
5081 Decoding routine handles them effectively anyway. */
5085 translation_table
= coding
->translation_table_for_decode
;
5086 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5087 translation_table
= Vstandard_translation_table_for_decode
;
5088 if (CHAR_TABLE_P (translation_table
))
5091 for (i
= 0; i
< 128; i
++)
5092 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5095 /* Some ASCII character should be translated. We give up
5100 if (coding
->heading_ascii
>= 0)
5101 /* Detection routine has already found how much we can skip at the
5103 *beg
+= coding
->heading_ascii
;
5107 begp_orig
= begp
= str
+ *beg
;
5108 endp_orig
= endp
= str
+ *end
;
5112 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5113 endp_orig
= endp
= begp
+ *end
- *beg
;
5116 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5117 || coding
->eol_type
== CODING_EOL_CRLF
);
5119 switch (coding
->type
)
5121 case coding_type_sjis
:
5122 case coding_type_big5
:
5123 /* We can skip all ASCII characters at the head. */
5124 if (coding
->heading_ascii
< 0)
5127 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\r') begp
++;
5129 while (begp
< endp
&& *begp
< 0x80) begp
++;
5131 /* We can skip all ASCII characters at the tail except for the
5132 second byte of SJIS or BIG5 code. */
5134 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\r') endp
--;
5136 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5137 /* Do not consider LF as ascii if preceded by CR, since that
5138 confuses eol decoding. */
5139 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5141 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] >= 0x80)
5145 case coding_type_iso2022
:
5146 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5147 /* We can't skip any data. */
5149 if (coding
->heading_ascii
< 0)
5151 /* We can skip all ASCII characters at the head except for a
5152 few control codes. */
5153 while (begp
< endp
&& (c
= *begp
) < 0x80
5154 && c
!= ISO_CODE_CR
&& c
!= ISO_CODE_SO
5155 && c
!= ISO_CODE_SI
&& c
!= ISO_CODE_ESC
5156 && (!eol_conversion
|| c
!= ISO_CODE_LF
))
5159 switch (coding
->category_idx
)
5161 case CODING_CATEGORY_IDX_ISO_8_1
:
5162 case CODING_CATEGORY_IDX_ISO_8_2
:
5163 /* We can skip all ASCII characters at the tail. */
5165 while (begp
< endp
&& (c
= endp
[-1]) < 0x80 && c
!= '\r') endp
--;
5167 while (begp
< endp
&& endp
[-1] < 0x80) endp
--;
5168 /* Do not consider LF as ascii if preceded by CR, since that
5169 confuses eol decoding. */
5170 if (begp
< endp
&& endp
< endp_orig
&& endp
[-1] == '\r' && endp
[0] == '\n')
5174 case CODING_CATEGORY_IDX_ISO_7
:
5175 case CODING_CATEGORY_IDX_ISO_7_TIGHT
:
5177 /* We can skip all characters at the tail except for 8-bit
5178 codes and ESC and the following 2-byte at the tail. */
5179 unsigned char *eight_bit
= NULL
;
5183 && (c
= endp
[-1]) != ISO_CODE_ESC
&& c
!= '\r')
5185 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5190 && (c
= endp
[-1]) != ISO_CODE_ESC
)
5192 if (!eight_bit
&& c
& 0x80) eight_bit
= endp
;
5195 /* Do not consider LF as ascii if preceded by CR, since that
5196 confuses eol decoding. */
5197 if (begp
< endp
&& endp
< endp_orig
5198 && endp
[-1] == '\r' && endp
[0] == '\n')
5200 if (begp
< endp
&& endp
[-1] == ISO_CODE_ESC
)
5202 if (endp
+ 1 < endp_orig
&& end
[0] == '(' && end
[1] == 'B')
5203 /* This is an ASCII designation sequence. We can
5204 surely skip the tail. But, if we have
5205 encountered an 8-bit code, skip only the codes
5207 endp
= eight_bit
? eight_bit
: endp
+ 2;
5209 /* Hmmm, we can't skip the tail. */
5221 *beg
+= begp
- begp_orig
;
5222 *end
+= endp
- endp_orig
;
5226 /* Like shrink_decoding_region but for encoding. */
5229 shrink_encoding_region (beg
, end
, coding
, str
)
5231 struct coding_system
*coding
;
5234 unsigned char *begp_orig
, *begp
, *endp_orig
, *endp
;
5236 Lisp_Object translation_table
;
5238 if (coding
->type
== coding_type_ccl
5239 || coding
->eol_type
== CODING_EOL_CRLF
5240 || coding
->eol_type
== CODING_EOL_CR
5241 || (coding
->cmp_data
&& coding
->cmp_data
->used
> 0))
5243 /* We can't skip any data. */
5246 if (coding
->type
== coding_type_no_conversion
5247 || coding
->type
== coding_type_raw_text
5248 || coding
->type
== coding_type_emacs_mule
5249 || coding
->type
== coding_type_undecided
)
5251 /* We need no conversion, but don't have to skip any data here.
5252 Encoding routine handles them effectively anyway. */
5256 translation_table
= coding
->translation_table_for_encode
;
5257 if (NILP (translation_table
) && !NILP (Venable_character_translation
))
5258 translation_table
= Vstandard_translation_table_for_encode
;
5259 if (CHAR_TABLE_P (translation_table
))
5262 for (i
= 0; i
< 128; i
++)
5263 if (!NILP (CHAR_TABLE_REF (translation_table
, i
)))
5266 /* Some ASCII character should be translated. We give up
5273 begp_orig
= begp
= str
+ *beg
;
5274 endp_orig
= endp
= str
+ *end
;
5278 begp_orig
= begp
= BYTE_POS_ADDR (*beg
);
5279 endp_orig
= endp
= begp
+ *end
- *beg
;
5282 eol_conversion
= (coding
->eol_type
== CODING_EOL_CR
5283 || coding
->eol_type
== CODING_EOL_CRLF
);
5285 /* Here, we don't have to check coding->pre_write_conversion because
5286 the caller is expected to have handled it already. */
5287 switch (coding
->type
)
5289 case coding_type_iso2022
:
5290 if (CODING_SPEC_ISO_INITIAL_DESIGNATION (coding
, 0) != CHARSET_ASCII
)
5291 /* We can't skip any data. */
5293 if (coding
->flags
& CODING_FLAG_ISO_DESIGNATE_AT_BOL
)
5295 unsigned char *bol
= begp
;
5296 while (begp
< endp
&& *begp
< 0x80)
5299 if (begp
[-1] == '\n')
5303 goto label_skip_tail
;
5307 case coding_type_sjis
:
5308 case coding_type_big5
:
5309 /* We can skip all ASCII characters at the head and tail. */
5311 while (begp
< endp
&& *begp
< 0x80 && *begp
!= '\n') begp
++;
5313 while (begp
< endp
&& *begp
< 0x80) begp
++;
5316 while (begp
< endp
&& endp
[-1] < 0x80 && endp
[-1] != '\n') endp
--;
5318 while (begp
< endp
&& *(endp
- 1) < 0x80) endp
--;
5325 *beg
+= begp
- begp_orig
;
5326 *end
+= endp
- endp_orig
;
5330 /* As shrinking conversion region requires some overhead, we don't try
5331 shrinking if the length of conversion region is less than this
5333 static int shrink_conversion_region_threshhold
= 1024;
5335 #define SHRINK_CONVERSION_REGION(beg, end, coding, str, encodep) \
5337 if (*(end) - *(beg) > shrink_conversion_region_threshhold) \
5339 if (encodep) shrink_encoding_region (beg, end, coding, str); \
5340 else shrink_decoding_region (beg, end, coding, str); \
5345 code_convert_region_unwind (arg
)
5348 inhibit_pre_post_conversion
= 0;
5349 Vlast_coding_system_used
= arg
;
5353 /* Store information about all compositions in the range FROM and TO
5354 of OBJ in memory blocks pointed by CODING->cmp_data. OBJ is a
5355 buffer or a string, defaults to the current buffer. */
5358 coding_save_composition (coding
, from
, to
, obj
)
5359 struct coding_system
*coding
;
5366 if (coding
->composing
== COMPOSITION_DISABLED
)
5368 if (!coding
->cmp_data
)
5369 coding_allocate_composition_data (coding
, from
);
5370 if (!find_composition (from
, to
, &start
, &end
, &prop
, obj
)
5374 && (!find_composition (end
, to
, &start
, &end
, &prop
, obj
)
5377 coding
->composing
= COMPOSITION_NO
;
5380 if (COMPOSITION_VALID_P (start
, end
, prop
))
5382 enum composition_method method
= COMPOSITION_METHOD (prop
);
5383 if (coding
->cmp_data
->used
+ COMPOSITION_DATA_MAX_BUNCH_LENGTH
5384 >= COMPOSITION_DATA_SIZE
)
5385 coding_allocate_composition_data (coding
, from
);
5386 /* For relative composition, we remember start and end
5387 positions, for the other compositions, we also remember
5389 CODING_ADD_COMPOSITION_START (coding
, start
- from
, method
);
5390 if (method
!= COMPOSITION_RELATIVE
)
5392 /* We must store a*/
5393 Lisp_Object val
, ch
;
5395 val
= COMPOSITION_COMPONENTS (prop
);
5399 ch
= XCAR (val
), val
= XCDR (val
);
5400 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5402 else if (VECTORP (val
) || STRINGP (val
))
5404 int len
= (VECTORP (val
)
5405 ? XVECTOR (val
)->size
: SCHARS (val
));
5407 for (i
= 0; i
< len
; i
++)
5410 ? Faref (val
, make_number (i
))
5411 : XVECTOR (val
)->contents
[i
]);
5412 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (ch
));
5415 else /* INTEGERP (val) */
5416 CODING_ADD_COMPOSITION_COMPONENT (coding
, XINT (val
));
5418 CODING_ADD_COMPOSITION_END (coding
, end
- from
);
5423 && find_composition (start
, to
, &start
, &end
, &prop
, obj
)
5426 /* Make coding->cmp_data point to the first memory block. */
5427 while (coding
->cmp_data
->prev
)
5428 coding
->cmp_data
= coding
->cmp_data
->prev
;
5429 coding
->cmp_data_start
= 0;
5432 /* Reflect the saved information about compositions to OBJ.
5433 CODING->cmp_data points to a memory block for the information. OBJ
5434 is a buffer or a string, defaults to the current buffer. */
5437 coding_restore_composition (coding
, obj
)
5438 struct coding_system
*coding
;
5441 struct composition_data
*cmp_data
= coding
->cmp_data
;
5446 while (cmp_data
->prev
)
5447 cmp_data
= cmp_data
->prev
;
5453 for (i
= 0; i
< cmp_data
->used
&& cmp_data
->data
[i
] > 0;
5454 i
+= cmp_data
->data
[i
])
5456 int *data
= cmp_data
->data
+ i
;
5457 enum composition_method method
= (enum composition_method
) data
[3];
5458 Lisp_Object components
;
5460 if (data
[0] < 0 || i
+ data
[0] > cmp_data
->used
)
5461 /* Invalid composition data. */
5464 if (method
== COMPOSITION_RELATIVE
)
5468 int len
= data
[0] - 4, j
;
5469 Lisp_Object args
[MAX_COMPOSITION_COMPONENTS
* 2 - 1];
5471 if (method
== COMPOSITION_WITH_RULE_ALTCHARS
5475 /* Invalid composition data. */
5477 for (j
= 0; j
< len
; j
++)
5478 args
[j
] = make_number (data
[4 + j
]);
5479 components
= (method
== COMPOSITION_WITH_ALTCHARS
5480 ? Fstring (len
, args
)
5481 : Fvector (len
, args
));
5483 compose_text (data
[1], data
[2], components
, Qnil
, obj
);
5485 cmp_data
= cmp_data
->next
;
5489 /* Decode (if ENCODEP is zero) or encode (if ENCODEP is nonzero) the
5490 text from FROM to TO (byte positions are FROM_BYTE and TO_BYTE) by
5491 coding system CODING, and return the status code of code conversion
5492 (currently, this value has no meaning).
5494 How many characters (and bytes) are converted to how many
5495 characters (and bytes) are recorded in members of the structure
5498 If REPLACE is nonzero, we do various things as if the original text
5499 is deleted and a new text is inserted. See the comments in
5500 replace_range (insdel.c) to know what we are doing.
5502 If REPLACE is zero, it is assumed that the source text is unibyte.
5503 Otherwise, it is assumed that the source text is multibyte. */
5506 code_convert_region (from
, from_byte
, to
, to_byte
, coding
, encodep
, replace
)
5507 int from
, from_byte
, to
, to_byte
, encodep
, replace
;
5508 struct coding_system
*coding
;
5510 int len
= to
- from
, len_byte
= to_byte
- from_byte
;
5511 int nchars_del
= 0, nbytes_del
= 0;
5512 int require
, inserted
, inserted_byte
;
5513 int head_skip
, tail_skip
, total_skip
= 0;
5514 Lisp_Object saved_coding_symbol
;
5516 unsigned char *src
, *dst
;
5517 Lisp_Object deletion
;
5518 int orig_point
= PT
, orig_len
= len
;
5520 int multibyte_p
= !NILP (current_buffer
->enable_multibyte_characters
);
5523 saved_coding_symbol
= coding
->symbol
;
5525 if (from
< PT
&& PT
< to
)
5527 TEMP_SET_PT_BOTH (from
, from_byte
);
5533 int saved_from
= from
;
5534 int saved_inhibit_modification_hooks
;
5536 prepare_to_modify_buffer (from
, to
, &from
);
5537 if (saved_from
!= from
)
5540 from_byte
= CHAR_TO_BYTE (from
), to_byte
= CHAR_TO_BYTE (to
);
5541 len_byte
= to_byte
- from_byte
;
5544 /* The code conversion routine can not preserve text properties
5545 for now. So, we must remove all text properties in the
5546 region. Here, we must suppress all modification hooks. */
5547 saved_inhibit_modification_hooks
= inhibit_modification_hooks
;
5548 inhibit_modification_hooks
= 1;
5549 Fset_text_properties (make_number (from
), make_number (to
), Qnil
, Qnil
);
5550 inhibit_modification_hooks
= saved_inhibit_modification_hooks
;
5553 if (! encodep
&& CODING_REQUIRE_DETECTION (coding
))
5555 /* We must detect encoding of text and eol format. */
5557 if (from
< GPT
&& to
> GPT
)
5558 move_gap_both (from
, from_byte
);
5559 if (coding
->type
== coding_type_undecided
)
5561 detect_coding (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5562 if (coding
->type
== coding_type_undecided
)
5564 /* It seems that the text contains only ASCII, but we
5565 should not leave it undecided because the deeper
5566 decoding routine (decode_coding) tries to detect the
5567 encodings again in vain. */
5568 coding
->type
= coding_type_emacs_mule
;
5569 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
5570 /* As emacs-mule decoder will handle composition, we
5571 need this setting to allocate coding->cmp_data
5573 coding
->composing
= COMPOSITION_NO
;
5576 if (coding
->eol_type
== CODING_EOL_UNDECIDED
5577 && coding
->type
!= coding_type_ccl
)
5579 detect_eol (coding
, BYTE_POS_ADDR (from_byte
), len_byte
);
5580 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
5581 coding
->eol_type
= CODING_EOL_LF
;
5582 /* We had better recover the original eol format if we
5583 encounter an inconsistent eol format while decoding. */
5584 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
5588 /* Now we convert the text. */
5590 /* For encoding, we must process pre-write-conversion in advance. */
5591 if (! inhibit_pre_post_conversion
5593 && SYMBOLP (coding
->pre_write_conversion
)
5594 && ! NILP (Ffboundp (coding
->pre_write_conversion
)))
5596 /* The function in pre-write-conversion may put a new text in a
5598 struct buffer
*prev
= current_buffer
;
5601 record_unwind_protect (code_convert_region_unwind
,
5602 Vlast_coding_system_used
);
5603 /* We should not call any more pre-write/post-read-conversion
5604 functions while this pre-write-conversion is running. */
5605 inhibit_pre_post_conversion
= 1;
5606 call2 (coding
->pre_write_conversion
,
5607 make_number (from
), make_number (to
));
5608 inhibit_pre_post_conversion
= 0;
5609 /* Discard the unwind protect. */
5612 if (current_buffer
!= prev
)
5615 new = Fcurrent_buffer ();
5616 set_buffer_internal_1 (prev
);
5617 del_range_2 (from
, from_byte
, to
, to_byte
, 0);
5618 TEMP_SET_PT_BOTH (from
, from_byte
);
5619 insert_from_buffer (XBUFFER (new), 1, len
, 0);
5621 if (orig_point
>= to
)
5622 orig_point
+= len
- orig_len
;
5623 else if (orig_point
> from
)
5627 from_byte
= CHAR_TO_BYTE (from
);
5628 to_byte
= CHAR_TO_BYTE (to
);
5629 len_byte
= to_byte
- from_byte
;
5630 TEMP_SET_PT_BOTH (from
, from_byte
);
5636 if (! EQ (current_buffer
->undo_list
, Qt
))
5637 deletion
= make_buffer_string_both (from
, from_byte
, to
, to_byte
, 1);
5640 nchars_del
= to
- from
;
5641 nbytes_del
= to_byte
- from_byte
;
5645 if (coding
->composing
!= COMPOSITION_DISABLED
)
5648 coding_save_composition (coding
, from
, to
, Fcurrent_buffer ());
5650 coding_allocate_composition_data (coding
, from
);
5653 /* Try to skip the heading and tailing ASCIIs. */
5654 if (coding
->type
!= coding_type_ccl
)
5656 int from_byte_orig
= from_byte
, to_byte_orig
= to_byte
;
5658 if (from
< GPT
&& GPT
< to
)
5659 move_gap_both (from
, from_byte
);
5660 SHRINK_CONVERSION_REGION (&from_byte
, &to_byte
, coding
, NULL
, encodep
);
5661 if (from_byte
== to_byte
5662 && (encodep
|| NILP (coding
->post_read_conversion
))
5663 && ! CODING_REQUIRE_FLUSHING (coding
))
5665 coding
->produced
= len_byte
;
5666 coding
->produced_char
= len
;
5668 /* We must record and adjust for this new text now. */
5669 adjust_after_insert (from
, from_byte_orig
, to
, to_byte_orig
, len
);
5673 head_skip
= from_byte
- from_byte_orig
;
5674 tail_skip
= to_byte_orig
- to_byte
;
5675 total_skip
= head_skip
+ tail_skip
;
5678 len
-= total_skip
; len_byte
-= total_skip
;
5681 /* For conversion, we must put the gap before the text in addition to
5682 making the gap larger for efficient decoding. The required gap
5683 size starts from 2000 which is the magic number used in make_gap.
5684 But, after one batch of conversion, it will be incremented if we
5685 find that it is not enough . */
5688 if (GAP_SIZE
< require
)
5689 make_gap (require
- GAP_SIZE
);
5690 move_gap_both (from
, from_byte
);
5692 inserted
= inserted_byte
= 0;
5694 GAP_SIZE
+= len_byte
;
5697 ZV_BYTE
-= len_byte
;
5700 if (GPT
- BEG
< BEG_UNCHANGED
)
5701 BEG_UNCHANGED
= GPT
- BEG
;
5702 if (Z
- GPT
< END_UNCHANGED
)
5703 END_UNCHANGED
= Z
- GPT
;
5705 if (!encodep
&& coding
->src_multibyte
)
5707 /* Decoding routines expects that the source text is unibyte.
5708 We must convert 8-bit characters of multibyte form to
5710 int len_byte_orig
= len_byte
;
5711 len_byte
= str_as_unibyte (GAP_END_ADDR
- len_byte
, len_byte
);
5712 if (len_byte
< len_byte_orig
)
5713 safe_bcopy (GAP_END_ADDR
- len_byte_orig
, GAP_END_ADDR
- len_byte
,
5715 coding
->src_multibyte
= 0;
5722 /* The buffer memory is now:
5723 +--------+converted-text+---------+-------original-text-------+---+
5724 |<-from->|<--inserted-->|---------|<--------len_byte--------->|---|
5725 |<---------------------- GAP ----------------------->| */
5726 src
= GAP_END_ADDR
- len_byte
;
5727 dst
= GPT_ADDR
+ inserted_byte
;
5730 result
= encode_coding (coding
, src
, dst
, len_byte
, 0);
5733 if (coding
->composing
!= COMPOSITION_DISABLED
)
5734 coding
->cmp_data
->char_offset
= from
+ inserted
;
5735 result
= decode_coding (coding
, src
, dst
, len_byte
, 0);
5738 /* The buffer memory is now:
5739 +--------+-------converted-text----+--+------original-text----+---+
5740 |<-from->|<-inserted->|<-produced->|--|<-(len_byte-consumed)->|---|
5741 |<---------------------- GAP ----------------------->| */
5743 inserted
+= coding
->produced_char
;
5744 inserted_byte
+= coding
->produced
;
5745 len_byte
-= coding
->consumed
;
5747 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
5749 coding_allocate_composition_data (coding
, from
+ inserted
);
5753 src
+= coding
->consumed
;
5754 dst
+= coding
->produced
;
5756 if (result
== CODING_FINISH_NORMAL
)
5761 if (! encodep
&& result
== CODING_FINISH_INCONSISTENT_EOL
)
5763 unsigned char *pend
= dst
, *p
= pend
- inserted_byte
;
5764 Lisp_Object eol_type
;
5766 /* Encode LFs back to the original eol format (CR or CRLF). */
5767 if (coding
->eol_type
== CODING_EOL_CR
)
5769 while (p
< pend
) if (*p
++ == '\n') p
[-1] = '\r';
5775 while (p
< pend
) if (*p
++ == '\n') count
++;
5776 if (src
- dst
< count
)
5778 /* We don't have sufficient room for encoding LFs
5779 back to CRLF. We must record converted and
5780 not-yet-converted text back to the buffer
5781 content, enlarge the gap, then record them out of
5782 the buffer contents again. */
5783 int add
= len_byte
+ inserted_byte
;
5786 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5787 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5788 make_gap (count
- GAP_SIZE
);
5790 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5791 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5792 /* Don't forget to update SRC, DST, and PEND. */
5793 src
= GAP_END_ADDR
- len_byte
;
5794 dst
= GPT_ADDR
+ inserted_byte
;
5798 inserted_byte
+= count
;
5799 coding
->produced
+= count
;
5800 p
= dst
= pend
+ count
;
5804 if (*p
== '\n') count
--, *--p
= '\r';
5808 /* Suppress eol-format conversion in the further conversion. */
5809 coding
->eol_type
= CODING_EOL_LF
;
5811 /* Set the coding system symbol to that for Unix-like EOL. */
5812 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
5813 if (VECTORP (eol_type
)
5814 && XVECTOR (eol_type
)->size
== 3
5815 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
5816 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
5818 coding
->symbol
= saved_coding_symbol
;
5824 if (coding
->type
!= coding_type_ccl
5825 || coding
->mode
& CODING_MODE_LAST_BLOCK
)
5827 coding
->mode
|= CODING_MODE_LAST_BLOCK
;
5830 if (result
== CODING_FINISH_INSUFFICIENT_SRC
)
5832 /* The source text ends in invalid codes. Let's just
5833 make them valid buffer contents, and finish conversion. */
5836 unsigned char *start
= dst
;
5838 inserted
+= len_byte
;
5842 dst
+= CHAR_STRING (c
, dst
);
5845 inserted_byte
+= dst
- start
;
5849 inserted
+= len_byte
;
5850 inserted_byte
+= len_byte
;
5856 if (result
== CODING_FINISH_INTERRUPT
)
5858 /* The conversion procedure was interrupted by a user. */
5861 /* Now RESULT == CODING_FINISH_INSUFFICIENT_DST */
5862 if (coding
->consumed
< 1)
5864 /* It's quite strange to require more memory without
5865 consuming any bytes. Perhaps CCL program bug. */
5870 /* We have just done the first batch of conversion which was
5871 stopped because of insufficient gap. Let's reconsider the
5872 required gap size (i.e. SRT - DST) now.
5874 We have converted ORIG bytes (== coding->consumed) into
5875 NEW bytes (coding->produced). To convert the remaining
5876 LEN bytes, we may need REQUIRE bytes of gap, where:
5877 REQUIRE + LEN_BYTE = LEN_BYTE * (NEW / ORIG)
5878 REQUIRE = LEN_BYTE * (NEW - ORIG) / ORIG
5879 Here, we are sure that NEW >= ORIG. */
5882 if (coding
->produced
<= coding
->consumed
)
5884 /* This happens because of CCL-based coding system with
5890 ratio
= (coding
->produced
- coding
->consumed
) / coding
->consumed
;
5891 require
= len_byte
* ratio
;
5895 if ((src
- dst
) < (require
+ 2000))
5897 /* See the comment above the previous call of make_gap. */
5898 int add
= len_byte
+ inserted_byte
;
5901 ZV
+= add
; Z
+= add
; ZV_BYTE
+= add
; Z_BYTE
+= add
;
5902 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5903 make_gap (require
+ 2000);
5905 ZV
-= add
; Z
-= add
; ZV_BYTE
-= add
; Z_BYTE
-= add
;
5906 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5909 if (src
- dst
> 0) *dst
= 0; /* Put an anchor. */
5911 if (encodep
&& coding
->dst_multibyte
)
5913 /* The output is unibyte. We must convert 8-bit characters to
5915 if (inserted_byte
* 2 > GAP_SIZE
)
5917 GAP_SIZE
-= inserted_byte
;
5918 ZV
+= inserted_byte
; Z
+= inserted_byte
;
5919 ZV_BYTE
+= inserted_byte
; Z_BYTE
+= inserted_byte
;
5920 GPT
+= inserted_byte
; GPT_BYTE
+= inserted_byte
;
5921 make_gap (inserted_byte
- GAP_SIZE
);
5922 GAP_SIZE
+= inserted_byte
;
5923 ZV
-= inserted_byte
; Z
-= inserted_byte
;
5924 ZV_BYTE
-= inserted_byte
; Z_BYTE
-= inserted_byte
;
5925 GPT
-= inserted_byte
; GPT_BYTE
-= inserted_byte
;
5927 inserted_byte
= str_to_multibyte (GPT_ADDR
, GAP_SIZE
, inserted_byte
);
5930 /* If we shrank the conversion area, adjust it now. */
5934 safe_bcopy (GAP_END_ADDR
, GPT_ADDR
+ inserted_byte
, tail_skip
);
5935 inserted
+= total_skip
; inserted_byte
+= total_skip
;
5936 GAP_SIZE
+= total_skip
;
5937 GPT
-= head_skip
; GPT_BYTE
-= head_skip
;
5938 ZV
-= total_skip
; ZV_BYTE
-= total_skip
;
5939 Z
-= total_skip
; Z_BYTE
-= total_skip
;
5940 from
-= head_skip
; from_byte
-= head_skip
;
5941 to
+= tail_skip
; to_byte
+= tail_skip
;
5945 if (! EQ (current_buffer
->undo_list
, Qt
))
5946 adjust_after_replace (from
, from_byte
, deletion
, inserted
, inserted_byte
);
5948 adjust_after_replace_noundo (from
, from_byte
, nchars_del
, nbytes_del
,
5949 inserted
, inserted_byte
);
5950 inserted
= Z
- prev_Z
;
5952 if (!encodep
&& coding
->cmp_data
&& coding
->cmp_data
->used
)
5953 coding_restore_composition (coding
, Fcurrent_buffer ());
5954 coding_free_composition_data (coding
);
5956 if (! inhibit_pre_post_conversion
5957 && ! encodep
&& ! NILP (coding
->post_read_conversion
))
5960 Lisp_Object saved_coding_system
;
5963 TEMP_SET_PT_BOTH (from
, from_byte
);
5965 record_unwind_protect (code_convert_region_unwind
,
5966 Vlast_coding_system_used
);
5967 saved_coding_system
= Vlast_coding_system_used
;
5968 Vlast_coding_system_used
= coding
->symbol
;
5969 /* We should not call any more pre-write/post-read-conversion
5970 functions while this post-read-conversion is running. */
5971 inhibit_pre_post_conversion
= 1;
5972 val
= call1 (coding
->post_read_conversion
, make_number (inserted
));
5973 inhibit_pre_post_conversion
= 0;
5974 coding
->symbol
= Vlast_coding_system_used
;
5975 Vlast_coding_system_used
= saved_coding_system
;
5976 /* Discard the unwind protect. */
5979 inserted
+= Z
- prev_Z
;
5982 if (orig_point
>= from
)
5984 if (orig_point
>= from
+ orig_len
)
5985 orig_point
+= inserted
- orig_len
;
5988 TEMP_SET_PT (orig_point
);
5993 signal_after_change (from
, to
- from
, inserted
);
5994 update_compositions (from
, from
+ inserted
, CHECK_BORDER
);
5998 coding
->consumed
= to_byte
- from_byte
;
5999 coding
->consumed_char
= to
- from
;
6000 coding
->produced
= inserted_byte
;
6001 coding
->produced_char
= inserted
;
6008 run_pre_post_conversion_on_str (str
, coding
, encodep
)
6010 struct coding_system
*coding
;
6013 int count
= SPECPDL_INDEX ();
6014 struct gcpro gcpro1
, gcpro2
;
6015 int multibyte
= STRING_MULTIBYTE (str
);
6018 Lisp_Object old_deactivate_mark
;
6020 record_unwind_protect (Fset_buffer
, Fcurrent_buffer ());
6021 record_unwind_protect (code_convert_region_unwind
,
6022 Vlast_coding_system_used
);
6023 /* It is not crucial to specbind this. */
6024 old_deactivate_mark
= Vdeactivate_mark
;
6025 GCPRO2 (str
, old_deactivate_mark
);
6027 buffer
= Fget_buffer_create (build_string (" *code-converting-work*"));
6028 buf
= XBUFFER (buffer
);
6030 delete_all_overlays (buf
);
6031 buf
->directory
= current_buffer
->directory
;
6032 buf
->read_only
= Qnil
;
6033 buf
->filename
= Qnil
;
6034 buf
->undo_list
= Qt
;
6035 eassert (buf
->overlays_before
== NULL
);
6036 eassert (buf
->overlays_after
== NULL
);
6038 set_buffer_internal (buf
);
6039 /* We must insert the contents of STR as is without
6040 unibyte<->multibyte conversion. For that, we adjust the
6041 multibyteness of the working buffer to that of STR. */
6043 buf
->enable_multibyte_characters
= multibyte
? Qt
: Qnil
;
6045 insert_from_string (str
, 0, 0,
6046 SCHARS (str
), SBYTES (str
), 0);
6048 inhibit_pre_post_conversion
= 1;
6050 call2 (coding
->pre_write_conversion
, make_number (BEG
), make_number (Z
));
6053 Vlast_coding_system_used
= coding
->symbol
;
6054 TEMP_SET_PT_BOTH (BEG
, BEG_BYTE
);
6055 call1 (coding
->post_read_conversion
, make_number (Z
- BEG
));
6056 coding
->symbol
= Vlast_coding_system_used
;
6058 inhibit_pre_post_conversion
= 0;
6059 Vdeactivate_mark
= old_deactivate_mark
;
6060 str
= make_buffer_string (BEG
, Z
, 1);
6061 return unbind_to (count
, str
);
6065 decode_coding_string (str
, coding
, nocopy
)
6067 struct coding_system
*coding
;
6071 struct conversion_buffer buf
;
6073 Lisp_Object saved_coding_symbol
;
6075 int require_decoding
;
6076 int shrinked_bytes
= 0;
6078 int consumed
, consumed_char
, produced
, produced_char
;
6081 to_byte
= SBYTES (str
);
6083 saved_coding_symbol
= coding
->symbol
;
6084 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6085 coding
->dst_multibyte
= 1;
6086 if (CODING_REQUIRE_DETECTION (coding
))
6088 /* See the comments in code_convert_region. */
6089 if (coding
->type
== coding_type_undecided
)
6091 detect_coding (coding
, SDATA (str
), to_byte
);
6092 if (coding
->type
== coding_type_undecided
)
6094 coding
->type
= coding_type_emacs_mule
;
6095 coding
->category_idx
= CODING_CATEGORY_IDX_EMACS_MULE
;
6096 /* As emacs-mule decoder will handle composition, we
6097 need this setting to allocate coding->cmp_data
6099 coding
->composing
= COMPOSITION_NO
;
6102 if (coding
->eol_type
== CODING_EOL_UNDECIDED
6103 && coding
->type
!= coding_type_ccl
)
6105 saved_coding_symbol
= coding
->symbol
;
6106 detect_eol (coding
, SDATA (str
), to_byte
);
6107 if (coding
->eol_type
== CODING_EOL_UNDECIDED
)
6108 coding
->eol_type
= CODING_EOL_LF
;
6109 /* We had better recover the original eol format if we
6110 encounter an inconsistent eol format while decoding. */
6111 coding
->mode
|= CODING_MODE_INHIBIT_INCONSISTENT_EOL
;
6115 if (coding
->type
== coding_type_no_conversion
6116 || coding
->type
== coding_type_raw_text
)
6117 coding
->dst_multibyte
= 0;
6119 require_decoding
= CODING_REQUIRE_DECODING (coding
);
6121 if (STRING_MULTIBYTE (str
))
6123 /* Decoding routines expect the source text to be unibyte. */
6124 str
= Fstring_as_unibyte (str
);
6125 to_byte
= SBYTES (str
);
6127 coding
->src_multibyte
= 0;
6130 /* Try to skip the heading and tailing ASCIIs. */
6131 if (require_decoding
&& coding
->type
!= coding_type_ccl
)
6133 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6135 if (from
== to_byte
)
6136 require_decoding
= 0;
6137 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6140 if (!require_decoding
6141 && !(SYMBOLP (coding
->post_read_conversion
)
6142 && !NILP (Ffboundp (coding
->post_read_conversion
))))
6144 coding
->consumed
= SBYTES (str
);
6145 coding
->consumed_char
= SCHARS (str
);
6146 if (coding
->dst_multibyte
)
6148 str
= Fstring_as_multibyte (str
);
6151 coding
->produced
= SBYTES (str
);
6152 coding
->produced_char
= SCHARS (str
);
6153 return (nocopy
? str
: Fcopy_sequence (str
));
6156 if (coding
->composing
!= COMPOSITION_DISABLED
)
6157 coding_allocate_composition_data (coding
, from
);
6158 len
= decoding_buffer_size (coding
, to_byte
- from
);
6159 allocate_conversion_buffer (buf
, len
);
6161 consumed
= consumed_char
= produced
= produced_char
= 0;
6164 result
= decode_coding (coding
, SDATA (str
) + from
+ consumed
,
6165 buf
.data
+ produced
, to_byte
- from
- consumed
,
6166 buf
.size
- produced
);
6167 consumed
+= coding
->consumed
;
6168 consumed_char
+= coding
->consumed_char
;
6169 produced
+= coding
->produced
;
6170 produced_char
+= coding
->produced_char
;
6171 if (result
== CODING_FINISH_NORMAL
6172 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6173 && coding
->consumed
== 0))
6175 if (result
== CODING_FINISH_INSUFFICIENT_CMP
)
6176 coding_allocate_composition_data (coding
, from
+ produced_char
);
6177 else if (result
== CODING_FINISH_INSUFFICIENT_DST
)
6178 extend_conversion_buffer (&buf
);
6179 else if (result
== CODING_FINISH_INCONSISTENT_EOL
)
6181 Lisp_Object eol_type
;
6183 /* Recover the original EOL format. */
6184 if (coding
->eol_type
== CODING_EOL_CR
)
6187 for (p
= buf
.data
; p
< buf
.data
+ produced
; p
++)
6188 if (*p
== '\n') *p
= '\r';
6190 else if (coding
->eol_type
== CODING_EOL_CRLF
)
6193 unsigned char *p0
, *p1
;
6194 for (p0
= buf
.data
, p1
= p0
+ produced
; p0
< p1
; p0
++)
6195 if (*p0
== '\n') num_eol
++;
6196 if (produced
+ num_eol
>= buf
.size
)
6197 extend_conversion_buffer (&buf
);
6198 for (p0
= buf
.data
+ produced
, p1
= p0
+ num_eol
; p0
> buf
.data
;)
6201 if (*p0
== '\n') *--p1
= '\r';
6203 produced
+= num_eol
;
6204 produced_char
+= num_eol
;
6206 /* Suppress eol-format conversion in the further conversion. */
6207 coding
->eol_type
= CODING_EOL_LF
;
6209 /* Set the coding system symbol to that for Unix-like EOL. */
6210 eol_type
= Fget (saved_coding_symbol
, Qeol_type
);
6211 if (VECTORP (eol_type
)
6212 && XVECTOR (eol_type
)->size
== 3
6213 && SYMBOLP (XVECTOR (eol_type
)->contents
[CODING_EOL_LF
]))
6214 coding
->symbol
= XVECTOR (eol_type
)->contents
[CODING_EOL_LF
];
6216 coding
->symbol
= saved_coding_symbol
;
6222 coding
->consumed
= consumed
;
6223 coding
->consumed_char
= consumed_char
;
6224 coding
->produced
= produced
;
6225 coding
->produced_char
= produced_char
;
6227 if (coding
->dst_multibyte
)
6228 newstr
= make_uninit_multibyte_string (produced_char
+ shrinked_bytes
,
6229 produced
+ shrinked_bytes
);
6231 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6233 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6234 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6235 if (shrinked_bytes
> from
)
6236 STRING_COPYIN (newstr
, from
+ produced
,
6237 SDATA (str
) + to_byte
,
6238 shrinked_bytes
- from
);
6239 free_conversion_buffer (&buf
);
6241 coding
->consumed
+= shrinked_bytes
;
6242 coding
->consumed_char
+= shrinked_bytes
;
6243 coding
->produced
+= shrinked_bytes
;
6244 coding
->produced_char
+= shrinked_bytes
;
6246 if (coding
->cmp_data
&& coding
->cmp_data
->used
)
6247 coding_restore_composition (coding
, newstr
);
6248 coding_free_composition_data (coding
);
6250 if (SYMBOLP (coding
->post_read_conversion
)
6251 && !NILP (Ffboundp (coding
->post_read_conversion
)))
6252 newstr
= run_pre_post_conversion_on_str (newstr
, coding
, 0);
6258 encode_coding_string (str
, coding
, nocopy
)
6260 struct coding_system
*coding
;
6264 struct conversion_buffer buf
;
6265 int from
, to
, to_byte
;
6267 int shrinked_bytes
= 0;
6269 int consumed
, consumed_char
, produced
, produced_char
;
6271 if (SYMBOLP (coding
->pre_write_conversion
)
6272 && !NILP (Ffboundp (coding
->pre_write_conversion
)))
6273 str
= run_pre_post_conversion_on_str (str
, coding
, 1);
6277 to_byte
= SBYTES (str
);
6279 /* Encoding routines determine the multibyteness of the source text
6280 by coding->src_multibyte. */
6281 coding
->src_multibyte
= STRING_MULTIBYTE (str
);
6282 coding
->dst_multibyte
= 0;
6283 if (! CODING_REQUIRE_ENCODING (coding
))
6285 coding
->consumed
= SBYTES (str
);
6286 coding
->consumed_char
= SCHARS (str
);
6287 if (STRING_MULTIBYTE (str
))
6289 str
= Fstring_as_unibyte (str
);
6292 coding
->produced
= SBYTES (str
);
6293 coding
->produced_char
= SCHARS (str
);
6294 return (nocopy
? str
: Fcopy_sequence (str
));
6297 if (coding
->composing
!= COMPOSITION_DISABLED
)
6298 coding_save_composition (coding
, from
, to
, str
);
6300 /* Try to skip the heading and tailing ASCIIs. */
6301 if (coding
->type
!= coding_type_ccl
)
6303 SHRINK_CONVERSION_REGION (&from
, &to_byte
, coding
, SDATA (str
),
6305 if (from
== to_byte
)
6306 return (nocopy
? str
: Fcopy_sequence (str
));
6307 shrinked_bytes
= from
+ (SBYTES (str
) - to_byte
);
6310 len
= encoding_buffer_size (coding
, to_byte
- from
);
6311 allocate_conversion_buffer (buf
, len
);
6313 consumed
= consumed_char
= produced
= produced_char
= 0;
6316 result
= encode_coding (coding
, SDATA (str
) + from
+ consumed
,
6317 buf
.data
+ produced
, to_byte
- from
- consumed
,
6318 buf
.size
- produced
);
6319 consumed
+= coding
->consumed
;
6320 consumed_char
+= coding
->consumed_char
;
6321 produced
+= coding
->produced
;
6322 produced_char
+= coding
->produced_char
;
6323 if (result
== CODING_FINISH_NORMAL
6324 || result
== CODING_FINISH_INTERRUPT
6325 || (result
== CODING_FINISH_INSUFFICIENT_SRC
6326 && coding
->consumed
== 0))
6328 /* Now result should be CODING_FINISH_INSUFFICIENT_DST. */
6329 extend_conversion_buffer (&buf
);
6332 coding
->consumed
= consumed
;
6333 coding
->consumed_char
= consumed_char
;
6334 coding
->produced
= produced
;
6335 coding
->produced_char
= produced_char
;
6337 newstr
= make_uninit_string (produced
+ shrinked_bytes
);
6339 STRING_COPYIN (newstr
, 0, SDATA (str
), from
);
6340 STRING_COPYIN (newstr
, from
, buf
.data
, produced
);
6341 if (shrinked_bytes
> from
)
6342 STRING_COPYIN (newstr
, from
+ produced
,
6343 SDATA (str
) + to_byte
,
6344 shrinked_bytes
- from
);
6346 free_conversion_buffer (&buf
);
6347 coding_free_composition_data (coding
);
6354 /*** 8. Emacs Lisp library functions ***/
6356 DEFUN ("coding-system-p", Fcoding_system_p
, Scoding_system_p
, 1, 1, 0,
6357 doc
: /* Return t if OBJECT is nil or a coding-system.
6358 See the documentation of `make-coding-system' for information
6359 about coding-system objects. */)
6367 if (! NILP (Fget (obj
, Qcoding_system_define_form
)))
6369 /* Get coding-spec vector for OBJ. */
6370 obj
= Fget (obj
, Qcoding_system
);
6371 return ((VECTORP (obj
) && XVECTOR (obj
)->size
== 5)
6375 DEFUN ("read-non-nil-coding-system", Fread_non_nil_coding_system
,
6376 Sread_non_nil_coding_system
, 1, 1, 0,
6377 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT. */)
6384 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6385 Qt
, Qnil
, Qcoding_system_history
, Qnil
, Qnil
);
6387 while (SCHARS (val
) == 0);
6388 return (Fintern (val
, Qnil
));
6391 DEFUN ("read-coding-system", Fread_coding_system
, Sread_coding_system
, 1, 2, 0,
6392 doc
: /* Read a coding system from the minibuffer, prompting with string PROMPT.
6393 If the user enters null input, return second argument DEFAULT-CODING-SYSTEM. */)
6394 (prompt
, default_coding_system
)
6395 Lisp_Object prompt
, default_coding_system
;
6398 if (SYMBOLP (default_coding_system
))
6399 default_coding_system
= SYMBOL_NAME (default_coding_system
);
6400 val
= Fcompleting_read (prompt
, Vcoding_system_alist
, Qnil
,
6401 Qt
, Qnil
, Qcoding_system_history
,
6402 default_coding_system
, Qnil
);
6403 return (SCHARS (val
) == 0 ? Qnil
: Fintern (val
, Qnil
));
6406 DEFUN ("check-coding-system", Fcheck_coding_system
, Scheck_coding_system
,
6408 doc
: /* Check validity of CODING-SYSTEM.
6409 If valid, return CODING-SYSTEM, else signal a `coding-system-error' error.
6410 It is valid if it is nil or a symbol with a non-nil `coding-system' property.
6411 The value of this property should be a vector of length 5. */)
6413 Lisp_Object coding_system
;
6415 Lisp_Object define_form
;
6417 define_form
= Fget (coding_system
, Qcoding_system_define_form
);
6418 if (! NILP (define_form
))
6420 Fput (coding_system
, Qcoding_system_define_form
, Qnil
);
6421 safe_eval (define_form
);
6423 if (!NILP (Fcoding_system_p (coding_system
)))
6424 return coding_system
;
6426 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
6430 detect_coding_system (src
, src_bytes
, highest
, multibytep
)
6431 const unsigned char *src
;
6432 int src_bytes
, highest
;
6435 int coding_mask
, eol_type
;
6436 Lisp_Object val
, tmp
;
6439 coding_mask
= detect_coding_mask (src
, src_bytes
, NULL
, &dummy
, multibytep
);
6440 eol_type
= detect_eol_type (src
, src_bytes
, &dummy
);
6441 if (eol_type
== CODING_EOL_INCONSISTENT
)
6442 eol_type
= CODING_EOL_UNDECIDED
;
6447 if (eol_type
!= CODING_EOL_UNDECIDED
)
6450 val2
= Fget (Qundecided
, Qeol_type
);
6452 val
= XVECTOR (val2
)->contents
[eol_type
];
6454 return (highest
? val
: Fcons (val
, Qnil
));
6457 /* At first, gather possible coding systems in VAL. */
6459 for (tmp
= Vcoding_category_list
; CONSP (tmp
); tmp
= XCDR (tmp
))
6461 Lisp_Object category_val
, category_index
;
6463 category_index
= Fget (XCAR (tmp
), Qcoding_category_index
);
6464 category_val
= Fsymbol_value (XCAR (tmp
));
6465 if (!NILP (category_val
)
6466 && NATNUMP (category_index
)
6467 && (coding_mask
& (1 << XFASTINT (category_index
))))
6469 val
= Fcons (category_val
, val
);
6475 val
= Fnreverse (val
);
6477 /* Then, replace the elements with subsidiary coding systems. */
6478 for (tmp
= val
; CONSP (tmp
); tmp
= XCDR (tmp
))
6480 if (eol_type
!= CODING_EOL_UNDECIDED
6481 && eol_type
!= CODING_EOL_INCONSISTENT
)
6484 eol
= Fget (XCAR (tmp
), Qeol_type
);
6486 XSETCAR (tmp
, XVECTOR (eol
)->contents
[eol_type
]);
6489 return (highest
? XCAR (val
) : val
);
6492 DEFUN ("detect-coding-region", Fdetect_coding_region
, Sdetect_coding_region
,
6494 doc
: /* Detect how the byte sequence in the region is encoded.
6495 Return a list of possible coding systems used on decoding a byte
6496 sequence containing the bytes in the region between START and END when
6497 the coding system `undecided' is specified. The list is ordered by
6498 priority decided in the current language environment.
6500 If only ASCII characters are found, it returns a list of single element
6501 `undecided' or its subsidiary coding system according to a detected
6504 If optional argument HIGHEST is non-nil, return the coding system of
6505 highest priority. */)
6506 (start
, end
, highest
)
6507 Lisp_Object start
, end
, highest
;
6510 int from_byte
, to_byte
;
6511 int include_anchor_byte
= 0;
6513 CHECK_NUMBER_COERCE_MARKER (start
);
6514 CHECK_NUMBER_COERCE_MARKER (end
);
6516 validate_region (&start
, &end
);
6517 from
= XINT (start
), to
= XINT (end
);
6518 from_byte
= CHAR_TO_BYTE (from
);
6519 to_byte
= CHAR_TO_BYTE (to
);
6521 if (from
< GPT
&& to
>= GPT
)
6522 move_gap_both (to
, to_byte
);
6523 /* If we an anchor byte `\0' follows the region, we include it in
6524 the detecting source. Then code detectors can handle the tailing
6525 byte sequence more accurately.
6527 Fix me: This is not a perfect solution. It is better that we
6528 add one more argument, say LAST_BLOCK, to all detect_coding_XXX.
6530 if (to
== Z
|| (to
== GPT
&& GAP_SIZE
> 0))
6531 include_anchor_byte
= 1;
6532 return detect_coding_system (BYTE_POS_ADDR (from_byte
),
6533 to_byte
- from_byte
+ include_anchor_byte
,
6535 !NILP (current_buffer
6536 ->enable_multibyte_characters
));
6539 DEFUN ("detect-coding-string", Fdetect_coding_string
, Sdetect_coding_string
,
6541 doc
: /* Detect how the byte sequence in STRING is encoded.
6542 Return a list of possible coding systems used on decoding a byte
6543 sequence containing the bytes in STRING when the coding system
6544 `undecided' is specified. The list is ordered by priority decided in
6545 the current language environment.
6547 If only ASCII characters are found, it returns a list of single element
6548 `undecided' or its subsidiary coding system according to a detected
6551 If optional argument HIGHEST is non-nil, return the coding system of
6552 highest priority. */)
6554 Lisp_Object string
, highest
;
6556 CHECK_STRING (string
);
6558 return detect_coding_system (SDATA (string
),
6559 /* "+ 1" is to include the anchor byte
6560 `\0'. With this, code detectors can
6561 handle the tailing bytes more
6563 SBYTES (string
) + 1,
6565 STRING_MULTIBYTE (string
));
6568 /* Subroutine for Fsafe_coding_systems_region_internal.
6570 Return a list of coding systems that safely encode the multibyte
6571 text between P and PEND. SAFE_CODINGS, if non-nil, is an alist of
6572 possible coding systems. If it is nil, it means that we have not
6573 yet found any coding systems.
6575 WORK_TABLE a char-table of which element is set to t once the
6576 element is looked up.
6578 If a non-ASCII single byte char is found, set
6579 *single_byte_char_found to 1. */
6582 find_safe_codings (p
, pend
, safe_codings
, work_table
, single_byte_char_found
)
6583 unsigned char *p
, *pend
;
6584 Lisp_Object safe_codings
, work_table
;
6585 int *single_byte_char_found
;
6588 Lisp_Object val
, ch
;
6589 Lisp_Object prev
, tail
;
6591 if (NILP (safe_codings
))
6592 goto done_safe_codings
;
6595 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6597 if (ASCII_BYTE_P (c
))
6598 /* We can ignore ASCII characters here. */
6600 if (SINGLE_BYTE_CHAR_P (c
))
6601 *single_byte_char_found
= 1;
6602 /* Check the safe coding systems for C. */
6603 ch
= make_number (c
);
6604 val
= Faref (work_table
, ch
);
6606 /* This element was already checked. Ignore it. */
6608 /* Remember that we checked this element. */
6609 Faset (work_table
, ch
, Qt
);
6611 for (prev
= tail
= safe_codings
; CONSP (tail
); tail
= XCDR (tail
))
6613 Lisp_Object elt
, translation_table
, hash_table
, accept_latin_extra
;
6617 if (CONSP (XCDR (elt
)))
6619 /* This entry has this format now:
6620 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6621 ACCEPT-LATIN-EXTRA ) */
6623 encodable
= ! NILP (Faref (XCAR (val
), ch
));
6627 translation_table
= XCAR (val
);
6628 hash_table
= XCAR (XCDR (val
));
6629 accept_latin_extra
= XCAR (XCDR (XCDR (val
)));
6634 /* This entry has this format now: ( CODING . SAFE-CHARS) */
6635 encodable
= ! NILP (Faref (XCDR (elt
), ch
));
6638 /* Transform the format to:
6639 ( CODING SAFE-CHARS TRANSLATION-TABLE HASH-TABLE
6640 ACCEPT-LATIN-EXTRA ) */
6641 val
= Fget (XCAR (elt
), Qcoding_system
);
6643 = Fplist_get (AREF (val
, 3),
6644 Qtranslation_table_for_encode
);
6645 if (SYMBOLP (translation_table
))
6646 translation_table
= Fget (translation_table
,
6647 Qtranslation_table
);
6649 = (CHAR_TABLE_P (translation_table
)
6650 ? XCHAR_TABLE (translation_table
)->extras
[1]
6653 = ((EQ (AREF (val
, 0), make_number (2))
6654 && VECTORP (AREF (val
, 4)))
6655 ? AREF (AREF (val
, 4), 16)
6657 XSETCAR (tail
, list5 (XCAR (elt
), XCDR (elt
),
6658 translation_table
, hash_table
,
6659 accept_latin_extra
));
6664 && ((CHAR_TABLE_P (translation_table
)
6665 && ! NILP (Faref (translation_table
, ch
)))
6666 || (HASH_TABLE_P (hash_table
)
6667 && ! NILP (Fgethash (ch
, hash_table
, Qnil
)))
6668 || (SINGLE_BYTE_CHAR_P (c
)
6669 && ! NILP (accept_latin_extra
)
6670 && VECTORP (Vlatin_extra_code_table
)
6671 && ! NILP (AREF (Vlatin_extra_code_table
, c
)))))
6677 /* Exclude this coding system from SAFE_CODINGS. */
6678 if (EQ (tail
, safe_codings
))
6680 safe_codings
= XCDR (safe_codings
);
6681 if (NILP (safe_codings
))
6682 goto done_safe_codings
;
6685 XSETCDR (prev
, XCDR (tail
));
6691 /* If the above loop was terminated before P reaches PEND, it means
6692 SAFE_CODINGS was set to nil. If we have not yet found an
6693 non-ASCII single-byte char, check it now. */
6694 if (! *single_byte_char_found
)
6697 c
= STRING_CHAR_AND_LENGTH (p
, pend
- p
, len
);
6699 if (! ASCII_BYTE_P (c
)
6700 && SINGLE_BYTE_CHAR_P (c
))
6702 *single_byte_char_found
= 1;
6706 return safe_codings
;
6709 DEFUN ("find-coding-systems-region-internal",
6710 Ffind_coding_systems_region_internal
,
6711 Sfind_coding_systems_region_internal
, 2, 2, 0,
6712 doc
: /* Internal use only. */)
6714 Lisp_Object start
, end
;
6716 Lisp_Object work_table
, safe_codings
;
6717 int non_ascii_p
= 0;
6718 int single_byte_char_found
= 0;
6719 const unsigned char *p1
, *p1end
, *p2
, *p2end
, *p
;
6721 if (STRINGP (start
))
6723 if (!STRING_MULTIBYTE (start
))
6725 p1
= SDATA (start
), p1end
= p1
+ SBYTES (start
);
6727 if (SCHARS (start
) != SBYTES (start
))
6734 CHECK_NUMBER_COERCE_MARKER (start
);
6735 CHECK_NUMBER_COERCE_MARKER (end
);
6736 if (XINT (start
) < BEG
|| XINT (end
) > Z
|| XINT (start
) > XINT (end
))
6737 args_out_of_range (start
, end
);
6738 if (NILP (current_buffer
->enable_multibyte_characters
))
6740 from
= CHAR_TO_BYTE (XINT (start
));
6741 to
= CHAR_TO_BYTE (XINT (end
));
6742 stop
= from
< GPT_BYTE
&& GPT_BYTE
< to
? GPT_BYTE
: to
;
6743 p1
= BYTE_POS_ADDR (from
), p1end
= p1
+ (stop
- from
);
6747 p2
= BYTE_POS_ADDR (stop
), p2end
= p2
+ (to
- stop
);
6748 if (XINT (end
) - XINT (start
) != to
- from
)
6754 /* We are sure that the text contains no multibyte character.
6755 Check if it contains eight-bit-graphic. */
6757 for (p
= p1
; p
< p1end
&& ASCII_BYTE_P (*p
); p
++);
6760 for (p
= p2
; p
< p2end
&& ASCII_BYTE_P (*p
); p
++);
6766 /* The text contains non-ASCII characters. */
6768 work_table
= Fmake_char_table (Qchar_coding_system
, Qnil
);
6769 safe_codings
= Fcopy_sequence (XCDR (Vcoding_system_safe_chars
));
6771 safe_codings
= find_safe_codings (p1
, p1end
, safe_codings
, work_table
,
6772 &single_byte_char_found
);
6774 safe_codings
= find_safe_codings (p2
, p2end
, safe_codings
, work_table
,
6775 &single_byte_char_found
);
6776 if (EQ (safe_codings
, XCDR (Vcoding_system_safe_chars
)))
6780 /* Turn safe_codings to a list of coding systems... */
6783 if (single_byte_char_found
)
6784 /* ... and append these for eight-bit chars. */
6785 val
= Fcons (Qraw_text
,
6786 Fcons (Qemacs_mule
, Fcons (Qno_conversion
, Qnil
)));
6788 /* ... and append generic coding systems. */
6789 val
= Fcopy_sequence (XCAR (Vcoding_system_safe_chars
));
6791 for (; CONSP (safe_codings
); safe_codings
= XCDR (safe_codings
))
6792 val
= Fcons (XCAR (XCAR (safe_codings
)), val
);
6796 return safe_codings
;
6800 /* Search from position POS for such characters that are unencodable
6801 accoding to SAFE_CHARS, and return a list of their positions. P
6802 points where in the memory the character at POS exists. Limit the
6803 search at PEND or when Nth unencodable characters are found.
6805 If SAFE_CHARS is a char table, an element for an unencodable
6808 If SAFE_CHARS is nil, all non-ASCII characters are unencodable.
6810 Otherwise, SAFE_CHARS is t, and only eight-bit-contrl and
6811 eight-bit-graphic characters are unencodable. */
6814 unencodable_char_position (safe_chars
, pos
, p
, pend
, n
)
6815 Lisp_Object safe_chars
;
6817 unsigned char *p
, *pend
;
6820 Lisp_Object pos_list
;
6826 int c
= STRING_CHAR_AND_LENGTH (p
, MAX_MULTIBYTE_LENGTH
, len
);
6829 && (CHAR_TABLE_P (safe_chars
)
6830 ? NILP (CHAR_TABLE_REF (safe_chars
, c
))
6831 : (NILP (safe_chars
) || c
< 256)))
6833 pos_list
= Fcons (make_number (pos
), pos_list
);
6840 return Fnreverse (pos_list
);
6844 DEFUN ("unencodable-char-position", Funencodable_char_position
,
6845 Sunencodable_char_position
, 3, 5, 0,
6847 Return position of first un-encodable character in a region.
6848 START and END specfiy the region and CODING-SYSTEM specifies the
6849 encoding to check. Return nil if CODING-SYSTEM does encode the region.
6851 If optional 4th argument COUNT is non-nil, it specifies at most how
6852 many un-encodable characters to search. In this case, the value is a
6855 If optional 5th argument STRING is non-nil, it is a string to search
6856 for un-encodable characters. In that case, START and END are indexes
6858 (start
, end
, coding_system
, count
, string
)
6859 Lisp_Object start
, end
, coding_system
, count
, string
;
6862 Lisp_Object safe_chars
;
6863 struct coding_system coding
;
6864 Lisp_Object positions
;
6866 unsigned char *p
, *pend
;
6870 validate_region (&start
, &end
);
6871 from
= XINT (start
);
6873 if (NILP (current_buffer
->enable_multibyte_characters
))
6875 p
= CHAR_POS_ADDR (from
);
6879 pend
= CHAR_POS_ADDR (to
);
6883 CHECK_STRING (string
);
6884 CHECK_NATNUM (start
);
6886 from
= XINT (start
);
6889 || to
> SCHARS (string
))
6890 args_out_of_range_3 (string
, start
, end
);
6891 if (! STRING_MULTIBYTE (string
))
6893 p
= SDATA (string
) + string_char_to_byte (string
, from
);
6894 pend
= SDATA (string
) + string_char_to_byte (string
, to
);
6897 setup_coding_system (Fcheck_coding_system (coding_system
), &coding
);
6903 CHECK_NATNUM (count
);
6907 if (coding
.type
== coding_type_no_conversion
6908 || coding
.type
== coding_type_raw_text
)
6911 if (coding
.type
== coding_type_undecided
)
6914 safe_chars
= coding_safe_chars (coding_system
);
6916 if (STRINGP (string
)
6917 || from
>= GPT
|| to
<= GPT
)
6918 positions
= unencodable_char_position (safe_chars
, from
, p
, pend
, n
);
6921 Lisp_Object args
[2];
6923 args
[0] = unencodable_char_position (safe_chars
, from
, p
, GPT_ADDR
, n
);
6924 n
-= XINT (Flength (args
[0]));
6926 positions
= args
[0];
6929 args
[1] = unencodable_char_position (safe_chars
, GPT
, GAP_END_ADDR
,
6931 positions
= Fappend (2, args
);
6935 return (NILP (count
) ? Fcar (positions
) : positions
);
6940 code_convert_region1 (start
, end
, coding_system
, encodep
)
6941 Lisp_Object start
, end
, coding_system
;
6944 struct coding_system coding
;
6947 CHECK_NUMBER_COERCE_MARKER (start
);
6948 CHECK_NUMBER_COERCE_MARKER (end
);
6949 CHECK_SYMBOL (coding_system
);
6951 validate_region (&start
, &end
);
6952 from
= XFASTINT (start
);
6953 to
= XFASTINT (end
);
6955 if (NILP (coding_system
))
6956 return make_number (to
- from
);
6958 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
6959 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
6961 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
6962 coding
.src_multibyte
= coding
.dst_multibyte
6963 = !NILP (current_buffer
->enable_multibyte_characters
);
6964 code_convert_region (from
, CHAR_TO_BYTE (from
), to
, CHAR_TO_BYTE (to
),
6965 &coding
, encodep
, 1);
6966 Vlast_coding_system_used
= coding
.symbol
;
6967 return make_number (coding
.produced_char
);
6970 DEFUN ("decode-coding-region", Fdecode_coding_region
, Sdecode_coding_region
,
6971 3, 3, "r\nzCoding system: ",
6972 doc
: /* Decode the current region from the specified coding system.
6973 When called from a program, takes three arguments:
6974 START, END, and CODING-SYSTEM. START and END are buffer positions.
6975 This function sets `last-coding-system-used' to the precise coding system
6976 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6977 not fully specified.)
6978 It returns the length of the decoded text. */)
6979 (start
, end
, coding_system
)
6980 Lisp_Object start
, end
, coding_system
;
6982 return code_convert_region1 (start
, end
, coding_system
, 0);
6985 DEFUN ("encode-coding-region", Fencode_coding_region
, Sencode_coding_region
,
6986 3, 3, "r\nzCoding system: ",
6987 doc
: /* Encode the current region into the specified coding system.
6988 When called from a program, takes three arguments:
6989 START, END, and CODING-SYSTEM. START and END are buffer positions.
6990 This function sets `last-coding-system-used' to the precise coding system
6991 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
6992 not fully specified.)
6993 It returns the length of the encoded text. */)
6994 (start
, end
, coding_system
)
6995 Lisp_Object start
, end
, coding_system
;
6997 return code_convert_region1 (start
, end
, coding_system
, 1);
7001 code_convert_string1 (string
, coding_system
, nocopy
, encodep
)
7002 Lisp_Object string
, coding_system
, nocopy
;
7005 struct coding_system coding
;
7007 CHECK_STRING (string
);
7008 CHECK_SYMBOL (coding_system
);
7010 if (NILP (coding_system
))
7011 return (NILP (nocopy
) ? Fcopy_sequence (string
) : string
);
7013 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7014 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7016 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7018 ? encode_coding_string (string
, &coding
, !NILP (nocopy
))
7019 : decode_coding_string (string
, &coding
, !NILP (nocopy
)));
7020 Vlast_coding_system_used
= coding
.symbol
;
7025 DEFUN ("decode-coding-string", Fdecode_coding_string
, Sdecode_coding_string
,
7027 doc
: /* Decode STRING which is encoded in CODING-SYSTEM, and return the result.
7028 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7029 if the decoding operation is trivial.
7030 This function sets `last-coding-system-used' to the precise coding system
7031 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7032 not fully specified.) */)
7033 (string
, coding_system
, nocopy
)
7034 Lisp_Object string
, coding_system
, nocopy
;
7036 return code_convert_string1 (string
, coding_system
, nocopy
, 0);
7039 DEFUN ("encode-coding-string", Fencode_coding_string
, Sencode_coding_string
,
7041 doc
: /* Encode STRING to CODING-SYSTEM, and return the result.
7042 Optional arg NOCOPY non-nil means it is OK to return STRING itself
7043 if the encoding operation is trivial.
7044 This function sets `last-coding-system-used' to the precise coding system
7045 used (which may be different from CODING-SYSTEM if CODING-SYSTEM is
7046 not fully specified.) */)
7047 (string
, coding_system
, nocopy
)
7048 Lisp_Object string
, coding_system
, nocopy
;
7050 return code_convert_string1 (string
, coding_system
, nocopy
, 1);
7053 /* Encode or decode STRING according to CODING_SYSTEM.
7054 Do not set Vlast_coding_system_used.
7056 This function is called only from macros DECODE_FILE and
7057 ENCODE_FILE, thus we ignore character composition. */
7060 code_convert_string_norecord (string
, coding_system
, encodep
)
7061 Lisp_Object string
, coding_system
;
7064 struct coding_system coding
;
7066 CHECK_STRING (string
);
7067 CHECK_SYMBOL (coding_system
);
7069 if (NILP (coding_system
))
7072 if (setup_coding_system (Fcheck_coding_system (coding_system
), &coding
) < 0)
7073 error ("Invalid coding system: %s", SDATA (SYMBOL_NAME (coding_system
)));
7075 coding
.composing
= COMPOSITION_DISABLED
;
7076 coding
.mode
|= CODING_MODE_LAST_BLOCK
;
7078 ? encode_coding_string (string
, &coding
, 1)
7079 : decode_coding_string (string
, &coding
, 1));
7082 DEFUN ("decode-sjis-char", Fdecode_sjis_char
, Sdecode_sjis_char
, 1, 1, 0,
7083 doc
: /* Decode a Japanese character which has CODE in shift_jis encoding.
7084 Return the corresponding character. */)
7088 unsigned char c1
, c2
, s1
, s2
;
7091 CHECK_NUMBER (code
);
7092 s1
= (XFASTINT (code
)) >> 8, s2
= (XFASTINT (code
)) & 0xFF;
7096 XSETFASTINT (val
, s2
);
7097 else if (s2
>= 0xA0 || s2
<= 0xDF)
7098 XSETFASTINT (val
, MAKE_CHAR (charset_katakana_jisx0201
, s2
, 0));
7100 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7104 if ((s1
< 0x80 || (s1
> 0x9F && s1
< 0xE0) || s1
> 0xEF)
7105 || (s2
< 0x40 || s2
== 0x7F || s2
> 0xFC))
7106 error ("Invalid Shift JIS code: %x", XFASTINT (code
));
7107 DECODE_SJIS (s1
, s2
, c1
, c2
);
7108 XSETFASTINT (val
, MAKE_CHAR (charset_jisx0208
, c1
, c2
));
7113 DEFUN ("encode-sjis-char", Fencode_sjis_char
, Sencode_sjis_char
, 1, 1, 0,
7114 doc
: /* Encode a Japanese character CHAR to shift_jis encoding.
7115 Return the corresponding code in SJIS. */)
7119 int charset
, c1
, c2
, s1
, s2
;
7123 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7124 if (charset
== CHARSET_ASCII
)
7128 else if (charset
== charset_jisx0208
7129 && c1
> 0x20 && c1
< 0x7F && c2
> 0x20 && c2
< 0x7F)
7131 ENCODE_SJIS (c1
, c2
, s1
, s2
);
7132 XSETFASTINT (val
, (s1
<< 8) | s2
);
7134 else if (charset
== charset_katakana_jisx0201
7135 && c1
> 0x20 && c2
< 0xE0)
7137 XSETFASTINT (val
, c1
| 0x80);
7140 error ("Can't encode to shift_jis: %d", XFASTINT (ch
));
7144 DEFUN ("decode-big5-char", Fdecode_big5_char
, Sdecode_big5_char
, 1, 1, 0,
7145 doc
: /* Decode a Big5 character which has CODE in BIG5 coding system.
7146 Return the corresponding character. */)
7151 unsigned char b1
, b2
, c1
, c2
;
7154 CHECK_NUMBER (code
);
7155 b1
= (XFASTINT (code
)) >> 8, b2
= (XFASTINT (code
)) & 0xFF;
7159 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7164 if ((b1
< 0xA1 || b1
> 0xFE)
7165 || (b2
< 0x40 || (b2
> 0x7E && b2
< 0xA1) || b2
> 0xFE))
7166 error ("Invalid BIG5 code: %x", XFASTINT (code
));
7167 DECODE_BIG5 (b1
, b2
, charset
, c1
, c2
);
7168 XSETFASTINT (val
, MAKE_CHAR (charset
, c1
, c2
));
7173 DEFUN ("encode-big5-char", Fencode_big5_char
, Sencode_big5_char
, 1, 1, 0,
7174 doc
: /* Encode the Big5 character CHAR to BIG5 coding system.
7175 Return the corresponding character code in Big5. */)
7179 int charset
, c1
, c2
, b1
, b2
;
7183 SPLIT_CHAR (XFASTINT (ch
), charset
, c1
, c2
);
7184 if (charset
== CHARSET_ASCII
)
7188 else if ((charset
== charset_big5_1
7189 && (XFASTINT (ch
) >= 0x250a1 && XFASTINT (ch
) <= 0x271ec))
7190 || (charset
== charset_big5_2
7191 && XFASTINT (ch
) >= 0x290a1 && XFASTINT (ch
) <= 0x2bdb2))
7193 ENCODE_BIG5 (charset
, c1
, c2
, b1
, b2
);
7194 XSETFASTINT (val
, (b1
<< 8) | b2
);
7197 error ("Can't encode to Big5: %d", XFASTINT (ch
));
7201 DEFUN ("set-terminal-coding-system-internal", Fset_terminal_coding_system_internal
,
7202 Sset_terminal_coding_system_internal
, 1, 1, 0,
7203 doc
: /* Internal use only. */)
7205 Lisp_Object coding_system
;
7207 CHECK_SYMBOL (coding_system
);
7208 setup_coding_system (Fcheck_coding_system (coding_system
), &terminal_coding
);
7209 /* We had better not send unsafe characters to terminal. */
7210 terminal_coding
.mode
|= CODING_MODE_INHIBIT_UNENCODABLE_CHAR
;
7211 /* Character composition should be disabled. */
7212 terminal_coding
.composing
= COMPOSITION_DISABLED
;
7213 /* Error notification should be suppressed. */
7214 terminal_coding
.suppress_error
= 1;
7215 terminal_coding
.src_multibyte
= 1;
7216 terminal_coding
.dst_multibyte
= 0;
7220 DEFUN ("set-safe-terminal-coding-system-internal", Fset_safe_terminal_coding_system_internal
,
7221 Sset_safe_terminal_coding_system_internal
, 1, 1, 0,
7222 doc
: /* Internal use only. */)
7224 Lisp_Object coding_system
;
7226 CHECK_SYMBOL (coding_system
);
7227 setup_coding_system (Fcheck_coding_system (coding_system
),
7228 &safe_terminal_coding
);
7229 /* Character composition should be disabled. */
7230 safe_terminal_coding
.composing
= COMPOSITION_DISABLED
;
7231 /* Error notification should be suppressed. */
7232 safe_terminal_coding
.suppress_error
= 1;
7233 safe_terminal_coding
.src_multibyte
= 1;
7234 safe_terminal_coding
.dst_multibyte
= 0;
7238 DEFUN ("terminal-coding-system", Fterminal_coding_system
,
7239 Sterminal_coding_system
, 0, 0, 0,
7240 doc
: /* Return coding system specified for terminal output. */)
7243 return terminal_coding
.symbol
;
7246 DEFUN ("set-keyboard-coding-system-internal", Fset_keyboard_coding_system_internal
,
7247 Sset_keyboard_coding_system_internal
, 1, 1, 0,
7248 doc
: /* Internal use only. */)
7250 Lisp_Object coding_system
;
7252 CHECK_SYMBOL (coding_system
);
7253 setup_coding_system (Fcheck_coding_system (coding_system
), &keyboard_coding
);
7254 /* Character composition should be disabled. */
7255 keyboard_coding
.composing
= COMPOSITION_DISABLED
;
7259 DEFUN ("keyboard-coding-system", Fkeyboard_coding_system
,
7260 Skeyboard_coding_system
, 0, 0, 0,
7261 doc
: /* Return coding system specified for decoding keyboard input. */)
7264 return keyboard_coding
.symbol
;
7268 DEFUN ("find-operation-coding-system", Ffind_operation_coding_system
,
7269 Sfind_operation_coding_system
, 1, MANY
, 0,
7270 doc
: /* Choose a coding system for an operation based on the target name.
7271 The value names a pair of coding systems: (DECODING-SYSTEM . ENCODING-SYSTEM).
7272 DECODING-SYSTEM is the coding system to use for decoding
7273 \(in case OPERATION does decoding), and ENCODING-SYSTEM is the coding system
7274 for encoding (in case OPERATION does encoding).
7276 The first argument OPERATION specifies an I/O primitive:
7277 For file I/O, `insert-file-contents' or `write-region'.
7278 For process I/O, `call-process', `call-process-region', or `start-process'.
7279 For network I/O, `open-network-stream'.
7281 The remaining arguments should be the same arguments that were passed
7282 to the primitive. Depending on which primitive, one of those arguments
7283 is selected as the TARGET. For example, if OPERATION does file I/O,
7284 whichever argument specifies the file name is TARGET.
7286 TARGET has a meaning which depends on OPERATION:
7287 For file I/O, TARGET is a file name.
7288 For process I/O, TARGET is a process name.
7289 For network I/O, TARGET is a service name or a port number
7291 This function looks up what specified for TARGET in,
7292 `file-coding-system-alist', `process-coding-system-alist',
7293 or `network-coding-system-alist' depending on OPERATION.
7294 They may specify a coding system, a cons of coding systems,
7295 or a function symbol to call.
7296 In the last case, we call the function with one argument,
7297 which is a list of all the arguments given to this function.
7299 usage: (find-operation-coding-system OPERATION ARGUMENTS ...) */)
7304 Lisp_Object operation
, target_idx
, target
, val
;
7305 register Lisp_Object chain
;
7308 error ("Too few arguments");
7309 operation
= args
[0];
7310 if (!SYMBOLP (operation
)
7311 || !INTEGERP (target_idx
= Fget (operation
, Qtarget_idx
)))
7312 error ("Invalid first argument");
7313 if (nargs
< 1 + XINT (target_idx
))
7314 error ("Too few arguments for operation: %s",
7315 SDATA (SYMBOL_NAME (operation
)));
7316 /* For write-region, if the 6th argument (i.e. VISIT, the 5th
7317 argument to write-region) is string, it must be treated as a
7318 target file name. */
7319 if (EQ (operation
, Qwrite_region
)
7321 && STRINGP (args
[5]))
7322 target_idx
= make_number (4);
7323 target
= args
[XINT (target_idx
) + 1];
7324 if (!(STRINGP (target
)
7325 || (EQ (operation
, Qopen_network_stream
) && INTEGERP (target
))))
7326 error ("Invalid argument %d", XINT (target_idx
) + 1);
7328 chain
= ((EQ (operation
, Qinsert_file_contents
)
7329 || EQ (operation
, Qwrite_region
))
7330 ? Vfile_coding_system_alist
7331 : (EQ (operation
, Qopen_network_stream
)
7332 ? Vnetwork_coding_system_alist
7333 : Vprocess_coding_system_alist
));
7337 for (; CONSP (chain
); chain
= XCDR (chain
))
7343 && ((STRINGP (target
)
7344 && STRINGP (XCAR (elt
))
7345 && fast_string_match (XCAR (elt
), target
) >= 0)
7346 || (INTEGERP (target
) && EQ (target
, XCAR (elt
)))))
7349 /* Here, if VAL is both a valid coding system and a valid
7350 function symbol, we return VAL as a coding system. */
7353 if (! SYMBOLP (val
))
7355 if (! NILP (Fcoding_system_p (val
)))
7356 return Fcons (val
, val
);
7357 if (! NILP (Ffboundp (val
)))
7359 val
= call1 (val
, Flist (nargs
, args
));
7362 if (SYMBOLP (val
) && ! NILP (Fcoding_system_p (val
)))
7363 return Fcons (val
, val
);
7371 DEFUN ("update-coding-systems-internal", Fupdate_coding_systems_internal
,
7372 Supdate_coding_systems_internal
, 0, 0, 0,
7373 doc
: /* Update internal database for ISO2022 and CCL based coding systems.
7374 When values of any coding categories are changed, you must
7375 call this function. */)
7380 for (i
= CODING_CATEGORY_IDX_EMACS_MULE
; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7384 val
= SYMBOL_VALUE (XVECTOR (Vcoding_category_table
)->contents
[i
]);
7387 if (! coding_system_table
[i
])
7388 coding_system_table
[i
] = ((struct coding_system
*)
7389 xmalloc (sizeof (struct coding_system
)));
7390 setup_coding_system (val
, coding_system_table
[i
]);
7392 else if (coding_system_table
[i
])
7394 xfree (coding_system_table
[i
]);
7395 coding_system_table
[i
] = NULL
;
7402 DEFUN ("set-coding-priority-internal", Fset_coding_priority_internal
,
7403 Sset_coding_priority_internal
, 0, 0, 0,
7404 doc
: /* Update internal database for the current value of `coding-category-list'.
7405 This function is internal use only. */)
7411 val
= Vcoding_category_list
;
7413 while (CONSP (val
) && i
< CODING_CATEGORY_IDX_MAX
)
7415 if (! SYMBOLP (XCAR (val
)))
7417 idx
= XFASTINT (Fget (XCAR (val
), Qcoding_category_index
));
7418 if (idx
>= CODING_CATEGORY_IDX_MAX
)
7420 coding_priorities
[i
++] = (1 << idx
);
7423 /* If coding-category-list is valid and contains all coding
7424 categories, `i' should be CODING_CATEGORY_IDX_MAX now. If not,
7425 the following code saves Emacs from crashing. */
7426 while (i
< CODING_CATEGORY_IDX_MAX
)
7427 coding_priorities
[i
++] = CODING_CATEGORY_MASK_RAW_TEXT
;
7432 DEFUN ("define-coding-system-internal", Fdefine_coding_system_internal
,
7433 Sdefine_coding_system_internal
, 1, 1, 0,
7434 doc
: /* Register CODING-SYSTEM as a base coding system.
7435 This function is internal use only. */)
7437 Lisp_Object coding_system
;
7439 Lisp_Object safe_chars
, slot
;
7441 if (NILP (Fcheck_coding_system (coding_system
)))
7442 Fsignal (Qcoding_system_error
, Fcons (coding_system
, Qnil
));
7443 safe_chars
= coding_safe_chars (coding_system
);
7444 if (! EQ (safe_chars
, Qt
) && ! CHAR_TABLE_P (safe_chars
))
7445 error ("No valid safe-chars property for %s",
7446 SDATA (SYMBOL_NAME (coding_system
)));
7447 if (EQ (safe_chars
, Qt
))
7449 if (NILP (Fmemq (coding_system
, XCAR (Vcoding_system_safe_chars
))))
7450 XSETCAR (Vcoding_system_safe_chars
,
7451 Fcons (coding_system
, XCAR (Vcoding_system_safe_chars
)));
7455 slot
= Fassq (coding_system
, XCDR (Vcoding_system_safe_chars
));
7457 XSETCDR (Vcoding_system_safe_chars
,
7458 nconc2 (XCDR (Vcoding_system_safe_chars
),
7459 Fcons (Fcons (coding_system
, safe_chars
), Qnil
)));
7461 XSETCDR (slot
, safe_chars
);
7469 /*** 9. Post-amble ***/
7476 /* Emacs' internal format specific initialize routine. */
7477 for (i
= 0; i
<= 0x20; i
++)
7478 emacs_code_class
[i
] = EMACS_control_code
;
7479 emacs_code_class
[0x0A] = EMACS_linefeed_code
;
7480 emacs_code_class
[0x0D] = EMACS_carriage_return_code
;
7481 for (i
= 0x21 ; i
< 0x7F; i
++)
7482 emacs_code_class
[i
] = EMACS_ascii_code
;
7483 emacs_code_class
[0x7F] = EMACS_control_code
;
7484 for (i
= 0x80; i
< 0xFF; i
++)
7485 emacs_code_class
[i
] = EMACS_invalid_code
;
7486 emacs_code_class
[LEADING_CODE_PRIVATE_11
] = EMACS_leading_code_3
;
7487 emacs_code_class
[LEADING_CODE_PRIVATE_12
] = EMACS_leading_code_3
;
7488 emacs_code_class
[LEADING_CODE_PRIVATE_21
] = EMACS_leading_code_4
;
7489 emacs_code_class
[LEADING_CODE_PRIVATE_22
] = EMACS_leading_code_4
;
7491 /* ISO2022 specific initialize routine. */
7492 for (i
= 0; i
< 0x20; i
++)
7493 iso_code_class
[i
] = ISO_control_0
;
7494 for (i
= 0x21; i
< 0x7F; i
++)
7495 iso_code_class
[i
] = ISO_graphic_plane_0
;
7496 for (i
= 0x80; i
< 0xA0; i
++)
7497 iso_code_class
[i
] = ISO_control_1
;
7498 for (i
= 0xA1; i
< 0xFF; i
++)
7499 iso_code_class
[i
] = ISO_graphic_plane_1
;
7500 iso_code_class
[0x20] = iso_code_class
[0x7F] = ISO_0x20_or_0x7F
;
7501 iso_code_class
[0xA0] = iso_code_class
[0xFF] = ISO_0xA0_or_0xFF
;
7502 iso_code_class
[ISO_CODE_CR
] = ISO_carriage_return
;
7503 iso_code_class
[ISO_CODE_SO
] = ISO_shift_out
;
7504 iso_code_class
[ISO_CODE_SI
] = ISO_shift_in
;
7505 iso_code_class
[ISO_CODE_SS2_7
] = ISO_single_shift_2_7
;
7506 iso_code_class
[ISO_CODE_ESC
] = ISO_escape
;
7507 iso_code_class
[ISO_CODE_SS2
] = ISO_single_shift_2
;
7508 iso_code_class
[ISO_CODE_SS3
] = ISO_single_shift_3
;
7509 iso_code_class
[ISO_CODE_CSI
] = ISO_control_sequence_introducer
;
7511 setup_coding_system (Qnil
, &keyboard_coding
);
7512 setup_coding_system (Qnil
, &terminal_coding
);
7513 setup_coding_system (Qnil
, &safe_terminal_coding
);
7514 setup_coding_system (Qnil
, &default_buffer_file_coding
);
7516 bzero (coding_system_table
, sizeof coding_system_table
);
7518 bzero (ascii_skip_code
, sizeof ascii_skip_code
);
7519 for (i
= 0; i
< 128; i
++)
7520 ascii_skip_code
[i
] = 1;
7522 #if defined (MSDOS) || defined (WINDOWSNT)
7523 system_eol_type
= CODING_EOL_CRLF
;
7525 system_eol_type
= CODING_EOL_LF
;
7528 inhibit_pre_post_conversion
= 0;
7536 Qtarget_idx
= intern ("target-idx");
7537 staticpro (&Qtarget_idx
);
7539 Qcoding_system_history
= intern ("coding-system-history");
7540 staticpro (&Qcoding_system_history
);
7541 Fset (Qcoding_system_history
, Qnil
);
7543 /* Target FILENAME is the first argument. */
7544 Fput (Qinsert_file_contents
, Qtarget_idx
, make_number (0));
7545 /* Target FILENAME is the third argument. */
7546 Fput (Qwrite_region
, Qtarget_idx
, make_number (2));
7548 Qcall_process
= intern ("call-process");
7549 staticpro (&Qcall_process
);
7550 /* Target PROGRAM is the first argument. */
7551 Fput (Qcall_process
, Qtarget_idx
, make_number (0));
7553 Qcall_process_region
= intern ("call-process-region");
7554 staticpro (&Qcall_process_region
);
7555 /* Target PROGRAM is the third argument. */
7556 Fput (Qcall_process_region
, Qtarget_idx
, make_number (2));
7558 Qstart_process
= intern ("start-process");
7559 staticpro (&Qstart_process
);
7560 /* Target PROGRAM is the third argument. */
7561 Fput (Qstart_process
, Qtarget_idx
, make_number (2));
7563 Qopen_network_stream
= intern ("open-network-stream");
7564 staticpro (&Qopen_network_stream
);
7565 /* Target SERVICE is the fourth argument. */
7566 Fput (Qopen_network_stream
, Qtarget_idx
, make_number (3));
7568 Qcoding_system
= intern ("coding-system");
7569 staticpro (&Qcoding_system
);
7571 Qeol_type
= intern ("eol-type");
7572 staticpro (&Qeol_type
);
7574 Qbuffer_file_coding_system
= intern ("buffer-file-coding-system");
7575 staticpro (&Qbuffer_file_coding_system
);
7577 Qpost_read_conversion
= intern ("post-read-conversion");
7578 staticpro (&Qpost_read_conversion
);
7580 Qpre_write_conversion
= intern ("pre-write-conversion");
7581 staticpro (&Qpre_write_conversion
);
7583 Qno_conversion
= intern ("no-conversion");
7584 staticpro (&Qno_conversion
);
7586 Qundecided
= intern ("undecided");
7587 staticpro (&Qundecided
);
7589 Qcoding_system_p
= intern ("coding-system-p");
7590 staticpro (&Qcoding_system_p
);
7592 Qcoding_system_error
= intern ("coding-system-error");
7593 staticpro (&Qcoding_system_error
);
7595 Fput (Qcoding_system_error
, Qerror_conditions
,
7596 Fcons (Qcoding_system_error
, Fcons (Qerror
, Qnil
)));
7597 Fput (Qcoding_system_error
, Qerror_message
,
7598 build_string ("Invalid coding system"));
7600 Qcoding_category
= intern ("coding-category");
7601 staticpro (&Qcoding_category
);
7602 Qcoding_category_index
= intern ("coding-category-index");
7603 staticpro (&Qcoding_category_index
);
7605 Vcoding_category_table
7606 = Fmake_vector (make_number (CODING_CATEGORY_IDX_MAX
), Qnil
);
7607 staticpro (&Vcoding_category_table
);
7610 for (i
= 0; i
< CODING_CATEGORY_IDX_MAX
; i
++)
7612 XVECTOR (Vcoding_category_table
)->contents
[i
]
7613 = intern (coding_category_name
[i
]);
7614 Fput (XVECTOR (Vcoding_category_table
)->contents
[i
],
7615 Qcoding_category_index
, make_number (i
));
7619 Vcoding_system_safe_chars
= Fcons (Qnil
, Qnil
);
7620 staticpro (&Vcoding_system_safe_chars
);
7622 Qtranslation_table
= intern ("translation-table");
7623 staticpro (&Qtranslation_table
);
7624 Fput (Qtranslation_table
, Qchar_table_extra_slots
, make_number (2));
7626 Qtranslation_table_id
= intern ("translation-table-id");
7627 staticpro (&Qtranslation_table_id
);
7629 Qtranslation_table_for_decode
= intern ("translation-table-for-decode");
7630 staticpro (&Qtranslation_table_for_decode
);
7632 Qtranslation_table_for_encode
= intern ("translation-table-for-encode");
7633 staticpro (&Qtranslation_table_for_encode
);
7635 Qsafe_chars
= intern ("safe-chars");
7636 staticpro (&Qsafe_chars
);
7638 Qchar_coding_system
= intern ("char-coding-system");
7639 staticpro (&Qchar_coding_system
);
7641 /* Intern this now in case it isn't already done.
7642 Setting this variable twice is harmless.
7643 But don't staticpro it here--that is done in alloc.c. */
7644 Qchar_table_extra_slots
= intern ("char-table-extra-slots");
7645 Fput (Qsafe_chars
, Qchar_table_extra_slots
, make_number (0));
7646 Fput (Qchar_coding_system
, Qchar_table_extra_slots
, make_number (0));
7648 Qvalid_codes
= intern ("valid-codes");
7649 staticpro (&Qvalid_codes
);
7651 Qemacs_mule
= intern ("emacs-mule");
7652 staticpro (&Qemacs_mule
);
7654 Qraw_text
= intern ("raw-text");
7655 staticpro (&Qraw_text
);
7657 Qutf_8
= intern ("utf-8");
7658 staticpro (&Qutf_8
);
7660 Qcoding_system_define_form
= intern ("coding-system-define-form");
7661 staticpro (&Qcoding_system_define_form
);
7663 defsubr (&Scoding_system_p
);
7664 defsubr (&Sread_coding_system
);
7665 defsubr (&Sread_non_nil_coding_system
);
7666 defsubr (&Scheck_coding_system
);
7667 defsubr (&Sdetect_coding_region
);
7668 defsubr (&Sdetect_coding_string
);
7669 defsubr (&Sfind_coding_systems_region_internal
);
7670 defsubr (&Sunencodable_char_position
);
7671 defsubr (&Sdecode_coding_region
);
7672 defsubr (&Sencode_coding_region
);
7673 defsubr (&Sdecode_coding_string
);
7674 defsubr (&Sencode_coding_string
);
7675 defsubr (&Sdecode_sjis_char
);
7676 defsubr (&Sencode_sjis_char
);
7677 defsubr (&Sdecode_big5_char
);
7678 defsubr (&Sencode_big5_char
);
7679 defsubr (&Sset_terminal_coding_system_internal
);
7680 defsubr (&Sset_safe_terminal_coding_system_internal
);
7681 defsubr (&Sterminal_coding_system
);
7682 defsubr (&Sset_keyboard_coding_system_internal
);
7683 defsubr (&Skeyboard_coding_system
);
7684 defsubr (&Sfind_operation_coding_system
);
7685 defsubr (&Supdate_coding_systems_internal
);
7686 defsubr (&Sset_coding_priority_internal
);
7687 defsubr (&Sdefine_coding_system_internal
);
7689 DEFVAR_LISP ("coding-system-list", &Vcoding_system_list
,
7690 doc
: /* List of coding systems.
7692 Do not alter the value of this variable manually. This variable should be
7693 updated by the functions `make-coding-system' and
7694 `define-coding-system-alias'. */);
7695 Vcoding_system_list
= Qnil
;
7697 DEFVAR_LISP ("coding-system-alist", &Vcoding_system_alist
,
7698 doc
: /* Alist of coding system names.
7699 Each element is one element list of coding system name.
7700 This variable is given to `completing-read' as TABLE argument.
7702 Do not alter the value of this variable manually. This variable should be
7703 updated by the functions `make-coding-system' and
7704 `define-coding-system-alias'. */);
7705 Vcoding_system_alist
= Qnil
;
7707 DEFVAR_LISP ("coding-category-list", &Vcoding_category_list
,
7708 doc
: /* List of coding-categories (symbols) ordered by priority.
7710 On detecting a coding system, Emacs tries code detection algorithms
7711 associated with each coding-category one by one in this order. When
7712 one algorithm agrees with a byte sequence of source text, the coding
7713 system bound to the corresponding coding-category is selected. */);
7717 Vcoding_category_list
= Qnil
;
7718 for (i
= CODING_CATEGORY_IDX_MAX
- 1; i
>= 0; i
--)
7719 Vcoding_category_list
7720 = Fcons (XVECTOR (Vcoding_category_table
)->contents
[i
],
7721 Vcoding_category_list
);
7724 DEFVAR_LISP ("coding-system-for-read", &Vcoding_system_for_read
,
7725 doc
: /* Specify the coding system for read operations.
7726 It is useful to bind this variable with `let', but do not set it globally.
7727 If the value is a coding system, it is used for decoding on read operation.
7728 If not, an appropriate element is used from one of the coding system alists:
7729 There are three such tables, `file-coding-system-alist',
7730 `process-coding-system-alist', and `network-coding-system-alist'. */);
7731 Vcoding_system_for_read
= Qnil
;
7733 DEFVAR_LISP ("coding-system-for-write", &Vcoding_system_for_write
,
7734 doc
: /* Specify the coding system for write operations.
7735 Programs bind this variable with `let', but you should not set it globally.
7736 If the value is a coding system, it is used for encoding of output,
7737 when writing it to a file and when sending it to a file or subprocess.
7739 If this does not specify a coding system, an appropriate element
7740 is used from one of the coding system alists:
7741 There are three such tables, `file-coding-system-alist',
7742 `process-coding-system-alist', and `network-coding-system-alist'.
7743 For output to files, if the above procedure does not specify a coding system,
7744 the value of `buffer-file-coding-system' is used. */);
7745 Vcoding_system_for_write
= Qnil
;
7747 DEFVAR_LISP ("last-coding-system-used", &Vlast_coding_system_used
,
7748 doc
: /* Coding system used in the latest file or process I/O.
7749 Also set by `encode-coding-region', `decode-coding-region',
7750 `encode-coding-string' and `decode-coding-string'. */);
7751 Vlast_coding_system_used
= Qnil
;
7753 DEFVAR_BOOL ("inhibit-eol-conversion", &inhibit_eol_conversion
,
7754 doc
: /* *Non-nil means always inhibit code conversion of end-of-line format.
7755 See info node `Coding Systems' and info node `Text and Binary' concerning
7756 such conversion. */);
7757 inhibit_eol_conversion
= 0;
7759 DEFVAR_BOOL ("inherit-process-coding-system", &inherit_process_coding_system
,
7760 doc
: /* Non-nil means process buffer inherits coding system of process output.
7761 Bind it to t if the process output is to be treated as if it were a file
7762 read from some filesystem. */);
7763 inherit_process_coding_system
= 0;
7765 DEFVAR_LISP ("file-coding-system-alist", &Vfile_coding_system_alist
,
7766 doc
: /* Alist to decide a coding system to use for a file I/O operation.
7767 The format is ((PATTERN . VAL) ...),
7768 where PATTERN is a regular expression matching a file name,
7769 VAL is a coding system, a cons of coding systems, or a function symbol.
7770 If VAL is a coding system, it is used for both decoding and encoding
7772 If VAL is a cons of coding systems, the car part is used for decoding,
7773 and the cdr part is used for encoding.
7774 If VAL is a function symbol, the function must return a coding system
7775 or a cons of coding systems which are used as above. The function gets
7776 the arguments with which `find-operation-coding-system' was called.
7778 See also the function `find-operation-coding-system'
7779 and the variable `auto-coding-alist'. */);
7780 Vfile_coding_system_alist
= Qnil
;
7782 DEFVAR_LISP ("process-coding-system-alist", &Vprocess_coding_system_alist
,
7783 doc
: /* Alist to decide a coding system to use for a process I/O operation.
7784 The format is ((PATTERN . VAL) ...),
7785 where PATTERN is a regular expression matching a program name,
7786 VAL is a coding system, a cons of coding systems, or a function symbol.
7787 If VAL is a coding system, it is used for both decoding what received
7788 from the program and encoding what sent to the program.
7789 If VAL is a cons of coding systems, the car part is used for decoding,
7790 and the cdr part is used for encoding.
7791 If VAL is a function symbol, the function must return a coding system
7792 or a cons of coding systems which are used as above.
7794 See also the function `find-operation-coding-system'. */);
7795 Vprocess_coding_system_alist
= Qnil
;
7797 DEFVAR_LISP ("network-coding-system-alist", &Vnetwork_coding_system_alist
,
7798 doc
: /* Alist to decide a coding system to use for a network I/O operation.
7799 The format is ((PATTERN . VAL) ...),
7800 where PATTERN is a regular expression matching a network service name
7801 or is a port number to connect to,
7802 VAL is a coding system, a cons of coding systems, or a function symbol.
7803 If VAL is a coding system, it is used for both decoding what received
7804 from the network stream and encoding what sent to the network stream.
7805 If VAL is a cons of coding systems, the car part is used for decoding,
7806 and the cdr part is used for encoding.
7807 If VAL is a function symbol, the function must return a coding system
7808 or a cons of coding systems which are used as above.
7810 See also the function `find-operation-coding-system'. */);
7811 Vnetwork_coding_system_alist
= Qnil
;
7813 DEFVAR_LISP ("locale-coding-system", &Vlocale_coding_system
,
7814 doc
: /* Coding system to use with system messages.
7815 Also used for decoding keyboard input on X Window system. */);
7816 Vlocale_coding_system
= Qnil
;
7818 /* The eol mnemonics are reset in startup.el system-dependently. */
7819 DEFVAR_LISP ("eol-mnemonic-unix", &eol_mnemonic_unix
,
7820 doc
: /* *String displayed in mode line for UNIX-like (LF) end-of-line format. */);
7821 eol_mnemonic_unix
= build_string (":");
7823 DEFVAR_LISP ("eol-mnemonic-dos", &eol_mnemonic_dos
,
7824 doc
: /* *String displayed in mode line for DOS-like (CRLF) end-of-line format. */);
7825 eol_mnemonic_dos
= build_string ("\\");
7827 DEFVAR_LISP ("eol-mnemonic-mac", &eol_mnemonic_mac
,
7828 doc
: /* *String displayed in mode line for MAC-like (CR) end-of-line format. */);
7829 eol_mnemonic_mac
= build_string ("/");
7831 DEFVAR_LISP ("eol-mnemonic-undecided", &eol_mnemonic_undecided
,
7832 doc
: /* *String displayed in mode line when end-of-line format is not yet determined. */);
7833 eol_mnemonic_undecided
= build_string (":");
7835 DEFVAR_LISP ("enable-character-translation", &Venable_character_translation
,
7836 doc
: /* *Non-nil enables character translation while encoding and decoding. */);
7837 Venable_character_translation
= Qt
;
7839 DEFVAR_LISP ("standard-translation-table-for-decode",
7840 &Vstandard_translation_table_for_decode
,
7841 doc
: /* Table for translating characters while decoding. */);
7842 Vstandard_translation_table_for_decode
= Qnil
;
7844 DEFVAR_LISP ("standard-translation-table-for-encode",
7845 &Vstandard_translation_table_for_encode
,
7846 doc
: /* Table for translating characters while encoding. */);
7847 Vstandard_translation_table_for_encode
= Qnil
;
7849 DEFVAR_LISP ("charset-revision-table", &Vcharset_revision_alist
,
7850 doc
: /* Alist of charsets vs revision numbers.
7851 While encoding, if a charset (car part of an element) is found,
7852 designate it with the escape sequence identifying revision (cdr part of the element). */);
7853 Vcharset_revision_alist
= Qnil
;
7855 DEFVAR_LISP ("default-process-coding-system",
7856 &Vdefault_process_coding_system
,
7857 doc
: /* Cons of coding systems used for process I/O by default.
7858 The car part is used for decoding a process output,
7859 the cdr part is used for encoding a text to be sent to a process. */);
7860 Vdefault_process_coding_system
= Qnil
;
7862 DEFVAR_LISP ("latin-extra-code-table", &Vlatin_extra_code_table
,
7863 doc
: /* Table of extra Latin codes in the range 128..159 (inclusive).
7864 This is a vector of length 256.
7865 If Nth element is non-nil, the existence of code N in a file
7866 \(or output of subprocess) doesn't prevent it to be detected as
7867 a coding system of ISO 2022 variant which has a flag
7868 `accept-latin-extra-code' t (e.g. iso-latin-1) on reading a file
7869 or reading output of a subprocess.
7870 Only 128th through 159th elements has a meaning. */);
7871 Vlatin_extra_code_table
= Fmake_vector (make_number (256), Qnil
);
7873 DEFVAR_LISP ("select-safe-coding-system-function",
7874 &Vselect_safe_coding_system_function
,
7875 doc
: /* Function to call to select safe coding system for encoding a text.
7877 If set, this function is called to force a user to select a proper
7878 coding system which can encode the text in the case that a default
7879 coding system used in each operation can't encode the text.
7881 The default value is `select-safe-coding-system' (which see). */);
7882 Vselect_safe_coding_system_function
= Qnil
;
7884 DEFVAR_BOOL ("coding-system-require-warning",
7885 &coding_system_require_warning
,
7886 doc
: /* Internal use only.
7887 If non-nil, on writing a file, `select-safe-coding-system-function' is
7888 called even if `coding-system-for-write' is non-nil. The command
7889 `universal-coding-system-argument' binds this variable to t temporarily. */);
7890 coding_system_require_warning
= 0;
7893 DEFVAR_BOOL ("inhibit-iso-escape-detection",
7894 &inhibit_iso_escape_detection
,
7895 doc
: /* If non-nil, Emacs ignores ISO2022's escape sequence on code detection.
7897 By default, on reading a file, Emacs tries to detect how the text is
7898 encoded. This code detection is sensitive to escape sequences. If
7899 the sequence is valid as ISO2022, the code is determined as one of
7900 the ISO2022 encodings, and the file is decoded by the corresponding
7901 coding system (e.g. `iso-2022-7bit').
7903 However, there may be a case that you want to read escape sequences in
7904 a file as is. In such a case, you can set this variable to non-nil.
7905 Then, as the code detection ignores any escape sequences, no file is
7906 detected as encoded in some ISO2022 encoding. The result is that all
7907 escape sequences become visible in a buffer.
7909 The default value is nil, and it is strongly recommended not to change
7910 it. That is because many Emacs Lisp source files that contain
7911 non-ASCII characters are encoded by the coding system `iso-2022-7bit'
7912 in Emacs's distribution, and they won't be decoded correctly on
7913 reading if you suppress escape sequence detection.
7915 The other way to read escape sequences in a file without decoding is
7916 to explicitly specify some coding system that doesn't use ISO2022's
7917 escape sequence (e.g `latin-1') on reading by \\[universal-coding-system-argument]. */);
7918 inhibit_iso_escape_detection
= 0;
7920 DEFVAR_LISP ("translation-table-for-input", &Vtranslation_table_for_input
,
7921 doc
: /* Char table for translating self-inserting characters.
7922 This is applied to the result of input methods, not their input. See also
7923 `keyboard-translate-table'. */);
7924 Vtranslation_table_for_input
= Qnil
;
7928 emacs_strerror (error_number
)
7933 synchronize_system_messages_locale ();
7934 str
= strerror (error_number
);
7936 if (! NILP (Vlocale_coding_system
))
7938 Lisp_Object dec
= code_convert_string_norecord (build_string (str
),
7939 Vlocale_coding_system
,
7941 str
= (char *) SDATA (dec
);
7949 /* arch-tag: 3a3a2b01-5ff6-4071-9afe-f5b808d9229d
7950 (do not change this comment) */