Daily bump.
[official-gcc.git] / gcc / reg-stack.c
blob1d9ea035cf44f77a09d8bedc2001240e0d646b64
1 /* Register to Stack convert for GNU compiler.
2 Copyright (C) 1992-2021 Free Software Foundation, Inc.
4 This file is part of GCC.
6 GCC is free software; you can redistribute it and/or modify it
7 under the terms of the GNU General Public License as published by
8 the Free Software Foundation; either version 3, or (at your option)
9 any later version.
11 GCC is distributed in the hope that it will be useful, but WITHOUT
12 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
13 or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
14 License for more details.
16 You should have received a copy of the GNU General Public License
17 along with GCC; see the file COPYING3. If not see
18 <http://www.gnu.org/licenses/>. */
20 /* This pass converts stack-like registers from the "flat register
21 file" model that gcc uses, to a stack convention that the 387 uses.
23 * The form of the input:
25 On input, the function consists of insn that have had their
26 registers fully allocated to a set of "virtual" registers. Note that
27 the word "virtual" is used differently here than elsewhere in gcc: for
28 each virtual stack reg, there is a hard reg, but the mapping between
29 them is not known until this pass is run. On output, hard register
30 numbers have been substituted, and various pop and exchange insns have
31 been emitted. The hard register numbers and the virtual register
32 numbers completely overlap - before this pass, all stack register
33 numbers are virtual, and afterward they are all hard.
35 The virtual registers can be manipulated normally by gcc, and their
36 semantics are the same as for normal registers. After the hard
37 register numbers are substituted, the semantics of an insn containing
38 stack-like regs are not the same as for an insn with normal regs: for
39 instance, it is not safe to delete an insn that appears to be a no-op
40 move. In general, no insn containing hard regs should be changed
41 after this pass is done.
43 * The form of the output:
45 After this pass, hard register numbers represent the distance from
46 the current top of stack to the desired register. A reference to
47 FIRST_STACK_REG references the top of stack, FIRST_STACK_REG + 1,
48 represents the register just below that, and so forth. Also, REG_DEAD
49 notes indicate whether or not a stack register should be popped.
51 A "swap" insn looks like a parallel of two patterns, where each
52 pattern is a SET: one sets A to B, the other B to A.
54 A "push" or "load" insn is a SET whose SET_DEST is FIRST_STACK_REG
55 and whose SET_DEST is REG or MEM. Any other SET_DEST, such as PLUS,
56 will replace the existing stack top, not push a new value.
58 A store insn is a SET whose SET_DEST is FIRST_STACK_REG, and whose
59 SET_SRC is REG or MEM.
61 The case where the SET_SRC and SET_DEST are both FIRST_STACK_REG
62 appears ambiguous. As a special case, the presence of a REG_DEAD note
63 for FIRST_STACK_REG differentiates between a load insn and a pop.
65 If a REG_DEAD is present, the insn represents a "pop" that discards
66 the top of the register stack. If there is no REG_DEAD note, then the
67 insn represents a "dup" or a push of the current top of stack onto the
68 stack.
70 * Methodology:
72 Existing REG_DEAD and REG_UNUSED notes for stack registers are
73 deleted and recreated from scratch. REG_DEAD is never created for a
74 SET_DEST, only REG_UNUSED.
76 * asm_operands:
78 There are several rules on the usage of stack-like regs in
79 asm_operands insns. These rules apply only to the operands that are
80 stack-like regs:
82 1. Given a set of input regs that die in an asm_operands, it is
83 necessary to know which are implicitly popped by the asm, and
84 which must be explicitly popped by gcc.
86 An input reg that is implicitly popped by the asm must be
87 explicitly clobbered, unless it is constrained to match an
88 output operand.
90 2. For any input reg that is implicitly popped by an asm, it is
91 necessary to know how to adjust the stack to compensate for the pop.
92 If any non-popped input is closer to the top of the reg-stack than
93 the implicitly popped reg, it would not be possible to know what the
94 stack looked like - it's not clear how the rest of the stack "slides
95 up".
97 All implicitly popped input regs must be closer to the top of
98 the reg-stack than any input that is not implicitly popped.
100 All explicitly referenced input operands may not "skip" a reg.
101 Otherwise we can have holes in the stack.
103 3. It is possible that if an input dies in an insn, reload might
104 use the input reg for an output reload. Consider this example:
106 asm ("foo" : "=t" (a) : "f" (b));
108 This asm says that input B is not popped by the asm, and that
109 the asm pushes a result onto the reg-stack, i.e., the stack is one
110 deeper after the asm than it was before. But, it is possible that
111 reload will think that it can use the same reg for both the input and
112 the output, if input B dies in this insn.
114 If any input operand uses the "f" constraint, all output reg
115 constraints must use the "&" earlyclobber.
117 The asm above would be written as
119 asm ("foo" : "=&t" (a) : "f" (b));
121 4. Some operands need to be in particular places on the stack. All
122 output operands fall in this category - there is no other way to
123 know which regs the outputs appear in unless the user indicates
124 this in the constraints.
126 Output operands must specifically indicate which reg an output
127 appears in after an asm. "=f" is not allowed: the operand
128 constraints must select a class with a single reg.
130 5. Output operands may not be "inserted" between existing stack regs.
131 Since no 387 opcode uses a read/write operand, all output operands
132 are dead before the asm_operands, and are pushed by the asm_operands.
133 It makes no sense to push anywhere but the top of the reg-stack.
135 Output operands must start at the top of the reg-stack: output
136 operands may not "skip" a reg.
138 6. Some asm statements may need extra stack space for internal
139 calculations. This can be guaranteed by clobbering stack registers
140 unrelated to the inputs and outputs.
142 Here are a couple of reasonable asms to want to write. This asm
143 takes one input, which is internally popped, and produces two outputs.
145 asm ("fsincos" : "=t" (cos), "=u" (sin) : "0" (inp));
147 This asm takes two inputs, which are popped by the fyl2xp1 opcode,
148 and replaces them with one output. The user must code the "st(1)"
149 clobber for reg-stack.c to know that fyl2xp1 pops both inputs.
151 asm ("fyl2xp1" : "=t" (result) : "0" (x), "u" (y) : "st(1)");
155 #include "config.h"
156 #include "system.h"
157 #include "coretypes.h"
158 #include "backend.h"
159 #include "target.h"
160 #include "rtl.h"
161 #include "tree.h"
162 #include "df.h"
163 #include "insn-config.h"
164 #include "memmodel.h"
165 #include "regs.h"
166 #include "emit-rtl.h" /* FIXME: Can go away once crtl is moved to rtl.h. */
167 #include "recog.h"
168 #include "varasm.h"
169 #include "rtl-error.h"
170 #include "cfgrtl.h"
171 #include "cfganal.h"
172 #include "cfgbuild.h"
173 #include "cfgcleanup.h"
174 #include "reload.h"
175 #include "tree-pass.h"
176 #include "rtl-iter.h"
177 #include "function-abi.h"
179 #ifdef STACK_REGS
181 /* We use this array to cache info about insns, because otherwise we
182 spend too much time in stack_regs_mentioned_p.
184 Indexed by insn UIDs. A value of zero is uninitialized, one indicates
185 the insn uses stack registers, two indicates the insn does not use
186 stack registers. */
187 static vec<char> stack_regs_mentioned_data;
189 #define REG_STACK_SIZE (LAST_STACK_REG - FIRST_STACK_REG + 1)
191 int regstack_completed = 0;
193 /* This is the basic stack record. TOP is an index into REG[] such
194 that REG[TOP] is the top of stack. If TOP is -1 the stack is empty.
196 If TOP is -2, REG[] is not yet initialized. Stack initialization
197 consists of placing each live reg in array `reg' and setting `top'
198 appropriately.
200 REG_SET indicates which registers are live. */
202 typedef struct stack_def
204 int top; /* index to top stack element */
205 HARD_REG_SET reg_set; /* set of live registers */
206 unsigned char reg[REG_STACK_SIZE];/* register - stack mapping */
207 } *stack_ptr;
209 /* This is used to carry information about basic blocks. It is
210 attached to the AUX field of the standard CFG block. */
212 typedef struct block_info_def
214 struct stack_def stack_in; /* Input stack configuration. */
215 struct stack_def stack_out; /* Output stack configuration. */
216 HARD_REG_SET out_reg_set; /* Stack regs live on output. */
217 int done; /* True if block already converted. */
218 int predecessors; /* Number of predecessors that need
219 to be visited. */
220 } *block_info;
222 #define BLOCK_INFO(B) ((block_info) (B)->aux)
224 /* Passed to change_stack to indicate where to emit insns. */
225 enum emit_where
227 EMIT_AFTER,
228 EMIT_BEFORE
231 /* The block we're currently working on. */
232 static basic_block current_block;
234 /* In the current_block, whether we're processing the first register
235 stack or call instruction, i.e. the regstack is currently the
236 same as BLOCK_INFO(current_block)->stack_in. */
237 static bool starting_stack_p;
239 /* This is the register file for all register after conversion. */
240 static rtx
241 FP_mode_reg[LAST_STACK_REG+1-FIRST_STACK_REG][(int) MAX_MACHINE_MODE];
243 #define FP_MODE_REG(regno,mode) \
244 (FP_mode_reg[(regno)-FIRST_STACK_REG][(int) (mode)])
246 /* Used to initialize uninitialized registers. */
247 static rtx not_a_num;
249 /* Forward declarations */
251 static int stack_regs_mentioned_p (const_rtx pat);
252 static void pop_stack (stack_ptr, int);
253 static rtx *get_true_reg (rtx *);
255 static int check_asm_stack_operands (rtx_insn *);
256 static void get_asm_operands_in_out (rtx, int *, int *);
257 static rtx stack_result (tree);
258 static void replace_reg (rtx *, int);
259 static void remove_regno_note (rtx_insn *, enum reg_note, unsigned int);
260 static int get_hard_regnum (stack_ptr, rtx);
261 static rtx_insn *emit_pop_insn (rtx_insn *, stack_ptr, rtx, enum emit_where);
262 static void swap_to_top (rtx_insn *, stack_ptr, rtx, rtx);
263 static bool move_for_stack_reg (rtx_insn *, stack_ptr, rtx);
264 static bool move_nan_for_stack_reg (rtx_insn *, stack_ptr, rtx);
265 static int swap_rtx_condition_1 (rtx);
266 static int swap_rtx_condition (rtx_insn *);
267 static void compare_for_stack_reg (rtx_insn *, stack_ptr, rtx, bool);
268 static bool subst_stack_regs_pat (rtx_insn *, stack_ptr, rtx);
269 static void subst_asm_stack_regs (rtx_insn *, stack_ptr);
270 static bool subst_stack_regs (rtx_insn *, stack_ptr);
271 static void change_stack (rtx_insn *, stack_ptr, stack_ptr, enum emit_where);
272 static void print_stack (FILE *, stack_ptr);
273 static rtx_insn *next_flags_user (rtx_insn *);
275 /* Return nonzero if any stack register is mentioned somewhere within PAT. */
277 static int
278 stack_regs_mentioned_p (const_rtx pat)
280 const char *fmt;
281 int i;
283 if (STACK_REG_P (pat))
284 return 1;
286 fmt = GET_RTX_FORMAT (GET_CODE (pat));
287 for (i = GET_RTX_LENGTH (GET_CODE (pat)) - 1; i >= 0; i--)
289 if (fmt[i] == 'E')
291 int j;
293 for (j = XVECLEN (pat, i) - 1; j >= 0; j--)
294 if (stack_regs_mentioned_p (XVECEXP (pat, i, j)))
295 return 1;
297 else if (fmt[i] == 'e' && stack_regs_mentioned_p (XEXP (pat, i)))
298 return 1;
301 return 0;
304 /* Return nonzero if INSN mentions stacked registers, else return zero. */
307 stack_regs_mentioned (const_rtx insn)
309 unsigned int uid, max;
310 int test;
312 if (! INSN_P (insn) || !stack_regs_mentioned_data.exists ())
313 return 0;
315 uid = INSN_UID (insn);
316 max = stack_regs_mentioned_data.length ();
317 if (uid >= max)
319 /* Allocate some extra size to avoid too many reallocs, but
320 do not grow too quickly. */
321 max = uid + uid / 20 + 1;
322 stack_regs_mentioned_data.safe_grow_cleared (max, true);
325 test = stack_regs_mentioned_data[uid];
326 if (test == 0)
328 /* This insn has yet to be examined. Do so now. */
329 test = stack_regs_mentioned_p (PATTERN (insn)) ? 1 : 2;
330 stack_regs_mentioned_data[uid] = test;
333 return test == 1;
336 static rtx ix86_flags_rtx;
338 static rtx_insn *
339 next_flags_user (rtx_insn *insn)
341 /* Search forward looking for the first use of this value.
342 Stop at block boundaries. */
344 while (insn != BB_END (current_block))
346 insn = NEXT_INSN (insn);
348 if (INSN_P (insn) && reg_mentioned_p (ix86_flags_rtx, PATTERN (insn)))
349 return insn;
351 if (CALL_P (insn))
352 return NULL;
354 return NULL;
357 /* Reorganize the stack into ascending numbers, before this insn. */
359 static void
360 straighten_stack (rtx_insn *insn, stack_ptr regstack)
362 struct stack_def temp_stack;
363 int top;
365 /* If there is only a single register on the stack, then the stack is
366 already in increasing order and no reorganization is needed.
368 Similarly if the stack is empty. */
369 if (regstack->top <= 0)
370 return;
372 temp_stack.reg_set = regstack->reg_set;
374 for (top = temp_stack.top = regstack->top; top >= 0; top--)
375 temp_stack.reg[top] = FIRST_STACK_REG + temp_stack.top - top;
377 change_stack (insn, regstack, &temp_stack, EMIT_BEFORE);
380 /* Pop a register from the stack. */
382 static void
383 pop_stack (stack_ptr regstack, int regno)
385 int top = regstack->top;
387 CLEAR_HARD_REG_BIT (regstack->reg_set, regno);
388 regstack->top--;
389 /* If regno was not at the top of stack then adjust stack. */
390 if (regstack->reg [top] != regno)
392 int i;
393 for (i = regstack->top; i >= 0; i--)
394 if (regstack->reg [i] == regno)
396 int j;
397 for (j = i; j < top; j++)
398 regstack->reg [j] = regstack->reg [j + 1];
399 break;
404 /* Return a pointer to the REG expression within PAT. If PAT is not a
405 REG, possible enclosed by a conversion rtx, return the inner part of
406 PAT that stopped the search. */
408 static rtx *
409 get_true_reg (rtx *pat)
411 for (;;)
412 switch (GET_CODE (*pat))
414 case SUBREG:
415 /* Eliminate FP subregister accesses in favor of the
416 actual FP register in use. */
418 rtx subreg = SUBREG_REG (*pat);
420 if (STACK_REG_P (subreg))
422 int regno_off = subreg_regno_offset (REGNO (subreg),
423 GET_MODE (subreg),
424 SUBREG_BYTE (*pat),
425 GET_MODE (*pat));
426 *pat = FP_MODE_REG (REGNO (subreg) + regno_off,
427 GET_MODE (subreg));
428 return pat;
430 pat = &XEXP (*pat, 0);
431 break;
434 case FLOAT_TRUNCATE:
435 if (!flag_unsafe_math_optimizations)
436 return pat;
437 /* FALLTHRU */
439 case FLOAT:
440 case FIX:
441 case FLOAT_EXTEND:
442 pat = &XEXP (*pat, 0);
443 break;
445 case UNSPEC:
446 if (XINT (*pat, 1) == UNSPEC_TRUNC_NOOP
447 || XINT (*pat, 1) == UNSPEC_FILD_ATOMIC)
448 pat = &XVECEXP (*pat, 0, 0);
449 return pat;
451 default:
452 return pat;
456 /* Set if we find any malformed asms in a block. */
457 static bool any_malformed_asm;
459 /* There are many rules that an asm statement for stack-like regs must
460 follow. Those rules are explained at the top of this file: the rule
461 numbers below refer to that explanation. */
463 static int
464 check_asm_stack_operands (rtx_insn *insn)
466 int i;
467 int n_clobbers;
468 int malformed_asm = 0;
469 rtx body = PATTERN (insn);
471 char reg_used_as_output[FIRST_PSEUDO_REGISTER];
472 char implicitly_dies[FIRST_PSEUDO_REGISTER];
473 char explicitly_used[FIRST_PSEUDO_REGISTER];
475 rtx *clobber_reg = 0;
476 int n_inputs, n_outputs;
478 /* Find out what the constraints require. If no constraint
479 alternative matches, this asm is malformed. */
480 extract_constrain_insn (insn);
482 preprocess_constraints (insn);
484 get_asm_operands_in_out (body, &n_outputs, &n_inputs);
486 if (which_alternative < 0)
488 /* Avoid further trouble with this insn. */
489 PATTERN (insn) = gen_rtx_USE (VOIDmode, const0_rtx);
490 return 0;
492 const operand_alternative *op_alt = which_op_alt ();
494 /* Strip SUBREGs here to make the following code simpler. */
495 for (i = 0; i < recog_data.n_operands; i++)
496 if (GET_CODE (recog_data.operand[i]) == SUBREG
497 && REG_P (SUBREG_REG (recog_data.operand[i])))
498 recog_data.operand[i] = SUBREG_REG (recog_data.operand[i]);
500 /* Set up CLOBBER_REG. */
502 n_clobbers = 0;
504 if (GET_CODE (body) == PARALLEL)
506 clobber_reg = XALLOCAVEC (rtx, XVECLEN (body, 0));
508 for (i = 0; i < XVECLEN (body, 0); i++)
509 if (GET_CODE (XVECEXP (body, 0, i)) == CLOBBER)
511 rtx clobber = XVECEXP (body, 0, i);
512 rtx reg = XEXP (clobber, 0);
514 if (GET_CODE (reg) == SUBREG && REG_P (SUBREG_REG (reg)))
515 reg = SUBREG_REG (reg);
517 if (STACK_REG_P (reg))
519 clobber_reg[n_clobbers] = reg;
520 n_clobbers++;
525 /* Enforce rule #4: Output operands must specifically indicate which
526 reg an output appears in after an asm. "=f" is not allowed: the
527 operand constraints must select a class with a single reg.
529 Also enforce rule #5: Output operands must start at the top of
530 the reg-stack: output operands may not "skip" a reg. */
532 memset (reg_used_as_output, 0, sizeof (reg_used_as_output));
533 for (i = 0; i < n_outputs; i++)
534 if (STACK_REG_P (recog_data.operand[i]))
536 if (reg_class_size[(int) op_alt[i].cl] != 1)
538 error_for_asm (insn, "output constraint %d must specify a single register", i);
539 malformed_asm = 1;
541 else
543 int j;
545 for (j = 0; j < n_clobbers; j++)
546 if (REGNO (recog_data.operand[i]) == REGNO (clobber_reg[j]))
548 error_for_asm (insn, "output constraint %d cannot be "
549 "specified together with %qs clobber",
550 i, reg_names [REGNO (clobber_reg[j])]);
551 malformed_asm = 1;
552 break;
554 if (j == n_clobbers)
555 reg_used_as_output[REGNO (recog_data.operand[i])] = 1;
560 /* Search for first non-popped reg. */
561 for (i = FIRST_STACK_REG; i < LAST_STACK_REG + 1; i++)
562 if (! reg_used_as_output[i])
563 break;
565 /* If there are any other popped regs, that's an error. */
566 for (; i < LAST_STACK_REG + 1; i++)
567 if (reg_used_as_output[i])
568 break;
570 if (i != LAST_STACK_REG + 1)
572 error_for_asm (insn, "output registers must be grouped at top of stack");
573 malformed_asm = 1;
576 /* Enforce rule #2: All implicitly popped input regs must be closer
577 to the top of the reg-stack than any input that is not implicitly
578 popped. */
580 memset (implicitly_dies, 0, sizeof (implicitly_dies));
581 memset (explicitly_used, 0, sizeof (explicitly_used));
582 for (i = n_outputs; i < n_outputs + n_inputs; i++)
583 if (STACK_REG_P (recog_data.operand[i]))
585 /* An input reg is implicitly popped if it is tied to an
586 output, or if there is a CLOBBER for it. */
587 int j;
589 for (j = 0; j < n_clobbers; j++)
590 if (operands_match_p (clobber_reg[j], recog_data.operand[i]))
591 break;
593 if (j < n_clobbers || op_alt[i].matches >= 0)
594 implicitly_dies[REGNO (recog_data.operand[i])] = 1;
595 else if (reg_class_size[(int) op_alt[i].cl] == 1)
596 explicitly_used[REGNO (recog_data.operand[i])] = 1;
599 /* Search for first non-popped reg. */
600 for (i = FIRST_STACK_REG; i < LAST_STACK_REG + 1; i++)
601 if (! implicitly_dies[i])
602 break;
604 /* If there are any other popped regs, that's an error. */
605 for (; i < LAST_STACK_REG + 1; i++)
606 if (implicitly_dies[i])
607 break;
609 if (i != LAST_STACK_REG + 1)
611 error_for_asm (insn,
612 "implicitly popped registers must be grouped "
613 "at top of stack");
614 malformed_asm = 1;
617 /* Search for first not-explicitly used reg. */
618 for (i = FIRST_STACK_REG; i < LAST_STACK_REG + 1; i++)
619 if (! implicitly_dies[i] && ! explicitly_used[i])
620 break;
622 /* If there are any other explicitly used regs, that's an error. */
623 for (; i < LAST_STACK_REG + 1; i++)
624 if (explicitly_used[i])
625 break;
627 if (i != LAST_STACK_REG + 1)
629 error_for_asm (insn,
630 "explicitly used registers must be grouped "
631 "at top of stack");
632 malformed_asm = 1;
635 /* Enforce rule #3: If any input operand uses the "f" constraint, all
636 output constraints must use the "&" earlyclobber.
638 ??? Detect this more deterministically by having constrain_asm_operands
639 record any earlyclobber. */
641 for (i = n_outputs; i < n_outputs + n_inputs; i++)
642 if (STACK_REG_P (recog_data.operand[i]) && op_alt[i].matches == -1)
644 int j;
646 for (j = 0; j < n_outputs; j++)
647 if (operands_match_p (recog_data.operand[j], recog_data.operand[i]))
649 error_for_asm (insn,
650 "output operand %d must use %<&%> constraint", j);
651 malformed_asm = 1;
655 if (malformed_asm)
657 /* Avoid further trouble with this insn. */
658 PATTERN (insn) = gen_rtx_USE (VOIDmode, const0_rtx);
659 any_malformed_asm = true;
660 return 0;
663 return 1;
666 /* Calculate the number of inputs and outputs in BODY, an
667 asm_operands. N_OPERANDS is the total number of operands, and
668 N_INPUTS and N_OUTPUTS are pointers to ints into which the results are
669 placed. */
671 static void
672 get_asm_operands_in_out (rtx body, int *pout, int *pin)
674 rtx asmop = extract_asm_operands (body);
676 *pin = ASM_OPERANDS_INPUT_LENGTH (asmop);
677 *pout = (recog_data.n_operands
678 - ASM_OPERANDS_INPUT_LENGTH (asmop)
679 - ASM_OPERANDS_LABEL_LENGTH (asmop));
682 /* If current function returns its result in an fp stack register,
683 return the REG. Otherwise, return 0. */
685 static rtx
686 stack_result (tree decl)
688 rtx result;
690 /* If the value is supposed to be returned in memory, then clearly
691 it is not returned in a stack register. */
692 if (aggregate_value_p (DECL_RESULT (decl), decl))
693 return 0;
695 result = DECL_RTL_IF_SET (DECL_RESULT (decl));
696 if (result != 0)
697 result = targetm.calls.function_value (TREE_TYPE (DECL_RESULT (decl)),
698 decl, true);
700 return result != 0 && STACK_REG_P (result) ? result : 0;
705 * This section deals with stack register substitution, and forms the second
706 * pass over the RTL.
709 /* Replace REG, which is a pointer to a stack reg RTX, with an RTX for
710 the desired hard REGNO. */
712 static void
713 replace_reg (rtx *reg, int regno)
715 gcc_assert (IN_RANGE (regno, FIRST_STACK_REG, LAST_STACK_REG));
716 gcc_assert (STACK_REG_P (*reg));
718 gcc_assert (GET_MODE_CLASS (GET_MODE (*reg)) == MODE_FLOAT
719 || GET_MODE_CLASS (GET_MODE (*reg)) == MODE_COMPLEX_FLOAT);
721 *reg = FP_MODE_REG (regno, GET_MODE (*reg));
724 /* Remove a note of type NOTE, which must be found, for register
725 number REGNO from INSN. Remove only one such note. */
727 static void
728 remove_regno_note (rtx_insn *insn, enum reg_note note, unsigned int regno)
730 rtx *note_link, this_rtx;
732 note_link = &REG_NOTES (insn);
733 for (this_rtx = *note_link; this_rtx; this_rtx = XEXP (this_rtx, 1))
734 if (REG_NOTE_KIND (this_rtx) == note
735 && REG_P (XEXP (this_rtx, 0)) && REGNO (XEXP (this_rtx, 0)) == regno)
737 *note_link = XEXP (this_rtx, 1);
738 return;
740 else
741 note_link = &XEXP (this_rtx, 1);
743 gcc_unreachable ();
746 /* Find the hard register number of virtual register REG in REGSTACK.
747 The hard register number is relative to the top of the stack. -1 is
748 returned if the register is not found. */
750 static int
751 get_hard_regnum (stack_ptr regstack, rtx reg)
753 int i;
755 gcc_assert (STACK_REG_P (reg));
757 for (i = regstack->top; i >= 0; i--)
758 if (regstack->reg[i] == REGNO (reg))
759 break;
761 return i >= 0 ? (FIRST_STACK_REG + regstack->top - i) : -1;
764 /* Emit an insn to pop virtual register REG before or after INSN.
765 REGSTACK is the stack state after INSN and is updated to reflect this
766 pop. WHEN is either emit_insn_before or emit_insn_after. A pop insn
767 is represented as a SET whose destination is the register to be popped
768 and source is the top of stack. A death note for the top of stack
769 cases the movdf pattern to pop. */
771 static rtx_insn *
772 emit_pop_insn (rtx_insn *insn, stack_ptr regstack, rtx reg,
773 enum emit_where where)
775 machine_mode raw_mode = reg_raw_mode[FIRST_STACK_REG];
776 rtx_insn *pop_insn;
777 rtx pop_rtx;
778 int hard_regno;
780 /* For complex types take care to pop both halves. These may survive in
781 CLOBBER and USE expressions. */
782 if (COMPLEX_MODE_P (GET_MODE (reg)))
784 rtx reg1 = FP_MODE_REG (REGNO (reg), raw_mode);
785 rtx reg2 = FP_MODE_REG (REGNO (reg) + 1, raw_mode);
787 pop_insn = NULL;
788 if (get_hard_regnum (regstack, reg1) >= 0)
789 pop_insn = emit_pop_insn (insn, regstack, reg1, where);
790 if (get_hard_regnum (regstack, reg2) >= 0)
791 pop_insn = emit_pop_insn (insn, regstack, reg2, where);
792 gcc_assert (pop_insn);
793 return pop_insn;
796 hard_regno = get_hard_regnum (regstack, reg);
798 gcc_assert (hard_regno >= FIRST_STACK_REG);
800 pop_rtx = gen_rtx_SET (FP_MODE_REG (hard_regno, raw_mode),
801 FP_MODE_REG (FIRST_STACK_REG, raw_mode));
803 if (where == EMIT_AFTER)
804 pop_insn = emit_insn_after (pop_rtx, insn);
805 else
806 pop_insn = emit_insn_before (pop_rtx, insn);
808 add_reg_note (pop_insn, REG_DEAD, FP_MODE_REG (FIRST_STACK_REG, raw_mode));
810 regstack->reg[regstack->top - (hard_regno - FIRST_STACK_REG)]
811 = regstack->reg[regstack->top];
812 regstack->top -= 1;
813 CLEAR_HARD_REG_BIT (regstack->reg_set, REGNO (reg));
815 return pop_insn;
818 /* Emit an insn before or after INSN to swap virtual register REG with
819 the top of stack. REGSTACK is the stack state before the swap, and
820 is updated to reflect the swap. A swap insn is represented as a
821 PARALLEL of two patterns: each pattern moves one reg to the other.
823 If REG is already at the top of the stack, no insn is emitted. */
825 static void
826 emit_swap_insn (rtx_insn *insn, stack_ptr regstack, rtx reg)
828 int hard_regno;
829 int other_reg; /* swap regno temps */
830 rtx_insn *i1; /* the stack-reg insn prior to INSN */
831 rtx i1set = NULL_RTX; /* the SET rtx within I1 */
833 hard_regno = get_hard_regnum (regstack, reg);
835 if (hard_regno == FIRST_STACK_REG)
836 return;
837 if (hard_regno == -1)
839 /* Something failed if the register wasn't on the stack. If we had
840 malformed asms, we zapped the instruction itself, but that didn't
841 produce the same pattern of register sets as before. To prevent
842 further failure, adjust REGSTACK to include REG at TOP. */
843 gcc_assert (any_malformed_asm);
844 regstack->reg[++regstack->top] = REGNO (reg);
845 return;
847 gcc_assert (hard_regno >= FIRST_STACK_REG);
849 other_reg = regstack->top - (hard_regno - FIRST_STACK_REG);
850 std::swap (regstack->reg[regstack->top], regstack->reg[other_reg]);
852 /* Find the previous insn involving stack regs, but don't pass a
853 block boundary. */
854 i1 = NULL;
855 if (current_block && insn != BB_HEAD (current_block))
857 rtx_insn *tmp = PREV_INSN (insn);
858 rtx_insn *limit = PREV_INSN (BB_HEAD (current_block));
859 while (tmp != limit)
861 if (LABEL_P (tmp)
862 || CALL_P (tmp)
863 || NOTE_INSN_BASIC_BLOCK_P (tmp)
864 || (NONJUMP_INSN_P (tmp)
865 && stack_regs_mentioned (tmp)))
867 i1 = tmp;
868 break;
870 tmp = PREV_INSN (tmp);
874 if (i1 != NULL_RTX
875 && (i1set = single_set (i1)) != NULL_RTX)
877 rtx i1src = *get_true_reg (&SET_SRC (i1set));
878 rtx i1dest = *get_true_reg (&SET_DEST (i1set));
880 /* If the previous register stack push was from the reg we are to
881 swap with, omit the swap. */
883 if (REG_P (i1dest) && REGNO (i1dest) == FIRST_STACK_REG
884 && REG_P (i1src)
885 && REGNO (i1src) == (unsigned) hard_regno - 1
886 && find_regno_note (i1, REG_DEAD, FIRST_STACK_REG) == NULL_RTX)
887 return;
889 /* If the previous insn wrote to the reg we are to swap with,
890 omit the swap. */
892 if (REG_P (i1dest) && REGNO (i1dest) == (unsigned) hard_regno
893 && REG_P (i1src) && REGNO (i1src) == FIRST_STACK_REG
894 && find_regno_note (i1, REG_DEAD, FIRST_STACK_REG) == NULL_RTX)
895 return;
897 /* Instead of
898 fld a
899 fld b
900 fxch %st(1)
901 just use
902 fld b
903 fld a
904 if possible. Similarly for fld1, fldz, fldpi etc. instead of any
905 of the loads or for float extension from memory. */
907 i1src = SET_SRC (i1set);
908 if (GET_CODE (i1src) == FLOAT_EXTEND)
909 i1src = XEXP (i1src, 0);
910 if (REG_P (i1dest)
911 && REGNO (i1dest) == FIRST_STACK_REG
912 && (MEM_P (i1src) || GET_CODE (i1src) == CONST_DOUBLE)
913 && !side_effects_p (i1src)
914 && hard_regno == FIRST_STACK_REG + 1
915 && i1 != BB_HEAD (current_block))
917 /* i1 is the last insn that involves stack regs before insn, and
918 is known to be a load without other side-effects, i.e. fld b
919 in the above comment. */
920 rtx_insn *i2 = NULL;
921 rtx i2set;
922 rtx_insn *tmp = PREV_INSN (i1);
923 rtx_insn *limit = PREV_INSN (BB_HEAD (current_block));
924 /* Find the previous insn involving stack regs, but don't pass a
925 block boundary. */
926 while (tmp != limit)
928 if (LABEL_P (tmp)
929 || CALL_P (tmp)
930 || NOTE_INSN_BASIC_BLOCK_P (tmp)
931 || (NONJUMP_INSN_P (tmp)
932 && stack_regs_mentioned (tmp)))
934 i2 = tmp;
935 break;
937 tmp = PREV_INSN (tmp);
939 if (i2 != NULL_RTX
940 && (i2set = single_set (i2)) != NULL_RTX)
942 rtx i2dest = *get_true_reg (&SET_DEST (i2set));
943 rtx i2src = SET_SRC (i2set);
944 if (GET_CODE (i2src) == FLOAT_EXTEND)
945 i2src = XEXP (i2src, 0);
946 /* If the last two insns before insn that involve
947 stack regs are loads, where the latter (i1)
948 pushes onto the register stack and thus
949 moves the value from the first load (i2) from
950 %st to %st(1), consider swapping them. */
951 if (REG_P (i2dest)
952 && REGNO (i2dest) == FIRST_STACK_REG
953 && (MEM_P (i2src) || GET_CODE (i2src) == CONST_DOUBLE)
954 /* Ensure i2 doesn't have other side-effects. */
955 && !side_effects_p (i2src)
956 /* And that the two instructions can actually be
957 swapped, i.e. there shouldn't be any stores
958 in between i2 and i1 that might alias with
959 the i1 memory, and the memory address can't
960 use registers set in between i2 and i1. */
961 && !modified_between_p (SET_SRC (i1set), i2, i1))
963 /* Move i1 (fld b above) right before i2 (fld a
964 above. */
965 remove_insn (i1);
966 SET_PREV_INSN (i1) = NULL_RTX;
967 SET_NEXT_INSN (i1) = NULL_RTX;
968 set_block_for_insn (i1, NULL);
969 emit_insn_before (i1, i2);
970 return;
976 /* Avoid emitting the swap if this is the first register stack insn
977 of the current_block. Instead update the current_block's stack_in
978 and let compensate edges take care of this for us. */
979 if (current_block && starting_stack_p)
981 BLOCK_INFO (current_block)->stack_in = *regstack;
982 starting_stack_p = false;
983 return;
986 machine_mode raw_mode = reg_raw_mode[FIRST_STACK_REG];
987 rtx op1 = FP_MODE_REG (hard_regno, raw_mode);
988 rtx op2 = FP_MODE_REG (FIRST_STACK_REG, raw_mode);
989 rtx swap_rtx
990 = gen_rtx_PARALLEL (VOIDmode,
991 gen_rtvec (2, gen_rtx_SET (op1, op2),
992 gen_rtx_SET (op2, op1)));
993 if (i1)
994 emit_insn_after (swap_rtx, i1);
995 else if (current_block)
996 emit_insn_before (swap_rtx, BB_HEAD (current_block));
997 else
998 emit_insn_before (swap_rtx, insn);
1001 /* Emit an insns before INSN to swap virtual register SRC1 with
1002 the top of stack and virtual register SRC2 with second stack
1003 slot. REGSTACK is the stack state before the swaps, and
1004 is updated to reflect the swaps. A swap insn is represented as a
1005 PARALLEL of two patterns: each pattern moves one reg to the other.
1007 If SRC1 and/or SRC2 are already at the right place, no swap insn
1008 is emitted. */
1010 static void
1011 swap_to_top (rtx_insn *insn, stack_ptr regstack, rtx src1, rtx src2)
1013 struct stack_def temp_stack;
1014 int regno, j, k;
1016 temp_stack = *regstack;
1018 /* Place operand 1 at the top of stack. */
1019 regno = get_hard_regnum (&temp_stack, src1);
1020 gcc_assert (regno >= 0);
1021 if (regno != FIRST_STACK_REG)
1023 k = temp_stack.top - (regno - FIRST_STACK_REG);
1024 j = temp_stack.top;
1026 std::swap (temp_stack.reg[j], temp_stack.reg[k]);
1029 /* Place operand 2 next on the stack. */
1030 regno = get_hard_regnum (&temp_stack, src2);
1031 gcc_assert (regno >= 0);
1032 if (regno != FIRST_STACK_REG + 1)
1034 k = temp_stack.top - (regno - FIRST_STACK_REG);
1035 j = temp_stack.top - 1;
1037 std::swap (temp_stack.reg[j], temp_stack.reg[k]);
1040 change_stack (insn, regstack, &temp_stack, EMIT_BEFORE);
1043 /* Handle a move to or from a stack register in PAT, which is in INSN.
1044 REGSTACK is the current stack. Return whether a control flow insn
1045 was deleted in the process. */
1047 static bool
1048 move_for_stack_reg (rtx_insn *insn, stack_ptr regstack, rtx pat)
1050 rtx *psrc = get_true_reg (&SET_SRC (pat));
1051 rtx *pdest = get_true_reg (&SET_DEST (pat));
1052 rtx src, dest;
1053 rtx note;
1054 bool control_flow_insn_deleted = false;
1056 src = *psrc; dest = *pdest;
1058 if (STACK_REG_P (src) && STACK_REG_P (dest))
1060 /* Write from one stack reg to another. If SRC dies here, then
1061 just change the register mapping and delete the insn. */
1063 note = find_regno_note (insn, REG_DEAD, REGNO (src));
1064 if (note)
1066 int i;
1068 /* If this is a no-op move, there must not be a REG_DEAD note. */
1069 gcc_assert (REGNO (src) != REGNO (dest));
1071 for (i = regstack->top; i >= 0; i--)
1072 if (regstack->reg[i] == REGNO (src))
1073 break;
1075 /* The destination must be dead, or life analysis is borked. */
1076 gcc_assert (get_hard_regnum (regstack, dest) < FIRST_STACK_REG);
1078 /* If the source is not live, this is yet another case of
1079 uninitialized variables. Load up a NaN instead. */
1080 if (i < 0)
1081 return move_nan_for_stack_reg (insn, regstack, dest);
1083 /* It is possible that the dest is unused after this insn.
1084 If so, just pop the src. */
1086 if (find_regno_note (insn, REG_UNUSED, REGNO (dest)))
1087 emit_pop_insn (insn, regstack, src, EMIT_AFTER);
1088 else
1090 regstack->reg[i] = REGNO (dest);
1091 SET_HARD_REG_BIT (regstack->reg_set, REGNO (dest));
1092 CLEAR_HARD_REG_BIT (regstack->reg_set, REGNO (src));
1095 control_flow_insn_deleted |= control_flow_insn_p (insn);
1096 delete_insn (insn);
1097 return control_flow_insn_deleted;
1100 /* The source reg does not die. */
1102 /* If this appears to be a no-op move, delete it, or else it
1103 will confuse the machine description output patterns. But if
1104 it is REG_UNUSED, we must pop the reg now, as per-insn processing
1105 for REG_UNUSED will not work for deleted insns. */
1107 if (REGNO (src) == REGNO (dest))
1109 if (find_regno_note (insn, REG_UNUSED, REGNO (dest)))
1110 emit_pop_insn (insn, regstack, dest, EMIT_AFTER);
1112 control_flow_insn_deleted |= control_flow_insn_p (insn);
1113 delete_insn (insn);
1114 return control_flow_insn_deleted;
1117 /* The destination ought to be dead. */
1118 if (get_hard_regnum (regstack, dest) >= FIRST_STACK_REG)
1119 gcc_assert (any_malformed_asm);
1120 else
1122 replace_reg (psrc, get_hard_regnum (regstack, src));
1124 regstack->reg[++regstack->top] = REGNO (dest);
1125 SET_HARD_REG_BIT (regstack->reg_set, REGNO (dest));
1126 replace_reg (pdest, FIRST_STACK_REG);
1129 else if (STACK_REG_P (src))
1131 /* Save from a stack reg to MEM, or possibly integer reg. Since
1132 only top of stack may be saved, emit an exchange first if
1133 needs be. */
1135 emit_swap_insn (insn, regstack, src);
1137 note = find_regno_note (insn, REG_DEAD, REGNO (src));
1138 if (note)
1140 replace_reg (&XEXP (note, 0), FIRST_STACK_REG);
1141 regstack->top--;
1142 CLEAR_HARD_REG_BIT (regstack->reg_set, REGNO (src));
1144 else if ((GET_MODE (src) == XFmode)
1145 && regstack->top < REG_STACK_SIZE - 1)
1147 /* A 387 cannot write an XFmode value to a MEM without
1148 clobbering the source reg. The output code can handle
1149 this by reading back the value from the MEM.
1150 But it is more efficient to use a temp register if one is
1151 available. Push the source value here if the register
1152 stack is not full, and then write the value to memory via
1153 a pop. */
1154 rtx push_rtx;
1155 rtx top_stack_reg = FP_MODE_REG (FIRST_STACK_REG, GET_MODE (src));
1157 push_rtx = gen_movxf (top_stack_reg, top_stack_reg);
1158 emit_insn_before (push_rtx, insn);
1159 add_reg_note (insn, REG_DEAD, top_stack_reg);
1162 replace_reg (psrc, FIRST_STACK_REG);
1164 else
1166 rtx pat = PATTERN (insn);
1168 gcc_assert (STACK_REG_P (dest));
1170 /* Load from MEM, or possibly integer REG or constant, into the
1171 stack regs. The actual target is always the top of the
1172 stack. The stack mapping is changed to reflect that DEST is
1173 now at top of stack. */
1175 /* The destination ought to be dead. However, there is a
1176 special case with i387 UNSPEC_TAN, where destination is live
1177 (an argument to fptan) but inherent load of 1.0 is modelled
1178 as a load from a constant. */
1179 if (GET_CODE (pat) == PARALLEL
1180 && XVECLEN (pat, 0) == 2
1181 && GET_CODE (XVECEXP (pat, 0, 1)) == SET
1182 && GET_CODE (SET_SRC (XVECEXP (pat, 0, 1))) == UNSPEC
1183 && XINT (SET_SRC (XVECEXP (pat, 0, 1)), 1) == UNSPEC_TAN)
1184 emit_swap_insn (insn, regstack, dest);
1185 else
1186 gcc_assert (get_hard_regnum (regstack, dest) < FIRST_STACK_REG
1187 || any_malformed_asm);
1189 gcc_assert (regstack->top < REG_STACK_SIZE);
1191 regstack->reg[++regstack->top] = REGNO (dest);
1192 SET_HARD_REG_BIT (regstack->reg_set, REGNO (dest));
1193 replace_reg (pdest, FIRST_STACK_REG);
1196 return control_flow_insn_deleted;
1199 /* A helper function which replaces INSN with a pattern that loads up
1200 a NaN into DEST, then invokes move_for_stack_reg. */
1202 static bool
1203 move_nan_for_stack_reg (rtx_insn *insn, stack_ptr regstack, rtx dest)
1205 rtx pat;
1207 dest = FP_MODE_REG (REGNO (dest), SFmode);
1208 pat = gen_rtx_SET (dest, not_a_num);
1209 PATTERN (insn) = pat;
1210 INSN_CODE (insn) = -1;
1212 return move_for_stack_reg (insn, regstack, pat);
1215 /* Swap the condition on a branch, if there is one. Return true if we
1216 found a condition to swap. False if the condition was not used as
1217 such. */
1219 static int
1220 swap_rtx_condition_1 (rtx pat)
1222 const char *fmt;
1223 int i, r = 0;
1225 if (COMPARISON_P (pat))
1227 PUT_CODE (pat, swap_condition (GET_CODE (pat)));
1228 r = 1;
1230 else
1232 fmt = GET_RTX_FORMAT (GET_CODE (pat));
1233 for (i = GET_RTX_LENGTH (GET_CODE (pat)) - 1; i >= 0; i--)
1235 if (fmt[i] == 'E')
1237 int j;
1239 for (j = XVECLEN (pat, i) - 1; j >= 0; j--)
1240 r |= swap_rtx_condition_1 (XVECEXP (pat, i, j));
1242 else if (fmt[i] == 'e')
1243 r |= swap_rtx_condition_1 (XEXP (pat, i));
1247 return r;
1250 static int
1251 swap_rtx_condition (rtx_insn *insn)
1253 rtx pat = PATTERN (insn);
1255 /* We're looking for a single set to an HImode temporary. */
1257 if (GET_CODE (pat) == SET
1258 && REG_P (SET_DEST (pat))
1259 && REGNO (SET_DEST (pat)) == FLAGS_REG)
1261 insn = next_flags_user (insn);
1262 if (insn == NULL_RTX)
1263 return 0;
1264 pat = PATTERN (insn);
1267 /* See if this is, or ends in, a fnstsw. If so, we're not doing anything
1268 with the cc value right now. We may be able to search for one
1269 though. */
1271 if (GET_CODE (pat) == SET
1272 && GET_CODE (SET_SRC (pat)) == UNSPEC
1273 && XINT (SET_SRC (pat), 1) == UNSPEC_FNSTSW)
1275 rtx dest = SET_DEST (pat);
1277 /* Search forward looking for the first use of this value.
1278 Stop at block boundaries. */
1279 while (insn != BB_END (current_block))
1281 insn = NEXT_INSN (insn);
1282 if (INSN_P (insn) && reg_mentioned_p (dest, insn))
1283 break;
1284 if (CALL_P (insn))
1285 return 0;
1288 /* We haven't found it. */
1289 if (insn == BB_END (current_block))
1290 return 0;
1292 /* So we've found the insn using this value. If it is anything
1293 other than sahf or the value does not die (meaning we'd have
1294 to search further), then we must give up. */
1295 pat = PATTERN (insn);
1296 if (GET_CODE (pat) != SET
1297 || GET_CODE (SET_SRC (pat)) != UNSPEC
1298 || XINT (SET_SRC (pat), 1) != UNSPEC_SAHF
1299 || ! dead_or_set_p (insn, dest))
1300 return 0;
1302 /* Now we are prepared to handle this. */
1303 insn = next_flags_user (insn);
1304 if (insn == NULL_RTX)
1305 return 0;
1306 pat = PATTERN (insn);
1309 if (swap_rtx_condition_1 (pat))
1311 int fail = 0;
1312 INSN_CODE (insn) = -1;
1313 if (recog_memoized (insn) == -1)
1314 fail = 1;
1315 /* In case the flags don't die here, recurse to try fix
1316 following user too. */
1317 else if (! dead_or_set_p (insn, ix86_flags_rtx))
1319 insn = next_flags_user (insn);
1320 if (!insn || !swap_rtx_condition (insn))
1321 fail = 1;
1323 if (fail)
1325 swap_rtx_condition_1 (pat);
1326 return 0;
1328 return 1;
1330 return 0;
1333 /* Handle a comparison. Special care needs to be taken to avoid
1334 causing comparisons that a 387 cannot do correctly, such as EQ.
1336 Also, a pop insn may need to be emitted. The 387 does have an
1337 `fcompp' insn that can pop two regs, but it is sometimes too expensive
1338 to do this - a `fcomp' followed by a `fstpl %st(0)' may be easier to
1339 set up. */
1341 static void
1342 compare_for_stack_reg (rtx_insn *insn, stack_ptr regstack,
1343 rtx pat_src, bool can_pop_second_op)
1345 rtx *src1, *src2;
1346 rtx src1_note, src2_note;
1348 src1 = get_true_reg (&XEXP (pat_src, 0));
1349 src2 = get_true_reg (&XEXP (pat_src, 1));
1351 /* ??? If fxch turns out to be cheaper than fstp, give priority to
1352 registers that die in this insn - move those to stack top first. */
1353 if ((! STACK_REG_P (*src1)
1354 || (STACK_REG_P (*src2)
1355 && get_hard_regnum (regstack, *src2) == FIRST_STACK_REG))
1356 && swap_rtx_condition (insn))
1358 std::swap (XEXP (pat_src, 0), XEXP (pat_src, 1));
1360 src1 = get_true_reg (&XEXP (pat_src, 0));
1361 src2 = get_true_reg (&XEXP (pat_src, 1));
1363 INSN_CODE (insn) = -1;
1366 /* We will fix any death note later. */
1368 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1370 if (STACK_REG_P (*src2))
1371 src2_note = find_regno_note (insn, REG_DEAD, REGNO (*src2));
1372 else
1373 src2_note = NULL_RTX;
1375 emit_swap_insn (insn, regstack, *src1);
1377 replace_reg (src1, FIRST_STACK_REG);
1379 if (STACK_REG_P (*src2))
1380 replace_reg (src2, get_hard_regnum (regstack, *src2));
1382 if (src1_note)
1384 if (*src2 == CONST0_RTX (GET_MODE (*src2)))
1386 /* This is `ftst' insn that can't pop register. */
1387 remove_regno_note (insn, REG_DEAD, REGNO (XEXP (src1_note, 0)));
1388 emit_pop_insn (insn, regstack, XEXP (src1_note, 0),
1389 EMIT_AFTER);
1391 else
1393 pop_stack (regstack, REGNO (XEXP (src1_note, 0)));
1394 replace_reg (&XEXP (src1_note, 0), FIRST_STACK_REG);
1398 /* If the second operand dies, handle that. But if the operands are
1399 the same stack register, don't bother, because only one death is
1400 needed, and it was just handled. */
1402 if (src2_note
1403 && ! (STACK_REG_P (*src1) && STACK_REG_P (*src2)
1404 && REGNO (*src1) == REGNO (*src2)))
1406 /* As a special case, two regs may die in this insn if src2 is
1407 next to top of stack and the top of stack also dies. Since
1408 we have already popped src1, "next to top of stack" is really
1409 at top (FIRST_STACK_REG) now. */
1411 if (get_hard_regnum (regstack, XEXP (src2_note, 0)) == FIRST_STACK_REG
1412 && src1_note && can_pop_second_op)
1414 pop_stack (regstack, REGNO (XEXP (src2_note, 0)));
1415 replace_reg (&XEXP (src2_note, 0), FIRST_STACK_REG + 1);
1417 else
1419 /* The 386 can only represent death of the first operand in
1420 the case handled above. In all other cases, emit a separate
1421 pop and remove the death note from here. */
1422 remove_regno_note (insn, REG_DEAD, REGNO (XEXP (src2_note, 0)));
1423 emit_pop_insn (insn, regstack, XEXP (src2_note, 0),
1424 EMIT_AFTER);
1429 /* Substitute hardware stack regs in debug insn INSN, using stack
1430 layout REGSTACK. If we can't find a hardware stack reg for any of
1431 the REGs in it, reset the debug insn. */
1433 static void
1434 subst_all_stack_regs_in_debug_insn (rtx_insn *insn, struct stack_def *regstack)
1436 subrtx_ptr_iterator::array_type array;
1437 FOR_EACH_SUBRTX_PTR (iter, array, &INSN_VAR_LOCATION_LOC (insn), NONCONST)
1439 rtx *loc = *iter;
1440 rtx x = *loc;
1441 if (STACK_REG_P (x))
1443 int hard_regno = get_hard_regnum (regstack, x);
1445 /* If we can't find an active register, reset this debug insn. */
1446 if (hard_regno == -1)
1448 INSN_VAR_LOCATION_LOC (insn) = gen_rtx_UNKNOWN_VAR_LOC ();
1449 return;
1452 gcc_assert (hard_regno >= FIRST_STACK_REG);
1453 replace_reg (loc, hard_regno);
1454 iter.skip_subrtxes ();
1459 /* Substitute new registers in PAT, which is part of INSN. REGSTACK
1460 is the current register layout. Return whether a control flow insn
1461 was deleted in the process. */
1463 static bool
1464 subst_stack_regs_pat (rtx_insn *insn, stack_ptr regstack, rtx pat)
1466 rtx *dest, *src;
1467 bool control_flow_insn_deleted = false;
1469 switch (GET_CODE (pat))
1471 case USE:
1472 /* Deaths in USE insns can happen in non optimizing compilation.
1473 Handle them by popping the dying register. */
1474 src = get_true_reg (&XEXP (pat, 0));
1475 if (STACK_REG_P (*src)
1476 && find_regno_note (insn, REG_DEAD, REGNO (*src)))
1478 /* USEs are ignored for liveness information so USEs of dead
1479 register might happen. */
1480 if (TEST_HARD_REG_BIT (regstack->reg_set, REGNO (*src)))
1481 emit_pop_insn (insn, regstack, *src, EMIT_AFTER);
1482 return control_flow_insn_deleted;
1484 /* Uninitialized USE might happen for functions returning uninitialized
1485 value. We will properly initialize the USE on the edge to EXIT_BLOCK,
1486 so it is safe to ignore the use here. This is consistent with behavior
1487 of dataflow analyzer that ignores USE too. (This also imply that
1488 forcibly initializing the register to NaN here would lead to ICE later,
1489 since the REG_DEAD notes are not issued.) */
1490 break;
1492 case VAR_LOCATION:
1493 gcc_unreachable ();
1495 case CLOBBER:
1497 rtx note;
1499 dest = get_true_reg (&XEXP (pat, 0));
1500 if (STACK_REG_P (*dest))
1502 note = find_reg_note (insn, REG_DEAD, *dest);
1504 if (pat != PATTERN (insn))
1506 /* The fix_truncdi_1 pattern wants to be able to
1507 allocate its own scratch register. It does this by
1508 clobbering an fp reg so that it is assured of an
1509 empty reg-stack register. If the register is live,
1510 kill it now. Remove the DEAD/UNUSED note so we
1511 don't try to kill it later too.
1513 In reality the UNUSED note can be absent in some
1514 complicated cases when the register is reused for
1515 partially set variable. */
1517 if (note)
1518 emit_pop_insn (insn, regstack, *dest, EMIT_BEFORE);
1519 else
1520 note = find_reg_note (insn, REG_UNUSED, *dest);
1521 if (note)
1522 remove_note (insn, note);
1523 replace_reg (dest, FIRST_STACK_REG + 1);
1525 else
1527 /* A top-level clobber with no REG_DEAD, and no hard-regnum
1528 indicates an uninitialized value. Because reload removed
1529 all other clobbers, this must be due to a function
1530 returning without a value. Load up a NaN. */
1532 if (!note)
1534 rtx t = *dest;
1535 if (COMPLEX_MODE_P (GET_MODE (t)))
1537 rtx u = FP_MODE_REG (REGNO (t) + 1, SFmode);
1538 if (get_hard_regnum (regstack, u) == -1)
1540 rtx pat2 = gen_rtx_CLOBBER (VOIDmode, u);
1541 rtx_insn *insn2 = emit_insn_before (pat2, insn);
1542 control_flow_insn_deleted
1543 |= move_nan_for_stack_reg (insn2, regstack, u);
1546 if (get_hard_regnum (regstack, t) == -1)
1547 control_flow_insn_deleted
1548 |= move_nan_for_stack_reg (insn, regstack, t);
1552 break;
1555 case SET:
1557 rtx *src1 = (rtx *) 0, *src2;
1558 rtx src1_note, src2_note;
1559 rtx pat_src;
1561 dest = get_true_reg (&SET_DEST (pat));
1562 src = get_true_reg (&SET_SRC (pat));
1563 pat_src = SET_SRC (pat);
1565 /* See if this is a `movM' pattern, and handle elsewhere if so. */
1566 if (STACK_REG_P (*src)
1567 || (STACK_REG_P (*dest)
1568 && (REG_P (*src) || MEM_P (*src)
1569 || CONST_DOUBLE_P (*src))))
1571 control_flow_insn_deleted |= move_for_stack_reg (insn, regstack, pat);
1572 break;
1575 switch (GET_CODE (pat_src))
1577 case CALL:
1579 int count;
1580 for (count = REG_NREGS (*dest); --count >= 0;)
1582 regstack->reg[++regstack->top] = REGNO (*dest) + count;
1583 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest) + count);
1586 replace_reg (dest, FIRST_STACK_REG);
1587 break;
1589 case REG:
1590 gcc_unreachable ();
1592 /* Fall through. */
1594 case FLOAT_TRUNCATE:
1595 case SQRT:
1596 case ABS:
1597 case NEG:
1598 /* These insns only operate on the top of the stack. It's
1599 possible that the tstM case results in a REG_DEAD note on the
1600 source. */
1602 if (src1 == 0)
1603 src1 = get_true_reg (&XEXP (pat_src, 0));
1605 emit_swap_insn (insn, regstack, *src1);
1607 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1609 if (STACK_REG_P (*dest))
1610 replace_reg (dest, FIRST_STACK_REG);
1612 if (src1_note)
1614 replace_reg (&XEXP (src1_note, 0), FIRST_STACK_REG);
1615 regstack->top--;
1616 CLEAR_HARD_REG_BIT (regstack->reg_set, REGNO (*src1));
1619 replace_reg (src1, FIRST_STACK_REG);
1620 break;
1622 case MINUS:
1623 case DIV:
1624 /* On i386, reversed forms of subM3 and divM3 exist for
1625 MODE_FLOAT, so the same code that works for addM3 and mulM3
1626 can be used. */
1627 case MULT:
1628 case PLUS:
1629 /* These insns can accept the top of stack as a destination
1630 from a stack reg or mem, or can use the top of stack as a
1631 source and some other stack register (possibly top of stack)
1632 as a destination. */
1634 src1 = get_true_reg (&XEXP (pat_src, 0));
1635 src2 = get_true_reg (&XEXP (pat_src, 1));
1637 /* We will fix any death note later. */
1639 if (STACK_REG_P (*src1))
1640 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1641 else
1642 src1_note = NULL_RTX;
1643 if (STACK_REG_P (*src2))
1644 src2_note = find_regno_note (insn, REG_DEAD, REGNO (*src2));
1645 else
1646 src2_note = NULL_RTX;
1648 /* If either operand is not a stack register, then the dest
1649 must be top of stack. */
1651 if (! STACK_REG_P (*src1) || ! STACK_REG_P (*src2))
1652 emit_swap_insn (insn, regstack, *dest);
1653 else
1655 /* Both operands are REG. If neither operand is already
1656 at the top of stack, choose to make the one that is the
1657 dest the new top of stack. */
1659 int src1_hard_regnum, src2_hard_regnum;
1661 src1_hard_regnum = get_hard_regnum (regstack, *src1);
1662 src2_hard_regnum = get_hard_regnum (regstack, *src2);
1664 /* If the source is not live, this is yet another case of
1665 uninitialized variables. Load up a NaN instead. */
1666 if (src1_hard_regnum == -1)
1668 rtx pat2 = gen_rtx_CLOBBER (VOIDmode, *src1);
1669 rtx_insn *insn2 = emit_insn_before (pat2, insn);
1670 control_flow_insn_deleted
1671 |= move_nan_for_stack_reg (insn2, regstack, *src1);
1673 if (src2_hard_regnum == -1)
1675 rtx pat2 = gen_rtx_CLOBBER (VOIDmode, *src2);
1676 rtx_insn *insn2 = emit_insn_before (pat2, insn);
1677 control_flow_insn_deleted
1678 |= move_nan_for_stack_reg (insn2, regstack, *src2);
1681 if (src1_hard_regnum != FIRST_STACK_REG
1682 && src2_hard_regnum != FIRST_STACK_REG)
1683 emit_swap_insn (insn, regstack, *dest);
1686 if (STACK_REG_P (*src1))
1687 replace_reg (src1, get_hard_regnum (regstack, *src1));
1688 if (STACK_REG_P (*src2))
1689 replace_reg (src2, get_hard_regnum (regstack, *src2));
1691 if (src1_note)
1693 rtx src1_reg = XEXP (src1_note, 0);
1695 /* If the register that dies is at the top of stack, then
1696 the destination is somewhere else - merely substitute it.
1697 But if the reg that dies is not at top of stack, then
1698 move the top of stack to the dead reg, as though we had
1699 done the insn and then a store-with-pop. */
1701 if (REGNO (src1_reg) == regstack->reg[regstack->top])
1703 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1704 replace_reg (dest, get_hard_regnum (regstack, *dest));
1706 else
1708 int regno = get_hard_regnum (regstack, src1_reg);
1710 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1711 replace_reg (dest, regno);
1713 regstack->reg[regstack->top - (regno - FIRST_STACK_REG)]
1714 = regstack->reg[regstack->top];
1717 CLEAR_HARD_REG_BIT (regstack->reg_set,
1718 REGNO (XEXP (src1_note, 0)));
1719 replace_reg (&XEXP (src1_note, 0), FIRST_STACK_REG);
1720 regstack->top--;
1722 else if (src2_note)
1724 rtx src2_reg = XEXP (src2_note, 0);
1725 if (REGNO (src2_reg) == regstack->reg[regstack->top])
1727 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1728 replace_reg (dest, get_hard_regnum (regstack, *dest));
1730 else
1732 int regno = get_hard_regnum (regstack, src2_reg);
1734 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1735 replace_reg (dest, regno);
1737 regstack->reg[regstack->top - (regno - FIRST_STACK_REG)]
1738 = regstack->reg[regstack->top];
1741 CLEAR_HARD_REG_BIT (regstack->reg_set,
1742 REGNO (XEXP (src2_note, 0)));
1743 replace_reg (&XEXP (src2_note, 0), FIRST_STACK_REG);
1744 regstack->top--;
1746 else
1748 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1749 replace_reg (dest, get_hard_regnum (regstack, *dest));
1752 /* Keep operand 1 matching with destination. */
1753 if (COMMUTATIVE_ARITH_P (pat_src)
1754 && REG_P (*src1) && REG_P (*src2)
1755 && REGNO (*src1) != REGNO (*dest))
1757 int tmp = REGNO (*src1);
1758 replace_reg (src1, REGNO (*src2));
1759 replace_reg (src2, tmp);
1761 break;
1763 case UNSPEC:
1764 switch (XINT (pat_src, 1))
1766 case UNSPEC_FIST:
1767 case UNSPEC_FIST_ATOMIC:
1769 case UNSPEC_FIST_FLOOR:
1770 case UNSPEC_FIST_CEIL:
1772 /* These insns only operate on the top of the stack. */
1774 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1775 emit_swap_insn (insn, regstack, *src1);
1777 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1779 if (STACK_REG_P (*dest))
1780 replace_reg (dest, FIRST_STACK_REG);
1782 if (src1_note)
1784 replace_reg (&XEXP (src1_note, 0), FIRST_STACK_REG);
1785 regstack->top--;
1786 CLEAR_HARD_REG_BIT (regstack->reg_set, REGNO (*src1));
1789 replace_reg (src1, FIRST_STACK_REG);
1790 break;
1792 case UNSPEC_FXAM:
1794 /* This insn only operate on the top of the stack. */
1796 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1797 emit_swap_insn (insn, regstack, *src1);
1799 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1801 replace_reg (src1, FIRST_STACK_REG);
1803 if (src1_note)
1805 remove_regno_note (insn, REG_DEAD,
1806 REGNO (XEXP (src1_note, 0)));
1807 emit_pop_insn (insn, regstack, XEXP (src1_note, 0),
1808 EMIT_AFTER);
1811 break;
1813 case UNSPEC_SIN:
1814 case UNSPEC_COS:
1815 case UNSPEC_FRNDINT:
1816 case UNSPEC_F2XM1:
1818 case UNSPEC_FRNDINT_ROUNDEVEN:
1819 case UNSPEC_FRNDINT_FLOOR:
1820 case UNSPEC_FRNDINT_CEIL:
1821 case UNSPEC_FRNDINT_TRUNC:
1823 /* Above insns operate on the top of the stack. */
1825 case UNSPEC_SINCOS_COS:
1826 case UNSPEC_XTRACT_FRACT:
1828 /* Above insns operate on the top two stack slots,
1829 first part of one input, double output insn. */
1831 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1833 emit_swap_insn (insn, regstack, *src1);
1835 /* Input should never die, it is replaced with output. */
1836 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1837 gcc_assert (!src1_note);
1839 if (STACK_REG_P (*dest))
1840 replace_reg (dest, FIRST_STACK_REG);
1842 replace_reg (src1, FIRST_STACK_REG);
1843 break;
1845 case UNSPEC_SINCOS_SIN:
1846 case UNSPEC_XTRACT_EXP:
1848 /* These insns operate on the top two stack slots,
1849 second part of one input, double output insn. */
1851 regstack->top++;
1852 /* FALLTHRU */
1854 case UNSPEC_TAN:
1856 /* For UNSPEC_TAN, regstack->top is already increased
1857 by inherent load of constant 1.0. */
1859 /* Output value is generated in the second stack slot.
1860 Move current value from second slot to the top. */
1861 regstack->reg[regstack->top]
1862 = regstack->reg[regstack->top - 1];
1864 gcc_assert (STACK_REG_P (*dest));
1866 regstack->reg[regstack->top - 1] = REGNO (*dest);
1867 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1868 replace_reg (dest, FIRST_STACK_REG + 1);
1870 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1872 replace_reg (src1, FIRST_STACK_REG);
1873 break;
1875 case UNSPEC_FPATAN:
1876 case UNSPEC_FYL2X:
1877 case UNSPEC_FYL2XP1:
1878 /* These insns operate on the top two stack slots. */
1880 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1881 src2 = get_true_reg (&XVECEXP (pat_src, 0, 1));
1883 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1884 src2_note = find_regno_note (insn, REG_DEAD, REGNO (*src2));
1886 swap_to_top (insn, regstack, *src1, *src2);
1888 replace_reg (src1, FIRST_STACK_REG);
1889 replace_reg (src2, FIRST_STACK_REG + 1);
1891 if (src1_note)
1892 replace_reg (&XEXP (src1_note, 0), FIRST_STACK_REG);
1893 if (src2_note)
1894 replace_reg (&XEXP (src2_note, 0), FIRST_STACK_REG + 1);
1896 /* Pop both input operands from the stack. */
1897 CLEAR_HARD_REG_BIT (regstack->reg_set,
1898 regstack->reg[regstack->top]);
1899 CLEAR_HARD_REG_BIT (regstack->reg_set,
1900 regstack->reg[regstack->top - 1]);
1901 regstack->top -= 2;
1903 /* Push the result back onto the stack. */
1904 regstack->reg[++regstack->top] = REGNO (*dest);
1905 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1906 replace_reg (dest, FIRST_STACK_REG);
1907 break;
1909 case UNSPEC_FSCALE_FRACT:
1910 case UNSPEC_FPREM_F:
1911 case UNSPEC_FPREM1_F:
1912 /* These insns operate on the top two stack slots,
1913 first part of double input, double output insn. */
1915 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1916 src2 = get_true_reg (&XVECEXP (pat_src, 0, 1));
1918 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
1919 src2_note = find_regno_note (insn, REG_DEAD, REGNO (*src2));
1921 /* Inputs should never die, they are
1922 replaced with outputs. */
1923 gcc_assert (!src1_note);
1924 gcc_assert (!src2_note);
1926 swap_to_top (insn, regstack, *src1, *src2);
1928 /* Push the result back onto stack. Empty stack slot
1929 will be filled in second part of insn. */
1930 if (STACK_REG_P (*dest))
1932 regstack->reg[regstack->top] = REGNO (*dest);
1933 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1934 replace_reg (dest, FIRST_STACK_REG);
1937 replace_reg (src1, FIRST_STACK_REG);
1938 replace_reg (src2, FIRST_STACK_REG + 1);
1939 break;
1941 case UNSPEC_FSCALE_EXP:
1942 case UNSPEC_FPREM_U:
1943 case UNSPEC_FPREM1_U:
1944 /* These insns operate on the top two stack slots,
1945 second part of double input, double output insn. */
1947 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1948 src2 = get_true_reg (&XVECEXP (pat_src, 0, 1));
1950 /* Push the result back onto stack. Fill empty slot from
1951 first part of insn and fix top of stack pointer. */
1952 if (STACK_REG_P (*dest))
1954 regstack->reg[regstack->top - 1] = REGNO (*dest);
1955 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
1956 replace_reg (dest, FIRST_STACK_REG + 1);
1959 replace_reg (src1, FIRST_STACK_REG);
1960 replace_reg (src2, FIRST_STACK_REG + 1);
1961 break;
1963 case UNSPEC_C2_FLAG:
1964 /* This insn operates on the top two stack slots,
1965 third part of C2 setting double input insn. */
1967 src1 = get_true_reg (&XVECEXP (pat_src, 0, 0));
1968 src2 = get_true_reg (&XVECEXP (pat_src, 0, 1));
1970 replace_reg (src1, FIRST_STACK_REG);
1971 replace_reg (src2, FIRST_STACK_REG + 1);
1972 break;
1974 case UNSPEC_FNSTSW:
1975 /* Combined fcomp+fnstsw generated for doing well with
1976 CSE. When optimizing this would have been broken
1977 up before now. */
1979 pat_src = XVECEXP (pat_src, 0, 0);
1980 if (GET_CODE (pat_src) == COMPARE)
1981 goto do_compare;
1983 /* Fall through. */
1985 case UNSPEC_NOTRAP:
1987 pat_src = XVECEXP (pat_src, 0, 0);
1988 gcc_assert (GET_CODE (pat_src) == COMPARE);
1989 goto do_compare;
1991 default:
1992 gcc_unreachable ();
1994 break;
1996 case COMPARE:
1997 do_compare:
1998 /* `fcomi' insn can't pop two regs. */
1999 compare_for_stack_reg (insn, regstack, pat_src,
2000 REGNO (*dest) != FLAGS_REG);
2001 break;
2003 case IF_THEN_ELSE:
2004 /* This insn requires the top of stack to be the destination. */
2006 src1 = get_true_reg (&XEXP (pat_src, 1));
2007 src2 = get_true_reg (&XEXP (pat_src, 2));
2009 src1_note = find_regno_note (insn, REG_DEAD, REGNO (*src1));
2010 src2_note = find_regno_note (insn, REG_DEAD, REGNO (*src2));
2012 /* If the comparison operator is an FP comparison operator,
2013 it is handled correctly by compare_for_stack_reg () who
2014 will move the destination to the top of stack. But if the
2015 comparison operator is not an FP comparison operator, we
2016 have to handle it here. */
2017 if (get_hard_regnum (regstack, *dest) >= FIRST_STACK_REG
2018 && REGNO (*dest) != regstack->reg[regstack->top])
2020 /* In case one of operands is the top of stack and the operands
2021 dies, it is safe to make it the destination operand by
2022 reversing the direction of cmove and avoid fxch. */
2023 if ((REGNO (*src1) == regstack->reg[regstack->top]
2024 && src1_note)
2025 || (REGNO (*src2) == regstack->reg[regstack->top]
2026 && src2_note))
2028 int idx1 = (get_hard_regnum (regstack, *src1)
2029 - FIRST_STACK_REG);
2030 int idx2 = (get_hard_regnum (regstack, *src2)
2031 - FIRST_STACK_REG);
2033 /* Make reg-stack believe that the operands are already
2034 swapped on the stack */
2035 regstack->reg[regstack->top - idx1] = REGNO (*src2);
2036 regstack->reg[regstack->top - idx2] = REGNO (*src1);
2038 /* Reverse condition to compensate the operand swap.
2039 i386 do have comparison always reversible. */
2040 PUT_CODE (XEXP (pat_src, 0),
2041 reversed_comparison_code (XEXP (pat_src, 0), insn));
2043 else
2044 emit_swap_insn (insn, regstack, *dest);
2048 rtx src_note [3];
2049 int i;
2051 src_note[0] = 0;
2052 src_note[1] = src1_note;
2053 src_note[2] = src2_note;
2055 if (STACK_REG_P (*src1))
2056 replace_reg (src1, get_hard_regnum (regstack, *src1));
2057 if (STACK_REG_P (*src2))
2058 replace_reg (src2, get_hard_regnum (regstack, *src2));
2060 for (i = 1; i <= 2; i++)
2061 if (src_note [i])
2063 int regno = REGNO (XEXP (src_note[i], 0));
2065 /* If the register that dies is not at the top of
2066 stack, then move the top of stack to the dead reg.
2067 Top of stack should never die, as it is the
2068 destination. */
2069 gcc_assert (regno != regstack->reg[regstack->top]);
2070 remove_regno_note (insn, REG_DEAD, regno);
2071 emit_pop_insn (insn, regstack, XEXP (src_note[i], 0),
2072 EMIT_AFTER);
2076 /* Make dest the top of stack. Add dest to regstack if
2077 not present. */
2078 if (get_hard_regnum (regstack, *dest) < FIRST_STACK_REG)
2079 regstack->reg[++regstack->top] = REGNO (*dest);
2080 SET_HARD_REG_BIT (regstack->reg_set, REGNO (*dest));
2081 replace_reg (dest, FIRST_STACK_REG);
2082 break;
2084 default:
2085 gcc_unreachable ();
2087 break;
2090 default:
2091 break;
2094 return control_flow_insn_deleted;
2097 /* Substitute hard regnums for any stack regs in INSN, which has
2098 N_INPUTS inputs and N_OUTPUTS outputs. REGSTACK is the stack info
2099 before the insn, and is updated with changes made here.
2101 There are several requirements and assumptions about the use of
2102 stack-like regs in asm statements. These rules are enforced by
2103 record_asm_stack_regs; see comments there for details. Any
2104 asm_operands left in the RTL at this point may be assume to meet the
2105 requirements, since record_asm_stack_regs removes any problem asm. */
2107 static void
2108 subst_asm_stack_regs (rtx_insn *insn, stack_ptr regstack)
2110 rtx body = PATTERN (insn);
2112 rtx *note_reg; /* Array of note contents */
2113 rtx **note_loc; /* Address of REG field of each note */
2114 enum reg_note *note_kind; /* The type of each note */
2116 rtx *clobber_reg = 0;
2117 rtx **clobber_loc = 0;
2119 struct stack_def temp_stack;
2120 int n_notes;
2121 int n_clobbers;
2122 rtx note;
2123 int i;
2124 int n_inputs, n_outputs;
2126 if (! check_asm_stack_operands (insn))
2127 return;
2129 /* Find out what the constraints required. If no constraint
2130 alternative matches, that is a compiler bug: we should have caught
2131 such an insn in check_asm_stack_operands. */
2132 extract_constrain_insn (insn);
2134 preprocess_constraints (insn);
2135 const operand_alternative *op_alt = which_op_alt ();
2137 get_asm_operands_in_out (body, &n_outputs, &n_inputs);
2139 /* Strip SUBREGs here to make the following code simpler. */
2140 for (i = 0; i < recog_data.n_operands; i++)
2141 if (GET_CODE (recog_data.operand[i]) == SUBREG
2142 && REG_P (SUBREG_REG (recog_data.operand[i])))
2144 recog_data.operand_loc[i] = & SUBREG_REG (recog_data.operand[i]);
2145 recog_data.operand[i] = SUBREG_REG (recog_data.operand[i]);
2148 /* Set up NOTE_REG, NOTE_LOC and NOTE_KIND. */
2150 for (i = 0, note = REG_NOTES (insn); note; note = XEXP (note, 1))
2151 i++;
2153 note_reg = XALLOCAVEC (rtx, i);
2154 note_loc = XALLOCAVEC (rtx *, i);
2155 note_kind = XALLOCAVEC (enum reg_note, i);
2157 n_notes = 0;
2158 for (note = REG_NOTES (insn); note; note = XEXP (note, 1))
2160 if (GET_CODE (note) != EXPR_LIST)
2161 continue;
2162 rtx reg = XEXP (note, 0);
2163 rtx *loc = & XEXP (note, 0);
2165 if (GET_CODE (reg) == SUBREG && REG_P (SUBREG_REG (reg)))
2167 loc = & SUBREG_REG (reg);
2168 reg = SUBREG_REG (reg);
2171 if (STACK_REG_P (reg)
2172 && (REG_NOTE_KIND (note) == REG_DEAD
2173 || REG_NOTE_KIND (note) == REG_UNUSED))
2175 note_reg[n_notes] = reg;
2176 note_loc[n_notes] = loc;
2177 note_kind[n_notes] = REG_NOTE_KIND (note);
2178 n_notes++;
2182 /* Set up CLOBBER_REG and CLOBBER_LOC. */
2184 n_clobbers = 0;
2186 if (GET_CODE (body) == PARALLEL)
2188 clobber_reg = XALLOCAVEC (rtx, XVECLEN (body, 0));
2189 clobber_loc = XALLOCAVEC (rtx *, XVECLEN (body, 0));
2191 for (i = 0; i < XVECLEN (body, 0); i++)
2192 if (GET_CODE (XVECEXP (body, 0, i)) == CLOBBER)
2194 rtx clobber = XVECEXP (body, 0, i);
2195 rtx reg = XEXP (clobber, 0);
2196 rtx *loc = & XEXP (clobber, 0);
2198 if (GET_CODE (reg) == SUBREG && REG_P (SUBREG_REG (reg)))
2200 loc = & SUBREG_REG (reg);
2201 reg = SUBREG_REG (reg);
2204 if (STACK_REG_P (reg))
2206 clobber_reg[n_clobbers] = reg;
2207 clobber_loc[n_clobbers] = loc;
2208 n_clobbers++;
2213 temp_stack = *regstack;
2215 /* Put the input regs into the desired place in TEMP_STACK. */
2217 for (i = n_outputs; i < n_outputs + n_inputs; i++)
2218 if (STACK_REG_P (recog_data.operand[i])
2219 && reg_class_subset_p (op_alt[i].cl, FLOAT_REGS)
2220 && op_alt[i].cl != FLOAT_REGS)
2222 /* If an operand needs to be in a particular reg in
2223 FLOAT_REGS, the constraint was either 't' or 'u'. Since
2224 these constraints are for single register classes, and
2225 reload guaranteed that operand[i] is already in that class,
2226 we can just use REGNO (recog_data.operand[i]) to know which
2227 actual reg this operand needs to be in. */
2229 int regno = get_hard_regnum (&temp_stack, recog_data.operand[i]);
2231 gcc_assert (regno >= 0);
2233 if ((unsigned int) regno != REGNO (recog_data.operand[i]))
2235 /* recog_data.operand[i] is not in the right place. Find
2236 it and swap it with whatever is already in I's place.
2237 K is where recog_data.operand[i] is now. J is where it
2238 should be. */
2239 int j, k;
2241 k = temp_stack.top - (regno - FIRST_STACK_REG);
2242 j = (temp_stack.top
2243 - (REGNO (recog_data.operand[i]) - FIRST_STACK_REG));
2245 std::swap (temp_stack.reg[j], temp_stack.reg[k]);
2249 /* Emit insns before INSN to make sure the reg-stack is in the right
2250 order. */
2252 change_stack (insn, regstack, &temp_stack, EMIT_BEFORE);
2254 /* Make the needed input register substitutions. Do death notes and
2255 clobbers too, because these are for inputs, not outputs. */
2257 for (i = n_outputs; i < n_outputs + n_inputs; i++)
2258 if (STACK_REG_P (recog_data.operand[i]))
2260 int regnum = get_hard_regnum (regstack, recog_data.operand[i]);
2262 gcc_assert (regnum >= 0);
2264 replace_reg (recog_data.operand_loc[i], regnum);
2267 for (i = 0; i < n_notes; i++)
2268 if (note_kind[i] == REG_DEAD)
2270 int regnum = get_hard_regnum (regstack, note_reg[i]);
2272 gcc_assert (regnum >= 0);
2274 replace_reg (note_loc[i], regnum);
2277 for (i = 0; i < n_clobbers; i++)
2279 /* It's OK for a CLOBBER to reference a reg that is not live.
2280 Don't try to replace it in that case. */
2281 int regnum = get_hard_regnum (regstack, clobber_reg[i]);
2283 if (regnum >= 0)
2284 replace_reg (clobber_loc[i], regnum);
2287 /* Now remove from REGSTACK any inputs that the asm implicitly popped. */
2289 for (i = n_outputs; i < n_outputs + n_inputs; i++)
2290 if (STACK_REG_P (recog_data.operand[i]))
2292 /* An input reg is implicitly popped if it is tied to an
2293 output, or if there is a CLOBBER for it. */
2294 int j;
2296 for (j = 0; j < n_clobbers; j++)
2297 if (operands_match_p (clobber_reg[j], recog_data.operand[i]))
2298 break;
2300 if (j < n_clobbers || op_alt[i].matches >= 0)
2302 /* recog_data.operand[i] might not be at the top of stack.
2303 But that's OK, because all we need to do is pop the
2304 right number of regs off of the top of the reg-stack.
2305 record_asm_stack_regs guaranteed that all implicitly
2306 popped regs were grouped at the top of the reg-stack. */
2308 CLEAR_HARD_REG_BIT (regstack->reg_set,
2309 regstack->reg[regstack->top]);
2310 regstack->top--;
2314 /* Now add to REGSTACK any outputs that the asm implicitly pushed.
2315 Note that there isn't any need to substitute register numbers.
2316 ??? Explain why this is true. */
2318 for (i = LAST_STACK_REG; i >= FIRST_STACK_REG; i--)
2320 /* See if there is an output for this hard reg. */
2321 int j;
2323 for (j = 0; j < n_outputs; j++)
2324 if (STACK_REG_P (recog_data.operand[j])
2325 && REGNO (recog_data.operand[j]) == (unsigned) i)
2327 regstack->reg[++regstack->top] = i;
2328 SET_HARD_REG_BIT (regstack->reg_set, i);
2329 break;
2333 /* Now emit a pop insn for any REG_UNUSED output, or any REG_DEAD
2334 input that the asm didn't implicitly pop. If the asm didn't
2335 implicitly pop an input reg, that reg will still be live.
2337 Note that we can't use find_regno_note here: the register numbers
2338 in the death notes have already been substituted. */
2340 for (i = 0; i < n_outputs; i++)
2341 if (STACK_REG_P (recog_data.operand[i]))
2343 int j;
2345 for (j = 0; j < n_notes; j++)
2346 if (REGNO (recog_data.operand[i]) == REGNO (note_reg[j])
2347 && note_kind[j] == REG_UNUSED)
2349 insn = emit_pop_insn (insn, regstack, recog_data.operand[i],
2350 EMIT_AFTER);
2351 break;
2355 for (i = n_outputs; i < n_outputs + n_inputs; i++)
2356 if (STACK_REG_P (recog_data.operand[i]))
2358 int j;
2360 for (j = 0; j < n_notes; j++)
2361 if (REGNO (recog_data.operand[i]) == REGNO (note_reg[j])
2362 && note_kind[j] == REG_DEAD
2363 && TEST_HARD_REG_BIT (regstack->reg_set,
2364 REGNO (recog_data.operand[i])))
2366 insn = emit_pop_insn (insn, regstack, recog_data.operand[i],
2367 EMIT_AFTER);
2368 break;
2373 /* Return true if a function call is allowed to alter some or all bits
2374 of any stack reg. */
2375 static bool
2376 callee_clobbers_any_stack_reg (const function_abi & callee_abi)
2378 for (unsigned regno = FIRST_STACK_REG; regno <= LAST_STACK_REG; regno++)
2379 if (callee_abi.clobbers_at_least_part_of_reg_p (regno))
2380 return true;
2381 return false;
2385 /* Substitute stack hard reg numbers for stack virtual registers in
2386 INSN. Non-stack register numbers are not changed. REGSTACK is the
2387 current stack content. Insns may be emitted as needed to arrange the
2388 stack for the 387 based on the contents of the insn. Return whether
2389 a control flow insn was deleted in the process. */
2391 static bool
2392 subst_stack_regs (rtx_insn *insn, stack_ptr regstack)
2394 rtx *note_link, note;
2395 bool control_flow_insn_deleted = false;
2396 int i;
2398 /* If the target of the call doesn't clobber any stack registers,
2399 Don't clear the arguments. */
2400 if (CALL_P (insn)
2401 && callee_clobbers_any_stack_reg (insn_callee_abi (insn)))
2403 int top = regstack->top;
2405 /* If there are any floating point parameters to be passed in
2406 registers for this call, make sure they are in the right
2407 order. */
2409 if (top >= 0)
2411 straighten_stack (insn, regstack);
2413 /* Now mark the arguments as dead after the call. */
2415 while (regstack->top >= 0)
2417 CLEAR_HARD_REG_BIT (regstack->reg_set, FIRST_STACK_REG + regstack->top);
2418 regstack->top--;
2423 /* Do the actual substitution if any stack regs are mentioned.
2424 Since we only record whether entire insn mentions stack regs, and
2425 subst_stack_regs_pat only works for patterns that contain stack regs,
2426 we must check each pattern in a parallel here. A call_value_pop could
2427 fail otherwise. */
2429 if (stack_regs_mentioned (insn))
2431 int n_operands = asm_noperands (PATTERN (insn));
2432 if (n_operands >= 0)
2434 /* This insn is an `asm' with operands. Decode the operands,
2435 decide how many are inputs, and do register substitution.
2436 Any REG_UNUSED notes will be handled by subst_asm_stack_regs. */
2438 subst_asm_stack_regs (insn, regstack);
2439 return control_flow_insn_deleted;
2442 if (GET_CODE (PATTERN (insn)) == PARALLEL)
2443 for (i = 0; i < XVECLEN (PATTERN (insn), 0); i++)
2445 if (stack_regs_mentioned_p (XVECEXP (PATTERN (insn), 0, i)))
2447 if (GET_CODE (XVECEXP (PATTERN (insn), 0, i)) == CLOBBER)
2448 XVECEXP (PATTERN (insn), 0, i)
2449 = shallow_copy_rtx (XVECEXP (PATTERN (insn), 0, i));
2450 control_flow_insn_deleted
2451 |= subst_stack_regs_pat (insn, regstack,
2452 XVECEXP (PATTERN (insn), 0, i));
2455 else
2456 control_flow_insn_deleted
2457 |= subst_stack_regs_pat (insn, regstack, PATTERN (insn));
2460 /* subst_stack_regs_pat may have deleted a no-op insn. If so, any
2461 REG_UNUSED will already have been dealt with, so just return. */
2463 if (NOTE_P (insn) || insn->deleted ())
2464 return control_flow_insn_deleted;
2466 /* If this a noreturn call, we can't insert pop insns after it.
2467 Instead, reset the stack state to empty. */
2468 if (CALL_P (insn)
2469 && find_reg_note (insn, REG_NORETURN, NULL))
2471 regstack->top = -1;
2472 CLEAR_HARD_REG_SET (regstack->reg_set);
2473 return control_flow_insn_deleted;
2476 /* If there is a REG_UNUSED note on a stack register on this insn,
2477 the indicated reg must be popped. The REG_UNUSED note is removed,
2478 since the form of the newly emitted pop insn references the reg,
2479 making it no longer `unset'. */
2481 note_link = &REG_NOTES (insn);
2482 for (note = *note_link; note; note = XEXP (note, 1))
2483 if (REG_NOTE_KIND (note) == REG_UNUSED && STACK_REG_P (XEXP (note, 0)))
2485 *note_link = XEXP (note, 1);
2486 insn = emit_pop_insn (insn, regstack, XEXP (note, 0), EMIT_AFTER);
2488 else
2489 note_link = &XEXP (note, 1);
2491 return control_flow_insn_deleted;
2494 /* Change the organization of the stack so that it fits a new basic
2495 block. Some registers might have to be popped, but there can never be
2496 a register live in the new block that is not now live.
2498 Insert any needed insns before or after INSN, as indicated by
2499 WHERE. OLD is the original stack layout, and NEW is the desired
2500 form. OLD is updated to reflect the code emitted, i.e., it will be
2501 the same as NEW upon return.
2503 This function will not preserve block_end[]. But that information
2504 is no longer needed once this has executed. */
2506 static void
2507 change_stack (rtx_insn *insn, stack_ptr old, stack_ptr new_stack,
2508 enum emit_where where)
2510 int reg;
2511 machine_mode raw_mode = reg_raw_mode[FIRST_STACK_REG];
2512 rtx_insn *update_end = NULL;
2513 int i;
2515 /* Stack adjustments for the first insn in a block update the
2516 current_block's stack_in instead of inserting insns directly.
2517 compensate_edges will add the necessary code later. */
2518 if (current_block
2519 && starting_stack_p
2520 && where == EMIT_BEFORE)
2522 BLOCK_INFO (current_block)->stack_in = *new_stack;
2523 starting_stack_p = false;
2524 *old = *new_stack;
2525 return;
2528 /* We will be inserting new insns "backwards". If we are to insert
2529 after INSN, find the next insn, and insert before it. */
2531 if (where == EMIT_AFTER)
2533 if (current_block && BB_END (current_block) == insn)
2534 update_end = insn;
2535 insn = NEXT_INSN (insn);
2538 /* Initialize partially dead variables. */
2539 for (i = FIRST_STACK_REG; i < LAST_STACK_REG + 1; i++)
2540 if (TEST_HARD_REG_BIT (new_stack->reg_set, i)
2541 && !TEST_HARD_REG_BIT (old->reg_set, i))
2543 old->reg[++old->top] = i;
2544 SET_HARD_REG_BIT (old->reg_set, i);
2545 emit_insn_before (gen_rtx_SET (FP_MODE_REG (i, SFmode), not_a_num),
2546 insn);
2549 /* Pop any registers that are not needed in the new block. */
2551 /* If the destination block's stack already has a specified layout
2552 and contains two or more registers, use a more intelligent algorithm
2553 to pop registers that minimizes the number of fxchs below. */
2554 if (new_stack->top > 0)
2556 bool slots[REG_STACK_SIZE];
2557 int pops[REG_STACK_SIZE];
2558 int next, dest, topsrc;
2560 /* First pass to determine the free slots. */
2561 for (reg = 0; reg <= new_stack->top; reg++)
2562 slots[reg] = TEST_HARD_REG_BIT (new_stack->reg_set, old->reg[reg]);
2564 /* Second pass to allocate preferred slots. */
2565 topsrc = -1;
2566 for (reg = old->top; reg > new_stack->top; reg--)
2567 if (TEST_HARD_REG_BIT (new_stack->reg_set, old->reg[reg]))
2569 dest = -1;
2570 for (next = 0; next <= new_stack->top; next++)
2571 if (!slots[next] && new_stack->reg[next] == old->reg[reg])
2573 /* If this is a preference for the new top of stack, record
2574 the fact by remembering it's old->reg in topsrc. */
2575 if (next == new_stack->top)
2576 topsrc = reg;
2577 slots[next] = true;
2578 dest = next;
2579 break;
2581 pops[reg] = dest;
2583 else
2584 pops[reg] = reg;
2586 /* Intentionally, avoid placing the top of stack in it's correct
2587 location, if we still need to permute the stack below and we
2588 can usefully place it somewhere else. This is the case if any
2589 slot is still unallocated, in which case we should place the
2590 top of stack there. */
2591 if (topsrc != -1)
2592 for (reg = 0; reg < new_stack->top; reg++)
2593 if (!slots[reg])
2595 pops[topsrc] = reg;
2596 slots[new_stack->top] = false;
2597 slots[reg] = true;
2598 break;
2601 /* Third pass allocates remaining slots and emits pop insns. */
2602 next = new_stack->top;
2603 for (reg = old->top; reg > new_stack->top; reg--)
2605 dest = pops[reg];
2606 if (dest == -1)
2608 /* Find next free slot. */
2609 while (slots[next])
2610 next--;
2611 dest = next--;
2613 emit_pop_insn (insn, old, FP_MODE_REG (old->reg[dest], raw_mode),
2614 EMIT_BEFORE);
2617 else
2619 /* The following loop attempts to maximize the number of times we
2620 pop the top of the stack, as this permits the use of the faster
2621 ffreep instruction on platforms that support it. */
2622 int live, next;
2624 live = 0;
2625 for (reg = 0; reg <= old->top; reg++)
2626 if (TEST_HARD_REG_BIT (new_stack->reg_set, old->reg[reg]))
2627 live++;
2629 next = live;
2630 while (old->top >= live)
2631 if (TEST_HARD_REG_BIT (new_stack->reg_set, old->reg[old->top]))
2633 while (TEST_HARD_REG_BIT (new_stack->reg_set, old->reg[next]))
2634 next--;
2635 emit_pop_insn (insn, old, FP_MODE_REG (old->reg[next], raw_mode),
2636 EMIT_BEFORE);
2638 else
2639 emit_pop_insn (insn, old, FP_MODE_REG (old->reg[old->top], raw_mode),
2640 EMIT_BEFORE);
2643 if (new_stack->top == -2)
2645 /* If the new block has never been processed, then it can inherit
2646 the old stack order. */
2648 new_stack->top = old->top;
2649 memcpy (new_stack->reg, old->reg, sizeof (new_stack->reg));
2651 else
2653 /* This block has been entered before, and we must match the
2654 previously selected stack order. */
2656 /* By now, the only difference should be the order of the stack,
2657 not their depth or liveliness. */
2659 gcc_assert (old->reg_set == new_stack->reg_set);
2660 gcc_assert (old->top == new_stack->top);
2662 /* If the stack is not empty (new_stack->top != -1), loop here emitting
2663 swaps until the stack is correct.
2665 The worst case number of swaps emitted is N + 2, where N is the
2666 depth of the stack. In some cases, the reg at the top of
2667 stack may be correct, but swapped anyway in order to fix
2668 other regs. But since we never swap any other reg away from
2669 its correct slot, this algorithm will converge. */
2671 if (new_stack->top != -1)
2674 /* Swap the reg at top of stack into the position it is
2675 supposed to be in, until the correct top of stack appears. */
2677 while (old->reg[old->top] != new_stack->reg[new_stack->top])
2679 for (reg = new_stack->top; reg >= 0; reg--)
2680 if (new_stack->reg[reg] == old->reg[old->top])
2681 break;
2683 gcc_assert (reg != -1);
2685 emit_swap_insn (insn, old,
2686 FP_MODE_REG (old->reg[reg], raw_mode));
2689 /* See if any regs remain incorrect. If so, bring an
2690 incorrect reg to the top of stack, and let the while loop
2691 above fix it. */
2693 for (reg = new_stack->top; reg >= 0; reg--)
2694 if (new_stack->reg[reg] != old->reg[reg])
2696 emit_swap_insn (insn, old,
2697 FP_MODE_REG (old->reg[reg], raw_mode));
2698 break;
2700 } while (reg >= 0);
2702 /* At this point there must be no differences. */
2704 for (reg = old->top; reg >= 0; reg--)
2705 gcc_assert (old->reg[reg] == new_stack->reg[reg]);
2708 if (update_end)
2710 for (update_end = NEXT_INSN (update_end); update_end != insn;
2711 update_end = NEXT_INSN (update_end))
2713 set_block_for_insn (update_end, current_block);
2714 if (INSN_P (update_end))
2715 df_insn_rescan (update_end);
2717 BB_END (current_block) = PREV_INSN (insn);
2721 /* Print stack configuration. */
2723 static void
2724 print_stack (FILE *file, stack_ptr s)
2726 if (! file)
2727 return;
2729 if (s->top == -2)
2730 fprintf (file, "uninitialized\n");
2731 else if (s->top == -1)
2732 fprintf (file, "empty\n");
2733 else
2735 int i;
2736 fputs ("[ ", file);
2737 for (i = 0; i <= s->top; ++i)
2738 fprintf (file, "%d ", s->reg[i]);
2739 fputs ("]\n", file);
2743 /* This function was doing life analysis. We now let the regular live
2744 code do it's job, so we only need to check some extra invariants
2745 that reg-stack expects. Primary among these being that all registers
2746 are initialized before use.
2748 The function returns true when code was emitted to CFG edges and
2749 commit_edge_insertions needs to be called. */
2751 static int
2752 convert_regs_entry (void)
2754 int inserted = 0;
2755 edge e;
2756 edge_iterator ei;
2758 /* Load something into each stack register live at function entry.
2759 Such live registers can be caused by uninitialized variables or
2760 functions not returning values on all paths. In order to keep
2761 the push/pop code happy, and to not scrog the register stack, we
2762 must put something in these registers. Use a QNaN.
2764 Note that we are inserting converted code here. This code is
2765 never seen by the convert_regs pass. */
2767 FOR_EACH_EDGE (e, ei, ENTRY_BLOCK_PTR_FOR_FN (cfun)->succs)
2769 basic_block block = e->dest;
2770 block_info bi = BLOCK_INFO (block);
2771 int reg, top = -1;
2773 for (reg = LAST_STACK_REG; reg >= FIRST_STACK_REG; --reg)
2774 if (TEST_HARD_REG_BIT (bi->stack_in.reg_set, reg))
2776 rtx init;
2778 bi->stack_in.reg[++top] = reg;
2780 init = gen_rtx_SET (FP_MODE_REG (FIRST_STACK_REG, SFmode),
2781 not_a_num);
2782 insert_insn_on_edge (init, e);
2783 inserted = 1;
2786 bi->stack_in.top = top;
2789 return inserted;
2792 /* Construct the desired stack for function exit. This will either
2793 be `empty', or the function return value at top-of-stack. */
2795 static void
2796 convert_regs_exit (void)
2798 int value_reg_low, value_reg_high;
2799 stack_ptr output_stack;
2800 rtx retvalue;
2802 retvalue = stack_result (current_function_decl);
2803 value_reg_low = value_reg_high = -1;
2804 if (retvalue)
2806 value_reg_low = REGNO (retvalue);
2807 value_reg_high = END_REGNO (retvalue) - 1;
2810 output_stack = &BLOCK_INFO (EXIT_BLOCK_PTR_FOR_FN (cfun))->stack_in;
2811 if (value_reg_low == -1)
2812 output_stack->top = -1;
2813 else
2815 int reg;
2817 output_stack->top = value_reg_high - value_reg_low;
2818 for (reg = value_reg_low; reg <= value_reg_high; ++reg)
2820 output_stack->reg[value_reg_high - reg] = reg;
2821 SET_HARD_REG_BIT (output_stack->reg_set, reg);
2826 /* Copy the stack info from the end of edge E's source block to the
2827 start of E's destination block. */
2829 static void
2830 propagate_stack (edge e)
2832 stack_ptr src_stack = &BLOCK_INFO (e->src)->stack_out;
2833 stack_ptr dest_stack = &BLOCK_INFO (e->dest)->stack_in;
2834 int reg;
2836 /* Preserve the order of the original stack, but check whether
2837 any pops are needed. */
2838 dest_stack->top = -1;
2839 for (reg = 0; reg <= src_stack->top; ++reg)
2840 if (TEST_HARD_REG_BIT (dest_stack->reg_set, src_stack->reg[reg]))
2841 dest_stack->reg[++dest_stack->top] = src_stack->reg[reg];
2843 /* Push in any partially dead values. */
2844 for (reg = FIRST_STACK_REG; reg < LAST_STACK_REG + 1; reg++)
2845 if (TEST_HARD_REG_BIT (dest_stack->reg_set, reg)
2846 && !TEST_HARD_REG_BIT (src_stack->reg_set, reg))
2847 dest_stack->reg[++dest_stack->top] = reg;
2851 /* Adjust the stack of edge E's source block on exit to match the stack
2852 of it's target block upon input. The stack layouts of both blocks
2853 should have been defined by now. */
2855 static bool
2856 compensate_edge (edge e)
2858 basic_block source = e->src, target = e->dest;
2859 stack_ptr target_stack = &BLOCK_INFO (target)->stack_in;
2860 stack_ptr source_stack = &BLOCK_INFO (source)->stack_out;
2861 struct stack_def regstack;
2862 int reg;
2864 if (dump_file)
2865 fprintf (dump_file, "Edge %d->%d: ", source->index, target->index);
2867 gcc_assert (target_stack->top != -2);
2869 /* Check whether stacks are identical. */
2870 if (target_stack->top == source_stack->top)
2872 for (reg = target_stack->top; reg >= 0; --reg)
2873 if (target_stack->reg[reg] != source_stack->reg[reg])
2874 break;
2876 if (reg == -1)
2878 if (dump_file)
2879 fprintf (dump_file, "no changes needed\n");
2880 return false;
2884 if (dump_file)
2886 fprintf (dump_file, "correcting stack to ");
2887 print_stack (dump_file, target_stack);
2890 /* Abnormal calls may appear to have values live in st(0), but the
2891 abnormal return path will not have actually loaded the values. */
2892 if (e->flags & EDGE_ABNORMAL_CALL)
2894 /* Assert that the lifetimes are as we expect -- one value
2895 live at st(0) on the end of the source block, and no
2896 values live at the beginning of the destination block.
2897 For complex return values, we may have st(1) live as well. */
2898 gcc_assert (source_stack->top == 0 || source_stack->top == 1);
2899 gcc_assert (target_stack->top == -1);
2900 return false;
2903 /* Handle non-call EH edges specially. The normal return path have
2904 values in registers. These will be popped en masse by the unwind
2905 library. */
2906 if (e->flags & EDGE_EH)
2908 gcc_assert (target_stack->top == -1);
2909 return false;
2912 /* We don't support abnormal edges. Global takes care to
2913 avoid any live register across them, so we should never
2914 have to insert instructions on such edges. */
2915 gcc_assert (! (e->flags & EDGE_ABNORMAL));
2917 /* Make a copy of source_stack as change_stack is destructive. */
2918 regstack = *source_stack;
2920 /* It is better to output directly to the end of the block
2921 instead of to the edge, because emit_swap can do minimal
2922 insn scheduling. We can do this when there is only one
2923 edge out, and it is not abnormal. */
2924 if (EDGE_COUNT (source->succs) == 1)
2926 current_block = source;
2927 change_stack (BB_END (source), &regstack, target_stack,
2928 (JUMP_P (BB_END (source)) ? EMIT_BEFORE : EMIT_AFTER));
2930 else
2932 rtx_insn *seq;
2933 rtx_note *after;
2935 current_block = NULL;
2936 start_sequence ();
2938 /* ??? change_stack needs some point to emit insns after. */
2939 after = emit_note (NOTE_INSN_DELETED);
2941 change_stack (after, &regstack, target_stack, EMIT_BEFORE);
2943 seq = get_insns ();
2944 end_sequence ();
2946 set_insn_locations (seq, e->goto_locus);
2947 insert_insn_on_edge (seq, e);
2948 return true;
2950 return false;
2953 /* Traverse all non-entry edges in the CFG, and emit the necessary
2954 edge compensation code to change the stack from stack_out of the
2955 source block to the stack_in of the destination block. */
2957 static bool
2958 compensate_edges (void)
2960 bool inserted = false;
2961 basic_block bb;
2963 starting_stack_p = false;
2965 FOR_EACH_BB_FN (bb, cfun)
2966 if (bb != ENTRY_BLOCK_PTR_FOR_FN (cfun))
2968 edge e;
2969 edge_iterator ei;
2971 FOR_EACH_EDGE (e, ei, bb->succs)
2972 inserted |= compensate_edge (e);
2974 return inserted;
2977 /* Select the better of two edges E1 and E2 to use to determine the
2978 stack layout for their shared destination basic block. This is
2979 typically the more frequently executed. The edge E1 may be NULL
2980 (in which case E2 is returned), but E2 is always non-NULL. */
2982 static edge
2983 better_edge (edge e1, edge e2)
2985 if (!e1)
2986 return e2;
2988 if (e1->count () > e2->count ())
2989 return e1;
2990 if (e1->count () < e2->count ())
2991 return e2;
2993 /* Prefer critical edges to minimize inserting compensation code on
2994 critical edges. */
2996 if (EDGE_CRITICAL_P (e1) != EDGE_CRITICAL_P (e2))
2997 return EDGE_CRITICAL_P (e1) ? e1 : e2;
2999 /* Avoid non-deterministic behavior. */
3000 return (e1->src->index < e2->src->index) ? e1 : e2;
3003 /* Convert stack register references in one block. Return true if the CFG
3004 has been modified in the process. */
3006 static bool
3007 convert_regs_1 (basic_block block)
3009 struct stack_def regstack;
3010 block_info bi = BLOCK_INFO (block);
3011 int reg;
3012 rtx_insn *insn, *next;
3013 bool control_flow_insn_deleted = false;
3014 bool cfg_altered = false;
3015 int debug_insns_with_starting_stack = 0;
3017 any_malformed_asm = false;
3019 /* Choose an initial stack layout, if one hasn't already been chosen. */
3020 if (bi->stack_in.top == -2)
3022 edge e, beste = NULL;
3023 edge_iterator ei;
3025 /* Select the best incoming edge (typically the most frequent) to
3026 use as a template for this basic block. */
3027 FOR_EACH_EDGE (e, ei, block->preds)
3028 if (BLOCK_INFO (e->src)->done)
3029 beste = better_edge (beste, e);
3031 if (beste)
3032 propagate_stack (beste);
3033 else
3035 /* No predecessors. Create an arbitrary input stack. */
3036 bi->stack_in.top = -1;
3037 for (reg = LAST_STACK_REG; reg >= FIRST_STACK_REG; --reg)
3038 if (TEST_HARD_REG_BIT (bi->stack_in.reg_set, reg))
3039 bi->stack_in.reg[++bi->stack_in.top] = reg;
3043 if (dump_file)
3045 fprintf (dump_file, "\nBasic block %d\nInput stack: ", block->index);
3046 print_stack (dump_file, &bi->stack_in);
3049 /* Process all insns in this block. Keep track of NEXT so that we
3050 don't process insns emitted while substituting in INSN. */
3051 current_block = block;
3052 next = BB_HEAD (block);
3053 regstack = bi->stack_in;
3054 starting_stack_p = true;
3058 insn = next;
3059 next = NEXT_INSN (insn);
3061 /* Ensure we have not missed a block boundary. */
3062 gcc_assert (next);
3063 if (insn == BB_END (block))
3064 next = NULL;
3066 /* Don't bother processing unless there is a stack reg
3067 mentioned or if it's a CALL_INSN. */
3068 if (DEBUG_BIND_INSN_P (insn))
3070 if (starting_stack_p)
3071 debug_insns_with_starting_stack++;
3072 else
3074 subst_all_stack_regs_in_debug_insn (insn, &regstack);
3076 /* Nothing must ever die at a debug insn. If something
3077 is referenced in it that becomes dead, it should have
3078 died before and the reference in the debug insn
3079 should have been removed so as to avoid changing code
3080 generation. */
3081 gcc_assert (!find_reg_note (insn, REG_DEAD, NULL));
3084 else if (stack_regs_mentioned (insn)
3085 || CALL_P (insn))
3087 if (dump_file)
3089 fprintf (dump_file, " insn %d input stack: ",
3090 INSN_UID (insn));
3091 print_stack (dump_file, &regstack);
3093 control_flow_insn_deleted |= subst_stack_regs (insn, &regstack);
3094 starting_stack_p = false;
3097 while (next);
3099 if (debug_insns_with_starting_stack)
3101 /* Since it's the first non-debug instruction that determines
3102 the stack requirements of the current basic block, we refrain
3103 from updating debug insns before it in the loop above, and
3104 fix them up here. */
3105 for (insn = BB_HEAD (block); debug_insns_with_starting_stack;
3106 insn = NEXT_INSN (insn))
3108 if (!DEBUG_BIND_INSN_P (insn))
3109 continue;
3111 debug_insns_with_starting_stack--;
3112 subst_all_stack_regs_in_debug_insn (insn, &bi->stack_in);
3116 if (dump_file)
3118 fprintf (dump_file, "Expected live registers [");
3119 for (reg = FIRST_STACK_REG; reg <= LAST_STACK_REG; ++reg)
3120 if (TEST_HARD_REG_BIT (bi->out_reg_set, reg))
3121 fprintf (dump_file, " %d", reg);
3122 fprintf (dump_file, " ]\nOutput stack: ");
3123 print_stack (dump_file, &regstack);
3126 insn = BB_END (block);
3127 if (JUMP_P (insn))
3128 insn = PREV_INSN (insn);
3130 /* If the function is declared to return a value, but it returns one
3131 in only some cases, some registers might come live here. Emit
3132 necessary moves for them. */
3134 for (reg = FIRST_STACK_REG; reg <= LAST_STACK_REG; ++reg)
3136 if (TEST_HARD_REG_BIT (bi->out_reg_set, reg)
3137 && ! TEST_HARD_REG_BIT (regstack.reg_set, reg))
3139 rtx set;
3141 if (dump_file)
3142 fprintf (dump_file, "Emitting insn initializing reg %d\n", reg);
3144 set = gen_rtx_SET (FP_MODE_REG (reg, SFmode), not_a_num);
3145 insn = emit_insn_after (set, insn);
3146 control_flow_insn_deleted |= subst_stack_regs (insn, &regstack);
3150 /* Amongst the insns possibly deleted during the substitution process above,
3151 might have been the only trapping insn in the block. We purge the now
3152 possibly dead EH edges here to avoid an ICE from fixup_abnormal_edges,
3153 called at the end of convert_regs. The order in which we process the
3154 blocks ensures that we never delete an already processed edge.
3156 Note that, at this point, the CFG may have been damaged by the emission
3157 of instructions after an abnormal call, which moves the basic block end
3158 (and is the reason why we call fixup_abnormal_edges later). So we must
3159 be sure that the trapping insn has been deleted before trying to purge
3160 dead edges, otherwise we risk purging valid edges.
3162 ??? We are normally supposed not to delete trapping insns, so we pretend
3163 that the insns deleted above don't actually trap. It would have been
3164 better to detect this earlier and avoid creating the EH edge in the first
3165 place, still, but we don't have enough information at that time. */
3167 if (control_flow_insn_deleted)
3168 cfg_altered |= purge_dead_edges (block);
3170 /* Something failed if the stack lives don't match. If we had malformed
3171 asms, we zapped the instruction itself, but that didn't produce the
3172 same pattern of register kills as before. */
3174 gcc_assert (regstack.reg_set == bi->out_reg_set || any_malformed_asm);
3175 bi->stack_out = regstack;
3176 bi->done = true;
3178 return cfg_altered;
3181 /* Convert registers in all blocks reachable from BLOCK. Return true if the
3182 CFG has been modified in the process. */
3184 static bool
3185 convert_regs_2 (basic_block block)
3187 basic_block *stack, *sp;
3188 bool cfg_altered = false;
3190 /* We process the blocks in a top-down manner, in a way such that one block
3191 is only processed after all its predecessors. The number of predecessors
3192 of every block has already been computed. */
3194 stack = XNEWVEC (basic_block, n_basic_blocks_for_fn (cfun));
3195 sp = stack;
3197 *sp++ = block;
3201 edge e;
3202 edge_iterator ei;
3204 block = *--sp;
3206 /* Processing BLOCK is achieved by convert_regs_1, which may purge
3207 some dead EH outgoing edge after the deletion of the trapping
3208 insn inside the block. Since the number of predecessors of
3209 BLOCK's successors was computed based on the initial edge set,
3210 we check the necessity to process some of these successors
3211 before such an edge deletion may happen. However, there is
3212 a pitfall: if BLOCK is the only predecessor of a successor and
3213 the edge between them happens to be deleted, the successor
3214 becomes unreachable and should not be processed. The problem
3215 is that there is no way to preventively detect this case so we
3216 stack the successor in all cases and hand over the task of
3217 fixing up the discrepancy to convert_regs_1. */
3219 FOR_EACH_EDGE (e, ei, block->succs)
3220 if (! (e->flags & EDGE_DFS_BACK))
3222 BLOCK_INFO (e->dest)->predecessors--;
3223 if (!BLOCK_INFO (e->dest)->predecessors)
3224 *sp++ = e->dest;
3227 cfg_altered |= convert_regs_1 (block);
3229 while (sp != stack);
3231 free (stack);
3233 return cfg_altered;
3236 /* Traverse all basic blocks in a function, converting the register
3237 references in each insn from the "flat" register file that gcc uses,
3238 to the stack-like registers the 387 uses. */
3240 static void
3241 convert_regs (void)
3243 bool cfg_altered = false;
3244 int inserted;
3245 basic_block b;
3246 edge e;
3247 edge_iterator ei;
3249 /* Initialize uninitialized registers on function entry. */
3250 inserted = convert_regs_entry ();
3252 /* Construct the desired stack for function exit. */
3253 convert_regs_exit ();
3254 BLOCK_INFO (EXIT_BLOCK_PTR_FOR_FN (cfun))->done = 1;
3256 /* ??? Future: process inner loops first, and give them arbitrary
3257 initial stacks which emit_swap_insn can modify. This ought to
3258 prevent double fxch that often appears at the head of a loop. */
3260 /* Process all blocks reachable from all entry points. */
3261 FOR_EACH_EDGE (e, ei, ENTRY_BLOCK_PTR_FOR_FN (cfun)->succs)
3262 cfg_altered |= convert_regs_2 (e->dest);
3264 /* ??? Process all unreachable blocks. Though there's no excuse
3265 for keeping these even when not optimizing. */
3266 FOR_EACH_BB_FN (b, cfun)
3268 block_info bi = BLOCK_INFO (b);
3270 if (! bi->done)
3271 cfg_altered |= convert_regs_2 (b);
3274 /* We must fix up abnormal edges before inserting compensation code
3275 because both mechanisms insert insns on edges. */
3276 inserted |= fixup_abnormal_edges ();
3278 inserted |= compensate_edges ();
3280 clear_aux_for_blocks ();
3282 if (inserted)
3283 commit_edge_insertions ();
3285 if (cfg_altered)
3286 cleanup_cfg (0);
3288 if (dump_file)
3289 fputc ('\n', dump_file);
3292 /* Convert register usage from "flat" register file usage to a "stack
3293 register file. FILE is the dump file, if used.
3295 Construct a CFG and run life analysis. Then convert each insn one
3296 by one. Run a last cleanup_cfg pass, if optimizing, to eliminate
3297 code duplication created when the converter inserts pop insns on
3298 the edges. */
3300 static bool
3301 reg_to_stack (void)
3303 basic_block bb;
3304 int i;
3305 int max_uid;
3307 /* Clean up previous run. */
3308 stack_regs_mentioned_data.release ();
3310 /* See if there is something to do. Flow analysis is quite
3311 expensive so we might save some compilation time. */
3312 for (i = FIRST_STACK_REG; i <= LAST_STACK_REG; i++)
3313 if (df_regs_ever_live_p (i))
3314 break;
3315 if (i > LAST_STACK_REG)
3316 return false;
3318 df_note_add_problem ();
3319 df_analyze ();
3321 mark_dfs_back_edges ();
3323 /* Set up block info for each basic block. */
3324 alloc_aux_for_blocks (sizeof (struct block_info_def));
3325 FOR_EACH_BB_FN (bb, cfun)
3327 block_info bi = BLOCK_INFO (bb);
3328 edge_iterator ei;
3329 edge e;
3330 int reg;
3332 FOR_EACH_EDGE (e, ei, bb->preds)
3333 if (!(e->flags & EDGE_DFS_BACK)
3334 && e->src != ENTRY_BLOCK_PTR_FOR_FN (cfun))
3335 bi->predecessors++;
3337 /* Set current register status at last instruction `uninitialized'. */
3338 bi->stack_in.top = -2;
3340 /* Copy live_at_end and live_at_start into temporaries. */
3341 for (reg = FIRST_STACK_REG; reg <= LAST_STACK_REG; reg++)
3343 if (REGNO_REG_SET_P (DF_LR_OUT (bb), reg))
3344 SET_HARD_REG_BIT (bi->out_reg_set, reg);
3345 if (REGNO_REG_SET_P (DF_LR_IN (bb), reg))
3346 SET_HARD_REG_BIT (bi->stack_in.reg_set, reg);
3350 /* Create the replacement registers up front. */
3351 for (i = FIRST_STACK_REG; i <= LAST_STACK_REG; i++)
3353 machine_mode mode;
3354 FOR_EACH_MODE_IN_CLASS (mode, MODE_FLOAT)
3355 FP_MODE_REG (i, mode) = gen_rtx_REG (mode, i);
3356 FOR_EACH_MODE_IN_CLASS (mode, MODE_COMPLEX_FLOAT)
3357 FP_MODE_REG (i, mode) = gen_rtx_REG (mode, i);
3360 ix86_flags_rtx = gen_rtx_REG (CCmode, FLAGS_REG);
3362 /* A QNaN for initializing uninitialized variables.
3364 ??? We can't load from constant memory in PIC mode, because
3365 we're inserting these instructions before the prologue and
3366 the PIC register hasn't been set up. In that case, fall back
3367 on zero, which we can get from `fldz'. */
3369 if ((flag_pic && !TARGET_64BIT)
3370 || ix86_cmodel == CM_LARGE || ix86_cmodel == CM_LARGE_PIC)
3371 not_a_num = CONST0_RTX (SFmode);
3372 else
3374 REAL_VALUE_TYPE r;
3376 real_nan (&r, "", 1, SFmode);
3377 not_a_num = const_double_from_real_value (r, SFmode);
3378 not_a_num = force_const_mem (SFmode, not_a_num);
3381 /* Allocate a cache for stack_regs_mentioned. */
3382 max_uid = get_max_uid ();
3383 stack_regs_mentioned_data.create (max_uid + 1);
3384 memset (stack_regs_mentioned_data.address (),
3385 0, sizeof (char) * (max_uid + 1));
3387 convert_regs ();
3389 free_aux_for_blocks ();
3390 return true;
3392 #endif /* STACK_REGS */
3394 namespace {
3396 const pass_data pass_data_stack_regs =
3398 RTL_PASS, /* type */
3399 "*stack_regs", /* name */
3400 OPTGROUP_NONE, /* optinfo_flags */
3401 TV_REG_STACK, /* tv_id */
3402 0, /* properties_required */
3403 0, /* properties_provided */
3404 0, /* properties_destroyed */
3405 0, /* todo_flags_start */
3406 0, /* todo_flags_finish */
3409 class pass_stack_regs : public rtl_opt_pass
3411 public:
3412 pass_stack_regs (gcc::context *ctxt)
3413 : rtl_opt_pass (pass_data_stack_regs, ctxt)
3416 /* opt_pass methods: */
3417 virtual bool gate (function *)
3419 #ifdef STACK_REGS
3420 return true;
3421 #else
3422 return false;
3423 #endif
3426 }; // class pass_stack_regs
3428 } // anon namespace
3430 rtl_opt_pass *
3431 make_pass_stack_regs (gcc::context *ctxt)
3433 return new pass_stack_regs (ctxt);
3436 /* Convert register usage from flat register file usage to a stack
3437 register file. */
3438 static unsigned int
3439 rest_of_handle_stack_regs (void)
3441 #ifdef STACK_REGS
3442 if (reg_to_stack ())
3443 df_insn_rescan_all ();
3444 regstack_completed = 1;
3445 #endif
3446 return 0;
3449 namespace {
3451 const pass_data pass_data_stack_regs_run =
3453 RTL_PASS, /* type */
3454 "stack", /* name */
3455 OPTGROUP_NONE, /* optinfo_flags */
3456 TV_REG_STACK, /* tv_id */
3457 0, /* properties_required */
3458 0, /* properties_provided */
3459 0, /* properties_destroyed */
3460 0, /* todo_flags_start */
3461 TODO_df_finish, /* todo_flags_finish */
3464 class pass_stack_regs_run : public rtl_opt_pass
3466 public:
3467 pass_stack_regs_run (gcc::context *ctxt)
3468 : rtl_opt_pass (pass_data_stack_regs_run, ctxt)
3471 /* opt_pass methods: */
3472 virtual unsigned int execute (function *)
3474 return rest_of_handle_stack_regs ();
3477 }; // class pass_stack_regs_run
3479 } // anon namespace
3481 rtl_opt_pass *
3482 make_pass_stack_regs_run (gcc::context *ctxt)
3484 return new pass_stack_regs_run (ctxt);