2016-09-25 François Dumont <fdumont@gcc.gnu.org>
[official-gcc.git] / gcc / reorg.c
blob40f143feddf0086df27f9f9440bb4a6487ba9b1f
1 /* Perform instruction reorganizations for delay slot filling.
2 Copyright (C) 1992-2016 Free Software Foundation, Inc.
3 Contributed by Richard Kenner (kenner@vlsi1.ultra.nyu.edu).
4 Hacked by Michael Tiemann (tiemann@cygnus.com).
6 This file is part of GCC.
8 GCC is free software; you can redistribute it and/or modify it under
9 the terms of the GNU General Public License as published by the Free
10 Software Foundation; either version 3, or (at your option) any later
11 version.
13 GCC is distributed in the hope that it will be useful, but WITHOUT ANY
14 WARRANTY; without even the implied warranty of MERCHANTABILITY or
15 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
16 for more details.
18 You should have received a copy of the GNU General Public License
19 along with GCC; see the file COPYING3. If not see
20 <http://www.gnu.org/licenses/>. */
22 /* Instruction reorganization pass.
24 This pass runs after register allocation and final jump
25 optimization. It should be the last pass to run before peephole.
26 It serves primarily to fill delay slots of insns, typically branch
27 and call insns. Other insns typically involve more complicated
28 interactions of data dependencies and resource constraints, and
29 are better handled by scheduling before register allocation (by the
30 function `schedule_insns').
32 The Branch Penalty is the number of extra cycles that are needed to
33 execute a branch insn. On an ideal machine, branches take a single
34 cycle, and the Branch Penalty is 0. Several RISC machines approach
35 branch delays differently:
37 The MIPS has a single branch delay slot. Most insns
38 (except other branches) can be used to fill this slot. When the
39 slot is filled, two insns execute in two cycles, reducing the
40 branch penalty to zero.
42 The SPARC always has a branch delay slot, but its effects can be
43 annulled when the branch is not taken. This means that failing to
44 find other sources of insns, we can hoist an insn from the branch
45 target that would only be safe to execute knowing that the branch
46 is taken.
48 The HP-PA always has a branch delay slot. For unconditional branches
49 its effects can be annulled when the branch is taken. The effects
50 of the delay slot in a conditional branch can be nullified for forward
51 taken branches, or for untaken backward branches. This means
52 we can hoist insns from the fall-through path for forward branches or
53 steal insns from the target of backward branches.
55 The TMS320C3x and C4x have three branch delay slots. When the three
56 slots are filled, the branch penalty is zero. Most insns can fill the
57 delay slots except jump insns.
59 Three techniques for filling delay slots have been implemented so far:
61 (1) `fill_simple_delay_slots' is the simplest, most efficient way
62 to fill delay slots. This pass first looks for insns which come
63 from before the branch and which are safe to execute after the
64 branch. Then it searches after the insn requiring delay slots or,
65 in the case of a branch, for insns that are after the point at
66 which the branch merges into the fallthrough code, if such a point
67 exists. When such insns are found, the branch penalty decreases
68 and no code expansion takes place.
70 (2) `fill_eager_delay_slots' is more complicated: it is used for
71 scheduling conditional jumps, or for scheduling jumps which cannot
72 be filled using (1). A machine need not have annulled jumps to use
73 this strategy, but it helps (by keeping more options open).
74 `fill_eager_delay_slots' tries to guess the direction the branch
75 will go; if it guesses right 100% of the time, it can reduce the
76 branch penalty as much as `fill_simple_delay_slots' does. If it
77 guesses wrong 100% of the time, it might as well schedule nops. When
78 `fill_eager_delay_slots' takes insns from the fall-through path of
79 the jump, usually there is no code expansion; when it takes insns
80 from the branch target, there is code expansion if it is not the
81 only way to reach that target.
83 (3) `relax_delay_slots' uses a set of rules to simplify code that
84 has been reorganized by (1) and (2). It finds cases where
85 conditional test can be eliminated, jumps can be threaded, extra
86 insns can be eliminated, etc. It is the job of (1) and (2) to do a
87 good job of scheduling locally; `relax_delay_slots' takes care of
88 making the various individual schedules work well together. It is
89 especially tuned to handle the control flow interactions of branch
90 insns. It does nothing for insns with delay slots that do not
91 branch.
93 On machines that use CC0, we are very conservative. We will not make
94 a copy of an insn involving CC0 since we want to maintain a 1-1
95 correspondence between the insn that sets and uses CC0. The insns are
96 allowed to be separated by placing an insn that sets CC0 (but not an insn
97 that uses CC0; we could do this, but it doesn't seem worthwhile) in a
98 delay slot. In that case, we point each insn at the other with REG_CC_USER
99 and REG_CC_SETTER notes. Note that these restrictions affect very few
100 machines because most RISC machines with delay slots will not use CC0
101 (the RT is the only known exception at this point). */
103 #include "config.h"
104 #include "system.h"
105 #include "coretypes.h"
106 #include "backend.h"
107 #include "target.h"
108 #include "rtl.h"
109 #include "tree.h"
110 #include "predict.h"
111 #include "tm_p.h"
112 #include "expmed.h"
113 #include "insn-config.h"
114 #include "emit-rtl.h"
115 #include "recog.h"
116 #include "insn-attr.h"
117 #include "resource.h"
118 #include "params.h"
119 #include "tree-pass.h"
122 /* First, some functions that were used before GCC got a control flow graph.
123 These functions are now only used here in reorg.c, and have therefore
124 been moved here to avoid inadvertent misuse elsewhere in the compiler. */
126 /* Return the last label to mark the same position as LABEL. Return LABEL
127 itself if it is null or any return rtx. */
129 static rtx
130 skip_consecutive_labels (rtx label_or_return)
132 rtx_insn *insn;
134 if (label_or_return && ANY_RETURN_P (label_or_return))
135 return label_or_return;
137 rtx_insn *label = as_a <rtx_insn *> (label_or_return);
139 for (insn = label; insn != 0 && !INSN_P (insn); insn = NEXT_INSN (insn))
140 if (LABEL_P (insn))
141 label = insn;
143 return label;
146 /* INSN uses CC0 and is being moved into a delay slot. Set up REG_CC_SETTER
147 and REG_CC_USER notes so we can find it. */
149 static void
150 link_cc0_insns (rtx_insn *insn)
152 rtx user = next_nonnote_insn (insn);
154 if (NONJUMP_INSN_P (user) && GET_CODE (PATTERN (user)) == SEQUENCE)
155 user = XVECEXP (PATTERN (user), 0, 0);
157 add_reg_note (user, REG_CC_SETTER, insn);
158 add_reg_note (insn, REG_CC_USER, user);
161 /* Insns which have delay slots that have not yet been filled. */
163 static struct obstack unfilled_slots_obstack;
164 static rtx *unfilled_firstobj;
166 /* Define macros to refer to the first and last slot containing unfilled
167 insns. These are used because the list may move and its address
168 should be recomputed at each use. */
170 #define unfilled_slots_base \
171 ((rtx_insn **) obstack_base (&unfilled_slots_obstack))
173 #define unfilled_slots_next \
174 ((rtx_insn **) obstack_next_free (&unfilled_slots_obstack))
176 /* Points to the label before the end of the function, or before a
177 return insn. */
178 static rtx_code_label *function_return_label;
179 /* Likewise for a simple_return. */
180 static rtx_code_label *function_simple_return_label;
182 /* Mapping between INSN_UID's and position in the code since INSN_UID's do
183 not always monotonically increase. */
184 static int *uid_to_ruid;
186 /* Highest valid index in `uid_to_ruid'. */
187 static int max_uid;
189 static int stop_search_p (rtx_insn *, int);
190 static int resource_conflicts_p (struct resources *, struct resources *);
191 static int insn_references_resource_p (rtx, struct resources *, bool);
192 static int insn_sets_resource_p (rtx, struct resources *, bool);
193 static rtx_code_label *find_end_label (rtx);
194 static rtx_insn *emit_delay_sequence (rtx_insn *, const vec<rtx_insn *> &,
195 int);
196 static void add_to_delay_list (rtx_insn *, vec<rtx_insn *> *);
197 static rtx_insn *delete_from_delay_slot (rtx_insn *);
198 static void delete_scheduled_jump (rtx_insn *);
199 static void note_delay_statistics (int, int);
200 static int get_jump_flags (const rtx_insn *, rtx);
201 static int mostly_true_jump (rtx);
202 static rtx get_branch_condition (const rtx_insn *, rtx);
203 static int condition_dominates_p (rtx, const rtx_insn *);
204 static int redirect_with_delay_slots_safe_p (rtx_insn *, rtx, rtx);
205 static int redirect_with_delay_list_safe_p (rtx_insn *, rtx,
206 const vec<rtx_insn *> &);
207 static int check_annul_list_true_false (int, const vec<rtx_insn *> &);
208 static void steal_delay_list_from_target (rtx_insn *, rtx, rtx_sequence *,
209 vec<rtx_insn *> *,
210 struct resources *,
211 struct resources *,
212 struct resources *,
213 int, int *, int *,
214 rtx *);
215 static void steal_delay_list_from_fallthrough (rtx_insn *, rtx, rtx_sequence *,
216 vec<rtx_insn *> *,
217 struct resources *,
218 struct resources *,
219 struct resources *,
220 int, int *, int *);
221 static void try_merge_delay_insns (rtx_insn *, rtx_insn *);
222 static rtx_insn *redundant_insn (rtx, rtx_insn *, const vec<rtx_insn *> &);
223 static int own_thread_p (rtx, rtx, int);
224 static void update_block (rtx_insn *, rtx);
225 static int reorg_redirect_jump (rtx_jump_insn *, rtx);
226 static void update_reg_dead_notes (rtx_insn *, rtx_insn *);
227 static void fix_reg_dead_note (rtx_insn *, rtx);
228 static void update_reg_unused_notes (rtx, rtx);
229 static void fill_simple_delay_slots (int);
230 static void fill_slots_from_thread (rtx_jump_insn *, rtx, rtx, rtx,
231 int, int, int, int,
232 int *, vec<rtx_insn *> *);
233 static void fill_eager_delay_slots (void);
234 static void relax_delay_slots (rtx_insn *);
235 static void make_return_insns (rtx_insn *);
237 /* A wrapper around next_active_insn which takes care to return ret_rtx
238 unchanged. */
240 static rtx
241 first_active_target_insn (rtx insn)
243 if (ANY_RETURN_P (insn))
244 return insn;
245 return next_active_insn (as_a <rtx_insn *> (insn));
248 /* Return true iff INSN is a simplejump, or any kind of return insn. */
250 static bool
251 simplejump_or_return_p (rtx insn)
253 return (JUMP_P (insn)
254 && (simplejump_p (as_a <rtx_insn *> (insn))
255 || ANY_RETURN_P (PATTERN (insn))));
258 /* Return TRUE if this insn should stop the search for insn to fill delay
259 slots. LABELS_P indicates that labels should terminate the search.
260 In all cases, jumps terminate the search. */
262 static int
263 stop_search_p (rtx_insn *insn, int labels_p)
265 if (insn == 0)
266 return 1;
268 /* If the insn can throw an exception that is caught within the function,
269 it may effectively perform a jump from the viewpoint of the function.
270 Therefore act like for a jump. */
271 if (can_throw_internal (insn))
272 return 1;
274 switch (GET_CODE (insn))
276 case NOTE:
277 case CALL_INSN:
278 return 0;
280 case CODE_LABEL:
281 return labels_p;
283 case JUMP_INSN:
284 case BARRIER:
285 return 1;
287 case INSN:
288 /* OK unless it contains a delay slot or is an `asm' insn of some type.
289 We don't know anything about these. */
290 return (GET_CODE (PATTERN (insn)) == SEQUENCE
291 || GET_CODE (PATTERN (insn)) == ASM_INPUT
292 || asm_noperands (PATTERN (insn)) >= 0);
294 default:
295 gcc_unreachable ();
299 /* Return TRUE if any resources are marked in both RES1 and RES2 or if either
300 resource set contains a volatile memory reference. Otherwise, return FALSE. */
302 static int
303 resource_conflicts_p (struct resources *res1, struct resources *res2)
305 if ((res1->cc && res2->cc) || (res1->memory && res2->memory)
306 || res1->volatil || res2->volatil)
307 return 1;
309 return hard_reg_set_intersect_p (res1->regs, res2->regs);
312 /* Return TRUE if any resource marked in RES, a `struct resources', is
313 referenced by INSN. If INCLUDE_DELAYED_EFFECTS is set, return if the called
314 routine is using those resources.
316 We compute this by computing all the resources referenced by INSN and
317 seeing if this conflicts with RES. It might be faster to directly check
318 ourselves, and this is the way it used to work, but it means duplicating
319 a large block of complex code. */
321 static int
322 insn_references_resource_p (rtx insn, struct resources *res,
323 bool include_delayed_effects)
325 struct resources insn_res;
327 CLEAR_RESOURCE (&insn_res);
328 mark_referenced_resources (insn, &insn_res, include_delayed_effects);
329 return resource_conflicts_p (&insn_res, res);
332 /* Return TRUE if INSN modifies resources that are marked in RES.
333 INCLUDE_DELAYED_EFFECTS is set if the actions of that routine should be
334 included. CC0 is only modified if it is explicitly set; see comments
335 in front of mark_set_resources for details. */
337 static int
338 insn_sets_resource_p (rtx insn, struct resources *res,
339 bool include_delayed_effects)
341 struct resources insn_sets;
343 CLEAR_RESOURCE (&insn_sets);
344 mark_set_resources (insn, &insn_sets, 0,
345 (include_delayed_effects
346 ? MARK_SRC_DEST_CALL
347 : MARK_SRC_DEST));
348 return resource_conflicts_p (&insn_sets, res);
351 /* Find a label at the end of the function or before a RETURN. If there
352 is none, try to make one. If that fails, returns 0.
354 The property of such a label is that it is placed just before the
355 epilogue or a bare RETURN insn, so that another bare RETURN can be
356 turned into a jump to the label unconditionally. In particular, the
357 label cannot be placed before a RETURN insn with a filled delay slot.
359 ??? There may be a problem with the current implementation. Suppose
360 we start with a bare RETURN insn and call find_end_label. It may set
361 function_return_label just before the RETURN. Suppose the machinery
362 is able to fill the delay slot of the RETURN insn afterwards. Then
363 function_return_label is no longer valid according to the property
364 described above and find_end_label will still return it unmodified.
365 Note that this is probably mitigated by the following observation:
366 once function_return_label is made, it is very likely the target of
367 a jump, so filling the delay slot of the RETURN will be much more
368 difficult.
369 KIND is either simple_return_rtx or ret_rtx, indicating which type of
370 return we're looking for. */
372 static rtx_code_label *
373 find_end_label (rtx kind)
375 rtx_insn *insn;
376 rtx_code_label **plabel;
378 if (kind == ret_rtx)
379 plabel = &function_return_label;
380 else
382 gcc_assert (kind == simple_return_rtx);
383 plabel = &function_simple_return_label;
386 /* If we found one previously, return it. */
387 if (*plabel)
388 return *plabel;
390 /* Otherwise, see if there is a label at the end of the function. If there
391 is, it must be that RETURN insns aren't needed, so that is our return
392 label and we don't have to do anything else. */
394 insn = get_last_insn ();
395 while (NOTE_P (insn)
396 || (NONJUMP_INSN_P (insn)
397 && (GET_CODE (PATTERN (insn)) == USE
398 || GET_CODE (PATTERN (insn)) == CLOBBER)))
399 insn = PREV_INSN (insn);
401 /* When a target threads its epilogue we might already have a
402 suitable return insn. If so put a label before it for the
403 function_return_label. */
404 if (BARRIER_P (insn)
405 && JUMP_P (PREV_INSN (insn))
406 && PATTERN (PREV_INSN (insn)) == kind)
408 rtx_insn *temp = PREV_INSN (PREV_INSN (insn));
409 rtx_code_label *label = gen_label_rtx ();
410 LABEL_NUSES (label) = 0;
412 /* Put the label before any USE insns that may precede the RETURN
413 insn. */
414 while (GET_CODE (temp) == USE)
415 temp = PREV_INSN (temp);
417 emit_label_after (label, temp);
418 *plabel = label;
421 else if (LABEL_P (insn))
422 *plabel = as_a <rtx_code_label *> (insn);
423 else
425 rtx_code_label *label = gen_label_rtx ();
426 LABEL_NUSES (label) = 0;
427 /* If the basic block reorder pass moves the return insn to
428 some other place try to locate it again and put our
429 function_return_label there. */
430 while (insn && ! (JUMP_P (insn) && (PATTERN (insn) == kind)))
431 insn = PREV_INSN (insn);
432 if (insn)
434 insn = PREV_INSN (insn);
436 /* Put the label before any USE insns that may precede the
437 RETURN insn. */
438 while (GET_CODE (insn) == USE)
439 insn = PREV_INSN (insn);
441 emit_label_after (label, insn);
443 else
445 if (targetm.have_epilogue () && ! targetm.have_return ())
446 /* The RETURN insn has its delay slot filled so we cannot
447 emit the label just before it. Since we already have
448 an epilogue and cannot emit a new RETURN, we cannot
449 emit the label at all. */
450 return NULL;
452 /* Otherwise, make a new label and emit a RETURN and BARRIER,
453 if needed. */
454 emit_label (label);
455 if (targetm.have_return ())
457 /* The return we make may have delay slots too. */
458 rtx_insn *pat = targetm.gen_return ();
459 rtx_insn *insn = emit_jump_insn (pat);
460 set_return_jump_label (insn);
461 emit_barrier ();
462 if (num_delay_slots (insn) > 0)
463 obstack_ptr_grow (&unfilled_slots_obstack, insn);
466 *plabel = label;
469 /* Show one additional use for this label so it won't go away until
470 we are done. */
471 ++LABEL_NUSES (*plabel);
473 return *plabel;
476 /* Put INSN and LIST together in a SEQUENCE rtx of LENGTH, and replace
477 the pattern of INSN with the SEQUENCE.
479 Returns the insn containing the SEQUENCE that replaces INSN. */
481 static rtx_insn *
482 emit_delay_sequence (rtx_insn *insn, const vec<rtx_insn *> &list, int length)
484 /* Allocate the rtvec to hold the insns and the SEQUENCE. */
485 rtvec seqv = rtvec_alloc (length + 1);
486 rtx seq = gen_rtx_SEQUENCE (VOIDmode, seqv);
487 rtx_insn *seq_insn = make_insn_raw (seq);
489 /* If DELAY_INSN has a location, use it for SEQ_INSN. If DELAY_INSN does
490 not have a location, but one of the delayed insns does, we pick up a
491 location from there later. */
492 INSN_LOCATION (seq_insn) = INSN_LOCATION (insn);
494 /* Unlink INSN from the insn chain, so that we can put it into
495 the SEQUENCE. Remember where we want to emit SEQUENCE in AFTER. */
496 rtx_insn *after = PREV_INSN (insn);
497 remove_insn (insn);
498 SET_NEXT_INSN (insn) = SET_PREV_INSN (insn) = NULL;
500 /* Build our SEQUENCE and rebuild the insn chain. */
501 start_sequence ();
502 XVECEXP (seq, 0, 0) = emit_insn (insn);
504 unsigned int delay_insns = list.length ();
505 gcc_assert (delay_insns == (unsigned int) length);
506 for (unsigned int i = 0; i < delay_insns; i++)
508 rtx_insn *tem = list[i];
509 rtx note, next;
511 /* Show that this copy of the insn isn't deleted. */
512 tem->set_undeleted ();
514 /* Unlink insn from its original place, and re-emit it into
515 the sequence. */
516 SET_NEXT_INSN (tem) = SET_PREV_INSN (tem) = NULL;
517 XVECEXP (seq, 0, i + 1) = emit_insn (tem);
519 /* SPARC assembler, for instance, emit warning when debug info is output
520 into the delay slot. */
521 if (INSN_LOCATION (tem) && !INSN_LOCATION (seq_insn))
522 INSN_LOCATION (seq_insn) = INSN_LOCATION (tem);
523 INSN_LOCATION (tem) = 0;
525 for (note = REG_NOTES (tem); note; note = next)
527 next = XEXP (note, 1);
528 switch (REG_NOTE_KIND (note))
530 case REG_DEAD:
531 /* Remove any REG_DEAD notes because we can't rely on them now
532 that the insn has been moved. */
533 remove_note (tem, note);
534 break;
536 case REG_LABEL_OPERAND:
537 case REG_LABEL_TARGET:
538 /* Keep the label reference count up to date. */
539 if (LABEL_P (XEXP (note, 0)))
540 LABEL_NUSES (XEXP (note, 0)) ++;
541 break;
543 default:
544 break;
548 end_sequence ();
550 /* Splice our SEQUENCE into the insn stream where INSN used to be. */
551 add_insn_after (seq_insn, after, NULL);
553 return seq_insn;
556 /* Add INSN to DELAY_LIST and return the head of the new list. The list must
557 be in the order in which the insns are to be executed. */
559 static void
560 add_to_delay_list (rtx_insn *insn, vec<rtx_insn *> *delay_list)
562 /* If INSN has its block number recorded, clear it since we may
563 be moving the insn to a new block. */
564 clear_hashed_info_for_insn (insn);
565 delay_list->safe_push (insn);
568 /* Delete INSN from the delay slot of the insn that it is in, which may
569 produce an insn with no delay slots. Return the new insn. */
571 static rtx_insn *
572 delete_from_delay_slot (rtx_insn *insn)
574 rtx_insn *trial, *seq_insn, *prev;
575 rtx_sequence *seq;
576 int i;
577 int had_barrier = 0;
579 /* We first must find the insn containing the SEQUENCE with INSN in its
580 delay slot. Do this by finding an insn, TRIAL, where
581 PREV_INSN (NEXT_INSN (TRIAL)) != TRIAL. */
583 for (trial = insn;
584 PREV_INSN (NEXT_INSN (trial)) == trial;
585 trial = NEXT_INSN (trial))
588 seq_insn = PREV_INSN (NEXT_INSN (trial));
589 seq = as_a <rtx_sequence *> (PATTERN (seq_insn));
591 if (NEXT_INSN (seq_insn) && BARRIER_P (NEXT_INSN (seq_insn)))
592 had_barrier = 1;
594 /* Create a delay list consisting of all the insns other than the one
595 we are deleting (unless we were the only one). */
596 auto_vec<rtx_insn *, 5> delay_list;
597 if (seq->len () > 2)
598 for (i = 1; i < seq->len (); i++)
599 if (seq->insn (i) != insn)
600 add_to_delay_list (seq->insn (i), &delay_list);
602 /* Delete the old SEQUENCE, re-emit the insn that used to have the delay
603 list, and rebuild the delay list if non-empty. */
604 prev = PREV_INSN (seq_insn);
605 trial = seq->insn (0);
606 delete_related_insns (seq_insn);
607 add_insn_after (trial, prev, NULL);
609 /* If there was a barrier after the old SEQUENCE, remit it. */
610 if (had_barrier)
611 emit_barrier_after (trial);
613 /* If there are any delay insns, remit them. Otherwise clear the
614 annul flag. */
615 if (!delay_list.is_empty ())
616 trial = emit_delay_sequence (trial, delay_list, XVECLEN (seq, 0) - 2);
617 else if (JUMP_P (trial))
618 INSN_ANNULLED_BRANCH_P (trial) = 0;
620 INSN_FROM_TARGET_P (insn) = 0;
622 /* Show we need to fill this insn again. */
623 obstack_ptr_grow (&unfilled_slots_obstack, trial);
625 return trial;
628 /* Delete INSN, a JUMP_INSN. If it is a conditional jump, we must track down
629 the insn that sets CC0 for it and delete it too. */
631 static void
632 delete_scheduled_jump (rtx_insn *insn)
634 /* Delete the insn that sets cc0 for us. On machines without cc0, we could
635 delete the insn that sets the condition code, but it is hard to find it.
636 Since this case is rare anyway, don't bother trying; there would likely
637 be other insns that became dead anyway, which we wouldn't know to
638 delete. */
640 if (HAVE_cc0 && reg_mentioned_p (cc0_rtx, insn))
642 rtx note = find_reg_note (insn, REG_CC_SETTER, NULL_RTX);
644 /* If a reg-note was found, it points to an insn to set CC0. This
645 insn is in the delay list of some other insn. So delete it from
646 the delay list it was in. */
647 if (note)
649 if (! FIND_REG_INC_NOTE (XEXP (note, 0), NULL_RTX)
650 && sets_cc0_p (PATTERN (XEXP (note, 0))) == 1)
651 delete_from_delay_slot (as_a <rtx_insn *> (XEXP (note, 0)));
653 else
655 /* The insn setting CC0 is our previous insn, but it may be in
656 a delay slot. It will be the last insn in the delay slot, if
657 it is. */
658 rtx_insn *trial = previous_insn (insn);
659 if (NOTE_P (trial))
660 trial = prev_nonnote_insn (trial);
661 if (sets_cc0_p (PATTERN (trial)) != 1
662 || FIND_REG_INC_NOTE (trial, NULL_RTX))
663 return;
664 if (PREV_INSN (NEXT_INSN (trial)) == trial)
665 delete_related_insns (trial);
666 else
667 delete_from_delay_slot (trial);
671 delete_related_insns (insn);
674 /* Counters for delay-slot filling. */
676 #define NUM_REORG_FUNCTIONS 2
677 #define MAX_DELAY_HISTOGRAM 3
678 #define MAX_REORG_PASSES 2
680 static int num_insns_needing_delays[NUM_REORG_FUNCTIONS][MAX_REORG_PASSES];
682 static int num_filled_delays[NUM_REORG_FUNCTIONS][MAX_DELAY_HISTOGRAM+1][MAX_REORG_PASSES];
684 static int reorg_pass_number;
686 static void
687 note_delay_statistics (int slots_filled, int index)
689 num_insns_needing_delays[index][reorg_pass_number]++;
690 if (slots_filled > MAX_DELAY_HISTOGRAM)
691 slots_filled = MAX_DELAY_HISTOGRAM;
692 num_filled_delays[index][slots_filled][reorg_pass_number]++;
695 /* Optimize the following cases:
697 1. When a conditional branch skips over only one instruction,
698 use an annulling branch and put that insn in the delay slot.
699 Use either a branch that annuls when the condition if true or
700 invert the test with a branch that annuls when the condition is
701 false. This saves insns, since otherwise we must copy an insn
702 from the L1 target.
704 (orig) (skip) (otherwise)
705 Bcc.n L1 Bcc',a L1 Bcc,a L1'
706 insn insn insn2
707 L1: L1: L1:
708 insn2 insn2 insn2
709 insn3 insn3 L1':
710 insn3
712 2. When a conditional branch skips over only one instruction,
713 and after that, it unconditionally branches somewhere else,
714 perform the similar optimization. This saves executing the
715 second branch in the case where the inverted condition is true.
717 Bcc.n L1 Bcc',a L2
718 insn insn
719 L1: L1:
720 Bra L2 Bra L2
722 INSN is a JUMP_INSN.
724 This should be expanded to skip over N insns, where N is the number
725 of delay slots required. */
727 static void
728 optimize_skip (rtx_jump_insn *insn, vec<rtx_insn *> *delay_list)
730 rtx_insn *trial = next_nonnote_insn (insn);
731 rtx_insn *next_trial = next_active_insn (trial);
732 int flags;
734 flags = get_jump_flags (insn, JUMP_LABEL (insn));
736 if (trial == 0
737 || !NONJUMP_INSN_P (trial)
738 || GET_CODE (PATTERN (trial)) == SEQUENCE
739 || recog_memoized (trial) < 0
740 || (! eligible_for_annul_false (insn, 0, trial, flags)
741 && ! eligible_for_annul_true (insn, 0, trial, flags))
742 || RTX_FRAME_RELATED_P (trial)
743 || can_throw_internal (trial))
744 return;
746 /* There are two cases where we are just executing one insn (we assume
747 here that a branch requires only one insn; this should be generalized
748 at some point): Where the branch goes around a single insn or where
749 we have one insn followed by a branch to the same label we branch to.
750 In both of these cases, inverting the jump and annulling the delay
751 slot give the same effect in fewer insns. */
752 if (next_trial == next_active_insn (JUMP_LABEL_AS_INSN (insn))
753 || (next_trial != 0
754 && simplejump_or_return_p (next_trial)
755 && JUMP_LABEL (insn) == JUMP_LABEL (next_trial)))
757 if (eligible_for_annul_false (insn, 0, trial, flags))
759 if (invert_jump (insn, JUMP_LABEL (insn), 1))
760 INSN_FROM_TARGET_P (trial) = 1;
761 else if (! eligible_for_annul_true (insn, 0, trial, flags))
762 return;
765 add_to_delay_list (trial, delay_list);
766 next_trial = next_active_insn (trial);
767 update_block (trial, trial);
768 delete_related_insns (trial);
770 /* Also, if we are targeting an unconditional
771 branch, thread our jump to the target of that branch. Don't
772 change this into a RETURN here, because it may not accept what
773 we have in the delay slot. We'll fix this up later. */
774 if (next_trial && simplejump_or_return_p (next_trial))
776 rtx target_label = JUMP_LABEL (next_trial);
777 if (ANY_RETURN_P (target_label))
778 target_label = find_end_label (target_label);
780 if (target_label)
782 /* Recompute the flags based on TARGET_LABEL since threading
783 the jump to TARGET_LABEL may change the direction of the
784 jump (which may change the circumstances in which the
785 delay slot is nullified). */
786 flags = get_jump_flags (insn, target_label);
787 if (eligible_for_annul_true (insn, 0, trial, flags))
788 reorg_redirect_jump (insn, target_label);
792 INSN_ANNULLED_BRANCH_P (insn) = 1;
796 /* Encode and return branch direction and prediction information for
797 INSN assuming it will jump to LABEL.
799 Non conditional branches return no direction information and
800 are predicted as very likely taken. */
802 static int
803 get_jump_flags (const rtx_insn *insn, rtx label)
805 int flags;
807 /* get_jump_flags can be passed any insn with delay slots, these may
808 be INSNs, CALL_INSNs, or JUMP_INSNs. Only JUMP_INSNs have branch
809 direction information, and only if they are conditional jumps.
811 If LABEL is a return, then there is no way to determine the branch
812 direction. */
813 if (JUMP_P (insn)
814 && (condjump_p (insn) || condjump_in_parallel_p (insn))
815 && !ANY_RETURN_P (label)
816 && INSN_UID (insn) <= max_uid
817 && INSN_UID (label) <= max_uid)
818 flags
819 = (uid_to_ruid[INSN_UID (label)] > uid_to_ruid[INSN_UID (insn)])
820 ? ATTR_FLAG_forward : ATTR_FLAG_backward;
821 /* No valid direction information. */
822 else
823 flags = 0;
825 return flags;
828 /* Return truth value of the statement that this branch
829 is mostly taken. If we think that the branch is extremely likely
830 to be taken, we return 2. If the branch is slightly more likely to be
831 taken, return 1. If the branch is slightly less likely to be taken,
832 return 0 and if the branch is highly unlikely to be taken, return -1. */
834 static int
835 mostly_true_jump (rtx jump_insn)
837 /* If branch probabilities are available, then use that number since it
838 always gives a correct answer. */
839 rtx note = find_reg_note (jump_insn, REG_BR_PROB, 0);
840 if (note)
842 int prob = XINT (note, 0);
844 if (prob >= REG_BR_PROB_BASE * 9 / 10)
845 return 2;
846 else if (prob >= REG_BR_PROB_BASE / 2)
847 return 1;
848 else if (prob >= REG_BR_PROB_BASE / 10)
849 return 0;
850 else
851 return -1;
854 /* If there is no note, assume branches are not taken.
855 This should be rare. */
856 return 0;
859 /* Return the condition under which INSN will branch to TARGET. If TARGET
860 is zero, return the condition under which INSN will return. If INSN is
861 an unconditional branch, return const_true_rtx. If INSN isn't a simple
862 type of jump, or it doesn't go to TARGET, return 0. */
864 static rtx
865 get_branch_condition (const rtx_insn *insn, rtx target)
867 rtx pat = PATTERN (insn);
868 rtx src;
870 if (condjump_in_parallel_p (insn))
871 pat = XVECEXP (pat, 0, 0);
873 if (ANY_RETURN_P (pat) && pat == target)
874 return const_true_rtx;
876 if (GET_CODE (pat) != SET || SET_DEST (pat) != pc_rtx)
877 return 0;
879 src = SET_SRC (pat);
880 if (GET_CODE (src) == LABEL_REF && LABEL_REF_LABEL (src) == target)
881 return const_true_rtx;
883 else if (GET_CODE (src) == IF_THEN_ELSE
884 && XEXP (src, 2) == pc_rtx
885 && ((GET_CODE (XEXP (src, 1)) == LABEL_REF
886 && LABEL_REF_LABEL (XEXP (src, 1)) == target)
887 || (ANY_RETURN_P (XEXP (src, 1)) && XEXP (src, 1) == target)))
888 return XEXP (src, 0);
890 else if (GET_CODE (src) == IF_THEN_ELSE
891 && XEXP (src, 1) == pc_rtx
892 && ((GET_CODE (XEXP (src, 2)) == LABEL_REF
893 && LABEL_REF_LABEL (XEXP (src, 2)) == target)
894 || (ANY_RETURN_P (XEXP (src, 2)) && XEXP (src, 2) == target)))
896 enum rtx_code rev;
897 rev = reversed_comparison_code (XEXP (src, 0), insn);
898 if (rev != UNKNOWN)
899 return gen_rtx_fmt_ee (rev, GET_MODE (XEXP (src, 0)),
900 XEXP (XEXP (src, 0), 0),
901 XEXP (XEXP (src, 0), 1));
904 return 0;
907 /* Return nonzero if CONDITION is more strict than the condition of
908 INSN, i.e., if INSN will always branch if CONDITION is true. */
910 static int
911 condition_dominates_p (rtx condition, const rtx_insn *insn)
913 rtx other_condition = get_branch_condition (insn, JUMP_LABEL (insn));
914 enum rtx_code code = GET_CODE (condition);
915 enum rtx_code other_code;
917 if (rtx_equal_p (condition, other_condition)
918 || other_condition == const_true_rtx)
919 return 1;
921 else if (condition == const_true_rtx || other_condition == 0)
922 return 0;
924 other_code = GET_CODE (other_condition);
925 if (GET_RTX_LENGTH (code) != 2 || GET_RTX_LENGTH (other_code) != 2
926 || ! rtx_equal_p (XEXP (condition, 0), XEXP (other_condition, 0))
927 || ! rtx_equal_p (XEXP (condition, 1), XEXP (other_condition, 1)))
928 return 0;
930 return comparison_dominates_p (code, other_code);
933 /* Return nonzero if redirecting JUMP to NEWLABEL does not invalidate
934 any insns already in the delay slot of JUMP. */
936 static int
937 redirect_with_delay_slots_safe_p (rtx_insn *jump, rtx newlabel, rtx seq)
939 int flags, i;
940 rtx_sequence *pat = as_a <rtx_sequence *> (PATTERN (seq));
942 /* Make sure all the delay slots of this jump would still
943 be valid after threading the jump. If they are still
944 valid, then return nonzero. */
946 flags = get_jump_flags (jump, newlabel);
947 for (i = 1; i < pat->len (); i++)
948 if (! (
949 #if ANNUL_IFFALSE_SLOTS
950 (INSN_ANNULLED_BRANCH_P (jump)
951 && INSN_FROM_TARGET_P (pat->insn (i)))
952 ? eligible_for_annul_false (jump, i - 1, pat->insn (i), flags) :
953 #endif
954 #if ANNUL_IFTRUE_SLOTS
955 (INSN_ANNULLED_BRANCH_P (jump)
956 && ! INSN_FROM_TARGET_P (XVECEXP (pat, 0, i)))
957 ? eligible_for_annul_true (jump, i - 1, pat->insn (i), flags) :
958 #endif
959 eligible_for_delay (jump, i - 1, pat->insn (i), flags)))
960 break;
962 return (i == pat->len ());
965 /* Return nonzero if redirecting JUMP to NEWLABEL does not invalidate
966 any insns we wish to place in the delay slot of JUMP. */
968 static int
969 redirect_with_delay_list_safe_p (rtx_insn *jump, rtx newlabel,
970 const vec<rtx_insn *> &delay_list)
972 /* Make sure all the insns in DELAY_LIST would still be
973 valid after threading the jump. If they are still
974 valid, then return nonzero. */
976 int flags = get_jump_flags (jump, newlabel);
977 unsigned int delay_insns = delay_list.length ();
978 unsigned int i = 0;
979 for (; i < delay_insns; i++)
980 if (! (
981 #if ANNUL_IFFALSE_SLOTS
982 (INSN_ANNULLED_BRANCH_P (jump)
983 && INSN_FROM_TARGET_P (delay_list[i]))
984 ? eligible_for_annul_false (jump, i, delay_list[i], flags) :
985 #endif
986 #if ANNUL_IFTRUE_SLOTS
987 (INSN_ANNULLED_BRANCH_P (jump)
988 && ! INSN_FROM_TARGET_P (delay_list[i]))
989 ? eligible_for_annul_true (jump, i, delay_list[i], flags) :
990 #endif
991 eligible_for_delay (jump, i, delay_list[i], flags)))
992 break;
994 return i == delay_insns;
997 /* DELAY_LIST is a list of insns that have already been placed into delay
998 slots. See if all of them have the same annulling status as ANNUL_TRUE_P.
999 If not, return 0; otherwise return 1. */
1001 static int
1002 check_annul_list_true_false (int annul_true_p,
1003 const vec<rtx_insn *> &delay_list)
1005 rtx_insn *trial;
1006 unsigned int i;
1007 FOR_EACH_VEC_ELT (delay_list, i, trial)
1008 if ((annul_true_p && INSN_FROM_TARGET_P (trial))
1009 || (!annul_true_p && !INSN_FROM_TARGET_P (trial)))
1010 return 0;
1012 return 1;
1015 /* INSN branches to an insn whose pattern SEQ is a SEQUENCE. Given that
1016 the condition tested by INSN is CONDITION and the resources shown in
1017 OTHER_NEEDED are needed after INSN, see whether INSN can take all the insns
1018 from SEQ's delay list, in addition to whatever insns it may execute
1019 (in DELAY_LIST). SETS and NEEDED are denote resources already set and
1020 needed while searching for delay slot insns. Return the concatenated
1021 delay list if possible, otherwise, return 0.
1023 SLOTS_TO_FILL is the total number of slots required by INSN, and
1024 PSLOTS_FILLED points to the number filled so far (also the number of
1025 insns in DELAY_LIST). It is updated with the number that have been
1026 filled from the SEQUENCE, if any.
1028 PANNUL_P points to a nonzero value if we already know that we need
1029 to annul INSN. If this routine determines that annulling is needed,
1030 it may set that value nonzero.
1032 PNEW_THREAD points to a location that is to receive the place at which
1033 execution should continue. */
1035 static void
1036 steal_delay_list_from_target (rtx_insn *insn, rtx condition, rtx_sequence *seq,
1037 vec<rtx_insn *> *delay_list, resources *sets,
1038 struct resources *needed,
1039 struct resources *other_needed,
1040 int slots_to_fill, int *pslots_filled,
1041 int *pannul_p, rtx *pnew_thread)
1043 int slots_remaining = slots_to_fill - *pslots_filled;
1044 int total_slots_filled = *pslots_filled;
1045 auto_vec<rtx_insn *, 5> new_delay_list;
1046 int must_annul = *pannul_p;
1047 int used_annul = 0;
1048 int i;
1049 struct resources cc_set;
1050 bool *redundant;
1052 /* We can't do anything if there are more delay slots in SEQ than we
1053 can handle, or if we don't know that it will be a taken branch.
1054 We know that it will be a taken branch if it is either an unconditional
1055 branch or a conditional branch with a stricter branch condition.
1057 Also, exit if the branch has more than one set, since then it is computing
1058 other results that can't be ignored, e.g. the HPPA mov&branch instruction.
1059 ??? It may be possible to move other sets into INSN in addition to
1060 moving the instructions in the delay slots.
1062 We can not steal the delay list if one of the instructions in the
1063 current delay_list modifies the condition codes and the jump in the
1064 sequence is a conditional jump. We can not do this because we can
1065 not change the direction of the jump because the condition codes
1066 will effect the direction of the jump in the sequence. */
1068 CLEAR_RESOURCE (&cc_set);
1070 rtx_insn *trial;
1071 FOR_EACH_VEC_ELT (*delay_list, i, trial)
1073 mark_set_resources (trial, &cc_set, 0, MARK_SRC_DEST_CALL);
1074 if (insn_references_resource_p (seq->insn (0), &cc_set, false))
1075 return;
1078 if (XVECLEN (seq, 0) - 1 > slots_remaining
1079 || ! condition_dominates_p (condition, seq->insn (0))
1080 || ! single_set (seq->insn (0)))
1081 return;
1083 /* On some targets, branches with delay slots can have a limited
1084 displacement. Give the back end a chance to tell us we can't do
1085 this. */
1086 if (! targetm.can_follow_jump (insn, seq->insn (0)))
1087 return;
1089 redundant = XALLOCAVEC (bool, XVECLEN (seq, 0));
1090 for (i = 1; i < seq->len (); i++)
1092 rtx_insn *trial = seq->insn (i);
1093 int flags;
1095 if (insn_references_resource_p (trial, sets, false)
1096 || insn_sets_resource_p (trial, needed, false)
1097 || insn_sets_resource_p (trial, sets, false)
1098 /* If TRIAL sets CC0, we can't copy it, so we can't steal this
1099 delay list. */
1100 || (HAVE_cc0 && find_reg_note (trial, REG_CC_USER, NULL_RTX))
1101 /* If TRIAL is from the fallthrough code of an annulled branch insn
1102 in SEQ, we cannot use it. */
1103 || (INSN_ANNULLED_BRANCH_P (seq->insn (0))
1104 && ! INSN_FROM_TARGET_P (trial)))
1105 return;
1107 /* If this insn was already done (usually in a previous delay slot),
1108 pretend we put it in our delay slot. */
1109 redundant[i] = redundant_insn (trial, insn, new_delay_list);
1110 if (redundant[i])
1111 continue;
1113 /* We will end up re-vectoring this branch, so compute flags
1114 based on jumping to the new label. */
1115 flags = get_jump_flags (insn, JUMP_LABEL (seq->insn (0)));
1117 if (! must_annul
1118 && ((condition == const_true_rtx
1119 || (! insn_sets_resource_p (trial, other_needed, false)
1120 && ! may_trap_or_fault_p (PATTERN (trial)))))
1121 ? eligible_for_delay (insn, total_slots_filled, trial, flags)
1122 : (must_annul || (delay_list->is_empty () && new_delay_list.is_empty ()))
1123 && (must_annul = 1,
1124 check_annul_list_true_false (0, *delay_list)
1125 && check_annul_list_true_false (0, new_delay_list)
1126 && eligible_for_annul_false (insn, total_slots_filled,
1127 trial, flags)))
1129 if (must_annul)
1131 /* Frame related instructions cannot go into annulled delay
1132 slots, it messes up the dwarf info. */
1133 if (RTX_FRAME_RELATED_P (trial))
1134 return;
1135 used_annul = 1;
1137 rtx_insn *temp = copy_delay_slot_insn (trial);
1138 INSN_FROM_TARGET_P (temp) = 1;
1139 add_to_delay_list (temp, &new_delay_list);
1140 total_slots_filled++;
1142 if (--slots_remaining == 0)
1143 break;
1145 else
1146 return;
1149 /* Record the effect of the instructions that were redundant and which
1150 we therefore decided not to copy. */
1151 for (i = 1; i < seq->len (); i++)
1152 if (redundant[i])
1153 update_block (seq->insn (i), insn);
1155 /* Show the place to which we will be branching. */
1156 *pnew_thread = first_active_target_insn (JUMP_LABEL (seq->insn (0)));
1158 /* Add any new insns to the delay list and update the count of the
1159 number of slots filled. */
1160 *pslots_filled = total_slots_filled;
1161 if (used_annul)
1162 *pannul_p = 1;
1164 rtx_insn *temp;
1165 FOR_EACH_VEC_ELT (new_delay_list, i, temp)
1166 add_to_delay_list (temp, delay_list);
1169 /* Similar to steal_delay_list_from_target except that SEQ is on the
1170 fallthrough path of INSN. Here we only do something if the delay insn
1171 of SEQ is an unconditional branch. In that case we steal its delay slot
1172 for INSN since unconditional branches are much easier to fill. */
1174 static void
1175 steal_delay_list_from_fallthrough (rtx_insn *insn, rtx condition,
1176 rtx_sequence *seq,
1177 vec<rtx_insn *> *delay_list,
1178 struct resources *sets,
1179 struct resources *needed,
1180 struct resources *other_needed,
1181 int slots_to_fill, int *pslots_filled,
1182 int *pannul_p)
1184 int i;
1185 int flags;
1186 int must_annul = *pannul_p;
1187 int used_annul = 0;
1189 flags = get_jump_flags (insn, JUMP_LABEL (insn));
1191 /* We can't do anything if SEQ's delay insn isn't an
1192 unconditional branch. */
1194 if (! simplejump_or_return_p (seq->insn (0)))
1195 return;
1197 for (i = 1; i < seq->len (); i++)
1199 rtx_insn *trial = seq->insn (i);
1201 /* If TRIAL sets CC0, stealing it will move it too far from the use
1202 of CC0. */
1203 if (insn_references_resource_p (trial, sets, false)
1204 || insn_sets_resource_p (trial, needed, false)
1205 || insn_sets_resource_p (trial, sets, false)
1206 || (HAVE_cc0 && sets_cc0_p (PATTERN (trial))))
1208 break;
1210 /* If this insn was already done, we don't need it. */
1211 if (redundant_insn (trial, insn, *delay_list))
1213 update_block (trial, insn);
1214 delete_from_delay_slot (trial);
1215 continue;
1218 if (! must_annul
1219 && ((condition == const_true_rtx
1220 || (! insn_sets_resource_p (trial, other_needed, false)
1221 && ! may_trap_or_fault_p (PATTERN (trial)))))
1222 ? eligible_for_delay (insn, *pslots_filled, trial, flags)
1223 : (must_annul || delay_list->is_empty ()) && (must_annul = 1,
1224 check_annul_list_true_false (1, *delay_list)
1225 && eligible_for_annul_true (insn, *pslots_filled, trial, flags)))
1227 if (must_annul)
1228 used_annul = 1;
1229 delete_from_delay_slot (trial);
1230 add_to_delay_list (trial, delay_list);
1232 if (++(*pslots_filled) == slots_to_fill)
1233 break;
1235 else
1236 break;
1239 if (used_annul)
1240 *pannul_p = 1;
1243 /* Try merging insns starting at THREAD which match exactly the insns in
1244 INSN's delay list.
1246 If all insns were matched and the insn was previously annulling, the
1247 annul bit will be cleared.
1249 For each insn that is merged, if the branch is or will be non-annulling,
1250 we delete the merged insn. */
1252 static void
1253 try_merge_delay_insns (rtx_insn *insn, rtx_insn *thread)
1255 rtx_insn *trial, *next_trial;
1256 rtx_insn *delay_insn = as_a <rtx_insn *> (XVECEXP (PATTERN (insn), 0, 0));
1257 int annul_p = JUMP_P (delay_insn) && INSN_ANNULLED_BRANCH_P (delay_insn);
1258 int slot_number = 1;
1259 int num_slots = XVECLEN (PATTERN (insn), 0);
1260 rtx next_to_match = XVECEXP (PATTERN (insn), 0, slot_number);
1261 struct resources set, needed, modified;
1262 auto_vec<std::pair<rtx_insn *, bool>, 10> merged_insns;
1263 int flags;
1265 flags = get_jump_flags (delay_insn, JUMP_LABEL (delay_insn));
1267 CLEAR_RESOURCE (&needed);
1268 CLEAR_RESOURCE (&set);
1270 /* If this is not an annulling branch, take into account anything needed in
1271 INSN's delay slot. This prevents two increments from being incorrectly
1272 folded into one. If we are annulling, this would be the correct
1273 thing to do. (The alternative, looking at things set in NEXT_TO_MATCH
1274 will essentially disable this optimization. This method is somewhat of
1275 a kludge, but I don't see a better way.) */
1276 if (! annul_p)
1277 for (int i = 1; i < num_slots; i++)
1278 if (XVECEXP (PATTERN (insn), 0, i))
1279 mark_referenced_resources (XVECEXP (PATTERN (insn), 0, i), &needed,
1280 true);
1282 for (trial = thread; !stop_search_p (trial, 1); trial = next_trial)
1284 rtx pat = PATTERN (trial);
1285 rtx oldtrial = trial;
1287 next_trial = next_nonnote_insn (trial);
1289 /* TRIAL must be a CALL_INSN or INSN. Skip USE and CLOBBER. */
1290 if (NONJUMP_INSN_P (trial)
1291 && (GET_CODE (pat) == USE || GET_CODE (pat) == CLOBBER))
1292 continue;
1294 if (GET_CODE (next_to_match) == GET_CODE (trial)
1295 /* We can't share an insn that sets cc0. */
1296 && (!HAVE_cc0 || ! sets_cc0_p (pat))
1297 && ! insn_references_resource_p (trial, &set, true)
1298 && ! insn_sets_resource_p (trial, &set, true)
1299 && ! insn_sets_resource_p (trial, &needed, true)
1300 && (trial = try_split (pat, trial, 0)) != 0
1301 /* Update next_trial, in case try_split succeeded. */
1302 && (next_trial = next_nonnote_insn (trial))
1303 /* Likewise THREAD. */
1304 && (thread = oldtrial == thread ? trial : thread)
1305 && rtx_equal_p (PATTERN (next_to_match), PATTERN (trial))
1306 /* Have to test this condition if annul condition is different
1307 from (and less restrictive than) non-annulling one. */
1308 && eligible_for_delay (delay_insn, slot_number - 1, trial, flags))
1311 if (! annul_p)
1313 update_block (trial, thread);
1314 if (trial == thread)
1315 thread = next_active_insn (thread);
1317 delete_related_insns (trial);
1318 INSN_FROM_TARGET_P (next_to_match) = 0;
1320 else
1321 merged_insns.safe_push (std::pair<rtx_insn *, bool> (trial, false));
1323 if (++slot_number == num_slots)
1324 break;
1326 next_to_match = XVECEXP (PATTERN (insn), 0, slot_number);
1329 mark_set_resources (trial, &set, 0, MARK_SRC_DEST_CALL);
1330 mark_referenced_resources (trial, &needed, true);
1333 /* See if we stopped on a filled insn. If we did, try to see if its
1334 delay slots match. */
1335 if (slot_number != num_slots
1336 && trial && NONJUMP_INSN_P (trial)
1337 && GET_CODE (PATTERN (trial)) == SEQUENCE
1338 && !(JUMP_P (XVECEXP (PATTERN (trial), 0, 0))
1339 && INSN_ANNULLED_BRANCH_P (XVECEXP (PATTERN (trial), 0, 0))))
1341 rtx_sequence *pat = as_a <rtx_sequence *> (PATTERN (trial));
1342 rtx filled_insn = XVECEXP (pat, 0, 0);
1344 /* Account for resources set/needed by the filled insn. */
1345 mark_set_resources (filled_insn, &set, 0, MARK_SRC_DEST_CALL);
1346 mark_referenced_resources (filled_insn, &needed, true);
1348 for (int i = 1; i < pat->len (); i++)
1350 rtx_insn *dtrial = pat->insn (i);
1352 CLEAR_RESOURCE (&modified);
1353 /* Account for resources set by the insn following NEXT_TO_MATCH
1354 inside INSN's delay list. */
1355 for (int j = 1; slot_number + j < num_slots; j++)
1356 mark_set_resources (XVECEXP (PATTERN (insn), 0, slot_number + j),
1357 &modified, 0, MARK_SRC_DEST_CALL);
1358 /* Account for resources set by the insn before DTRIAL and inside
1359 TRIAL's delay list. */
1360 for (int j = 1; j < i; j++)
1361 mark_set_resources (XVECEXP (pat, 0, j),
1362 &modified, 0, MARK_SRC_DEST_CALL);
1363 if (! insn_references_resource_p (dtrial, &set, true)
1364 && ! insn_sets_resource_p (dtrial, &set, true)
1365 && ! insn_sets_resource_p (dtrial, &needed, true)
1366 && (!HAVE_cc0 || ! sets_cc0_p (PATTERN (dtrial)))
1367 && rtx_equal_p (PATTERN (next_to_match), PATTERN (dtrial))
1368 /* Check that DTRIAL and NEXT_TO_MATCH does not reference a
1369 resource modified between them (only dtrial is checked because
1370 next_to_match and dtrial shall to be equal in order to hit
1371 this line) */
1372 && ! insn_references_resource_p (dtrial, &modified, true)
1373 && eligible_for_delay (delay_insn, slot_number - 1, dtrial, flags))
1375 if (! annul_p)
1377 rtx_insn *new_rtx;
1379 update_block (dtrial, thread);
1380 new_rtx = delete_from_delay_slot (dtrial);
1381 if (thread->deleted ())
1382 thread = new_rtx;
1383 INSN_FROM_TARGET_P (next_to_match) = 0;
1385 else
1386 merged_insns.safe_push (std::pair<rtx_insn *, bool> (dtrial,
1387 true));
1389 if (++slot_number == num_slots)
1390 break;
1392 next_to_match = XVECEXP (PATTERN (insn), 0, slot_number);
1394 else
1396 /* Keep track of the set/referenced resources for the delay
1397 slots of any trial insns we encounter. */
1398 mark_set_resources (dtrial, &set, 0, MARK_SRC_DEST_CALL);
1399 mark_referenced_resources (dtrial, &needed, true);
1404 /* If all insns in the delay slot have been matched and we were previously
1405 annulling the branch, we need not any more. In that case delete all the
1406 merged insns. Also clear the INSN_FROM_TARGET_P bit of each insn in
1407 the delay list so that we know that it isn't only being used at the
1408 target. */
1409 if (slot_number == num_slots && annul_p)
1411 unsigned int len = merged_insns.length ();
1412 for (unsigned int i = len - 1; i < len; i--)
1413 if (merged_insns[i].second)
1415 update_block (merged_insns[i].first, thread);
1416 rtx_insn *new_rtx = delete_from_delay_slot (merged_insns[i].first);
1417 if (thread->deleted ())
1418 thread = new_rtx;
1420 else
1422 update_block (merged_insns[i].first, thread);
1423 delete_related_insns (merged_insns[i].first);
1426 INSN_ANNULLED_BRANCH_P (delay_insn) = 0;
1428 for (int i = 0; i < XVECLEN (PATTERN (insn), 0); i++)
1429 INSN_FROM_TARGET_P (XVECEXP (PATTERN (insn), 0, i)) = 0;
1433 /* See if INSN is redundant with an insn in front of TARGET. Often this
1434 is called when INSN is a candidate for a delay slot of TARGET.
1435 DELAY_LIST are insns that will be placed in delay slots of TARGET in front
1436 of INSN. Often INSN will be redundant with an insn in a delay slot of
1437 some previous insn. This happens when we have a series of branches to the
1438 same label; in that case the first insn at the target might want to go
1439 into each of the delay slots.
1441 If we are not careful, this routine can take up a significant fraction
1442 of the total compilation time (4%), but only wins rarely. Hence we
1443 speed this routine up by making two passes. The first pass goes back
1444 until it hits a label and sees if it finds an insn with an identical
1445 pattern. Only in this (relatively rare) event does it check for
1446 data conflicts.
1448 We do not split insns we encounter. This could cause us not to find a
1449 redundant insn, but the cost of splitting seems greater than the possible
1450 gain in rare cases. */
1452 static rtx_insn *
1453 redundant_insn (rtx insn, rtx_insn *target, const vec<rtx_insn *> &delay_list)
1455 rtx target_main = target;
1456 rtx ipat = PATTERN (insn);
1457 rtx_insn *trial;
1458 rtx pat;
1459 struct resources needed, set;
1460 int i;
1461 unsigned insns_to_search;
1463 /* If INSN has any REG_UNUSED notes, it can't match anything since we
1464 are allowed to not actually assign to such a register. */
1465 if (find_reg_note (insn, REG_UNUSED, NULL_RTX) != 0)
1466 return 0;
1468 /* Scan backwards looking for a match. */
1469 for (trial = PREV_INSN (target),
1470 insns_to_search = MAX_DELAY_SLOT_INSN_SEARCH;
1471 trial && insns_to_search > 0;
1472 trial = PREV_INSN (trial))
1474 /* (use (insn))s can come immediately after a barrier if the
1475 label that used to precede them has been deleted as dead.
1476 See delete_related_insns. */
1477 if (LABEL_P (trial) || BARRIER_P (trial))
1478 return 0;
1480 if (!INSN_P (trial))
1481 continue;
1482 --insns_to_search;
1484 pat = PATTERN (trial);
1485 if (GET_CODE (pat) == USE || GET_CODE (pat) == CLOBBER)
1486 continue;
1488 if (rtx_sequence *seq = dyn_cast <rtx_sequence *> (pat))
1490 /* Stop for a CALL and its delay slots because it is difficult to
1491 track its resource needs correctly. */
1492 if (CALL_P (seq->element (0)))
1493 return 0;
1495 /* Stop for an INSN or JUMP_INSN with delayed effects and its delay
1496 slots because it is difficult to track its resource needs
1497 correctly. */
1499 if (INSN_SETS_ARE_DELAYED (seq->insn (0)))
1500 return 0;
1502 if (INSN_REFERENCES_ARE_DELAYED (seq->insn (0)))
1503 return 0;
1505 /* See if any of the insns in the delay slot match, updating
1506 resource requirements as we go. */
1507 for (i = seq->len () - 1; i > 0; i--)
1508 if (GET_CODE (seq->element (i)) == GET_CODE (insn)
1509 && rtx_equal_p (PATTERN (seq->element (i)), ipat)
1510 && ! find_reg_note (seq->element (i), REG_UNUSED, NULL_RTX))
1511 break;
1513 /* If found a match, exit this loop early. */
1514 if (i > 0)
1515 break;
1518 else if (GET_CODE (trial) == GET_CODE (insn) && rtx_equal_p (pat, ipat)
1519 && ! find_reg_note (trial, REG_UNUSED, NULL_RTX))
1520 break;
1523 /* If we didn't find an insn that matches, return 0. */
1524 if (trial == 0)
1525 return 0;
1527 /* See what resources this insn sets and needs. If they overlap, or
1528 if this insn references CC0, it can't be redundant. */
1530 CLEAR_RESOURCE (&needed);
1531 CLEAR_RESOURCE (&set);
1532 mark_set_resources (insn, &set, 0, MARK_SRC_DEST_CALL);
1533 mark_referenced_resources (insn, &needed, true);
1535 /* If TARGET is a SEQUENCE, get the main insn. */
1536 if (NONJUMP_INSN_P (target) && GET_CODE (PATTERN (target)) == SEQUENCE)
1537 target_main = XVECEXP (PATTERN (target), 0, 0);
1539 if (resource_conflicts_p (&needed, &set)
1540 || (HAVE_cc0 && reg_mentioned_p (cc0_rtx, ipat))
1541 /* The insn requiring the delay may not set anything needed or set by
1542 INSN. */
1543 || insn_sets_resource_p (target_main, &needed, true)
1544 || insn_sets_resource_p (target_main, &set, true))
1545 return 0;
1547 /* Insns we pass may not set either NEEDED or SET, so merge them for
1548 simpler tests. */
1549 needed.memory |= set.memory;
1550 IOR_HARD_REG_SET (needed.regs, set.regs);
1552 /* This insn isn't redundant if it conflicts with an insn that either is
1553 or will be in a delay slot of TARGET. */
1555 unsigned int j;
1556 rtx_insn *temp;
1557 FOR_EACH_VEC_ELT (delay_list, j, temp)
1558 if (insn_sets_resource_p (temp, &needed, true))
1559 return 0;
1561 if (NONJUMP_INSN_P (target) && GET_CODE (PATTERN (target)) == SEQUENCE)
1562 for (i = 1; i < XVECLEN (PATTERN (target), 0); i++)
1563 if (insn_sets_resource_p (XVECEXP (PATTERN (target), 0, i), &needed,
1564 true))
1565 return 0;
1567 /* Scan backwards until we reach a label or an insn that uses something
1568 INSN sets or sets something insn uses or sets. */
1570 for (trial = PREV_INSN (target),
1571 insns_to_search = MAX_DELAY_SLOT_INSN_SEARCH;
1572 trial && !LABEL_P (trial) && insns_to_search > 0;
1573 trial = PREV_INSN (trial))
1575 if (!INSN_P (trial))
1576 continue;
1577 --insns_to_search;
1579 pat = PATTERN (trial);
1580 if (GET_CODE (pat) == USE || GET_CODE (pat) == CLOBBER)
1581 continue;
1583 if (rtx_sequence *seq = dyn_cast <rtx_sequence *> (pat))
1585 bool annul_p = false;
1586 rtx_insn *control = seq->insn (0);
1588 /* If this is a CALL_INSN and its delay slots, it is hard to track
1589 the resource needs properly, so give up. */
1590 if (CALL_P (control))
1591 return 0;
1593 /* If this is an INSN or JUMP_INSN with delayed effects, it
1594 is hard to track the resource needs properly, so give up. */
1596 if (INSN_SETS_ARE_DELAYED (control))
1597 return 0;
1599 if (INSN_REFERENCES_ARE_DELAYED (control))
1600 return 0;
1602 if (JUMP_P (control))
1603 annul_p = INSN_ANNULLED_BRANCH_P (control);
1605 /* See if any of the insns in the delay slot match, updating
1606 resource requirements as we go. */
1607 for (i = seq->len () - 1; i > 0; i--)
1609 rtx_insn *candidate = seq->insn (i);
1611 /* If an insn will be annulled if the branch is false, it isn't
1612 considered as a possible duplicate insn. */
1613 if (rtx_equal_p (PATTERN (candidate), ipat)
1614 && ! (annul_p && INSN_FROM_TARGET_P (candidate)))
1616 /* Show that this insn will be used in the sequel. */
1617 INSN_FROM_TARGET_P (candidate) = 0;
1618 return candidate;
1621 /* Unless this is an annulled insn from the target of a branch,
1622 we must stop if it sets anything needed or set by INSN. */
1623 if ((!annul_p || !INSN_FROM_TARGET_P (candidate))
1624 && insn_sets_resource_p (candidate, &needed, true))
1625 return 0;
1628 /* If the insn requiring the delay slot conflicts with INSN, we
1629 must stop. */
1630 if (insn_sets_resource_p (control, &needed, true))
1631 return 0;
1633 else
1635 /* See if TRIAL is the same as INSN. */
1636 pat = PATTERN (trial);
1637 if (rtx_equal_p (pat, ipat))
1638 return trial;
1640 /* Can't go any further if TRIAL conflicts with INSN. */
1641 if (insn_sets_resource_p (trial, &needed, true))
1642 return 0;
1646 return 0;
1649 /* Return 1 if THREAD can only be executed in one way. If LABEL is nonzero,
1650 it is the target of the branch insn being scanned. If ALLOW_FALLTHROUGH
1651 is nonzero, we are allowed to fall into this thread; otherwise, we are
1652 not.
1654 If LABEL is used more than one or we pass a label other than LABEL before
1655 finding an active insn, we do not own this thread. */
1657 static int
1658 own_thread_p (rtx thread, rtx label, int allow_fallthrough)
1660 rtx_insn *active_insn;
1661 rtx_insn *insn;
1663 /* We don't own the function end. */
1664 if (thread == 0 || ANY_RETURN_P (thread))
1665 return 0;
1667 /* We have a non-NULL insn. */
1668 rtx_insn *thread_insn = as_a <rtx_insn *> (thread);
1670 /* Get the first active insn, or THREAD_INSN, if it is an active insn. */
1671 active_insn = next_active_insn (PREV_INSN (thread_insn));
1673 for (insn = thread_insn; insn != active_insn; insn = NEXT_INSN (insn))
1674 if (LABEL_P (insn)
1675 && (insn != label || LABEL_NUSES (insn) != 1))
1676 return 0;
1678 if (allow_fallthrough)
1679 return 1;
1681 /* Ensure that we reach a BARRIER before any insn or label. */
1682 for (insn = prev_nonnote_insn (thread_insn);
1683 insn == 0 || !BARRIER_P (insn);
1684 insn = prev_nonnote_insn (insn))
1685 if (insn == 0
1686 || LABEL_P (insn)
1687 || (NONJUMP_INSN_P (insn)
1688 && GET_CODE (PATTERN (insn)) != USE
1689 && GET_CODE (PATTERN (insn)) != CLOBBER))
1690 return 0;
1692 return 1;
1695 /* Called when INSN is being moved from a location near the target of a jump.
1696 We leave a marker of the form (use (INSN)) immediately in front
1697 of WHERE for mark_target_live_regs. These markers will be deleted when
1698 reorg finishes.
1700 We used to try to update the live status of registers if WHERE is at
1701 the start of a basic block, but that can't work since we may remove a
1702 BARRIER in relax_delay_slots. */
1704 static void
1705 update_block (rtx_insn *insn, rtx where)
1707 /* Ignore if this was in a delay slot and it came from the target of
1708 a branch. */
1709 if (INSN_FROM_TARGET_P (insn))
1710 return;
1712 emit_insn_before (gen_rtx_USE (VOIDmode, insn), where);
1714 /* INSN might be making a value live in a block where it didn't use to
1715 be. So recompute liveness information for this block. */
1717 incr_ticks_for_insn (insn);
1720 /* Similar to REDIRECT_JUMP except that we update the BB_TICKS entry for
1721 the basic block containing the jump. */
1723 static int
1724 reorg_redirect_jump (rtx_jump_insn *jump, rtx nlabel)
1726 incr_ticks_for_insn (jump);
1727 return redirect_jump (jump, nlabel, 1);
1730 /* Called when INSN is being moved forward into a delay slot of DELAYED_INSN.
1731 We check every instruction between INSN and DELAYED_INSN for REG_DEAD notes
1732 that reference values used in INSN. If we find one, then we move the
1733 REG_DEAD note to INSN.
1735 This is needed to handle the case where a later insn (after INSN) has a
1736 REG_DEAD note for a register used by INSN, and this later insn subsequently
1737 gets moved before a CODE_LABEL because it is a redundant insn. In this
1738 case, mark_target_live_regs may be confused into thinking the register
1739 is dead because it sees a REG_DEAD note immediately before a CODE_LABEL. */
1741 static void
1742 update_reg_dead_notes (rtx_insn *insn, rtx_insn *delayed_insn)
1744 rtx link, next;
1745 rtx_insn *p;
1747 for (p = next_nonnote_insn (insn); p != delayed_insn;
1748 p = next_nonnote_insn (p))
1749 for (link = REG_NOTES (p); link; link = next)
1751 next = XEXP (link, 1);
1753 if (REG_NOTE_KIND (link) != REG_DEAD
1754 || !REG_P (XEXP (link, 0)))
1755 continue;
1757 if (reg_referenced_p (XEXP (link, 0), PATTERN (insn)))
1759 /* Move the REG_DEAD note from P to INSN. */
1760 remove_note (p, link);
1761 XEXP (link, 1) = REG_NOTES (insn);
1762 REG_NOTES (insn) = link;
1767 /* Called when an insn redundant with start_insn is deleted. If there
1768 is a REG_DEAD note for the target of start_insn between start_insn
1769 and stop_insn, then the REG_DEAD note needs to be deleted since the
1770 value no longer dies there.
1772 If the REG_DEAD note isn't deleted, then mark_target_live_regs may be
1773 confused into thinking the register is dead. */
1775 static void
1776 fix_reg_dead_note (rtx_insn *start_insn, rtx stop_insn)
1778 rtx link, next;
1779 rtx_insn *p;
1781 for (p = next_nonnote_insn (start_insn); p != stop_insn;
1782 p = next_nonnote_insn (p))
1783 for (link = REG_NOTES (p); link; link = next)
1785 next = XEXP (link, 1);
1787 if (REG_NOTE_KIND (link) != REG_DEAD
1788 || !REG_P (XEXP (link, 0)))
1789 continue;
1791 if (reg_set_p (XEXP (link, 0), PATTERN (start_insn)))
1793 remove_note (p, link);
1794 return;
1799 /* Delete any REG_UNUSED notes that exist on INSN but not on REDUNDANT_INSN.
1801 This handles the case of udivmodXi4 instructions which optimize their
1802 output depending on whether any REG_UNUSED notes are present.
1803 we must make sure that INSN calculates as many results as REDUNDANT_INSN
1804 does. */
1806 static void
1807 update_reg_unused_notes (rtx insn, rtx redundant_insn)
1809 rtx link, next;
1811 for (link = REG_NOTES (insn); link; link = next)
1813 next = XEXP (link, 1);
1815 if (REG_NOTE_KIND (link) != REG_UNUSED
1816 || !REG_P (XEXP (link, 0)))
1817 continue;
1819 if (! find_regno_note (redundant_insn, REG_UNUSED,
1820 REGNO (XEXP (link, 0))))
1821 remove_note (insn, link);
1825 static vec <rtx> sibling_labels;
1827 /* Return the label before INSN, or put a new label there. If SIBLING is
1828 non-zero, it is another label associated with the new label (if any),
1829 typically the former target of the jump that will be redirected to
1830 the new label. */
1832 static rtx_insn *
1833 get_label_before (rtx_insn *insn, rtx sibling)
1835 rtx_insn *label;
1837 /* Find an existing label at this point
1838 or make a new one if there is none. */
1839 label = prev_nonnote_insn (insn);
1841 if (label == 0 || !LABEL_P (label))
1843 rtx_insn *prev = PREV_INSN (insn);
1845 label = gen_label_rtx ();
1846 emit_label_after (label, prev);
1847 LABEL_NUSES (label) = 0;
1848 if (sibling)
1850 sibling_labels.safe_push (label);
1851 sibling_labels.safe_push (sibling);
1854 return label;
1857 /* Scan a function looking for insns that need a delay slot and find insns to
1858 put into the delay slot.
1860 NON_JUMPS_P is nonzero if we are to only try to fill non-jump insns (such
1861 as calls). We do these first since we don't want jump insns (that are
1862 easier to fill) to get the only insns that could be used for non-jump insns.
1863 When it is zero, only try to fill JUMP_INSNs.
1865 When slots are filled in this manner, the insns (including the
1866 delay_insn) are put together in a SEQUENCE rtx. In this fashion,
1867 it is possible to tell whether a delay slot has really been filled
1868 or not. `final' knows how to deal with this, by communicating
1869 through FINAL_SEQUENCE. */
1871 static void
1872 fill_simple_delay_slots (int non_jumps_p)
1874 rtx_insn *insn, *trial, *next_trial;
1875 rtx pat;
1876 int i;
1877 int num_unfilled_slots = unfilled_slots_next - unfilled_slots_base;
1878 struct resources needed, set;
1879 int slots_to_fill, slots_filled;
1880 auto_vec<rtx_insn *, 5> delay_list;
1882 for (i = 0; i < num_unfilled_slots; i++)
1884 int flags;
1885 /* Get the next insn to fill. If it has already had any slots assigned,
1886 we can't do anything with it. Maybe we'll improve this later. */
1888 insn = unfilled_slots_base[i];
1889 if (insn == 0
1890 || insn->deleted ()
1891 || (NONJUMP_INSN_P (insn)
1892 && GET_CODE (PATTERN (insn)) == SEQUENCE)
1893 || (JUMP_P (insn) && non_jumps_p)
1894 || (!JUMP_P (insn) && ! non_jumps_p))
1895 continue;
1897 /* It may have been that this insn used to need delay slots, but
1898 now doesn't; ignore in that case. This can happen, for example,
1899 on the HP PA RISC, where the number of delay slots depends on
1900 what insns are nearby. */
1901 slots_to_fill = num_delay_slots (insn);
1903 /* Some machine description have defined instructions to have
1904 delay slots only in certain circumstances which may depend on
1905 nearby insns (which change due to reorg's actions).
1907 For example, the PA port normally has delay slots for unconditional
1908 jumps.
1910 However, the PA port claims such jumps do not have a delay slot
1911 if they are immediate successors of certain CALL_INSNs. This
1912 allows the port to favor filling the delay slot of the call with
1913 the unconditional jump. */
1914 if (slots_to_fill == 0)
1915 continue;
1917 /* This insn needs, or can use, some delay slots. SLOTS_TO_FILL
1918 says how many. After initialization, first try optimizing
1920 call _foo call _foo
1921 nop add %o7,.-L1,%o7
1922 b,a L1
1925 If this case applies, the delay slot of the call is filled with
1926 the unconditional jump. This is done first to avoid having the
1927 delay slot of the call filled in the backward scan. Also, since
1928 the unconditional jump is likely to also have a delay slot, that
1929 insn must exist when it is subsequently scanned.
1931 This is tried on each insn with delay slots as some machines
1932 have insns which perform calls, but are not represented as
1933 CALL_INSNs. */
1935 slots_filled = 0;
1936 delay_list.truncate (0);
1938 if (JUMP_P (insn))
1939 flags = get_jump_flags (insn, JUMP_LABEL (insn));
1940 else
1941 flags = get_jump_flags (insn, NULL_RTX);
1943 if ((trial = next_active_insn (insn))
1944 && JUMP_P (trial)
1945 && simplejump_p (trial)
1946 && eligible_for_delay (insn, slots_filled, trial, flags)
1947 && no_labels_between_p (insn, trial)
1948 && ! can_throw_internal (trial))
1950 rtx_insn **tmp;
1951 slots_filled++;
1952 add_to_delay_list (trial, &delay_list);
1954 /* TRIAL may have had its delay slot filled, then unfilled. When
1955 the delay slot is unfilled, TRIAL is placed back on the unfilled
1956 slots obstack. Unfortunately, it is placed on the end of the
1957 obstack, not in its original location. Therefore, we must search
1958 from entry i + 1 to the end of the unfilled slots obstack to
1959 try and find TRIAL. */
1960 tmp = &unfilled_slots_base[i + 1];
1961 while (*tmp != trial && tmp != unfilled_slots_next)
1962 tmp++;
1964 /* Remove the unconditional jump from consideration for delay slot
1965 filling and unthread it. */
1966 if (*tmp == trial)
1967 *tmp = 0;
1969 rtx_insn *next = NEXT_INSN (trial);
1970 rtx_insn *prev = PREV_INSN (trial);
1971 if (prev)
1972 SET_NEXT_INSN (prev) = next;
1973 if (next)
1974 SET_PREV_INSN (next) = prev;
1978 /* Now, scan backwards from the insn to search for a potential
1979 delay-slot candidate. Stop searching when a label or jump is hit.
1981 For each candidate, if it is to go into the delay slot (moved
1982 forward in execution sequence), it must not need or set any resources
1983 that were set by later insns and must not set any resources that
1984 are needed for those insns.
1986 The delay slot insn itself sets resources unless it is a call
1987 (in which case the called routine, not the insn itself, is doing
1988 the setting). */
1990 if (slots_filled < slots_to_fill)
1992 /* If the flags register is dead after the insn, then we want to be
1993 able to accept a candidate that clobbers it. For this purpose,
1994 we need to filter the flags register during life analysis, so
1995 that it doesn't create RAW and WAW dependencies, while still
1996 creating the necessary WAR dependencies. */
1997 bool filter_flags
1998 = (slots_to_fill == 1
1999 && targetm.flags_regnum != INVALID_REGNUM
2000 && find_regno_note (insn, REG_DEAD, targetm.flags_regnum));
2001 struct resources fset;
2002 CLEAR_RESOURCE (&needed);
2003 CLEAR_RESOURCE (&set);
2004 mark_set_resources (insn, &set, 0, MARK_SRC_DEST);
2005 if (filter_flags)
2007 CLEAR_RESOURCE (&fset);
2008 mark_set_resources (insn, &fset, 0, MARK_SRC_DEST);
2010 mark_referenced_resources (insn, &needed, false);
2012 for (trial = prev_nonnote_insn (insn); ! stop_search_p (trial, 1);
2013 trial = next_trial)
2015 next_trial = prev_nonnote_insn (trial);
2017 /* This must be an INSN or CALL_INSN. */
2018 pat = PATTERN (trial);
2020 /* Stand-alone USE and CLOBBER are just for flow. */
2021 if (GET_CODE (pat) == USE || GET_CODE (pat) == CLOBBER)
2022 continue;
2024 /* Check for resource conflict first, to avoid unnecessary
2025 splitting. */
2026 if (! insn_references_resource_p (trial, &set, true)
2027 && ! insn_sets_resource_p (trial,
2028 filter_flags ? &fset : &set,
2029 true)
2030 && ! insn_sets_resource_p (trial, &needed, true)
2031 /* Can't separate set of cc0 from its use. */
2032 && (!HAVE_cc0 || ! (reg_mentioned_p (cc0_rtx, pat) && ! sets_cc0_p (pat)))
2033 && ! can_throw_internal (trial))
2035 trial = try_split (pat, trial, 1);
2036 next_trial = prev_nonnote_insn (trial);
2037 if (eligible_for_delay (insn, slots_filled, trial, flags))
2039 /* In this case, we are searching backward, so if we
2040 find insns to put on the delay list, we want
2041 to put them at the head, rather than the
2042 tail, of the list. */
2044 update_reg_dead_notes (trial, insn);
2045 delay_list.safe_insert (0, trial);
2046 update_block (trial, trial);
2047 delete_related_insns (trial);
2048 if (slots_to_fill == ++slots_filled)
2049 break;
2050 continue;
2054 mark_set_resources (trial, &set, 0, MARK_SRC_DEST_CALL);
2055 if (filter_flags)
2057 mark_set_resources (trial, &fset, 0, MARK_SRC_DEST_CALL);
2058 /* If the flags register is set, then it doesn't create RAW
2059 dependencies any longer and it also doesn't create WAW
2060 dependencies since it's dead after the original insn. */
2061 if (TEST_HARD_REG_BIT (fset.regs, targetm.flags_regnum))
2063 CLEAR_HARD_REG_BIT (needed.regs, targetm.flags_regnum);
2064 CLEAR_HARD_REG_BIT (fset.regs, targetm.flags_regnum);
2067 mark_referenced_resources (trial, &needed, true);
2071 /* If all needed slots haven't been filled, we come here. */
2073 /* Try to optimize case of jumping around a single insn. */
2074 if ((ANNUL_IFTRUE_SLOTS || ANNUL_IFFALSE_SLOTS)
2075 && slots_filled != slots_to_fill
2076 && delay_list.is_empty ()
2077 && JUMP_P (insn)
2078 && (condjump_p (insn) || condjump_in_parallel_p (insn))
2079 && !ANY_RETURN_P (JUMP_LABEL (insn)))
2081 optimize_skip (as_a <rtx_jump_insn *> (insn), &delay_list);
2082 if (!delay_list.is_empty ())
2083 slots_filled += 1;
2086 /* Try to get insns from beyond the insn needing the delay slot.
2087 These insns can neither set or reference resources set in insns being
2088 skipped, cannot set resources in the insn being skipped, and, if this
2089 is a CALL_INSN (or a CALL_INSN is passed), cannot trap (because the
2090 call might not return).
2092 There used to be code which continued past the target label if
2093 we saw all uses of the target label. This code did not work,
2094 because it failed to account for some instructions which were
2095 both annulled and marked as from the target. This can happen as a
2096 result of optimize_skip. Since this code was redundant with
2097 fill_eager_delay_slots anyways, it was just deleted. */
2099 if (slots_filled != slots_to_fill
2100 /* If this instruction could throw an exception which is
2101 caught in the same function, then it's not safe to fill
2102 the delay slot with an instruction from beyond this
2103 point. For example, consider:
2105 int i = 2;
2107 try {
2108 f();
2109 i = 3;
2110 } catch (...) {}
2112 return i;
2114 Even though `i' is a local variable, we must be sure not
2115 to put `i = 3' in the delay slot if `f' might throw an
2116 exception.
2118 Presumably, we should also check to see if we could get
2119 back to this function via `setjmp'. */
2120 && ! can_throw_internal (insn)
2121 && !JUMP_P (insn))
2123 int maybe_never = 0;
2124 rtx pat, trial_delay;
2126 CLEAR_RESOURCE (&needed);
2127 CLEAR_RESOURCE (&set);
2128 mark_set_resources (insn, &set, 0, MARK_SRC_DEST_CALL);
2129 mark_referenced_resources (insn, &needed, true);
2131 if (CALL_P (insn))
2132 maybe_never = 1;
2134 for (trial = next_nonnote_insn (insn); !stop_search_p (trial, 1);
2135 trial = next_trial)
2137 next_trial = next_nonnote_insn (trial);
2139 /* This must be an INSN or CALL_INSN. */
2140 pat = PATTERN (trial);
2142 /* Stand-alone USE and CLOBBER are just for flow. */
2143 if (GET_CODE (pat) == USE || GET_CODE (pat) == CLOBBER)
2144 continue;
2146 /* If this already has filled delay slots, get the insn needing
2147 the delay slots. */
2148 if (GET_CODE (pat) == SEQUENCE)
2149 trial_delay = XVECEXP (pat, 0, 0);
2150 else
2151 trial_delay = trial;
2153 /* Stop our search when seeing a jump. */
2154 if (JUMP_P (trial_delay))
2155 break;
2157 /* See if we have a resource problem before we try to split. */
2158 if (GET_CODE (pat) != SEQUENCE
2159 && ! insn_references_resource_p (trial, &set, true)
2160 && ! insn_sets_resource_p (trial, &set, true)
2161 && ! insn_sets_resource_p (trial, &needed, true)
2162 && (!HAVE_cc0 && ! (reg_mentioned_p (cc0_rtx, pat) && ! sets_cc0_p (pat)))
2163 && ! (maybe_never && may_trap_or_fault_p (pat))
2164 && (trial = try_split (pat, trial, 0))
2165 && eligible_for_delay (insn, slots_filled, trial, flags)
2166 && ! can_throw_internal (trial))
2168 next_trial = next_nonnote_insn (trial);
2169 add_to_delay_list (trial, &delay_list);
2170 if (HAVE_cc0 && reg_mentioned_p (cc0_rtx, pat))
2171 link_cc0_insns (trial);
2173 delete_related_insns (trial);
2174 if (slots_to_fill == ++slots_filled)
2175 break;
2176 continue;
2179 mark_set_resources (trial, &set, 0, MARK_SRC_DEST_CALL);
2180 mark_referenced_resources (trial, &needed, true);
2182 /* Ensure we don't put insns between the setting of cc and the
2183 comparison by moving a setting of cc into an earlier delay
2184 slot since these insns could clobber the condition code. */
2185 set.cc = 1;
2187 /* If this is a call, we might not get here. */
2188 if (CALL_P (trial_delay))
2189 maybe_never = 1;
2192 /* If there are slots left to fill and our search was stopped by an
2193 unconditional branch, try the insn at the branch target. We can
2194 redirect the branch if it works.
2196 Don't do this if the insn at the branch target is a branch. */
2197 if (slots_to_fill != slots_filled
2198 && trial
2199 && jump_to_label_p (trial)
2200 && simplejump_p (trial)
2201 && (next_trial = next_active_insn (JUMP_LABEL_AS_INSN (trial))) != 0
2202 && ! (NONJUMP_INSN_P (next_trial)
2203 && GET_CODE (PATTERN (next_trial)) == SEQUENCE)
2204 && !JUMP_P (next_trial)
2205 && ! insn_references_resource_p (next_trial, &set, true)
2206 && ! insn_sets_resource_p (next_trial, &set, true)
2207 && ! insn_sets_resource_p (next_trial, &needed, true)
2208 && (!HAVE_cc0 || ! reg_mentioned_p (cc0_rtx, PATTERN (next_trial)))
2209 && ! (maybe_never && may_trap_or_fault_p (PATTERN (next_trial)))
2210 && (next_trial = try_split (PATTERN (next_trial), next_trial, 0))
2211 && eligible_for_delay (insn, slots_filled, next_trial, flags)
2212 && ! can_throw_internal (trial))
2214 /* See comment in relax_delay_slots about necessity of using
2215 next_real_insn here. */
2216 rtx_insn *new_label = next_real_insn (next_trial);
2218 if (new_label != 0)
2219 new_label = get_label_before (new_label, JUMP_LABEL (trial));
2220 else
2221 new_label = find_end_label (simple_return_rtx);
2223 if (new_label)
2225 add_to_delay_list (copy_delay_slot_insn (next_trial),
2226 &delay_list);
2227 slots_filled++;
2228 reorg_redirect_jump (as_a <rtx_jump_insn *> (trial),
2229 new_label);
2234 /* If this is an unconditional jump, then try to get insns from the
2235 target of the jump. */
2236 rtx_jump_insn *jump_insn;
2237 if ((jump_insn = dyn_cast <rtx_jump_insn *> (insn))
2238 && simplejump_p (jump_insn)
2239 && slots_filled != slots_to_fill)
2240 fill_slots_from_thread (jump_insn, const_true_rtx,
2241 next_active_insn (JUMP_LABEL_AS_INSN (insn)),
2242 NULL, 1, 1, own_thread_p (JUMP_LABEL (insn),
2243 JUMP_LABEL (insn), 0),
2244 slots_to_fill, &slots_filled, &delay_list);
2246 if (!delay_list.is_empty ())
2247 unfilled_slots_base[i]
2248 = emit_delay_sequence (insn, delay_list, slots_filled);
2250 if (slots_to_fill == slots_filled)
2251 unfilled_slots_base[i] = 0;
2253 note_delay_statistics (slots_filled, 0);
2257 /* Follow any unconditional jump at LABEL, for the purpose of redirecting JUMP;
2258 return the ultimate label reached by any such chain of jumps.
2259 Return a suitable return rtx if the chain ultimately leads to a
2260 return instruction.
2261 If LABEL is not followed by a jump, return LABEL.
2262 If the chain loops or we can't find end, return LABEL,
2263 since that tells caller to avoid changing the insn.
2264 If the returned label is obtained by following a crossing jump,
2265 set *CROSSING to true, otherwise set it to false. */
2267 static rtx
2268 follow_jumps (rtx label, rtx_insn *jump, bool *crossing)
2270 rtx_insn *insn;
2271 rtx_insn *next;
2272 int depth;
2274 *crossing = false;
2275 if (ANY_RETURN_P (label))
2276 return label;
2278 rtx_insn *value = as_a <rtx_insn *> (label);
2280 for (depth = 0;
2281 (depth < 10
2282 && (insn = next_active_insn (value)) != 0
2283 && JUMP_P (insn)
2284 && JUMP_LABEL (insn) != NULL_RTX
2285 && ((any_uncondjump_p (insn) && onlyjump_p (insn))
2286 || ANY_RETURN_P (PATTERN (insn)))
2287 && (next = NEXT_INSN (insn))
2288 && BARRIER_P (next));
2289 depth++)
2291 rtx this_label_or_return = JUMP_LABEL (insn);
2293 /* If we have found a cycle, make the insn jump to itself. */
2294 if (this_label_or_return == label)
2295 return label;
2297 /* Cannot follow returns and cannot look through tablejumps. */
2298 if (ANY_RETURN_P (this_label_or_return))
2299 return this_label_or_return;
2301 rtx_insn *this_label = as_a <rtx_insn *> (this_label_or_return);
2302 if (NEXT_INSN (this_label)
2303 && JUMP_TABLE_DATA_P (NEXT_INSN (this_label)))
2304 break;
2306 if (!targetm.can_follow_jump (jump, insn))
2307 break;
2308 if (!*crossing)
2309 *crossing = CROSSING_JUMP_P (jump);
2310 value = this_label;
2312 if (depth == 10)
2313 return label;
2314 return value;
2317 /* Try to find insns to place in delay slots.
2319 INSN is the jump needing SLOTS_TO_FILL delay slots. It tests CONDITION
2320 or is an unconditional branch if CONDITION is const_true_rtx.
2321 *PSLOTS_FILLED is updated with the number of slots that we have filled.
2323 THREAD is a flow-of-control, either the insns to be executed if the
2324 branch is true or if the branch is false, THREAD_IF_TRUE says which.
2326 OPPOSITE_THREAD is the thread in the opposite direction. It is used
2327 to see if any potential delay slot insns set things needed there.
2329 LIKELY is nonzero if it is extremely likely that the branch will be
2330 taken and THREAD_IF_TRUE is set. This is used for the branch at the
2331 end of a loop back up to the top.
2333 OWN_THREAD and OWN_OPPOSITE_THREAD are true if we are the only user of the
2334 thread. I.e., it is the fallthrough code of our jump or the target of the
2335 jump when we are the only jump going there.
2337 If OWN_THREAD is false, it must be the "true" thread of a jump. In that
2338 case, we can only take insns from the head of the thread for our delay
2339 slot. We then adjust the jump to point after the insns we have taken. */
2341 static void
2342 fill_slots_from_thread (rtx_jump_insn *insn, rtx condition,
2343 rtx thread_or_return, rtx opposite_thread, int likely,
2344 int thread_if_true, int own_thread, int slots_to_fill,
2345 int *pslots_filled, vec<rtx_insn *> *delay_list)
2347 rtx new_thread;
2348 struct resources opposite_needed, set, needed;
2349 rtx_insn *trial;
2350 int lose = 0;
2351 int must_annul = 0;
2352 int flags;
2354 /* Validate our arguments. */
2355 gcc_assert (condition != const_true_rtx || thread_if_true);
2356 gcc_assert (own_thread || thread_if_true);
2358 flags = get_jump_flags (insn, JUMP_LABEL (insn));
2360 /* If our thread is the end of subroutine, we can't get any delay
2361 insns from that. */
2362 if (thread_or_return == NULL_RTX || ANY_RETURN_P (thread_or_return))
2363 return;
2365 rtx_insn *thread = as_a <rtx_insn *> (thread_or_return);
2367 /* If this is an unconditional branch, nothing is needed at the
2368 opposite thread. Otherwise, compute what is needed there. */
2369 if (condition == const_true_rtx)
2370 CLEAR_RESOURCE (&opposite_needed);
2371 else
2372 mark_target_live_regs (get_insns (), opposite_thread, &opposite_needed);
2374 /* If the insn at THREAD can be split, do it here to avoid having to
2375 update THREAD and NEW_THREAD if it is done in the loop below. Also
2376 initialize NEW_THREAD. */
2378 new_thread = thread = try_split (PATTERN (thread), thread, 0);
2380 /* Scan insns at THREAD. We are looking for an insn that can be removed
2381 from THREAD (it neither sets nor references resources that were set
2382 ahead of it and it doesn't set anything needs by the insns ahead of
2383 it) and that either can be placed in an annulling insn or aren't
2384 needed at OPPOSITE_THREAD. */
2386 CLEAR_RESOURCE (&needed);
2387 CLEAR_RESOURCE (&set);
2389 /* If we do not own this thread, we must stop as soon as we find
2390 something that we can't put in a delay slot, since all we can do
2391 is branch into THREAD at a later point. Therefore, labels stop
2392 the search if this is not the `true' thread. */
2394 for (trial = thread;
2395 ! stop_search_p (trial, ! thread_if_true) && (! lose || own_thread);
2396 trial = next_nonnote_insn (trial))
2398 rtx pat, old_trial;
2400 /* If we have passed a label, we no longer own this thread. */
2401 if (LABEL_P (trial))
2403 own_thread = 0;
2404 continue;
2407 pat = PATTERN (trial);
2408 if (GET_CODE (pat) == USE || GET_CODE (pat) == CLOBBER)
2409 continue;
2411 /* If TRIAL conflicts with the insns ahead of it, we lose. Also,
2412 don't separate or copy insns that set and use CC0. */
2413 if (! insn_references_resource_p (trial, &set, true)
2414 && ! insn_sets_resource_p (trial, &set, true)
2415 && ! insn_sets_resource_p (trial, &needed, true)
2416 && (!HAVE_cc0 || (! (reg_mentioned_p (cc0_rtx, pat)
2417 && (! own_thread || ! sets_cc0_p (pat)))))
2418 && ! can_throw_internal (trial))
2420 rtx_insn *prior_insn;
2422 /* If TRIAL is redundant with some insn before INSN, we don't
2423 actually need to add it to the delay list; we can merely pretend
2424 we did. */
2425 if ((prior_insn = redundant_insn (trial, insn, *delay_list)))
2427 fix_reg_dead_note (prior_insn, insn);
2428 if (own_thread)
2430 update_block (trial, thread);
2431 if (trial == thread)
2433 thread = next_active_insn (thread);
2434 if (new_thread == trial)
2435 new_thread = thread;
2438 delete_related_insns (trial);
2440 else
2442 update_reg_unused_notes (prior_insn, trial);
2443 new_thread = next_active_insn (trial);
2446 continue;
2449 /* There are two ways we can win: If TRIAL doesn't set anything
2450 needed at the opposite thread and can't trap, or if it can
2451 go into an annulled delay slot. But we want neither to copy
2452 nor to speculate frame-related insns. */
2453 if (!must_annul
2454 && ((condition == const_true_rtx
2455 && (own_thread || !RTX_FRAME_RELATED_P (trial)))
2456 || (! insn_sets_resource_p (trial, &opposite_needed, true)
2457 && ! may_trap_or_fault_p (pat)
2458 && ! RTX_FRAME_RELATED_P (trial))))
2460 old_trial = trial;
2461 trial = try_split (pat, trial, 0);
2462 if (new_thread == old_trial)
2463 new_thread = trial;
2464 if (thread == old_trial)
2465 thread = trial;
2466 pat = PATTERN (trial);
2467 if (eligible_for_delay (insn, *pslots_filled, trial, flags))
2468 goto winner;
2470 else if (!RTX_FRAME_RELATED_P (trial)
2471 && ((ANNUL_IFTRUE_SLOTS && ! thread_if_true)
2472 || (ANNUL_IFFALSE_SLOTS && thread_if_true)))
2474 old_trial = trial;
2475 trial = try_split (pat, trial, 0);
2476 if (new_thread == old_trial)
2477 new_thread = trial;
2478 if (thread == old_trial)
2479 thread = trial;
2480 pat = PATTERN (trial);
2481 if ((must_annul || delay_list->is_empty ()) && (thread_if_true
2482 ? check_annul_list_true_false (0, *delay_list)
2483 && eligible_for_annul_false (insn, *pslots_filled, trial, flags)
2484 : check_annul_list_true_false (1, *delay_list)
2485 && eligible_for_annul_true (insn, *pslots_filled, trial, flags)))
2487 rtx_insn *temp;
2489 must_annul = 1;
2490 winner:
2492 if (HAVE_cc0 && reg_mentioned_p (cc0_rtx, pat))
2493 link_cc0_insns (trial);
2495 /* If we own this thread, delete the insn. If this is the
2496 destination of a branch, show that a basic block status
2497 may have been updated. In any case, mark the new
2498 starting point of this thread. */
2499 if (own_thread)
2501 rtx note;
2503 update_block (trial, thread);
2504 if (trial == thread)
2506 thread = next_active_insn (thread);
2507 if (new_thread == trial)
2508 new_thread = thread;
2511 /* We are moving this insn, not deleting it. We must
2512 temporarily increment the use count on any referenced
2513 label lest it be deleted by delete_related_insns. */
2514 for (note = REG_NOTES (trial);
2515 note != NULL_RTX;
2516 note = XEXP (note, 1))
2517 if (REG_NOTE_KIND (note) == REG_LABEL_OPERAND
2518 || REG_NOTE_KIND (note) == REG_LABEL_TARGET)
2520 /* REG_LABEL_OPERAND could be
2521 NOTE_INSN_DELETED_LABEL too. */
2522 if (LABEL_P (XEXP (note, 0)))
2523 LABEL_NUSES (XEXP (note, 0))++;
2524 else
2525 gcc_assert (REG_NOTE_KIND (note)
2526 == REG_LABEL_OPERAND);
2528 if (jump_to_label_p (trial))
2529 LABEL_NUSES (JUMP_LABEL (trial))++;
2531 delete_related_insns (trial);
2533 for (note = REG_NOTES (trial);
2534 note != NULL_RTX;
2535 note = XEXP (note, 1))
2536 if (REG_NOTE_KIND (note) == REG_LABEL_OPERAND
2537 || REG_NOTE_KIND (note) == REG_LABEL_TARGET)
2539 /* REG_LABEL_OPERAND could be
2540 NOTE_INSN_DELETED_LABEL too. */
2541 if (LABEL_P (XEXP (note, 0)))
2542 LABEL_NUSES (XEXP (note, 0))--;
2543 else
2544 gcc_assert (REG_NOTE_KIND (note)
2545 == REG_LABEL_OPERAND);
2547 if (jump_to_label_p (trial))
2548 LABEL_NUSES (JUMP_LABEL (trial))--;
2550 else
2551 new_thread = next_active_insn (trial);
2553 temp = own_thread ? trial : copy_delay_slot_insn (trial);
2554 if (thread_if_true)
2555 INSN_FROM_TARGET_P (temp) = 1;
2557 add_to_delay_list (temp, delay_list);
2559 if (slots_to_fill == ++(*pslots_filled))
2561 /* Even though we have filled all the slots, we
2562 may be branching to a location that has a
2563 redundant insn. Skip any if so. */
2564 while (new_thread && ! own_thread
2565 && ! insn_sets_resource_p (new_thread, &set, true)
2566 && ! insn_sets_resource_p (new_thread, &needed,
2567 true)
2568 && ! insn_references_resource_p (new_thread,
2569 &set, true)
2570 && (prior_insn
2571 = redundant_insn (new_thread, insn,
2572 *delay_list)))
2574 /* We know we do not own the thread, so no need
2575 to call update_block and delete_insn. */
2576 fix_reg_dead_note (prior_insn, insn);
2577 update_reg_unused_notes (prior_insn, new_thread);
2578 new_thread
2579 = next_active_insn (as_a<rtx_insn *> (new_thread));
2581 break;
2584 continue;
2589 /* This insn can't go into a delay slot. */
2590 lose = 1;
2591 mark_set_resources (trial, &set, 0, MARK_SRC_DEST_CALL);
2592 mark_referenced_resources (trial, &needed, true);
2594 /* Ensure we don't put insns between the setting of cc and the comparison
2595 by moving a setting of cc into an earlier delay slot since these insns
2596 could clobber the condition code. */
2597 set.cc = 1;
2599 /* If this insn is a register-register copy and the next insn has
2600 a use of our destination, change it to use our source. That way,
2601 it will become a candidate for our delay slot the next time
2602 through this loop. This case occurs commonly in loops that
2603 scan a list.
2605 We could check for more complex cases than those tested below,
2606 but it doesn't seem worth it. It might also be a good idea to try
2607 to swap the two insns. That might do better.
2609 We can't do this if the next insn modifies our destination, because
2610 that would make the replacement into the insn invalid. We also can't
2611 do this if it modifies our source, because it might be an earlyclobber
2612 operand. This latter test also prevents updating the contents of
2613 a PRE_INC. We also can't do this if there's overlap of source and
2614 destination. Overlap may happen for larger-than-register-size modes. */
2616 if (NONJUMP_INSN_P (trial) && GET_CODE (pat) == SET
2617 && REG_P (SET_SRC (pat))
2618 && REG_P (SET_DEST (pat))
2619 && !reg_overlap_mentioned_p (SET_DEST (pat), SET_SRC (pat)))
2621 rtx_insn *next = next_nonnote_insn (trial);
2623 if (next && NONJUMP_INSN_P (next)
2624 && GET_CODE (PATTERN (next)) != USE
2625 && ! reg_set_p (SET_DEST (pat), next)
2626 && ! reg_set_p (SET_SRC (pat), next)
2627 && reg_referenced_p (SET_DEST (pat), PATTERN (next))
2628 && ! modified_in_p (SET_DEST (pat), next))
2629 validate_replace_rtx (SET_DEST (pat), SET_SRC (pat), next);
2633 /* If we stopped on a branch insn that has delay slots, see if we can
2634 steal some of the insns in those slots. */
2635 if (trial && NONJUMP_INSN_P (trial)
2636 && GET_CODE (PATTERN (trial)) == SEQUENCE
2637 && JUMP_P (XVECEXP (PATTERN (trial), 0, 0)))
2639 rtx_sequence *sequence = as_a <rtx_sequence *> (PATTERN (trial));
2640 /* If this is the `true' thread, we will want to follow the jump,
2641 so we can only do this if we have taken everything up to here. */
2642 if (thread_if_true && trial == new_thread)
2644 steal_delay_list_from_target (insn, condition, sequence,
2645 delay_list, &set, &needed,
2646 &opposite_needed, slots_to_fill,
2647 pslots_filled, &must_annul,
2648 &new_thread);
2649 /* If we owned the thread and are told that it branched
2650 elsewhere, make sure we own the thread at the new location. */
2651 if (own_thread && trial != new_thread)
2652 own_thread = own_thread_p (new_thread, new_thread, 0);
2654 else if (! thread_if_true)
2655 steal_delay_list_from_fallthrough (insn, condition, sequence,
2656 delay_list, &set, &needed,
2657 &opposite_needed, slots_to_fill,
2658 pslots_filled, &must_annul);
2661 /* If we haven't found anything for this delay slot and it is very
2662 likely that the branch will be taken, see if the insn at our target
2663 increments or decrements a register with an increment that does not
2664 depend on the destination register. If so, try to place the opposite
2665 arithmetic insn after the jump insn and put the arithmetic insn in the
2666 delay slot. If we can't do this, return. */
2667 if (delay_list->is_empty () && likely
2668 && new_thread && !ANY_RETURN_P (new_thread)
2669 && NONJUMP_INSN_P (new_thread)
2670 && !RTX_FRAME_RELATED_P (new_thread)
2671 && GET_CODE (PATTERN (new_thread)) != ASM_INPUT
2672 && asm_noperands (PATTERN (new_thread)) < 0)
2674 rtx pat = PATTERN (new_thread);
2675 rtx dest;
2676 rtx src;
2678 /* We know "new_thread" is an insn due to NONJUMP_INSN_P (new_thread)
2679 above. */
2680 trial = as_a <rtx_insn *> (new_thread);
2681 pat = PATTERN (trial);
2683 if (!NONJUMP_INSN_P (trial)
2684 || GET_CODE (pat) != SET
2685 || ! eligible_for_delay (insn, 0, trial, flags)
2686 || can_throw_internal (trial))
2687 return;
2689 dest = SET_DEST (pat), src = SET_SRC (pat);
2690 if ((GET_CODE (src) == PLUS || GET_CODE (src) == MINUS)
2691 && rtx_equal_p (XEXP (src, 0), dest)
2692 && (!FLOAT_MODE_P (GET_MODE (src))
2693 || flag_unsafe_math_optimizations)
2694 && ! reg_overlap_mentioned_p (dest, XEXP (src, 1))
2695 && ! side_effects_p (pat))
2697 rtx other = XEXP (src, 1);
2698 rtx new_arith;
2699 rtx_insn *ninsn;
2701 /* If this is a constant adjustment, use the same code with
2702 the negated constant. Otherwise, reverse the sense of the
2703 arithmetic. */
2704 if (CONST_INT_P (other))
2705 new_arith = gen_rtx_fmt_ee (GET_CODE (src), GET_MODE (src), dest,
2706 negate_rtx (GET_MODE (src), other));
2707 else
2708 new_arith = gen_rtx_fmt_ee (GET_CODE (src) == PLUS ? MINUS : PLUS,
2709 GET_MODE (src), dest, other);
2711 ninsn = emit_insn_after (gen_rtx_SET (dest, new_arith), insn);
2713 if (recog_memoized (ninsn) < 0
2714 || (extract_insn (ninsn),
2715 !constrain_operands (1, get_preferred_alternatives (ninsn))))
2717 delete_related_insns (ninsn);
2718 return;
2721 if (own_thread)
2723 update_block (trial, thread);
2724 if (trial == thread)
2726 thread = next_active_insn (thread);
2727 if (new_thread == trial)
2728 new_thread = thread;
2730 delete_related_insns (trial);
2732 else
2733 new_thread = next_active_insn (trial);
2735 ninsn = own_thread ? trial : copy_delay_slot_insn (trial);
2736 if (thread_if_true)
2737 INSN_FROM_TARGET_P (ninsn) = 1;
2739 add_to_delay_list (ninsn, delay_list);
2740 (*pslots_filled)++;
2744 if (!delay_list->is_empty () && must_annul)
2745 INSN_ANNULLED_BRANCH_P (insn) = 1;
2747 /* If we are to branch into the middle of this thread, find an appropriate
2748 label or make a new one if none, and redirect INSN to it. If we hit the
2749 end of the function, use the end-of-function label. */
2750 if (new_thread != thread)
2752 rtx label;
2753 bool crossing = false;
2755 gcc_assert (thread_if_true);
2757 if (new_thread && simplejump_or_return_p (new_thread)
2758 && redirect_with_delay_list_safe_p (insn,
2759 JUMP_LABEL (new_thread),
2760 *delay_list))
2761 new_thread = follow_jumps (JUMP_LABEL (new_thread), insn,
2762 &crossing);
2764 if (ANY_RETURN_P (new_thread))
2765 label = find_end_label (new_thread);
2766 else if (LABEL_P (new_thread))
2767 label = new_thread;
2768 else
2769 label = get_label_before (as_a <rtx_insn *> (new_thread),
2770 JUMP_LABEL (insn));
2772 if (label)
2774 reorg_redirect_jump (insn, label);
2775 if (crossing)
2776 CROSSING_JUMP_P (insn) = 1;
2781 /* Make another attempt to find insns to place in delay slots.
2783 We previously looked for insns located in front of the delay insn
2784 and, for non-jump delay insns, located behind the delay insn.
2786 Here only try to schedule jump insns and try to move insns from either
2787 the target or the following insns into the delay slot. If annulling is
2788 supported, we will be likely to do this. Otherwise, we can do this only
2789 if safe. */
2791 static void
2792 fill_eager_delay_slots (void)
2794 rtx_insn *insn;
2795 int i;
2796 int num_unfilled_slots = unfilled_slots_next - unfilled_slots_base;
2798 for (i = 0; i < num_unfilled_slots; i++)
2800 rtx condition;
2801 rtx target_label, insn_at_target;
2802 rtx_insn *fallthrough_insn;
2803 auto_vec<rtx_insn *, 5> delay_list;
2804 rtx_jump_insn *jump_insn;
2805 int own_target;
2806 int own_fallthrough;
2807 int prediction, slots_to_fill, slots_filled;
2809 insn = unfilled_slots_base[i];
2810 if (insn == 0
2811 || insn->deleted ()
2812 || ! (jump_insn = dyn_cast <rtx_jump_insn *> (insn))
2813 || ! (condjump_p (jump_insn) || condjump_in_parallel_p (jump_insn)))
2814 continue;
2816 slots_to_fill = num_delay_slots (jump_insn);
2817 /* Some machine description have defined instructions to have
2818 delay slots only in certain circumstances which may depend on
2819 nearby insns (which change due to reorg's actions).
2821 For example, the PA port normally has delay slots for unconditional
2822 jumps.
2824 However, the PA port claims such jumps do not have a delay slot
2825 if they are immediate successors of certain CALL_INSNs. This
2826 allows the port to favor filling the delay slot of the call with
2827 the unconditional jump. */
2828 if (slots_to_fill == 0)
2829 continue;
2831 slots_filled = 0;
2832 target_label = JUMP_LABEL (jump_insn);
2833 condition = get_branch_condition (jump_insn, target_label);
2835 if (condition == 0)
2836 continue;
2838 /* Get the next active fallthrough and target insns and see if we own
2839 them. Then see whether the branch is likely true. We don't need
2840 to do a lot of this for unconditional branches. */
2842 insn_at_target = first_active_target_insn (target_label);
2843 own_target = own_thread_p (target_label, target_label, 0);
2845 if (condition == const_true_rtx)
2847 own_fallthrough = 0;
2848 fallthrough_insn = 0;
2849 prediction = 2;
2851 else
2853 fallthrough_insn = next_active_insn (jump_insn);
2854 own_fallthrough = own_thread_p (NEXT_INSN (jump_insn), NULL_RTX, 1);
2855 prediction = mostly_true_jump (jump_insn);
2858 /* If this insn is expected to branch, first try to get insns from our
2859 target, then our fallthrough insns. If it is not expected to branch,
2860 try the other order. */
2862 if (prediction > 0)
2864 fill_slots_from_thread (jump_insn, condition, insn_at_target,
2865 fallthrough_insn, prediction == 2, 1,
2866 own_target,
2867 slots_to_fill, &slots_filled, &delay_list);
2869 if (delay_list.is_empty () && own_fallthrough)
2871 /* Even though we didn't find anything for delay slots,
2872 we might have found a redundant insn which we deleted
2873 from the thread that was filled. So we have to recompute
2874 the next insn at the target. */
2875 target_label = JUMP_LABEL (jump_insn);
2876 insn_at_target = first_active_target_insn (target_label);
2878 fill_slots_from_thread (jump_insn, condition, fallthrough_insn,
2879 insn_at_target, 0, 0, own_fallthrough,
2880 slots_to_fill, &slots_filled,
2881 &delay_list);
2884 else
2886 if (own_fallthrough)
2887 fill_slots_from_thread (jump_insn, condition, fallthrough_insn,
2888 insn_at_target, 0, 0, own_fallthrough,
2889 slots_to_fill, &slots_filled, &delay_list);
2891 if (delay_list.is_empty ())
2892 fill_slots_from_thread (jump_insn, condition, insn_at_target,
2893 next_active_insn (insn), 0, 1, own_target,
2894 slots_to_fill, &slots_filled, &delay_list);
2897 if (!delay_list.is_empty ())
2898 unfilled_slots_base[i]
2899 = emit_delay_sequence (jump_insn, delay_list, slots_filled);
2901 if (slots_to_fill == slots_filled)
2902 unfilled_slots_base[i] = 0;
2904 note_delay_statistics (slots_filled, 1);
2908 static void delete_computation (rtx_insn *insn);
2910 /* Recursively delete prior insns that compute the value (used only by INSN
2911 which the caller is deleting) stored in the register mentioned by NOTE
2912 which is a REG_DEAD note associated with INSN. */
2914 static void
2915 delete_prior_computation (rtx note, rtx_insn *insn)
2917 rtx_insn *our_prev;
2918 rtx reg = XEXP (note, 0);
2920 for (our_prev = prev_nonnote_insn (insn);
2921 our_prev && (NONJUMP_INSN_P (our_prev)
2922 || CALL_P (our_prev));
2923 our_prev = prev_nonnote_insn (our_prev))
2925 rtx pat = PATTERN (our_prev);
2927 /* If we reach a CALL which is not calling a const function
2928 or the callee pops the arguments, then give up. */
2929 if (CALL_P (our_prev)
2930 && (! RTL_CONST_CALL_P (our_prev)
2931 || GET_CODE (pat) != SET || GET_CODE (SET_SRC (pat)) != CALL))
2932 break;
2934 /* If we reach a SEQUENCE, it is too complex to try to
2935 do anything with it, so give up. We can be run during
2936 and after reorg, so SEQUENCE rtl can legitimately show
2937 up here. */
2938 if (GET_CODE (pat) == SEQUENCE)
2939 break;
2941 if (GET_CODE (pat) == USE
2942 && NONJUMP_INSN_P (XEXP (pat, 0)))
2943 /* reorg creates USEs that look like this. We leave them
2944 alone because reorg needs them for its own purposes. */
2945 break;
2947 if (reg_set_p (reg, pat))
2949 if (side_effects_p (pat) && !CALL_P (our_prev))
2950 break;
2952 if (GET_CODE (pat) == PARALLEL)
2954 /* If we find a SET of something else, we can't
2955 delete the insn. */
2957 int i;
2959 for (i = 0; i < XVECLEN (pat, 0); i++)
2961 rtx part = XVECEXP (pat, 0, i);
2963 if (GET_CODE (part) == SET
2964 && SET_DEST (part) != reg)
2965 break;
2968 if (i == XVECLEN (pat, 0))
2969 delete_computation (our_prev);
2971 else if (GET_CODE (pat) == SET
2972 && REG_P (SET_DEST (pat)))
2974 int dest_regno = REGNO (SET_DEST (pat));
2975 int dest_endregno = END_REGNO (SET_DEST (pat));
2976 int regno = REGNO (reg);
2977 int endregno = END_REGNO (reg);
2979 if (dest_regno >= regno
2980 && dest_endregno <= endregno)
2981 delete_computation (our_prev);
2983 /* We may have a multi-word hard register and some, but not
2984 all, of the words of the register are needed in subsequent
2985 insns. Write REG_UNUSED notes for those parts that were not
2986 needed. */
2987 else if (dest_regno <= regno
2988 && dest_endregno >= endregno)
2990 int i;
2992 add_reg_note (our_prev, REG_UNUSED, reg);
2994 for (i = dest_regno; i < dest_endregno; i++)
2995 if (! find_regno_note (our_prev, REG_UNUSED, i))
2996 break;
2998 if (i == dest_endregno)
2999 delete_computation (our_prev);
3003 break;
3006 /* If PAT references the register that dies here, it is an
3007 additional use. Hence any prior SET isn't dead. However, this
3008 insn becomes the new place for the REG_DEAD note. */
3009 if (reg_overlap_mentioned_p (reg, pat))
3011 XEXP (note, 1) = REG_NOTES (our_prev);
3012 REG_NOTES (our_prev) = note;
3013 break;
3018 /* Delete INSN and recursively delete insns that compute values used only
3019 by INSN. This uses the REG_DEAD notes computed during flow analysis.
3021 Look at all our REG_DEAD notes. If a previous insn does nothing other
3022 than set a register that dies in this insn, we can delete that insn
3023 as well.
3025 On machines with CC0, if CC0 is used in this insn, we may be able to
3026 delete the insn that set it. */
3028 static void
3029 delete_computation (rtx_insn *insn)
3031 rtx note, next;
3033 if (HAVE_cc0 && reg_referenced_p (cc0_rtx, PATTERN (insn)))
3035 rtx_insn *prev = prev_nonnote_insn (insn);
3036 /* We assume that at this stage
3037 CC's are always set explicitly
3038 and always immediately before the jump that
3039 will use them. So if the previous insn
3040 exists to set the CC's, delete it
3041 (unless it performs auto-increments, etc.). */
3042 if (prev && NONJUMP_INSN_P (prev)
3043 && sets_cc0_p (PATTERN (prev)))
3045 if (sets_cc0_p (PATTERN (prev)) > 0
3046 && ! side_effects_p (PATTERN (prev)))
3047 delete_computation (prev);
3048 else
3049 /* Otherwise, show that cc0 won't be used. */
3050 add_reg_note (prev, REG_UNUSED, cc0_rtx);
3054 for (note = REG_NOTES (insn); note; note = next)
3056 next = XEXP (note, 1);
3058 if (REG_NOTE_KIND (note) != REG_DEAD
3059 /* Verify that the REG_NOTE is legitimate. */
3060 || !REG_P (XEXP (note, 0)))
3061 continue;
3063 delete_prior_computation (note, insn);
3066 delete_related_insns (insn);
3069 /* If all INSN does is set the pc, delete it,
3070 and delete the insn that set the condition codes for it
3071 if that's what the previous thing was. */
3073 static void
3074 delete_jump (rtx_insn *insn)
3076 rtx set = single_set (insn);
3078 if (set && GET_CODE (SET_DEST (set)) == PC)
3079 delete_computation (insn);
3082 static rtx_insn *
3083 label_before_next_insn (rtx_insn *x, rtx scan_limit)
3085 rtx_insn *insn = next_active_insn (x);
3086 while (insn)
3088 insn = PREV_INSN (insn);
3089 if (insn == scan_limit || insn == NULL_RTX)
3090 return NULL;
3091 if (LABEL_P (insn))
3092 break;
3094 return insn;
3097 /* Return TRUE if there is a NOTE_INSN_SWITCH_TEXT_SECTIONS note in between
3098 BEG and END. */
3100 static bool
3101 switch_text_sections_between_p (const rtx_insn *beg, const rtx_insn *end)
3103 const rtx_insn *p;
3104 for (p = beg; p != end; p = NEXT_INSN (p))
3105 if (NOTE_P (p) && NOTE_KIND (p) == NOTE_INSN_SWITCH_TEXT_SECTIONS)
3106 return true;
3107 return false;
3111 /* Once we have tried two ways to fill a delay slot, make a pass over the
3112 code to try to improve the results and to do such things as more jump
3113 threading. */
3115 static void
3116 relax_delay_slots (rtx_insn *first)
3118 rtx_insn *insn, *next;
3119 rtx_sequence *pat;
3120 rtx trial;
3121 rtx_insn *delay_insn;
3122 rtx target_label;
3124 /* Look at every JUMP_INSN and see if we can improve it. */
3125 for (insn = first; insn; insn = next)
3127 rtx_insn *other;
3128 bool crossing;
3130 next = next_active_insn (insn);
3132 /* If this is a jump insn, see if it now jumps to a jump, jumps to
3133 the next insn, or jumps to a label that is not the last of a
3134 group of consecutive labels. */
3135 if (is_a <rtx_jump_insn *> (insn)
3136 && (condjump_p (insn) || condjump_in_parallel_p (insn))
3137 && !ANY_RETURN_P (target_label = JUMP_LABEL (insn)))
3139 rtx_jump_insn *jump_insn = as_a <rtx_jump_insn *> (insn);
3140 target_label
3141 = skip_consecutive_labels (follow_jumps (target_label, jump_insn,
3142 &crossing));
3143 if (ANY_RETURN_P (target_label))
3144 target_label = find_end_label (target_label);
3146 if (target_label
3147 && next_active_insn (as_a<rtx_insn *> (target_label)) == next
3148 && ! condjump_in_parallel_p (jump_insn)
3149 && ! (next && switch_text_sections_between_p (jump_insn, next)))
3151 delete_jump (jump_insn);
3152 continue;
3155 if (target_label && target_label != JUMP_LABEL (jump_insn))
3157 reorg_redirect_jump (jump_insn, target_label);
3158 if (crossing)
3159 CROSSING_JUMP_P (jump_insn) = 1;
3162 /* See if this jump conditionally branches around an unconditional
3163 jump. If so, invert this jump and point it to the target of the
3164 second jump. Check if it's possible on the target. */
3165 if (next && simplejump_or_return_p (next)
3166 && any_condjump_p (jump_insn)
3167 && target_label
3168 && (next_active_insn (as_a<rtx_insn *> (target_label))
3169 == next_active_insn (next))
3170 && no_labels_between_p (jump_insn, next)
3171 && targetm.can_follow_jump (jump_insn, next))
3173 rtx label = JUMP_LABEL (next);
3175 /* Be careful how we do this to avoid deleting code or
3176 labels that are momentarily dead. See similar optimization
3177 in jump.c.
3179 We also need to ensure we properly handle the case when
3180 invert_jump fails. */
3182 ++LABEL_NUSES (target_label);
3183 if (!ANY_RETURN_P (label))
3184 ++LABEL_NUSES (label);
3186 if (invert_jump (jump_insn, label, 1))
3188 delete_related_insns (next);
3189 next = jump_insn;
3192 if (!ANY_RETURN_P (label))
3193 --LABEL_NUSES (label);
3195 if (--LABEL_NUSES (target_label) == 0)
3196 delete_related_insns (target_label);
3198 continue;
3202 /* If this is an unconditional jump and the previous insn is a
3203 conditional jump, try reversing the condition of the previous
3204 insn and swapping our targets. The next pass might be able to
3205 fill the slots.
3207 Don't do this if we expect the conditional branch to be true, because
3208 we would then be making the more common case longer. */
3210 if (simplejump_or_return_p (insn)
3211 && (other = prev_active_insn (insn)) != 0
3212 && any_condjump_p (other)
3213 && no_labels_between_p (other, insn)
3214 && 0 > mostly_true_jump (other))
3216 rtx other_target = JUMP_LABEL (other);
3217 target_label = JUMP_LABEL (insn);
3219 if (invert_jump (as_a <rtx_jump_insn *> (other), target_label, 0))
3220 reorg_redirect_jump (as_a <rtx_jump_insn *> (insn), other_target);
3223 /* Now look only at cases where we have a filled delay slot. */
3224 if (!NONJUMP_INSN_P (insn) || GET_CODE (PATTERN (insn)) != SEQUENCE)
3225 continue;
3227 pat = as_a <rtx_sequence *> (PATTERN (insn));
3228 delay_insn = pat->insn (0);
3230 /* See if the first insn in the delay slot is redundant with some
3231 previous insn. Remove it from the delay slot if so; then set up
3232 to reprocess this insn. */
3233 if (redundant_insn (pat->insn (1), delay_insn, vNULL))
3235 update_block (pat->insn (1), insn);
3236 delete_from_delay_slot (pat->insn (1));
3237 next = prev_active_insn (next);
3238 continue;
3241 /* See if we have a RETURN insn with a filled delay slot followed
3242 by a RETURN insn with an unfilled a delay slot. If so, we can delete
3243 the first RETURN (but not its delay insn). This gives the same
3244 effect in fewer instructions.
3246 Only do so if optimizing for size since this results in slower, but
3247 smaller code. */
3248 if (optimize_function_for_size_p (cfun)
3249 && ANY_RETURN_P (PATTERN (delay_insn))
3250 && next
3251 && JUMP_P (next)
3252 && PATTERN (next) == PATTERN (delay_insn))
3254 rtx_insn *after;
3255 int i;
3257 /* Delete the RETURN and just execute the delay list insns.
3259 We do this by deleting the INSN containing the SEQUENCE, then
3260 re-emitting the insns separately, and then deleting the RETURN.
3261 This allows the count of the jump target to be properly
3262 decremented.
3264 Note that we need to change the INSN_UID of the re-emitted insns
3265 since it is used to hash the insns for mark_target_live_regs and
3266 the re-emitted insns will no longer be wrapped up in a SEQUENCE.
3268 Clear the from target bit, since these insns are no longer
3269 in delay slots. */
3270 for (i = 0; i < XVECLEN (pat, 0); i++)
3271 INSN_FROM_TARGET_P (XVECEXP (pat, 0, i)) = 0;
3273 trial = PREV_INSN (insn);
3274 delete_related_insns (insn);
3275 gcc_assert (GET_CODE (pat) == SEQUENCE);
3276 add_insn_after (delay_insn, trial, NULL);
3277 after = delay_insn;
3278 for (i = 1; i < pat->len (); i++)
3279 after = emit_copy_of_insn_after (pat->insn (i), after);
3280 delete_scheduled_jump (delay_insn);
3281 continue;
3284 /* Now look only at the cases where we have a filled JUMP_INSN. */
3285 rtx_jump_insn *delay_jump_insn =
3286 dyn_cast <rtx_jump_insn *> (delay_insn);
3287 if (! delay_jump_insn || !(condjump_p (delay_jump_insn)
3288 || condjump_in_parallel_p (delay_jump_insn)))
3289 continue;
3291 target_label = JUMP_LABEL (delay_jump_insn);
3292 if (target_label && ANY_RETURN_P (target_label))
3293 continue;
3295 /* If this jump goes to another unconditional jump, thread it, but
3296 don't convert a jump into a RETURN here. */
3297 trial = skip_consecutive_labels (follow_jumps (target_label,
3298 delay_jump_insn,
3299 &crossing));
3300 if (ANY_RETURN_P (trial))
3301 trial = find_end_label (trial);
3303 if (trial && trial != target_label
3304 && redirect_with_delay_slots_safe_p (delay_jump_insn, trial, insn))
3306 reorg_redirect_jump (delay_jump_insn, trial);
3307 target_label = trial;
3308 if (crossing)
3309 CROSSING_JUMP_P (delay_jump_insn) = 1;
3312 /* If the first insn at TARGET_LABEL is redundant with a previous
3313 insn, redirect the jump to the following insn and process again.
3314 We use next_real_insn instead of next_active_insn so we
3315 don't skip USE-markers, or we'll end up with incorrect
3316 liveness info. */
3317 trial = next_real_insn (target_label);
3318 if (trial && GET_CODE (PATTERN (trial)) != SEQUENCE
3319 && redundant_insn (trial, insn, vNULL)
3320 && ! can_throw_internal (trial))
3322 /* Figure out where to emit the special USE insn so we don't
3323 later incorrectly compute register live/death info. */
3324 rtx_insn *tmp = next_active_insn (as_a<rtx_insn *> (trial));
3325 if (tmp == 0)
3326 tmp = find_end_label (simple_return_rtx);
3328 if (tmp)
3330 /* Insert the special USE insn and update dataflow info.
3331 We know "trial" is an insn here as it is the output of
3332 next_real_insn () above. */
3333 update_block (as_a <rtx_insn *> (trial), tmp);
3335 /* Now emit a label before the special USE insn, and
3336 redirect our jump to the new label. */
3337 target_label = get_label_before (PREV_INSN (tmp), target_label);
3338 reorg_redirect_jump (delay_jump_insn, target_label);
3339 next = insn;
3340 continue;
3344 /* Similarly, if it is an unconditional jump with one insn in its
3345 delay list and that insn is redundant, thread the jump. */
3346 rtx_sequence *trial_seq =
3347 trial ? dyn_cast <rtx_sequence *> (PATTERN (trial)) : NULL;
3348 if (trial_seq
3349 && trial_seq->len () == 2
3350 && JUMP_P (trial_seq->insn (0))
3351 && simplejump_or_return_p (trial_seq->insn (0))
3352 && redundant_insn (trial_seq->insn (1), insn, vNULL))
3354 target_label = JUMP_LABEL (trial_seq->insn (0));
3355 if (ANY_RETURN_P (target_label))
3356 target_label = find_end_label (target_label);
3358 if (target_label
3359 && redirect_with_delay_slots_safe_p (delay_jump_insn,
3360 target_label, insn))
3362 update_block (trial_seq->insn (1), insn);
3363 reorg_redirect_jump (delay_jump_insn, target_label);
3364 next = insn;
3365 continue;
3369 /* See if we have a simple (conditional) jump that is useless. */
3370 if (! INSN_ANNULLED_BRANCH_P (delay_jump_insn)
3371 && ! condjump_in_parallel_p (delay_jump_insn)
3372 && prev_active_insn (as_a<rtx_insn *> (target_label)) == insn
3373 && ! BARRIER_P (prev_nonnote_insn (as_a<rtx_insn *> (target_label)))
3374 /* If the last insn in the delay slot sets CC0 for some insn,
3375 various code assumes that it is in a delay slot. We could
3376 put it back where it belonged and delete the register notes,
3377 but it doesn't seem worthwhile in this uncommon case. */
3378 && (!HAVE_cc0
3379 || ! find_reg_note (XVECEXP (pat, 0, XVECLEN (pat, 0) - 1),
3380 REG_CC_USER, NULL_RTX)))
3382 rtx_insn *after;
3383 int i;
3385 /* All this insn does is execute its delay list and jump to the
3386 following insn. So delete the jump and just execute the delay
3387 list insns.
3389 We do this by deleting the INSN containing the SEQUENCE, then
3390 re-emitting the insns separately, and then deleting the jump.
3391 This allows the count of the jump target to be properly
3392 decremented.
3394 Note that we need to change the INSN_UID of the re-emitted insns
3395 since it is used to hash the insns for mark_target_live_regs and
3396 the re-emitted insns will no longer be wrapped up in a SEQUENCE.
3398 Clear the from target bit, since these insns are no longer
3399 in delay slots. */
3400 for (i = 0; i < XVECLEN (pat, 0); i++)
3401 INSN_FROM_TARGET_P (XVECEXP (pat, 0, i)) = 0;
3403 trial = PREV_INSN (insn);
3404 delete_related_insns (insn);
3405 gcc_assert (GET_CODE (pat) == SEQUENCE);
3406 add_insn_after (delay_jump_insn, trial, NULL);
3407 after = delay_jump_insn;
3408 for (i = 1; i < pat->len (); i++)
3409 after = emit_copy_of_insn_after (pat->insn (i), after);
3410 delete_scheduled_jump (delay_jump_insn);
3411 continue;
3414 /* See if this is an unconditional jump around a single insn which is
3415 identical to the one in its delay slot. In this case, we can just
3416 delete the branch and the insn in its delay slot. */
3417 if (next && NONJUMP_INSN_P (next)
3418 && label_before_next_insn (next, insn) == target_label
3419 && simplejump_p (insn)
3420 && XVECLEN (pat, 0) == 2
3421 && rtx_equal_p (PATTERN (next), PATTERN (pat->insn (1))))
3423 delete_related_insns (insn);
3424 continue;
3427 /* See if this jump (with its delay slots) conditionally branches
3428 around an unconditional jump (without delay slots). If so, invert
3429 this jump and point it to the target of the second jump. We cannot
3430 do this for annulled jumps, though. Again, don't convert a jump to
3431 a RETURN here. */
3432 if (! INSN_ANNULLED_BRANCH_P (delay_jump_insn)
3433 && any_condjump_p (delay_jump_insn)
3434 && next && simplejump_or_return_p (next)
3435 && (next_active_insn (as_a<rtx_insn *> (target_label))
3436 == next_active_insn (next))
3437 && no_labels_between_p (insn, next))
3439 rtx label = JUMP_LABEL (next);
3440 rtx old_label = JUMP_LABEL (delay_jump_insn);
3442 if (ANY_RETURN_P (label))
3443 label = find_end_label (label);
3445 /* find_end_label can generate a new label. Check this first. */
3446 if (label
3447 && no_labels_between_p (insn, next)
3448 && redirect_with_delay_slots_safe_p (delay_jump_insn,
3449 label, insn))
3451 /* Be careful how we do this to avoid deleting code or labels
3452 that are momentarily dead. See similar optimization in
3453 jump.c */
3454 if (old_label)
3455 ++LABEL_NUSES (old_label);
3457 if (invert_jump (delay_jump_insn, label, 1))
3459 int i;
3461 /* Must update the INSN_FROM_TARGET_P bits now that
3462 the branch is reversed, so that mark_target_live_regs
3463 will handle the delay slot insn correctly. */
3464 for (i = 1; i < XVECLEN (PATTERN (insn), 0); i++)
3466 rtx slot = XVECEXP (PATTERN (insn), 0, i);
3467 INSN_FROM_TARGET_P (slot) = ! INSN_FROM_TARGET_P (slot);
3470 delete_related_insns (next);
3471 next = insn;
3474 if (old_label && --LABEL_NUSES (old_label) == 0)
3475 delete_related_insns (old_label);
3476 continue;
3480 /* If we own the thread opposite the way this insn branches, see if we
3481 can merge its delay slots with following insns. */
3482 if (INSN_FROM_TARGET_P (pat->insn (1))
3483 && own_thread_p (NEXT_INSN (insn), 0, 1))
3484 try_merge_delay_insns (insn, next);
3485 else if (! INSN_FROM_TARGET_P (pat->insn (1))
3486 && own_thread_p (target_label, target_label, 0))
3487 try_merge_delay_insns (insn,
3488 next_active_insn (as_a<rtx_insn *> (target_label)));
3490 /* If we get here, we haven't deleted INSN. But we may have deleted
3491 NEXT, so recompute it. */
3492 next = next_active_insn (insn);
3497 /* Look for filled jumps to the end of function label. We can try to convert
3498 them into RETURN insns if the insns in the delay slot are valid for the
3499 RETURN as well. */
3501 static void
3502 make_return_insns (rtx_insn *first)
3504 rtx_insn *insn;
3505 rtx_jump_insn *jump_insn;
3506 rtx real_return_label = function_return_label;
3507 rtx real_simple_return_label = function_simple_return_label;
3508 int slots, i;
3510 /* See if there is a RETURN insn in the function other than the one we
3511 made for END_OF_FUNCTION_LABEL. If so, set up anything we can't change
3512 into a RETURN to jump to it. */
3513 for (insn = first; insn; insn = NEXT_INSN (insn))
3514 if (JUMP_P (insn) && ANY_RETURN_P (PATTERN (insn)))
3516 rtx t = get_label_before (insn, NULL_RTX);
3517 if (PATTERN (insn) == ret_rtx)
3518 real_return_label = t;
3519 else
3520 real_simple_return_label = t;
3521 break;
3524 /* Show an extra usage of REAL_RETURN_LABEL so it won't go away if it
3525 was equal to END_OF_FUNCTION_LABEL. */
3526 if (real_return_label)
3527 LABEL_NUSES (real_return_label)++;
3528 if (real_simple_return_label)
3529 LABEL_NUSES (real_simple_return_label)++;
3531 /* Clear the list of insns to fill so we can use it. */
3532 obstack_free (&unfilled_slots_obstack, unfilled_firstobj);
3534 for (insn = first; insn; insn = NEXT_INSN (insn))
3536 int flags;
3537 rtx kind, real_label;
3539 /* Only look at filled JUMP_INSNs that go to the end of function
3540 label. */
3541 if (!NONJUMP_INSN_P (insn))
3542 continue;
3544 if (GET_CODE (PATTERN (insn)) != SEQUENCE)
3545 continue;
3547 rtx_sequence *pat = as_a <rtx_sequence *> (PATTERN (insn));
3549 if (!jump_to_label_p (pat->insn (0)))
3550 continue;
3552 if (JUMP_LABEL (pat->insn (0)) == function_return_label)
3554 kind = ret_rtx;
3555 real_label = real_return_label;
3557 else if (JUMP_LABEL (pat->insn (0)) == function_simple_return_label)
3559 kind = simple_return_rtx;
3560 real_label = real_simple_return_label;
3562 else
3563 continue;
3565 jump_insn = as_a <rtx_jump_insn *> (pat->insn (0));
3567 /* If we can't make the jump into a RETURN, try to redirect it to the best
3568 RETURN and go on to the next insn. */
3569 if (!reorg_redirect_jump (jump_insn, kind))
3571 /* Make sure redirecting the jump will not invalidate the delay
3572 slot insns. */
3573 if (redirect_with_delay_slots_safe_p (jump_insn, real_label, insn))
3574 reorg_redirect_jump (jump_insn, real_label);
3575 continue;
3578 /* See if this RETURN can accept the insns current in its delay slot.
3579 It can if it has more or an equal number of slots and the contents
3580 of each is valid. */
3582 flags = get_jump_flags (jump_insn, JUMP_LABEL (jump_insn));
3583 slots = num_delay_slots (jump_insn);
3584 if (slots >= XVECLEN (pat, 0) - 1)
3586 for (i = 1; i < XVECLEN (pat, 0); i++)
3587 if (! (
3588 #if ANNUL_IFFALSE_SLOTS
3589 (INSN_ANNULLED_BRANCH_P (jump_insn)
3590 && INSN_FROM_TARGET_P (pat->insn (i)))
3591 ? eligible_for_annul_false (jump_insn, i - 1,
3592 pat->insn (i), flags) :
3593 #endif
3594 #if ANNUL_IFTRUE_SLOTS
3595 (INSN_ANNULLED_BRANCH_P (jump_insn)
3596 && ! INSN_FROM_TARGET_P (pat->insn (i)))
3597 ? eligible_for_annul_true (jump_insn, i - 1,
3598 pat->insn (i), flags) :
3599 #endif
3600 eligible_for_delay (jump_insn, i - 1,
3601 pat->insn (i), flags)))
3602 break;
3604 else
3605 i = 0;
3607 if (i == XVECLEN (pat, 0))
3608 continue;
3610 /* We have to do something with this insn. If it is an unconditional
3611 RETURN, delete the SEQUENCE and output the individual insns,
3612 followed by the RETURN. Then set things up so we try to find
3613 insns for its delay slots, if it needs some. */
3614 if (ANY_RETURN_P (PATTERN (jump_insn)))
3616 rtx_insn *prev = PREV_INSN (insn);
3618 delete_related_insns (insn);
3619 for (i = 1; i < XVECLEN (pat, 0); i++)
3620 prev = emit_insn_after (PATTERN (XVECEXP (pat, 0, i)), prev);
3622 insn = emit_jump_insn_after (PATTERN (jump_insn), prev);
3623 emit_barrier_after (insn);
3625 if (slots)
3626 obstack_ptr_grow (&unfilled_slots_obstack, insn);
3628 else
3629 /* It is probably more efficient to keep this with its current
3630 delay slot as a branch to a RETURN. */
3631 reorg_redirect_jump (jump_insn, real_label);
3634 /* Now delete REAL_RETURN_LABEL if we never used it. Then try to fill any
3635 new delay slots we have created. */
3636 if (real_return_label != NULL_RTX && --LABEL_NUSES (real_return_label) == 0)
3637 delete_related_insns (real_return_label);
3638 if (real_simple_return_label != NULL_RTX
3639 && --LABEL_NUSES (real_simple_return_label) == 0)
3640 delete_related_insns (real_simple_return_label);
3642 fill_simple_delay_slots (1);
3643 fill_simple_delay_slots (0);
3646 /* Try to find insns to place in delay slots. */
3648 static void
3649 dbr_schedule (rtx_insn *first)
3651 rtx_insn *insn, *next, *epilogue_insn = 0;
3652 int i;
3653 bool need_return_insns;
3655 /* If the current function has no insns other than the prologue and
3656 epilogue, then do not try to fill any delay slots. */
3657 if (n_basic_blocks_for_fn (cfun) == NUM_FIXED_BLOCKS)
3658 return;
3660 /* Find the highest INSN_UID and allocate and initialize our map from
3661 INSN_UID's to position in code. */
3662 for (max_uid = 0, insn = first; insn; insn = NEXT_INSN (insn))
3664 if (INSN_UID (insn) > max_uid)
3665 max_uid = INSN_UID (insn);
3666 if (NOTE_P (insn)
3667 && NOTE_KIND (insn) == NOTE_INSN_EPILOGUE_BEG)
3668 epilogue_insn = insn;
3671 uid_to_ruid = XNEWVEC (int, max_uid + 1);
3672 for (i = 0, insn = first; insn; i++, insn = NEXT_INSN (insn))
3673 uid_to_ruid[INSN_UID (insn)] = i;
3675 /* Initialize the list of insns that need filling. */
3676 if (unfilled_firstobj == 0)
3678 gcc_obstack_init (&unfilled_slots_obstack);
3679 unfilled_firstobj = XOBNEWVAR (&unfilled_slots_obstack, rtx, 0);
3682 for (insn = next_active_insn (first); insn; insn = next_active_insn (insn))
3684 rtx target;
3686 /* Skip vector tables. We can't get attributes for them. */
3687 if (JUMP_TABLE_DATA_P (insn))
3688 continue;
3690 if (JUMP_P (insn))
3691 INSN_ANNULLED_BRANCH_P (insn) = 0;
3692 INSN_FROM_TARGET_P (insn) = 0;
3694 if (num_delay_slots (insn) > 0)
3695 obstack_ptr_grow (&unfilled_slots_obstack, insn);
3697 /* Ensure all jumps go to the last of a set of consecutive labels. */
3698 if (JUMP_P (insn)
3699 && (condjump_p (insn) || condjump_in_parallel_p (insn))
3700 && !ANY_RETURN_P (JUMP_LABEL (insn))
3701 && ((target = skip_consecutive_labels (JUMP_LABEL (insn)))
3702 != JUMP_LABEL (insn)))
3703 redirect_jump (as_a <rtx_jump_insn *> (insn), target, 1);
3706 init_resource_info (epilogue_insn);
3708 /* Show we haven't computed an end-of-function label yet. */
3709 function_return_label = function_simple_return_label = NULL;
3711 /* Initialize the statistics for this function. */
3712 memset (num_insns_needing_delays, 0, sizeof num_insns_needing_delays);
3713 memset (num_filled_delays, 0, sizeof num_filled_delays);
3715 /* Now do the delay slot filling. Try everything twice in case earlier
3716 changes make more slots fillable. */
3718 for (reorg_pass_number = 0;
3719 reorg_pass_number < MAX_REORG_PASSES;
3720 reorg_pass_number++)
3722 fill_simple_delay_slots (1);
3723 fill_simple_delay_slots (0);
3724 if (!targetm.no_speculation_in_delay_slots_p ())
3725 fill_eager_delay_slots ();
3726 relax_delay_slots (first);
3729 /* If we made an end of function label, indicate that it is now
3730 safe to delete it by undoing our prior adjustment to LABEL_NUSES.
3731 If it is now unused, delete it. */
3732 if (function_return_label && --LABEL_NUSES (function_return_label) == 0)
3733 delete_related_insns (function_return_label);
3734 if (function_simple_return_label
3735 && --LABEL_NUSES (function_simple_return_label) == 0)
3736 delete_related_insns (function_simple_return_label);
3738 need_return_insns = false;
3739 need_return_insns |= targetm.have_return () && function_return_label != 0;
3740 need_return_insns |= (targetm.have_simple_return ()
3741 && function_simple_return_label != 0);
3742 if (need_return_insns)
3743 make_return_insns (first);
3745 /* Delete any USE insns made by update_block; subsequent passes don't need
3746 them or know how to deal with them. */
3747 for (insn = first; insn; insn = next)
3749 next = NEXT_INSN (insn);
3751 if (NONJUMP_INSN_P (insn) && GET_CODE (PATTERN (insn)) == USE
3752 && INSN_P (XEXP (PATTERN (insn), 0)))
3753 next = delete_related_insns (insn);
3756 obstack_free (&unfilled_slots_obstack, unfilled_firstobj);
3758 /* It is not clear why the line below is needed, but it does seem to be. */
3759 unfilled_firstobj = XOBNEWVAR (&unfilled_slots_obstack, rtx, 0);
3761 if (dump_file)
3763 int i, j, need_comma;
3764 int total_delay_slots[MAX_DELAY_HISTOGRAM + 1];
3765 int total_annul_slots[MAX_DELAY_HISTOGRAM + 1];
3767 for (reorg_pass_number = 0;
3768 reorg_pass_number < MAX_REORG_PASSES;
3769 reorg_pass_number++)
3771 fprintf (dump_file, ";; Reorg pass #%d:\n", reorg_pass_number + 1);
3772 for (i = 0; i < NUM_REORG_FUNCTIONS; i++)
3774 need_comma = 0;
3775 fprintf (dump_file, ";; Reorg function #%d\n", i);
3777 fprintf (dump_file, ";; %d insns needing delay slots\n;; ",
3778 num_insns_needing_delays[i][reorg_pass_number]);
3780 for (j = 0; j < MAX_DELAY_HISTOGRAM + 1; j++)
3781 if (num_filled_delays[i][j][reorg_pass_number])
3783 if (need_comma)
3784 fprintf (dump_file, ", ");
3785 need_comma = 1;
3786 fprintf (dump_file, "%d got %d delays",
3787 num_filled_delays[i][j][reorg_pass_number], j);
3789 fprintf (dump_file, "\n");
3792 memset (total_delay_slots, 0, sizeof total_delay_slots);
3793 memset (total_annul_slots, 0, sizeof total_annul_slots);
3794 for (insn = first; insn; insn = NEXT_INSN (insn))
3796 if (! insn->deleted ()
3797 && NONJUMP_INSN_P (insn)
3798 && GET_CODE (PATTERN (insn)) != USE
3799 && GET_CODE (PATTERN (insn)) != CLOBBER)
3801 if (GET_CODE (PATTERN (insn)) == SEQUENCE)
3803 rtx control;
3804 j = XVECLEN (PATTERN (insn), 0) - 1;
3805 if (j > MAX_DELAY_HISTOGRAM)
3806 j = MAX_DELAY_HISTOGRAM;
3807 control = XVECEXP (PATTERN (insn), 0, 0);
3808 if (JUMP_P (control) && INSN_ANNULLED_BRANCH_P (control))
3809 total_annul_slots[j]++;
3810 else
3811 total_delay_slots[j]++;
3813 else if (num_delay_slots (insn) > 0)
3814 total_delay_slots[0]++;
3817 fprintf (dump_file, ";; Reorg totals: ");
3818 need_comma = 0;
3819 for (j = 0; j < MAX_DELAY_HISTOGRAM + 1; j++)
3821 if (total_delay_slots[j])
3823 if (need_comma)
3824 fprintf (dump_file, ", ");
3825 need_comma = 1;
3826 fprintf (dump_file, "%d got %d delays", total_delay_slots[j], j);
3829 fprintf (dump_file, "\n");
3831 if (ANNUL_IFTRUE_SLOTS || ANNUL_IFFALSE_SLOTS)
3833 fprintf (dump_file, ";; Reorg annuls: ");
3834 need_comma = 0;
3835 for (j = 0; j < MAX_DELAY_HISTOGRAM + 1; j++)
3837 if (total_annul_slots[j])
3839 if (need_comma)
3840 fprintf (dump_file, ", ");
3841 need_comma = 1;
3842 fprintf (dump_file, "%d got %d delays", total_annul_slots[j], j);
3845 fprintf (dump_file, "\n");
3848 fprintf (dump_file, "\n");
3851 if (!sibling_labels.is_empty ())
3853 update_alignments (sibling_labels);
3854 sibling_labels.release ();
3857 free_resource_info ();
3858 free (uid_to_ruid);
3859 crtl->dbr_scheduled_p = true;
3862 /* Run delay slot optimization. */
3863 static unsigned int
3864 rest_of_handle_delay_slots (void)
3866 if (DELAY_SLOTS)
3867 dbr_schedule (get_insns ());
3869 return 0;
3872 namespace {
3874 const pass_data pass_data_delay_slots =
3876 RTL_PASS, /* type */
3877 "dbr", /* name */
3878 OPTGROUP_NONE, /* optinfo_flags */
3879 TV_DBR_SCHED, /* tv_id */
3880 0, /* properties_required */
3881 0, /* properties_provided */
3882 0, /* properties_destroyed */
3883 0, /* todo_flags_start */
3884 0, /* todo_flags_finish */
3887 class pass_delay_slots : public rtl_opt_pass
3889 public:
3890 pass_delay_slots (gcc::context *ctxt)
3891 : rtl_opt_pass (pass_data_delay_slots, ctxt)
3894 /* opt_pass methods: */
3895 virtual bool gate (function *);
3896 virtual unsigned int execute (function *)
3898 return rest_of_handle_delay_slots ();
3901 }; // class pass_delay_slots
3903 bool
3904 pass_delay_slots::gate (function *)
3906 /* At -O0 dataflow info isn't updated after RA. */
3907 if (DELAY_SLOTS)
3908 return optimize > 0 && flag_delayed_branch && !crtl->dbr_scheduled_p;
3910 return false;
3913 } // anon namespace
3915 rtl_opt_pass *
3916 make_pass_delay_slots (gcc::context *ctxt)
3918 return new pass_delay_slots (ctxt);
3921 /* Machine dependent reorg pass. */
3923 namespace {
3925 const pass_data pass_data_machine_reorg =
3927 RTL_PASS, /* type */
3928 "mach", /* name */
3929 OPTGROUP_NONE, /* optinfo_flags */
3930 TV_MACH_DEP, /* tv_id */
3931 0, /* properties_required */
3932 0, /* properties_provided */
3933 0, /* properties_destroyed */
3934 0, /* todo_flags_start */
3935 0, /* todo_flags_finish */
3938 class pass_machine_reorg : public rtl_opt_pass
3940 public:
3941 pass_machine_reorg (gcc::context *ctxt)
3942 : rtl_opt_pass (pass_data_machine_reorg, ctxt)
3945 /* opt_pass methods: */
3946 virtual bool gate (function *)
3948 return targetm.machine_dependent_reorg != 0;
3951 virtual unsigned int execute (function *)
3953 targetm.machine_dependent_reorg ();
3954 return 0;
3957 }; // class pass_machine_reorg
3959 } // anon namespace
3961 rtl_opt_pass *
3962 make_pass_machine_reorg (gcc::context *ctxt)
3964 return new pass_machine_reorg (ctxt);